Information Technology: DOD's Acquisition Policies and Guidance Need to Incorporate Additional Best Practices and Controls
Published: 2004-07-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The way in which the Department of Defense (DOD) has historically acquired its business systems has been cited as a root cause for its limited success in delivering promised system capabilities and benefits on time and within budget. In response, DOD recently revised its systems acquisition policies and guidance to incorporate best practices, including those pertaining to business systems. GAO was asked to determine whether DOD's revised systems acquisition policies and guidance (1) are consistent with industry best practices, including those pertaining to commercial component-based systems, and (2) provide the necessary controls to ensure that DOD component organizations adhere to the practices.
section_title: What GAO Found
paragraphs: DOD's revised policies and guidance largely incorporate 10 best practices for acquiring any type of information technology (IT) business system. For example, the revisions include the requirement that acquisitions be economically justified on the basis of costs, benefits, and risks. However, the revisions generally do not incorporate 8 best practices relating to the acquisition of commercial component-based systems. For example, they do not address basing any decision to modify commercial components on a thorough analysis of the impact of doing so or on preparing system users for the business process and job roles and responsibilities changes that are embedded in the functionality of commercial IT products. In total, GAO found that DOD's acquisition policies and guidance fully incorporate 8 of the 18 best practices that they were evaluated against, partially incorporate 5 practices, and do not incorporate the remaining 5 practices. DOD intends to expand its acquisition guidance to incorporate additional best practices by September 30, 2004, but department officials cite other priorities as a reason why they have not been able to complete this effort and could not provide a plan specifying how this will be accomplished. Until DOD's revised policies and guidance incorporate key systems acquisition best practices, the risk that system investments will not consistently deliver promised capabilities and benefits on time and within budget is increased. DOD's revised policies also do not contain sufficient controls to ensure that DOD components appropriately follow the best practices that are incorporated in its policies and guidance. According to acquisition best practices experts, as well as GAO's internal control guidance, controls are effective if they are backed by measurements that are verified. Although the revised policies and guidance require acquisition managers to examine and, as appropriate, adopt best practices, they do not cite what that examination entails. DOD believes existing controls are sufficient, even though these controls do not provide for measuring and validating the practices' use. Without specific requirements to measure and validate the use of best practices, the risk that they will not be followed increases, which, in turn, increases the risk that system investments will not meet expectations.

section_title: Letter
paragraphs: This report responds to your request that we assess the Department of  Defense’s (DOD) recently revised policies and guidance for acquiring  business systems for incorporation of acquisition best practices. As you  know, the way in which DOD has historically acquired information  technology (IT) systems has been cited as a root cause of these systems  failing to deliver promised capabilities and benefits on time and within  budget. The use of these best practices—which includes those practices  pertaining to any business system, whether custom developed or based on  commercially available product components, as well as those unique to  commercial component-based systems—is intended to improve on this  performance.
As agreed with your offices, our objectives were to determine whether  DOD’s recently revised policies and guidance for acquiring business  systems (1) are consistent with industry best practices, including those  pertaining to commercial component-based systems, and (2) provide the  necessary controls to ensure that DOD component organizations adhere to  best practices. We conducted our work between December 2003 and May  2004 in accordance with generally accepted government auditing  standards. Details of our objectives, scope, and methodology are included  in appendix I.
subsections: section_title: Background
paragraphs: Best practices are tried and proven methods, processes, techniques, and  activities that organizations define and use to minimize risks and maximize  chances for success. As we have previously reported, using best practices  can result in better outcomes—including cost savings; improved service  and product quality; and, ultimately, a better return on investment. For  example, two software engineering analyses of nearly 200 systems  acquisitions projects indicated that teams using systems acquisition best  practices produced cost savings of at least 11 percent over similar projects  conducted by teams that did not employ the kind of rigor and discipline  embedded in these practices. In addition, our research shows that best  practices are a significant factor in successful acquisition outcomes,  including increasing the likelihood that programs and projects will be  executed within cost and schedule estimates.
DOD, GAO, and the Congress have all advocated the use of best practices.  For example, in September 2000, DOD established a steering group to  promote the use of systems acquisition best practices and lessons learned.  Further, our 2001 report cited the benefits of DOD adoption of best  practices and provided recommendations for establishing a mechanism for  sharing best practices throughout DOD. In the fiscal year 2003 Defense  Authorization Act, the Congress used our recommendations in directing  DOD to expand its use of best practices. Specifically, it required the Under  Secretary of Defense (Acquisition, Technology, and Logistics (AT&L)) and  the Assistant Secretary of Defense (Command, Control, Communications,  and Intelligence—now called Networks and Information Integration  (NII))—to identify and serve as a clearinghouse for information regarding  software acquisition best practices in the public and private sectors. In  response, DOD assigned AT&L responsibility for serving as that  clearinghouse. Further, the Defense Information Systems Agency created a  Web site to provide information about acquisition best practices.
subsections: section_title: DOD Relies Extensively on IT Systems to Perform a Variety of Business Functions
paragraphs: DOD is one of the largest and most complex organizations in the world. In  fiscal year 2003, DOD reported that its operations involved over $1 trillion  in assets, nearly $1.6 trillion in liabilities, disbursements of more than $416  billion, and approximately 3.3 million military and civilian personnel.  Execution of these operations spans a wide range of defense organizations,  including the military services, defense agencies and field activities, and  various combatant and joint operation commands. To execute these  military operations, the department performs an assortment of business  functions, including logistics management, procurement, healthcare  management, and financial management.
To support its business functions, DOD reports that it currently relies on  about 2,300 IT systems, including accounting, acquisition, logistics, and  personnel systems. Moreover, its future investment in business systems is  expected to be sizable. For fiscal year 2004, DOD requested approximately  $28 billion in IT funding to support a wide range of military and business  operations. Approximately $9 billion of this amount is to support primarily  command and control systems, and the remaining $19 billion is to support  the operation, maintenance, and modernization of business systems.
section_title: Overview of DOD’s Acquisition Management Framework
paragraphs: Since the 1980s, DOD’s oversight of its systems acquisitions had been  defined by a series of three documents—commonly called the 5000  series—that provided the policies and guidance for departmental efforts to  acquire service capabilities and systems:   DOD Directive 5000.1, The Defense Acquisition System—describes the  management principles for DOD’s acquisition programs.
DOD Instruction 5000.2, Operation of the Defense Acquisition  System—outlines the framework for managing acquisition programs.
DOD 5000.2-R, Mandatory Procedures for Major Defense Acquisition  Programs (MDAPS) and Major Automated Information System  (MAIS) Acquisition Programs—provides the mandatory procedures  for acquiring major defense programs.
These documents have been revised several times. Most recently, in  October 2002, the Deputy Secretary of Defense determined that the  existing versions of these three documents required further revisions to  improve acquisition efficiency, flexibility, creativity, and innovation. As a  result, the Deputy Secretary canceled the existing versions of each  document and instructed the Under Secretary for AT&L; the Assistant  Secretary for NII; and the Director, Operational Test and Evaluation  (OT&E) to jointly revise the documents. (Table 1 describes selected  responsibilities of these three entities.) The revised directive and  instruction were issued in May 2003. Both were shortened and modified to  focus on required outcomes and legal requirements and to eliminate the  “how-to” details in the previous versions. In doing so, DOD intended the  revisions to provide program managers with more flexibility in executing  their respective programs.
DOD 5000.2-R was renamed the Interim Defense Acquisition Guidebook  and was made optional guidance on best practices and lessons learned.  (See fig. 1.) According to DOD officials, improvements to the guidebook  are currently under way. Until they are completed, DOD 5000.2-R serves as  the guidebook.
According to the revised policy, an acquisition is a directed, funded effort  that provides a new, improved, or continuing materiel, weapon or  information system, or service capability. The revised 5000 series applies to  acquisitions conducted by all of the department’s organizational  components. These components include the military services and the  defense agencies, such as the Defense Information Systems Agency.
The 5000 series describes a management framework that is intended to  translate mission needs and requirements into systems acquisition  programs. To accomplish this, the framework specifies five phases:   Concept refinement: Intended to refine the initial system concept and  produce a strategy for acquiring a system capability. A decision is made  at the end of this phase (milestone A decision) whether to move to  technology development.
Technology development: Intended to determine the appropriate set of  technologies to be integrated into the system by iteratively assessing the  viability of various technologies while simultaneously refining user  requirements. Once the technology has been demonstrated in a relevant  environment, a decision is made at the end of this phase (milestone B  decision) whether to move to system development and demonstration.
System development and demonstration: Intended to develop a system  or a system increment and demonstrate through developer testing that  the system/system increment can function in its target environment. A  decision is made at the end of this phase (milestone C decision) whether  to move to production and deployment.
Production and deployment: Intended to achieve an operational  capability that satisfies the mission needs, as verified through  independent operational test and evaluation, and ensures that the  system is implemented at all applicable locations.
Operations and support: Intended to provide a support program to  meet operational support requirements and sustain the system in the  most cost-effective manner over its total life cycle.
According to the framework, an acquisition program may begin at  milestone A, B, or C, and its progress depends on obtaining sufficient  knowledge to make an informed decision about whether to continue to the  next acquisition phase. Although the framework permits programs to be  managed as a single project, DOD Instruction 5000.2 states that the  department prefers an evolutionary acquisition strategy that delivers a  mature product in increments. Under such a strategy, the instruction states  that each increment is to begin with a milestone B decision, and the  production and deployment phase of each increment is to begin with a  milestone C decision. Figure 2 provides a simplified diagram of the  department’s acquisition management framework.
Typically, the Under Secretary for AT&L or a designee serves as the  investment decision authority for DOD acquisitions, but the Assistant  Secretary for NII/Chief Information Officer or a designee serves as the  decision authority for IT systems acquisitions.
section_title: Past Evaluations of DOD Business Systems Have Revealed Acquisition Management Weaknesses
paragraphs: Due in part to its long-standing and pervasive IT acquisition management  weaknesses, DOD has had limited success in acquiring IT resources to  replace its outdated business systems. Both inspector general and  departmental studies have cited these weaknesses on a variety of  acquisition projects. We have also reported on business systems acquisition  weaknesses. For example, in 2002 we reported that the Defense Logistics  Agency did not have effective corporate processes for consistently  acquiring software (the most costly and complex component of systems),  and that the agency did not have a software process improvement program  in place to effectively strengthen its software acquisition processes. In  2002, we also reported acquisition management problems with the Military  Health System’s acquisition of DOD’s primary medical information system,  including weaknesses in incremental economic justification, risk  management, and contract management. In 2003, we reported that DOD  had not economically justified four finance and accounting systems that  have an estimated cost of more than $1 billion. In each of our reports, we  have made recommendations for strengthening acquisition management  through the adoption of best practices. DOD has largely agreed with our  recommendations, but its progress to date in implementing them across the  department has been uneven.
section_title: DOD’s Acquisition Policy and Guidance Are Consistent with Some, but Not All, Key Acquisition Best Practices
paragraphs: We and others, such as Carnegie Mellon University’s Software Engineering  Institute, have identified and promoted the use of a number of best  practices associated with acquiring IT systems. For the purposes of this  report, we have identified 18 relevant best practices and grouped them into  two categories: (1) 10 best practices for acquiring any type of business  system and (2) 8 complementary best practices that relate specifically to  acquiring commercial component-based business systems. Examples of  best practices relevant to any business systems acquisition include  ensuring that (1) reasonable planning for all parts of the acquisition occur,  (2) a clear understanding of system requirements exists, and (3) risks are  proactively identified and systematically mitigated. Examples of best  practices relevant to commercial component-based systems acquisitions  include ensuring that (1) commercial product modification is effectively  controlled, (2) relationships among commercial products are understood  before acquisition decisions are made, and (3) the organizational impact of  using new system functionality is proactively managed. Each of these  practices is composed of from one to eight activities and is described in  table 2. DOD officials responsible for revising the 5000 series told us that  each of these 18 practices are relevant to DOD business systems  acquisitions. Appendix II provides additional details on each of these  practices.
DOD’s acquisition policies and guidance largely incorporate the 10 best  practices that are relevant to any business systems acquisition. More  specifically, they fully incorporate 7 of the 10 practices and partially  incorporate the other 3 practices. However, they generally do not  incorporate the 8 best practices that relate to acquiring commercial  component-based business systems. In particular, they fully incorporate 1  best practice, partially incorporate 2, and do not incorporate the remaining  5. (See fig. 3 for a summary of our analysis.) At our request, DOD officials  responsible for the 5000 series also assessed it against those 18 practices,  and we incorporated information from their assessment into ours. These  officials also told us that the acquisition guidebook is currently being  expanded to incorporate additional best practices, but they did not provide  us with a plan for accomplishing this. Until this is accomplished, DOD is  increasing the risk that important and beneficial best practices will not be  followed and that DOD business systems investments will not deliver  promised capabilities and benefits on time and within budget.
subsections: section_title: The 5000 Series Largely Incorporates Best Practices Relevant to Any Business Systems Acquisition
paragraphs: Of the 10 best practices that we categorized as relevant to the acquisition of  any business system, whether custom-developed or developed using  commercial packages and products, essentially all have been incorporated  into DOD’s acquisition policies and guidance. (See table 3 for our detailed  comparative analysis of the 5000 series against the 10 best practices.) For  example, those practices aimed at ensuring that the acquisition is well  planned, that the system is adequately tested and evaluated against  contractual requirements, and that the requirements are clearly defined  and understood by all stakeholders are all addressed in the 5000 series.  Similarly, for the 3 practices that are not fully addressed, this is the case  because one activity associated with the practice is not addressed.  According to DOD officials responsible for revising the 5000 series, the  policies contain those best practices mandated by either law or DOD  regulation, and other, optional best practices are contained in the interim  guidance represented by the former DOD 5000.2-R.
Nevertheless, the activities that are missing from the 3 practices are  important, and their absence increases the risk that the activities, and thus  the practice, will not be adequately performed. In turn, this increases the  risk that acquisition projects will fall short of expectations. The best  practice aimed at ensuring that risks are proactively identified and  systematically mitigated is a case in point. This practice has five activities  associated with it, one of which the 5000 series does not address—project  reviews include the status of identified risks. As with all the activities under  this practice, this activity plays an important role in ensuring that the  appropriate level of attention and visibility is regularly given to risk  identification and mitigation to ensure that it is effectively executed.  Conversely, if the activities are not provided for in policy and guidance, it is  unlikely that they will be performed, and it is likely that acquisition risks  will become cost, schedule, and performance problems.
section_title: The 5000 Series Generally Does Not Incorporate Best Practices Relevant to Commercial Component-Based Business Systems Acquisitions
paragraphs: Of the 8 best practices relevant to acquiring commercial component-based  business systems, few have been incorporated into DOD systems  acquisition policies and guidance. (See table 4 for our detailed comparative  analysis of the 5000 series against the 8 best practices.) For example, while  the practice aimed at ensuring that adequate planning takes place for  integrating commercial products with legacy systems is incorporated into  the 5000 series, practices associated with closely controlling any  modification to the software of these packages and products, thoroughly  analyzing and understanding the dependencies among commercial  products before acquiring them, and proactively managing the institutional  change that results from implementing the functionality in commercial  packages and products are not incorporated. According to DOD officials  responsible for revising the 5000 series, these practices were not included  in the recently revised version of DOD’s acquisition policies because they  included only those in existing law or regulation.
Nevertheless, the absence of these practices from the 5000 series increases  the risk that the practices will not be performed, which, in turn, increases  the risk that acquisition projects will fall short of expectations. The  practice intended to ensure development of a quality solicitation and  selection of a best-qualified contractor illustrates this. Specifically, this  practice calls for contract bidders to be evaluated on their ability to  implement commercial components. This evaluation is important because  integrating and implementing these component products is sufficiently  different from developing customized system solutions; it requires different  core competencies and experiences to be successful. By explicitly taking  this into consideration in evaluating and selecting a contractor, the risk of  contract award to a less-than-best-qualified contractor is reduced.
section_title: DOD Officials Report That They Are in the Process of Revising the Interim Defense Acquisition Guidebook
paragraphs: The DOD officials responsible for revising the 5000 series told us they  recognize the need for the 5000 series to incorporate additional best  practices. To this end, they reported that efforts are under way to expand  the Interim Defense Acquisition Guidebook to include additional best  practices and lessons learned across the department. However, the officials  could not provide us with a documented plan and associated  documentation showing how this task will be accomplished, what  resources are needed and assigned to accomplish it, when it will be  accomplished, and where the department stands in accomplishing it.  Instead, the officials told us that progress on it has been slowed by other  priorities, such as the need to first revise DOD Directive 5000.1 and DOD  Instruction 5000.2. They also said that there is only a small number of  staff available to work on what they described as being an extensive  revision of the guidebook. According to these officials, 80 to 90 percent of  the revision has been completed and reviewed and their goal is to publish  the initial version of the revised guidebook by September 30, 2004. In our  view, until the missing best practices that we cite in this report are included  in DOD’s acquisition policies and guidance, the chance that business  systems acquisitions will follow the policies and guidance and consistently  produce a successful outcome is diminished.
section_title: DOD’s Acquisition Policies Do Not Contain Sufficient Controls to Ensure That the Requirement Is Met for Appropriately Applying Best Practices
paragraphs: Federal laws and regulations define the need for effective controls over  agency programs, and controls are a key factor in achieving program  results, minimizing operational problems, and managing evolving demands  and priorities. Controls over defined processes, procedures, and support  activities are considered effective if they entail measuring and verifying  whether a given practice is followed. Without sufficient controls, it is  unlikely that practices will be consistently employed, which, in turn,  increases the probability that the positive program and project outcomes  these practices are designed to produce will not occur.
DOD’s acquisition policy requires program managers and investment  decision authorities to examine and, as appropriate, adopt best practices.  However, neither the policies nor the accompanying guidance explain what  “examine” means, including whether practice use is to be measured and  verified. Instead, the policies state that any issues regarding the intent of  the 5000 series, which would include whether practice adoption is to be  measured and validated, shall be resolved by the investment decision  authority, meaning that it is entirely up to this individual what information  relative to the use of best practices is relevant and necessary to ensure that  best practices are appropriately followed. According to the Chairman of  the Defense Acquisition Policy Working Group, this control is sufficient,  and explicit requirements for measuring and validating the use of best  practices are not necessary.
In our view, not requiring that decision authorities examine the  measurement and validation of best practices’ use increases the chance  that important best practices will not be appropriately followed, as required by DOD policy. As we have previously reported, a lack of explicit  controls that require review of relevant information at key decision points  raises the risk of making uninformed project decisions, as well as the risk  that investments will not meet cost, schedule, capability, and benefit  commitments.
section_title: Conclusions
paragraphs: DOD recognizes the importance of business systems acquisition best  practices by including best practices in their revised policy and guidance.  However, other practices that, if followed, could increase the odds of  acquisitions delivering promised system capabilities and benefits on time  and within budget have yet to be similarly included. In particular, those  practices associated with the successful acquisition of commercial  component-based business systems have not been sufficiently  incorporated into either the policies or the guidance. Moreover, effective  controls for ensuring that best practices are appropriately followed are not  adequately provided for in the policies. Although DOD officials intend to  expand the coverage of best practices in future versions of DOD’s  acquisition guidance, it is unclear what the scope, nature, and status of  these intentions are because explicit plans for revising the guidance and  associated progress reports were not available. Until DOD incorporates the  best practices we found missing in the 5000 series, and until it strengthens  the means by which the appropriate use of these practices will be ensured,  its business systems acquisitions will be exposed to unnecessary risk.  Therefore, it is important for DOD to treat further revisions of its  acquisition policy and guidance relative to business systems as a priority  and move quickly to incorporate missing best practices and associated  controls for ensuring that the practices are followed.
section_title: Recommendations for Executive Action
paragraphs: To improve DOD’s ability to acquire business systems, we recommend that  the Secretary of Defense direct the Under Secretary for AT&L, in  collaboration with the Assistant Secretary for NII and the Director, OT&E,  to take the following actions: 1. Develop and implement an explicit plan for incorporating into the 5000  series the best practices and associated activities currently missing  from the series. We recommend that the plan specify tasks to be  performed, resources needed and assigned, and milestones for  completing tasks.  2. We further recommend that progress against this plan be tracked and  reported as appropriate, and that the plan, at a minimum, incorporate  each of the following best practice activities:  Product line requirements—rather than just the requirements for the  system being acquired—are an explicit consideration in each  acquisition.
Acquisition project management activities are communicated to all  stakeholders.
Acquisition reviews include the status of identified risks.
Modification of commercial components is discouraged and allowed  only if justified by a thorough analysis of life-cycle costs and benefits.
Modification or upgrades to deployed versions of system components  are centrally controlled, and unilateral user release changes are  precluded.
Acquisition decisions about commercial components are based on  deliberate and thorough research, analysis, and evaluation of the  components’ interdependencies.
Acquisition plans provide for preparing users for the impact that the  business processes embedded in the commercial components will have  on their respective roles and responsibilities.
Changes affecting how users will be expected to use the system to  execute their jobs are actively managed.
Systems integration contractors are explicitly evaluated on their ability  to implement commercial components.
Investment decisions throughout a system’s life cycle are based on a  continuous set of tradeoffs among capabilities available in commercial  components (current and future), the architectural environment in  which the system is to operate, defined system requirements, and  existing cost/schedule constraints.
Evaluation criteria are established for selecting among commercial  component options that include both defined system requirements and  vendor/commercial product characteristics.  3. To ensure that the best practices provided for in DOD acquisition  policies and guidance are appropriately followed, we also recommend  that the above recommended plan incorporate steps to include in  DOD’s acquisition policies a provision for measurement and  verification of best practices.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, signed by the Principal  Director for Command, Control, Communications, Space, and Information  Technology Programs in the office of the DOD Assistant Secretary for  Networks and Information Integration, DOD agreed with the importance  and relevance of the best practices that we cite in the report. Additionally,  DOD agreed with 2 of our 13 recommendations for incorporating additional  best practices, stating that the department would incorporate the 2  practices in its policies and guidance.
DOD also partially agreed with 9 of our recommendations for incorporating  additional practices, stating that it would consider augmenting its coverage  of 5 of the practices and that it believed that 4 practices already existed in  its policies and guidance. With regard to the 5 practices, DOD stated that it  needed to review each practice further and determine the need for its  emphasis or endorsement in the 5000 series. We understand DOD’s desire  to carefully consider changes to its acquisition policies and guidance, and  believe that such careful deliberation is consistent with the spirit of our  recommendations.
With regard to the remaining 4 practices that DOD partially agreed with, we  do not agree with the department’s comment that these best practices  adequately exist in the 5000 series. For example, DOD commented that  because its existing policies and guidance provide for the use of integrated  product teams, which, according to DOD, are a means for promoting  collaboration and facilitating communication among stakeholders, its  policies and guidance therefore already provide for communicating  information about management of a given acquisition project to all relevant  project stakeholders. While we do not question the use of integrated  product teams as a way to communicate information, the point of our  recommendation is that there needs to be an explicit recognition in policy  or guidance of the type of information to be communicated and with whom  it is to be communicated. Restated, our recommendation for incorporating  the best practice of communicating acquisition management activities to all  stakeholders is intended to permit communication vehicles, such as  integrated product teams, to be more effective by explicitly providing for  this best practice in relevant policies and guidance. As another example,  we do not agree with DOD’s comment that 2 of the best practices that we  recommended for incorporation in its policies and guidance—preparing  users for the impact that business process changes embedded in  commercial components will have on their roles and responsibilities, and  actively managing changes in how users will use new systems—are already  sufficiently contained in the 5000 series. In particular, while we agree that  the series references an acquisition management toolkit that addresses  these 2 best practices, this reference is provided only once in the 5000  series, and this reference is only in relation to one phase of the acquisition  cycle (the technology development phase). Given the importance and  relevance of these practices to successful implementation of commercial  component-based systems, our position, and thus the basis for our  recommendations, is that the practices’ implementation would be more  likely to occur if the practices were visible and better recognized in all  relevant stages of DOD’s acquisition cycle.
Also in its comments, DOD did not agree with our recommendations to  develop and implement an explicit plan to govern its ongoing and future  policy and guidance revision activities, specifically stating that the  recommendation was inappropriate and offering updated information on  the status of and associated milestone for completing its activities. While  we have updated our report to include the revised status and milestone  information, we do not agree with DOD that a plan governing these efforts  is not needed. Given the importance of DOD's acquisition policies and  guidance, and the need for their continuous review and update to reflect  new acquisition best practices, we believe that having an explicit plan that  defines how and when these policies and guidance will be incorporated is  essential. Among other things, a plan would highlight the resource  constraints that this revision effort has been subject to, would allow  measurement against defined milestones, and would allow disclosure of  progress and impediments.
DOD also did not agree with our recommendation to add stronger controls  for ensuring adherence to the best practices that are contained in its  acquisition policies and guidance, stating that its existing oversight process  includes the necessary compliance activities. We disagree. As we state in  the report, DOD’s existing policy leaves these compliance activities to the  discretion of the program manager and the investment decision authority,  and it does not provide for measurement and verification of the use of best  practices, both of which are recognized components of effective control  processes.
A copy of DOD’s comments is reprinted in appendix III, along with our  response.
We are sending copies of this report to the Chairmen and Ranking Minority  Members of the Senate and House Committees on Armed Services;  Subcommittees on Defense, Senate and House Committees on  Appropriations; and the Subcommittee on Military Readiness, House  Committee on Armed Services. We are also sending copies to the Director,  Office of Management and Budget; the Secretary of Defense; the Under  Secretary of Defense (AT&L); the Assistant Secretary of Defense  (NII)/Chief Information Officer; and the Director, OT&E. We will make  copies available to others on request. This report will also be available at  no charge on our Web site at http://www.gao.gov.
If you or your staff has any questions concerning this report, please contact  me at (202) 512-3439. I can also be reached by e-mail at hiter@gao.gov.  Other contacts and key contributors to this report are listed in appendix IV.
section_title: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine whether the Department of Defense’s  (DOD) revised systems acquisition policies for acquiring information  technology (IT) business systems (1) are consistent with industry best  practices, including those pertaining to commercial component-based  systems, and (2) provide the necessary controls to ensure that the  department’s component organizations adhere to the practices.
To accomplish the first objective, we identified the DOD policies and  guidance relevant to business systems. These policies and guidance are  contained in three documents—DOD Directive 5000.1, DOD Instruction  5000.2, and the Interim Defense Acquisition Guidebook—and are  generally referred to as the 5000 series. We then reviewed each of these  documents and discussed with DOD officials responsible for developing  and revising the documents what steps were taken to incorporate best  practices into each document. The DOD officials that we interviewed were  from the offices of the Under Secretary of Defense (Acquisition,  Technology, and Logistics (AT&L)); the Assistant Secretary of Defense  (Networks and Information Integration (NII)); and the Director,  Operational Test and Evaluation (OT&E). Next, we researched prior GAO  reports; the work of federally funded research and development  organizations, such as the Software Engineering Institute and The  Aerospace Corporation; and other authoritative sources to identify  business systems acquisition best practices. Our research produced 18 best  practices, including associated activities, that we placed into two  categories—one category for the practices that are relevant to any business  systems acquisition and one category for the practices that are relevant to  commercial component-based business systems acquisitions. In particular,  we drew extensively from the Software Engineering Institute’s Software  Acquisition Capability Maturity Model.  In doing so, we selected practices  from the model’s repeatable level of process maturity, which is level two on  the model’s five-level scale. We used the repeatable level of process  maturity because it is intended to provide the necessary process discipline  to allow an organization to repeat earlier successes on similar projects. In  addition, we included one Software Acquisition Capability Maturity Model  level-three process area—risk management—because many experts  consider it to be one of the most important process areas. We did not  attempt to develop an exhaustive list of best practices and, in fact, fully  recognize that additional best practices exist, such as ensuring that the  appropriate level of human capital knowledge, skills, and abilities are  employed, as well as that additional activities for the practices that we have  identified exist, such as those configuration management activities  associated with identifying, controlling, reporting on, and auditing  configuration items and components. For the purposes of this report, we  identified those practices that are embodied, recognized, and accepted  acquisition models or frameworks, as well as those practices that are now  being recognized as being unique to commercial component-based systems  and for which there appears to be general agreement, including agreement  with DOD officials responsible for revising the 5000 series, that the  practices are relevant and important. Last, we analyzed each of the DOD  5000 series documents to determine whether the documents addressed,  either directly or indirectly by reference to another authoritative  document, the 18 best practices that we identified. Based on this analysis,  we judged whether the 5000 series documents fully, partially, or did not  incorporate each best practice. In making these judgments, we used the  following criteria:  To fully incorporate the practice, the 5000 series addressed all of the  practice’s activities.
To partially incorporate the practice, the 5000 series addressed some,  but not all, of the practice’s activities.
To not incorporate the practice, the 5000 series did not address any of  the practice’s activities.
Additionally, we provided the DOD officials responsible for revising the  5000 series with the 18 practices that we identified to obtain their views on  whether the practices were relevant to DOD business systems acquisitions.  The officials agreed that they were. We also requested that these officials  perform their own assessment of the 5000 series against these practices,  and we used these officials’ assessment in making our judgments as to  whether the practices were fully, partially, or not incorporated into DOD’s  acquisition policies and guidance. For a number of the activities, DOD  identified the Federal Acquisition Regulation and the Defense Federal  Acquisition Regulation Supplement as evidence that the activity was  being performed within a particular practice. We accepted that as proof  that the activity was being covered within DOD’s business systems  acquisition policy.
To address our second objective, we researched federal internal control  standards and controls inherent in the business systems acquisition best  practices that we identified. In particular, we reviewed the Software  Engineering Institute’s Software Acquisition Capability Maturity Model  framework and GAO’s internal control standards. We then analyzed DOD’s  revised acquisition policies and guidance to identify whether these controls  were cited and to provide assurance that relevant best practices were being  followed. We also interviewed DOD officials responsible for revising the  5000 series to determine reasons why controls were addressed or not  addressed in the policies and guidance.
section_title: Best Practices
paragraphs: Additional information on each of the 18 best practices that we identified is  provided in this appendix.
subsections: section_title: Best Practices Relevant to Any IT Business Systems Acquisition
paragraphs: Purpose: To ensure that reasonable planning for all parts of the acquisition  is conducted.
Description:  Acquisition planning is the process for conducting and  documenting acquisition planning activities beginning early and covering  all parts of the project. It extends to all acquisition areas, such as  budgeting, scheduling, resource estimating, risk identification, and  requirements definition, as well as the overall acquisition strategy.  Acquisition planning begins with the earliest identification of a requirement  that is to be satisfied through an acquisition.
Activities: (1) Plans are prepared during acquisition planning and  maintained throughout the acquisition. (2) Planning addresses the entire  acquisition process, including life cycle support of the products being  acquired. (3) The acquisition organization has a written policy for planning  the acquisition. (4) Responsibility for acquisition planning activities is  designated.
Purpose: To ensure that the acquisition is consistent with the organization’s  enterprise architecture.
Description: Architectural alignment is the process for analyzing and  verifying that the proposed architecture of the system being acquired is  consistent with the enterprise architecture for the organization acquiring  the system. Such alignment is needed to ensure that acquired systems can  interoperate and are not unnecessarily duplicative of one another.  Exceptions to this alignment requirement are permitted, but only when  justified and only when granted an explicit waiver from the architecture. A  particular architectural consideration is whether requirements that extend  beyond the specific system being acquired should be considered when  selecting system components. Such product line (i.e., systems that are  developed from a common set of assets and share a common and managed  set of features) considerations can provide substantial production  economies over acquiring systems from scratch.
Activities: (1) The system being acquired is assessed for alignment with  the enterprise architecture at key life cycle decision points, and any  deviations from the architecture are explicitly understood and justified by  an explicit waiver to the architecture. (2) Product line requirements— rather than just the requirements for the system being acquired—are an  explicit consideration in each acquisition. 3. Contract Tracking and Oversight  Purpose: To ensure that contract activities are performed in accordance  with contractual requirements.
Description:  Contract tracking and oversight is the process by which  contractual agreements are established and contractor efforts to satisfy  those agreements are supervised. It involves information sharing between  the acquirer and contractor to ensure that contractual requirements are  understood, that there are regular measurements to disclose overall project  status and whether problems exist, and that there are appropriate  incentives for ensuring that cost and schedule commitments are met and  that quality products are delivered. Contract tracking and oversight begins  with the award of the contract and ends at the conclusion of the contract’s  period of performance.
Activities:  (1) The acquiring organization has sufficient insight into the  contractor’s activities to manage and control the contractor and ensure that  contract requirements are met. (2) The acquiring organization and  contractor maintain ongoing communication; commitments are agreed to  and implemented by both parties. (3) All contract changes are managed  throughout the life of the contract. (4) The acquiring organization has a  written policy for contract tracking and oversight. (5) Responsibility for  contract tracking and oversight activities is designated. (6) The acquiring  organization involves contracting specialists in the execution of the  contract. (7) A quantitative set of software and system metrics is used to  define and measure product quality and contractor performance. (8) In addition to incentives for meeting cost and schedule estimates,  measurable, metrics-based product quality incentives are explicitly cited in  the contract.
Purpose: To ensure that system investments have an adequate economic  justification.
Description: Economic justification is the process for ensuring that  acquisition decisions are based on reliable analyses of the proposed  investment’s likely costs versus benefits over its useful life, as well as an  analysis of the risks associated with actually realizing the acquisition’s  forecasted benefits for its estimated costs. Moreover, it entails minimizing  the risk and uncertainty of large acquisitions that require spending large  sums of money over many years by breaking the acquisition into smaller,  incremental acquisitions. Economic justification is not a one-time event,  but rather is performed throughout an acquisition’s life cycle in order to  permit informed investment decision making.
Activities: (1) System investment decisions are made on the basis of  reliable analyses of estimated system life cycle costs, expected benefits,  and anticipated risks. (2) Large systems acquisitions are (to the maximum  extent practical) divided into a series of smaller, incremental acquisition  efforts, and investment decisions on these smaller efforts are made on the  basis of reliable analyses of estimated costs, expected benefits, and  anticipated risks.
Purpose: To ensure that evidence showing that the contract products  satisfy the defined requirements are provided prior to accepting contractor  products.
Description: Evaluation is the process by which contract deliverables are  analyzed to determine whether they meet contract requirements. It  includes developing criteria such as product acceptance criteria to be  included into both the solicitation package and the contract. It should be  conducted continuously throughout the contract period as products are  delivered. It begins with development of the products’ requirements and  ends when the acquisition is completed.
Activities: (1) Evaluation requirements are developed in conjunction with  the contractual requirements and are maintained over the life of the  acquisition. (2) Evaluations are planned and conducted throughout the  total acquisition period to provide an integrated approach that satisfies  evaluation requirements and takes advantage of all evaluation results. (3)  Evaluations provide an objective basis to support the product acceptance  decision. (4) The acquisition organization has a written policy for managing  the evaluation of the acquired products. (5) Responsibility for evaluation  activities is designated.
Purpose: To ensure that the project office and its supporting organizations  function efficiently and effectively.
Description: Project management is the process for planning, organizing,  staffing, directing, and managing all project-office-related activities, such  as defining project tasks, estimating and securing resources, scheduling  activities and tasks, training, and accepting products. Project management  begins when the project office is formed and ends when the acquisition is  completed.
Activities: (1) Project management activities are planned, organized,  controlled, and communicated. (2) The performance, cost, and schedule of  the acquisition are continually measured, compared with planned  objectives, and controlled. (3) Problems discovered during the acquisition  are managed and controlled. (4) The acquisition organization has a written  policy for project management. (5) Responsibility for project management  is designated. 7.  Requirements Development and Management Purpose: To ensure that contractual requirements are clearly defined and  understood by the acquisition stakeholders.
Description: Requirements development is the process for developing and  documenting contractual requirements, including evaluating opportunities  for reusing existing assets. It involves participation from end users to  ensure that product requirements are well understood, and that optional  versus mandatory requirements are clearly delineated. Requirements  management is the process for establishing and maintaining agreement on  the contractual requirements among the various stakeholders and for  ensuring that the requirements are traceable, verifiable, and controlled.  This involves baselining the requirements and controlling subsequent  requirements changes. Requirements development and management begins  when the solicitation’s requirements are documented and ends when  system responsibility is transferred to the support organization.
Activities: (1) Contractual requirements are developed, managed, and  maintained. (2) The end user and other affected groups have input into the  contractual requirements over the life of the acquisition. (3) Contractual  requirements are traceable and verifiable. (4) The contractual requirements  baseline is established prior to release of the solicitation package. (5) The  acquisition organization has a written policy for establishing and managing  the contractual requirements. (6) Responsibility for requirements  development and management is designated. (7) Requirements that are  mandatory versus optional are clearly delineated and used in deciding what  requirements can be eliminated or postponed to meet other project goals,  such as cost and schedule constraints.
Purpose: To ensure that risks are identified and systematically mitigated.
Description: Risk management is the process for identifying potential  acquisition problems and taking appropriate steps to avoid their becoming  actual problems. It includes risk identification and categorization based on  estimated impact, development of risk mitigation strategies, and execution  of and reporting on the strategies. Risk management occurs early and  continuously in the acquisition life cycle.
Activities: (1) Projectwide participation in the identification and mitigation  of risks is encouraged. (2) The defined acquisition process provides for the  identification, analysis, and mitigation of risks. (3) Milestone reviews  include the status of identified risks. (4) The acquisition organization has a  written policy for managing acquisition risk. (5) Responsibility for  acquisition risk management activities is designated.
Purpose: To ensure that a quality solicitation is produced and a best- qualified contractor selected.
Description: Solicitation is the process for developing, documenting, and  issuing the solicitation package; developing and implementing a plan to  evaluate responses; conducting contract negotiations; and awarding the  contract. Solicitation ends with contract award.
Activities: (1) The solicitation package includes the contractual  requirements and the proposal evaluation criteria. (2) The technical and  management elements of proposals are evaluated to ensure that the  requirements of the contract will be satisfied. (3) The selection official  selects a supplier who is qualified to satisfy the contract’s requirements. (4)  The acquiring organization has a written policy for conducting the  solicitation. (5) Responsibility for the solicitation is designated. (6) A  selection official has been designated to be responsible for the selection  process and decision. (7) The acquiring team includes contracting  specialists to support contract administration. 10. Transition to Support Purpose: To ensure proper transfer of the system from the acquisition  organization to the eventual support organization.
Description: Transition to support is the process for developing and  implementing the plans for transitioning products to the support  organization. This includes engaging relevant stakeholders in the  acquisition and sharing information about the system’s supporting  infrastructure. Transition to support begins with requirements  development and ends when the responsibility for the products is turned  over to the support organization.
Activities:  (1) The acquiring organization ensures that the support  organization has the capacity and capability to provide the required  support. (2) There is no loss in continuity of support to the products during  transition from the supplier to the support organization. (3) Configuration  management of the products is maintained throughout the transition. (4)  The acquiring organization has a written policy for transitioning products  to the support organization. (5) The acquiring organization ensures that the  support organization is involved in planning for transition to support. (6)  Responsibility for transition to support activities is designated.
section_title: Complementary Best Practices Relevant to Commercial Component-Based IT Business Systems Acquisitions
paragraphs: Purpose: To ensure that commercial product modification is effectively  controlled.
Description: Component modification is the process for limiting the  chances of a commercial product being modified to the point that it  becomes a one-of-a-kind solution because doing so can result in extensive  life cycle costs. Such modifications, if not incorporated into the  commercially available version of the product by the supplier, mean that  every product release has to be modified in accordance with the custom  changes, thus precluding realization of some of the benefit of using a  commercial product.
Activity: (1) Modification of commercial components is discouraged and  allowed only if justified by a thorough analysis of life cycle costs and  benefits.
Purpose: To ensure the integrity and consistency of system commercial  components.
Description: Configuration management relative to commercial  component-based systems is the process for ensuring that changes to the  commercial components of a system are strictly controlled. It recognizes  that when using commercial components, it is the vendor, not the  acquisition or support organization, that controls the release of new  component versions and that new versions are released frequently. Thus,  acquisition management needs to provide for both receiving new product  releases and controlling the implementation of these releases.
Activities: (1) Project plans explicitly provide for evaluation, acquisition,  and implementation of new, often frequent, product releases. (2)  Modification or upgrades to deployed versions of system components are  centrally controlled, and unilateral user release changes are precluded.
Purpose: To ensure that relationships between commercial products are  understood before acquisition decisions are made.
Description: Dependency analysis relative to commercial component- based systems is the process for determining and understanding the  characteristics of these products so that inherent dependencies among  them can be considered before they are acquired. It involves recognizing  that the logical and physical relationships among products impact one  another. This is necessary because commercial products are built around  each vendor’s functional and architectural assumptions and paradigms,  such as approaches to error handling and data access, and these  assumptions and paradigms are likely to be different among products from  different sources. Such differences complicate product integration.  Further, some commercial products have built-in dependencies with other  products that, if not known, can further complicate integration.
Activity: (1) Decisions about the acquisition of commercial components  are based on deliberate and thorough research, analysis, and evaluation of  the components’ interdependencies.4.  Legacy Systems Integration Planning Purpose: To ensure reasonable planning for integration of commercial  products with existing systems.
Description:  Legacy systems integration planning is the process for  ensuring that the time and resources needed to integrate existing systems  with the system being acquired are identified and provided for. It involves  identifying which legacy systems will interact with the system being  acquired and what kinds and levels of testing are required. Integration  planning recognizes that, although some commercial products may provide  mechanisms and information that is helpful in integration with legacy  systems, the unavailability of the source code for commercial products and  the different organizations that are responsible for the two will likely  require additional time and effort.
Activity: (1) Project plans explicitly provide for the necessary time and  resources for integrating commercial components with legacy systems. 5.  Organization Change Management Purpose: To ensure that the organizational impact of using new system  functionality is proactively managed.
Description: Organization change management relative to commercial  component-based systems is the process for preparing system users for the  business process changes that will accompany implementation of the  system. It involves engaging users and communicating the nature of  anticipated changes to system users through training on how jobs will  change. This is necessary because commercial products are created with  the developers’ expectations of how they will be used, and the products’  functionality may require the organization implementing the system to  change existing business processes.
Activities: (1) Project plans explicitly provide for preparing users on the  impact that the business processes embedded in the commercial  components will have on the user’s respective roles and responsibilities.  (2) The introduction and adoption of changes to how users will be  expected to execute their jobs are actively managed.
Purpose: To ensure that a quality solicitation is produced and a best- qualified contractor is selected.
Description: Solicitation relative to commercial component-based systems  is the process for ensuring that a capable contractor is selected. It involves  ensuring that the selected contractor has experience with integrating  commercial component products. This is important because expertise in  developing custom system solutions is different from expertise in  implementing commercial components; it requires different core  competencies and experiences to be successful.
Activity: (1) Systems integration contractors are explicitly evaluated on  their ability to implement commercial components.
Purpose: To ensure that system requirements alone do not drive the system  solution.
Description: Tradeoff analysis relative to commercial product-based  systems is the process for analyzing and understanding the tradeoffs  among competing acquisition variables so as to produce informed  acquisition decision making. It involves planning and executing  acquisitions in a manner that recognizes four competing interests: defined  system requirements, the architectural environment (current and future) in  which the system needs to operate, acquisition cost and schedule  constraints, and the availability of products in the commercial marketplace  (current and future). This analysis should be performed early and  continuously throughout an acquisition’s life cycle.
Activity: (1) Investment decisions throughout a system’s life cycle are  based on tradeoffs among the availability of commercial products (current  and future), the architectural environment in which the system is to operate (current and future), defined system requirements, and acquisition  cost/schedule constraints.8. Vendor and Product Research and Evaluation Purpose: To ensure that vendor and product characteristics are understood  before acquisition decisions are made.
Description: Vendor and product research and evaluation relative to  commercial component-based systems is the process for obtaining reliable  information about both the product being considered and the vendor  offering the product. It involves taking additional steps beyond vendor  representations, such as obtaining information about the vendor’s history,  obtaining information on the vendor’s business strategy relative to  evolution and support of the product, and evaluating copies of the product  in a test environment.
Activities: (1) Commercial component and vendor options are researched,  evaluated/tested, and understood, both early and continuously. (2) A set of  evaluation criteria for selecting among commercial component options is  established that includes both defined system requirements and  vendor/commercial product characteristics (e.g., customer satisfaction  with company and product line).
section_title: Comments from the Department of Defense
paragraphs: The following are GAO’s comments on the Department of Defense’s letter  dated July 1, 2004.
subsections: section_title: GAO Comments
paragraphs: 1. We disagree. While we have updated our report to reflect the additional  information provided in DOD’s comments on the status of its efforts  and the associated milestone, the importance of revising and  maintaining DOD's acquisition policies and guidance, and their  incorporation of acquisition best practices, makes it essential to have  an explicit plan. Among other things, a plan would highlight the  resource constraints that this revision effort has been subject to, would  allow measurement against defined milestones, and would allow  disclosure of progress and impediments.  2. See comment 1. 3. We do not question DOD’s statement that it has an initial repository of  custom software components that can help adapt commercial  components to defense use and reuse. However, this repository does  not satisfy the product line requirements in this best practice and our  recommendation. According to the Software Engineering Institute, the  product line requirements best practice involves more than just reuse.  Under the approach described in DOD’s comments, reuse generally  involves items, such as software modules or components, that  developers are encouraged to use. However, the product line  requirements best practice is not simply encouraging reuse of items in a  repository. Rather, it is planned, enabled, and enforced reuse of such  assets as requirements, models, and architectures that have been  designed and optimized for use in multiple systems. In short, it is  proactive rather than reactive reuse.  4. We have modified our recommendation to use the terminology  “acquisition project management activities” instead of “acquisition  management activities” to eliminate any confusion. Further, we do not  question DOD’s use of integrated product teams as a way to  communicate information. Further, the point of our recommendation is  that there needs to be an explicit recognition in policy and guidance of  the type of information to be communicated and to whom.  Incorporating this best practice is based on the need to ensure that  communication vehicles, such as integrated product teams, are  effective.  5. We agree that the 5000 series contains information on acquisition risk  management, as we state in our report, and that one of the purposes of  DOD’s acquisition framework is to reduce risks. However, there is no  provision in DOD’s acquisition policy or guidelines to ensure that the  status of identified risks are discussed at key decision points. For  example, DOD policy states that a criterion for passing milestone A is  “management and mitigation of technology risk.” However, it does not  provide for what is to be done to manage and mitigate risks, and it does  not provide for reviewing risk status at milestones B or C.  6. While we agree that the toolkit provides relevant change management  information, the toolkit is referenced only once in the 5000 series; and  the reference is only in relation to one phase of the acquisition cycle  (the technology development phase). Given the importance and  relevance of this practice to the successful implementation of  commercial component-based systems, our position, and thus the basis  for our recommendation, is that the best practice’s implementation  would more likely occur if it was visible and recognized in all relevant  stages of DOD’s acquisition cycle.  7.
See comment 6. 8. While the regulations and guidance that DOD cited and referenced in its  acquisition policies discuss the use of information on contractors’ past  performance, they do not discuss evaluating systems integration  contractors on their ability to implement commercial components,  which is the point of the best practice. Further, DOD’s objection to  incorporating this best practice is not consistent with its own  comments. Specifically, DOD commented that it has already taken  steps, through its enterprise systems initiative, to establish blanket  agreements with five contractors who were evaluated on, among other  things, “explicitly proven abilities to implement commercial systems  solutions.” Additionally, while we appreciate DOD’s concern that  incorporation of this best practice can have unintended consequences,  it is important to also recognize that our recommendation is not  intended to restrict a contracting officer’s options. Rather, it is intended  to be one of the many factors considered in the source selection  process when it is relevant.  9. We support the Defense Acquisition Executive’s decision to add more  specifics on systems engineering to the 5000 series, including  provisions that address this best practice and recommendation.  10. See comment 8.  11. We disagree. While we do not question that statutory and regulatory  compliance are referenced in DOD’s integrated process team and  milestone decision point processes, we do not believe that these  reviews are adequately defined with respect to implementation of best  practices because DOD’s policy does not require that the practices’ use  be measured and verified. Rather, it leaves these reviews to the  discretion of the program manager and investment decision authority.  As we state in our report, not requiring that the use of best practices be  measured and verified increases the chance that the practices will not  be followed. Therefore, our position remains that DOD’s policies do not  provide effective controls for ensuring that best practices are  appropriately followed.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: Carl L. Higginbotham, (404) 679-1824 Staff acknowledgments In addition to the individual named above, key contributors to this report  included Nabajyoti Barkakati, Nancy Glover, Madhav Panwar, Morgan  Walts, and Thomas Wright.
subsections: section_title: GAO’s Mission
paragraphs: The Government Accountability Office, the audit, evaluation and  investigative arm of Congress, exists to support Congress in meeting its  constitutional responsibilities and to help improve the performance and  accountability of the federal government for the American people. GAO  examines the use of public funds; evaluates federal programs and policies;  and provides analyses, recommendations, and other assistance to help  Congress make informed oversight, policy, and funding decisions. GAO’s  commitment to good government is reflected in its core values of  accountability, integrity, and reliability.
section_title: Obtaining Copies of GAO Reports and Testimony
paragraphs: The fastest and easiest way to obtain copies of GAO documents at no cost  is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts  newly released reports, testimony, and correspondence on its Web site. To  have GAO e-mail you a list of newly posted products every afternoon, go to  www.gao.gov and select “Subscribe to Updates.”
subsections: section_title: Order by Mail or Phone
section_title: To Report Fraud, Waste, and Abuse in Federal Programs
section_title: Congressional Relations
section_title: Public Affairs