NASA: Projects Need More Disciplined Oversight and Management to Address Key Challenges
Released: 2009-03-05T00:00:00

Letter

Acquisition Management Problems Persist

We assessed 18 projects in NASA’s current portfolio. Four were in the “formulation” phase, a time when system concepts and technologies are still being explored and 14 were in the “implementation” phase, where system design is completed, scientific instruments are integrated, and a spacecraft is fabricated. When implementation begins, it is expected that project officials know enough about a project’s requirements and what resources are necessary to meet those requirements that they can reliably predict the cost and schedule necessary to achieve its goals. Reaching this point requires investment. In some cases, projects that we reviewed spent 2 to 5 years and up to $100 million or more before being able to formally set cost and schedule estimates.

Ten of the projects in our assessment for which we received data and that had entered the implementation phase experienced significant cost and/or schedule growth from their project baselines. Based on our analysis, development costs for projects in our review increased by an average of almost 13 percent from their baseline cost estimates—all in just two or three years—including one that went up more than 50 percent. It should be noted that a number of these projects had experienced considerably more cost growth before a baseline was established in response to statutory reporting requirements. Our analysis also shows that projects in our review had an average delay of 11 months to their launch dates.

We found challenges in five areas that occurred throughout the various projects we reviewed that can contribute to project cost and schedule growth. These are not necessarily unique to NASA projects and many have been identified in other weapon and space systems that we have reviewed and have been prevalent in the agency for decades.

Technology maturity. Four of the 13 projects in our assessment for which we received data and that had entered the implementation phase did so without first maturing all critical technologies, that is they did not know that technologies central to the project’s success could work as intended before beginning the process of fabricating the spacecraft. This means that knowledge needed to make these technologies work remained unknown well into development. Consequences accrue to projects that are still working to mature technologies well into system development, when they should be focusing on maturing system design and preparing for production. Simply put, projects that start with mature technologies experience less cost growth than those that start with immature technologies.

Design stability. The majority of the projects in our assessment that held a critical design review did so without first achieving a stable design. If design stability is not achieved, but product development continues, costly re-designs to address changes to project requirements and unforeseen challenges can occur. All of the projects in our assessment that had reached their critical design review and that provided data on engineering drawings experienced some growth in the total number of design drawings after their critical design review. Growth ranged from 8 percent to, in the case of two projects, well over 100 percent. Some of this increase can be attributed to change in system design after critical design review.

Complexity of heritage technology. More than half the projects in the implementation phase—eight of them—encountered challenges in integrating or modifying heritage technologies. Additionally, two projects in formulation—Ares I and Orion—also encountered this problem. We found that the projects that relied on heritage technologies underestimated the effort required to modify them to the necessary form, fit, or function.

Contractor performance. Six of the seven projects that cited contractor performance as a challenge also experienced significant cost and/or schedule growth. For example, through our discussions with the project offices, we were informed that contractors encountered technical and design problems with hardware that disrupted development progress.

Development partner performance. Five of the thirteen projects we reviewed encountered challenges with a development partner. In these cases, the development partner could not meet its commitments to the project within planned timeframes. This may have been a result of problems within the specific development partner organization or as a result of problems faced by a contractor to that development partner.

Common Acquisition Management Challenges Persist between NASA and DOD

The challenges we identified in the NASA assessment are similar to ones we have identified in other weapon systems, including Defense space systems. We testified last year that DOD space system cost growth was attributable to programs starting before they have assurance that capabilities being pursued can be achieved within available resources and time constraints. For example, DOD’s National Polar-orbiting Operational Environmental Satellite System (NPOESS) has increased in cost from roughly $6 billion to over $11 billion because of challenges with maturing key technologies. We have also tied acquisition problems in space systems to inadequate contracting strategies and contract and program management weaknesses. Further, we issued a report in 2006 that found DOD space system cost estimates were consistently optimistic. For example, DOD’s Space-Based Infrared High program was originally expected to cost about $4 billion and is now expected to cost more than $10 billion.

We have found these problems are largely rooted in the failure to match the customer’s needs with the developer’s resources—technical knowledge, timing, and funding—when starting product development. In other words, commitments were made to achieving certain capabilities without knowing whether technologies and/or designs being pursued could really work as intended. Time and costs were consistently underestimated. As we have discussed in previous work on space systems at both DOD and NASA, a knowledge-based approach to acquisitions, regardless of the uniqueness or complexity of the system is beneficial because it allows program managers the opportunity to gain enough knowledge to identify potential challenges earlier in development and make more realistic assumptions about what they can achieve.

NASA Is Making a Concerted Effort to Reduce High Risk in Acquisition Management but More Needs to Be Done

NASA has also taken significant steps to improve in the high risk area of acquisition management. For example, NASA revised its acquisition and engineering polices to incorporate elements of a knowledge-based approach that should allow the agency to make informed decisions. The agency is also instituting a new approach whereby senior leadership is reviewing acquisition strategies earlier in the process and has developed broad procurement tenets to guide the agency’s procurement practices. Further, NASA is working to improve management oversight of project cost, schedule, and technical performance with the establishment of a baseline performance review with senior management. In order to improve it’s contracting and procurement process, NASA has instituted an agency wide standard contract-writing application intended to ensure all contracts include the most up-to-date NASA contract clauses and to improve the efficiency of the contracting process. NASA is also requiring project managers to quantify the program risks they identify and collect more consistent data on project cost and technologies. It is taking other actions to enhance cost estimating methodologies and to ensure that independent estimates are used.

These changes brought the policy more in line with best practices for product development. However, as we previously reported, NASA lacks defined requirements across centers and mission directorates for consistent metrics that demonstrate knowledge attainment through the development cycle. In order for a disciplined approach to take hold, we would expect project officials across the agency to be held accountable for following the same required policies.

More steps also need to be taken to manage risk factors that NASA believes are outside of its control. NASA asserts that contractor deficiencies, launch manifest issues, partner performance, and funding instability are to blame for the significant cost and schedule growth on many of its projects that we reviewed. Such unforeseen events, however, should be addressed in project-level, budgeting and resource planning through the development of adequate levels of contingency funds. NASA cannot be expected to predict unforeseen challenges, but being disciplined while managing resources, conducting active oversight of contractors, and working closely with partners can put projects in a better position to mitigate these risks should they occur. Realistically planning for and retiring technical or engineering risks early in product development allows the project to target reserves to issues NASA believes are outside of its control.

In conclusion, managing resources as effectively and efficiently as possible is more important than ever for NASA. The agency is undertaking a new multi-billion dollar program to develop the next generation of spacecraft for human spaceflight and at a time when it is faced with increasing demands to support important scientific missions, including the study of climate change, and to increase aeronautics research and development. By allowing major investment commitments to continue to be made with unknowns about technology and design readiness, contractor capabilities, requirements, and/or funding, NASA will merely be exacerbating the inherent risks it already faces in developing and delivering new space systems. Programs will likely continue to experience problems that require more time and money to address than anticipated. Over the long run, the extra investment required to address these problems may well prevent NASA from pursuing more critical science and space exploration missions. By contrast, by continuing to implement its acquisition management reforms and ensuring programs do not move forward with such unknowns, NASA can better align customer expectations with resources, minimize problems that could hurt programs, and maximize it ability to meet increased demands.

Madam Chairwoman, this concludes my statement. I will be happy to answer any questions that you have.

GAO Contacts and Staff Acknowledgments

For additional information, please contact Cristina Chaplain at 202-512- 4841 or chaplainc@gao.gov. Individuals making contributions to this testimony include Jim Morrison, Assistant Director; Shelby S. Oakley, Assistant Director; Greg Campbell; Richard A. Cederholm; Brendan S. Culley; Deanna R. Laufer; Kenneth E. Patton; and Letisha T. Watson.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.