Business Modernization: Disciplined Processes Needed to Better Manage NASA's Integrated Financial Management Program
Published: 2003-11-21T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The National Aeronautics and Space Administration (NASA) has struggled to implement a fully integrated financial management system. The lack of such a system has affected the agency's ability to control program costs, raising concerns about the management of its most costly programs, including the space shuttle program and the International Space Station. In April 2000 NASA initiated the Integrated Financial Management Program (IFMP)--its third effort to improve the agencywide management of its resources. Implementation is expected by fiscal year 2006 with an estimated life-cycle cost of nearly $1 billion. This report (1) assesses NASA's methodology for preparing the current life-cycle cost estimate for implementing IFMP, (2) determines whether NASA's current schedule is reasonable, and (3) evaluates NASA's processes for ensuring adequate cost contingencies.
section_title: What GAO Found
paragraphs: The uncertain reliability of cost estimates, optimistic schedules, and insufficient processes for ensuring adequate funding reserves have put NASA's latest financial management modernization effort at risk. Over the past several years, IFMP's life-cycle cost estimates have fluctuated, and NASA's current estimate is 14 percent greater than the previous estimate. The reliability of these estimates is uncertain because disciplined costestimating processes required by NASA and recognized as best practices were not used in preparing them. For example, IFMP's current life-cycle cost estimate did not include the full cost likely to be incurred during the life of the program, including certain operations costs and costs to retire the system. In addition, NASA did not consistently use breakdowns of work in preparing the cost estimate, as recommended by NASA guidance. In cases where work breakdowns were used, the agency did not always show the connection between the work breakdown estimates and the official program cost estimate. This has been a weakness since the inception. Although more than half of the IFMP modules have been implemented--including the Core Financial module, which is considered the backbone of IFMP--the system may not be fully implemented by the end of fiscal year 2006 as planned. Efforts to complete the integrated system as quickly as possible might have resulted in schedule margins that are insufficient to manage program challenges--such as personnel shortages, uncertainties about software availability, and Office of Management and Budget (OMB) initiatives to implement electronic systems for agency business processes governmentwide. These OMB initiatives have put IFMP in a reactive mode and are already affecting planning for the payroll, procurement, and travel components of the integrated system, which could result in additional schedule delays and cost growth. Finally, reserve funding for IFMP contingencies may be insufficient, which is particularly problematic, given the program's unreliable cost estimates and optimistic schedule. One module--Budget Formulation--is already experiencing potential shortfalls in its reserves, and project officials expressed concerns that the module's functionality may have to be reduced. Yet the program continues to establish funding reserves based on reserve levels set by other high-risk NASA programs, such as NASA's space flight program--not on analyses of the potential cost impact of risks and unknowns specific to IFMP, as required by NASA guidance. Moreover, the program did not quantify the cost impact of high-criticality risks--also required by NASA--or link its risks to funding reserves to help IFMP develop realistic budget estimates.

section_title: Letter
paragraphs: About 90 percent of the National Aeronautics and Space Administration’s  (NASA) annual budget is spent on contractors. Yet since 1990 we have  identified NASA’s contract management as a high-risk area—in part,  because the agency has failed to implement a modern, fully integrated  financial management system. As we reported in January 2003, the lack of  such a system has hampered NASA’s ability to oversee contracts; control  program costs; and ensure an effective human capital management  strategy, raising serious concerns about NASA’s management of its largest  and most costly programs, including the space shuttle program and the  International Space Station.
In April 2000 NASA initiated its third and most recent effort to implement  a modernized financial management system: the Integrated Financial  Management Program (IFMP). Through IFMP, NASA plans to employ  multiple software applications to improve the agencywide management of  its financial, physical, and human resources. NASA expects to complete  IFMP’s implementation in fiscal year 2006 with an estimated life-cycle cost  totaling nearly $1 billion, including operations and maintenance costs.  However, several of the system’s modules remain to be developed, and the  program faces significant challenges in completing them as planned. Given  its previous failures, NASA is under tremendous pressure to ensure that  IFMP is implemented quickly and can achieve its goals.
In April 2003 we issued an interim report on IFMP, which found that  NASA was not following key best practices for acquiring and  implementing IFMP. As agreed, we continued our review in three areas:  (1) whether NASA is acquiring and implementing IFMP in the context of  an enterprise architecture, (2) the extent to which the Core Financial  module will address NASA’s external reporting requirements, and  (3) NASA’s life-cycle cost estimate and schedule for IFMP. We are  responding to the first two issues in separate reports. This report  addresses the third issue—IFMP’s life-cycle cost estimate and schedule.  Specifically, you asked that we (1) assess the reliability of NASA’s  methodology for preparing the current cost estimate for implementing  IFMP, (2) determine whether NASA’s current schedule is reasonable in  terms of progress to date and available resources, and (3) evaluate NASA’s  processes for ensuring the adequacy of reserve funding for contingencies  to mitigate the potential impact of identified program risks and unknowns.  In addition, we have summarized our findings on the three areas  previously cited in a separate report.
subsections: section_title: Background
paragraphs: For more than a decade, we have reported that the lack of a modern  integrated financial management system to produce accurate and reliable  information has hampered NASA’s ability to oversee contracts and  develop good cost estimates for NASA’s programs. In 1990 NASA’s lack of  effective systems and processes for overseeing contractor’s activities  prompted us to identify NASA’s contract management as a high-risk area.  In July 2002 we reported that the accuracy of NASA’s $5 billion cost  growth estimate for the International Space Station was questionable and  that the agency might have difficulty preparing a reliable life-cycle cost  estimate because a modern integrated financial management system  was not available to track and maintain the data needed for estimating and  controlling costs.  NASA’s lack of a fully integrated financial management  system has also hurt the agency’s ability to collect, maintain, and report  the full cost of its projects and programs. For example, in March 2002 we  testified that NASA was unable to provide us with detailed support for the  amounts that it reported to the Congress as obligated against space station  and related shuttle program cost limits as required by the National  Aeronautics and Space Administration Authorization Act of 2000.
IFMP is designed as an integrated system to replace the separate  and incompatible financial management systems used by NASA’s  10 centers. According to the IFMP Program Director, the new system  will provide better decision data, consistent information across centers,  and improved functionality.
Unlike NASA’s previous efforts to modernize its financial management  system, IFMP does not rely on a single contractor. NASA selected System  Applications and Products (SAP) to provide its “best of suite” software  and contracted for implementation services under a separate contract.  NASA has also broken the project into modules that will be implemented  individually—instead of all at once—on the basis of the availability of  proven commercial-off-the-shelf software products. IFMP initially  segmented implementation into 14 modules but has since reorganized the  program into 9. Some of these modules may be further broken out and  others added, depending on the scope of OMB’s e-Government initiatives  and other considerations. Table 1 describes the modules that currently  comprise the system and their status.
When NASA announced in June 2003 that the Core Financial module had  been implemented at all of its centers, only about two-thirds of the  financial events needed for day-to-day financial operations and external  reporting had been implemented. In addition, we found that NASA  deferred implementation of other key core financial module capabilities  and created new problems in recording certain financial transactions.  Thus, full functionality of the system has been deferred, increasing the risk  of additional costs and potentially affecting the implementation of future  modules.
As we reported in April 2003, NASA is not following key best practices for  acquiring and implementing IFMP. For example, NASA has not analyzed  the interdependencies between selected and proposed IFMP components,  and it does not have a methodology for doing so. By acquiring IFMP  components without first understanding system component relationships,  NASA has increased its risk of implementing a system that will not  optimize mission performance and will cost more and take longer to  implement than necessary. In addition, in implementing the Core Financial  module, NASA faces risks in the areas of user needs and requirements  management because the agency did not consider the information needs  of key system users and is relying on a requirements management process  that does not require the documentation of detailed system requirements  prior to system implementation and testing.
section_title: Reliability of IFMP’s Current Life-Cycle Cost Estimate Is Uncertain Owing to a Lack of Disciplined Processes
paragraphs: The reliability of the current life-cycle cost estimate—which has fluctuated  since the initial estimate and is 14 percent greater than the previous  estimate established in February 2002—is uncertain because disciplined  cost-estimating processes required by NASA and recognized as best  practices were not used in preparing the estimate. Specifically, IFMP’s  life-cycle cost estimate did not include the full cost likely to be incurred  during the life of the program. In addition, breakdowns of work to  be performed—or Work Breakdown Structure (WBS)—were not  consistently used in preparing the cost estimate. In cases where work  breakdowns were used to prepare the estimate, the agency did not always  provide a clear audit trail. NASA has made some improvements in the  program’s financial management, such as hiring personnel to provide  oversight and consistency for the cost-estimating process. However, until  NASA uses more disciplined processes such as breakdowns of work in  preparing the program’s cost estimate, the reliability of the life-cycle cost  estimate will be uncertain and the program will have difficulty with  controlling costs.
subsections: section_title: IFMP’s Life-Cycle Cost Estimates Show Overall Increase
paragraphs: Since the program began, cost estimates for IFMP’s 10-year life cycle— fiscal years 2001 through 2010—have fluctuated and increased overall, as  shown in figure 1.
NASA’s current IFMP life-cycle cost estimate totals $982.7 million—an  increase of $121.8 million, or 14 percent, over the previous IFMP life-cycle  cost estimate. The estimate comprises IFMP direct program costs, NASA’s  enterprise support, and civil service salaries/benefits. (See table 2.)
Although direct program costs decreased by $9.5 million, these costs  were shifted to the enterprise support component of the estimate with  the program’s decision to fund only 1 year’s worth of operations and  maintenance, rather than 2 years’ worth from the direct program budget.  In addition, NASA anticipates that operations costs for fiscal years 2007  through 2010—estimated at $137.8 million—will be funded by the NASA  Shared Services Center (NSSC), a planned initiative to consolidate various  agency services such as purchasing and human resources. (See table 3.) As  a result, the fiscal year 2004 budget for the IFMP direct program portion of  implementing the system is $497.5 million.
In March 2003 an independent cost estimate team concluded that there is  an 85 percent confidence level that the direct program portion can be  successfully completed with the available funding of $497.5 million.  However, the direct program portion represents only about half of the  total life-cycle cost estimate. In addition, the team’s conclusion was  contingent on two optimistic assumptions: that there would be no  schedule disruptions and no increase in requirements.
section_title: Disciplined Processes Required by NASA Were Not Used in Preparing IFMP’s Cost Estimates
paragraphs: Reflecting OMB guidance and the best practices of government and  industry leaders, NASA requires that life-cycle cost estimates be prepared  on a full-cost basis, that estimates be summarized according to the current  breakdown of work to be performed, and that major changes be tracked to  the life-cycle cost. OMB guidance calls for a disciplined budget process to  ensure that performance goals are met with the least risk and the lowest  life-cycle cost, which includes direct and indirect costs, operations and  maintenance, and disposal. The Software Engineering Institute (SEI)  echoes the need for reliable cost-estimating processes in managing  software implementations—identifying tasks to be estimated, mapping the  estimates to the breakdown of work to be performed, and having a clear  audit trail are among SEI’s requisites for producing reliable cost estimates.
Despite NASA requirements and OMB and SEI guidance, IFMP did not  prepare a full life-cycle cost estimate—that is, all direct and indirect costs  for planning, procurement, operations and maintenance, and disposal  were not included. For example, the life-cycle cost estimate does not  include the following:  the cost to operate and maintain the system beyond 2010;  the cost of retiring the system;    enterprise travel costs, which are provided monthly by the NASA centers;  and  the cost of nonleased NASA facilities for housing IFMP.
In addition, IFMP did not prepare WBS estimates for active modules—that  is, those currently being implemented. According to NASA guidance,  breaking down work into smaller units helps facilitate cost estimating  and project and contract management, and helps ensure that relevant  costs are not omitted. The guidance also states that the WBS should  encompass both in-house and contractor efforts. According to the IFMP  Deputy Program Director, WBS estimates are not prepared for active  modules because information such as contract task orders can be used to  prepare the cost estimates. However, there is not one overriding contract  where each module is considered a deliverable at a fixed price. Rather,   numerous contracts at both the project and center level for a module’s  implementation—many of which can be awarded for a level of effort at  agreed-upon fixed rates at various phases in the implementation. Without  a WBS estimate for the project as a whole, NASA cannot ensure that all  relevant contractor costs are included in the cost estimate. In addition,  using contract task orders to prepare the cost estimate would not ensure  that government in-house costs are included in the life-cycle cost estimate.
Finally, for modules in the planning phase, the program utilized NASA’s  subject matter experts and professional cost estimators to prepare  business case analyses. However, although these analyses contained WBS  cost estimates, the audit trail from the WBS estimate to the program’s life- cycle cost estimate was not always clear. Without a clear audit trail, it is  difficult to determine whether the differences between the detailed  WBS estimates and the official program cost estimate are appropriate.  The lack of a clear audit trail has been a weakness since the inception of  the program. For example, IFMP was unable to provide us with traceable  support for its baseline cost estimate for direct program costs.
NASA has made some improvements that should help the program prepare  better cost estimates. In May 2002 the NASA Administrator appointed an  executive to provide leadership and accountability in the direction and  operation of the system. The NASA headquarters program office also  hired a business manager to oversee and provide consistency for the  cost-estimating process and provide an analyst to review enterprise  support costs.
section_title: Program Schedule May Be Optimistic
paragraphs: Although NASA guidance requires sufficient program schedule margins  to manage risks, efforts to complete the integrated system as quickly as  possible might have resulted in a schedule that is too compressed to  accommodate program challenges, such as personnel shortages and  uncertainties about software’s availability. If the program schedule margin  is too compressed, the program could incur additional risks, including  added cost growth as well as failure to meet IFMP’s schedule objectives.  OMB’s e-Government initiatives—which aim to streamline agency  business processes and eliminate redundant systems governmentwide— could also provide challenges for NASA’s IFMP planning.  As a result, the  program schedule may be optimistic.
While implementing the Core Financial module (see table 1), IFMP has  faced human resource challenges, and the program continues to face these  challenges with other modules. For example, personnel shortages at  Marshall Space Flight Center for several months affected the Core  Financial project and other projects. In this case, a schedule slip was  avoided, but during fiscal year 2002, the shortages resulted in nearly  $400,000 for extra hours worked by center employees. Human resource  challenges are also affecting the Budget Formulation module. The  simultaneous implementation of this module with the Core Financial  module—an action advised against by a contractor conducting a  lessons-learned study—placed heavy demand on already scarce resources  and added complexity to the program. As a result the schedule for  implementing the Budget Formulation module has already slipped.  Sometimes, relying more on contractor personnel can alleviate  shortfalls in civil service personnel, but a recent Budget Formulation  project status report indicated that the implementation contractor might  also have difficulties acquiring and/or retaining qualified personnel. The  implementation schedules for the remaining modules overlap, putting the  program at further risk of schedule slippages.
Uncertainty regarding software availability also puts the program at risk  for completing the integrated system on schedule. For example, complete  software solutions and requirements for IFMP’s Contract Administration  module have not yet been determined. Although contract-document-  generation software is available and tailored to meet the unique interface  and reporting requirements of the federal government, the “best of suite”  software solution—SAP—does not currently meet these requirements.  NASA faces the same challenge with IFMP’s Human Resources  Management module. NASA’s monthly status reports show that the  program is working with SAP to develop a software solution for the  Human Resources Management module that will meet federal government  requirements, but the outcome is uncertain. In addition, the program could  adopt an e-Government solution for its Human Resources Management  module rather than the SAP solution.
Inserting e-Government solutions into IFMP planning—which calls  for using “best of suite” software—could create more difficult interface  development and a less-integrated system, thus interrupting the program’s  cost and schedule. E-Government initiatives are already affecting  NASA’s planning for the payroll, procurement, and travel modules in  the integrated system. For example, the payroll function, which was  once part of the Human Resources Management module, will likely  become a separate module under e-Government. Similarly, the Contract  Administration module has been split into two components: one for  procurement document generation, for which software is available  although requirements are not finalized, and one for the remainder of  NASA’s Contract Administration requirements, for which requirements  and software are currently unknown. Furthermore, e-Travel could replace  the Travel Management module, which has already been implemented.
According to the program’s fiscal year 2002 Independent Annual Review,  e-Government initiatives are forcing the program into a reactionary mode,  thus increasing risk to the program’s success. The review specifically  noted that (1) the benefits of a fully integrated system could be lost under  e-Government, (2) the scope of IFMP and timing of future projects’  implementation have become uncertain, and (3) cost increases and  schedule slippage to accommodate directives may occur.
section_title: Processes Insufficient to Ensure Adequate Funding Set Aside for Contingencies
paragraphs: In addition to the uncertain reliability of IFMP’s life-cycle cost estimates  and optimistic schedules, NASA cannot ensure that the funding set aside  for program contingencies is sufficient because the program did not  consistently perform in-depth analyses of the potential cost impact of risks  and unknowns specific to IFMP, as required by NASA guidance. Moreover,  the program did not quantify the cost impact of identified risks, link its  risks to funding reserves, or consistently set aside cost contingencies for  these risks.
subsections: section_title: In-Depth Analysis Not Performed in Establishing Cost Contingencies
paragraphs: NASA guidance stipulates that programs incorporate financial reserves,  schedule margins, and technical performance margins to provide the  flexibility needed to manage risks. According to the guidance, financial  reserves are to be established and maintained commensurate with  programmatic, technical, cost, and schedule risks. In other words, cost  contingencies should be tailored to the specific risks associated with a  particular program or project. In addition, NASA guidance suggests that  tools such as Probabilistic Risk Assessment can help in analyzing risk.
Although NASA’s business case analyses include a risk assessment and  recommended reserve levels, we found no evidence that these  recommended levels were used in establishing the actual reserve levels for  the IFMP module projects. Regardless, the actual levels established did not  match the recommended levels in the business case analyses in most  cases. We found that reserves for some IFMP modules—both in the  planning and active phase—were based not on IFMP-specific risks but on  reserve levels for other high-risk NASA programs. For example, for a  number of IFMP modules, reserves were set at levels used for spacecraft  implementations—typically about 30 percent—because industry  experience showed that large cost overruns in system implementations  such as IFMP are common. Yet it is unclear whether this reserve margin  is adequate for IFMP because the effect of IFMP-specific risks and  assumptions—such as uncertainties relating to software, schedule, and  OMB’s e-Government initiatives—were not analyzed. In addition, some  of the enterprises supporting the module projects described their method  of establishing funding reserves as a combination of rules of thumb  and guesswork.
The Budget Formulation module has already experienced shortfalls in  its reserves, and project officials expressed concerns that the module’s  functionality may have to be reduced. As of April 2003, the module  had expended its baseline reserves, which were established at about  20 percent on the basis of the level of risk for space flight missions—not  on the risks specific to the module. Although the project was able to bring  its budget back into balance by obtaining an agreement with SAP to limit  overtime pay to time in excess of 50 hours per week, its remaining  reserves total only $83,000 to cover all contingencies—including those that  could require changes to the Budget Formulation module.
section_title: Cost Impact of Identified Risks Not Quantified or Linked to Cost Contingencies
paragraphs: NASA requires programs to quantify the cost impact of high-criticality  risks and to determine to what extent reserves may be exhausted,  should the risks become reality. According to SEI, estimating the potential  cost and schedule impact for all identified risks is an element of good  estimating practice. Quantifying the cost impact of identified risks and  clearly and consistently linking the risk database to funding reserves helps  programs develop realistic budget estimates.
While IFMP identifies program risks, analyzes their severity, and plans  mitigation actions, the program typically does not prepare a cost impact  analysis for identified risks nor does it consistently link identified risks to  funding reserves to ensure that funds are available, should the risk occur.  For example, in February 2003, the Travel Management Project found that  some components of the Travel Management module might not satisfy  individual centers, be funded, or be technically feasible. However, the  cost impact of this risk, as well as others, was not quantified. Similarly,  in June 2003, the Budget Formulation module did not quantify the  cost impact of a number of identified risks. Without estimating the  potential cost impact of these risks, NASA cannot determine whether  it has sufficient reserves to cover the risks—which is particularly  problematic for Budget Formulation, since virtually no reserves remain  for this module.
Furthermore, in its July 2003 monthly status report, the IFMP headquarters  office identified three high-criticality risks that could have a cost impact  on the overall program; however, no liens were set aside against reserves  for these risks:    Reductions to out-year budgets could affect the implementation of future  integrated modules or the ongoing evolution of existing modules.
An e-Government solution may be adopted for human resources  management rather than the IFMP solution, resulting in more difficult  interface development and a less-than-integrated solution.
E-Government initiatives and policy decisions could disrupt IFMP  modules, resulting in delays or additional resource impacts.
An independent cost estimate team identified and quantified the impact of  two IFMP program risks, indicating that the cost and schedule impact of a  risk on a program or project can be sizeable. First, the team identified a  high-probability risk that NASA’s “full cost requirement”—in which all  direct and indirect agency costs, including civil service personnel costs,  are tied to individual programs and projects—could affect the Budget  Formulation module. The team estimated this risk at $2 million to  $3 million, with a potential schedule slip of 3 to 6 months. The Budget  Formulation Project is currently trying to determine what impact it may  have. The second risk identified by the independent cost review team— that the Core Financial module may be transitioned to operations  before all integration points are addressed—could be more costly.  The team estimated this risk at $10.5 million to $20 million, also with a  potential 3- to 6-month schedule slip. However, the team considered this  risk as having a low probability of occurrence.
section_title: Conclusion
paragraphs: NASA is at a critical juncture and faces major challenges in improving  contract management and controlling costs. These challenges seriously  affect the agency’s ability to effectively manage its largest and most  costly programs. A modern integrated financial management system, as  envisioned in IFMP, is critical to ensuring that NASA has accurate and  reliable information to successfully meet these challenges. NASA has  made some improvements during the past year, such as hiring personnel  to provide the cost-estimating process with oversight and consistency.  However, if IFMP continues to ignore disciplined processes in estimating  program costs and impacts, it is unlikely that the program will meet  its goals.
section_title: Recommendations for Executive Action
paragraphs: To ensure that IFMP’s life-cycle cost estimate conforms to NASA guidance  and best practices, we recommend that the NASA Administrator direct  IFMP to do the following:    Prepare cost estimates by the current Work Breakdown Structure for the  remaining modules.
Provide a clear audit trail between detailed WBS estimates and the  program’s cost estimate for the remaining modules.
Prepare a full life-cycle cost estimate for the entire IFMP that meets  NASA’s life-cycle cost and full cost guidance.
To ensure that contingencies are funded in accordance with NASA  guidance and best practices, we recommend that the NASA Administrator  direct IFMP to do the following:    Utilize a systematic, logical, and comprehensive tool, such as Probabilistic  Risk Assessment, in establishing the level of financial reserves for the  remaining module projects and tailor the analysis to risks specific to IFMP.
Quantify the cost impact of at least all risks with a high likelihood of  occurrence and a high magnitude of impact to facilitate the continuing  analysis necessary to maintain adequate reserve levels.
Establish a clear link between the program’s risk database and financial  reserves.
section_title: Agency Comments and Our Evaluation
paragraphs: Although NASA concurred with our recommendations for corrective  action, NASA indicated that its current processes are adequate for (1)  preparing WBS cost estimates, (2) estimating life-cycle costs, and (3)  establishing reserves on the basis of IFMP-specific risks. The agency cited  its business case analyses as the methodology through which it is  accomplishing these tasks.
We disagree that NASA’s current processes are adequate, and our  recommendations are aimed at improving these processes. As discussed in  this report, while NASA prepares WBS cost estimates for IFMP modules in  the planning phases by using business case analyses, it does not prepare  WBS cost estimates for active modules. And although IFMP indicates that  preparing cost estimates by using contract task orders is an appropriate  methodology, this approach will not ensure that all relevant costs,  including both contractor and government in-house costs, are included in  the life-cycle cost estimate. Regarding contract costs, there is not one  overriding contract where each module is considered a deliverable at a  fixed price. Rather, there are numerous contracts at both the project and  center level for implementing modules—many of which can be awarded  for a level of effort at agreed-upon fixed rates at various phases in the  implementation. Without a WBS estimate for the project as a whole, NASA  cannot ensure that all relevant contractor costs are included in the cost  estimate. In addition, using contract task orders to prepare the cost  estimate would not ensure that government in-house costs are included in  the life-cycle cost estimate.
According to NASA, IFMP will improve its business case analyses by  providing better estimates of operational costs through the expected life of  the module, retirement costs, and other full life-cycle costs. However, as  discussed in this report, an audit trail is needed between the detailed  estimates contained in the business case analyses and the program’s life- cycle cost estimate to ensure that these improvements are reflected in the  program’s official cost estimate.
Finally, as discussed in this report, although NASA’s business case  analyses include recommended reserve levels, we found no evidence that  these recommended levels were used in establishing the actual reserve  levels for the IFMP module projects. Regardless, the actual levels  established did not match the recommended levels in most cases. We  found that the program established funding reserves on the basis of  reserve levels set by other high-risk NASA programs, rather than on IFMP- specific risks as required by NASA guidance.
section_title: Scope and Methodology
paragraphs: To assess the reliability of NASA’s methodology for preparing the  current cost estimate for IFMP, we reviewed program and project-level  documentation to obtain an understanding of NASA’s current cost  estimate and its major components and the methodology used to develop  the estimate. We also interviewed program and project officials to clarify  our understanding of the cost estimate and how NASA derived it. In  addition, we compared the program’s cost-estimating methodology with  SEI best practices, OMB requirements, and NASA’s own procedures and  guidance. Finally, we reviewed internal and independent analyses of the  cost estimate. We did not attempt to validate NASA’s estimate; rather, we  reviewed NASA’s processes for preparing its estimate.
To determine whether NASA’s current schedule is reasonable in terms  of progress to date and available resources, we reviewed the program’s  schedule objectives and NASA’s policies for managing program and  project schedules. We monitored the schedule and risks to the schedule  through our review of the program’s monthly status reports and internal  NASA briefings. We interviewed program and project officials to ascertain  NASA’s progress against the schedule.
To evaluate NASA’s processes for ensuring the adequacy of cost  contingencies to mitigate the potential impact of identified program risks  and unknowns, we reviewed governmentwide and NASA policies and  SEI best practices for managing risk and establishing cost contingencies.  We also interviewed program officials at NASA headquarters and project  managers to obtain an understanding of how reserve levels were  established and maintained for the program. We then compared IFMP’s  processes for ensuring adequate cost contingencies with processes  dictated by OMB and NASA guidance and by best practices.
To accomplish our work, we visited NASA headquarters, Washington,  D.C.; Marshall Space Flight Center, Alabama; and Goddard Space  Flight Center, Maryland. We also contacted officials at Glenn Research  Center, Ohio.
We performed our review from April through September 2003 in  accordance with generally accepted government auditing standards.
As agreed with your offices, unless you announce its contents earlier,  we will not distribute this report further until 30 days from its date. At  that time, we will send copies to interested congressional committees;  the NASA Administrator; and the Director, Office of Management and  Budget. We will make copies available to others upon request. In addition,  the report will be available at no charge on the GAO Web site  at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or lia@gao.gov. Key contributors to this  report are acknowledged in appendix I.
section_title: Appendix I: Comments from the National Aeronautics and Space Administration
section_title: Appendix II: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: Staff making key contributions to this report were Jerry Herley, Erin  Schoening, LaTonya Miller, and Karen Sloan.