Telecommunications: GSA Has Accumulated Adequate Funding for Transition to New Contracts but Needs Cost Estimation Policy
Published: 2007-02-23T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The General Services Administration (GSA) and its customer agencies are preparing to transition new governmentwide telecommunications contracts known as the Networx program. GSA estimated the costs for which it is responsible to be $151.5 million. This report addresses (1) the soundness of the analysis GSA used to derive the estimate of funding that would be required for the transition and (2) whether GSA will have accumulated adequate funding to pay for transition costs. In performing this work, GAO reviewed cost estimation best practices, analyzed relevant GSA documents, and performed an uncertainty analysis on GSA's estimate.
section_title: What GAO Found
paragraphs: GSA did not use sound analysis when estimating the amount of funding needed to meet its transition-related commitments. Specifically, its analysis was not sufficiently accurate, comprehensive, documented, or validated. A primary weakness is that the estimate is largely based an assumption--known as the transition traffic factor--that 76 percent of the services provided under the current contracts would be moved to a different provider under the Networx contracts. However, according to program officials, this assumption is intentionally conservative and represents a worst-case scenario that is unlikely to occur. Additionally, GSA may have double-counted a cost and did not update its analysis to reflect a nearly 2-year delay. Finally, GSA did not document significant assumptions and data sources used in its analysis, or validate it. These weaknesses can be attributed in part to the lack of a cost estimation policy that reflects best practices. While GSA's intentionally conservative approach minimizes the risk that it would have inadequate funds to pay for committed transition costs, it increases the risk that GSA will retain excess funds that could be used for other purposes. GSA has accumulated adequate funding to support its anticipated transition costs. As of fiscal year-end 2006, GSA had approximately $142 million in a transition reserve. GAO analysis of the estimate indicates it is unlikely that GSA will need more than it has already accumulated to fund the transition. Specifically, the $142 million already retained will be adequate to cover anticipated costs 96 percent of the time. The recent merger of two GSA funds gives the agency additional flexibility that reduces its need to accumulate the entire $151.5 million it estimated would be needed. With Networx contracts scheduled to be awarded starting in March 2007, GSA will soon have the information necessary to reassess the main assumption underlying its estimate--the transition traffic factor--and address the weaknesses GAO identified. Once this has been accomplished, GSA can reevaluate the funding needed to meet anticipated commitments.

section_title: Letter
paragraphs: As innovations in telecommunications services continue to transform the  way the federal government conducts business, the General Services  Administration’s (GSA) governmentwide telecommunications acquisition  programs offer federal agencies the opportunity to apply innovative  services and solutions to their operations. With the current set of  governmentwide telecommunications contracts approaching expiration,  GSA and its customer agencies will have to transition the services  acquired under these contracts to their replacements, known collectively  as Networx.
GSA will incur program management costs associated with planning and  executing this transition. It has also made a commitment to absorb certain  agency transition costs. To ensure it would have the funds necessary to  pay for these costs, GSA estimated that it would need to set aside  approximately $151.5 million.
This report responds to your request that we determine (1) the soundness  of the analysis GSA used to derive the estimate of funding that would be  required for the transition and (2) whether GSA will have accumulated  adequate funding to pay for its transition management costs. To  accomplish the first objective, we conducted a search of over 250  documents from both government and industry for examples of best  practices in the field of cost estimation and identified common  characteristics among them. We determined that high-quality, reliable  estimates should be accurate, comprehensive, well-documented, and  validated. Using these characteristics, we analyzed transition estimate  documentation developed by GSA, documentation provided by GSA on the  previous transition, the Networx Request for Proposals, and other relevant  documents. To accomplish the second objective, we analyzed financial  and operational documents from related GSA programs. In addition, we  interviewed GSA program officials about both objectives and conducted  an analysis of GSA’s estimate to examine the effects of varying the main  cost driver in the estimate. We conducted our work at GSA’s Washington,  D.C., area headquarters between June 2006 and January 2007 in  accordance with generally accepted government auditing standards. A  detailed discussion of our objectives, scope, and methodology can be  found in appendix I.
subsections: section_title: Background
paragraphs: As part of its mission of providing federal agencies with acquisition  services and solutions at best value, GSA’s technology programs offer  agencies options to acquire needed telecommunications services. An  option chosen by more than 135 agencies is the FTS2001 program, which  consists of two large governmentwide telecommunications contracts—one  awarded to Sprint in December 1998 and the other to MCI in January  1999—and FTS2001 crossover contracts.
GSA is planning to replace the FTS2001 contracts, FTS2001 crossover  contracts, and separate wireless contracts with a new set of contracts.  Collectively known as the Networx program, these new contracts are to  provide governmentwide telecommunications services through two  indefinite-delivery/indefinite-quantity acquisitions—Networx Universal  and Networx Enterprise. The Universal acquisition is expected to satisfy  the requirements for a full range of national and international network  services and, according to GSA, to ensure the continuity of broad-ranging  services with global geographic coverage rendered under expiring  contracts. The Enterprise acquisition is expected to offer agencies  leading-edge services and solutions with less extensive geographic and  service requirements than Universal. The services required in these  contracts focus on Internet-based offerings and related security and  management services.
GSA expects the transition to begin when Networx Universal awards are  made in March 2007 and to continue until fiscal year 2010. Because the  FTS2001 contracts with Sprint Nextel Corporation and Verizon Business  expired in December 2006 and January 2007, respectively, GSA and the  incumbent vendors negotiated separate sole-source contracts that  essentially extend the terms of the FTS2001 contracts for 42 months.  These sole-source contracts were awarded to ensure uninterrupted service  and allow agencies adequate time to complete the transition to Networx.
GSA is working with representatives of federal agencies to prepare for the  upcoming transition to Networx, both directly and through the  Interagency Management Council (IMC), a group of senior federal  information resource officials who advise GSA on issues related to  telecommunications contracts. The IMC has worked with GSA to  document lessons learned from the transition to FTS2001 that began in the  late 1990s, GSA’s most recent governmentwide telecommunications  transition. One important lesson learned was that GSA and agency plans  for funding transition expenses should be determined early to allow  agencies to gauge the impact of transition expenses on their budgets.  Specifically, the lessons-learned document recommended that guidelines  be established to allow agencies to complete the financial planning  required to ensure that the resources needed for transition were available.  This led the IMC and GSA to develop a Taxonomy and Allocation of  Transition Costs document that identified which Networx transition costs  would be borne by GSA and which would be borne by transitioning  agencies.
This taxonomy document indicated that GSA will incur or reimburse  agencies, including:    GSA contractor support costs to, for example, aid in planning for the  transition and oversight of Networx contractors. GSA officials also  indicated that contractor support will be used to develop a methodology  for tracking transition progress and the establishment of a transition  coordination center; and   certain costs incurred by agencies during transition.
In 2004, following the development of the taxonomy document, GSA  generated an estimate of its costs for the transition. GSA’s methodology  for its estimate was to:    develop assumptions for the estimate,    define and develop a baseline for the estimate based on experiences and  lessons learned from the previous transition,    determine the network growth as well as the total business volume  projections for fiscal year 2006 based on historical traffic and cost trends,   define the transition cost elements,    define and develop a formula for calculating an estimate for each cost  element, and   design and develop the estimated transition cost model for sensitivity  analysis.
Using this methodology, GSA estimated that it would need a total of $151.5  million for a 30-month transition, most of which would be used to  reimburse agencies’ transition costs. Table 1 below details the transition  costs GSA expects to pay.
GSA planned to pay for its costs and the reimbursement of certain agency  costs using its Information Technology (IT) Fund. The IT Fund was a full- cost recovery revolving fund whereby GSA fully recovered all costs of its  technology programs and its operations via estimated fee rates. The fees,  which are charged to agencies for the use of GSA contracts cover the  direct costs of its operations—such as the development and management  of contract vehicles—and indirect costs associated with its headquarters,  such as support for the Offices of the Chief Information Officer and the  Chief Financial Officer. The IT Fund allowed GSA to stabilize rates for its  services when expenses varied and was used to provide funding for the  previous transition to FTS2001. The IT Fund also contained a working  capital reserve that was used to offset losses due to fluctuations in  business volumes and other unexpected contingencies. At each fiscal year- end, the uncommitted balance of funds remaining was to be transferred to  the general fund of the treasury as miscellaneous receipts.
Recently, a new law changed the structure of the IT Fund and GSA’s  organization. On October 6, 2006, the General Services Administration  Modernization Act combined the IT Fund with another GSA revolving  fund—the General Supply Fund—to create the Acquisition Services Fund.  This legislation, which also merged GSA’s technology and supply  programs, made all capital assets and balances remaining in the IT and  General Supply Funds available for the purposes of the Acquisition  Services Fund. According to GSA, the Federal Acquisition Service will  increase agency savings, enhance GSA’s capability to meet customer  requirements for excellence, and improve internal efficiencies.
section_title: GSA’s Analysis Was Not Sound
paragraphs: The analysis GSA used to derive its estimate of $151.5 million was not  sound because it was not sufficiently accurate, comprehensive,  documented, or validated. The analysis was not sufficiently accurate  because it is largely based on the assumption that agencies will transition  76 percent of the services acquired under the current FTS2001 contracts to  a different provider under Networx—an intentionally conservative  scenario that GSA program officials believe is unlikely to occur. Further,  the analysis has not been updated after a nearly 2-year delay in the  contract award. While GSA appears to have included all pertinent costs, it  may have double-counted a cost, calling into question the  comprehensiveness of its analysis. In addition, GSA did not document  significant assumptions and data. Finally, GSA’s analysis was not validated  by an independent cost estimate nor was an uncertainty analysis  performed that would allow GSA to quantify the level of confidence it has  in its estimate. These weaknesses can be attributed in part to the lack of a  cost estimation policy that would help to ensure that such estimates are  developed using best practices. While an intentionally conservative  approach minimizes the risk that GSA would have inadequate funds to pay  for committed transition costs, it increases the risk that GSA will retain  excess funds that could be used for other purposes.
subsections: section_title: GSA’s Analysis Was Not Sufficiently Accurate
paragraphs: Estimates are accurate when they are not overly conservative, based on an  assessment of the most likely costs, and adjusted properly for inflation.
Best practices further dictate that as schedules change, cost estimates  should be revised to provide management with insight into the current  program status, effective control of the program, and the ability to balance  resources and the budget.
The analysis GSA used to derive the estimate, however, was not  sufficiently accurate. First, the estimate’s main cost driver is based on the  assumption that agencies will transition 76 percent of the services  acquired under the current FTS2001 contracts to a different provider  under Networx—60 percent due to agencies being forced to change  providers and 16 percent due to voluntary changes. However, according to  program officials, the 76 percent “transition traffic factor” is intentionally  conservative and represents a worst-case scenario that is unlikely to  occur.
The 76 percent transition traffic factor is also overly conservative when  compared with the previous transition. Then, approximately 60 percent of  services were shifted from an incumbent to a different provider under  FTS2001, the bulk being forced to change providers when an incumbent  providing more than half of the services (AT&T) was not awarded an  FTS2001 contract. This forced shift happened in part because GSA limited  the FTS2001 awards to only two vendors. In contrast, GSA has placed no  such limit on the number of Networx vendors, and therefore, all  incumbent FTS2001 vendors could potentially be awarded a Networx  contract. Further, because the most-used FTS2001 incumbent (Verizon  Business) provides approximately 50 percent of the services under the  current FTS2001 contracts, the assumption of a 60 percent forced shift  would only be realized if no awards were made to Verizon Business and at  least one other incumbent. Finally, GSA’s assumption of a 16 percent  voluntary shift is significantly higher than the 3 percent that voluntarily  changed providers during the previous transition.
Program officials could not identify any basis for the assumption that  voluntary changes will reach 16 percent. However, the officials stated that  the percentage of voluntary changes could be higher because agencies will  likely have more options when selecting vendors and because of improved  guidance on federal requirements regarding the fair opportunity process,  which is intended to give each awardee an equal opportunity to compete  for agencies’ telecommunications services requirements based on  agency-established selection criteria.
Second, the analysis has not been updated to reflect schedule changes.  When the estimate was developed in 2004, GSA expected to award the first  Networx Universal contracts in mid-2005. Since that time, delays have  pushed the time frame back almost 2 years. Universal awards are now  expected in March 2007. GSA’s analysis has not been adjusted to reflect  additional inflation during this period, which could increase the estimates  total by as much as $9 million.
Finally, GSA has not revised its analysis to reflect an assessment of most  likely costs using currently available information. The analysis assumes  that reimbursable agency transition costs would be greater than during the  previous transition due to growth in the volume of services ordered. While  GSA estimated that service levels would grow 60 percent over the previous  transition by the time of contract award, as of August 2006, service levels  had actually grown by 55.9 percent and are now expected to remain stable.  The overestimation of service growth added approximately $1.7 million to  the estimate.
section_title: GSA’s Analysis May Not Be Comprehensive
paragraphs: Estimates are comprehensive when their level of detail ensures that all  pertinent costs are included and no costs are double-counted. It is  important to ensure the completeness, consistency, and realism of the  information contained in the cost estimate.
GSA appears to have included all pertinent costs in its analysis; however,  according to officials, a specific agency cost valued at $4.3 million may  have been double-counted. For several of the transition costs estimated,  GSA based its calculations on the actual charges incurred during the  previous transition. However, officials indicated that a specific agency  cost that is estimated separately in its current estimate may have already  been included in a more general category of its previous costs. Despite this  uncertainty, GSA calculated the separate total for this specific agency cost  based on current service levels and added it to its estimate. As a result, it  is likely this specific agency cost is double-counted and therefore  overstated.
section_title: GSA’s Analysis Was Not Adequately Documented
paragraphs: Cost estimates are well-documented when they can be easily repeated or  updated and can be traced to original sources through auditing. Rigorous  documentation increases the credibility of an estimate and helps support  an organization’s decision-making process. The documentation should  explicitly identify the primary methods, calculations, results, rationales or  assumptions, and sources of the data used to generate each cost element.
GSA provided us documentation of its methodology, the calculations it  used to derive each cost element, results, and many of the previous  transition costs. However, it did not document significant assumptions.  Specifically, GSA did not document the rationale behind its 76 percent  transition traffic factor or why it used a 30-month time period for the  transition—two key assumptions of its analysis.
GSA also did not provide documentation of certain data sources.  Specifically, program officials could not provide supporting data used to  estimate an agency transition cost valued at $4.7 million. In addition, GSA  could not document the data sources used to estimate costs for contractor  support in planning and implementing the transition. While many costs in  its estimate are based on the charges incurred during the previous  transition, GSA officials stated that it was not appropriate to use previous  costs as a basis for the contractor cost element. These officials explained  that unlike the previous transition, GSA would not provide agencies with  on-site contractor support. Officials made this decision, in part, because  the 2½ years of transition planning that has occurred to date is expected to  result in better preparation by agencies and the ability for them to  facilitate their transitions without direct assistance from GSA or its  contractors. Instead of basing their projection of contractor costs on prior  charges, program officials told us that GSA management decided that  contractor support costs should not exceed $35 million. Program officials  could not provide any data or analysis to support this decision.
section_title: GSA’s Analysis Was Not Validated
paragraphs: Estimates are adequately validated when they have been cross-checked  with an independent cost estimate, and when a level of uncertainty  associated with the estimate is identified. An independent cost estimate  provides the estimator with an unbiased test of the reasonableness of the  estimate and reduces the cost risk associated with the project by  demonstrating that alternate methods generate similar results. In  performing an uncertainty analysis, an entity examines the effects of  varying multiple elements and, as a result, is able to express a level of  confidence in its estimate.
GSA did not validate its analysis against an independent cost estimate or  perform an uncertainty analysis. GSA program officials could not provide  a rationale of why these activities were not performed.
section_title: Quantifiable Weaknesses Are Small, but Effect of Estimate’s Intentionally Conservative Assumption Could Be Significant
paragraphs: The cumulative effect of the quantifiable weaknesses we identified in  GSA’s analysis is relatively small, resulting in an underestimation of $3  million, or roughly 2 percent of the total $151.5 million estimate (as shown  in table 2). The underestimation of costs, related to inflation during the  extended delay in making award, was offset by the overestimation of  service growth and the possible double-counting of a specific agency  transition cost.
In contrast, if the actual level of services transitioning to a different vendor  is lower than the 76 percent transition traffic factor assumed by GSA, the  effect will be greater. Because this factor is the primary cost driver in the  estimate, significant changes in transitioning traffic result in similarly  significant changes in total costs. This effect can be illustrated using GSA’s  transition cost estimate model (see fig. 1). For example, if a transition  traffic factor of 66 percent is used (GSA’s assumption of 16 percent for a  voluntary transition plus a forced shift of 50 percent that would occur if  Verizon Business is not awarded a Networx contract), the total estimated  cost of the transition falls to about $136 million. Similarly, if all  incumbents are awarded Networx contracts and only GSA’s assumption of  a 16 percent voluntary transition occurs, the total estimated cost of the  transition is reduced to approximately $40 million. GSA officials explained  that they used a risk-averse transition traffic factor to minimize the  possibility of underestimating costs.
section_title: GSA Does Not Have a Policy to Ensure That Cost Estimates Are Sound
paragraphs: The weaknesses in GSA’s analysis can be attributed in part to the lack of a  policy requiring cost estimates to be developed using best practices.  Officials from the Offices of the Chief Acquisition Officer, the Chief  Information Officer, and the program office’s Controller confirmed that  GSA does not have centralized policy or guidance on cost estimation.  Instead, the officials that prepared the estimate stated that they based  their work loosely on best practices learned as a result of past experiences  and were comfortable with the estimate. Officials believe there is no  reason to revise their estimate to address the issues we raised in this  report because their purpose in producing it was to minimize the risk of  underestimating costs. While GSA’s approach minimized the risk of having  inadequate funds to fulfill its commitments, it increased the risk that GSA  will retain excess funds that could be used for other purposes.
section_title: GSA Has Accumulated Adequate Funding for the Transition
paragraphs: GSA has accumulated adequate funding to support its anticipated  commitments related to the Networx transition. As of fiscal year-end 2006,  GSA accumulated approximately $142 million in a reserve dedicated to  costs associated with the transition from FTS2001 to Networx. The  uncertainty analysis we performed on GSA’s cost estimate indicates that  the funding GSA has already accumulated will most likely be adequate to  pay for expected transition costs. By varying the transition traffic factor,  our analysis indicates that the $142 million already retained should be  adequate to cover expected expenses 96 percent of the time. Figure 2  illustrates the probability that a particular level of funding will be adequate  to account for the total actual costs incurred. For example, the $151.5  million estimate represents a confidence level of 100 percent: there is  almost no chance that costs will exceed the estimate. The full detail of our  methodology is detailed in appendix II.
The merger of the IT and General Supply Funds provides GSA with  additional flexibility, further reducing the need to retain the entire amount  of the estimate. As discussed, legislation combined the two funds into an  Acquisition Services Fund, making their capital assets and balances  available for the purposes of the new Federal Acquisition Service. The  legislation also allows the Federal Acquisition Service to establish a  reserve to retain surplus revenues from GSA’s technology and service  programs specifically for the purpose of offsetting losses and other  unexpected contingencies. Further, at the end of each year, the statute  requires GSA to return to the Treasury any funds not expended or held in a  working capital reserve. During fiscal year 2006, GSA’s technology  programs experienced losses, but its supply programs reported overall  positive earnings of approximately $126 million. If operations continue in  this fashion, excess revenue will be available in the combined fund to  offset losses or account for contingencies, such as the Networx transition,  be retained within the Acquisition Services Fund, or be returned to the  Treasury.
With Networx contracts scheduled to be awarded starting in March 2007,  GSA will soon be in a position to reassess its main assumption, the  transition traffic factor, and the resulting level of funding needed to meet  anticipated commitments. Unless GSA revises its estimate, it risks  unnecessarily retaining funds that could be reallocated to other agency  priorities or returned to the Treasury.
section_title: Conclusions
paragraphs: While GSA achieved its goal of minimizing the risk that it would have  inadequate funds to pay its transition commitments, the analysis used to  develop the transition cost estimate was not sound because it was not  sufficiently accurate, comprehensive, documented, or validated. The  weaknesses can be attributed in part to a lack of a policy at GSA to ensure  that such estimates are developed using best practices and make the best  use of agency resources. While the quantifiable effect of the weaknesses in  accuracy and comprehensiveness is small, the effect of potential  inaccuracies resulting from the intentional use of an overly conservative  assumption about a main cost driver could be more significant. Without  the use of a cost estimation policy that reflects best practices, GSA could  continue to produce similarly unsound estimates, increasing the risk that it  will unnecessarily retain funds that could be reallocated for other  purposes.
Despite the weaknesses in its analysis, GSA has accumulated adequate  funding to support its anticipated commitments related to the Networx  transition. Our analysis indicates that it is highly unlikely that GSA will  need more than the $142 million it has already accumulated. In addition,  the merger of two revolving funds gives it increased flexibility in meeting  costs. Once Networx contracts are awarded beginning in March 2007, GSA  will be able to forecast the number of forced transitions more accurately,  and, if necessary, reduce the amount of funding it plans to accumulate in  the future or free already accumulated funds for other purposes. This  reassessment will also be an opportunity for GSA to address the other  weaknesses in its analysis that resulted from its deviation from best  practices.
section_title: Recommendations
paragraphs: To improve GSA’s program management, we are making two  recommendations. First, to ensure that future cost estimates are sound  and can be used as a reliable basis for decisions, we recommend that the  GSA Administrator establish a policy for cost estimation efforts at GSA.  Specifically, this policy should reflect best practices by requiring that  estimates are:    accurate (not overly conservative, based on an assessment of the most  likely costs, and adjusted properly for inflation);   comprehensive (their level of detail ensures that all pertinent costs are  included and no costs are double-counted);    well-documented (can be easily repeated or updated and can be traced to  original sources through auditing); and   validated (they have been cross-checked with an independent cost  estimate and a level of uncertainty associated with the estimate has been  identified.)
Second, to ensure the most efficient use of federal funds, we recommend  that the Administrator revise the transition cost estimate following the  award of contracts under the Networx program. Specifically, this revision  should reflect best practices, include a more precise transition traffic  factor, and address the overestimation of service growth, the possible  double-counting of a nonrecurring charge, and the effects of inflation  during the extended delay in making awards. If the results of this new  estimate indicate that the full $151.5 million is not needed to reasonably  support the transition effort, GSA should reallocate any excess funds for  other purposes allowable within the Acquisition Services Fund or return  them to the Treasury.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, the GSA Administrator  concurred with our recommendations and emphasized the importance of  supporting a successful governmentwide telecommunications transition.  To address our recommendations, she stated that GSA will issue improved  policy guidance on cost estimating and review and adjust the cost estimate  as additional information becomes available.
The Administrator also commented that GSA’s transition cost estimate  was within 2 percent of our analysis using the same assumptions.  However, while we identified only $3 million in quantifiable errors  (roughly 2 percent of GSA’s total estimate), our report also states that if  the extent of transitioning services is significantly lower than GSA’s  intentionally conservative assumption, the actual costs of the transition  could be considerably less than GSA’s estimate. Specifically, if all  incumbents are awarded Networx contracts and only GSA’s assumption of  a 16 percent voluntary transition occurs, the total cost of the transition  could be reduced to approximately $40 million—over $110 million less  than GSA’s estimate.
In addition, the Administrator stated that GSA does not concur with the  entirety of the draft report. She presented two main objections. First, she  questioned whether our reported findings were balanced given the facts  and results presented. We clarified our report to ensure that our findings  better reflect the information we discussed. Second, she raised a concern  that we incorrectly suggested comparability between the pending Networx  transition and the prior transition. We maintain that the two transitions are  comparable, particularly because GSA’s analysis is based, in part, on the  results of the previous transition. Specifically, GSA’s analysis relied on  experiences and lessons learned from the previous transition to establish  costs for the current estimate.
Appendix III provides the full text of GSA’s comments. GSA also provided  technical comments that have been incorporated in this report, as  appropriate.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the GSA  Administrator and interested congressional committees. We will also make  copies available to others upon request. In addition, the report will be  available at no charge on the GAO Web site at http://www.gao.gov.
Should you or your offices have any questions about matters discussed in  this report, please contact me at (202) 512-6240 or by e-mail at  koontzl@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  GAO staff who made major contributions to this report are listed in  appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine (1) the soundness of the analysis the  General Services Administration (GSA) used to derive the estimate of  funding that would be required for the transition and (2) whether GSA will  have accumulated adequate funding to pay for its transition management  costs.
To determine the soundness of GSA’s analysis, we conducted an intensive  search of over 250 source documents of both government and industry  literature for examples of best practices in the field of cost estimation.  This included literature on cost estimation from the Society of Cost  Estimating and Analysis, the Department of the U.S. Army, the Office of  the Secretary of Defense, the National Aeronautics and Space  Administration, and the Department of Energy. This indicated that high  quality, reliable cost estimates are:    accurate (not overly conservative, based on an assessment of the most  likely costs, and adjusted properly for inflation);   comprehensive (their level of detail ensures that all pertinent costs are  included and no costs are double-counted);   well-documented (can be easily repeated or updated and can be traced to  original sources through auditing); and   validated (they have been cross-checked with an independent cost  estimate and a level of uncertainty associated with the estimate has been  identified).
To determine the extent to which GSA followed these practices, we  analyzed documentation supporting the transition estimate,  documentation provided by GSA on the previous transition estimate, and  the Networx Request for Proposals. We also interviewed GSA Networx  program managers and attended a GSA sponsored transition conference  and meetings of the Interagency Management Council Transition Working  Group.
To address whether GSA has accumulated adequate funding to pay for its  transition management costs, we obtained and analyzed Cost and Capital  Requirements Plans for the Information Technology Fund submitted to the  Office of Management and Budget and legislation for GSA’s Information  Technology, General Supply, and Acquisition Services Funds. In addition,  to determine the extent of funding held in GSA’s related accounts, we  analyzed financial statements for GSA’s technology and service programs.  To verify the reliability of these records, we obtained and analyzed the  results of the most recent GSA financial audits and audit reports from  GSA’s Inspector General, and we interviewed GSA’s independent financial  auditor regarding the quality control procedures in place. The independent  auditor did not specifically review the dollar amounts in the Information  Technology or General Supply Funds for accuracy but did test the controls  in place for compliance with laws and regulations. This auditor stated that  there were no reportable findings associated with either fund, and it was  reasonable to assume that these accounts were fairly stated. As a result,  we determined the data were sufficiently reliable for the purposes of this  report. We also interviewed officials from the Office of Management and  Budget and officials from GSA’s technology and service programs, Office  of the Chief Acquisition Officer, Office of the Chief Financial Officer, and  Office of the Chief Information Officer.
To assess the adequacy of the level of funding already accumulated by  GSA, we performed an uncertainty analysis for GSA’s estimate using a  Monte Carlo simulation. A Monte Carlo simulation provides a perspective  on the potential variability of the cost estimate should the facts,  circumstances, or assumptions change. We chose to vary only the  transition traffic factor because it is the main driver of the costs in GSA’s  estimate. To carry out this simulation, we identified a minimum,  maximum, and median value for the transition traffic factor based on  information received from GSA.
We conducted our work between May 2006 and January 2007 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: Uncertainty Analysis for GSA’s Transition Estimate
paragraphs: An uncertainty analysis provides decision makers with a perspective on  the potential variability of the estimate should the facts, circumstances,  and assumptions change. By examining the effects of varying the estimates  elements, a degree of uncertainty about the estimate can be expressed,  possibly as an estimated range or qualified by some factor of confidence.  For example, an estimate that produces a 100 percent confidence level  indicates that, based on the methodology used to create that estimate,  there is almost no chance that costs will exceed the estimate.
We performed our uncertainty analysis on GSA’s estimate using a Monte  Carlo simulation. We chose to vary only the transition traffic factor  because it is the main driver of the costs in GSA’s estimate. To carry out  this analysis, we identified a minimum, maximum, and median value for  the transition traffic factor, based on information received from GSA.
The transition traffic factor represents the possible percentage of services  under FTS2001 that may transition to a different provider under the  Networx contracts. For this factor, we chose a minimum of 3 percent,  which represents the voluntary shift to a different vendor that occurred  during GSA’s previous transition to FTS2001. For the maximum, we used  76 percent, as this value was chosen by GSA officials as a worst-case  scenario in an effort to mitigate the risks of underestimating costs. The  median of these two numbers is 39.5 percent. Table 3 shows the variations  of each factor used in our uncertainty analysis.
section_title: Appendix III: Comments from the GSA Administrator
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, James R. Sweetman, Jr., Assistant  Director; Jamey A. Collins; Neil J. Doherty; Jennifer K. Echard; Wilfred B.  Holloway; Ethan J. Iczkovitz; Frank Maguire; Karen A. Richey; Glenn D.  Slocum; and Amos A. Tevelow made key contributions to this report.