Over the years, FAA has taken a number of steps to better manage its ATC modernization program. In 1995, based on the premise that FAA would be better able to manage ATC modernization if it were not constrained by federal acquisition laws, FAA requested and Congress enacted legislation that exempted the agency from most federal procurement laws and regulations and directed FAA to develop and implement a new acquisition management system that would address the unique needs of the agency. In 1996, FAA implemented its Acquisition Management System, which provides high-level acquisition policy and guidance for selecting and controlling ATC system acquisitions through all phases of the acquisition life cycle. In February 2004, FAA created the performance-based ATO to control and improve FAA’s investments and operations. ATO incorporated FAA’s former Research and Acquisitions and Air Traffic Services organizations—essentially those that develop and acquire systems and those that operate them—into a single organization. ATO catalogues its acquisition programs in its Capital Investment Plan (CIP). The fiscal year 2006 CIP contained 120 funded acquisition programs and their anticipated total budgets. The 120 acquisition programs include 37 that have had acquisition program baselines approved by FAA’s Joint Resources Council (JRC). These baselined programs include communications, navigation, and surveillance systems that are key to air traffic control operations. Acquisition program baselines show, among other things, executive agreement on an acquisition’s estimated budget and schedule. The JRC also approves rebaselining, through which the agency documents and approves changes to a program’s budget and schedule. Of the 37 baselined programs, 29 also have an exhibit 300, a document prepared for the Office of Management and Budget (OMB) that provides investment justification and management plans for major ATC acquisitions. OMB requires agencies to submit exhibit 300s for major investments, such as those that require special management attention due to the investment’s importance to the agency, have significant program or policy implications, or that the agency defines as major. Figure 1 provides a breakout of the programs in the fiscal year 2006 CIP. The 83 programs that are not baselined include facilities and infrastructure programs; a variety of mission support programs such as training, information technology services, contractual services, and ancillary systems; and systems that are commercially available and ready for ATO to use without modification. FAA’s annual goals and measures for performance reporting, including the two for performance in managing acquisitions, are described in the agency’s annual strategic plan, known as the FAA Flight Plan. The Flight Plan, which FAA began publishing in 2004, sets forth goals, objectives, strategies, initiatives, and specific performance targets for the agency. The Government Performance and Results Act of 1993 requires each agency to prepare and submit to the President and Congress annual reports on program performance for the previous fiscal year. As an administration within the Department of Transportation, FAA is not required to prepare a separate report, but has elected to do so following the statutory framework and guidance for federal agencies. Each year, FAA reports its level of success in meeting its two acquisition performance targets, as well as its other performance targets, in its Performance and Accountability Report. The full suite of FAA’s performance measures for fiscal year 2006 is listed in appendix III. In our past work, we identified nine key attributes of successful performance measures used to evaluate agencies’ performance goals and measures. We determined that eight of these were applicable to our study of ATO’s on-budget and on-schedule acquisition goals and performance measures. See appendix I for more information on our methods. The eight key attributes to which we compared ATO’s acquisition performance measures are as follows: 1. Linkage. Measure is aligned with division- and agencywide goals and mission and clearly communicated throughout the organization. 2. Measurable target. Measure has a numerical goal. 3. Limited overlap. Measure provides new information beyond that provided by other measures. 4. Governmentwide priorities. Each measure covers a priority such as quality, timeliness, and cost of service. 5. Objectivity. Measure is reasonably free from significant bias or manipulation. 6. Reliability. Measure produces the same result under similar conditions. 7. Core program activities. Measure covers the activities that an entity is expected to perform to support the intent of the program. 8. Clarity. Measure is clearly stated and the name and definition are consistent with the methodology used to calculate it. ATO developed its performance targets to be consistent with targets set in the Department of Transportation’s strategic plan, OMB guidance, and the Federal Acquisition Streamlining Act of 1994, which call for other federal agencies to establish cost and schedule goals for acquisitions and to achieve at least 90 percent of those goals. In FAA’s latest Flight Plan, covering fiscal years 2008 through 2012, the performance targets for acquisitions are stated as follows: In fiscal year 2008, 90 percent of major system acquisition investments are within 10 percent of annual budget and maintain through fiscal year 2012. In fiscal year 2008, 90 percent of major system acquisition investments are on schedule and maintain through fiscal year 2012. FAA began using the basic structure of its current annual acquisition performance measures in 2003 when FAA sought to achieve 80 percent of designated milestones and maintain 80 percent of critical program costs within 10 percent of the total budget as published in the CIP. In 2005, ATO split the measure into separate targets for budget and schedule. ATO’s acquisitions targets gradually increased to 90 percent within 10 percent of budget and 90 percent on schedule by fiscal year 2008. Although ATO measures performance of all acquisitions, it reports performance only on major acquisitions in its annual Performance and Accountability Report. At the beginning of each fiscal year, ATO managers identify major acquisitions for performance reporting. According to ATO officials, their selections are based on a number of program characteristics; key among these is an acquisition’s criticality to the NAS. ATO officials told us that judging a program as critical could be based on a number of factors, such as the program having an OMB exhibit 300 or a baseline document. In addition, officials stated that the selected acquisitions were meant to represent a cross section of ATC system acquisitions within ATO. From fiscal years 2003 through 2006, the number of acquisition programs selected for annual performance reporting varied between 29 and 42, or roughly a quarter of ATO’s total acquisitions portfolio each year. ATO measures budget and schedule performance against selected acquisitions’ most recently approved estimates. To measure on-budget performance, ATO compares the total amount budgeted for each selected acquisition (i.e., the estimated budget to complete an acquisition) reported in the January CIP with the corresponding amount reported in the August CIP. ATO considers any program with budget growth of more than 10 percent in this 8-month time frame as not on budget. At the end of the fiscal year, ATO divides the number of selected acquisitions that are considered on-budget by the total number of selected acquisitions and then reports the result as the percentage of major acquisitions within 10 percent of annual budget. To measure schedule performance, at the start of each fiscal year ATO managers select a minimum of two schedule milestones from each major acquisition selected for performance reporting that year. Milestones could be dates to complete activities such as a final installation or establishing a procurement plan. At the end of the fiscal year, ATO divides the number of selected milestones that were met by the total number of selected milestones. ATO reports the result as the percentage of major programs that are on schedule. ATO’s acquisition performance measures meet four of eight key attributes for successful performance measures that we have identified, but the lack of objective criteria for selecting programs for performance measurement reduces objectivity, reliability, and assurance that core programs are included; the clarity of the performance measures could also be improved. Taken together, the lack of successful attributes, combined with the 1-year focus of the performance measures, may not provide a valid measure of ATO’s acquisition performance. We determined that eight key attributes of successful performance measures were applicable to our study of ATO’s two acquisition performance measures. Our analysis determined that ATO met half of these key attributes, as detailed in table 1. Although ATO provides some general guidance for selecting acquisitions for performance reporting, executive judgment was the primary basis for ATO’s selections. ATO described the scope of its performance measure for managing acquisitions in its Portfolio of Goals for fiscal year 2006 as follows: “FAA’s Air Traffic Organization (ATO) Service Units select specific programs that are determined to provide a capital asset to the NAS. For FY06, 31 acquisition programs will be tracked and monitored. Most of the programs selected are considered ‘major’ and must submit an exhibit 300. Those that do not provide exhibit 300s are included because they contribute an asset to the NAS with a useful life of more than two years. The designation of ‘critical acquisition programs’ in the title of this performance target expresses the critical value of the program to the NAS.” As this description illustrates, ATO’s guidance in designating programs as major allows for a significant amount of professional judgment and does not clearly define which programs should be included or excluded for performance reporting. Figure 2 illustrates the types of acquisitions that ATO selected for performance reporting in fiscal year 2006. ATO’s guidance lacks objectivity in that it does not indicate specifically what is to be observed and in which population or conditions. Contrary to ATO’s statement in the Portfolio of Goals that most of the programs selected are considered major and must have an OMB exhibit 300 prepared, only 14 of the 29 programs selected that year actually had an OMB exhibit 300. Additionally, about half of the programs with an OMB exhibit 300 were not selected for performance reporting.