Defense Acquisitions: Space-Based Infrared System-low at Risk of Missing Initial Deployment Date
Released: 2001-02-28T00:00:00

Letter

Background

Based on its experiences with the launching of short-range theater missiles by Iraq during the 1991 Persian Gulf War, DOD concluded that expanded theater missile warning capabilities were needed and it began planning for an improved infrared satellite sensor capability that would support both long-range strategic and short-range theater ballistic missile warning and defense operations. In 1994, DOD studied consolidating various infrared space requirements, such as for ballistic missile warning and defense, technical intelligence, and battlespace characterization, and it selected SBIRS to replace and enhance the capabilities provided by the Defense Support Program. The Defense Support Program is a strategic surveillance and early warning satellite system with an infrared capability to detect long-range ballistic missile launches that has been operational for about 30 years. DOD has previously attempted to replace the Defense Support Program with the Advanced Warning System in the early 1980s; the Boost Surveillance and Tracking System in the late 1980s; the Follow-on Early Warning System in the early 1990s; and the Alert, Locate, and Report Missiles System in the mid-1990s. These attempts failed due to immature technology, high cost, and affordability issues. SBIRS is to use more sophisticated infrared technologies than the Defense Support Program to enhance the detection of strategic and theater ballistic missile launches and the performance of the missile-tracking function.

The SBIRS development effort consists of two programs—SBIRS-high and SBIRS-low. SBIRS-high is to consist of four satellites operating in geosynchronous earth orbit and sensors on two host satellites operating in a highly elliptical orbit. SBIRS-high will replace Defense Support Program satellites and is primarily to provide enhanced strategic and theater ballistic missile warning capabilities. The SBIRS-high program includes the consolidation of the three existing Defense Support Program ground facilities—two overseas and one in the United States—at a single U.S. ground station to reduce operations and maintenance costs. The program is in the engineering and manufacturing development phase, with a scheduled launch of the first SBIRS-high satellite in fiscal year 2005.

The SBIRS-low program is currently in the program definition and risk reduction acquisition phase and is expected to consist of about 24 satellites in low earth orbit, but it could consist of more or less satellites, depending on the results of contractor cost and performance studies. The primary purpose of SBIRS-low is to support both national and theater missile defense by tracking ballistic missiles and discriminating between the warheads and other objects, such as decoys, that separate from the missile bodies throughout the middle portion of their flights. Its deployment schedule is tied to fiscal year 2010, the date when these capabilities are needed by the National Missile Defense System. According to DOD, the first SBIRS-low satellites need to be launched in fiscal year 2006 if full deployment is to be accomplished by fiscal year 2010.

Due to the importance Congress has placed on the deployment of a National Missile Defense System, Congress has maintained a high level of interest in the SBIRS-low program and has included in legislative provisions dates by which the first satellites are to be launched and initial operational capability is to occur. The National Defense Authorization Act for Fiscal Year 2000 is the latest expression of such interest. It defines, in section 231, the SBIRS-low baseline schedule as a program schedule that includes a first launch of a SBIRS-low satellite to be made during fiscal year 2006. This provision also requires that before the Secretary of the Air Force makes any changes to the SBIRS-low baseline schedule he must obtain the approval of the Director of the Ballistic Missile Defense Organization.

SBIRS-low Program Employs a High-Risk Acquisition Strategy

The Air Force’s current SBIRS-low acquisition schedule is at high risk of not delivering the system on time or at cost or with expected performance. Specifically, satellite development and production are scheduled to occur concurrently and the results of a 1-year flight test that is to test and finalize the design of the satellites will not be available until more than 5 years after the program enters production. The software required for SBIRS-low to perform all its missions is to be developed concurrent with the deployment of the satellites and is not to be completed until more than 3 years after the first SBIRS-low satellites are to be launched.

On-Orbit Test Results Will Not Be Available to Support Production Decision

Under the Air Force’s previous schedules for SBIRS-low, the results of an on-orbit flight demonstration of crucial satellite functions and capabilities were to be available and used to support the decision to enter satellite production; however, the current schedule does not provide such test results in time to support the production decision. In February 1999, the Air Force established the current acquisition schedule (see fig.1) for the SBIRS-low program, which includes a program definition and risk reduction phase, a concurrent development and production phase, and a 1-year on-orbit test with the first six SBIRS-low satellites produced (to be launched with two launches—three satellites per launch).

The decision to enter the engineering and manufacturing development phase and production phases is to be made in the third quarter of fiscal year 2002. The 1-year on-orbit test, which is intended to test and finalize the design of the satellites, will not be completed until January 2008, more than 5 years after development and production is to start. In contrast, under previous schedules (see app. I), the Air Force had stressed the importance of on-orbit tests, stating they were critical to support the decision to enter production. According to the Air Force, its decision to enter the engineering and manufacturing development and production phases will now be based on information obtained from the ground demonstrations performed under the program definition and risk reduction contracts and from other completed on-orbit demonstration programs such as the Midcourse Space Experiment and the Miniature Sensor Technology Integration Program. These program results, however, may be of limited utility to SBIRS-low. For example, according to Air Force officials, they plan to use information on midcourse discrimination collected by the Midcourse Space Experiment in their decision concerning SBIRS-low development and production. However, according to DOD’s Director of Operational Test and Evaluation, the Midcourse Space Experiment did not collect discrimination data on objects representative of those that SBIRS-low must be able to discriminate.

According to the Air Force, launches are not to be resumed until after the 1-year on-orbit test period has been completed, test results have been reviewed, and modifications, if required, have been made to the remaining satellites. However, the production of satellites will not stop during the 1-year on-orbit test. As a result, by the time the test is to be completed in fiscal year 2008, 9 satellites will have been produced (including the first 6 used for flight-testing), an additional 21 satellites will be in various stages of production, and at least $1.9 billion of the $2.4 billion (then-year) cost for these 30 satellites will have been expended or committed. Because the on-orbit test results for crucial functions and capabilities is not to be available until more than 5 years after the start of production, there is a risk that design changes will be required for satellites in production. For example, if parts that have already been purchased for the SBIRS-low operational satellites became obsolete because their acquisition was based on the initial system design, new parts may be required, program costs will increase, and the schedule will slip. Also, additional changes may be necessary to the satellite configuration that could affect not just long lead items, but also modifications may be required to satellite components already produced.

In a July 1999 memorandum to the Under Secretary of Defense for Acquisition and Technology, DOD’s Director of Operational Test and Evaluation expressed concern that the new (current) schedule eliminated critical on-orbit experiments that were to be conducted under the flight demonstration. The Director stated that while the restructured program schedule includes ground demonstrations that were previously lacking from the SBIRS-low program, considering the many technical challenges and high risk in the program, DOD must seek every opportunity to obtain early on-orbit experience. According to the Director, many of the functions and capabilities that must be demonstrated (and would have been demonstrated under the flight demonstration) before SBIRS-low exits the program definition and risk reduction phase and enters the engineering and manufacturing development phase are impossible to demonstrate with only ground tests. For example, the Director stated that DOD has no flight experience where two or more satellites in low earth orbit have communicated with each other. He stated that this was challenging because of the dynamically changing positions of orbiting satellites relative to each other and the high data rates needed to transmit data between satellites thousands of kilometers apart. Another example cited by the Director where DOD has no flight experience is with coordinating the operation of acquisition and tracking infrared sensors, both of which are to be mounted on each SBIRS-low satellite. Specifically, when the acquisition sensor detects the heat from a missile’s booster motor, it must determine and relay highly accurate information on the missile’s position to the tracking sensor. The tracking sensor must then point to the proper location in space, find the missile, and begin tracking the missile. All of these activities must occur within short time frames (seconds) to support missile defense.

We have reported on numerous occasions about the risks associated with program concurrency and of initiating production without adequate testing. In a 1990 testimony, we cited the Navy’s F/A-18 aircraft, the Air Force’s B-1B Bomber, and the Navy’s AEGIS Destroyer as examples where a rush to production without adequate testing resulted in increased costs, lower than expected performance, or both. In 1994 and 1995, we reported that programs are often permitted to begin production with little or no scrutiny and that the consequences have included procurement of substantial inventories of unsatisfactory weapons requiring costly modifications to achieve satisfactory performance, and in some cases, deployment of substandard systems to combat forces. In 2000, we reported that programs were allowed to begin production before the contractors and the government had conducted enough testing to know whether the systems’ design would meet requirements.

Evolutionary Software Development Plan Increases Program Risk

In December 1999, the SBIRS-low program office concluded that development of software to perform all SBIRS-low missions, as originally scheduled, could not be completed 1 year before the scheduled first launch of SBIRS-low satellites in fiscal year 2006. According to the Air Force, this conclusion was based on lessons learned from other programs under which software development efforts were underestimated. As a result, to maintain the fiscal year 2006 first launch, the program office plans to use an evolutionary software development approach under which software is to be developed in increments. The software needed to support all SBIRS-low missions will not be completed (ready for use for satellite operations) until March 2010, over 3 years after the first satellites are launched.

According to Air Force officials, DOD traditionally completes software required to support satellite systems 1 year before the scheduled first launch of a new satellite system. DOD established this practice to reduce risk by ensuring that all system problems have been identified and resolved, and that the personnel operating the systems have been adequately trained. This was the original plan for the SBIRS-low program.

Under the evolutionary approach, software will be developed to support satellite launches, early on-orbit testing, ballistic missile defense, and integration with SBIRS-high, followed by the software needed to support ancillary missions, such as technical intelligence and battlespace characterization. Figure 2 shows the schedule for the incremental development and completion of the software relative to the launch and testing schedule for the SBIRS-low satellites.

As figure 2 shows, by the time the on-orbit test period for the first six SBIRS-low satellites is to begin in fiscal year 2007, the first two increments of software are to be completed. According to program office officials, these two increments of software will provide all of the capabilities the ground control system and the satellites need to support and perform the on-orbit test. The third increment, the ground control and space related software required to operate the full satellite constellation in support of ballistic missile defense, is not to be completed until fiscal year 2008. The fourth software increment, which is to be completed in mid-fiscal year 2009, is to integrate SBIRS-low with SBIRS-high. The fifth increment, which is to be completed in mid-fiscal year 2010, is to add the software required for SBIRS-low to perform ancillary missions such as technical intelligence, battlespace characterization, and space surveillance. Thus, the software required to support all of SBIRS-low missions is not to be completed until over 3 years after the first satellites are launched. While this evolutionary approach reduces schedule pressure for completing the ground control and space software before the first launch in fiscal year 2006, it increases the risk that software may not be available when needed or perform as required. Under the traditional approach, all software would have been completed in fiscal year 2005, 1 year before the launch of the first satellites.

SBIRS-low Program Includes Immature Critical Technologies

The SBIRS-low program has high technical risks because some critical satellite technologies have been judged to be immature for the current stage of the program. Specifically, the SBIRS-low program office rated the maturity of five of six critical technologies at levels that constitute high risk the technologies will not be available when needed.

In developing a complex system, an assessment of the maturity levels of critical technologies can provide information on the risks those maturity levels pose if the technologies are to be included in the development. For example, in a previous report, we discuss a tool, referred to as Technology Readiness Levels, the National Aeronautics and Space Administration and Air Force Research Laboratory use to determine the readiness of technologies to be incorporated into a weapon system. The readiness levels are measured along a scale of one to nine, starting with paper studies of the basic concept and ending with a technology that has proven itself in actual usage on the intended product. The Air Force Research Laboratory considers a readiness level of six to be an acceptable risk for a program entering the Program Definition and Risk Reduction phase—the Laboratory considers lower readiness levels at this stage to translate to high program cost, schedule, and performance risks. Reaching a readiness level of six denotes a significant transition point for technology development in which the technology moves from component testing in a laboratory environment to demonstrating a model or prototype in a relevant environment.

At our request, the SBIRS-low program office rated the maturity, as of the start of the Program Definition and Risk Reduction phase, of six technologies critical to the success of the SBIRS-low program. The program office rated five of the six technologies at levels that, according to criteria used by the Air Force Research Laboratory, constitute high risk in the ability of the program to meet its objectives. A detailed description of the Technology Readiness Levels is provided in appendix II.

As shown in figure 3, SBIRS-low entered the Program Definition and Risk Reduction phase with a number of critical subsystem technologies with maturities below a readiness level of six.

Specifically, the program office rated the maturity of the (1) scanning infrared sensor, which is to acquire ballistic missiles in the early stages of flight, at a readiness level of four; (2) tracking infrared sensor, which is to track missiles, warheads, and other objects such as debris and decoys during the middle and later stages of flight, at a readiness level of four; (3) fore optics cryocooler and (4) tracking infrared sensor cryocooler, which are needed to cool the tracking sensor optics and other sensor components to enable the sensor to detect missile objects in space, at readiness levels of four; (5) satellite communications crosslinks, which enable satellites to communicate with each other, at a readiness level of five; and (6) on-board computer processors, critical for performing complex and autonomous satellite operations for providing missile warning and location information within short time frames, at a level of six. So critical are each of these subsystem technologies is that if one is not available when needed, SBIRS-low would be unable to perform its mission. And, in sum, five of six critical technologies are at a low maturity level, causing high program risk.

Alternative Approaches to SBIRS-low Have Not Been Assessed

Current DOD acquisition policy and procedures require that assessments be made of the cost and mission effectiveness of space systems to alternative terrestrial—land, sea, and air—systems. Despite this requirement, DOD has not adequately analyzed or identified cost-effective alternatives to SBIRS-low that could satisfy critical missile defense requirements such as a Navy ship-based radar capability. Compliance with this requirement would seem especially important, given the high risks identified with the SBIRS-low program.

Terrestrial alternatives to SBIRS-low are not being considered. While competing SBIRS-low contractors are performing cost and trade studies on the various options that could satisfy program requirements, none of these studies is to consider the cost-effectiveness of terrestrial alternatives. The most recent study assessing alternatives to SBIRS-low was performed in 1994; however, according to an Air Force Space Command official, the study’s scope was focused only on options that would use space-based infrared sensors; terrestrial options were not included. According to Air Force Space Command officials, terrestrial alternatives to SBIRS-low do not exist.

Studies on various aspects of the National Missile Defense System by the Ballistic Missile Defense Organization and other organizations have pointed out that alternatives to SBIRS-low may exist. For example, the Ballistic Missile Defense Organization’s June 1999 study, which assessed whether and how the Navy Theater Wide program, a DOD program to develop a ship-based theater missile defense capability, could be upgraded to provide a limited national missile defense capability, cited the potential utility of sea-based radars to a national missile defense capability. Specifically, the report states that properly deployed ship-based radars can provide a forward-based radar warning and tracking function against many of the potential ballistic missile threats to the United States, and that because the radars would be difficult to target due to the mobility and unknown locations of ships, the radars would add robustness against enemy attacks, particularly before SBIRS-low is available.

In a 1999 RAND issue paper that dealt with an assessment of the planning for the National Missile Defense System, the authors suggest that ground-based radars could potentially be used to provide midcourse tracking and cueing for interceptors. Specifically, they conclude that the planned initial capability of the National Missile Defense System is inadequate and suggest that an interim solution be considered to enhance the system’s capabilities against more sophisticated, larger, and more geographically dispersed ballistic missile threats prior to the next planned enhancement to the missile defense system. They suggest that one aspect of the interim solution could include deploying additional ground-based radars to perform ballistic missile tracking and discrimination functions, or alternately, speeding the deployment of SBIRS-low.

Conclusions

The Air Force is implementing a high-risk acquisition schedule for the SBIRS-low program in an attempt to deploy the system starting in fiscal year 2006 to support the National Missile Defense System. The highly concurrent acquisition schedule has evolved because of design, development, and technology challenges, as well as the importance Congress has placed on the deployment of a National Missile Defense capability. Although the schedule includes on-orbit tests to finalize satellite design and performance, the results will not be available in time to be useful for informed decision-making related to satellite design and production.

In addition, the Air Force’s evolutionary software development approach creates risk because it delays completion of the software needed to support all SBIRS-low missions over 3 years after the first launch of SBIRS-low satellites.

Finally, critical satellite technologies that have been judged to be immature for the current phase of the program, place program success in peril.

Due to these deficiencies, the SBIRS-low program is at high risk of not delivering the system on time or at cost or with expected performance. In spite of the high risk that SBIRS-low will not be available to support the National Missile Defense System when needed, DOD has not identified alternatives or interim solutions.

Recommendations for Executive Action

In order to reduce the cost, schedule, performance, and technical risks in the SBIRS-low program, we recommend that the Secretary of Defense direct the Secretary of the Air Force, with the approval of the Director of the Ballistic Missile Defense Organization, to develop a schedule that reduces concurrency and risks, and that sets more realistic and achievable cost, schedule, and performance goals. In addition, the Secretary of Defense should assess the impact of the revised schedule on the National Missile Defense program and provide the results of the assessment to Congress.

We also recommend that the Secretary of Defense direct the Director, Ballistic Missile Defense Organization, to analyze and develop, as appropriate, and in compliance with DOD acquisition policy and procedures, alternative approaches to satisfy critical missile defense midcourse tracking and discrimination requirements in case SBIRS-low cannot be deployed when needed (based on the resulting lower risk SBIRS-low schedule, threat analyses, and missile defense program schedules).

Agency Comments and Our Evaluation

In written comments to a draft of this report, DOD generally agreed with our recommendations. DOD also pointed out that it is taking actions that it believes will address our recommendations. These actions begin to address our concerns, but they are not yet completed or approved, and it is not clear yet whether they will fully address the risks identified by our review. Therefore, our recommendations are still relevant.

Our first recommendation deals with restructuring the SBIRS-low acquisition schedule to reduce cost, schedule, performance, and technical risks; assessing the impact of the restructured schedule on the National Missile Defense program; and providing the results of the assessment to Congress. DOD stated it has developed a proposed update to the SBIRS-low acquisition strategy that it believes addresses our concerns for concurrency in the production phase, while still retaining the fiscal year 2006 first launch date. For example, DOD’s proposed strategy would delay the full operational capability date of the first SBIRS-low constellation by 1 year and allow for additional ground demonstration program activities and on-orbit testing, thus reducing concurrency between production and testing, while maintaining the schedule for implementation of the full constellation. DOD stated that the Ballistic Missile Defense Organization is assessing the impact of this delay on the National Missile Defense program. This proposal has not been approved and will be reviewed for final decision by the Under Secretary of Defense for Acquisition, Technology, and Logistics in May 2001.

Since DOD’s proposed update to the SBIRS-low acquisition strategy has not been approved (due to cost concerns) and will not be considered again for approval until May 2001, we did not assess the proposed strategy in any detail. On the surface, the additional on-orbit testing does somewhat reduce production concurrency. However, even with this additional testing, the program still appears to have high concurrency risk, for example, with substantial long lead time procurement before testing results are complete. Therefore, we believe our recommendation is still appropriate in relation to the new proposal or in light of any changes to DOD’s new proposal.

With regard to our second recommendation, DOD stated that it has initiated a study to address viable alternatives to SBIRS-low capabilities and will provide the results of the study to the Deputy Secretary of Defense on March 1, 2001. While initiation of this study is a good beginning, until it is complete, we cannot assess the extent to which alternatives will be identified and whether critical missile defense requirements allocated to SBIRS-low will be satisfied.

DOD’s comments are reprinted in appendix III. DOD also provided separate technical comments that we have incorporated in this report where appropriate.

Scope and Methodology

To evaluate risks of the current acquisition schedule, we had discussions with officials of the Under Secretary of Defense for Acquisition, Technology, and Logistics; the Under Secretary of Defense for the Comptroller; the Assistant Secretary of Defense for Command, Control, Communications, and Intelligence; the Office of the Director, Operational Test and Evaluation; the Office of Program Analysis and Evaluation; and the Assistant Secretary of the Air Force for Acquisition, all in Washington, D.C. We also held discussions with, and reviewed documents from, officials of the SBIRS program office in Los Angeles, California; the U.S. and Air Force Space Commands, Colorado Springs, Colorado; the Defense Contract Management Agency offices in Van Nuys, California, and Phoenix, Arizona; the Air Force Operational Test and Evaluation Center, Buckley Air National Guard Base, Aurora, Colorado; TRW, Inc., Redondo Beach, California; and Spectrum Astro, Gilbert, Arizona.

To evaluate technical risks of the program, we had discussions with, and reviewed documents from, officials of the program office; the Office of the Director, Operational Test and Evaluation; and the Air Force Research Laboratory, Albuquerque, New Mexico. We also discussed technical risks with TRW and Spectrum Astro.

To determine whether DOD has assessed alternative approaches to SBIRS-low, we had discussions with the Ballistic Missile Defense Organization, in Washington, D.C.; the program office; and U.S. and Air Force Space Commands. We also reviewed two related studies by the Ballistic Missile Defense Organization and the RAND Corporation.

We performed our work from May 1999 through December 2000 in accordance with generally accepted government auditing standards.

If you or your staff have any questions concerning this report, please call me on (404) 679-1900. The GAO contact and staff acknowledgments are listed in appendix IV.

Space-Based Infrared System-low Program History

Original Schedule

The Department of Defense’s (DOD) original 1995 schedule for Space-Based Infrared System-low (SBIRS)-low called for (1) a launch of a two-satellite flight demonstration—both satellites on one launch vehicle—in the first quarter of fiscal year 1999; (2) a deployment decision in fiscal year 2000 after key technologies and operating concepts were validated by the demonstration satellites; and (3) launches of SBIRS-low satellites—3 satellites per launch vehicle—beginning in fiscal year 2006.

According to Air Force officials, the satellite flight demonstration was critical to validate the integration of key technologies and operational concepts that are crucial to national missile defense and other SBIRS missions such as technical intelligence and battlespace characterization. The primary emphasis was to be on the ability to detect and track ballistic missiles and their warheads throughout flight and distinguish between missile warheads and decoys. The Air Force planned to test these satellites’ ability to perform national missile defense functions against live theater and national missile defense targets and to use the demonstration and test results to model and simulate the full performance capability of a constellation of operational SBIRS-low satellites.

According to the program officials who established this acquisition strategy, performing this function autonomously while in orbit is one of the most complex and technologically challenging operational concepts ever attempted. They also stressed that the two-flight demonstration satellites would have provided an informed basis for deciding whether the program was ready to enter the engineering and manufacturing development and production phases of the acquisition process. They stated that a National Missile Defense System with space-based sensors depended on a successful flight demonstration program and that proceeding into the engineering and manufacturing development and production phases before demonstrating this capability would not provide an opportunity to assess lessons learned, thus introducing unacceptable risk into the program.

Figure 4 shows the original acquisition schedule for a fiscal year 2006 first launch of SBIRS-low satellites.

Under this schedule, the first year of the planned 2-year flight demonstration would have been completed in the first quarter of fiscal year 2000, about the same time the program was scheduled to enter the pre-engineering and manufacturing development phase. The first year results from the demonstration could have influenced requirements development and system design during this phase. The second year of the demonstration would have been completed in the first quarter of fiscal year 2001, about the same time the program was scheduled to enter the engineering and manufacturing development and production phases. Thus, DOD would have had almost 2-years of information on the demonstration satellites’ performance to consider in deciding whether the system should enter the engineering and manufacturing development and production phases.

Accelerated Schedule Established in December 1996

DOD did not implement the original schedule because Congress required in the National Defense Authorization Act for Fiscal Year 1996 that DOD establish a program baseline to include a first launch of SBIRS-low satellites in fiscal year 2002. The Defense Science Board, at DOD’s request, assessed the viability of accelerating the first launch from fiscal year 2006 to fiscal year 2002 and found it would not be viable; however, it did determine that the first launch could be accelerated to fiscal year 2004.

Subsequently, DOD informed Congress that the first launch of SBIRS-low satellites could not begin in fiscal year 2002 because technical, funding, and management problems had delayed the scheduled launch of the two demonstration satellites from the first quarter to the third quarter of fiscal year 1999. According to Air Force officials, this delay prevented basing a milestone decision to enter the engineering and manufacturing development and production phases of the SBIRS-low acquisition process, scheduled for the first quarter of fiscal year 2000, on the results of the planned flight demonstration. However, in December 1996, DOD committed to accelerating the first launch of SBIRS-low satellites to fiscal year 2004. Figure 5 shows the acquisition schedule for the flight demonstration and a fiscal year 2004 first launch of SBIRS-low.

Under this acquisition schedule, the demonstration satellites were to be launched in the third quarter of fiscal year 1999, two quarters later than scheduled under the original schedule. Consequently, the flight demonstration and the pre-engineering and manufacturing development phase would have run concurrently and the demonstration results could not have influenced the development of requirements and the system design as they could have under the original schedule. However, the first year of the flight demonstration would still have been completed about 4 months before the start of the engineering and manufacturing development and production phases, which were still scheduled to begin in the first quarter of fiscal year 2001 as they were under the original schedule. As a result, DOD would have had the information from the first year of the demonstration satellites’ performance, which it considered the most critical in deciding whether the system should enter these phases, to support a fiscal year 2004 deployment.

Technology Readiness Levels and Their Definitions

Lowest level of technology readiness. Scientific research begins to be translated into applied research and development. Examples might include paper studies of a technology’s basic properties.

Invention begins. Once basic principles are observed, practical applications can be invented. The application is speculative, and there is no proof or detailed analysis to support the assumption. Examples are still limited to paper studies.

Active research and development is initiated. This includes analytical studies and laboratory studies to physically validate analytical predictions of separate elements of the technology. Examples include components that are not yet integrated or representative.

Basic technological components are integrated to establish that the pieces will work together. This is relatively “low fidelity” compared to the eventual system. Examples include integration of “ad hoc” hardware in a laboratory.

Fidelity of breadboard technology increases significantly. The basic technological components are integrated with reasonably realistic supporting elements so that the technology can be tested in a simulated environment. Examples include “high fidelity” laboratory integration of components.

Representative model or prototype system, which is well beyond the breadboard tested for technology readiness level 5, is tested in a relevant environment. Represents a major step up in a technology’s demonstrated readiness. Examples include testing a prototype in a high fidelity laboratory environment or in simulated operational environment.

Prototype near or at planned operational system. Represents a major step up from technology readiness level 6, requiring the demonstration of an actual system prototype in an operational environment, such as in an aircraft, vehicle, or space. Examples include testing the prototype in a test bed aircraft.

Technology has been proven to work in its final form and under expected conditions. In almost all cases, this technology readiness level represents the end of true system development. Examples include developmental test and evaluation of the system in its intended weapon system to determine if it meets design specifications.

Actual application of the technology in its final form and under mission conditions, such as those encountered in operational test and evaluation. In almost all cases, this is the end of the last “bug fixing” aspects of true system development. Examples include using the system under operational mission conditions.

Comments From the Department of Defense

The following are our comments on DOD’s letter dated December 14, 2000.

GAO Comments

1. We are cognizant of the fact that the National Missile Defense program is driving the need date for SBIRS-low and not the converse. We do not intend to suggest that the SBIRS-low acquisition schedule be a driver for the National Missile Defense program schedule. Our primary goal in making this recommendation is to help ensure SBIRS-low is acquired at lower risk and will satisfy critical missile defense requirements. This is why we are also making our second recommendation—to develop alternative approaches to satisfy critical missile defense midcourse tracking and discrimination requirements in case SBIRS-low (under a new lower risk schedule) cannot be deployed when needed. 2. We disagree that we misstate the risks of the SBIRS-low incremental software development strategy. We recognize that an evolutionary, or incremental, approach to software development is valid. However, an acquisition approach such as the original SBIRS-low approach that calls for the completion of all software prior to the first launch poses less risk than one that does not, that is, the evolutionary or current approach. From the perspective of meeting the schedule for a first launch in fiscal year 2006, the evolutionary software development approach may reduce schedule risk because, according to the Air Force, the first launch date would be unachievable under the original strategy due to an underestimation of the software development effort. However, from the perspective of comparing the evolutionary software development approach with the original approach, there is increased program risk associated with the evolutionary approach because there is less assurance the software will be completed when needed with the mission capabilities specified. 3. While we agree that a revised acquisition strategy would likely increase costs, cost increases associated with program delays or rework could also occur under the current schedule. Due to the highly concurrent acquisition schedule, we believe that there is substantial risk that delays and rework resulting from the production of hardware and software that fail to satisfy requirements may occur—resulting in cost increases if the current schedule is strictly adhered to. We believe that early effort to understand acquisition options and the associated costs is important.

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the person named above, Ted B. Baird, Richard Y. Horiuchi, and Robert W. Stewart of the Denver Field Office and David G. Hubbell and Dale M. Yuge of the Los Angeles Field Office made key contributions to this report.

Ordering Information

The first copy of each GAO report is free. Additional copies of reports are $2 each. A check or money order should be made out to the Superintendent of Documents. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders by phone: (202) 512-6000 fax: (202) 512-6061 TDD (202) 512-2537 Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

To Report Fraud, Waste, or Abuse in Federal Programs

Web site: http://www.gao.gov/fraudnet/fraudnet.htm e-mail: fraudnet@gao.gov 1-800-424-5454 (automated answering system)