Systems engineering and test and evaluation are critical parts of the weapon system acquisition process and how well these activities are conducted early in the acquisition cycle can greatly affect program outcomes. Systems engineering translates customer needs into specific product requirements for which requisite technological, software, engineering, and production capabilities can be identified through requirements analysis, design, and testing. Early systems engineering provides the knowledge that weapon system requirements are achievable with available resources such as technologies, time, people, and money. It allows a product developer to identify and resolve performance and resource gaps before product development begins by reducing requirements, deferring them to the future, or increasing the estimated cost for the weapon system’s development. Systems engineering plays a fundamental role in the establishment of the business case for a weapon acquisition program by providing information to DOD officials to make tradeoffs between requirements and resources. Systems engineering is then applied throughout the acquisition process to manage the engineering and technical risk in designing, developing, and producing a weapon system. The systems engineering processes should be applied prior to the start of a new weapon acquisition program and then continuously throughout the life-cycle. Test and evaluation provides information about the capabilities of a weapon system and can assist in managing program risk. There are generally two broad categories of testing: developmental and operational. Developmental testing is used to verify the status of technical progress, substantiate achievement of contract technical performance, and certify readiness for initial operational testing. Early developmental testing reduces program risks by evaluating performance at progressively higher component and subsystem levels, thus allowing program officials to identify problems early in the acquisition process. Developmental testing officials in the Office of the Secretary of Defense and the military services provide guidance and assistance to program managers on how to develop sound test plans. The amount of developmental testing actually conducted however, is controlled by the program manager and the testing requirements explicitly specified in the development contract. In contrast, operational testing determines if a weapon system provides operationally useful capability to the warfighter. It involves field testing a weapon system, under realistic conditions, to determine the effectiveness and suitability of the weapon for use in combat by military users, and the evaluation of the results of such tests. DOD’s Director of Operational Test and Evaluation conducts independent assessments of programs and reports the results to the Secretary of Defense and Congress. In 2008, the Defense Science Board reported that operational testing over the previous 10 years showed that there had been a dramatic increase in the number of weapon systems that did not meet their suitability requirements. The board found that failure rates were caused by several factors, notably the lack of a disciplined systems engineering process early in development and a robust reliability growth program. The board also found that weaknesses in developmental testing, acquisition workforce reductions and retirements, limited government oversight, increased complexity of emerging weapon systems, and increased reliance on commercial standards (in lieu of military specifications and standards) all contributed to these failure rates. For example, over the last 15 years, all service acquisition and test organizations experienced significant personnel cuts, including the loss of a large number of the most experienced technical and management personnel, including subject matter experts, without an adequate replacement pipeline. The services now rely heavily on contractors to help support these activities. Over the past two decades, the prominence of the developmental testing and systems engineering communities within the Office of the Secretary of Defense has continuously evolved, as the following examples illustrate. In 1992, a systems engineering directorate did not exist and the developmental test function was part of the Office of the Director of Test and Evaluation, which reported directly to the Under Secretary of Defense for Acquisition. At that time, the director had direct access to the Under Secretary on an array of issues related to test policy, test assets, and the workforce. In 1994, the Development Test, Systems Engineering and Evaluation office was formed. This organization effectively expanded the responsibilities of the former testing organization to formally include systems engineering. The organization had two deputy directors: the Deputy Director, Development Test and Evaluation, and the Deputy Director, Systems Engineering. This organization was dissolved in 1999. From 1999 to 2006, systems engineering and developmental testing responsibilities were aligned under a variety of offices. The responsibility for managing test ranges and resources, for example, was transferred to the Director of Operational Test and Evaluation. This function was later moved to the Test Resource Management Center, which reports directly to AT&L, where it remains today. In 2004, a Director of Systems Engineering was re-established and then in 2006 this became the System and Software Engineering Directorate. Developmental testing activities were part of this directorate’s responsibilities. As a result, systems engineering and developmental testing issues were reported indirectly to AT&L through the Deputy Under Secretary for Acquisition and Technology. Congress passed the Weapon Systems Acquisition Reform Act of 2009 (Reform Act)—the latest in a series of congressional actions taken to strengthen the defense acquisition system. The Reform Act establishes a Director of Systems Engineering and a Director of Developmental Test and Evaluation within the Office of the Secretary of Defense and defines the responsibilities of both offices. The Reform Act requires the services to develop, implement, and report on their plans for ensuring that systems engineering and developmental testing functions are adequately staffed to meet the Reform Act requirements. In addition, it requires the directors to report to Congress on March 31 of each year on military service and major defense acquisition program systems engineering and developmental testing activities from the previous year. For example, the report is to include a discussion of the extent to which major defense acquisition programs are fulfilling the objectives of their systems engineering and developmental test and evaluation master plans, as well as provide an assessment of the department’s organization and capabilities to perform these activities. Figure 1 shows some of the major reorganizations over the past two decades, including the most recent change where DOD decided to place the two new directors’ offices under the Director of Defense Research and Engineering. DOD has made progress in implementing the systems engineering and developmental test and evaluation provisions of the Reform Act, but has not yet developed performance criteria that would help assess the effectiveness of the changes. Some requirements, such as the establishment of the two new offices, have been fully implemented. The implementation of other requirements, such as the review and approval of systems engineering and developmental test and evaluation plans, has begun but requires sustained efforts. The department has not fully implemented other requirements. For example, DOD has begun development of joint guidance that will identify measurable performance criteria to be included in the systems engineering and developmental testing plans. DOD initially decided that one discretionary provision of the act—naming the Director of Developmental Test and Evaluation also as the Director of the Test Resource Management Center—would not be implemented. However, the Director of Defense Research and Engineering is currently examining the implications of this organizational change. It will be several years before the full impact of the Reform Act provisions is known. The offices of the Director of Systems Engineering and Developmental Test and Evaluation were officially established by the Under Secretary of Defense for AT&L in June 2009 to be his principal advisors on systems engineering and developmental testing matters. The directors took office 3 months and 9 months later, respectively, and are working on obtaining the funding, workforce, and office space needed to accomplish their responsibilities. The directors have also completed evaluations of the military services’ organizations and capabilities for conducting systems engineering and developmental testing, and identified areas for improvement. These evaluations were based on reports provided by the services that were also required by the Reform Act. As shown in table 1, many of the requirements that have been implemented will require ongoing efforts. The directors have the responsibility for reviewing and approving systems engineering and developmental test and evaluation plans as well as the ongoing responsibility to monitor the systems engineering and developmental test and evaluation activities of major defense acquisition programs. During fiscal year 2009, the Director of Systems Engineering reviewed 22 systems engineering plans and approved 16, while the Director of Developmental Test and Evaluation reviewed and approved 25 developmental test and evaluation plans within the test and evaluation master plans. Both offices are monitoring and reviewing activities on a number of major acquisition programs, including the Virginia Class Submarine, the Stryker Family of Vehicles, and the C-130 Avionics Modernization Program. Once their offices are fully staffed, the directors plan to increase efforts in reviewing and approving applicable planning documents and monitoring the activities of about 200 major defense acquisition and information system programs. Evaluations of 42 weapon systems were included in the directors’ first annual joint report to Congress. The individual systems engineering program assessments were consistent in that they typically included information on 10 areas, including requirements, critical technologies, technical risks, reliability, integration, and manufacturing. In some cases, the assessments also included an overall evaluation of whether the program was low, medium, or high risk; the reasons why; and a general discussion of recommendations or efforts the director has made to help program officials reduce any identified risk. Examples include the following. In an operational test readiness assessment of the EA-18G aircraft, the Director of Systems Engineering found multiple moderate-level risks related to software, communications, and mission planning and made recommendations to reduce the risks. The program acted on the risks and recommendations identified in the assessment and delayed the start of initial operational testing by 6 weeks to implement the fixes. It has completed initial operational testing and was found to be effective and suitable by Navy testers. The Director of Operational Test and Evaluation rated the system effective but not suitable, and stated that follow-on testing has been scheduled to verify correction of noted deficiencies. The program received approval to enter full rate production and is rated as a low risk in the joint annual report. The systems engineering assessment of the Global Hawk program was high risk pending the determination of actual system capability; it also stated that there is a high probability that the system will fail operational testing. The assessment cited numerous issues, including questions regarding the system’s ability to meet mission reliability requirements, poor system availability, and the impact of simultaneous weapon system block builds (concurrency).