text: b' April 29, 2003\n\n\n\n\nAcquisition\n\n\nAcquisition of the Suite\nof Integrated Radio\nFrequency Countermeasures\n(D-2003-083)\n\n\n\n\n Department of Defense\n Office of the Inspector General\nQuality Integrity Accountability\n\x0c Additional Copies\n\n To obtain additional copies of this report, visit the Web site of the Inspector\n General of the Department of Defense at www.dodig.osd.mil/audit/reports or\n contact the Secondary Reports Distribution Unit of the Audit Followup and\n Technical Support Directorate at (703) 604-8937 (DSN 664-8937) or fax (703)\n 604-8932.\n\n Suggestions for Future Audits\n\n To suggest ideas for or to request future audits, contact the Audit Followup and\n Technical Support Directorate at (703) 604-8940 (DSN 664-8940) or fax (703)\n 604-8932. Ideas and requests can also be mailed to:\n\n OAIG-AUD (ATTN: AFTS Audit Suggestions)\n Inspector General of the Department of Defense\n 400 Army Navy Drive (Room 801)\n Arlington, VA 22202-4704\n\n Defense Hotline\n\n To report fraud, waste, or abuse, contact the Defense Hotline by calling (800)\n 424-9098; by sending an electronic message to Hotline@dodig.osd.mil; or by\n writing to the Defense Hotline, The Pentagon, Washington, DC 20301-1900. The\n identity of each writer and caller is fully protected.\n\n\n\n\nAcronyms\n\nC4I Command, Control, Communications, Computers, and Intelligence\nDFARS Defense Federal Acquisition Regulation Supplement\nORD Operational Requirements Document\nSIRFC Suite of Integrated Radio Frequency Countermeasures\nUSSOCOM U.S. Special Operations Command\n\x0c\x0c Office of the Inspector General of the Department of Defense\nReport No. D-2003-083 April 29, 2003\n (Project No. D2002AE-0121)\n\n Acquisition of the Suite of Integrated\n Radio Frequency Countermeasures\n\n Executive Summary\n\nWho Should Read This Report and Why? Those who are specifically involved in the\nmanagement, support, and oversight of the Suite of Integrated Radio Frequency\nCountermeasures (SIRFC) should read this report because it discusses acquisition issues\nthat must be addressed before the SIRFC program progresses further through the\nacquisition process.\n\nBackground. Initiated in 1984, the SIRFC is a state-of-the-art, lightweight, fully\nintegrated radio frequency countermeasures and situational awareness system designed to\nprotect rotary-wing and fixed-wing aircraft against fire control radars and semi-active\nmissiles for both air-to-air and surface-to-air weapons systems. At program initiation, the\nArmy and the U.S. Special Operations Command (USSOCOM) planned to procure\nSIRFC units. Through FY 2003, the Army programmed $138.1 million in research,\ndevelopment, test, and evaluation funds and $32.8 million in procurement funds for the\nSIRFC. However in September 2001, the Army withdrew procurement funding for the\nSIRFC program for FY 2003 and beyond. Because the Army delayed its SIRFC\nprocurement plans, USSOCOM programmed $239.7 million in procurement funding for\nFY 2003 through FY 2007 to procure, integrate, and install 97 SIRFC systems on special\noperations rotary-wing aircraft. The SIRFC is an Army Acquisition Category III\nprogram that is included on the Director, Operational Test and Evaluation oversight list.\n\nResults. Overall, the SIRFC program needs improved management controls in the areas\nof program management, key performance parameters, and test and evaluation before it\nenters the full-rate production phase of the acquisition process.\n\n \xe2\x80\xa2 Roles and responsibilities for day-to-day management of the SIRFC program\n were unresolved because of Army and USSOCOM indecision concerning\n which organization would manage the program. As a result, neither the Army\n nor USSOCOM had updated the operational requirements document; the\n command, control, communications, computers, and intelligence support plan;\n the test and evaluation master plan; and the program protection plan\xe2\x80\x94key\n documents that are needed to effectively manage the program. Determining\n which organization will manage the continued acquisition of the SIRFC\n system and updating program documentation will help decision makers make\n needed programmatic decisions (finding A).\n\n \xe2\x80\xa2 The Army Aviation Center did not include any key performance parameters in\n the operational requirements document for the SIRFC. As a result, program\n decision makers do not have criteria needed to make informed decisions\n concerning continuation of the program at program reviews and user\n\x0c requirements are at greater risk of not being met. Incorporating key\n performance parameters in the SIRFC operational requirements document,\n including a parameter for interoperability, before further decisions are made\n to procure additional SIRFC systems should reduce program risk (finding B).\n\n \xe2\x80\xa2 The Program Executive Officer, Intelligence, Electronic Warfare, and Sensors\n authorized the SIRFC program to enter low-rate initial production even\n though the Army Test and Evaluation Command concluded that the system, as\n designed, was not sufficiently mature to be considered operationally effective,\n suitable, and survivable. As a result, USSOCOM contracted to procure seven\n SIRFC systems at an estimated cost of $19.6 million without assurance that\n they can successfully pass planned operational tests before the full-rate\n production decision review. Delaying further low-rate initial production\n decisions for the SIRFC until adequate test results are available would enable\n decision makers to determine whether the system is affordable and can\n successfully accomplish its intended mission (finding C).\n\nSee the Findings section of this report for the detailed recommendations.\n\nManagement Comments and Audit Response. We received comments from the Army\nAcquisition Executive (the Army) and the USSOCOM Acquisition Executive. Although\nthe Army nonconcurred with findings A, B, and C, it concurred with the\nrecommendations to decide which organization will manage the SIRFC system through\nthe full-rate production phase of the acquisition process and to update the operational\nrequirements document; the command, control, communications, computers and\nintelligence support plan; the test and evaluation master plan; and the program protection\nplan.\n\nThe USSOCOM Acquisition Executive concurred with the recommendations in the\nreport with the exception of the recommendation to delay further low-rate initial\nproduction decisions for the SIRFC until test results show that the system is potentially\noperationally effective, suitable, and survivable as integrated on the MH-47 Chinook\nrotary-wing aircraft. The USSOCOM Acquisition Executive stated that SIRFC system\nintegration does not affect the SIRFC design. He further stated that the program manager\npostponed the low-rate initial production decision for 4 months until recent test results\nwere reviewed to determine the readiness of SIRFC, as an aircraft subsystem, for low-\nrate initial production. However, as concluded in the Army Test and Evaluation\nCommand Report of June 2001, the operational effectiveness and suitability of the\nSIRFC design will not be known until the SIRFC is operationally tested on the MH-47\nChinook. Accordingly, we still believe that the USSOCOM Acquisition Executive, to\nreduce program risks, needs to adhere to the exit criteria established in the Army\nacquisition decision memorandum, May 30, 2002, requiring successful implementation\nof corrective actions, meeting established reliability exit criteria, and having formal test\nevaluation results of the SIRFC as integrated on the MH-47 Chinook before approving\nadditional low-rate initial production decisions. Further, the USSOCOM response did\nnot satisfy the intent of the recommendation to measure the extent that aircraft\nsurvivability is improved with the system and to assess the affordability of the system\nbecause the independent evaluation that the USSOCOM response referenced did not\nmeasure them. Accordingly, we request that the Acquisition Executive, U.S. Special\nOperations Command provide additional comments in response to the final report by\nJune 30, 2003. (See the Findings section of the report for a discussion of the\nmanagement comments and the Management Comments section of the report for the\ncomplete text of the comments.)\n\n\n ii\n\x0cTable of Contents\n\nExecutive Summary i\n\nBackground 1\n\nObjectives 2\n\nFindings\n A. Program Management 3\n B. Key Performance Parameters 8\n C. Readiness for Low-Rate Initial Production 11\n\nAppendixes\n A. Scope and Methodology 19\n Management Control Program Review 20\n Prior Coverage 21\n B. Monitoring Contractor Cost and Schedule Performance on the\n Technology Insertion Contract 23\n C. Army and U.S. Special Operations Command Program Management 25\n D. Definition of Technical Terms 27\n E. Audit Responses to Army on the Report 30\n F. Report Distribution 34\n\nManagement Comments\n Department of the Army 37\n U.S. Special Operations Command 42\n\x0cSource: U.S. Army Technology Applications Program Office, Fort Eustis, VA\n\nSuite of Integrated Radio Frequency Countermeasures (AN/ALQ-211)\n\x0cBackground\n The Suite of Integrated Radio Frequency Countermeasures (SIRFC), an Army\n acquisition category III program, was initiated in 1984 to meet operational\n requirements for a modular radio frequency electronic countermeasures system\n for Army rotary-wing and fixed-wing aircraft and U.S. Army Special Operations\n Command special operations aircraft. The SIRFC is to provide electronic\n countermeasures against fire control radars and semi-active missiles for both air-\n to-air and surface-to-air weapons systems, radio frequency warning, electronic\n countermeasures protection, and enhanced aircrew situational awareness.\n Appendix D provides definitions of technical terms used in this report.\n\n In September 2001, the Army withdrew procurement funding for the SIRFC\n program for FY 2003 and beyond. As a result of the Army\xe2\x80\x99s decision to remove\n procurement funding for the system and the events of September 11, 2001, the\n U.S. Special Operations Command (USSOCOM)1 made advanced aircraft\n survivability equipment its number one requirement to meet worldwide\n commitments and reinforced its plans to integrate the SIRFC on the MH-47\n Chinook and the MH-60 Blackhawk rotary-wing aircraft. The DoD Comptroller,\n in Program Decision Memorandum II, December 15, 2001, provided USSOCOM\n with $239.7 million in funding to procure, integrate, and install 97 SIRFC systems\n on special operations aircraft from FY 2003 through FY 2007. The Army\n obligated $128.2 million in research, development, test, and evaluation funds and\n $32.8 million in procurement funds for SIRFC through FY 2002 and programmed\n another $9.9 million for research, development, test, and evaluation in FY 2003.\n\n In March 2002, the Program Executive Officer, Intelligence, Electronic Warfare,\n and Sensors approved the SIRFC to enter low-rate initial production in support of\n USSOCOM-funded requirements. In the acquisition decision memorandum, the\n Program Executive Officer approved a low-rate initial production quantity of up\n to eight SIRFC systems for aircraft integration, operational testing, and\n subsequent fielding, and to support establishment of an initial production base.\n On March 28, 2002, USSOCOM contracted for seven low-rate initial production\n systems, which are scheduled for delivery beginning in September 2003. The\n Acquisition Program Baseline, March 2002, showed that the Army planned to\n hold the full-rate production decision review in the first quarter of FY 2004. In\n July 2002, the USSOCOM program office revised the date of the planned full-rate\n production decision review to the first quarter of FY 2005.\n\n In June 2002, the Army removed $51.4 million from the $61.3 million in\n research, development, test, and evaluation funds that were included in the Army\n Research, Development, Test, and Evaluation Budget Item Justification (R-2A\n Exhibit), February 2002, for FY 2003 through FY 2007. The Army will use the\n remaining $9.9 million in research, development, test, and evaluation funds for\n1\n USSOCOM, established on April 16, 1987, consists of three Service Component commands: the Army\n Special Operations Command, the Naval Special Warfare Command, and the Air Force Special\n Operations Command. All special operations forces of the Army, Navy, and Air Force are under\n USSOCOM. SIRFC program documentation used the terms USSOCOM and SOCOM [Special\n Operations Command] interchangeably.\n\n\n\n 1\n\x0c FY 2003 to address technology insertion issues. The draft Army Program\n Objective Memorandum for FY 2004 through FY 2009 also showed zero funding\n for the SIRFC program. Because of a congressional mandate,2 the Office of the\n Director, Operational Test and Evaluation included the SIRFC on its test and\n evaluation oversight list.\n\n USSOCOM will also use the SIRFC as the aircraft survivability equipment suite\n controller and radio frequency countermeasures system for installation on the\n CV-22 tilt-rotor aircraft.3 The Principal Deputy for the USSOCOM Acquisition\n Executive stated that the CV-22 Program Office purchased four SIRFC systems\n as contractor furnished equipment under a commercial contract between\n Bell/Boeing and International Telephone and Telegraph. The SIRFC program\n office anticipates that the CV-22 Program Office will purchase additional SIRFC\n systems as government furnished equipment through the Army\xe2\x80\x99s SIRFC\n production contract beginning in FY 2004.\n\n\nObjectives\n The audit objective was to evaluate the overall management of the SIRFC\n program. Because the program was in the engineering and manufacturing\n development acquisition phase, we determined whether management was cost-\n effectively readying the program for the production phase of the acquisition\n process. We also reviewed the adequacy of the management control program as it\n related to our audit objective. See Appendix A for a discussion of the scope and\n methodology, our review of the management control program, and prior coverage.\n Monitoring of the contractor cost and schedule performance is discussed in\n Appendix B.\n\n\n\n\n2\n House of Representatives Report 103-357 (1993) requires the Secretary of Defense to develop a DoD test\n and evaluation process for electronic warfare systems and to report annually on the progress toward\n meeting this process.\n3\n As part of our review of the acquisition of the SIRFC, we requested program documentation for the\n CV-22. As of December 2002, the CV-22 Program Office had not provided any documentation for\n review.\n\n\n\n\n 2\n\x0c A. Program Management\n Roles and responsibilities for day-to-day management of the SIRFC\n program were unresolved because of Army and USSOCOM indecision\n concerning which organization would manage the program. Neither\n organization updated the operational requirements document; the\n command, control, communications, computers, and intelligence support\n plan; the test and evaluation master plan; and the program protection\n plan\xe2\x80\x94key documents that are needed to effectively manage the program.\n As a result, acquisition decision makers do not have the information they\n need to make informed programmatic decisions.\n\n\nAcquisition Responsibilities and Program Documentation\n Policy\n DoD has established written policy governing the assignment of program\n management responsibilities and the preparation and update of the operational\n requirements document; command, control, communications, computers, and\n intelligence support plan; the test and evaluation master plan; and the program\n protection plan as follows.\n\n Program Management Responsibilities. Memorandum of the Secretary of\n Defense on, \xe2\x80\x9cThe Defense Acquisition System,\xe2\x80\x9d October 30, 2002, requires that a\n program manager be designated for each acquisition program. USSOCOM\n Directive 70-1, \xe2\x80\x9cAcquisition Management Procedures,\xe2\x80\x9d November 24, 1999,\n requires that a memorandum of agreement between the Military Departments and\n USSOCOM be established to delineate responsibilities of USSOCOM and the\n managing Service for each Service-managed acquisition program.\n\n Operational Requirements Document. Chairman of the Joint Chiefs of Staff\n Instruction 3170.01B requires that the operational requirements document (ORD)\n be updated when necessary and prior to each acquisition milestone to incorporate\n results of the activities during each acquisition phase.\n Command, Control, Communications, Computers, and Intelligence (C4I)\n Support Plans. DoD Interim Defense Acquisition guidebook, October 30, 2002,\n requires that DoD Components develop C4I support plans early in the acquisition\n process for all acquisition programs when they connect in any way to the\n communications and information infrastructure. DoD Instruction 4630.8,\n \xe2\x80\x9cProcedures for Interoperability and Supportability of Information Technology\n (IT) and National Security Systems (NSS),\xe2\x80\x9d May 2, 2002, also requires that DoD\n Components identify program interoperability requirements in the C4I support\n plan.\n\n Test and Evaluation Master Plan. The Interim Defense Acquisition\n Guidebook, October 30, 2002, states that the program manager should update the\n test and evaluation master plan at milestones and decision reviews, when the\n\n\n\n 3\n\x0c ORD or C4I support plan is significantly modified, or when the program has\n changed significantly. USSOCOM Directive 70-1 requires that each program\n manager document a test plan in the single acquisition master plan for\n development test and operational test activities. Additionally, the Directive states\n that the program manager should update the single acquisition master plan as the\n program matures and when significant events affecting the program occur.\n\n Program Protection Plan. DoD Directive 5200.39, \xe2\x80\x9cSecurity, Intelligence, and\n Counterintelligence Support to Acquisition Program Protection,\xe2\x80\x9d September 10,\n 1997, establishes policy and assigns responsibilities to protect and prevent\n unauthorized or inadvertent disclosure, or loss of critical program information.\n DoD Manual 5200.1-M, \xe2\x80\x9cAcquisition Systems Protection Program,\xe2\x80\x9d March 16,\n 1994, requires program managers to update the program protection plan at each\n milestone review and upon the occurrence of a change in status of the critical\n program information.\n\n\nArmy and USSOCOM Program Management Responsibilities\n Roles and responsibilities for day-to-day management of the SIRFC program\n were unresolved because of Army and USSOCOM indecision concerning which\n organization would manage the SIRFC program after the Army removed funding\n for the program in September 2001.\n\n Program office personnel stated that the decision concerning program\n management responsibilities would be resolved in September 2002. As of\n December 2002, the Army Acquisition Executive and the USSOCOM\n Acquisition Executive still had not decided which organization will manage the\n SIRFC program through the full-rate production phase of the acquisition process.\n For further details concerning the reason for the Army\xe2\x80\x99s withdrawal from the\n SIRFC program and subsequent USSOCOM actions on the program, see\n Appendix C.\n\n\nUpdate of Program Documents\n As a result of the Army\xe2\x80\x99s decision to withdraw from the program in\n September 2001, the ORD, the C4I support plan, the test and evaluation master\n plan, and the program protection plan needed to be updated to show the revised\n requirements and plans to provide the milestone decision authority with the\n information needed to make an informed decision at the SIRFC full-rate\n production review. Because of Army and USSOCOM indecision concerning\n program management responsibilities, the requirements and planning documents\n were not updated as required.\n\n Operational Requirements Document. Chairman of the Joint Chiefs of Staff\n Instruction 3170.01B, requires that ORDs be updated when necessary and prior to\n each acquisition milestone to incorporate results of the activities during each\n acquisition phase. Because USSOCOM is now the sole user for the SIRFC, the\n\n\n 4\n\x0cORD needs to be revised to identify USSOCOM requirements for the SIRFC.\nSpecifically, USSOCOM needs to identify system capabilities, characteristics,\nkey performance parameters, and the planned operational use for the SIRFC\nsystem in the ORD. Operational testers need the updated ORD to develop test\nplans for the dedicated operational test and evaluation that will be performed to\nsupport the full-rate production decision review in the first quarter of FY 2005.\n\nC4I Support Plan. DoD Instruction 4630.8 requires that program managers\ndevelop a C4I support plan for all acquisition programs to document\ninteroperability and supportability requirements. The SIRFC program office\nbelieved that a plan was not necessary because the SIRFC system did not\nexchange or receive data with systems outside the aircraft. Because the SIRFC\nsystem must interoperate with other systems on the aircraft, the program office\nneeds a C4I support plan to identify SIRFC information exchange requirements.\nOperational testers use the C4I support plan to develop and conduct tests to\ndemonstrate the satisfaction of system interoperability requirements before the\nfull-rate production decision. During the audit, the Army Program Office began\npreparing a C4I support plan.\n\nTest and Evaluation Master Plan. USSOCOM Directive 70-1 requires that the\nprogram office develop a test plan for developmental test and operational test\nactivities. The test plan provides a framework to generate detailed test and\nevaluation plans for developmental and operational tests needed to support key\ndecision points.\n\nThe test plan that was approved in September 1997 included plans to test the\nSIRFC on the Longbow Apache to demonstrate the satisfaction of Army\noperational requirements. With the Army\xe2\x80\x99s withdrawal from the SIRFC program,\nthe test plan needs to be updated to plan and conduct operational tests on the\nMH-47 Chinook and the MH-60 Blackhawk rotary-wing aircraft to demonstrate\nthe satisfaction of USSOCOM operational requirements for the SIRFC before the\nfull-rate production decision review in the first quarter of FY 2005. The deputy\nproduct manager stated that the test and evaluation master plan was under\nrevision. She anticipated that the revised test and evaluation master plan would\nbe approved in the Spring of 2003.\nProgram Protection Plan. DoD Directive 5200.39 requires program managers\nto identify critical elements of their programs and develop a program protection\nplan. DoD 5200.1-M requires the program manager to update the program\nprotection plan at each milestone review, when a change occurs in the status of\nthe critical program information, and before each acquisition phase. The Science\nApplications International Corporation prepared the SIRFC program protection\nplan that was dated March 10, 1994. In preparing the program protection plan,\nthe Science Applications International Corporation did not have information on\nthe selection and application of security measures and countermeasures that are\nnecessary to protect critical information for the SIRFC throughout its acquisition\nlife. The information was not available because the program office did not\nprovide the Army Intelligence Agency with the information needed to prepare a\nMulti-Discipline Counter Intelligence Threat Assessment to be included in the\n1994 program protection plan. Subsequently, in August 2001, the deputy\n\n\n\n 5\n\x0c product manager stated that the required information was provided to the Army\n Intelligence Agency; however, as of December 2002, the Army Intelligence\n Agency had not initiated an assessment.\n\n In addition, when the Science Applications International Corporation developed\n the program protection plan, it addressed the advanced threat radar jammer\n component, but not the threat warning system component of the SIRFC. Without\n a complete and updated program protection plan, the program office may\n unknowingly release critical program information and technologies. To help\n prevent unauthorized disclosure or inadvertent transfer of leading-edge\n technologies and sensitive data or systems, the program protection plan should be\n kept up-to-date to safeguard critical program and technology information.\n\n\nConclusion\n The Army and USSOCOM need to decide future program management\n responsibilities for the SIRFC program and to update program documentation to\n show the effects of the Army\xe2\x80\x99s withdrawal on program planning and\n management. Without updated documents, acquisition decision makers do not\n have the information they need to make informed programmatic decisions.\n\n\nArmy Comments on the Finding and Audit Response\n A summary of Army comments on the finding and audit responses is in\n Appendix E.\n\n\nRecommendations and Management Comments\n A.1. We recommend that the Army Acquisition Executive and the U.S.\n Special Operations Command Acquisition Executive decide which\n organization will manage the Suite of Integrated Radio Frequency\n Countermeasures system through the full-rate production phase of the\n acquisition process.\n\n Army Comments. The Program Executive Officer, Intelligence, Electronic\n Warfare, and Sensors, responding for the Army Acquisition Executive, concurred,\n stating that operational control of the SIRFC program was transferred to the U.S.\n Special Operations Command effective January 6, 2003, and that a formal\n transition plan was being jointly staffed for approval with the Army and U.S.\n Special Operations Command Acquisition Executives.\n\n U.S. Special Operations Command Comments. The Acquisition Executive,\n U.S. Special Operations Command concurred, stating that operational control of\n\n\n\n\n 6\n\x0cthe SIRFC program was transferred to USSOCOM effective January 6, 2003. He\nstated that transition of the SIRFC program from the Army product management\noffice to USSOCOM would be completed by September 30, 2003.\n\nA.2. We recommend that the office assigned responsibility for the future\nmanagement of the Suite of Integrated Radio Frequency Countermeasures\nprogram update the operational requirements document; the command,\ncontrol, communications, computers and intelligence support plan; the test\nand evaluation master plan; and the program protection plan to show\nrequired program changes caused by the Army\xe2\x80\x99s withdrawal from the Suite\nof Integrated Radio Frequency Countermeasures program.\n\nArmy Comments. The Program Executive Officer, responding for the Army\nAcquisition Executive, concurred, stating that the Army product management\noffice and the USSOCOM Technology Applications Program Office jointly\nprepared a single acquisition management plan (the Plan) in March 2002, to\nsupport the FY 2002 low-rate initial production decision. The Plan identified\nprogram changes caused by the Army\xe2\x80\x99s withdrawal of procurement funds from\nthe program and addressed cost, schedule, performance, test and evaluation,\nprogram affordability, and risk assessments for special operations aircraft\napplications based on USSOCOM production funds. He further stated that\nUSSOCOM would handle subsequent updates of the documents.\n\nU.S. Special Operations Command Comments. The Acquisition Executive,\nU.S. Special Operations Command concurred, stating that the operational\nrequirements document; the command, control, communications, computers and\nintelligence support plan; the test and evaluation master plan; and the program\nprotection plan were being updated to reflect just USSOCOM requirements and\nwould be signed in the Spring of 2003.\n\n\n\n\n 7\n\x0c B. Key Performance Parameters\n The Army Aviation Center did not include any key performance\n parameters for the SIRFC in the ORD. This condition occurred because\n the Army Aviation Center was not aware of the requirement that key\n performance parameters for acquisition category III programs be placed in\n ORDs. As a result, program decision makers do not have criteria needed\n to make informed decisions concerning continuation of the program at\n program reviews and user requirements are at greater risk of not being\n met.\n\n\nOperational Requirements and Interoperability Policy\n Operational Requirements Policy. Memorandum of the Secretary of Defense\n on, \xe2\x80\x9cThe Defense Acquisition System,\xe2\x80\x9d October 30, 2002, and Chairman of the\n Joint Chiefs of Staff Instruction 3170.01B provide DoD policy for identifying key\n performance parameters in operational requirements documents.\n\n Secretary of Defense Policy Memorandum. The memorandum states\n that before a program enters the system development and demonstration phase of\n the acquisition process, key performance parameters must be identified and\n validated to guide the efforts of that phase. Additionally, the memorandum states\n that key performance parameters should be refined as needed.\n\n Joint Staff Policy. Chairman of the Joint Chiefs of Staff\n Instruction 3170.01B requires that DoD Components include key performance\n parameters that are mission-critical system requirements in the ORD. Key\n performance parameters represent those capabilities and characteristics of the\n system so significant that failure to meet the threshold value of performance can\n be cause for the program to be reevaluated or the program to be reassessed or\n terminated.\n\n Interoperability Policy. Chairman of the Joint Chiefs of Staff\n Instruction 3170.01B; Chairman of the Joint Chiefs of Staff Instruction 6212.01B,\n \xe2\x80\x9cInteroperability and Supportability of National Security Systems and\n Information Technology Systems,\xe2\x80\x9d May 8, 2000; and Memorandum of the Joint\n Requirements Oversight Council on, \xe2\x80\x9cPolicy for Updating Operational\n Requirements Documents to Incorporate Interoperability Key Performance\n Parameter and Cost,\xe2\x80\x9d November 16, 1999, provide DoD policy on identifying a\n key performance parameter for interoperability in ORDs.\n\n Joint Staff Policy. Chairman of the Joint Chiefs of Staff\n Instruction 3170.01B requires that the lead DoD Component include a key\n performance parameter for interoperability in ORDs to allow for cross-system\n operation. The interoperability key performance parameter is derived from the\n information exchange requirements that characterize the information exchanges\n performed by the proposed system. The interoperability key performance\n parameters in an ORD define the level of interoperability for the proposed system.\n\n\n\n 8\n\x0c Chairman of the Joint Chiefs of Staff Instruction 6212.01B states that the ORD\n must be certified before each milestone, regardless of acquisition category, for\n conformance with joint national security systems and interoperability standards.\n\n Joint Requirements Oversight Council Policy Memorandum. The\n memorandum requires that DoD Components include an interoperability key\n performance parameter in the ORD of all systems that entered the engineering\n and manufacturing development phase of the acquisition process before April 1,\n 2000.\n\n\nOperational Requirements Document\n The Army Aviation Center prepared the initial ORD for the SIRFC on March 2,\n 1994, based on the 1984 Aircraft Survivability Equipment Required Operational\n Capability document. Subsequently, the Army Aviation Center revised the ORD\n in May 1995, August 1998, and June 1999. Key performance parameters were\n omitted from those versions of the ORD.\n\n The Army Aviation Center did not include key performance parameters in the\n ORD because it was not aware of the requirement that key performance\n parameters for acquisition category III programs be placed in ORDs. Program\n office personnel stated that the Army planned to add a key performance parameter\n for interoperability in an update to the ORD. As of November 2002, the ORD\n was still being updated. When interoperability is identified as a key performance\n parameter in the ORD, the Director for Command, Control, Communications, and\n Computers (J-6) validates that the interoperability key performance parameter\n was adequately tested and that the test results were certified. The Joint\n Interoperability Test Command certifies system test results during the\n interoperability system test certification. Testing may be performed with other\n developmental or operational testing whenever possible to conserve resources.\n However, interoperability testing and test certification must be addressed as an\n integral part of the requirements generation process before the planned full-rate\n production decision review in the first quarter of FY 2005.\n\n\nEffect of Not Including Key Performance Parameters in the\n Operational Requirements Document\n Without key performance parameters that specify which SIRFC system\n requirements are considered essential for successful mission accomplishment,\n acquisition decision makers do not have the criteria needed to make informed\n decisions concerning continuation of the program at program reviews.\n Accordingly, USSOCOM needs to update the ORD for the SIRFC to specify key\n performance parameters for those system capabilities or characteristics that are\n considered essential to reach the overall desired capabilities of the system for\n special operations rotary-wing aircraft.\n\n\n\n\n 9\n\x0cArmy Comments on the Finding and Audit Response\n A summary of Army comments on the finding and audit responses is in\n Appendix E.\n\n\nRecommendation and Management Comments\n B. We recommend that the Commander, U.S. Special Operations Command\n update the operational requirements document for the Suite of Integrated\n Radio Frequency Countermeasures to include key performance parameters\n for critical user requirements and system interoperability requirements\n before further production decisions.\n U.S. Special Operations Command Comments. The Acquisition Executive,\n U.S. Special Operations Command concurred, stating that the operational\n requirements document would be updated to include key performance parameters\n for critical user requirements and system interoperability requirements before the\n next production decision.\n\n\n\n\n 10\n\x0c C. Readiness for Low-Rate Initial\n Production\n The Program Executive Officer, Intelligence, Electronic Warfare, and\n Sensors authorized the SIRFC program to enter low-rate initial production\n even though the Army Test and Evaluation Command concluded that the\n system, as designed, was not sufficiently mature to be considered\n operationally effective, suitable, and survivable. The Program Executive\n Officer made the decision to enable USSOCOM to meet its SIRFC\n requirement for an initial operational capability in FY 2005 for the MH-47\n Chinook special operations rotary-wing aircraft. As a result, USSOCOM\n has contracted to procure seven SIRFC systems at an estimated cost of\n $19.6 million without assurance that the SIRFC units procured can\n successfully pass planned operational tests before the full-rate production\n decision review.\n\n\nLow-Rate Initial Production Decision Policy\n Memorandum of the Secretary of Defense on, \xe2\x80\x9cThe Defense Acquisition System,\xe2\x80\x9d\n October 30, 2002, provides testing policies pertaining to low-rate and full-rate\n production for all DoD acquisition programs. DoD Manual 4245.7-M,\n \xe2\x80\x9cTransition from Development to Production,\xe2\x80\x9d Change 1, February 13, 1989,\n provides guidance to acquisition managers on minimizing risks associated with\n transitioning from development to production. The transition process involves\n two decision points: approval to enter low-rate initial production and approval to\n enter full-rate production. The low-rate initial production decision is critical\n because it starts the contractor production line.\n\n Secretary of Defense Policy Memorandum. The memorandum states that\n system entrance into the production and deployment phase depends on the\n following criteria: acceptable performance in development, test and evaluation,\n and operational assessment; mature software capability; no significant\n manufacturing risks; a manufacturing process in control; an approved ORD;\n acceptable interoperability; acceptable operational supportability; and\n demonstration that the system is affordable, optimally funded, and properly\n phased for rapid acquisition throughout the life cycle.\n\n DoD Manual 4245.7-M. DoD Manual 4245.7-M provides acquisition managers\n with an overlay of risk-reducing activities in design, testing, and production for\n the sequential acquisition program milestones. The manual states that program\n failure in one of the design, test, and production processes often results in a\n failure to do well in all areas. As a result, programs become high risk and\n equipment is deployed later and at a far greater cost.\n\n\n\n\n 11\n\x0cPlatform Used For Limited User Test\n The test and evaluation master plan for the SIRFC, September 16, 1997, stated\n that the Longbow Apache would be used as the test platform during the\n engineering and manufacturing development phase to demonstrate SIRFC\n capabilities. From September through October 2001, the Operational Test\n Command, a subordinate organization of the Army Test and Evaluation\n Command, conducted the SIRFC limited user test with the Longbow Apache to\n support the low-rate initial production decision.\n\n In September 2001, the Army withdrew program funding for the SIRFC. As a\n result, USSOCOM obtained funding in December 2001 to continue with\n procuring 97 SIRFC systems for special operations aircraft. Because USSOCOM\n special operations aircraft did not include the Longbow Apache, the Army, as\n program manager, decided to equip the MH-47 Chinook rotary-wing aircraft with\n the SIRFC system first.\n\n Technology Applications Program Office officials stated that it was too late to\n change the test platform and believed that if the SIRFC system worked on the\n Longbow Apache, it could be integrated into other platforms. The Army Test and\n Evaluation Command\xe2\x80\x99s system assessment stated that because the difference\n between platforms and their respective SIRFC integrations reduced the relevance\n of Longbow Apache-based test results for other aircraft, they recommended that\n the additional platforms needed to be tested.\n\n\nResults of Operational Tests\n The Program Executive Officer, Intelligence, Electronic Warfare, and Sensors\n authorized SIRFC to enter low-rate initial production even though the Army Test\n and Evaluation Command (the Command) and the Director, Operational Test and\n Evaluation concluded that the system was not sufficiently mature to be considered\n operationally effective, suitable, and survivable.\n\n Army Test and Evaluation Command. The Command, in its system\n assessment for the SIRFC dated June 21, 2002, stated that the SIRFC system was\n not sufficiently mature at that time to be considered operationally effective,\n suitable, and survivable. Further, the Command recommended that the program\n office make 15 system improvements before allowing the SIRFC system to enter\n into low-rate initial production and 11 modifications to the test and evaluation\n strategy before testing the SIRFC system again. The Command assessed SIRFC\n effectiveness, suitability, and survivability based on:\n\n \xe2\x80\xa2 the \xe2\x80\x9cCritical Operational Issues and Criteria,\xe2\x80\x9d August 22, 1997;\n\n \xe2\x80\xa2 the Operational Requirements Document for the SIRFC, Change 1,\n April 18, 1996;\n\n\n\n\n 12\n\x0c \xe2\x80\xa2 the mission needs and minimum operational performance requirement in\n the Operational Requirements Document for the Longbow Apache,\n April 1, 1994; and\n\n \xe2\x80\xa2 the evaluation issues and associated measures of effectiveness and\n performance data obtained during developmental and operational testing.\n\n The test results obtained for SIRFC effectiveness, suitability, and survivability\n during developmental and operational testing follow.\n\n Effectiveness Issues. The Command stated that the SIRFC system\n demonstrated potential effectiveness, which it defined as the ability of SIRFC to\n provide threat warning and countermeasures in response to the radio frequency\n threat. Specifically, the SIRFC system was effective at identifying two of seven\n tested threats; however, the Command stated that further system improvement in\n identifying additional threats depended on the resolution of technical risks.\n Further, the Command stated that neither SIRFC nor any other radar warning\n system4 may be able to provide threat identification to the accuracy standards\n specified in the ORD. Among the deficiencies that the Command noted were\n frequent false warnings of threat radars, overlapping threat symbols, and\n inadequate warnings of threat search radars.\n\n Another effectiveness concern was the jamming countermeasures capability of the\n SIRFC, which must be able to degrade the threat\xe2\x80\x99s ability to engage aircraft and\n reduce the effectiveness of guns or missiles fired at the aircraft. Although the\n SIRFC limited user test indicated improved survivability of the Longbow Apache\n against some radio frequency threat systems, the jammer effectiveness varied with\n threat type and aircraft range. Also, the test results showed that the jammer\n created the potential for interference between Army rotary-wing aircraft when\n multiple SIRFC-equipped platforms were deployed in the same area. The\n necessary separation of SIRFC-equiped platforms imposed significant operational\n restrictions affecting Army attack helicopter doctrine. The ORD and the draft test\n and evaluation master plan, November 28, 2001, specify that SIRFC will be used\n in attack helicopter missions. However, the USSOCOM program office stated\n that the Army attack helicopter doctrine did not apply to USSOCOM rotary-wing\n aircraft operations. Accordingly, USSOCOM needs to reassess and revise, if\n appropriate, the ORD requirement that SIRFC equipped platforms will operate in\n attack helicopter missions.\n\n Output power was also an effectiveness issue. The Command noted that the\n existing output power of the SIRFC jammer did not meet the jamming\n requirements of special operations aircraft because those aircraft have larger radar\n cross sections than the Longbow Apache. To overcome this issue, the\n USSOCOM program office stated that it planned to use either two jammers or a\n jammer with twice the power output of the jammer used on the Longbow Apache\n on special operations aircraft to mitigate the output power deficiency of the\n SIRFC jammer.\n\n4\n Radar warning systems enhance aircrew knowledge of the threat environment and facilitate evasive\n maneuvers to avoid the threat, thereby reducing the occurrence of engagements.\n\n\n\n 13\n\x0c Suitability Issues. The Command evaluated SIRFC suitability in terms of\n software reload capability; integrated logistics support; interoperability;\n compatibility; reliability; availability; and maintainability and concluded that the\n SIRFC system, as integrated on the Longbow Apache, was not suitable. The\n Command stated that the number of issues identified in the areas of reliability,\n built-in test, maintainability, maturity and stability of design, and manpower and\n personnel integration was more significant than the magnitude of any single issue.\n During the limited user test, for example, the demonstrated reliability of the\n system was 5 hours, with a lower confidence limit of 3 hours. The demonstrated\n reliability of 5 hours is significantly lower than the ORD requirement for\n 300 hours mean-time between mission affecting failures. The results of the\n contractor\xe2\x80\x99s reliability growth test5 showed that the SIRFC system could meet the\n ORD reliability threshold of 300 hours. However, the Command stated that the\n data from the limited user test and the reliability growth test could not be\n combined, nor did the data instill high confidence that the SIRFC could meet its\n mission reliability requirements. Further, the Command stated that the number of\n design changes and the poor reliability test results suggested that changes were\n needed in the maintenance concept.\n\n Survivability Issues. Survivability requirements include the ability of the\n SIRFC to withstand and to operate in electromagnetic, nuclear, biological, and\n chemical environments and not to be susceptible to information attacks. The\n Command stated that the SIRFC was not survivable in any of those environments.\n For example, the Command stated that the SIRFC system was not\n electromagnetically compatible with other subsystems of the Longbow Apache\n and did not comply with electromagnetic environmental effects tests of\n subsystems in accordance with Military Standard 461D, \xe2\x80\x9cRequirements for the\n Control of Electromagnetic Interference Emissions and Susceptibility,\xe2\x80\x9d\n January 11, 1993. As a result, the Command stated that the SIRFC was at risk for\n electromagnetic interference and electromagnetic compatibility problems when\n integrated on platforms and deployed in an operational environment. In this\n regard, the USSOCOM Program Office stated that it believed that the\n electromagnetic interference and compatibility problems experienced on the\n Longbow Apache were not USSOCOM issues because those subsystems\n interfered with on the Longbow Apache were not installed on special operations\n aircraft. Until the SIRFC system is tested on USSOCOM rotary-wing aircraft,\n knowledge concerning whether subsystems on USSOCOM rotary-wing aircraft\n will experience similar electromagnetic interference and compatibility problems\n with the SIRFC will not be known.\n\n Director, Operational Test and Evaluation. The Director, Operational Test and\n Evaluation tasked the Institute for Defense Analysis to assess the operational\n effectiveness, suitability, and survivability of the SIRFC system. On August 21,\n 2002, the Institute for Defense Analysis released its draft assessment to the\n Director, Operational Test and Evaluation. Overall, the Institute for Defense\n Analysis concurred with the system assessment that the Army Test and\n Evaluation Command issued in June 2002.\n5\n Reliability growth testing is an iterative process intended to rapidly and steadily improve reliability using\n a systematic engineering process of test-analyze-fix-retest, where equipment is tested under actual,\n simulated, or accelerated environments.\n\n\n\n 14\n\x0cSystem Improvements Identified at Low-Rate Initial\n Production Program Review\n The SIRFC contractor advised the program office of at least 10 known SIRFC\n deficiencies that it wanted to fix before conducting the limited user test. Because\n the SIRFC program manager needed approval to enter low-rate initial production\n in March 2002 to meet the planned initial operational capability date for the\n European Command scheduled for FY 2005, he decided to go forward with the\n limited user test before the contractor corrected the identified SIRFC deficiencies.\n After the limited user test, in March 2002, the program office awarded the SIRFC\n contractor a corrective actions contract to correct 37 deficiencies. Of those\n 37 deficiencies, 35 deficiencies involved SIRFC software deficiencies with the\n electronic countermeasures, the complex radio frequency, and the sensor fusion\n processor. The remaining two deficiencies related to the wideband receiver and\n smearing noise. The contractor and Army program office were of the opinion that\n successful correction of the 37 deficiencies would solve most of the SIRFC\n performance deficiencies noted in the Army Test and Evaluation Command\xe2\x80\x99s\n system assessment.\n\n\nStatement of Urgency for the SIRFC\n In a memorandum dated October 22, 2001, the Program Executive Officer,\n Aviation requested that the SIRFC procurement be accelerated in support of\n Operation Enduring Freedom. The Program Executive Officer stated that it was\n imperative for SIRFC to be procured expeditiously to equip special operations\n aircraft. He stated that failure to immediately purchase SIRFC systems would\n result in degraded aircraft readiness and a high probability of increased flight\n crew casualties.\n\n\nLow-Rate Initial Production Decision\n In February 2002, the Program Executive Officer, Intelligence, Electronic\n Warfare, and Sensors conducted a meeting with Army program office\n representatives, USSOCOM, and the Army Test and Evaluation Command to\n discuss a limited procurement decision in response to the statement of urgency.\n Although the final test results were not favorable, USSOCOM stated its\n willingness to accept the system with its known deficiencies because the\n demonstrated SIRFC capabilities were better than existing radio frequency\n electronic countermeasures on special operations aircraft.\n\n In March 2002, the Program Executive Officer approved the procurement of up to\n eight SIRFC systems under low-rate initial production because he believed that\n the Army had a reasonable corrective actions plan that included risk management\n and risk mitigation. Additionally, he stated that no additional SIRFC systems\n\n\n\n\n 15\n\x0c would be procured under low-rate initial production until it was fully and\n successfully integrated with USSOCOM aircraft and the reliability problems were\n resolved.\n\n\nConclusion\n Test results available to the Army and USSOCOM program offices did not\n provide evidence that SIRFC system capabilities were better than those of\n existing systems on special operations aircraft or that low-rate initial production\n was justified in March 2002. No pre-set threshold that defined existing\n capabilities of special operations aircraft was established in the ORD or the test\n and evaluation master plan. Also, there was no side-by-side testing of the SIRFC\n with those existing systems on the aircraft to compare against. Therefore, the\n opinion that SIRFC was better than the existing capabilities on special operations\n aircraft was not supported by available test results before the low-rate production\n decision in March 2002.\n\n\nArmy Comments on the Finding and Audit Response\n A summary of Army comments on the finding and audit responses is in\n Appendix E.\n\n\nRecommendations, Management Comments, and Audit\n Response\n C.1 We recommend that the Army Acquisition Executive and the U.S.\n Special Operations Command Acquisition Executive not approve further\n low-rate initial production decisions for the Suite of Integrated Radio\n Frequency Countermeasures system until test results show that the system is\n potentially operationally effective, suitable, and survivable as integrated on\n the MH-47 Chinook rotary-wing aircraft.\n\n Army Comments. The Program Executive Officer, Intelligence, Electronic\n Warfare and Sensors neither concurred nor nonconcurred; however, he stated that\n because the U.S. Special Operations Command Acquisition Executive was now\n the milestone decision authority for the SIRFC program, the Army did not have a\n formal position for this recommendation.\n\n U.S. Special Operations Command Comments. The Acquisition Executive,\n U.S. Special Operations Command nonconcurred, stating that the SIRFC system,\n as an aircraft subsystem, was designed to perform across multiple platforms and\n that the integration of the system on the MH-47 Chinook had no impact on the\n system design. However, the Acquisition Executive stated that the program\n manager had postponed the low-rate initial procurement decision for 4 months.\n During that 4-month period, the program manager will review system test results\n\n\n 16\n\x0cto determine the readiness of the system, as an aircraft subsystem, for low-rate\ninitial production.\n\nAudit Response. The comments of the Acquisition Executive, U.S. Special\nOperations Command were unresponsive to the intent of the recommendation. In\nMarch 2002, the Program Executive Officer, Intelligence, Electronic Warfare and\nSensors approved a low-rate initial production decision of up to eight SIRFC\nsystems. The approval was granted with the caveat that additional low-rate initial\nproduction units would be withheld pending a decision review that would\ndemonstrate, among other issues:\n\n \xe2\x80\xa2 evidence of reliability fixes that demonstrate an achievable growth path\n towards meeting the reliability exit criteria;\n\n \xe2\x80\xa2 appropriate rationale to support additional low-rate initial production\n quantities; and\n\n \xe2\x80\xa2 formal test and evaluation results that can support a decision regarding\n additional low-rate initial production quantities.\n\nThe Army Test and Evaluation Command, in its system assessment of June 21,\n2002, stated that the SIRFC system was not sufficiently mature to be considered\noperationally effective, suitable, and survivable and that the differences between\nplatforms and their respective SIRFC integrations reduced the relevance of the\ntest results that were based on the Army Longbow Apache for other aircraft, such\nas the USSOCOM MH-47 Chinook rotary-wing aircraft. The Command\nrecommended that those additional platforms be tested with the SIRFC before\nmaking production decisions. Because the SIRFC has yet to be operationally\ntested on the MH-47 Chinook, the aircraft that the SIRFC will be fielded on, we\nbelieve that to reduce program risks, the USSOCOM Acquisition Executive needs\nto adhere to the exit criteria established in the Army acquisition decision\nmemorandum for the SIRFC, May 5, 2002, before approving additional low-rate\ninitial production quantities. Accordingly, we request the Acquisition Executive,\nU.S. Special Operations Command reconsider his position on this\nrecommendation and provide additional comments on the final report.\nC.2. We recommend that the Commander, U.S. Special Operations\nCommand compare the performance achieved with the Suite of Integrated\nRadio Frequency Countermeasures to the performance of existing aircraft\nradio frequency and jamming systems, measure the extent that aircraft\nsurvivability is improved within the system, and assess the affordability of\nthe system.\n\nU.S. Special Operations Command Comments. The Acquisition Executive,\nU.S. Special Operations Command concurred, stating that on January 8, 2003, the\nIntelligence and Information Warfare Directorate, Army Communications\nElectronics Command provided an independent evaluation of the performance of\nexisting radio frequency systems compared to the Suite of Integrated Radio\nFrequency Countermeasures system.\n\n\n\n\n 17\n\x0cAudit Response. The comments of the Acquisition Executive, U.S. Special\nOperations Command did not satisfy the intent of the recommendation. On\nJanuary 8, 2003, the Intelligence and Information Warfare Directorate (the\nDirectorate), Army Communications Electronics Command presented an\nindependent evaluation of the performance of existing radio frequency systems\ncompared to the SIRFC system. However, the Directorate did not compare\nSIRFC performance as demonstrated through operational test results with the\nactual performance of existing radio frequency systems. Instead, the Directorate\ncompared the performance specifications of existing radio frequency systems to\nthe performance specifications of the SIRFC. Accordingly, the comparison did\nnot measure the extent that aircraft survivability was improved with the SIRFC or\nassess the affordability of the system compared to using existing aircraft radio\nfrequency and jamming systems. Therefore, we request that the Acquisition\nExecutive, U.S. Special Operations Command provide additional comments based\non a comparison of actual SIRFC test data to measure the extent that aircraft\nsurvivability is improved using the SIRFC system and to assess the affordability\nof the SIRFC system.\n\n\n\n\n 18\n\x0cAppendix A. Scope and Methodology\n We reviewed documentation dated from August 1989 to August 2002. We used\n criteria and references cited in the Memorandum of the Secretary of Defense on,\n \xe2\x80\x9cThe Defense Acquisition System;\xe2\x80\x9d Interim Defense Acquisition Guidebook;\n DoD Instruction 4630.8, DoD Instruction 5010.40; DoD Directive 5010.38,\n \xe2\x80\x9cManagement Control (MC) Program;\xe2\x80\x9d DoD Directive 5200.39; DoD\n Manual 4245.7-M; DoD Manual 5200.1-M; Chairman of the Joint Chiefs of Staff\n Instructions 3170.01B and 6212.01B; Memorandum of the Joint Requirements\n Oversight Council on, \xe2\x80\x9cPolicy for Updating Operational Requirements\n Documents to Incorporate Interoperability Key Performance Parameter and\n Cost;\xe2\x80\x9d Military Standard 461D; Defense Federal Acquisition Regulation\n Supplement 252.242-7005, \xe2\x80\x9cCost/Schedule Status Report;\xe2\x80\x9d and USSOCOM\n Directive 70-1 to perform the audit.\n\n To accomplish the audit objective, we took the following steps.\n\n \xe2\x80\xa2 We reviewed the \xe2\x80\x9cSIRFC Operational Requirements Document,\xe2\x80\x9d March 2,\n 1994, and its revisions, dated May 1995, August 1998, and June 1999 to\n determine whether the user adequately defined system requirements, key\n performance parameters, and interoperability requirements. We also\n discussed planned ORD revisions and USSOCOM requirements with\n personnel in the Technology Applications Program Office.\n\n \xe2\x80\xa2 We reviewed the \xe2\x80\x9cSingle Acquisition Management Plan,\xe2\x80\x9d March 2002;\n the \xe2\x80\x9cSIRFC Test and Evaluation Master Plan,\xe2\x80\x9d September 16, 1997; and\n the \xe2\x80\x9cProgram Protection Plan,\xe2\x80\x9d March 10, 1994, to determine whether the\n program office developed and effectively implemented program\n documentation, such as the acquisition strategy, acquisition plan, test and\n evaluation master plan, and program protection plan. We discussed the\n contents of those documents with program office personnel. We also\n questioned program office personnel regarding the preparation of a C4I\n Support Plan.\n \xe2\x80\xa2 We reviewed the results of the \xe2\x80\x9cArmy Test and Evaluation Command\n System Assessment for the Suite of Integrated Radio Frequency\n Countermeasures,\xe2\x80\x9d June 21, 2002, and the Institute for Defense Analysis\n \xe2\x80\x9cIndependent Evaluation of the Suite of Integrated Radio Frequency\n Countermeasures,\xe2\x80\x9d DRAFT, August 21, 2002, to determine the results of\n the limited user test and whether the Command considered the SIRFC\n design mature enough to support a low-rate initial production decision.\n We discussed the test results in the Army\xe2\x80\x99s report with personnel from\n offices of the Director, Operational Test and Evaluation; the Army Test\n and Evaluation Command; the Technology Applications Program Office;\n the Communications and Electronics Command; and with contractor\n personnel from International Telephone and Telegraph. We also\n discussed the results from the Institute for Defense Analysis DRAFT\n report with personnel from the Office of the Director, Operational Test\n and Evaluation.\n\n\n\n 19\n\x0c \xe2\x80\xa2 We reviewed the following Army contracts: DAAB07-94-C-M504,\n awarded July 26, 1994, for SIRFC engineering and manufacturing\n development; DAAB07-00-C-B410, awarded July 17, 2000, for SIRFC\n technology insertion; DAAB07-01-D-B010, awarded April 26, 2001, for\n production, corrective actions, and special operations aircraft; and\n USSOCOM contract USZA95-02-C-0009, March 28, 2002, for SIRFC\n integration, interface control drawings, manufacturing improvements, and\n hardware. Additionally we reviewed price negotiation memorandums for\n those contracts. We also reviewed the cost and schedule status reports on\n contract DAAB07-94-C-M504 from January 26, 2002, through June 26,\n 2002, to evaluate the adequacy of contract cost and schedule monitoring.\n Further, we reviewed the memorandums of agreement between the\n Defense Contract Management Agency and the Army program office\n dated June 18, 1997; May 23, 2001; and August 1, 2002.\n\n \xe2\x80\xa2 We discussed with the Defense Contract Management Agency its\n involvement with the SIRFC program office and its monitoring of\n contractor cost and schedule status reports.\n\n \xe2\x80\xa2 We reviewed management controls related to the audit objective and\n management self-evaluations for information technology and purchase\n card assessable units.\n\n We performed the audit from April 2002 through December 2002 in accordance\n with generally accepted government auditing standards.\n\n Use of Computer-Processed Data. We did not use computer-processed data to\n perform this audit.\n\n Use of Technical Assistance. An electrical engineer from the Technical\n Assessment Division, Office of the Assistant Inspector General for Auditing,\n DoD, assisted in reviewing the technical requirements in the ORD, the test and\n evaluation master plan, and the system assessment report.\n\n General Accounting Office High-Risk Area. The General Accounting Office\n has identified several high-risk areas in DoD. This report provides coverage of\n the DoD Weapon System Acquisition high-risk area.\n\n\nManagement Control Program Review\n DoD Directive 5010.38, \xe2\x80\x9cManagement Control (MC) Program,\xe2\x80\x9d August 26, 1996,\n and DoD Instruction 5010.40, \xe2\x80\x9cManagement Control (MC) Program Procedures,\xe2\x80\x9d\n August 28, 1996, require DoD organizations to implement a comprehensive\n system of management controls that provides reasonable assurance that programs\n are operating as intended and to evaluate the adequacy of the controls.\n\n Scope of the Review of the Management Control Program. In accordance\n with DoD policy, acquisition managers are to use program cost, schedule, and\n performance parameters as control objectives to implement the requirements of\n\n\n 20\n\x0c DoD Directive 5010.40. Accordingly, we limited our review to management\n controls directly related to SIRFC program cost, schedule, and performance. We\n reviewed management\xe2\x80\x99s completed self-evaluations for information technology\n and purchase card assessable units. Management did not complete a self-\n assessment for the acquisition management assessable unit.\n\n Adequacy of Management Controls. We identified a material management\n control weakness as defined by DoD Instruction 5010.40. Management controls\n were not adequate to ensure that the program manager and user updated program\n documentation as required. Recommendations A.2. and B., if implemented, will\n ensure adherence to regulatory requirements. We will provide a copy of the\n report to the senior official responsible for management controls in the Offices of\n the Assistant Secretary of the Army (Financial Management and Comptroller) and\n the Comptroller, USSOCOM.\n\n Adequacy of Management Self-Evaluation. Although the SIRFC program\n office performed self-evaluations, it did not conduct a self-evaluation for the\n acquisition management assessable unit. Accordingly, the program office did not\n identify the material management control weaknesses that the audit identified.\n\n\nManagement Comments on Management Control Program\n Review and Audit Response\n Program Executive Office, Intelligence, Electronic Warfare, and Sensors\n Comments. The Program Executive Officer provided comments concerning the\n \xe2\x80\x9cManagement Control Program Review.\xe2\x80\x9d He stated that the audit identified the\n lack of updated program management and user documentation as a management\n control weakness. In this regard, he stated that the Program Executive Office,\n Intelligence, Electronic Warfare, and Sensors had exercised judicious\n management controls for essential program cost, schedule, and performance\n parameters and that, in the Army\xe2\x80\x99s opinion, no material weakness is present. For\n the complete text of the Program Executive Officer\xe2\x80\x99s comments, see the\n Management Comments section of this report.\n\n Audit Response. Without preparing updated program documentation, the\n milestone decision authority did not have essential information, such as key\n performance parameters in the ORD and a C4I Support Plan to document\n interoperability and supportability requirements, to make an informed decision\n concerning the readiness of the SIRFC for low-rate initial production.\n\n\nPrior Coverage\n During the last 5 years, the General Accounting Office (GAO) and the Inspector\n General of the Department of Defense (IG DoD) issued two reports that address\n the SIRFC. Unrestricted General Accounting Office and Inspector General of the\n Department of Defense reports can be accessed at http://www.gao.gov and\n http://www.dodig.osd.mil/audit/reports, respectively.\n\n\n 21\n\x0cGeneral Accounting Office\n GAO Report No. GAO-01-448, \xe2\x80\x9cElectronic Warfare: The Army Can\n Reduce Its Risks in Developing New Radar Countermeasures System,\xe2\x80\x9d\n April 2001\n\nInspector General of the Department of Defense (IG DoD)\n IG DoD Report No. D-2002-0035, \xe2\x80\x9cProtection of Strategic Assets Against\n the Radio Frequency Threat,\xe2\x80\x9d January 4, 2002 CLASSIFIED\n\n\n\n\n 22\n\x0cAppendix B. Monitoring Contractor Cost and\n Schedule Performance on the\n Technology Insertion Contract\n On July 17, 2000, the program office awarded the technology insertion contract\n DAAB07-00-C-B410 to ITT to redesign, develop and test SIRFC engineering,\n manufacturing and development components. The procuring contracting officer\n did not include the Defense Federal Acquisition Regulation Supplement (DFARS)\n 252.242-7005, \xe2\x80\x9cCost/Schedule Status Report,\xe2\x80\x9d clause in the SIRFC technology\n insertion contract. DFARS 252.242-7005 requires the contractor to provide cost\n and schedule status reports. The procuring contracting officer stated that the\n DFARS 252.242-7005 clause was not included in the contract because the SIRFC\n program manager stated that there was limited funding for the technology\n insertion contract and that cost and schedule status reports were not a cost-\n effective tool for the technology insertion contract. Further, the procuring\n contracting officer advised that the program office would assess contract cost and\n schedule performance under the initial development contract and that the\n contractor would provide a quarterly report to the program manager on contract\n progress, status, and management after the basic contract was closed. Despite the\n program manager\xe2\x80\x99s assurance that the contract cost and schedule performance\n would be monitored, the program office stopped analyzing the basic contract cost\n and schedule information in September 2001, even though 2 years remained on\n that contract. As of July 2002, several work breakdown structure identification\n categories on the technology insertion contract had cost overruns totaling\n $306,268. The program office could not explain those cost overruns because the\n financial data had not been analyzed in more than a year. Even so, the program\n office maintained that the contract was not experiencing any cost overruns.\n\n DFARS 252.242-7005 requires contractors to provide, at a minimum, a time-\n phased, budgeted cost of work schedule, the budgeted cost of work performed, the\n actual cost of work performed, budgeted-at-completion data, and estimate-at-\n completion data. By comparing those types of data, cost and schedule variances\n can be calculated and managers can use this information to predict possible\n schedule delays and cost overruns. The contractor did not provide the data for the\n technology insertion contract with the data provided for the basic contract.\n Further, the August 2002 quarterly report did not provide data on work\n breakdown structure to identify cost overruns. Instead, the report identified the\n contract value, the cumulative amount spent to date, and the remaining contract\n balance.\n\n Although the technology insertion contract was awarded in July 2000, the\n contractor did not begin submitting the quarterly report until August 2002. By\n that time, the contractor had spent 98 percent of the contract funding. Without the\n quarterly reports, the program office did not have sufficient cost information to\n monitor the contractor\xe2\x80\x99s performance.\n\n\n\n\n 23\n\x0cNo recommendations are being made concerning the technology insertion\ncontract because the contract is expected to be completed in July 2003. However,\nthe contracting officer should include DFARS 252.242-7005 in future SIRFC\ncontracts, such as the low-rate initial production contract, to assess and manage\nthe contractor\xe2\x80\x99s cost and schedule performance.\n\n\n\n\n 24\n\x0cAppendix C. Army and U.S. Special Operations\n Command Program Management\n Army Program Management. On July 18, 1994, the Program Executive Office,\n Aviation approved the SIRFC program for entry into the engineering and\n manufacturing development phase of the acquisition process. In October 2001,\n the Army reassigned the milestone decision authority responsibility for the\n program to the Program Executive Office, Intelligence, Electronic Warfare, and\n Sensors. The Product Manager, Radio Frequency Countermeasures continued\n with responsibility for day-to-day management of the program pending an\n agreement between USSOCOM and the Army Acquisition Executive concerning\n which organization would assume program management responsibility after the\n Army withdrew from the SIRFC program in September 2001.\n\n According to the Product manager, the Army removed funding for the SIRFC\n program because it had to choose between funding the infrared countermeasures\n program or the SIRFC. Both programs are designed to improve the protection of\n rotary-wing and fixed-wing aircraft. Because Army officials considered the\n infrared threat greater than the radio frequency threat, the Army decided to focus\n its available procurement funding on the infrared countermeasures system rather\n than on the SIRFC system.\n\n Although the Army no longer supports fielding a SIRFC system for each rotary-\n wing and fixed-wing aircraft, Army officials stated that a validated requirement\n remains for the radio frequency countermeasures capability. Accordingly, Army\n plans to revisit the requirement for the SIRFC system in the upcoming program\n objective memorandum for FY 2005 through FY 2010 and develop a revised\n strategy to meet the radio frequency threat.\n\n Army Special Operations Command Actions. In November 2001, the Army\n Special Operations Command stated in a memorandum to the Army Chief of Staff\n that its number one materiel problem was enhancing the combat survivability of\n Army special operations aircraft. Further, the Command stated that it needed the\n SIRFC system to satisfy that need, and that without immediate funding, the\n SIRFC program would not be executed. As a result, the DoD Comptroller, in\n Program Decision Memorandum II, December 15, 2001, provided USSOCOM\n with $239.7 million in funding for FY 2003 through FY 2007 to procure,\n integrate, and install 97 SIRFC systems on special operations aircraft.\n\n On March 25, 2002, the Program Executive Officer, Intelligence, Electronic\n Warfare, and Sensors verbally authorized low-rate initial production of the SIRFC\n for special operations aircraft. On May 30, 2002, the Program Executive Officer\n issued a signed acquisition decision memorandum stating that sufficient\n USSOCOM procurement funds were available to support low-rate initial\n production quantities and future production quantities through the Future Years\n Defense Program.\n\n\n\n\n 25\n\x0c Subsequently, on March 28, 2002, the Technology Applications Program Office,1\n Army Special Operations Command (USSOCOM program office), awarded an\n undefinitized2 low-rate initial production contract to International Telephone and\n Telegraph Industries, Avionics Division to procure and integrate seven SIRFC\n systems for use on the MH-47 Chinook and the MH-60 Blackhawk special\n operations rotary-wing aircraft. The contract had a not-to-exceed value of\n $45 million. As of November 2002, the USSOCOM program office had not\n definitized3 the contract.\n\n\n\n\n1\n The Technology Applications Program Office is the USSOCOM program office responsible for\n management of special operations-peculiar modifications to Army rotary wing aircraft.\n2\n A contract in which the terms, specifications, or price are not agreed on before performance begins.\n3\n To agree on the contract terms, specifications, and price, which converts the undefinitized contract into a\n definitive contract.\n\n\n\n 26\n\x0cAppendix D. Definition of Technical Terms\n Acquisition Category. An acquisition category is an attribute of an acquisition\n program that determines the program level of review, decision authority, and\n applicable procedures. The acquisition categories consist of I, major Defense\n acquisition programs; IA, major automated information systems; II, major\n systems; and III, all other acquisition programs.\n\n Acquisition Phase. An acquisition phase represents all the tasks and activities\n needed to bring a program to the next major milestone. Phases provide a logical\n means of progressively translating broadly stated mission needs into well-defined\n system-specific requirements and ultimately into operationally effective, suitable,\n and survivable systems.\n\n Acquisition Strategy. An acquisition strategy is a business and technical\n management approach designed to achieve program objectives within the\n resource constraints imposed. It is the framework for planning, directing,\n contracting for, and managing a program. It provides a master schedule for\n research, development, test, production, fielding, modification, postproduction\n management, and other activities essential for program success. The acquisition\n strategy is a basis for formulating functional plans and strategies.\n\n Cost and Schedule Status Report. The cost and schedule status report provides\n contract cost and schedule performance information for program management.\n\n Critical Program Information. Critical program information includes\n information, technologies, or systems that, if compromised, would degrade\n combat effectiveness, shorten the expected combat-effective life of the system, or\n significantly alter program direction. This includes classified military\n information or unclassified controlled information about such programs,\n technologies, or systems.\n\n Effectiveness. Effectiveness is the extent to which the goals of the system are\n attained, or the degree to which a system can be elected to achieve a set of specific\n mission requirements.\n\n Engineering and Manufacturing Development. Engineering and\n manufacturing development is the third phase of the acquisition process where the\n program office and its contractors fully develop, engineer, design, fabricate, test,\n and evaluate the systems and the principal items necessary for support.\n\n Full-Rate Production. Full-rate production is contracting for economic\n production quantities following stabilization of the system design and validation\n of the production process.\n\n Information Exchange Requirements. Information exchange requirements\n characterize the information exchanges to be performed by a proposed system and\n identify who exchanges what information with whom, why the information is\n necessary, and how the users will employ that information.\n\n\n\n 27\n\x0cInteroperability. Interoperability is the ability of systems, units, or forces to\nprovide services to, or accept services from, other systems, units, or forces and to\nuse the services so exchanged to enable them to operate effectively together.\nInteroperability is a key performance parameter.\n\nKey Performance Parameters. Key performance parameters are a critical\nsubset of the performance parameters found in the ORD, and are included in the\nperformance portion of the acquisition program baseline. Each key performance\nparameter has a threshold and objective value. Key performance parameters\nrepresent those capabilities or characteristics so significant that failure to meet the\nthreshold value of performance can be cause for the concept or system selected to\nbe reevaluated or the program to be reassessed or terminated.\n\nLow-Rate Initial Production. Low-rate initial production is the minimum\nnumber of systems to provide production representative articles for operational\ntest and evaluation, to establish an initial production base, and to permit an\norderly increase in the production rate sufficient to lead to a full-rate production\nupon successful completion of operational testing.\n\nMilestone. A milestone is the point at which a recommendation is made and\napproval sought regarding starting or continuing an acquisition program.\n\nMilestone Decision Authority. The milestone decision authority is the\nindividual designated in accordance with criteria established by the Under\nSecretary of Defense (Acquisition, Technology and Logistics) to approve entry of\nan acquisition program into the next phase of the acquisition process.\n\nObjective. The objective is the performance value that is desired by the user and\nwhich the program manager is attempting to obtain. The objective value\nrepresents an operationally meaningful, time critical, and cost-effective increment\nabove the performance threshold for each program parameter.\n\nOperational Assessment. An operational assessment is an evaluation of\noperational effectiveness and operational suitability made by an independent\noperational test organization, with user support as required, on other than\nproduction systems. The focus of an operational assessment is on significant\ntrends noted in development efforts, programmatic voids, risk areas, adequacy of\nrequirements, and the ability of the program to support adequate operational\ntesting. An operational assessment may be conducted at any time using\ntechnology demonstrators, prototypes, mock-ups, engineering development\nmodels, or simulations, but will not substitute for the initial operational test and\nevaluation necessary to support full-rate production decisions.\n\nOperational Requirements Document (ORD). The ORD is a formatted\nstatement containing performance and related operational performance parameters\nfor the proposed concept or system.\n\nOperational Suitability. Suitability is the degree to which a system can be\nplaced satisfactorily in field use with consideration being given to availability,\ncompatibility, transportability, interoperability, reliability, wartime usage rates,\n\n\n\n 28\n\x0cmaintainability, safety, human factors, manpower supportability, logistic\nsupportability, natural environmental effects, documentation, and training\nrequirements.\n\nProgram Protection Plan. A program protection plan is a comprehensive plan\nto safeguard critical program and technology information that is associated with a\ndefense acquisition program. The level of detail and complexity of the plan will\nvary based on the criticality of the program, system, information, and the phase of\nthe acquisition process being addressed.\n\nSurvivability. Survivability is the capability of a system and its crew to avoid or\nwithstand a man-made hostile environment without suffering an abortive\nimpairment of its ability to accomplish its designated mission.\n\nSystem Development and Demonstration Phase. The purpose of the system\ndevelopment and demonstration phase of the acquisition process is to develop a\nsystem, reduce program risk, ensure operational supportability, design for\nproducibility, ensure affordability, and demonstrate system integration,\ninteroperability, and utility.\n\nTest and Evaluation Master Plan. A test and evaluation master plan documents\nthe overall structure and objectives of the test and evaluation program. It\nprovides a framework within which to generate detailed test and evaluation plans\nand it documents schedule and resource implications associated with the test and\nevaluation program. It identifies the necessary developmental test and evaluation,\noperational test and evaluation, and live-fire test and evaluation activities. It\nrelates program schedule, test management strategy and structure, and required\nresources to critical operational issues, critical technical parameters, objectives\nand thresholds documented in the ORD, evaluation criteria, and milestone\ndecision points.\n\nThreshold. The threshold is the minimum acceptable value that, in the user\xe2\x80\x99s\njudgment, is necessary to satisfy the need. If threshold values are not achieved,\nprogram performance is seriously degraded, and the program may be too costly or\nmay no longer be timely.\n\n\n\n\n 29\n\x0cAppendix E. Audit Responses to Army Comments\n on the Report\n Our detailed responses to the comments from the Program Executive Officer,\n Intelligence, Electronic Warfare, and Sensors on statements in the draft report\n follow. The complete text of those comments is in the Management Comments\n section of this report.\n\n\nArmy Comments On the Background Section of the Report\n and Audit Response\n Army Comments. The Program Executive Officer provided additional facts and\n clarifications on the background section of the report on funding for the program,\n the full-rate production decision, and the purchase of SIRFC units as Government\n furnished equipment.\n\n Funding. The Program Executive Officer stated that although the Army\n had withdrawn SIRFC procurement funding for FY 2003 and beyond, the\n FY 2003 President\xe2\x80\x99s Budget showed research, development, and test and\n evaluation funds in the Army SIRFC program line from FY 2004 through\n FY 2008.\n\n Audit Response. The FY 2003 President\xe2\x80\x99s Budget did contain research,\n development, and test and evaluation funds in the Army SIRFC program line for\n FY 2004 through FY 2008. As stated in the report, the Army, in June 2002, did\n remove $51.4 million of the $61.3 million in research, development, test, and\n evaluation funds planned for FY 2004 and beyond that were included in the\n February 2002, Army Research, Development, Test, and Evaluation Budget Item\n Justification.\n\n Full-Rate Production Decision. The Program Executive Officer stated\n that at the time of the low-rate initial production decision, in March 2002, the\n SIRFC full-rate production decision was scheduled for the first quarter of\n FY 2004 not FY 2005.\n\n Audit Response. We revised the report in response to the Army\n comment.\n\n SIRFC as Government Furnished Equipment. The Program Executive\n Officer stated that plans to purchase SIRFC for the CV-22 program as\n Government furnished equipment on the Army\xe2\x80\x99s SIRFC production contract,\n beginning in FY 2004, were not part of the March 2002 low-rate initial\n production decision, deliberations, and economic discussions as discussed with\n the milestone decision authority.\n\n\n\n\n 30\n\x0c Response. We did not state in the report that the plans for purchasing\n SIRFC units for the CV-22 program as Government furnished equipment were\n part of the low-rate initial production decision, deliberations, or economic\n considerations discussed with the milestone decision authority in March 2002.\n However, the Program Executive Officer, Intelligence, Electronic Warfare and\n Sensors (the milestone decision authority) did approve the SIRFC Single\n Acquisition Management Plan (SAMP) 00-1, in March 2002, which indicated that\n the CV-22 Program Management Office anticipated purchasing SIRFC units as\n Government furnished equipment through the SIRFC production contract,\n beginning in FY 2004.\n\n\nArmy Comments on Finding A and Audit Response\n Army Comments. The Program Executive Officer stated that during his\n execution of the SIRFC program, there was no indecision or question regarding\n management responsibilities. Action was not taken to realign management\n responsibilities under USSOCOM until after the formal decision to zero all Army\n funding for SIRFC for FY 2004 and beyond, as documented in the Army\xe2\x80\x99s\n FY 2004 President\xe2\x80\x99s Budget Submission to the Office of the Secretary of Defense\n in January 2003. Accordingly, in January 2003, the Army officially transferred\n operational control of the SIRFC program to USSOCOM. He also stated that the\n product manager for SIRFC retained day-to-day management responsibility and\n that management responsibilities and chain of command were clearly defined at\n all times. Further, the Program Executive Officer stated that adequate\n information was presented to him to justify and support a low-rate initial\n production decision for quantities in support of USSOCOM needs, even though\n several documents were not updated before the decision in March 2002.\n\n Response. Because of the Army\xe2\x80\x99s withdrawal from the SIRFC program in\n September 2001, Army and USSOCOM program management officials were\n unclear as to which organization, Army or USSOCOM, would manage the\n continual acquisition of the SIRFC system. During the audit, the Product\n Manager, Radio Frequency Countermeasures and the Assistant Program Manager,\n Technology Applications Program Office advised that a decision concerning\n program management responsibilities would be made by September 2002. As of\n the date of the draft report, December 16, 2002, the Army and USSOCOM\n acquisition executives still had not made a decision concerning program\n management responsibilities. As a consequence, program management officials\n in the Army and USSOCOM had not taken responsibility for updating program\n documentation as required in DoD guidance.\n\n Finding A emphasized that the operational requirements document; the command,\n control, communications, computers, and intelligence support plan; the test and\n evaluation master plan; and the program protection plan\xe2\x80\x94key documents that\n were not updated as required in DoD, Chairman of the Joint Chiefs of Staff, and\n USSOCOM instructions and regulations. As a result of the Army\xe2\x80\x99s decision to\n withdraw from the program in September 2001, those key program documents\n needed to be updated to show the program\xe2\x80\x99s revised requirements and acquisition\n\n\n\n 31\n\x0c plans, which the milestone decision authority needs to make an informed decision\n at the full-rate production review scheduled for the SIRFC.\n\n\nArmy Comments on Finding B and Audit Response\n Army Comments. The Program Executive Officer stated that the ORD,\n approved in March 1994, and subsequently updated in August 1995 and\n September 1999, preceded DoD policy that required the implementation of key\n performance parameters. Further, he stated that the SIRFC program office, in\n coordination with the 160th Special Operations Aviation Regiment, proposed\n critical performance objectives and threshold parameters for the low-rate initial\n production phase of the program that were based on the existing ORD, and that\n were subsequently carried forward in the acquisition program baseline approved\n by the milestone decision authority. The SIRFC acquisition program baseline\n included key measurable performance, schedule, cost, and supportability criteria\n linked to operational requirements that in the event of a breach or a potential\n breach would cause the milestone decision authority to reassess the program for\n continuation, modification, or termination. In addition, the SIRFC acquisition\n decision memorandum, May 2002, identified exit criteria that must be satisfied\n before proceeding to an additional low-rate initial production or full-rate\n production decision.\n\n Response. As discussed in the finding, Chairman of the Joint Chiefs of Staff\n Instruction 3170.01B and its predecessors required that DoD Components include\n key performance parameters that are mission-critical system requirements in the\n ORD. Further, the Memorandum of the Joint Requirements Oversight Council on\n \xe2\x80\x9cPolicy for Updating Operational Requirements Documents to Incorporate\n Interoperability Key Performance Parameter and Cost,\xe2\x80\x9d November 16, 1999,\n requires that DoD Components include an interoperability key performance\n parameter in the ORD of all systems that entered the engineering and\n manufacturing development phase of the acquisition process before April 1, 2000.\n As reported, the latest ORD for the SIRFC program, dated September 1999, did\n not include any key performance parameters. Accordingly, we made the\n recommendation that USSOCOM update the ORD to include key performance\n parameters as required. A good starting point for identifying key performance\n parameters for the SIRFC would be the critical performance objective and\n threshold parameters that the milestone decision authority mentioned and\n approved in the acquisition program baseline in March 2002.\n\n\nArmy Comments on Finding C and Audit Response\n Army Comments. The Program Executive Officer stated that he reached his\n low-rate initial production decision for the SIRFC program through a series of\n multiple reviews with the program manager, USSOCOM, and the Army Test and\n Evaluation Command that focused on reliability; producibility; technical issues,\n status, and fixes; risk mitigation; and overall key programmatic parameters.\n\n\n 32\n\x0cBased on those reviews, he reached a reasoned decision to support proceeding\nwith low-rate initial production articles to establish production readiness, to prove\nout fixes through aircraft integration testing, and to support a formal material\nrelease to field. Additionally, he stated that the decision was further balanced and\nbounded by adding measurable exit criteria in the low-rate initial production\nacquisition decision memorandum.\n\nResponse. We do not doubt that the Program Executive Officer reached his low-\nrate initial production decision through a series of multiple reviews with the\nSIRFC program manager, USSOCOM, and the Army Test and Evaluation\nCommand. However, the Program Executive Officer still made the decision to\nallow the SIRFC to enter low-rate initial production when the test results provided\nat those reviews were not positive; that is:\n\n \xe2\x80\xa2 the Army Test and Evaluation Command and the Director, Operational\n Test and Evaluation concluded that the system was not sufficiently mature\n to be considered operationally effective, suitable, and survivable;\n\n \xe2\x80\xa2 the Army Test and Evaluation Command recommended that the SIRFC\n program office make 15 system improvements before allowing the SIRFC\n system to enter into low-rate initial production; and\n\n \xe2\x80\xa2 the Army Test and Evaluation Command stated that the differences\n between platforms and their respective SIRFC integrations reduced the\n relevance of the Army Longbow Apache\xe2\x80\x99s test results for the aircraft,\n (such as the USSOCOM MH-47 Chinook rotary-wing aircraft) and\n recommended that those additional platforms be tested with the SIRFC,\n\nAs indicated in the report and the System Acquisition Management Plan that was\nprepared in support of the low-rate initial production decision in March 2002, the\nlow-rate initial production decision was needed by March 2002 to meet the\nUSSOCOM initial operating capability date in FY 2005 for the MH-47 Chinook\nrotary-wing aircraft. As of April 2002, no test results were available for the\nSIRFC as integrated on the MH-47 Chinook.\n\nWe commend the Program Executive Officer for adding measurable exit criteria\nin the low-rate initial production acquisition decision memorandum concerning\nfurther low-rate initial production decisions. Because the system was immature\nwhen the first low-rate initial production decision was made, prudent management\npractices would dictate that measurable exit criteria be met before management\napproves further production decisions.\n\n\n\n\n 33\n\x0cAppendix F. Report Distribution\n\nOffice of the Secretary of Defense\nUnder Secretary of Defense for Acquisition, Technology, and Logistics\nUnder Secretary of Defense (Comptroller)/Chief Financial Officer\n Deputy Chief Financial Officer\n Deputy Comptroller (Program/Budget)\nDirector, Operational Test and Evaluation\n\nDepartment of the Army\nAssistant Secretary of the Army (Financial Management and Comptroller)\nAssistant Secretary of the Army (Acquisition, Logistics, and Technology)\n Program Executive Officer, Intelligence, Electronic Warfare, and Sensors\n Product Manager, Suite of Integrated Radio Frequency Countermeasures\nAuditor General, Department of the Army\nCommander, Army Test and Evaluation Command\n Commander, Army Evaluation Center\n\nDepartment of the Navy\nNaval Inspector General\nAuditor General, Department of the Navy\n\nDepartment of the Air Force\nAssistant Secretary of the Air Force (Financial Management and Comptroller)\nAuditor General, Department of the Air Force\n\nUnified Command\nCommander, U.S. Special Operations Command\n Commander, Army Special Operations Command\n Program Executive Officer, Maritime and Rotary\n System Acquisition Manager, Army Special Operations Aviation\n Program Manager, Technology Applications Program Office\n\nNon-Defense Federal Organization\nOffice of Management and Budget\n\n\n\n\n 34\n\x0cCongressional Committees and Subcommittees, Chairman and\n Ranking Minority Member\nSenate Committee on Appropriations\nSenate Subcommittee on Defense, Committee on Appropriations\nSenate Committee on Armed Services\nSenate Committee on Governmental Affairs\nHouse Committee on Appropriations\nHouse Subcommittee on Defense, Committee on Appropriations\nHouse Committee on Armed Services\nHouse Committee on Government Reform\nHouse Subcommittee on Government Efficiency and Financial Management, Committee\n on Government Reform\nHouse Subcommittee on National Security, Emerging Threats, and International\n Relations, Committee on Government Reform\nHouse Subcommittee on Technology, Information Policy, Intergovernmental Relations,\n and the Census, Committee on Government Reform\n\n\n\n\n 35\n\x0c\x0cDepartment of the Army Comments\n\n\n\n\n 37\n\x0cFinal Report\n Reference\n\n\n\n\nRevised\n\n\nRevised\n\n\n\n\n 38\n\x0c39\n\x0c40\n\x0c41\n\x0cU.S. Special Operations Command Comments\n\n\n\n\n 42\n\x0c43\n\x0cFinal Report\n Reference\n\n\n\n\nPage 7\n\n\n\n\nPage 10\n\n\nPage 16\n\n\n\n\nPage 17\n\n\n\n\n 44\n\x0cTeam Members\nThe Acquisition Management Directorate, Office of the Assistant Inspector\nGeneral for Auditing of the Department of Defense prepared this report.\nPersonnel of the Office of the Inspector General of the Department of Defense\nwho contributed to the report are listed below.\n\nMs. Mary L. Ugone\nMr. John E. Meling\nMs. Susan J. Lippolis\nMr. Douglas P. Neville\nMs. Barbara S. Wright\nMr. Bradley M. Heller\nMr. Kevin B. Palmer\nMs. Shelley M. Farber\nMr. Anh H. Tran\nMs. Jacqueline N. Pugh\n\x0c'