In June 2011, we reported that S&T met some of its oversight requirements for T&E of acquisition programs we reviewed, but additional steps were needed to ensure that all requirements were met.Specifically, since DHS issued the T&E directive in May 2009, S&T reviewed or approved T&E documents and plans for programs undergoing testing, and conducted independent assessments for the programs that completed operational testing during this time period. S&T officials told us that they also provided input and reviewed other T&E documentation, such as components’ documents describing the programs’ performance requirements, as required by the T&E directive. DHS senior level officials considered S&T’s T&E assessments and input in deciding whether programs were ready to proceed to the next acquisition phase. However, S&T did not consistently document its review and approval of components’ test agents—a government entity or independent contractor carrying out independent operational testing for a major acquisition—or document its review of other component acquisition documents, such as those establishing programs’ operational requirements, as required by the T&E directive. For example, 8 of the 11 acquisition programs we reviewed had hired test agents, but documentation of S&T approval of these agents existed for only 3 of these 8 programs. We reported that approving test agents is important to ensure that they are independent of the program and that they meet requirements of the T&E directive. S&T officials agreed that they did not have a mechanism in place requiring a consistent method for documenting their review or approval and the extent to which the review or approval criteria were met. We reported that without mechanisms in place for documenting its review or approval of acquisition documents and T&E requirements, such as approving test agents, it is difficult for DHS or a third party to review and validate S&T’s decision-making process and ensure that it is overseeing components’ T&E efforts in accordance with acquisition and T&E directives and internal control standards for the federal government. As a result, we recommended that S&T develop a mechanism to document both its approval of operational test agents and component acquisitions documentation to ensure that these meet the requirements of the DHS T&E directive. S&T concurred and reported that the agency has since developed internal procedures to ensure that the approval of test agents and component acquisition documents are documented. We also reported in June 2011 that S&T and DHS component officials stated that they face challenges in overseeing T&E across DHS components which fell into 4 categories: (1) ensuring that a program’s operational requirements—the key performance requirements that must be met for a program to achieve its intended goals—can be effectively tested; (2) working with DHS component program staff who have limited T&E expertise and experience; (3) using existing T&E directives and guidance to oversee complex information technology acquisitions; and (4) ensuring that components allow sufficient time for T&E while remaining within program cost and schedule estimates. Both S&T and DHS, more broadly, have begun initiatives to address some of these challenges, such as establishing a T&E council to disseminate best practices to component program managers, and developing specific guidance for testing and evaluating information technology acquisitions. In addition, as part of S&T’s recent reorganization, the agency has developed a new division specifically geared toward assisting components in developing requirements that can be tested, among other things. However, since these efforts have only recently been initiated to address these DHS-wide challenges, it is too soon to determine their effectiveness. Since 2009, S&T has undertaken a series of efforts related to its organizational structure. S&T underwent a new strategic planning process, developed new strategic goals, and conducted a reorganization intended to better achieve its strategic goals. These efforts were implemented after a 2009 National Academy of Public Administration study found that S&T’s organizational structure posed communication challenges across the agency and that the agency lacked a cohesive strategic plan and mechanisms to assess performance in a systematic way, among other things. In August 2010, S&T reorganized to align its structure with its top strategic goals, allow for easier interaction among senior leadership, and reduce the number of personnel directly reporting to the Under Secretary of S&T. Additionally, after the Under Secretary was confirmed in November 2009, S&T instituted a new strategic planning process which helped inform the development of new strategic goals. The new strategic goals announced in August 2010 include: rapidly developing and delivering knowledge, analyses, and innovative solutions that advance the mission of DHS; leveraging its expertise to assist DHS components’ efforts to establish operational requirements, and select and acquire needed technologies; strengthening the Homeland Security Enterprise and First Responders’ capabilities to protect the homeland and respond to disasters; conducting, catalyzing, and surveying scientific discoveries and inventions relevant to existing and emerging homeland security challenges; and fostering a culture of innovation and learning in S&T and across DHS that addresses mission needs with scientific, analytic, and technical rigor. According to S&T, the agency has developed a draft strategic plan that provides its overall approach to meeting these strategic goals, which is currently in the process of being finalized. Moreover, according to testimony by the Undersecretary of S&T in March 2011, to ensure that individual R&D projects are meeting their goals, S&T has committed to an annual review of its portfolio of basic and applied R&D and all proposed “new start” projects. According to S&T, the review process uses metrics determined by S&T, with input from DHS components, that are aligned with DHS priorities. These metrics consider: the impact on the customer’s mission; the ability to transition these products to the field; whether the investment positions S&T for the future; whether the projects are aligned with customer requirements; whether S&T has the appropriate level of customer interaction; and whether S&T is sufficiently innovative in the way it is approaching its challenges. We are currently reviewing DHS and S&T’s processes for prioritizing, coordinating, and measuring the results of its R&D efforts for the Senate Committee on Homeland Security and Governmental Affairs and we will report on this issue next year. Our prior work related to R&D at other federal agencies could provide insight for S&T as it moves forward with new structures and processes operating within potential fiscal constraints. During the 1990s, we issued a series of reports on federal efforts to restructure R&D in the wake of changing priorities and efforts to balance the federal budget. More recently, we have issued reports on R&D issues at the Department of Defense (DOD), Department of Energy (DOE), the Environmental Protection Agency (EPA), and DHS. Although the specific recommendations and issues vary from department to department, there are key findings across this body of work that could potentially help inform S&T’s efforts to meet DHS’s R&D needs, as well as Congressional oversight of these activities. Since our assessment of R&D efforts at DHS is currently under way, we have not determined the extent to which these key findings from our prior work are applicable to DHS’s R&D efforts or the extent to which DHS already has similar efforts under way. However, our prior work could provide valuable insights into how DHS could leverage the private sector to help conduct R&D, restructure R&D efforts in response to fiscal constraints, and develop comprehensive strategies to mitigate the risk of duplication and overlap. For example: We reported on federal agencies that have restructured their research and development efforts in response to fiscal constraints. For example, in January 1998, we reported on efforts by federal agencies, such as DOD, the DOE National Laboratories, and NASA, to streamline their R&D activities and infrastructure. We reported that restructuring research, development, testing and evaluation to meet current and future needs required interagency agreements and cross- agency efforts, in addition to ongoing individual efforts. Additionally, we reported on five elements that were useful in the successful restructuring of R&D in corporate and foreign government organizations. For example, we found that successful restructuring of R&D activities included having a core mission that supports overall goals and strategies, clear definitions of those responsible for supporting that mission, and accurate data on total costs of the organization’s activities. In addition, we have reported that comprehensive strategies mitigate risk of duplication and overlap. For example, we reported in March 2011 that DOD did not have a comprehensive approach to manage and oversee the breadth of its activities for developing new capabilities in response to urgent warfighter needs, including entities engaged in experimentation and rapid prototyping to accelerate the transition of technologies to the warfighter, and lacked visibility over the full range of its efforts.issue guidance that defined roles, responsibilities, and authorities across the department to lead its efforts. DOD agreed with this recommendation. As a result, we recommended that DOD Within DHS itself, we reported in May 2004 that DHS did not have a strategic plan to guide its R&D efforts. We recommended that DHS complete a strategic R&D plan and ensure that the plan was integrated with homeland security R&D conducted by other federal agencies. We also recommended that DHS develop criteria for distributing annual funding and for making long-term investments in laboratory capabilities, as well as develop guidelines that detailed how DOE’s laboratories would compete for funding with private sector and academic entities. DHS agreed with our recommendations. While S&T developed a 5-year R&D plan in 2008 to guide its efforts and is currently finalizing a new strategic plan to align its own R&D investments and goals, DHS has not yet completed a strategic plan to align all R&D efforts across the department, as we previously recommended. Our work on DOE National Laboratories provides additional insights related to oversight of R&D efforts that could be useful for DHS S&T. In 1995, we reported that DOE’s national laboratories did not have clearly defined missions focused on accomplishing DOE’s changing objectives and national priorities.