Information Sharing Environment: Definition of the Results to Be Achieved in Improving Terrorism-Related Information Sharing Is Needed to Guide Implementation and Assess Progress
Released: 2008-07-23T00:00:00

Letter

Background

Federal Law and Policy Call for the Development of an ISE

Because of the information-sharing weaknesses among federal departments and agencies that became apparent after September 11, the Congress and the administration have called for a number of terrorism- related information-sharing initiatives, including the development of an ISE, as the following instances illustrate: Section 1016 of the Intelligence Reform and Terrorism Prevention Act of 2004 (Intelligence Reform Act), enacted December 17, 2004, as amended by the Implementing Recommendations of the 9/11 Commission Act of 2007 (9/11 Commission Act), enacted August 3, 2007, requires the President to take action to facilitate the sharing of terrorism-related information by establishing an ISE. The Act required the President to, among other things, appoint a Program Manager to plan for, oversee implementation of, and manage the ISE, and established an ISC to assist the President and Program Manager in these duties. In addition, the Act required the President, with the assistance of the Program Manager, to submit to Congress a report containing an implementation plan for the ISE no later than 1 year after the date of enactment (enacted December 17, 2004) and specified 11 elements to be included in the plan. These elements include, among other things, the function, capabilities, resources, and concept for the design of the ISE; project plan; budget estimates; performance metrics and measures; and defined roles for all stakeholders. The Act also required annual performance management reports, beginning not later than 2 years after enactment, on the state of the ISE and of information sharing across the federal government.

On December 16, 2005, the President issued a memorandum to implement measures consistent with establishing and supporting the ISE. The memorandum sets forth five information sharing guidelines: (a) defining common standards for how information is acquired, accessed, shared, and used within the ISE; (b) developing a common framework for sharing information between and among executive departments and agencies; state, local, and tribal governments; law enforcement agencies; and the private sector; (c) standardizing the procedures for sensitive but unclassified information; (d) facilitating the sharing of information between executive departments and agencies and foreign governments; and (e) protecting the information privacy rights and other legal rights of Americans. The memorandum also directs the heads of executive departments and agencies to actively work to promote a culture of information sharing within their respective agencies and that ongoing information-sharing efforts be leveraged in the development of the ISE.

In October 2007, the President issued a National Strategy for Information Sharing. The strategy is focused on improving the sharing of homeland security, terrorism, and law enforcement information related to terrorism within and among all levels of government and the private sector and articulates the administration’s vision on terrorism-related information sharing. The strategy notes guiding principles and efforts taken to improve information sharing across all levels of government, the private sector, and foreign partners to date. It also contains an appendix that elaborates on the roles of federal, state, local, and tribal authorities in information sharing and expands on the role of state and major urban area fusion centers.

Scope and Purpose of the ISE

The ISE is not bounded by a single federal agency or component. While the Program Manager has been placed within the Office of the Director of National Intelligence, from an operational perspective, the ISE is to reach across all levels of government as well as the private sector and foreign partners. As such, the program is a broad-based coordination and collaboration effort among various stakeholders. In essence, the ISE can be viewed as a set of cross-cutting communication links—encompassing policies, processes, technologies—among and between the various entities that gather, analyze, and share terrorism-related information. According to officials at the Office of the Program Manager, their focus is primarily to ensure that all appropriate terrorism-related information is made available to analysts and others who need it when they need it. The Program Manager is not responsible for the collection or analysis of terrorism- related information.

The ISE implementation plan, released by the Program Manager in November 2006, is to be the guiding document describing how the ISE is to be implemented. This plan addressed at a very general and preliminary level the ISE’s information-sharing strategy, roles, and needs. The document set out to include: (1) an operational concept; (2) the implementation overview; (3) a summary of desired operational capabilities; (4) means to develop an architecture and standards; (5) an approach to sharing with non-federal partners; (6) ISE enabling activities; (7) implementation management; (8) recommendations on a structure for expansion and future management; and (9) a summary of implementation actions. The plan also acknowledged numerous challenges to be addressed, including promoting a culture of information sharing, protecting information privacy, and handling terrorism-related information. Under the plan, the ISE is comprised of five “communities of interest,” encompassing intelligence, law enforcement, defense, homeland security, and foreign affairs. Each community may comprise multiple federal organizations and other stakeholders; information is to be shared across these communities.

Key ISE Players and Roles

ISE leadership lies with the presidentially appointed Program Manager, for whom the Intelligence Reform Act, as amended, lays out specific requirements. Pursuant to the Act, the Program Manager, in consultation with the head of any affected department or agency, has governmentwide authority over the sharing of terrorism-related information within the scope of the ISE and is required to plan for, oversee implementation of, and manage the ISE. For example, the Program Manager, in consultation with the ISC and consistent with the direction and policies issued by the President, the Director of National Intelligence, and the Director of the Office of Management and Budget, is to issue governmentwide procedures, guidelines, instructions, and functional standards, as appropriate, for the management, development, and proper operation of the ISE. In fulfilling this responsibility, the Program Manager must, among other things, take into account the varying missions and security requirements of agencies participating in the ISE and ensure the protection of privacy and civil liberties. The implementation plan further described areas of responsibility in broad terms for the Program Manager. The plan states, for example, that the Program Manager is to “act as the central agent to improve terrorism-related information sharing among ISE participants by working with them to remove barriers, facilitate change, and ensure that ISE implementation proceeds efficiently and effectively.” In interpreting these responsibilities, the Program Manager has exercised discretion by focusing on, for example, facilitating information sharing across the five ISE communities. To support the development of the ISE, as of June 2008 the Program Manager has a staff of about 11 government staff and 31 contractors organized into three divisions—technology, policy and planning, and business process.

Interagency support and advice to the Program Manager on the development of the ISE is provided through the ISC. The ISC is chaired by the Program Manager and is currently composed of 16 other members, each designees of: the Secretaries of State, Treasury, Interior, Transportation, Health and Human Services, Commerce, Energy, and Homeland Security; the Department of Defense’s Office of the Secretary of Defense as well as the Joint Chiefs of Staff; the Attorney General; the Director of National Intelligence; the Director of the Central Intelligence Agency; the Director of the Office of Management and Budget; the Director of the FBI; and the Director of the National Counterterrorism Center. The ISC is an advisory body, which among other things, is expected to advise the President and the Program Manager on development of policies, procedures, guidelines, roles, and standards necessary to establish, implement, and maintain the ISE; work to ensure coordination among the federal agencies participating in the establishment, implementation, and maintenance of the ISE; and identify and recommend solutions to gaps between existing technologies, programs, and systems used by federal agencies for sharing information and the parameters of the proposed information-sharing environment.

The ISC and Program Manager are supported by various task and working groups. For example, the Foreign Government Information Sharing Working Group, with coordination and assistance from the PM-ISE, helped develop a checklist of issues to be taken into account in negotiating international agreements. Similarly, an Alerts and Notifications Working Group was established to assist the PM-ISE and ISC members in their efforts to identify the alerts and notifications to be available to federal and non-federal ISE participants.

Another area of roles and responsibilities for the ISE lies with individual federal agencies (including those that belong to the ISC and those that do not), state and local governments, and private sector entities. In accordance with the Intelligence Reform Act, as amended, any federal department or agency using or possessing intelligence or terrorism-related information, operating a system in the ISE, or otherwise participating or expecting to participate in the ISE must fully comply with information- sharing policies, procedures, guidelines, rules and standards established pursuant to the ISE. The departments and agencies must further ensure the provision of adequate resources for systems and activities supporting operation of and participation in the ISE, ensure full department or agency cooperation in the development of the ISE to implement governmentwide information sharing, and submit, as requested, any reports on the implementation of ISE requirements within the department or agency. State and local governments also play a role in the ISE through, for example, their law enforcement efforts to prevent crimes. As such, these governments are coordinated with and participate in implementing the ISE. Private sector organizations may share terrorism-related information on a voluntary basis through existing or newly developed ISE mechanisms as well. For example, the ISE leverages existing national plans such as the National Infrastructure Protection Plan, which established mechanisms for public and private sector organizations to share critical infrastructure information on 17 critical infrastructure sectors, such as banking and finance, energy, chemical, and transportation.

Initial Steps to Define a Structure and Approach to Implement the ISE Have Been Taken, but Work Remains to Define What the ISE Is to Include, to Design How It Will Operate, and to Outline Measurable Steps and Time Frames to Achieve Implementation and Desired Results

To guide the design and implementation of the ISE, the Program Manager has issued an implementation plan, completed a number of tasks contained in it, and other independent and ongoing information-sharing initiatives have been integrated into the ISE, but the plan does not include some important elements needed to implement the ISE. The plan provides an initial structure and approach for ISE design and implementation, as well as describes a two-phased approach for implementing the ISE by June 2009. Completed activities include, among other things, development of proposed common terrorism information sharing standards (CTISS) for sharing terrorism-related information. In addition, other federal, state, and local initiatives to enhance information sharing across the government have been or are being incorporated into the ISE. Based on existing federal guidance as well as our prior work and the work of others, standard practices in program and project management for defining, designing, and executing programs include (1) defining the program’s scope, roles and responsibilities, and specific results to be achieved, along with the individual projects needed to achieve these results, and (2) developing a road map, or program plan, to establish an order for executing specific projects needed to obtain defined programmatic results within a specified time frame and measuring progress and cost in doing so. While efforts to date may represent the groundwork needed to facilitate terrorism-related information sharing in the future, work remains to define and communicate the scope and desired results to be achieved by the ISE, the specific milestones and time frames for achieving the results, and the individual projects and the sequence of projects needed to achieve these results. Without such elements the Program Manager risks not being able to effectively manage and implement the ISE.

The Implementation Plan Provides an Initial Structure and Approach for Designing and Implementing the ISE

Issued in November 2006, the implementation plan provides an initial structure and approach for ISE design and implementation and incorporates Presidential Guidelines as well as ISE requirements spelled out in the Intelligence Reform Act. For example, the plan includes steps towards developing standardized procedures for managing, handling, and disseminating sensitive but unclassified information as well as protecting information privacy, as called for in the Presidential Guidelines. For the most part, the plan also maps out a timeline for further defining what information, business processes, and technologies are to be included in the ISE and exploring approaches for implementing the ISE. For example, the plan describes a two-phased approach to implementing the ISE by June 2009, with Phase 1 scheduled for the November 2006 to June 2007 time frame and generally covering set-up activities and building relationships among stakeholders and Phase 2, beginning July 2007, covering design as well as implementation. This approach is intended to develop the ISE incrementally over a 3-year period. The two phases are comprised of 89 action items organized by priority areas. These priority areas address important aspects of the ISE, from defining information- sharing capabilities and technologies to protecting privacy and measuring performance (see table 1).

Forty-eight of the action items, all part of Phase 1, were to be completed by June 2007. Of these 48, 18 were completed on time and an additional 15 were completed by March 2008 (see app. I for details). Examples of completed activities covered by these action items include: The development of proposed common terrorism information sharing standards—a set of standard operating procedures intended to govern how information is to be acquired, accessed, shared, and used within the ISE. According to the Program Manager, the proposed standards document the rules, conditions, guidelines, and characteristics of business processes, production methods, and products supporting terrorism-related information sharing. These standards are intended to address the Presidential Guideline that required the Director of National Intelligence— in coordination with the Secretaries of State, Defense, Homeland Security, and the Attorney General—to develop and issue such standards. These standards are an important early activity because of the structure they are intended to establish for sharing across all ISE stakeholders.

The development of procedures and markings for sensitive but unclassified information to facilitate the exchange of information among ISE participants. We reported in March 2006 that federal agencies use numerous sensitive but unclassified designations that govern how this information must be handled, protected, and controlled and that the confusion caused by these multiple designations creates information- sharing challenges. Therefore, we recommended the issuance of a policy that consolidates sensitive but unclassified designations where possible and addresses their consistent application across agencies. Consistent with our recommendation, in May 2008 the Administration established controlled unclassified information (CUI) as the single categorical designation throughout the executive branch and established a corresponding CUI framework for designating, marking, safeguarding, and disseminating information designated as CUI. Once implemented, this effort could help improve access to information and improve information sharing.

Establishment of an initial operating capability for the Interagency Threat Assessment and Coordination Group (ITACG). The purpose of the ITACG is to support the efforts of the National Counterterrorism Center to produce federally-coordinated terrorism-related information products intended for dissemination to state, local, tribal, and private sector partners through existing channels established by federal departments and agencies. This effort is expected to help address concerns that federally produced terrorism-related information that state, local, tribal, and private sector organizations need for law enforcement and homeland security purposes is sometimes conflicting or not getting to them.

The establishment of a Federal Fusion Center Coordination Group to identify federal resources to support the development and maintenance of a network of state-sponsored fusion centers. Most states and many local governments have created state and local fusion centers to address gaps in information sharing, such as those that occurred on 9/11. These centers are collaborative efforts to detect, prevent, investigate, and respond to criminal and terrorist activities. In October 2007, we issued a report on the characteristics of and challenges for fusion centers and stated that the centers were particularly concerned about sustaining their operations over the long term. We recommended that this group, through the ISC and the Program Manager, determine and articulate the federal government’s role in, and whether it expects to provide resources to, fusion centers over the long-term to help ensure their sustainability. According to ISE program management officials, work is ongoing to (1) complete a baseline capability assessment of designated state and major urban-area fusion centers and (2) develop a coordinated federal support plan that articulates resources being provided to the fusion centers.

The implementation of electronic directory services pages to help identify sources where terrorism information may be located within the federal government, as called for in the Intelligence Reform Act. In meeting this requirement, the electronic directory services are described as a collection of directories that enables ISE users to search for and locate information by accessing the appropriate people, organizations, data, and services related to the counterterrorism mission. The Program Manager expects to develop similar directories for state, local, and tribal stakeholders.

Furthermore, work has been done towards accomplishing some action items that are not yet complete. For example, agencies, with leadership from the PM-ISE, have been working to develop a core training module intended to provide an introduction to the ISE and to further promote the development of a culture of information sharing. The incomplete action items are generally those that require a greater level of stakeholder involvement and, according to officials at the Office of the Program Manager are taking longer than anticipated to complete, but will not delay work on Phase 2 items. However, the action items do not address all the activities that must be completed to implement the ISE, according to officials at the Office of the Program Manager, and several activities identified in the implementation plan will not be implemented as identified in the plan. For example, one activity identified in the plan included the implementation of an electronic directory of services containing green pages in the unclassified domain. As identified in the plan, the green pages were to provide a searchable listing of counterterrorism-related information-sharing resources, systems, and data repositories to support users searching for specific data and capabilities. Further, the pages were to provide system descriptions and technical and operational contact information for gaining access. However, according to officials at the Office of the Program Manager, aggregating the information for the green pages would no longer enable the information to be posted in an unclassified domain. Therefore, the green pages will no longer be completed for the sensitive but unclassified security domain. Appendix I provides further detail on the status of each Phase 1 action item.

Federal, State, and Local Agency Initiatives Are Being Leveraged to Enhance Information Sharing and Guide Implementation of the ISE

Federal, state, and local agencies have their own initiatives to enhance information sharing across the government that are being leveraged in designing and implementing the ISE. Examples of these initiatives include: The Director of National Intelligence (DNI) issued a 100-day plan in April 2007, followed by a 500-day plan in September 2007 that focused on integrating the intelligence agencies and their missions in a collaborative manner. One area of focus in these plans is improved information sharing. As a result of this effort, the DNI reported that an implementation plan was developed to standardize identity and access policies across agencies, networks, and systems. The 100-day plan notes that as it is implemented, its results are intended to be leveraged by the Program Manager as part of the ISE because it is anticipated to improve communication within the intelligence community—one of the five communities that have been designated as critical to the ISE.

The National Counterterrorism Center (NCTC) was established in 2004 in response to recommendations from the 9/11 Commission to operate as a partnership of intelligence agencies so that they can analyze and disseminate national intelligence data. The center works to ensure that intelligence agencies have access to and receive all-source intelligence support needed to execute their counterterrorism plans or perform independent, alternative, and mission-oriented analysis.

As previously noted, in recognition of fusion centers as important mechanisms for information sharing, the federal government—including the Department of Homeland Security (DHS), the Department of Justice (DOJ), and the Program Manager—is taking steps to partner with these centers. Although they were created primarily to improve information sharing within the state or local area, the implementation plan identifies the creation of an integrated national network of fusion centers to promote two-way sharing with the federal government, as discussed earlier. Toward developing this network, the Program Manager and stakeholder agencies have sponsored fusion center conferences and provided staff, technical assistance, and funding to these centers.

The FBI’s Terrorist Screening Center (TSC)—established in September 2003—maintains the U.S. government’s consolidated watch list of known or suspected terrorists and sends records from the list to agencies to support terrorism-related screening. The 9/11 Commission determined that agencies’ failures to share information they had on several of the terrorists was a major factor in the lead-up to the 9/11 attacks, and we recommended in a 2003 report that agencies develop such a consolidated database of terrorist records. In response, the TSC created its consolidated database, which was completed in 2004. The TSC receives the majority of its watch list records from the NCTC, which compiles the information on known or suspected international terrorists from federal agencies. The FBI provides information on known or suspected terrorists who operate within the United States. The TSC consolidates this information and sends it to federal agencies that use it for screening purposes, such as the screening of visa applicants and airline passengers. As noted in the annual report, the founding of the TSC is considered to be a key milestone in establishing the ISE. We and the Inspector General for the Department of Justice have also recommended ways in which agencies can enhance the watch list and agencies’ terrorist-screening processes, such as addressing vulnerabilities and creating an interagency governing entity.

Further Detailing What the ISE Is to Achieve and How It Will Operate Should Better Guide Implementation

The Program Manager, together with the ISE stakeholders, have followed standard practices in program and project management for defining, designing, and executing programs by identifying action items and strategic goals to be achieved in the implementation plan. However, work remains in, among other things, defining and communicating the scope and desired results to be achieved by the ISE, the specific milestones to be attained, and the individual projects—or initiatives—and execution sequence needed to achieve these results and implement the ISE. Standard practices in program and project management include (1) defining the program scope, roles and responsibilities, and specific results to be achieved, along with the individual projects needed to achieve these results, and (2) developing a road map, or program plan, to establish an order for executing specific projects needed to obtain defined programmatic results within a specified time frame and measuring progress and cost in doing so.

Further Defining and Communicating Key Elements of the ISE Will Help Address the Limitations of the ISE and Further Describe How the ISE Is to Operate

First, toward defining the scope of the ISE, the implementation plan restates the text of the Intelligence Reform Act, noting that the ISE encompasses “the sharing of terrorism information in a manner consistent with national security and with applicable legal standards relating to privacy and civil liberties” and that the ISE is defined as “an approach that facilitates the sharing of terrorism information.” Indeed, this is a broad scope requiring the Program Manager and stakeholders, such as members of the Information Sharing Council, to further define what the ISE, as a program, is to include as well as the scope of what it can address. Fundamentally, the Program Manager and stakeholders are still trying to fully define the scope and design of the ISE, and a more complete set of activities needed to achieve it than those that were included in the implementation plan, including, for example all of the terrorism-related information that should be a part of the ISE; what types of terrorism-related information ISE participants have and where such information resides; how the information can be put into a “shared space” so that a cross- sector of users can easily access and study information from different agencies; how this access can be provided while still protecting sensitive information and privacy interests; what information systems and networks will be integrated as part of the ISE and how; and methods for motivating agencies to invest in the ISE, be held accountable for ensuring that all relevant information is made available to ISE stakeholders, and identifying and implementing the specific projects needed to ensure the ISE runs effectively.

Further, the plan notes that the Intelligence Reform Act requires that the ISE ensure direct and continuous online electronic access to information and presents several action items intended to identify approaches for sharing information, including the use of technologies. However, the plan does not lay out a set of action items with related milestones for identifying, among other things, needed resources such as all the information to be made available as part of the ISE, the source of the information, and what limitations exist in making this counter-terrorism information available. In accordance with standard practices for program management, these are all elements critical for conveying the scope of what the ISE is to include, garnering an understanding among stakeholders of needs to be met as part of implementing the ISE, and identifying restrictions in stakeholder abilities to do so.

We recognize that defining all of these elements is a complex undertaking, especially because of the numerous ISE stakeholders that need to coordinate and the many existing and often stovepiped or independent methods stakeholders use for meeting their information needs that often were not developed with sharing in mind. Nevertheless, further defining and communicating key elements of the ISE, such as the scope and expected results, along with a road map for meeting needs in accordance with standard practices for program management will help, among other things, communicate the breadth and limitations of the ISE as a program and further describe how the ISE is to operate.

Second, the plan does not communicate the scope, or parameters, of stakeholder roles and responsibilities in such a way that stakeholders can understand what they will be held accountable for in implementing and operating the ISE. For example, the plan identifies the Program Manager’s role as responsible for information sharing across the government, overseeing the implementation of and managing the ISE, and working together with the ISC, but does not articulate aspects of how the Program Manager has interpreted this role in contrast to that of other stakeholders. For instance, the officials at the Office of the Program Manager noted: The Program Manager’s office works on developing or improving existing business processes that affect information sharing among two or more of the five ISE communities, but does not focus on processes that are internal to ISE members unless they directly impact the wider ISE. Agencies, therefore, are to define ISE related business processes and other requirements internal to their organizations along with how the information will be used and drive their own analytical efforts.

The Program Manager’s role focuses on determining if a policy, business process, legal or technical issue is preventing the sharing of information between two or more communities and on helping to resolve these types of issues rather than issues that impact sharing within a community, such as homeland security.

This information on the parameters of the Program Manager’s role and responsibilities was not transparently communicated in the plan but is critical for stakeholders, the Congress, and other policy makers to clearly understand, provide for accountability, and ensure the ISE is effectively implemented. Without clearly understanding their roles and responsibilities, stakeholders may not adequately prepare for and provide each other the information and services needed to prevent terrorist attacks. According to officials at the Office of the PM-ISE, departments and agencies, not the Program Manager alone, are responsible for defining the ISE’s scope and expected end state. Accordingly, in November 2007 they held a first-time off-site with ISC members to focus on ISE priorities, clarify responsibilities, and emphasize the importance of everyone’s active participation and leadership. Moreover, the meeting was held to rectify any misperceptions and reinforce that all ISE stakeholders are to define the ISE. However, according to officials at the Office of the Program Manager, problems in department and agency participation make it difficult for the ISC to function as an advisory body for ISE implementation. Among other things, officials noted that departments and agencies do not always provide representatives with the authority to speak on behalf of the agency and inconsistent attendance by ISC representatives has been an issue.

Since issuance of the plan, on October 31, 2007, the National Strategy for Information Sharing was issued, in part, further communicating the scope of the ISE and stakeholder roles. The strategy reaffirmed that stakeholders at all levels of government, the private sector, and foreign allies play a role in the ISE. The strategy also outlined some responsibilities for ISE stakeholders at the state, local, and tribal government levels. In addition, the strategy further defined the role of the Program Manager as also assisting in the development of ISE standards and practices. However, the strategy did not further clarify the parameters of the Program Manager’s role and what is within the scope of his responsibilities in “managing” the ISE and improving information sharing versus other ISE stakeholders.

Third, the Program Manager and stakeholders are still in the process of defining the programmatic results to be achieved by the ISE as well as the associated milestones and projects needed, as standard practices in program management suggest for effective program planning and performance measurement. Existing federal guidance as well as our work and the work of others indicates that programs should have overarching strategic goals that state the program’s aim or purpose, that define how it will be carried out over a period of time, are outcome oriented, and that are expressed so that progress in achieving the goals can be tracked and measured. Moreover, these longer-term strategic goals should be supported by interim performance goals (e.g., annual performance goals) that are also measurable, define the results to be achieved within specified time frames, and provide for a way to track annual and overall progress (e.g., through measures and metrics). The implementation plan, as an early step in planning for the ISE, identifies six strategic ISE goals to be achieved. These goals include, for instance, to the maximum extent possible, the ISE is to function in a decentralized, distributed, and coordinated manner. However, the plan does not define what this goal means, set up interim or annual goals and associated time sensitive milestones to be built upon to achieve the overall goal, or define how agencies will measure and ensure progress in meeting this goal in the interim or overall. Instead, the plan notes that performance measures will be developed at a later date. Moreover, the plan does not present the projects and the sequence in which they need to be implemented to achieve this strategic goal in the near term or in the future, or the specific resources needed and stakeholder responsibilities. Therefore, work remains in developing the road map for achieving this strategic goal. Since the plan’s issuance, officials in the office of the Program Manager and stakeholders have developed several performance measures and, as of March 2008, were in the process of further refining them. Yet, our review of a draft of these performance measures showed that they continue to focus on counting activities accomplished rather than results achieved and do not yet outline the sequence of projects needed to implement the ISE and measurably report on progress in doing so.

Further, the plan identifies seven priority areas to be addressed in implementing the ISE. These include, for example, sharing with partners outside the federal government, promoting a culture of information sharing, and establishing ISE operational capabilities. But like the strategic goals, the priority areas represent general tasks and themes to be addressed as part of the ISE and do not define expected results in a measurable form, along with supporting performance goals, measures, and deadlines for achieving the programmatic results. Without these elements, ISE stakeholders may not understand the interim or final ISE they are to achieve, assess progress towards implementing the ISE, or hold stakeholders accountable for their contributions in ensuring that the ISE succeeds.

Fourth, although required by the Intelligence Reform Act, the implementation plan did not provide a budget estimate that identified the incremental costs associated with designing, testing, integrating, deploying, and operating the ISE but indicated that steps to develop a budget estimate would be taken in the future. In part, this is because the ISE was in such an early stage of development that it would be difficult for agencies to know what to cost out for an estimate. Developing a budget estimate, however, is a commonly used tool for effective program management. While the Program Manager has been working with agencies and the Office of Management and Budget to determine the cost of implementing the ISE, officials at the Office of the Program Manager stated that the total cost of the ISE has not yet been accounted for and that attaining an overall estimate may not be achievable. This is because it is difficult for agencies to isolate and separate out what actions they are undertaking solely to implement the ISE versus ongoing operations. We recognize that attaining an accurate and reliable cost estimate for the ISE is a difficult undertaking, complicated further by the fact that stakeholders are still defining the scope of the ISE, results to be attained, and the projects to support it. However, without information on how much the ISE will cost, Congress and stakeholders will be unable to determine whether the expenses associated with the ISE are worth the results attained and in some cases unable to determine what has been accomplished given the expended resources. Toward addressing this cost issue, the PM-ISE, in collaboration with OMB, has since issued program guidance intended to assist in estimating and tracking ISE costs in ISE priority areas, such as suspicious activity reporting, developing ISE shared space, and alerts, warnings, and notifications.

Finally, while the implementation plan states that Phase 1 will conclude with the development of a detailed plan for implementation, including goals, measures, and targets, a revised plan will not be issued. Instead, officials at the Office of the Program Manager indicated that they consider the implementation plan to be a living document with initiatives identified at the outset of development being refined as needed based on experience. Officials at the Office of the Program Manager acknowledged that the 89 action items contained in the plan do not address all of the activities that must be completed to implement the ISE. This is because at the time the plan was produced, agreement on how the ISE is to function and what it is to include had not been reached among the stakeholders. Work toward reaching these agreements remains ongoing. Therefore, program officials stated that an assessment of the ISE’s progress based on the action items identified in the plan alone would not give a true sense of progress toward a fully functioning and executed ISE. Accordingly, the PM-ISE intends to adjust the plan, beginning with refinements in the next annual report. For example, according to officials at the Office of the Program Manager, to avoid delaying progress, the office plans to revise and update certain implementation plan actions in the course of developing the June 2008 Annual Report. In addition, officials at the Office of the Program Manager stated that based on their experience in Phase 1, they are deleting action items that are no longer valid and updating others to reflect the ISE’s current approach for implementation.

Making midcourse corrections to further determine and articulate the end design of the ISE, or at least more accurately specify what is to be achieved in the near term and at various milestones thereafter, is in accordance with standard practices in program and project management. However, given the ISE’s many stakeholders and the work that remains to be done in defining the scope of the ISE, the desired results to be achieved, and the supporting projects and milestones, it is important that the revisions, in accordance with standard practices for program management, provide for an effective road map to implement the ISE and measure achieved progress in implementing the ISE and in improving information sharing. Without such a road map, the Program Manager and stakeholders risk not being able to effectively manage and implement the ISE.

An ISE Enterprise Architecture Framework Has Been Developed, but Its Usefulness May Be Limited without Further Defining ISE Results

Subsequent to the implementation plan, in August 2007, the Program Manager issued the ISE Enterprise Architecture Framework Version 1.0 (ISE EAF), a planning document and tool intended to further inform ISE implementation efforts, but its usefulness in guiding the ISE to meet terrorism-related information-sharing needs may be hindered by the lack of defined programmatic results to be achieved. As reported by the Program Manager, the ISE EAF is to help improve information-sharing practices, reduce barriers to sharing, and institutionalize sharing by providing a new construct, or framework, for planning, installing, and operating nationwide information resources within the ISE. Such resources may include, for example, business processes and information technologies. Further, as noted in the EAF, it is to be used to guide the implementation of the ISE, accounting for current capabilities and setting the direction and steps towards the envisioned or To-Be capabilities. Because the ISE is composed of many organizations, the ISE EAF can be looked at as a collection of independent stakeholder enterprise architectures that were initially designed to support individual missions, but are now being leveraged to facilitate terrorism-related information sharing among these organizations. In doing so, the ISE EAF is to assist in identifying the relationships needed to facilitate terrorism information sharing among these organizations and is to serve as a tool for understanding what, where, and for what purpose current capabilities and resources, such as information technology systems, may exist.

Enterprise architectures generally use strategic planning elements to align potential system solutions with program needs. While the ISE EAF is intended to augment organizations’ enterprise architectures for the purpose of sharing terrorism-related information, work remains to determine the ISE’s desired program outcomes or specific results to be achieved, potentially limiting the effectiveness of the ISE EAF in guiding the ISE to meet terrorism-related information sharing needs. Unlike agency enterprise architectures, the ISE EAF does not seek to identify, for example, business processes and information flows at an operational level, the level at which organizations determine how specific investments in technologies will be used to support business needs and provide needed information. Instead, the ISE EAF relies on the prerogative of individual departments and agencies to define operational processes and information flows as part of their enterprise architectures. Officials at the Office of the Program Manager noted that OMB and the ISC agencies were very specific about the level of detail the ISE EAF was to take, noting that the ISE EAF helps inform, but not direct, how departments and agencies do their work at the operational level—individually or together. However, without further defining outcomes to be achieved and identifying how individual agencies are to work together to meet ISE information-sharing needs at the level where work is done, the ISE EAF may be limited in its usefulness for improving the sharing of terrorism-related information.

The Program Manager Has Issued the First Annual Report and Is Developing Initial Performance Measures, but Neither Can Yet Be Used to Determine How Much Progress Has Been Made and What Remains

To describe progress in implementing the ISE to date, the Program Manager issued an annual report—in response to the Intelligence Reform Act’s requirement for a yearly performance management report—in September 2007 that highlighted individual accomplishments and included annual performance goals and has since developed some performance measures, but neither effort shows how much measurable progress has been made toward implementing the ISE and how much remains to be done. In keeping with federal guidance, our work, and the work of others in strategic planning, performance measurement, and program management, the annual report contained four performance goals for 2008. Additionally, some initial performance measures have been developed, but they do not address all aspects of the annual performance goals or strategic goals and do not show how they represent interim milestones to ensure attainment of desired results or outcomes. According to officials at the Office of the Program Manager, these performance measures are currently being refined in consultation with the ISC to provide the needed framework to measure real progress made. We acknowledge that creating such measures is difficult, particularly since the program is still being designed, but until these measures are refined to account for and communicate progress and results, future attempts to measure and report on progress will be hampered.

The Annual Report Cited Accomplishments Made in Implementing the ISE, but Not the Extent of Progress Achieved and Remaining Work

The annual report conveyed individual ISE-related accomplishments as of September 2007 but did not provide Congress and policy makers with information on what portion of the ISE has been completed as a result of this work and what portion remains. The report lists the preliminary actions taken to prepare for establishing the ISE, such as designation of the Program Manager, the President’s memorandum providing guidelines for the ISE, and submission of the implementation plan to the Congress. The report also cites individual accomplishments that contribute to the ISE, some of which were accomplished under the implementation plan— such as establishment of an electronic directory service for users to find contact information for organizations that have counter-terrorism missions—and others achieved prior to and or separate from efforts to create the ISE. For instance, the report cites several accomplishments attained prior to the December 2004 Intelligence Reform Act and its call for an ISE, including the creation of the National Counterterrorism Center (NCTC) in August 2004 and the establishment of the Terrorist Screening Center (TSC) in 2003. In part, because ISE implementation remains in the early stages, the annual report highlighted these discrete accomplishments without putting them in an overall context that showed how much progress has been made and remains toward implementing the ISE. While, as previously noted, the implementation plan identified a two phased approach for implementing the ISE along with 89 action items—the only means presented in the implementation plan for tracking completion of ISE implementation—the report did not provide a one-for-one reporting on the status of these action items as steps for implementing the ISE or identify how much of the implementation had been completed. Thus, the Congress and policy makers do not yet have the information they need to assess the amount and rate of progress, remaining gaps, and the need for any intervening strategies.

Performance Measures Are Being Developed Although They Do Not Yet Address All Aspects of the Annual Performance Goals

In accordance with existing federal guidance as well as our work and the work of others in strategic planning, performance measurement, and program management, programs should have overarching strategic goals that state the program’s aim or purpose, define how it will be carried out over a period of time, are outcome oriented, and are expressed so that progress in achieving the goals can be tracked and measured. Moreover, these longer-term strategic goals should be supported by interim performance goals (e.g., annual performance goals) that are also measurable, define the results to be achieved within specified time frames, and provide for a way to measure and track annual and overall progress (e.g., through measure and metrics). Accordingly, the implementation plan contained six overall strategic goals and the annual report contained four annual performance goals for 2008, as shown in tables 2 and 3.

While not reflected in the first annual report, the Program Manager and agencies have begun to develop performance measures to improve future reporting on progress in implementing the ISE and information sharing overall, but these measures focus on counting activities accomplished rather than results achieved to show the extent of ISE implementation and attaining the ISE’s strategic goals. In accordance with our work and federal guidance on strategic planning and performance measurement, the newly developed measures represent an effort to more concretely and quantitatively assess progress in implementing the ISE and improving information sharing. The performance measures include, for example, the number of ISE organizations with a procedure in place for acquiring and processing reports on suspicious activities potentially related to terrorism, but not how the reports are used and what difference they are making in sharing to help prevent terrorist attacks. Similarly, the measures attempt to assess the creation of a culture of sharing by tabulating the percentage of relevant ISE organizations that have an information-sharing governance body or process in place, but not by measuring the outcome—such as how and to what extent cultural change is being achieved. Indeed, these measures are an important first step in providing quantitative data for assessing progress made in information sharing and help to inform Congress and other stakeholders on specific information sharing improvements. But, taking the measures to the next step—from counting activities to results or outcomes—while difficult, is important to assess results achieved.

The Program Manager and ISE stakeholders have not yet developed measures to address all aspects of the annual performance goals. For example, one 2008 performance goal identified in the annual report is to establish capabilities that allow ISE participants to create and use quality terrorism-related information by improving business processes, developing a common enterprise architecture framework, refining common standards, and instituting effective resource management for governmentwide programs. Based on the description of this performance goal, one ISE performance measure that supports this goal includes attaining the percentage of applicable ISE organizations that have adopted the common terrorism information sharing standards during the past or preceding fiscal year(s). However, performance measures in support of all topics identified in the goal, such as instituting effective resource management for governmentwide programs, have not been developed. Further, the performance measures are not presented in a way that explains how they represent milestones toward attaining the strategic goals or intended outcomes. According to officials at the Office of the Program Manager, as of March 2008, they are refining their measures in consultation with the ISC to provide an improved framework to measure progress made. Yet, our review of a draft of these performance measures showed that they continue to focus on counting activities accomplished rather than results achieved. We acknowledge that creating such measures is difficult, particularly since the program is still being designed, but until these measures are refined to account for and communicate progress and results, future attempts to measure and report on progress will be hampered.

Conclusions

Although the Program Manager and stakeholders have made progress in implementing a number of initiatives, successfully implementing the ISE remains a daunting task. While efforts to date may represent the groundwork needed to facilitate terrorism-related information sharing in the future, over 3 years after passage of the Intelligence Reform Act, the ISE is still without a clear definition of the specific results to be achieved as part of the ISE or the projects, stakeholder contributions, and other means needed to achieve these results. The Program Manager, together with the ISE stakeholders, have followed standard practices in program and project management for defining, designing, and executing programs by identifying action items and strategic goals to be achieved in the implementation plan. However, work remains in, among other things, defining and communicating the scope and desired results to be achieve by the ISE, specific milestones to achieve the results, and the individual projects and execution sequence needed to achieve these results and implement the ISE. Until this work is complete, further efforts may result in independent contributions to improving information sharing rather than an ISE with improved and coordinated sharing of terrorism-related information among stakeholders, a critical need exposed by the terrorist attacks of September 11. Given that the ISE requires extensive buy-in from stakeholders and the Program Manager is relying on stakeholders to provide technology and other resources to make the ISE work, it is critical to develop a road map for implementing the ISE and improving information sharing that communicates the scope and specific results to be achieved by the ISE, the key milestones and individual projects needed to implement the ISE, needed resources, and stakeholder responsibilities. Without such a road map, the Program Manager risks not being able to effectively manage and implement the ISE.

Furthermore, efforts to report on progress to date have provided examples of individual actions taken to improve information sharing but have not yet included an accounting of how far the Program Manager and stakeholder agencies are in achieving an effectively functioning ISE and what remains to be done. By not doing so, stakeholders do not have a measurable way to ensure that the sharing of terrorism-related information has improved and by how much, nor the information needed to understand the resources and time frames required to achieve the intended results of the ISE. Until the Program Manager and stakeholders more fully define the specific results the ISE is to attain and develop a set of measures to assess progress in achieving the goals—including, at a minimum, what has been done and what remains to be accomplished— Congress and stakeholders will not know how far the nation has come in implementing an ISE intended to improve governmentwide information sharing.

Recommendations

To help ensure that the ISE is on a measurable track to success, we are recommending that the Program Manager, with full participation of relevant stakeholders (e.g., agencies and departments on the ISC), take the following two actions more fully define the scope and specific results to be achieved by the ISE along with the key milestones and individual projects or initiatives needed to achieve these results, and develop a set of performance measures that show the extent to which the ISE has been implemented and sharing improved—including, at a minimum, what has been and remains to be accomplished—so as to more effectively account for and communicate progress and results.

Agency Comments and Our Evaluation

We requested comments on a draft of this report from the Secretaries of Defense, Homeland Security, and State, as well as the Attorney General, the Director of National Intelligence, and the Program Manager for the ISE or their designees. In a June 6, 2008, letter, the Program Manager for the ISE provided written comments, which are summarized below and included in their entirety in appendix II.

The Program Manager generally agreed with our recommendations to more fully define the scope and results to be achieved by the ISE and develop a comprehensive set of performance measures that show the extent to which the ISE has been implemented and sharing improved.

While the Program Manager agreed with our recommendations, he commented that the ISE is a governmentwide transformational effort— emphasizing that the ISE is an evolutionary process—and not a traditional “program.” Therefore, according to the Program Manager, trying to audit this interagency initiative strictly within program parameters presents problems. We agree that the ISE is a governmentwide transformational effort, that it is not a traditional “program,” and that it involves an evolutionary process. In fact, our report states that the ISE is not bounded by a single federal agency or component and that it is a broad-based coordination and collaboration effort among various stakeholders. While we agree that the ISE is not a traditional “program,” in that it is not operated and funded by a single department or agency, it is an activity that does receive government funding and can be reviewed using program and project management principles. As such, we based our evaluation of the ISE on a broad set of program and project management criteria, including the Government Performance and Results Act of 1993, related guidance issued by OMB, and our prior work on results-oriented government, program management, and federal coordination and collaboration. Further, while we recognize that approaches to implementing the ISE and improving information sharing may evolve over time as technologies and needs change, calling the ISE an evolutionary process does not exempt it from following the practices outlined in our report. Following these practices will help ensure that reports of progress by the Program Manager on behalf of the ISE at large are based on measures of results achieved toward implementing the ISE—that is measured based on what the ISE is to be, include, and accomplish in, for example, 3 years—rather than ad-hoc claims of progress.

With regard to efforts for assessing the ISE’s progress, the Program Manager noted that in the 2007 annual report he introduced a performance management approach and his office has since established a performance baseline—in the fall of 2007— and measured agencies’ progress against this baseline through an assessment performed in the spring of 2008. Our report acknowledges these efforts. However, our review of the performance measures developed in support of the performance management approach shows that these measures: (1) focus on counting activities accomplished rather than results achieved to show the extent of ISE implementation and attaining the ISE’s strategic goals and (2) are not presented in a way that explains how the measures represent milestones toward attaining the strategic goals identified in the implementation plan or intended outcomes. In his comments, the Program Manager further noted that the June 2008 annual report, which was not released by the time we issued this report, would provide more current data on performance measurement. However, our review of a draft of the measures to be incorporated in the 2008 report showed that they continue to focus on counting activities accomplished rather than results achieved. Unless the 2008 report corrects these shortfalls and establishes a performance management mechanism whereby short-term annual goals serve as steps for assessing the ISE’s progress towards achieving longer- term strategic goals, it and future reports on progress will fail to provide the Congress and other policy makers the meaningful information needed to understand what progress has been made in attaining the defined strategic results for the ISE and improving information sharing.

Finally, the Program Manager said that although the report mentions that one of the challenges of the ISE is interagency attention and priority to ISE initiatives, the report does not make any recommendations in this regard. We agree that interagency collaboration in the ISE is a challenge and individual departments and agencies, not the Program Manager alone, have responsibilities in implementing the ISE. However, to effectively hold these agencies accountable for ISE progress, existing issues identified in our report—such as defining the outcomes to be achieved and defining clear roles and responsibilities—must first be addressed. Given the ISE’s many stakeholders and recognizing the Program Manager’s key leadership role for managing the ISE, we maintain that these issues must be addressed by the Program Manager, with full participation of relevant stakeholders (e.g., agencies and departments on the ISC). Without doing so, the Program Manager may continue to face challenges in attaining agency buy-in and holding stakeholders accountable for ISE progress.

Officials in the Office of the Program Manager also provided technical comments on the draft that have been incorporated, as appropriate.

The Secretaries of Defense, Homeland Security, and State; the Attorney General; and the Director of National Intelligence responded that they did not have any comments on the report.

As agreed with your offices, unless you publicly release the contents of this report earlier, we plan no further distribution until 30 days from the report date. We will then send copies of this report to the Program Manager for the ISE, the Director of National Intelligence, and the Secretaries of the Departments of Defense, Homeland Security, Justice, and State; and interested congressional committees. In addition, this report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact either Eileen Larence at 202-512-8777 or larencee@gao.gov, or David Powner at 202-512-9286 or pownerd@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Status of Phase I Action Items as of March 1, 2008

Table 1 below provides the status of each of the 48 Phase 1 action items identified in the ISE implementation plan as of July 9, 2007, nine days after their planned completion date and as of March 1, 2008. These action items encompass many areas for development in the ISE, ranging from activities such as identifying capabilities and technology to privacy protection and performance measures. As the table indicates, based on our analysis of status information reported by the Program Manager, at the end of phase one’s scheduled completion, 18 of 48 action items had been completed and 30 remained incomplete. Eight months later, 33 of 48 action items had been completed, with 15 remaining incomplete. In determining the status of the action items, we reviewed documentation provided by the Program Manager, but did not evaluate the effectiveness of the actions taken.

Appendix II: Comments from Office of the Program Manager for the Information Sharing Environment

Appendix III: GAO Contacts and Acknowledgments

GAO Contacts

Acknowledgments

In addition to the contact named above, Susan H. Quinlan, Assistant Director; Richard Ascarate; Jason Barnosky; Amy Bernstein; Joseph Cruz; Thomas Lombardi; Lori Martinez; and Marcia Washington made key contributions to this report.