Homeland Security: Opportunities Exist to Enhance Collaboration at 24/7 Operations Centers Staffed by Multiple DHS Agencies
Published: 2006-10-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Because terrorists do not operate on a 9-5 schedule, the Department of Homeland Security (DHS) and its operational components have established information gathering and analysis centers that conduct activities 24 hours a day, 7 days a week, 365 days a year. Staff at these operations centers work to help detect, deter, and prevent terrorist acts. DHS has determined that out of 25 operations centers, four require higher levels of collaboration that can only be provided by personnel from multiple DHS agencies, and other federal, and sometimes state and local, agencies. For these four multi-agency operations centers, this report (1) describes their missions, products, functions, and customers and (2) assesses the extent to which DHS efforts to promote collaboration among the multiple agencies responsible for the centers reflect key practices for enhancing and sustaining collaborative efforts. To do so, GAO visited operations centers, reviewed data and reports from the centers, and interviewed center and other DHS officials.
section_title: What GAO Found
paragraphs: Each of the four multi-agency 24/7/365 operations centers has a different mission and therefore produces different products, yet all contribute to the larger mission of DHS and have similar functions and customers. Customs and Border Protection runs two of the four multi-agency operations centers--the National Targeting Center and the Air and Marine Operations Center. The former monitors the international movement of potential terrorists and produces reports on suspect individuals; the latter maintains situational awareness of the nation's airspace, general aviation, and sea-lanes and produces reports on suspicious private air and marine craft. The Transportation Security Administration's operations center monitors passengers on commercial flights; works to mitigate the vulnerabilities of commercial airports, rail stations, and pipelines, the National Capital Region, and critical infrastructure across the nation; and produces reports on these topics. DHS's Operations Directorate runs the National Operations Center Interagency Watch and works to enhance efficiency and collaboration among DHS components. This operations center has a more strategic mission in that it uses information gathered by the other operations centers to provide overall national situational awareness, and it prepares security briefs for federal, state, and local law enforcement agencies. Opportunities exist to enhance collaboration among 24/7/365 multi-agency operations centers. While DHS has leveraged resources by having staff from multiple agencies work together, the centers lack joint strategies for collaboration and staffing needs assessments, and they have not established a definition of watchstander roles for all agencies at each center. The centers also lack standards and procedures for using DHS's primary information sharing network; mechanisms to monitor, evaluate, and report on results; and reinforced accountability through agency plans and reports. GAO's previous work has shown that such practices are effective in enhancing and sustaining collaboration among federal agencies. The establishment of DHS's Operations Directorate in 2005 provides a means to promote implementation of more collaborative practices at the centers.

section_title: Letter
paragraphs: The Homeland Security Act of 2002 established the Department of  Homeland Security (DHS) with the primary mission of preventing terrorist  attacks against the United States. To accomplish this mission, DHS must  be aware of the potential risks and vulnerabilities faced by the nation,  including terrorist threats to our transportation infrastructure (such as  rail, aviation, and shipping); terrorists entering our country through land,  air, and sea ports; and terrorists operating within our borders. Because  terrorists do not operate on a 9-to-5 schedule, DHS and some of its  operational components (six DHS agencies and DHS’s Operations  Directorate) have established information gathering and/or analysis  centers that conduct activities 24 hours a day, 7 days a week, 365 days a  year. Staff at these operations centers conduct monitoring and  surveillance activities to help detect, deter, and prevent terrorist acts by  providing real-time situational awareness to a variety of federal, state, and  local governments and private-sector entities. According to DHS, this  constant situational awareness is needed to facilitate an immediate  response during times of crisis or other national incidents.
The six DHS agencies and the DHS Operations Directorate that run  national operations centers “24/7/365” gather and/or analyze information  through monitoring and surveillance activities to help detect, deter, and  prevent terrorist acts across the entire United States. In addition, DHS  conducts other 24/7/365 activities, such as telecommunications services,  security alarm monitoring, and region-specific operations at a variety of  other centers. We identified a total of 20 national and 5 regional DHS  centers that conduct 24/7/365 activities. Twenty-one of the 25 centers  employ staff from one DHS agency on a regular full-time basis and perform  agency-specific functions. Appendix I provides details on the missions and  functions of these centers.
DHS has determined that the remaining 4 centers have broader security  missions and require higher levels of collaboration that can only be  provided by personnel from multiple DHS agencies, and other federal, and  sometimes state and local, agencies. In January 2003, we designated  implementation and transformation of the new Department of Homeland  Security as high risk based on three factors: the enormity of the effort, the  challenges faced by the components, and the potential impact of failure to  effectively carry out the homeland security mission. Given the critical  homeland security role played by these operations centers and the  opportunity to facilitate the department’s transformation efforts by  maximizing collaboration at the program level, this report focuses on  these four national operations centers that are “multi-agency,” that is,  staffed by personnel from more than one agency within DHS, along with  other federal, and sometimes state and local, agencies. The 4 centers are  described in table 1.
To assess the collaboration among DHS agencies working at each multi- agency 24/7/365 operations center, this report answers the following  questions:  1.  What are the missions, functions, and products of the multi-agency  24/7/365 DHS operations centers and who are their customers?  2.  To what extent has DHS implemented key practices for enhancing  and sustaining collaboration at these multi-agency centers?
To answer our first objective, we analyzed information obtained from the  responsible component agencies and the Operations Directorate on the  mission and functions of all of the 24/7/365 activities in DHS. We visited all  4 multi-agency centers, as well as centers operated by other component  agencies including the Federal Emergency Management Agency, the U.S.  Coast Guard, and the U.S. Secret Service to observe their operations,  interview officials responsible for managing the centers, and identify  centers that employed staff from multiple DHS agencies. From the 4  national operations centers that employed staff from multiple DHS  component agencies--the Air and Marine Operations Center (AMOC), the  National Targeting Center (NTC), the Transportation Security Operations  Center (TSOC), and the National Operations Center-Interagency Watch  (NOC-Watch)—we obtained additional information on both the products  the centers regularly developed and their primary customers. We also  interviewed several staff assigned to centers from participating DHS  component agencies—referred to as watchstanders—to discuss their  roles and responsibilities at the centers and the overall mission of the  centers to which they had been assigned.
To answer our second objective, we met with the acting director and other  responsible officials from the Operations Directorate to discuss its role  and responsibilities. We reviewed transition, management integration, and  planning and policy documents as well as strategic plans and annual  performance reports and planning documents from DHS and its  component agencies. We also reviewed and analyzed the results of studies  undertaken by DHS to assess and improve coordination and collaboration  at the multi-agency centers as well as reports from GAO, the  Congressional Research Service, the DHS Office of Inspector General, and  others that addressed the integration, coordination, and collaboration of  departmentwide program functions. We then assessed DHS’s efforts  related to integration, coordination, and collaboration at the multi-agency  centers to determine the extent to which they reflect DHS’s application of  the key practices we have found can help enhance and sustain  collaboration among federal agencies and found to be at the center of  successful mergers and transformations.
We conducted our work from October 2005 through September 2006 in  accordance with generally accepted government auditing standards.  Appendix II includes more detailed information on our scope and  methodology.
subsections: section_title: Background
subsections: section_title: Twenty-two Agencies Merged to Form DHS in the Aftermath of 9/11
paragraphs: DHS was created in response to the terrorist attacks on September 11,  2001. Not since the creation of the Department of Defense in 1947 has the  federal government undertaken an organizational merger of this  magnitude. Enacted on November 25, 2002, the Homeland Security Act  established DHS by merging 22 distinct agencies and organizations with  multiple missions, values, and cultures. The 22 agencies whose powers  were absorbed or in part assumed by DHS came from eight different  departments (Agriculture, Commerce, Defense, Energy, Health and Human  Services, Justice, Transportation, and the Treasury) and two independent  offices (the Federal Emergency Management Agency and the General  Services Administration). In addition, DHS merged responsibilities from  former agencies to create some new agencies, such as Customs and  Border Protection. On March 1, 2003, DHS officially began operations as a  new department. DHS is among the largest federal government agencies,  with approximately 180,000 employees and an estimated budget of $43.6  billion for fiscal year 2007.
DHS’s mission is to lead the unified national effort to secure America,  prevent and deter terrorist attacks, protect against and respond to threats  and hazards to the nation, ensure safe and secure borders, welcome lawful  immigrants and visitors, and promote the free flow of commerce. Six of  the seven primary operational agencies, and the Operations Directorate of  the department, have identified the need to conduct activities in support of  the homeland security mission 24 hours a day, 7 days a week, 365 days a  year. The department’s July 2006 organizational chart, as illustrated in  figure 1, highlights these six agencies and the Operations Directorate.
section_title: Three DHS Components Sponsor Multi-Agency Operations Centers
paragraphs: The three components of DHS that have overall responsibility for the four  multi-agency 24/7/365 operations centers were created in response to the  events of September 11, 2001, and the subsequent establishment of DHS.  By merging portions of the Immigration and Naturalization Service and the  U.S. Department of Agriculture with elements of U.S. Customs, CBP was  created as part of DHS in 2003 to protect the nation’s borders in order to  prevent terrorists and terrorist weapons from entering or exiting the  United States while facilitating the flow of legitimate trade and travel. CBP  sponsors two 24/7/365 multi-agency operations centers: the Air and Marine  Operations Center and the National Targeting Center. TSA, established in  2001 (as part of the Department of Transportation), and incorporated into  DHS in 2003, protects the nation’s transportation systems to ensure  freedom of movement for people and commerce and sponsors the  Transportation Security Operations Center. DHS established the Office of  Operations Coordination (referred to as the Operations Directorate) after  a broad internal review in 2005. The Operations Directorate, which  sponsors the National Operations Center (includes the previous Homeland  Security Operations Center), is responsible for coordinating internal and  external operational issues throughout the department, conducting  incident management, and facilitating rapid staff planning and execution.  The three sponsoring components provide overall direction and  management for their respective centers.
section_title: Transformation Challenges and Practices That Can Help Enhance and Sustain Collaboration among Federal Agencies
paragraphs: We have previously reported that establishing the new DHS is an  enormous undertaking and the new department needs to build a  successful transformation that does the following: instills the organization  with important management principles; rapidly implements a phased-in  transition plan; leverages the new department and other agencies in  executing the national homeland security strategy; and builds  collaborative partnerships with federal, state, local, and private-sector  organizations.
DHS faces significant management and organizational transformation  challenges as it works to protect the nation from terrorism and  simultaneously establish itself. For these reasons, in January 2005, we  continued to designate the implementation and transformation of the  department as high risk. DHS’s Inspector General reported, in December  2004, that integrating DHS’s many separate components into a single,  effective, efficient, and economical department remains one of its biggest  challenges. We also reported in 2005 that agencies can enhance and  sustain their collaborative efforts by engaging in eight key management  practices:  defining and articulating a common outcome;    establishing mutually reinforcing or joint strategies;   identifying and addressing needs by leveraging resources;    agreeing on roles and responsibilities;    establishing compatible policies, procedures, and other means to  operate across agency boundaries;    developing mechanisms to monitor, evaluate, and report on results of  collaborative efforts;  reinforcing agency accountability for collaborative efforts through  agency plans and reports; and  reinforcing individual accountability for collaborative efforts through  performance management systems.
Although there is no commonly accepted definition for collaboration, in  our previous assessment of collaborative efforts among federal agencies  we defined it as any joint activity by two or more organizations that is  intended to produce more public value than could be produced when the  organizations act alone. This report focuses on the actions DHS and its  components have taken to make collaboration at multi-agency operations  centers as effective as possible. Joint activities take place at operations  centers where multiple components staff watchstander positions and  provide liaison, expertise, and access to information that would not  otherwise be on hand.  For this report, we selected the first seven of the  eight key practices listed above and assessed the first two key practices  together, thereby reducing our focus to six areas. We did not address the  eighth practice—reinforcing individual accountability for collaborative  efforts through performance management systems—because an in-depth  examination of component agencies’ performance management systems  was beyond the scope of this review.
section_title: While the Four Multi- Agency Operations Centers’ Missions and Products Differ, Functions and Customers Are Similar
paragraphs: The four multi-agency operations centers each have their own mission and  generate different products while performing similar functions and sharing  a number of customers. The missions of the AMOC, NTC, and TSOC are  tactical, including such activities as monitoring the nation’s airspace, the  movement of potential terrorists, and the passengers on commercial  flights, respectively. NOC-Watch’s mission is more strategic in that it  collects information gathered by the other multi-agency operations  centers and provides a national perspective on situational awareness.  The products of the four multi-agency operations centers reflect their  different missions and range from reports on suspect individuals traveling  on commercial flights to reports on suspicious private air and marine craft.  The multi-agency operations centers all share some common functions:  maintaining situational awareness and information sharing and  communications; coordinating internal operations and coordinating  among federal, state, local, tribal, and private-sector entities; and  managing incidents and making decisions. While all the multi-agency  operations centers share common customers, such as foreign, federal,  state, and local governments, the NOC-Watch has a larger number of  customers, given its role as a hub for overall situational awareness.
subsections: section_title: The Different Missions of the Four Multi-Agency Operations Centers Result in Different Products
paragraphs: Of the four multi-agency operations centers, three—AMOC, NTC and  TSOC—have tactical yet different missions and provide different products  that reflect their respective missions. The NOC-Watch has a more strategic  mission in providing an overall assessment of situational awareness.
subsections: section_title: Air and Marine Operations Center
paragraphs: The AMOC’s primary mission is to detect, sort, track, and facilitate the  interdiction of criminal entities throughout the Western Hemisphere, by  utilizing integrated air and marine forces, the latest technology, and  tactical intelligence. AMOC’s maintains day-to-day, around-the-clock  airspace situational awareness of the nation’s borders through  identification and detection of foreign and domestic threats. Created in  1988 by the U.S. Customs Service and located in Southern California, the  AMOC was established as the Air and Marine Operations Center on March  1, 2003. In addition to CBP and U.S. Coast Guard personnel, the AMOC is  staffed by the Federal Aviation Administration, and the Department of  Defense National Guard Bureau-Air National Guard, as well as a  representative of the government of Mexico.
AMOC staff use surveillance systems and databases to detect, identify, and  track potential threats, and to coordinate the apprehension of criminals  using law enforcement air, marine, and ground interdiction forces. Staff  utilize a surveillance system that includes an extensive network of over  200 ground-based radar and satellite tracking systems throughout North  America and the Caribbean. Staff also use numerous law enforcement and  Federal Aviation Administration databases to ensure that U.S. entry policy  and procedures are followed. Figure 2 shows the variety of information  and data sources employed by the AMOC.
Staff can conduct detailed research from a transnational and criminal  threat perspective to identify suspect persons, aircraft, and marine vessels.  AMOC staff use the resulting information to coordinate air and marine law  enforcement activity with various agencies such as the U.S. Coast Guard  and Immigration and Customs Enforcement; federal, state, and local law  enforcement; the Department of Defense; U.S. and foreign air traffic  control facilities; and foreign government coordination centers. The  AMOC Daily Intelligence Report focuses on the nation’s borders involving  suspicious private air and marine craft that are detected by radar,  eyewitnesses, or surveillance aircraft.
section_title: National Targeting Center
paragraphs: The NTC’s mission is to coordinate and support all agency field-level anti- terrorism activities by providing tactical targeting and analytical research,  and to be a single point of reference for all agency anti-terrorism efforts.  NTC monitors the movement of potential terrorists and prevents them and  any weapons of terror from entering and exiting our country through land,  air, and sea ports. Established on October 22, 2001, under the U.S.  Customs Service, the NTC, located in Northern Virginia, began 24/7/365  operations November 10, 2001. In addition to CBP personnel, the NTC is  staffed by the U.S. Coast Guard, Immigration and Customs Enforcement,  Federal Air Marshal Service, and the Transportation Security  Administration.
NTC staff use sophisticated information-gathering techniques and  analytical tools to look at data containing passenger and flight  information. These data include lists of known terrorists, foreign visitors  whose official authorization permitting entry into and travel within the  United States has elapsed (visa overstays), passport information, and  cargo listings to seek potential matches. Any inconsistency identified in  the data can trigger additional analysis. Figure 3 shows the variety of  sources of information and data sources employed by the NTC.
NTC works with a variety of federal stakeholders. For example, the NTC  works with the Federal Bureau of Investigation’s Terrorist Screening  Center to identify persons on the National Terrorist Watch List. NTC staff  also provide information from CBP’s Advance Passenger Information  System for TSA’s performance of a risk assessment on crewmembers on  international flights. Federal Air Marshals use information developed by  the NTC to determine if they need to put resources on a specific flight.  Using NTC capabilities to screen crew, vessel, and cargo, along with other  information, the U.S. Coast Guard determines vessels and crewmembers  that warrant further surveillance or assessment and can prioritize its  inspection efforts. NTC also helps in implementing the pilot Immigration  Advisory Program by reviewing advance information on travelers  forwarded by program teams to identify travelers at foreign airports that  may present a risk or warrant more intensive examination before they  board aircraft bound for the United States. (Passengers whose travel  documents are invalid, expired, or otherwise may have been altered,  counterfeited or obtained through fraud are advised, as is the airline,  before they leave their foreign location that they will likely be deemed  inadmissible and denied entry upon arrival in the United States.) The NTC  reports we reviewed primarily identified individuals at and between  domestic ports of entry and certain critical foreign ports.
section_title: Transportation Security Operations Center
paragraphs: The TSOC’s mission is to provide situational awareness and information  sharing in day-to-day coordination and incident management for all  transportation security related operations and issues worldwide by  monitoring, responding to, and investigating security incidents involving  all transportation sectors. TSOC maintains situational awareness of  passengers on commercial flights and works to minimize and mitigate  security vulnerabilities of the National Capital Region and critical  infrastructure such as commercial airports, rail stations, and pipelines.  The TSOC, located in Northern Virginia, began 24/7/365 operations in  August 2003. The National Capital Region Command Center constitutes  the multi-agency element of the TSOC because it is staffed by other DHS  component agencies—specifically the U.S. Secret Service and Customs  and Border Protection. In addition, representatives of organizations  outside of DHS such as the Federal Bureau of Investigation, District of  Columbia Metro Police, Federal Aviation Administration, U.S. Capitol  Police, and the U.S. Air Force (Northeast Air Defense Sector) provide  watchstanders for the TSOC.
As part of its mission, TSOC staff coordinate with federal, state, and local  homeland security entities to identify activities that might indicate a threat  to national security and isolate indications of impending attack while  assessing their potential impact. The TSOC also furnishes alerts and  reports to field security organizations while combining intelligence with  operational information across all modes of transportation. Last, it  monitors incidents and crises, including national special events such as  presidential inaugurations and the Super Bowl, for TSA headquarters and  makes recommendations to DHS leadership. Figure 4 shows the modes of  transportation monitored by the TSOC.
The TSOC reports we reviewed provided information on incidents across  all modes of transportation, including National Capital Region security  incidents, critical infrastructure, and individuals of interest related to the  No-Fly List.
section_title: National Operations Center- Interagency Watch
paragraphs: The NOC-Watch is designed to perform a more strategic mission than the  other three multi-agency operations centers. NOC-Watch acts as the  primary national-level coordination point for awareness of events that may  affect national security or safety. The center is responsible for combining  and sharing of information, communications, and operations coordination  pertaining to the prevention of terrorist attacks and domestic incident  management by facilitating information sharing with other federal, state,  local, tribal, and nongovernmental entities and by fusing law enforcement,  national intelligence, emergency response, and private-sector reporting.  Created as the Homeland Security Operations Center and located in  Northwest Washington, D.C., the center was established on February 19,  2003, and redesignated the National Operations Center on May 25, 2006.  The NOC-Watch is the 24/7/365 element of the center. In addition to staff  from the Operations Directorate, the NOC-Watch includes other DHS staff  from 20 components and offices such as representatives from the U.S.  Secret Service, Federal Protective Service, Federal Air Marshal Service,  Transportation Security Administration, Customs and Border Protection,  U.S. Coast Guard, Federal Emergency Management Agency, U.S. Border  Patrol, U.S. Citizenship and Immigration Services, National Biological  Surveillance Group, U.S. Computer Emergency Readiness Team, Domestic  Nuclear Detection Office, and other DHS directorates. The NOC-Watch  also includes representatives from 35 other federal, state, and local  agencies such as the Central Intelligence Agency; Defense Intelligence  Agency; National Security Agency; National Geospatial-Intelligence  Agency; Federal Bureau of Investigation; Department of Interior (U.S. Park  Police); Drug Enforcement Administration; Alcohol, Tobacco, Firearms  and Explosives; Virginia State Police; Fairfax County Police; and the New  York, Boston, and Los Angeles police departments; and a number of other  organizations.
NOC-Watch staff use information gathered and communicated by the three  tactical centers; other DHS operation centers; other federal, state, and  local entities; and a wide variety of other information sources to provide  overall national situational awareness related to homeland security. The  NOC-Watch reports, via the DHS Director of Operations, to the Secretary  of Homeland Security and coordinates directly with the White House and  focuses on two goals: (1) the detection, prevention, and deterrence of  terrorist attacks and (2) domestic incident management during crises and  disasters or national special events. Figure 5 shows some of the sources of  information and agencies with which that information is shared.
Situation reports prepared by the Operations Directorate’s NOC-Watch  that we reviewed contained information reported from other DHS  subcomponents and operations centers such as the TSOC, NTC, and  AMOC, as well as external intelligence and law enforcement agencies, and  the private sector. The NOC-Watch also prepares a Homeland Security  Operations Morning Brief that provides information to federal, state, and  local law enforcement agencies on the national picture at the sensitive but  unclassified level.
section_title: The Four Multi-Agency Operations Centers Share Common Functions and Customers
paragraphs: All four centers conduct common functions to maintain situational  awareness and communicate and coordinate with other federal, state, and  local governments, as well as private-sector entities. The centers do so to  support both the mission of the sponsoring component organization and  the underlying homeland security mission of DHS. On the basis of our  discussions with center officials and our assessment of documents they  provided, we summarized these functions and found that all DHS multi- agency operations centers perform 9 of 11 functions identified in table 2.  (According to TSOC officials, the TSOC does not coordinate with foreign  governments, and NTC and TSOC officials said they do not exercise  command and control functions.)
Multi-agency operations centers’ customers include federal, state, and  local governments and private-sector entities, along with foreign  governments. The NOC-Watch has a larger number of overall customers;  as the national-level multi-agency hub for situational awareness and a  common operating picture, the NOC-Watch provides information to a  wider range of government customers, including federal executive  leadership, and intelligence and law enforcement agencies at the federal,  state, and local level.
section_title: Opportunities Exist to Enhance Collaboration at DHS’s Four Multi- Agency Operations Centers
paragraphs: DHS has leveraged its resources—one key collaborative practice—by  having staff from multiple agencies work together at the four operations  centers. However, opportunities exist to further implement this and the  other relevant practices that our previous work has identified as important  to enhancing and sustaining collaboration among federal agencies. For  example, not all of the components responsible for managing the  operation centers had    established goals to define and articulate a common outcome and  mutually reinforcing or joint strategies for collaboration (related to two  of our key practices);    assessed staffing needs to leverage resources;    defined roles and responsibilities of watchstanders from agencies other  than the managing one;    applied standards, policies, and procedures for DHS’s information  sharing network to provide a means to operate across agency  boundaries;    prepared mechanisms to monitor, evaluate, and report on results of the  operations centers to reinforce for collaborative efforts; and  reinforced agency accountability for collaboration efforts through  agency plans and reports.
The Operations Directorate, established in November 2005 to improve  operational efficiency and coordination, provides DHS with an opportunity  to more consistently implement these practices that can enhance and  sustain collaboration among federal agencies at multi-agency operations  centers.
subsections: section_title: The Four Multi-Agency Operations Centers Lack Documented Goals and Joint Strategies
paragraphs: The three DHS components responsible for the four multi-agency centers  have not developed and documented common goals or joint strategies for  their operation that our work has shown could enhance collaboration  among the agencies. Officials at the four multi-agency operations centers  we visited said they did consider formally documenting working  agreements but concluded it was not essential since all of the agencies  involved were part of DHS. Officials from the NOC said that the lack of  formal agreements is a reflection of the speed with which the center was  established and the inherent flexibility offered to DHS agencies in order to  get them to staff the operation center positions. Nonetheless, as the DHS  Office of Inspector General has reported, memorandums of understanding  are valuable tools for establishing protocols for managing a national-level  program between two organizations. Within DHS, external and internal  memorandums of agreement and other interagency joint operating plans  are often used to document common organizational goals and how  agencies will work together. For example:    The National Interdiction Command and Control Plan among the  Department of Defense, Office of National Drug Control Policy, and the  AMOC highlights an agreement between a DHS component and other  federal agencies.
The Joint Field Office Activation and Operations Interagency  Integrated Standard Operating Procedure describes how a temporary  federal multi-agency coordination center should be established locally  to facilitate field-level domestic incident management activities related  to prevention, preparedness, response, and recovery and addresses the  roles and responsibilities of multiple DHS components such as the  Federal Emergency Management Agency and Immigration and  Customs Enforcement and other federal agencies such as the Federal  Bureau of Investigation.
Guidelines Governing Interaction Between ICE’s Office of  Investigations and CBP’s Office of Border Patrol documents a  memorandum of understanding between the Office of Investigations at  Immigration and Customs Enforcement and CBP’s Border Patrol,  entered into in November 2004, that governs the interaction between  the two components and formalizes roles and responsibilities in order  to further enhance information sharing.
Thus, although some DHS components have established a variety of  internal and external working agreements, memorandums, and in the case  of the Joint Field Offices, standard operating procedures, DHS’s  Operations Directorate, which is responsible for coordinating operations,  has not provided guidance on how and when such agreements should be  used to improve collaboration among the sponsoring and participating  components at the operations centers we reviewed. Nor have any of these  centers documented goals or joint strategies using these or other types of  agreements.
Our previous work has shown that memorandums of agreement or  understanding and strategic plans can both be used to establish common  goals and define joint strategies for how agencies will work together.  According to our work, collaborative efforts are further enhanced when  staff working across agency lines define and articulate a common federal  outcome, or purpose, that is consistent with their respective agency goals  and missions. Joint strategies or mutual agreements also contribute to  another key area when they are used as a vehicle for identifying and  defining more specific expectations of the roles and responsibilities of  staff provided by collaborating agencies.
section_title: The Four Multi-Agency Operations Centers Are at Varying Stages of Assessing Staffing Needs
paragraphs: The extent to which officials responsible for managing the four multi- agency operations centers had conducted needs assessments to determine  the staffing requirements of each center varied. For example, CBP officials  conducted an evaluation in June 2005 that addressed AMOC capabilities  and continuing staffing needs related to AMOC personnel, but did not  address the need for, or responsibilities of, U.S. Coast Guard staff at the  center. AMOC officials did cite a requirement for additional staff from the  U.S. Coast Guard, as well as a requirement for an Immigration and  Customs Enforcement position in a subsequent strategic planning effort  (although these requirements had not been filled). However, there was not  a specific assessment supporting the need for these staff positions. NTC  officials had not conducted a staffing needs assessment but said they plan  to conduct an assessment based upon current targeting programs, the  scheduled expansion of existing programs, and the onset of additional  CBP targeting programs. They said they plan to include data on the volume  of telephone calls handled by the center and the number of information  requests completed by the NTC in support of CBP targeting and  operations, and they expect to complete the assessment in October 2006.  TSOC and NOC-Watch officials said they had not documented a needs  analysis for staff from agencies other than the sponsoring agency. Instead,  they viewed the cross-agency staffing requirement as a historical edict  based on a general assumption that other agency staff expertise was  needed to fulfill the mission of their operations center.
Our work has shown that collaborating agencies should identify the  resources, including human resources, needed to initiate or sustain their  collaborative effort and take steps to leverage those resources. Because  each agency, or component, has different strengths and limitations,  assessing these varying levels allows them collectively to obtain additional  resources otherwise unavailable individually. Formal assessment of the  need for all participating agencies’ staff to perform specific functions  allows for the leveraging of resources to more effectively meet the  operational needs of each agency or component.
section_title: Three of the Four Multi- Agency Operations Centers Have Not Established a Definition of Watchstander Roles and Responsibilities for All Agencies at Each Center
paragraphs: While three of the four multi-agency operations centers had developed  descriptions for the watchstander position staffed by their own agency,  only one center—the AMOC—had developed a position description for  staff assigned to the center from another DHS agency. At the AMOC,  center officials require that Coast Guard staff meet a standardized set of  requirements for radar watchstanders. The other centers relied on the  components providing staff to define their watchstanders’ roles and  responsibilities. Lack of a consistent definition for the watchstander  position may lead to people at the same center in the same role performing  the same responsibilities differently or not at all.
Our work has shown that defining roles and responsibilities both enhances  and sustains collaboration among federal agencies. Because of the  potentially critical, time-sensitive need for decisive action at 24/7/365  operations centers, it is important that the roles and responsibilities of  watchstanders are described and understood stood by both the staff and  the officials responsible for managing the operations centers. Further, a  definition of the watchstander role and responsibilities is important for  supporting agency officials who must make staffing decisions about  assigning qualified and knowledgeable personnel to the centers. Finally,  internal controls standards require that management and employees  establish a positive control environment as a foundation for strong  organizational internal controls. According to the standard, one activity  that agency officials may consider implementing as part of the control  environment is to identify, define, and provide formal, up-to-date job  descriptions or other means of identifying and defining job-specific tasks.
section_title: DHS Did Not Provide All Multi-Agency Operations Centers with Standards, Policies, and Procedures for Use of Its Information Sharing Network to Operate across Agency Boundaries
paragraphs: To collaborate by sharing information through DHS’s primary information  sharing system, the Homeland Security Information Network (HSIN),  agencies participating in multi-agency operations centers need to be  connected to the network and have the guidance that enables its use. In  the course of our work, we learned that CBP’s National Targeting Center  could not collaborate with other users of HSIN because the system was  not connected for NTC watchstanders. Other concerns about the use of  HSIN to enhance coordination and collaboration have also been identified  by the DHS Inspector General. According to the Inspector General, DHS  did not provide adequate user guidance, including clear information  sharing processes, training, and reference materials needed to effectively  implement HSIN. The report noted that in the absence of clear DHS  direction, users were unsure of how to use the system. Though DHS  officials said other networks such as the Secret Internet Protocol Router  Network and the Joint Worldwide Intelligence Communications System  are primarily used for coordination of intelligence analysis, the  connectivity problem with the primary DHS-wide information sharing  system, HSIN, remained unresolved as of September 2006.
Our work has shown that to facilitate collaboration, agencies need to  address the compatibility of standards, policies, procedures, and data  systems used in the collaborative effort. Furthermore, as agencies bring  diverse cultures to the collaborative effort, it is important to address these  differences to enable a cohesive working relationship and to create the  mutual trust required to enhance and sustain the collaborative effort.  Frequent communication among collaborating agencies is another means  to facilitate working across agency boundaries and prevent  misunderstanding. The lack of standards, policies, and procedures for use  of HSIN at DHS operations centers could limit the frequency and  effectiveness of communications among the centers.
section_title: Three of Four Multi- Agency Operations Centers Have Not Developed Methods to Monitor, Evaluate, and Report Results of Joint Efforts
paragraphs: With the exception of AMOC, the multi-agency centers have not developed  methods to monitor, evaluate, and report the results of joint efforts. For  example, the Office of Management and Budget’s assessment of the NOC- Watch for 2005 determined that center officials had not established  effective annual or long-term performance goals. Nor were performance  measures or other mechanisms in place to monitor and evaluate the joint  efforts of multiple DHS agencies at the TSOC and NTC. In response to a  report by the DHS Office of Inspector General in March 2004 that found  the AMOC did not have organizational performance measures and  individual performance standards to assess AMOC’s effectiveness and  productivity, AMOC officials reported to the Inspector General that they  began collecting data in January 2004 on a daily basis to measure  productivity for the overall operations center as well as individual  watchstanders, including U.S. Coast Guard representatives.
Our work has shown that developing means to monitor, evaluate, and  report areas of improvement allow agencies to enhance collaboration.  Developing performance measures and mechanisms to monitor and  evaluate the contributions can help management, key decision makers,  and both stakeholders and customers obtain feedback through internal  reports in order to improve operational effectiveness and policy.  Developing goals and providing performance results can also help  reinforce accountability through joint planning and reporting of  collaborative efforts.
section_title: The Four Multi-Agency Operations Centers Are at Various Stages of Reinforcing Accountability for Collaborative Efforts through Joint Agency Planning and Reporting
paragraphs: Neither DHS nor the component agencies responsible for managing the  four multi-agency operations centers consistently discuss or include a  description of the contribution of collaborative efforts of the multi-agency  operations centers in their strategic or annual performance plans and  reports. The most recent DHS strategic plan, issued in 2004, neither  included a discussion of performance goals nor addressed the joint  operations of the multi-agency centers. The plan reported only that DHS  “will provide integrated logistical support to ensure a rapid and effective  response and coordinate among Department of Homeland Security and  other federal, state, and local operations centers consistent with national  incident command protocols.” CBP’s 2005 annual report on the operations  of the NTC does, however, include a section dedicated to the contributions  of the external liaisons in describing the roles and responsibilities of other  DHS agency personnel including the Federal Air Marshal Service,  Immigration and Customs Enforcement, and the U.S. Coast Guard, and the  accomplishments they have made in the center’s operations. In addition,  the AMOC strategic plan for 2005 generally discussed the importance of  collaboration with other component agencies and included a goal to  strengthen component agency partnerships to maximize homeland  security strategies. Reports of the components responsible for managing  the other centers do not address the roles and contributions of other  supporting agencies in accomplishing the centers’ missions. DHS agencies  responsible for providing staff to support watchstander positions for  multi-agency operations centers managed by other agencies also do not  address their participation in the operations of the operations center in  strategic plans or performance reports. In general, managing and  supporting agencies that do mention the operations centers do not include  any discussion of the relationship between the participating agencies’  missions or strategies and those of the centers.
Our work has shown that federal agencies can use their strategic and  annual performance plans as tools to drive collaboration with other  agencies and partners and establish complementary goals and strategies  for achieving results. These performance plans can also be used to ensure  that goals are consistent and, if possible, mutually reinforcing.  Accountability is also reinforced when strategic and annual performance  plans help to align agency policy with collaborative goals. A public  accounting through published strategic and annual performance plans and  reports makes agencies answerable for collaboration.
section_title: DHS’s Operations Directorate Has an Opportunity to Help Ensure That Key Practices for Collaboration Are Implemented at the Multi- Agency Operations Centers
paragraphs: DHS established a new Office of Operations Coordination in November  2005 (referred to as the Operations Directorate) to increase its ability to  prepare for, prevent, and respond to terrorist attacks and other  emergencies and improve coordination and efficiency of operations. In  responding to a draft of this report, DHS cited a number of efforts that the  new directorate plans to take to fulfill this leadership role.  Among other  things, DHS said it plans to conduct an independent study, initiated in  September 2006, to leverage technical and analytical expertise to support  expanding the capabilities of the Operations Directorate.  In addition, DHS  said it plans to move elements of the National Operations Center to the  Transportation Security Operations Center in 2007 and, ultimately to  colocate the DHS headquarters and all the DHS component headquarters  along with their respective staffs and operations centers at one location.  DHS also cited the development of a new working group that is developing  a national command and coordination capability.  While we agree that  these leadership efforts proposed by the Operations Directorate could  further enhance collaboration among DHS’s component agencies, because  DHS officials did not provide any information or documentation of these  efforts in response to our requests during the course of the review, we  were unable to determine the extent to which these efforts are likely to  enhance and sustain departmental collaboration.  Nonetheless, further  departmental focus on the key practices we have identified could enhance  collaboration among the component agencies. For example, at the time of  our review, the directorate had not taken steps to gather information on  the resources available at each center. The director’s office did not have  ready access to information such as centers’ budgets or other financial  information needed for reporting across the components, the number of  staff employed at the multi-agency centers, or the number and type of  operations centers managed by the various components. After being  directed to the components for budget and staffing information, we found  that the managing components of the multi-agency operations centers also  did not have ready access to up-to-date information on the number of staff  the centers employed. Such information could be useful to the  directorate’s efforts to develop a national command and coordination  capability and further enhance collaboration among the components with  multi-agency operations centers. Directorate officials said that the  Operations Directorate had not assumed its full range of responsibilities  due to not being fully staffed until March 2006 and because of the revisions  to the National Response Plan formalized in May 2006. In responding to a  draft of this report, DHS said that the Operations Directorate does not  have the authority to direct or exercise control over other components’  operations centers with respect to administration and support, including  organization, staffing, control of resources and equipment, personnel  management, logistics, and training. Nonetheless, while the Operations  Directorate lacks authority to direct the actions of the other components’  operations centers and obtaining compatible data may be difficult since  the reporting systems of several centers were in place prior to the creation  of DHS, without compatible staffing and financial data Operations  Directorate leadership officials are hampered in their ability to understand  and compare the relative personnel and operating costs of the 24/7/365  operations centers and use such information to promote the expected  unity of effort within the department.
Enhanced leadership from the Operations Directorate to support  consistent reporting of operations centers’ budgets and staffing could also  support collaborative actions in two of the previously mentioned key  areas: assessing staffing needs to leverage resources, and applying  standards, policies, and procedures to operate across agency boundaries.  In the absence of leadership to support these and other collaborative  efforts, DHS officials have not yet taken full advantage of an opportunity  to meet the directorate’s responsibilities.
section_title: Conclusions
paragraphs: The establishment of the Operations Directorate with the express intent of  enhancing collaboration and coordination among the department’s  operational components provides an opportunity to implement practices  that could enhance collaboration among DHS agencies working together  at each multi-agency 24/7/365 operations center. Having staff from multiple  agencies work together is a way of leveraging resources, one key practice  for enhancing collaboration. However, those resources may not be used to  their full potential if other steps to enhance collaboration are not taken,  and the Operations Directorate could provide guidance to help ensure that  the sponsors of the operations centers take the appropriate steps. There  are multi-agency operations centers that lack common goals and joint  strategies; clearly defined roles and responsibilities; compatible standards,  policies, and procedures for information networking; consistent staffing  assessments; prepared mechanisms to monitor, evaluate, and report on the  results of collaborative efforts; and reinforced agency accountability  through agency plans and reports. Our previous work has shown that  these are all critical components in enhancing collaboration among federal  agencies. Given that the collaboration in multi-agency operations centers  focuses on gathering and disseminating information on real-time  situational awareness related to disasters and possible terrorist activity, it  is important that the staff at the centers achieve the most effective  collaboration possible.
section_title: Recommendations for Executive Action
paragraphs: To provide a setting for more effective collaboration among the staff at  each multi-agency 24/7/365 operations center, we recommend that the  Secretary of the Department of Homeland Security charge the Director of  the Operations Directorate with developing and providing guidance and  helping to ensure the agencies that sponsor the centers take the following  six actions:    define common goals and joint strategies;    clarify the roles and responsibilities for watchstanders;  implement compatible standards, policies, and procedures for using  DHS’s information network to provide a means of operating across  agency boundaries;    conduct staffing needs assessments;  implement mechanisms to monitor, evaluate, and report on the results  of collaborative efforts; and    address collaborative efforts at the four multi-agency operations  centers in plans and reports on the level of each operation center’s  managing agency.
section_title: Agency Comments and Our Evaluation
paragraphs: On October 16, 2006, DHS provided written comments on a draft of this  report (see app. III.)  DHS agreed with the six recommended actions to  enhance collaboration at the DHS multi-agency operations centers and  said it planned to take action to implement the practices.  In the draft  report, we said that the Operations Directorate had not yet taken actions  to fulfill its leadership role and that a lack of leadership by the Operations  Directorate to support consistent reporting of operations centers’ budgets  and staffing limits collaborative actions. DHS did not agree that leadership  provided by the Operation Directorate to support collaboration is lacking  and provided a number of examples of leadership efforts.  Among other  things, DHS noted plans to conduct an independent study, initiated in  September 2006, to leverage technical and analytical expertise to support  expanding the capabilities of the Operations Directorate.  In addition, DHS  said it plans to move elements of the National Operations Center to the  Transportation Security Operations Center in 2007 and, ultimately to  colocate the DHS headquarters and all the DHS component headquarters  along with their respective staffs and operations centers at one location.   We identified the planned actions in the report and agree that these  leadership efforts by the Operations Directorate have the potential to  further enhance collaboration among DHS’s component agencies, along  with the key practices suggested by our efforts to review collaboration  among agencies across the federal government. However, because  Operations Directorate officials did not provide any information or  documentation of these efforts in response to our requests during the  course of the review, we were unable to determine the extent to which  these efforts are likely to enhance and sustain departmental collaboration.
In addition, DHS officials cited what they considered to be misconceptions  expressed in the draft report.  They said that the Operations Directorate  does not have the administrative, budgetary, programmatic, or command  and control authority to direct or exercise control over other component’s  operations centers.  They also said that our draft incorrectly reported that  the National Operations Center replaced the Homeland Security  Operations Center.  Although it was not our intent to imply that the  Operations Directorate has administrative, budgetary, programmatic, or  command and control authority to direct or exercise control over other  component’s operations centers, we added a clarifying reference to  address DHS’s concern.  Finally, although we reported that the new  National Operations Center includes (rather than replaced) the previous  Homeland Security Operations Center, we also added a footnote to further  clarify that the scope of responsibilities of the new National Operations  Center is greater than that of the Homeland Security Operations Center.
We are sending copies of this report to the Senate Committee on  Homeland Security and Governmental Affairs, the Permanent  Subcommittee on Investigations, the Secretary of Homeland Security; the  Assistant Secretary of the Transportation Security Administration, the  Commissioner of Customs and Border Protection, and interested  congressional committees. We will also make copies available to others on  request. In addition, the report will be available on GAO’s Web site at  http://www.gao.gov.
If you or your staff have any questions about this report, or wish to discuss  the matter further, please contact me at (202) 512-8777 or  Jonespl@gao.gov. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  Key contributors to this report are listed in appendix IV.
section_title: Appendix I: Missions of 24/7/365 DHS Centers Staffed by One DHS Component
subsections: section_title: # Center
paragraphs: To monitor cyber security, respond to incidents, and direct  communications.
To assist in the initiation, coordination, restoration, and  reconstitution of national security and emergency  preparedness telecommunications services or facilities  under all conditions, crises, or emergencies.
To provide warning and intelligence analysis to inform field  operators, industry, and TSA leadership.
To provide support to for scheduling, law enforcement  situations, crisis management, and safety and security-  related matters.
To provide information on significant incidents from field  and sector offices, providing situational awareness to the  Commissioner and senior CBP management.
6.  Caribbean Air Marine Operations  Center (Regional Operations)
To utilize integrated air and marine forces, technology, and  tactical intelligence to detect, sort, track, and facilitate the  interdiction of criminal entities throughout the Caribbean  area.
7.  National Airspace Security  Operations Center (Regional  Operations)
To utilize integrated air forces, technology, and tactical  intelligence to maintain air domain awareness, and detect,  sort, track, and facilitate the interception of intruder aircraft  throughout the National Capital Region.
To monitor radio communications among CBP personnel  for officer safety purposes, and to coordinate tactical  communications and analytical investigative support to  various DHS and other law enforcement agencies to  support homeland security.
To provide senior management with daily reports and  coordination on all significant incidents, events, and  matters that have an impact on the mission of ICE and  DHS.
To provide timely, effective classified intelligence support  to ICE headquarters and field personnel by serving as a  clearinghouse for the screening, evaluation, processing,  exploitation, dissemination, and coordination of intelligence  information.
To provide timely immigration status and identification  information to federal, state, and local law enforcement  agencies on aliens suspected, arrested, or convicted of  criminal activity.
12.  Federal Protective Service Mega- Center System (4 regional centers)
To provide alarm monitoring and dispatch services to all  federally owned and leased buildings.
To maintain national situational awareness and to monitor  emerging incidents or potential incidents with possible  operational consequences (becomes multi-agency under  incident surge conditions).
To facilitate, in coordination with the NOC, the distribution  of warnings, alerts, and bulletins to the entire emergency  management community using a variety of  communications systems.
15.  Mobile Emergency Response  Support Operations Centers   (5 regional centers)
To serve as the emergency operations center for FEMA  regions and associated state operations centers providing  time-sensitive information flow affecting federal  involvement and provide a deployed operations center  platform using assigned mobile assets during all  catastrophic events.
To provide command, control, communication, and  monitoring for ensuring the security of the White House  complex and surrounding grounds.
To coordinate communications for the receipt,  coordination, and dissemination of protective intelligence  information and activities that require immediate action in  support of protection assignments.  Also provides “as  needed” information and coordination support for the  service.  18.  U.S. Coast Guard Command Center To gather, coordinate, and disseminate information as the  direct representative of the Coast Guard Commandant and  the National Response Center.  Serves as the primary  communications link of priority operational and  administrative matters between USCG field units, District  and Area Commanders, senior Coast Guard officials, DHS  officials, the White House, other federal agencies, state  and local officials, and foreign governments.
19.  Intelligence Coordination Center  (includes three 24/7/365 watch  locations with one, the Intel Plot,  colocated at U.S. Coast Guard  Command Center)
To function as the national-level coordinator for collection,  analysis, production, and dissemination of Coast Guard  intelligence.  Provides all-source, tailored, and integrated  intelligence and intelligence services to DHS, Coast  Guard, Commandant/staff, intelligence community,  combatant commanders, and other services and agencies.
The Intel Plot provides predictive and comprehensive  intelligence support to priority requirements of the  Commandant and senior staff at Coast Guard  headquarters.
To serve as the single federal point of contact for all  pollution incident reporting.  Also serves as a  communications center in receiving, evaluating, and  relaying information to predesignated federal responders,  and advises FEMA of potential major disaster situations.
21.  Regional Command Centers (46)
Area Command Centers (2)
To serve as points of coordination at various organizational  levels for operational command, control, communications,  intelligence, and analysis.
District Command Centers (9)
Sector Command Centers (35)
section_title: Appendix II: Objectives, Scope, and Methodology
paragraphs: Our overall objective was to assess the collaboration among the four  multi-agency DHS operations centers. The key questions addressed were  as follows:  1.   What are the missions, functions, and products of the multi-agency  24/7/365 DHS operations centers and who are their customers?  2.  To what extent has DHS implemented key practices for enhancing and  sustaining collaboration at these multi-agency centers?
To answer our first objective, we obtained and reviewed information on  the missions and functions of all 24/7/365 operations centers in DHS. We  visited centers managed by the Operations Directorate, U.S. Customs and  Border Protection, the Federal Emergency Management Agency, the  Transportation Security Administration, the U.S. Coast Guard, and the  Secret Service to observe their operations, interview officials responsible  for managing the centers, and identify centers that employed staff from  multiple DHS agencies. We identified four centers that employed staff  from multiple DHS component agencies: the Air and Marine Operations  Center, the National Targeting Center, the Transportation Security  Operations Center, and the National Operations Center-Interagency  Watch. We gathered and analyzed information regarding the products the  multi-agency centers developed on a regular basis and the primary  customers served by the centers.
To answer our second objective, we met with responsible officials of the  NOC-Watch and the acting Director of the Operations Directorate to  discuss the roles and responsibilities of the new organization established  as a result of the department’s Second Stage Review. We discussed the  transition, current operations, and policy and procedures put in place by  the Operations Directorate since the reorganization. We also met with  officials from TSA, USCG, CBP, ICE, and the Operations Directorate to  discuss how staff are assigned by these agencies to the four multi-agency  operations centers. We spoke with watchstanders assigned to several of  the centers from other DHS component agencies to discuss their roles and  responsibilities at the centers, and the overall mission of the centers to  which they had been assigned. We reviewed planning and policy  documents including DHS’s strategic plans and performance and  accountability reports as well as our prior reports and reports from DHS’s  Inspector General that addressed DHS management issues. For the four  national operations centers we identified as multi-agency DHS centers, we  also reviewed strategic plans, standard operating procedures, and annual  reports and performance and accountability reports. We assessed DHS’s  efforts and actions taken by the Operations Directorate to encourage  coordination among the multi-agency centers and to promote  collaboration among the staff representing DHS agencies at the centers to  determine the extent that they reflected consideration of key practices  that our previous work has shown can enhance and sustain a collaborative  relationship among federal agencies. Eight practices we identified to  enhance and sustain collaboration are identified below:    defining and articulating a common outcome;    establishing mutually reinforcing or joint strategies;   identifying and addressing needs by leveraging resources;    agreeing on roles and responsibilities;    establishing compatible policies, procedures, and other means to  operate across agency boundaries;    developing mechanisms to monitor, evaluate, and report on results;  reinforcing agency accountability for collaboration efforts through  agency plans and reports; and,  reinforcing individual accountability for collaborative efforts through  performance management systems.
For the purposes of this review, we selected the first seven of the eight  practices. We combined our discussion of the implementation of the first  two practices—defining and articulating a common outcome and  establishing mutually reinforcing or joint strategies. We did not address  the eighth practice—reinforcing individual accountability for collaborative  efforts through performance management systems—because an in-depth  examination of component agencies’ performance management systems  was beyond the scope of this review. We selected examples that, in our  best judgment, clearly illustrated and strongly supported the need for  improvement in specific areas where the key practices could be  implemented.
We conducted our work from October 2005 through September 2006 in  accordance with generally accepted government auditing standards.
section_title: Appendix III: Comments from the Department of Homeland Security
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, Christopher Keisling, Kathleen  Ebert, Dorian Dunbar, Scott Behen, Keith Wandtke, Amanda Miller,  Christine Davis, and Willie Commons III made key contributions to this  report. Additional assistance was provided by Katherine Davis.