Cyber Analysis and Warning: DHS Faces Challenges in Establishing a Comprehensive National Capability
Published: 2008-07-31T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Cyber analysis and warning capabilities are critical to thwarting computer-based (cyber) threats and attacks. The Department of Homeland Security (DHS) established the United States Computer Emergency Readiness Team (US-CERT) to, among other things, coordinate the nation's efforts to prepare for, prevent, and respond to cyber threats to systems and communications networks. GAO's objectives were to (1) identify key attributes of cyber analysis and warning capabilities, (2) compare these attributes with US-CERT's current capabilities to identify whether there are gaps, and (3) identify US-CERT's challenges to developing and implementing key attributes and a successful national cyber analysis and warning capability. To address these objectives, GAO identified and analyzed related documents, observed operations at numerous entities, and interviewed responsible officials and experts.
section_title: What GAO Found
paragraphs: Cyber analysis and warning capabilities include (1) monitoring network activity to detect anomalies, (2) analyzing information and investigating anomalies to determine whether they are threats, (3) warning appropriate officials with timely and actionable threat and mitigation information, and (4) responding to the threat. GAO identified 15 key attributes associated with these capabilities. While US-CERT's cyber analysis and warning capabilities include aspects of each of the key attributes, they do not fully incorporate all of them. For example, as part of its monitoring, US-CERT obtains information from numerous external information sources; however, it has not established a baseline of our nation's critical network assets and operations. In addition, while it investigates if identified anomalies constitute actual cyber threats or attacks as part of its analysis, it does not integrate its work into predictive analyses. Further, it provides warnings by developing and distributing a wide array of notifications; however, these notifications are not consistently actionable or timely. US-CERT faces a number of newly identified and ongoing challenges that impede it from fully incorporating the key attributes and thus being able to coordinate the national efforts to prepare for, prevent, and respond to cyber threats. The newly identified challenge is creating warnings that are consistently actionable and timely. Ongoing challenges that GAO previously identified, and made recommendations to address, include employing predictive analysis and operating without organizational stability and leadership within DHS, including possible overlapping roles and responsibilities. Until US-CERT addresses these challenges and fully incorporates all key attributes, it will not have the full complement of cyber analysis and warning capabilities essential to effectively performing its national mission.

section_title: Letter
paragraphs: The rapid increase in computer connectivity has revolutionized the way  The rapid increase in computer connectivity has revolutionized the way  that our government, our nation, and much of the world communicate and  that our government, our nation, and much of the world communicate and  conduct business. While the benefits have been enormous, this widespread  conduct business. While the benefits have been enormous, this widespread  interconnectivity also poses significant risks to our nation’s computer- interconnectivity also poses significant risks to our nation’s computer- reliant critical operations. Establishing analytical and warning capabilities  reliant critical operations. Establishing analytical and warning capabilities  is essential to thwarting computer-based, or cyber, threats and attacks.  is essential to thwarting computer-based, or cyber, threats and attacks.  Cyber analysis and warning capabilities include (1) monitoring network  Cyber analysis and warning capabilities include (1) monitoring network  activity to detect anomalies, (2) analyzing information and investigating  activity to detect anomalies, (2) analyzing information and investigating  anomalies to determine whether they are threats, (3) warning appropriate  anomalies to determine whether they are threats, (3) warning appropriate  officials with timely and actionable threat and mitigation information, and  officials with timely and actionable threat and mitigation information, and  (4) responding to the threat.  (4) responding to the threat.
Federal law and policy direct the Department of Homeland Security (DHS)  Federal law and policy direct the Department of Homeland Security (DHS)  to establish such capabilities for our nation. To fulfill this requirement, the  to establish such capabilities for our nation. To fulfill this requirement, the  department established the United States Computer Emergency Readiness  department established the United States Computer Emergency Readiness  Team (US-CERT) to develop and implement these capabilities and, in  Team (US-CERT) to develop and implement these capabilities and, in  doing so, coordinate the nation’s efforts to prepare for, prevent, and  doing so, coordinate the nation’s efforts to prepare for, prevent, and  respond to cyber threats and attacks.  respond to cyber threats and attacks.
Our objectives were to (1) identify key attributes of cyber analysis and  Our objectives were to (1) identify key attributes of cyber analysis and  warning capabilities, (2) compare these attributes with US-CERT’s current  warning capabilities, (2) compare these attributes with US-CERT’s current  analysis and warning capabilities to identify whether there are gaps, and  analysis and warning capabilities to identify whether there are gaps, and  (3) identify US-CERT’s challenges to developing and implementing key  (3) identify US-CERT’s challenges to developing and implementing key  attributes and a successful national cyber analysis and warning capability.  attributes and a successful national cyber analysis and warning capability.  To identify key attributes, we identified and analyzed relevant laws,  To identify key attributes, we identified and analyzed relevant laws,  strategies, policies, reports, and studies; observed cyber analysis and  strategies, policies, reports, and studies; observed cyber analysis and  warning operations at numerous entities; and interviewed responsible  warning operations at numerous entities; and interviewed responsible  officials and experts from federal and nonfederal entities. To determine  US-CERT’s current capabilities and related challenges, we analyzed DHS’s  policies, procedures, and program plans and interviewed relevant officials.  Appendix I provides further details on our objectives, scope, and  methodology.
We conducted this performance audit from June 2007 to July 2008 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: Increasing computer interconnectivity—most notably growth in the use of  the Internet—has revolutionized the way that our government, our nation,  and much of the world communicate and conduct business. While the  benefits have been enormous, they are accompanied by significant risks to  the nation’s computer systems and to the critical operations and  infrastructures that those systems support.
subsections: section_title: Cyber Threats and Incidents Adversely Affect the Nation’s Critical Infrastructure
paragraphs: Different types of cyber threats from numerous sources may adversely  affect computers, software, a network, an agency’s operations, an  industry, or the Internet itself. Cyber threats can be unintentional or  intentional. Unintentional threats can be caused by software upgrades or  maintenance procedures that inadvertently disrupt systems. Intentional  threats include both targeted and untargeted attacks. A targeted attack  occurs when a group or individual specifically attacks a cyber asset. An  untargeted attack occurs when the intended target of the attack is  uncertain, such as when a virus, worm, or malware is released on the  Internet with no specific target.
subsections: section_title: Threats to the Nation’s Critical Infrastructure Are Proliferating
paragraphs: There is increasing concern among both government officials and industry  experts regarding the potential for a cyber attack on the national critical  infrastructure, including the infrastructure’s control systems. The  Department of Defense (DOD) and the Federal Bureau of Investigation,  among others, have identified multiple sources of threats to our nation’s  critical infrastructure, including foreign nation states engaged in  information warfare, domestic criminals, hackers, virus writers, and  disgruntled employees working within an organization. In addition, there  is concern about the growing vulnerabilities to our nation as the design,  manufacture, and service of information technology have moved  overseas. For example, according to media reports, technology has been  shipped to the United States from foreign countries with viruses on the  storage devices. Further, U.S. authorities are concerned about the  prospect of combined physical and cyber attacks, which could have  devastating consequences. For example, a cyber attack could disable a  security system in order to facilitate a physical attack. Table 2 lists sources  of threats that have been identified by the U.S. intelligence community and  others.
The nation’s critical infrastructure operates in an environment of  increasing and dynamic threats, and adversaries are becoming more agile  and sophisticated. Terrorists, transnational criminals, and intelligence  services use various cyber tools that can deny access, degrade the integrity  of, intercept, or destroy data and jeopardize the security of the nation’s  critical infrastructure (see table 3).
The growing number of known vulnerabilities increases the potential  number of attacks. By exploiting software vulnerabilities, hackers and  others who spread malicious code can cause significant damage, ranging  from defacing Web sites to taking control of entire systems and thereby  being able to read, modify, or delete sensitive information; disrupt  operations; launch attacks against other organizations’ systems; or destroy  systems. Reports of attacks involving critical infrastructure demonstrate  that a serious attack could be devastating, as the following examples  illustrate.
In June 2003, the U.S. government issued a warning concerning a virus that  specifically targeted financial institutions. Experts said the BugBear.b  virus was programmed to determine whether a victim had used an e-mail  address for any of the roughly 1,300 financial institutions listed in the  virus’s code. If a match was found, the software attempted to collect and  document user input by logging keystrokes and then provide this  information to a hacker, who could use it in attempts to break into the  banks’ networks.
In August 2006, two Los Angeles city employees hacked into computers  controlling the city’s traffic lights and disrupted signal lights at four  intersections, causing substantial backups and delays. The attacks were  launched prior to an anticipated labor protest by the employees.
In October 2006, a foreign hacker penetrated security at a water filtering  plant in Harrisburg, Pennsylvania. The intruder planted malicious software  that was capable of affecting the plant’s water treatment operations.
In May 2007, Estonia was the reported target of a denial-of-service cyber  attack with national consequences. The coordinated attack created mass  outages of its government and commercial Web sites.
In March 2008, the Department of Defense reported that in 2007 computer  networks operated by Defense, other federal agencies, and defense-related  think tanks and contractors were targets of cyber warfare intrusion  techniques. Although those responsible were not definitively  substantiated, the attacks appeared to have originated in China.
As these examples illustrate, attacks resulting in the incapacitation or  destruction of the nation’s critical infrastructures could have a debilitating  impact on national and economic security and on public health and safety.
section_title: Federal Law and Policy Establish the Need for National Cyber Analysis and Warning
paragraphs: To protect the nation’s critical computer-dependent infrastructures against  cyber threats and attacks, federal law and policy have identified the need  to enhance cybersecurity and establish cyber analytical and warning  capabilities, which are sometimes referred to as “indications and  warnings.” The laws and policies include (1) the Homeland Security Act of  2002, (2) the National Strategy to Secure Cyberspace, (3) Homeland  Security Presidential Directive 7, and (4) the National Response  Framework. In addition, the President issued in January 2008 Homeland  Security Presidential Directive 23, which, according to US-CERT officials,  has provisions that affect cyber analysis and warning efforts of the federal  government.
subsections: section_title: Homeland Security Act of 2002
paragraphs: The Homeland Security Act of 2002 established the Department of  Homeland Security and gave it lead responsibility for preventing terrorist  attacks in the United States, reducing the vulnerability of the United States  to terrorist attacks, and minimizing the damage and assisting in recovery  from attacks that do occur. The act assigned the department, among  other things, a number of critical infrastructure protection responsibilities,  including gathering of threat information, including cyber-related, from  law enforcement, intelligence sources, and other agencies of the federal,  state, and local governments and private sector entities to identify, assess,  and understand threats; carrying out assessments of the vulnerabilities of  key resources to determine the risks posed by attacks; and integrating  information, analyses, and vulnerability assessments in order to identify  priorities for protection. In addition, the department is responsible for  disseminating, as appropriate, information that it analyzes—both within  the department and to other federal, state, and local government agencies  and private sector entities—to assist in the deterrence, prevention,  preemption of, or response to terrorist acts.
section_title: National Strategy to Secure Cyberspace
paragraphs: The National Strategy to Secure Cyberspace proposes that a public/private  architecture be provided for analyzing, warning, and managing incidents of  national significance. The strategy states that cyber analysis includes  both (1) tactical analytical support during a cyber incident and   (2) strategic analyses of threats. Tactical support involves providing  current information on specific factors associated with incidents under  investigation or specific identified vulnerabilities. Examples of tactical  support include analysis of (1) a computer virus delivery mechanism to  issue immediate guidance on ways to prevent or mitigate damage related  to an imminent threat or (2) a specific computer intrusion or set of  intrusions to determine the perpetrator, motive, and method of attack.  Strategic analysis is predictive in that it looks beyond one specific incident  to consider a broader set of incidents or implications that may indicate a  potential future threat of national importance. For example, strategic  analyses may identify long-term vulnerability and threat trends that  provide advance warnings of increased risk, such as emerging attack  methods. Strategic analyses are intended to provide policymakers with  information that they can use to anticipate and prepare for attacks,  thereby diminishing the damage from such attacks.
section_title: Homeland Security Presidential Directive 7
paragraphs: Homeland Security Presidential Directive 7 (HSPD 7) directs DHS to,  among other things, serve as the focal point for securing cyberspace. This  includes analysis, warning, information sharing, vulnerability reduction,  mitigation, and recovery efforts for critical infrastructure information  systems. It also directs DHS to develop a national indications and  warnings architecture for infrastructure protection and capabilities,  including cyber, that will facilitate an understanding of baseline  infrastructure operations, the identification of indicators and precursors to  an attack, and create a surge capacity for detecting and analyzing patterns  of potential attacks.
In May 2005, we reported that DHS has many cybersecurity-related roles  and responsibilities, including developing and enhancing national cyber  analysis and warning capabilities. However, we found that DHS had not  fully addressed all its cybersecurity-related responsibilities and that it  faced challenges that impeded its ability to fulfill its responsibilities. These  challenges included having organizational stability and authority, hiring  employees, establishing information sharing and effective partnerships,  and developing strategic analysis and warning. We made  recommendations to the Secretary of Homeland Security to engage  appropriate stakeholders to prioritize key cybersecurity responsibilities,  develop a prioritized list of key activities to addressing underlying  challenges, and identify performance measures and milestones for  fulfilling its responsibilities and for addressing its challenges. We did not  make new recommendations regarding cyber-related analysis and warning  because our previous recommendations had not been fully implemented.  Specifically, in 2001, we recommended that responsible executive branch  officials and agencies establish a capability for strategic analysis of  computer-based threats, including developing a methodology, acquiring  expertise, and obtaining infrastructure data.
section_title: National Response Framework
paragraphs: The National Response Framework, issued by DHS in January 2008,  provides guidance to coordinate cyber incident response among federal  entities and, upon request, state and local governments and private sector  entities. Specifically, the Cyber Incident Annex describes the framework  for federal cyber incident response in the event of a cyber-related incident  of national significance affecting the critical national processes. Further,  the annex formalizes the National Cyber Response Coordination Group  (NCRCG). As established under the preceding National Response Plan, the  NCRCG continues to be cochaired by DHS’s National Cyber Security  Division (NCSD), the Department of Justice’s Computer Crime and  Intellectual Property Section, and the DOD. It is to bring together officials  from all agencies that have responsibility for cybersecurity and the sector- specific agencies identified in HSPD 7. The group coordinates  intergovernmental and public/private preparedness and response to and  recovery from national-level cyber incidents and physical attacks that have  significant cyber-related consequences. During and in anticipation of such  an incident, the NCRCG’s senior-level membership is responsible for  providing subject matter expertise, recommendations, and strategic policy  support and ensuring that the full range of federal capabilities is deployed  in a coordinated and effective fashion.
section_title: Homeland Security Presidential Directive 23
paragraphs: In January 2008, the President issued HSPD 23—also referred to as  National Security Presidential Directive 54 and the President’s “Cyber  Initiative”—to improve the federal government’s cybersecurity efforts,  including protecting against intrusion attempts and better anticipating  future threats. While the directive is a classified document, US-CERT  officials stated that it includes steps to enhance cyber analysis related  efforts, such as requirements that federal agencies implement a centralized  monitoring tool and that the federal government reduce the number of  connections to the Internet, referred to as Trusted Internet Connections.
section_title: DHS Established US-CERT to Provide National Cyber Analysis and Warning
paragraphs: To help protect the nation’s information infrastructure, DHS established  the US-CERT. It is currently positioned within the NCSD of DHS’s Office  of Cybersecurity and Communications. Figure 1 shows the position of  these offices within DHS’s organizational structure.
US-CERT is to serve as a focal point for the government’s interaction with  federal and nonfederal entities on a 24-hour-a-day, 7-day-a-week basis  regarding cyber-related analysis, warning, information sharing, major  incident response, and national-level recovery efforts. It is charged with  aggregating and disseminating cybersecurity information to improve  warning of and response to incidents, increasing coordination of response  information, reducing vulnerabilities, and enhancing prevention and  protection. In addition, the organization is to collect incident reports from  all federal agencies and assist agencies in their incident response efforts. It  is also to accept incident reports when voluntarily submitted by other  public and private entities and assist them in their response efforts, as  requested.
US-CERT is composed of five branches, as shown in figure 2: Operations,  Situational Awareness, Law Enforcement and Intelligence, Future  Operations, and Mission Support. Each branch has specific responsibilities    The Operations branch is to receive and respond to incidents, disseminate  reasoned and actionable cybersecurity information, and analyze various  types of data to improve overall understanding of current or emerging  cyber threats affecting the nation’s critical infrastructure.
The Situational Awareness branch is to identify, analyze, and comprehend  broad network activity and to support incident handling and analysis of  cybersecurity trends for federal agencies so that they may increase their  own situational awareness and reduce cyber threats and vulnerabilities. As  part of its responsibilities, the branch is responsible for managing the  information garnered from the US-CERT Einstein program, which obtains  network flow data from federal agencies, and analyzing the traffic patterns  and behavior. This information is then combined with other relevant data  to (1) detect potential deviations and identify how Internet activities are  likely to affect federal agencies and (2) provide insight into the health of  the Internet and into suspicious activities.
The Law Enforcement and Intelligence branch is to facilitate information  sharing and collaboration among law enforcement agencies, the  intelligence community, and US-CERT through the presence of liaisons  from those organizations at US-CERT.
The Future Operations branch was established in January 2007 to lead or  participate in the development of related policies, protocols, procedures,  and plans to support US-CERT’s coordination of national response to  cyber incidents.
The Mission Support branch is to manage US-CERT’s communications  mechanisms, including reports, alerts, notices, and its public and classified  Web site content.
section_title: Cyber Analysis and Warning Encompasses Four Key Capabilities
paragraphs: Our research and observations at federal and nonfederal entities show that  cyber analysis and warning typically encompasses four key capabilities:    Monitoring—detecting cyber threats, attacks, and vulnerabilities and  establishing a baseline of system and communication network assets and  normal traffic.
Analysis—using the information or intelligence gathered from monitoring  to hypothesize about what the threat might be, investigate it with technical  and contextual expertise and identify the threat and its impact, and  determine possible mitigation steps. Analysis may be initiated in reaction  to a detected anomaly. This is a tactical approach intended to triage  information during a cyber incident and help make decisions. It may also  be predictive, proactively reviewing data collected during monitoring to  look at cyber events and the network environment to find trends, patterns,  or anomaly correlations that indicate more serious attacks or future  threats.
Warning—developing and issuing informal and formal notifications that  alert recipients in advance of potential or imminent, as well as ongoing,  cyber threats or attacks. Warnings are intended to alert entities to the  presence of cyber attack, help delineate the relevance and immediacy of  cyber attacks, provide information on how to remediate vulnerabilities  and mitigate incidents, or make overall statements about the health and  welfare of the Internet.
Response—taking actions to contain an incident, manage the protection  of network operations, and recover from damages when vulnerabilities are  revealed or when cyber incidents occur. In addition, response includes  lessons learned and cyber threat data being documented and integrated  back into the capabilities to improve overall cyber analysis and warning.
Through our consultations with experts, we found that the terminology  may vary, but the functions of these capabilities are fairly consistent  across cyber analysis and warning entities. Figure 3 depicts the basic  process of cyber analysis and warning capabilities.
Typically, cyber analysis and warning is executed, or managed, from a  central focal point known as an operation center or watch center. Such  centers can serve a single organization or a number of organizations.  Centers generally include physically and electronically connected  multidisciplinary teams with access to a variety of communication and  software tools. The teams are made up of specialized analysts, sometimes  referred to as watch standers, with a combination of expertise in  information security, intelligence, and cyber forensics. Teams may also  include subject area experts with specialized expertise in certain critical  infrastructure sectors, industries, or technologies. The centers operate  tools that integrate data and facilitate analysis by the watch standers. The  data come from a multitude of sources, including internal or external  monitoring, human or signals intelligence, analytical results, warnings  from other entities, and information collected from previous threat  responses. Centers decide when and how to issue formal and informal  warnings that contribute to further analysis or provide information that  aids in decisions about how to respond to an incident.
Depending on the size and organizational structure of an organization, the  analysis and warning team may work with incident response teams during  a cyber incident. The incident response team manages the decisions  required for handling an incident using information discovered during  monitoring, analysis, and warning. The team may also coordinate with  those responsible for information security for the organization in order to  assess risks, remediate vulnerabilities, and prepare for and respond to  attacks.
section_title: Fifteen Key Attributes Essential to Establishing Cyber Analysis and Warning Capabilities
paragraphs: Our research and past experience at federal and nonfederal entities  identified 15 key attributes associated with the cyber analysis and warning  capabilities of monitoring, analysis, warning, and response. These  attributes are displayed in table 4, which is followed by a detailed  description by capability of each attribute.
subsections: section_title: Monitoring
paragraphs: Monitoring provides the data used to understand one’s operating  environment and detect changes that indicate the presence of anomalies  that may be cyber attacks. It encompasses five key attributes:  1.  Establishing a baseline understanding of network assets and  normal network traffic volume and flow  In order to detect unusual activity in network traffic or changes in an  operating environment, organizations require knowledge of ordinary  traffic and environmental conditions. This knowledge forms the baseline  against which changes or anomalies can be detected, identified, and  mitigated. A baseline is established through activities such as creating an  accurate inventory of systems, prioritizing resources and assets,  maintaining an understanding of the expected volume and nature of  network traffic, and instituting operational procedures such as procedures  for handling incidents. Without a baseline, it may be difficult to effectively  detect threats or respond to a warning with the appropriate resources.  2.  Assessing risks to network assets  Assessments should be conducted to determine what risks are posed by  combinations of threats and vulnerabilities and inform the monitoring  capability so that it is focused on the most critical assets. According to  CERT® Coordination Center (CERT/CC) officials, having a baseline  knowledge of networks and systems and their associated risks in advance  helps individual organizations understand what threats they may be  susceptible to, what resources are at risk, and what the potential damage  of an attack might be. Risks should be prioritized and mitigated until a  reasonable acceptable level of risk is reached.  3.  Obtain internal information on network operations via  technical tools and user reports  Another key attribute is monitoring traffic on internal networks using (1)  network and information security-related technology tools and (2) reports  on network activity. As table 5 shows, various technologies can be used  for internal network monitoring to help compile and identify patterns in  network data. Each type of technology may detect anomalies that the  other types of software cannot.
These technologies can be used to examine data logs from networks on a  24-hour-a-day, 7-day-a-week schedule in an effort to identify   (1) precursors and indicators of cyber threats or other anomalies and   (2) the occurrence of known attacks. The data logged from these  technologies are typically prepared using automated tools to help analysts  observe or detect a single anomaly or to discover patterns in data over  time. According to several federal and nonfederal entities, hands-on  monitoring by trained analysts is essential because it can be difficult for  automated tools to identify anomalies and incidents. For example, some  automated signature-based tools focus on known threats and may not  automatically recognize or alert analysts to new attack patterns or new  threat delivery techniques. Other intrusion detection systems can produce  large numbers of alerts indicating a problem when one does not exist  (false positives); therefore, an analyst must look into anomalies more  closely to see if detected intrusions are indications of a threat or simply an  equipment malfunction.  4.  Obtaining external information on threats, vulnerabilities, and  incidents through various relationships, alerts, and other  sources  External monitoring includes observing and receiving information that is  either publicly or not publicly available for the purpose of maintaining  environmental or situational awareness, detecting anomalies, and  providing data for analysis, warning, and response. External sources of  information include  formal relationships, such as with and between critical infrastructure  sector-related information sharing and analysis centers (ISAC); federal  agencies, including military, civilian, law enforcement, and intelligence  agencies; international computer emergency response team organizations;  the CERT/CC and vendors under contract for services;  informal relationships established on a personal basis between analysts  located at different operations centers;   alerts issued by federal, state, and local governments;    alerts issued by commercial external sources such as network security and    vulnerability databases, standards, and frameworks such as the National  Vulnerability Database, the Common Vulnerability and Exposures List,  Common Vulnerability Scoring System, and the Open Vulnerability  Assessment Language;   media outlets, such as television news and newspapers; and    Web sites, such as law enforcement entities’ sites, known hacker and  criminal sites and chat rooms, and cooperative cyber analysis and warning  services. 5.  Detecting anomalous activities  Continuous monitoring occurs in order to detect significant changes from  the baseline operations or the occurrence of an attack through an already  known threat or vulnerability. It is ultimately the detection of an  anomaly—observed internally or received from external information—and  the recognition of its relevance that triggers analysis of the incident to  begin.
section_title: Analysis
paragraphs: Analysis uses technical methods in combination with contextual expertise  to hypothesize about the threat and associated risks concerning an  anomaly and, if necessary, determine mitigation solutions. It encompasses  four key attributes:  1.  Verifying that an anomaly is an incident  Once an anomaly is detected, it should be verified whether it is a genuine  cyber incident by determining that the data are from a trusted source and  are accurate. For example, if the anomaly was identified by an internal  sensor, analysts start by confirming that the sensor was working correctly  and not indicating a false positive. If the anomaly was reported by an  external source, analysts try to determine the trustworthiness of that  source and begin to identify internal and external corroborating sources.  Anomalies that are verified may require in-depth investigation and incident  handling or more observation through monitoring.  2.  Investigating the incident to identify the type of cyber attack,  estimate impacts, and collect evidence  Once the anomaly is verified as a potential, impending, or occurring  incident, analysts should combine information from multiple sources  and/or perform investigative testing using available tools. Analysis often  occurs through collaboration between analysts, the exchange of  notifications and warnings, and the use of analytical research techniques.  Analysts use these techniques to investigate the type of attack, its source  (where it originates), its target (whom it affects), and the immediate risk  to network assets and mission performance. In addition, these techniques  are used to compile evidence for law enforcement. Techniques for  investigation include    comparing and correlating additional monitoring data available with the  anomaly to determine what other internal and external entities are  experiencing;   comparing data about the anomaly with standardized databases to  determine if the threats are known; and   performing investigations, such as cyber forensic examinations, reverse  engineering, malware analysis, and isolating anomalies in a test  environment such as a honeypot or a sandbox.  3.  Identifying possible actions to mitigate the impact of the  Analysis should culminate in identifying essential details about an anomaly  such as what specific vulnerabilities are exploited or what impacts are  expected for a specific incident. Steps should then be taken to identify  alternative courses of action to mitigate the risks of the incident according  to the severity of the exploit, available resources, and mission priorities.  Such steps may include isolating the affected system to prevent further  compromise, disabling the affected service that is being exploited, or  blocking the connections providing the attacker a route into the network  environment. These courses of action may lead to more analysis or be  used to support the warning capability.  4.  Integrating results into predictive analysis of broader  implications or potential future attacks  Information resulting from analysis of an individual incident should be  used to determine any broader implications and predict and protect  against future threats. This type of effort, or predictive analysis, should  look beyond one specific incident to consider a broader set of incidents or  implications that may indicate a potential threat of importance. For  example, it may include detailed trend analysis of threats that have  occurred over a certain period of time that is issued in public reports that  discuss current trends, predict future incident activity, or emerging attack  methods. However, according to many experts, this type of predictive  analysis is complex and it is still difficult to predict future threats with  current data.
section_title: Warning
paragraphs: Warnings are intended to alert entities to the presence of anomalies, help  delineate the relevancy and immediacy of cyber attacks, provide  information on how to remediate vulnerabilities and mitigate incidents, or  make overall statements about the health and welfare of the Internet.  Warning includes three key attributes:  1.  Developing notifications that are targeted and actionable  Warning messages should be targeted to the appropriate audience and  provide details that are accurate, specific, and relevant enough to be acted  upon. Developing actionable notifications requires providing the right  incident information to the right person or group. If a single group is the  only target of a threat, a warning directly to it may be more appropriate  than a general public announcement. In addition, warnings are tailored to  address technical or nontechnical recipients. Some warnings may be more  appropriate for chief information officers, while other may include  technical details for network administrators. Although notifications and  warnings are delivered throughout incident handling, it is important to  reach a balance between releasing actionable information and disclosing  warnings too often, which can overwhelm the recipients and stretch  limited resources. By addressing the specific audience, warnings avoid  overwhelming recipients with extraneous or irrelevant information.
Also, recipients of notifications and warnings need to be able to use them  to protect or defend their networks against cyber attacks. For example,  many organizations have designated thresholds that determine how and  when warnings are issued. To do so, the messages must include specific  and accurate information about the incident as it relates to the recipient’s  monitoring, analysis, or response capabilities. An actionable warning may  also include recommendations about how to respond to an incident.  Federal and nonfederal entities also noted that sensitivity of information  and privacy are key considerations when trying to develop an actionable  warning. Warnings are sanitized or stripped of identifying or proprietary  information in order to protect the privacy of individuals or entities  involved in the incident. In addition, the federal government and its private  sector partners must also adhere to procedures to make sure that they  share useful information at the appropriate clearance level.  2.  Providing notifications in a timely manner  Warnings are intended to give information to recipients as early as  possible—preferably in advance of a cyber attack—to give them time to  take appropriate action. In addition, the National Institute of Standards  and Technology (NIST) provides guidance to federal agencies that  describes when incidents are considered reportable and how long they  may take to report them to US-CERT. Similarly, several ISACs stated that  they have procedures that determine when and how warnings are issued  and when and how members should report incidents.  3. Distributing notifications using the most appropriate  communications methods  Once a warning is developed, it is important to determine the best method  for getting that message out without overwhelming the public or incident  handlers. Warnings can be provided both informally and formally. Informal  warnings between colleagues with established trusted relationships can  happen quickly and without significant regard to the organizational  structure. Formal warnings, which are typically held to a higher standard  of accuracy by recipients than informal warnings, come in many forms,  such as e-mail bulletins, vulnerability alerts, Web postings, targeted  warnings to a specific entity, or broad security notices to the general  public. In addition to specific formal warnings, operations centers that  perform analysis and warning for multiple organizations, such as the  ISACs and commercial vendors, use level-based or color-coded alert  systems on their Web sites to quickly notify members and the public of the  general threat status of the infrastructure or Internet. Changing from one  level or color to another indicates that the threat level is increasing or  decreasing. These same organizations send alerts about threats and  vulnerabilities to members only or may issue specific warnings to a single  organization that has been identified through analysis as being targeted by  a cyber threat.
section_title: Response
paragraphs: Response includes actions to contain an incident, manage the protection  of network operations, and recover from damages when vulnerabilities are  revealed or when cyber incidents occur. It encompasses three key  attributes:  1.  Containing and mitigating the incident  When an incident is identified, immediate steps should be taken to protect  network assets. Decisions are made to control further impacts on the  network and then eliminate the threat. These actions may include  installing a software patch, blocking a port known to be used by a  particular threat, or deploying other appropriate network resources. In the  case of a serious threat, the decision may be to turn off the network  gateway and temporarily isolate the network from the Internet, depending  upon what assets are at risk. One industry expert noted that investigation  may occur before any mitigation steps are taken in order to consider the  necessity of law enforcement involvement. On the other hand, if little is  known about a threat and it does not appear to endanger critical assets, a  decision might be made to watch the threat emerge in a contained area to  allow for further monitoring and analysis. Decisions to act or not are  based on acceptable risks, available resources, and ability to remedy the  known threat. In addition, decisions must be made in the context of the  impact that actions will have on other related efforts, such as a law  enforcement investigation.  2.  Recovering from damage and remediating vulnerabilities  Once an incident is contained and mitigated, restoring damaged areas of  the network to return it to its baseline becomes a priority. To understand  the damage, a cyber damage or loss assessment may be conducted to  identify, among other things, how the incident was discovered, what  network(s) were affected, when the incident occurred, who attacked the  network and by what methods, what was the intention of the attacker,  what occurred during the attack, and what is the impact or severity of the  incident. The recovery efforts may involve restoring or reinstalling  computers, network devices, applications, or systems that have been  compromised.
Taking action to remediate vulnerabilities in a network may also result  from analysis and incident management. Entities work to discover and  reduce the number of vulnerabilities in their computers, network devices,  applications, or systems.  3.  Evaluating actions and incorporating lessons learned  Entities should ensure that threat data, results, and lessons learned are  evaluated and appropriately incorporated to improve the overall cyber  analysis and warning capability. For example, teams can be used to  simulate network threats by purposefully attacking a network in order to  see how the network responds. From these simulations, an evaluation can  be made about the response, and recommendations on how to improve  can be developed. In addition, cyber simulations allow critical  infrastructure organizations to prepare for threat scenarios and to test  analysis, warning, and response capabilities. NIST guidance also states  that holding lessons learned meetings after major incidents is helpful in  improving security measures and the incident handling process itself.
section_title: US-CERT’s Capabilities Include Some but Not All Aspects of Key Attributes
paragraphs: US-CERT has established cyber analysis and warning capabilities that  include aspects of each of the key attributes. However, they do not fully  incorporate all of them.
subsections: section_title: Monitoring Capability Includes Most but Not All Aspects of Key Attributes
paragraphs: US-CERT has established capabilities that include aspects of key  attributes of monitoring. For example, it obtains internal network  operation information via technical tools and Einstein; obtains external  information on threats, vulnerabilities, and incidents; and detects  anomalous activities based on the information it receives. However, its  capabilities do not fully incorporate all of the key attributes of monitoring.  For example, it has not established a baseline of our nation’s critical  infrastructure information systems. Table 6 shows our analysis of its  monitoring capability.
As part of the President’s Cyber Initiative, DHS has a lead role for several  provisions that, if implemented appropriately, could address key  monitoring deficiencies, such as not having a comprehensive national  baseline and sufficient external information on threats, vulnerabilities, and  incidents. According to testimony by the Under Secretary for the National  Protection and Programs Directorate, the initiative makes the Einstein  program mandatory across all federal agencies. In addition, DHS plans to  enhance Einstein’s capabilities to be a real-time intrusion detection and  situational awareness system. Further, DHS, along with the Office of  Management and Budget (OMB), is responsible for working with federal  agencies to reduce the number of Trusted Internet Connections used by  the federal government. According to DHS and OMB officials, these  initiatives will enhance the ability of the US-CERT to monitor federal  systems for cyber attacks and other threats. According to US-CERT  officials, the reduction in Trusted Internet Connections, along with the  positioning of Einstein in front of those connections to the Internet, will  help provide a governmentwide baseline and view of the traffic entering  and leaving federal networks as well as access to the content of the traffic.  In addition, according to the Assistant Secretary for Cybersecurity and  Communications, the recently announced National Cybersecurity Center,  which reports directly to the Secretary of Homeland Security, will be  responsible for ensuring coordination among the cyber-related efforts  across the federal government, including improving the sharing of incident  and threat information. However, the efforts to use Einstein, reduce  Internet connections, and implement the National Cybersecurity Center  are in their early stages and have not yet been fully planned or  implemented, so whether these efforts will fully address all five of the  monitoring attributes is not known at this time.
section_title: Analysis Capability Does Not Fully Incorporate All Aspects of Key Attributes
paragraphs: US-CERT has established capabilities that include key attributes of  analysis. For example, it verifies anomalies, performs investigations, and  identifies possible courses of action. However, its capabilities do not fully  incorporate other attributes because of technical and human resource  constraints and the gaps in the monitoring capability. Table 8 shows our  analysis of the organization’s analysis capability.
As part of the Cyber Initiative, the organization has received additional  resources to develop the next version of the Einstein situational  awareness tool. According to US-CERT officials, this new version, referred  to as Einstein 2.0, will provide real-time intrusion detection monitoring, a  content analysis capability, and automated analysis functions that are  currently manual. In addition, it has received authorization for an  additional 30 government and 50 contractor employee full-time  equivalents. According to US-CERT officials, they plan to fill the additional  positions by leveraging graduates of the Scholarship for Service program,  which provides cybersecurity-related scholarships to students willing to  serve the federal government for a time commitment. However, these  efforts are in their early stages and have not yet been fully planned or  implemented. Consequently, whether these efforts will fully address all  four of the analysis attributes is not known at this time.
section_title: Warning Capability Exhibits Some but Not All Characteristics of Key Attributes
paragraphs: The organization has established capabilities that include key attributes of  warning. For example, it develops and distributes a number of attack and  other notifications targeted to different audiences with varying frequency.  However, according to customers, these warning products are not  consistently actionable and timely. Table 8 shows our analysis of the  organization’s warning capability. Tables 9 and 10 show types of warning  products and the quantity of products issued during fiscal year 2007.  (This page left blank intentionally)
As part of the Cyber Initiative, the enhancements to the Einstein program,  as well as the reduction in the number of Trusted Internet Connections  can lead to more complete data. According to US-CERT officials, the  improved data will lead to an enhanced warning capability that could  provide the ability to issue targeted and actionable alerts in advance of  actual cyber attacks. However, these efforts are in their early stages and  have not yet been fully planned or implemented; thus, it is not clear  whether these efforts will fully address the three warning attributes.
section_title: Response Capability Satisfies Some but Not All Aspects of Key Attributes
paragraphs: US-CERT possesses a limited response capability to assist other entities in  the containment, mitigation, and recovery from significant cyber incidents.  For example, while it provides on-site assistance to various entities, its  ability to provide response at the national level is hindered by limitations  in the resources available and authority over affected entities. Table 11  shows our analysis of its response capability.
To improve the organization’s response capability, US-CERT officials  stated that they needed to perform internal exercises that test its national- level response capability more often than every 2 years, as is the case with  the Cyber Storm exercise. It plans to develop “tabletop” exercises to more  frequently test its response capabilities. In addition, according to NCSD  officials, they are working collaboratively with other federal and  nonfederal working groups to improve their performance measures so that  they can understand the value and use of their products and make  continuous improvements. However, until they do so, it is not clear  whether these efforts will lead to US-CERT fully addressing the three  response attributes.
section_title: US-CERT Faces New and Ongoing Challenges to Fulfilling Its Mission
paragraphs: US-CERT faces a number of newly identified and ongoing challenges that  impede it from fully implementing the key attributes and in turn  establishing cyber analysis and warning capabilities essential to  coordinating the national effort to prepare for, prevent, and respond to  cyber threats. The new challenge is creating warnings that are actionable  and timely—it does not consistently issue warning and other notifications  that its customers find useful. In addition, US-CERT continues to face four  challenges that we previously identified: (1) employing predictive cyber  analysis, (2) developing more trusted relationships to encourage  information sharing, (3) having sufficient analytical and technical  capabilities, and (4) operating without organizational stability and  leadership within DHS. Until DHS addresses these challenges and fully  incorporates all key attributes into its capabilities, it will not have the full  complement of cyber analysis and warning capabilities essential to  effectively performing its national mission.
subsections: section_title: New Challenge Involves Creating Warnings That Are Actionable and Timely
paragraphs: Developing and disseminating cyber threat warnings to enable customers  to effectively mitigate a threat in advance of an attack can be challenging  for the US-CERT. According to the organization’s Acting Deputy Director,  it serves as the nation’s cyber analysis and warning center and must  ensure that its warnings are accurate. In addition, owners of classified or  law enforcement information must review and agree to the release of  related information. Therefore, the organization’s products are subjected  to a stringent review and revision process that could adversely affect the  timeliness of its products—potentially adding days to the release if  classified or law enforcement information must be removed from the  product. For example, an official from a cybersecurity-focused  organization at a university stated that the alerts from US-CERT generally  arrive a day or two after they might have been helpful. An official from  another private entity stated that the bureaucratic process US-CERT must  follow prevents it from providing useful alerts in a timely manner and that  as a result, it does not have the credibility to drive a reaction when an alert  is finally issued. Another private sector official stated that, in some cases,  the organization gets information on cyber incidents and attacks faster  from media sources than US-CERT because its analysts need time to verify  the reliability of the data they receive.
In addition, according to federal officials responsible for determining  cyber-related threats, US-CERT, as well as other organizations with  cybersecurity-related responsibilities, must also balance the need to  develop and release warnings with the activities of other organizations,  such as law enforcement and intelligence support, to identify and mitigate  cyber threats. For example, the release of a warning to address a threat or  attack may also alert the intruders that their methods have been  discovered and cause them to change their methods prior to the  completion of an investigation about their activities.
Further, when there is sensitive information to share, US-CERT officials  stated that on numerous occasions, they were unable to share the details  of threats to customers’ networks because no one within the federal  agency or nonfederal entity possessed a security clearance high enough to  receive the information. In some organizations, the individuals who do  possess security clearances are in the upper echelons of the organization  and do not possess a cyber or information security background. As a  result, they are not always able to accurately comprehend and relay the  threat information to those who would actually handle the mitigation  efforts. In September 2007, we reported that DHS lacked a rapid, efficient  process for disseminating sensitive information to private industry owners  and operators of critical infrastructures. We recommended that DHS  establish a rapid and secure process for sharing sensitive vulnerability  information with critical infrastructure stakeholders, including vendors,  owners, and operators; however, DHS has not yet fulfilled this  recommendation.
To provide actionable information to its customers, the organization  attempts to combine incident information with related cyber threat  information to determine the seriousness of the attack. However,  according to the Acting Director of US-CERT, its efforts are limited by  other federal entities’ abilities to determine specific cyber threats to the  nation’s critical infrastructure. One reason for the lack of cyber threat data  is that the task is complex and difficult and there are no established,  generally accepted methodologies for performing such analysis. In  addition, such entities are hampered by the limited number of analysts  dedicated to cyber threat identification. For example, in January 2008, the  Director of HITRAC stated that only 5 percent of HITRAC’s total number  of analyst positions was focused on analyzing and identifying cyber threats  to our nation’s critical information infrastructure. According to the  director, it had received approval to double the number of cyber-related  analysts and was in the process of filling those positions. In addition, the  director stated that HITRAC’s primary focus is on identifying physical  threats.
section_title: Ongoing Challenges Involve Establishing Predictive Analysis, Trusted Relationships, Analytical and Technical Capabilities, and a Stable Organization
paragraphs: US-CERT faces ongoing challenges that we identified in previous reports  as impeding DHS’s ability to fulfill its cyber critical infrastructure  protection responsibilities.
Employing predictive cyber analysis—US-CERT has been unable to  establish the solid foundation needed to perform predictive cyber analysis  that would enable it to determine any broader implications from ongoing  network activity, predict or protect against future threats, or identify  emerging attack methods prior to an attack. Since 2001, we have identified  the challenges associated with establishing strategic, predictive analysis  and warning and have made recommendations that responsible executive  branch officials and agencies establish such capabilities, including  developing methodologies. According to the Acting Director of US-CERT,  it has not been able to establish such capabilities because there is not a  generally accepted methodology for performing predictive cyber analysis  and warning. In addition, officials from US-CERT and other federal and  nonfederal entities with cyber analysis and warning capabilities stated that  while they can determine the motivations for the various threat sources to  use cyber attacks, it is a formidable task to foresee prior to attacks how  those threats would actually conduct attacks and to establish indicators to  recognize that such cyber attacks are about to occur. Also, the relative  newness of the cyber analysis and warning discipline and immaturity of  the related methodologies and tools add to the complexity.
Developing more trusted relationships to encourage information  sharing—Implementing cyber analysis and warning capabilities, including  all of the key attributes, requires that entities be willing and able to share  information, including details about incidents, threats, vulnerabilities, and  network operations. However, US-CERT continues to be challenged to  develop relationships with external sources that would encourage  information sharing. For example, nonfederal entities do not consistently  fully disclose incident and other data—they filter sensitive details from the  data reported, thus reducing its value to US-CERT. The lack of such  relationships negatively affects the organization’s cyber analysis and  warning capability.
In 2005, we reported that entities within critical infrastructure sectors  possess an inherent disincentive to share cybersecurity information with  DHS. Much of their concern was that the potential release of sensitive  information could increase the threat they face. In addition, when  information was shared, it was not clear whether the information would  be shared with other entities, such as other federal entities, state and local  entities, law enforcement, or various regulators, or how it would be used  or protected from disclosure. Alternatively, sector representatives  expressed concerns that DHS was not effectively communicating  information with them and had not matched private sector efforts to share  valuable information with a corresponding level of trusted information  sharing. We also identified information sharing in support of homeland  security as a high-risk area in 2005, and we noted that establishing an  effective two-way exchange of information to help detect, prevent, and  mitigate potential terrorist attacks requires an extraordinary level of  cooperation and perseverance among federal, state, and local  governments and the private sector.
Federal and nonfederal officials raised similar concerns about the ability  to develop trusted relationships and share information with and between  cyber analysis and warning entities, including US-CERT. For example,  frequent staff turnover at NCSD and US-CERT hindered the ability to build  trusted relationships with both public and private entities. Federal and  nonfederal officials stated that reliance was placed on personal  relationships to support sharing of sensitive information about  cybersecurity and cyber incidents. However, according to the NCSD  director, six senior staff members within the Office of Cybersecurity and  Communications (the national focal point for addressing cybersecurity  issues) were leaving for various reasons, affecting the ability to develop  such relationships. In addition, private sector officials stated that their  organizations continued to be hesitant to share information on  vulnerabilities and threats because of the fear that such sharing might  negatively affect their financial bottom line. For example, private sector  officials stated that it was difficult to share unfiltered information with  their respective infrastructure sector ISAC because a competitor operated  the ISAC, thus negatively affecting the information received by US-CERT.
Having sufficient analytical and technical capabilities—Obtaining  and retaining adequately trained cyber analysts and acquiring up-to-date  technological tools to implement the analysis capability attributes is an  ongoing challenge to US-CERT and other analysis and warning centers,  hindering their ability to respond to increasingly fast, nimble, and  sophisticated cyber attacks. As we have reported, NCSD has had difficulty  hiring personnel to fill vacant positions. We reported that once it found  qualified candidates, some candidates decided not to apply or withdrew  their applications because it took too long to be hired. This is still a  concern because current staff has limited organizational backup and, in  some cases, performs multiple roles. In addition, a private sector official  stated that it is not clear whether or not the government has the number of  technical analysts necessary to perform analysis on large and complex  data sets that are generated whether or not an incident is in progress or  not.
Keeping cyber analysts trained and up to date on the latest cybersecurity  tools and techniques can be difficult. For example, a DOD official  representing one of its cyber analysis and warning centers stated that its  analysts must develop their expertise on the job because there is no formal  training program available that teaches them how to detect and perform  analysis of an anomaly or intrusion. A private sector official stated that  while analysts are often trained to use existing tools, their understanding  of the key attributes of analysis is often limited, resulting in a solution too  late to be helpful.
Analysts also need the appropriate technological tools to handle the  volume, velocity, and variety of malicious data and activity they are faced  with, according to federal officials. For example, although the Einstein  flow data are collected in real time, the actual analysis is manually  intensive and does not occur simultaneously or in real time. Another  limiting factor of Einstein data is that US-CERT is unable to analyze the  content of the potentially malicious traffic and must rely on the affected  agency to perform any analysis of the content of the traffic. Thus both the  reaction time to determine the intent of the anomalous activity and the  necessary actions to address it are significantly slowed. In addition,  officials from one private sector entity questioned if agencies can  sufficiently protect their networks using the tools they are mandated to  use.
As part of the efforts to address the President’s Cyber Initiative, US-CERT  recently received approval to fill 80 new positions—30 government and 50  contractor—and is attempting to fill these analytical positions by  extending offers to candidates in the National Science Foundation’s  Scholarship for Service Program. However, these positions have yet to be  completely filled with qualified candidates.
Operating without organizational stability and authority—We have  identified challenges regarding DHS’s organizational stability, leadership,  and authority that affect US-CERT’s ability to successfully perform its  mission. In the past, we have reported that the lack of stable leadership  has diminished NCSD’s ability to maintain trusted relationships with its  infrastructure partners and has hindered its ability to adequately plan and  execute activities. While DHS has taken steps to fill key positions,  organizational instability among cybersecurity officials continues to affect  NCSD and thus US-CERT. For example, at least six senior staff members  were leaving DHS’s Office of Cybersecurity and Communications,  including the NCSD Director. Losing senior staff members in such large  numbers has negatively affected the agency’s long-term planning and  hampered the ability of NCSD/US-CERT to establish trusted relationships  with public and private entities and to build adequate functions to carry  out its mission, including expanded cyber analysis and warning  capabilities, according to the official.
Furthermore, when new senior leadership has joined DHS,  NCSD/US-CERT’s objectives were reassessed and redirected, thus  affecting NCSD’s ability to have a consistent long-term strategy, according  to the former official. For example, senior officials wanted to broaden the  role and focus of US-CERT by having it provide centralized network  monitoring for the entire federal government on a 24-hour-a-day, 7-day-a- week basis. However, the Director of NCSD disagreed with this strategy,  stating that each federal agency should have its own 24-hour-a-day, 7-day- a-week incident-handling capability (either in-house or contracted out) to  respond to incidents affecting its own network. He viewed US-CERT as a  fusion center that would provide analysis and warning for national-level  incidents, support federal agency incident-handling capabilities during  crisis situations, and offer a mechanism for federal agencies to coordinate  with law enforcement.
The organization’s future position in the government’s efforts to establish  a national-level cyber analysis and warning capability is uncertain.  Specifically, Homeland Security Presidential Directive 23, which is  classified, creates questions about US-CERT’s future role as the focal point  for national cyber analysis and warning. In addition, DHS established a  new National Cybersecurity Center at a higher organizational level, which  may diminish the Assistant Secretary of Cyber Security and  Communications’ authority as the focal point for the federal government’s  cybersecurity-related critical infrastructure protection efforts, and thus  US-CERT’s role as the central provider of cyber analysis and warning  capabilities across federal and nonfederal critical infrastructure entities.
As stated above, we did not make new recommendations in 2005 regarding  cyber analysis and warning because our previous recommendations had  not yet been fully implemented. At the time, we did recommend that the  Secretary of Homeland Security require NCSD to develop a prioritized list  of key activities for addressing the underlying challenges related to  information sharing, hiring staff with appropriate capabilities, and  organizational stability and authority. In addition, we recommended that  performance measures and milestones for performing activities to address  these challenges be identified. However, since that time, DHS has not  provided evidence that it has taken actions on these activities.
section_title: Conclusions
paragraphs: In seeking to counter the growing cyber threats to the nation’s critical  infrastructures, DHS has established a range of cyber analysis and warning  capabilities, such as monitoring federal Internet traffic and the issuance of  routine warnings to federal and nonfederal customers. However, while  DHS has actions under way aimed at helping US-CERT better fulfill  attributes identified as critical to demonstrating a capability, US-CERT still  does not exhibit aspects of the attributes essential to having a truly  national capability. It lacks a comprehensive baseline understanding of the  nation’s critical information infrastructure operations, does not monitor all  critical infrastructure information systems, does not consistently provide  actionable and timely warnings, and lacks the capacity to assist in  mitigation and recovery in the event of multiple, simultaneous incidents of  national significance.
Planned actions could help to mitigate deficiencies. For example, as part  of the Cyber Initiative, US-CERT plans to enhance its Einstein situational  awareness tool so that it has real-time intrusion detection monitoring, a  content analysis capability, and automated analysis functions. By placing  the tool in front of Trusted Internet Connections, officials expect to obtain  a governmentwide baseline view of the traffic and content entering and  leaving federal networks. US-CERT also plans to hire 80 additional cyber  analysts and to increase the frequency of exercises that test its national- level response capability.
However, at this point, it is unclear whether these actions will help US- CERT—or whatever organizational structure is ultimately charged with  coordinating national cyber analysis and warning efforts—achieve the  objectives set forth in policy. DHS faces a number of challenges that  impede its ability to achieve its objectives, including fostering trusted  relationships with critical infrastructure sectors, hiring and retaining  skilled cyber analysts, ensuring that US-CERT warning products provide  useful information in advance of attacks, enhancing predictive analysis,  and ensuring that any changes brought about by HSPD 23 are marked by  well-defined and transparent lines of authority and responsibility. We  identified most of these challenges in our prior reviews and made broad  recommendations to address them. DHS’s actions to address these  challenges have not been adequate. Because of this, addressing these  challenges is as critical as ever to overcome the growing and formidable  threats against our nation’s critical cyber infrastructure. If these  challenges are not addressed, US-CERT will not be able to provide an  effective national cyber analysis and warning capability.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Homeland Security take four actions  to fully establish a national cyber analysis and warning capability.  Specifically, the Secretary should address deficiencies in each of the  attributes identified for    monitoring, including establish a comprehensive baseline understanding  of the nation’s critical information infrastructure and engage appropriate  nonfederal stakeholders to support a national-level cyber monitoring  capability;   analysis, including expanding its capabilities to investigate incidents;    warning, including ensuring consistent notifications that are targeted,  actionable, and timely; and response, including ensuring that US-CERT provides assistance in the  mitigation of and recovery from simultaneous severe incidents, including  incidents of national significance.
We also recommend that the Secretary address the challenges that impede  DHS from fully implementing the key attributes, including the following 6  items:    engaging appropriate stakeholders in federal and nonfederal entities to  determine ways to develop closer working and more trusted relationships;   expeditiously hiring sufficiently trained cyber analysts and developing  strategies for hiring and retaining highly qualified cyber analysts; identifying and acquiring technological tools to strengthen cyber analytical  capabilities and handling the steadily increasing workload;    developing predictive analysis capabilities by defining terminology,  methodologies, and indicators, and engaging appropriate stakeholders in  other federal and nonfederal entities; filling key management positions and developing strategies for hiring and  retaining those officials; and   ensuring that there are distinct and transparent lines of authority and  responsibility assigned to DHS organizations with cybersecurity roles and  responsibilities, including the Office of Cybersecurity and  Communications and the National Cybersecurity Center.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report (see app. II), signed by the  Director of DHS’s GAO/OIG Liaison Office, the department concurred with  9 of our 10 recommendations. It also described actions planned and under  way to implement the 9 recommendations. In particular, the department  said that to fully establish a cyber analysis and warning capability, it plans  to continue expansion of the Einstein intrusion detection system and  increase US-CERT’s staffing. In addition, to address the challenges that  impede DHS from fully implementing key cyber analysis and warning  attributes, the department stated that it plans to continue to build new  relationships and grow existing ones with stakeholders. Further, to  strengthen its analysis and warning capability and develop its predictive  analysis capability, the department cited, among other things, its planned  implementation of an upgraded version of Einstein.
DHS took exception to our last recommendation, stating that the  department had developed a concept-of-operations document that clearly  defined roles and responsibilities for the National Cybersecurity Center  and NCSD. However, this concept-of-operations document is still in draft,  and the department could not provide a date for when the document  would be finalized and implemented.
DHS also commented on the report’s description of US-CERT as “the  center.” Specifically, DHS was concerned that referring to US-CERT as the  center might lead to confusion with the department’s newly established  National Cybersecurity Center. DHS requested that we remove references  to US-CERT as the center. We agree with this comment and have  incorporated it in the report where appropriate.
In addition to its written response, the department provided technical  comments that have been incorporated in the report where appropriate.
We also incorporated technical comments provided by other entities  involved in this review.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to interested  congressional committees, the Secretary of Homeland Security, and other  interested parties. We also will make copies available to others upon  request. In addition, this report will be available at no charge on GAO’s  Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  David Powner at (202) 512-9286, or pownerd@gao.gov, or Dr. Nabajyoti  Barkakati at (202) 512-4499, or barkakatin@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this report. Major contributors to this report are listed in  appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) identify key attributes of cyber analysis and  warning capabilities, (2) compare these attributes with the United States  Computer Emergency Readiness Team’s (US-CERT) current analysis and  warning capabilities to identify whether there are gaps, and (3) identify  US-CERT’s challenges to developing and implementing key attributes and  a successful national cyber analysis and warning capability.
To identify key attributes of cyber analysis and warning capabilities, we  identified entities based on our previous work related to cyber critical  infrastructure protection, information security, and information sharing  and analyzed relevant laws, strategies, and policies. In addition, we  solicited suggestions from a variety of sources familiar with cyber analysis  and warning organizations, including GAO’s chief information technology  officer and members of our Executive Council on Information  Management and Technology, which is a group of executives with  extensive experience in information technology management who advise  us on major information management issues affecting federal agencies. On  the basis of the entities identified, we selected those that were relevant  and agreed to participate. We then gathered and analyzed policies, reports,  and surveys; made site visits to observe the operation of cyber analysis  and warning capabilities; conducted structured interviews; and received  written responses to structured interview questions. These activities were  performed, as appropriate, at the following entities:    Department of Defense: Commander and Deputy Commander of the Joint  Task Force—Global Network Operations and Director of the Defense  Information Systems Agency; Commanding Officer, Navy Cyber Defense  Operations Command; Chief Information Officer and Electronic Data  Service officials of the Navy’s Global Network Operations Center. We also  toured the Joint Task Force’s Global Network Operations Center; the  Navy’s Cyber Defense Operation Command Center; and the Navy Marine  Corps Intranet Network’s Operations Center, Computer Incident Response  Team Laboratory, Request Management Center, and Enterprise Global  Networks Operations Center.
Department of Energy: the Associate Chief Information Officer for Cyber  Security for the Department of Energy and other relevant officials, and the  Chief Information Officer of the National Nuclear Security Administration  and other relevant officials.
Department of Homeland Security: the Director of the National Cyber  Security Division, the Acting Director of the National Cyber Security  Division, and the Acting Director of US-CERT.
National Institute of Standards and Technology: the Director of the  Information Technology Laboratory and officials from the Information  Technology Laboratory’s Computer Security Division.
Private sector: Carnegie Mellon University’s CERT® Coordination Center,  Internet Storm Center, LUMETA, Microsoft, MITRE, National Association  of State Chief Information Officers, SANS Institute, SRI International, and  Symantec.
Information sharing and analysis centers representing the following  sectors: financial services, information technology, states, surface  transportation, and research and education.
Federal agencies in the intelligence community.
On the basis of the evidence gathered and our observations regarding each  entity’s capabilities and operations, we determined the key common  attributes of cyber analysis and warning capabilities. To verify the  attributes we identified, we solicited comments from each entity regarding  the attributes identified and incorporated the comments as appropriate.
To determine US-CERT’s current national analysis and warning  capabilities and compare them with the attributes identified to determine  whether there were any gaps, we gathered and analyzed a variety of US- CERT policies, procedures, and program plans to identify the  organization’s key activities related to cyber analysis and warning. We also  observed US-CERT operations. In addition, we held interviews with key  US-CERT officials, including the Director and Acting Director of the  National Cyber Security Division, the Acting Director and Deputy Director  of the US-CERT, and other relevant officials, to further clarify and confirm  the key initiatives we identified through our analysis of the  aforementioned documents. In addition, we interviewed the Director of  Intelligence for the Department of Homeland Security’s Homeland  Infrastructure Threat and Risk Analysis Center to determine that  organization’s interaction with US-CERT and its role regarding identifying  cyber threats. We also interviewed the Deputy Director of the Department  of Homeland Security’s National Cybersecurity Center to obtain  information about its concept-of-operations document. We then compared  those activities to the key attributes of cyber analysis and warning  capabilities in order to determine US-CERT’s ability to provide cyber  analysis and warning and identify any related gaps.
To identify US-CERT’s challenges to developing and implementing the key  attributes and a successful national cyber analysis and warning capability,  we gathered and analyzed relevant documents, such as past GAO reports  and studies by various cybersecurity-related entities, and interviewed key  federal and nonfederal officials regarding the challenges associated with  cyber analysis and warning. On the basis of the information received and  our knowledge of the issues, we determined the major challenges to  developing and implementing the key attributes and a successful national  cyber analysis and warning capability.
We performed this performance audit between June 2007 and July 2008 in  the Washington, D.C., metropolitan area; Atlanta, Georgia; Bloomington,  Indiana; Pittsburgh, Pennsylvania; and Norfolk, Virginia; in accordance  with generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on our  audit objectives.
section_title: Appendix II: Comments from the Department of Homeland Security
section_title: Appendix III: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the persons named above, Neil Doherty, Michael Gilmore,  Barbarol James, Kenneth A. Johnson, Kush K. Malhotra, Gary Mountjoy,  Jennifer Stavros-Turner, and Amos Tevelow made key contributions to  this report.