1

` 

 

Challenges Remain in Securing the Nation’s 

Cyber Infrastructure 

OIG-07-48 June 2007

DEPARTMENT OF HOMELAND SECURITY 

Office of Inspector General 
Office of Inspector General 

U.S. Department of Homeland Security 

Washington, DC 20528

 June 5, 2007 

Preface 

The Department of Homeland Security (DHS) Office of Inspector General (OIG) was established by 

the Homeland Security Act of 2002 (Public Law 107-296) by amendment to the Inspector General 

Act of 1978. This is one of a series of audit, inspection, and special reports prepared as part of our 

oversight responsibilities to promote economy, efficiency, and effectiveness within the department. 

 

This report addresses the strengths and weaknesses of the National Cyber Security Division’s 

implementation of its mission to coordinate security of the nation’s cyber infrastructure. It is based 

on interviews with employees and officials of relevant agencies and institutions, direct observations, 

technical scans, and a review of applicable documents. 

The recommendations herein have been developed to the best knowledge available to our office, and 

have been discussed in draft with those responsible for implementation. It is our hope that this 

report will result in more effective, efficient, and economical operations. We express our 

appreciation to all of those who contributed to the preparation of this report. 

 

Richard L. Skinner 

Inspector General 
Table of Contents/Abbreviations 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

 Executive Summary...................................................................................................................1 

Background................................................................................................................................2 

Results of Audit .........................................................................................................................4 

Progress Made In Securing Cyberspace..........................................................................4 

Better Management of Strategic Plan Is Needed ............................................................5 

Establish Priorities to Ensure Critical Tasks Are Completed .........................................5 

NCSD Needs to Improve the Monitoring of Its Milestones ...........................................6 

Recommendations...........................................................................................................7 

Management Comments and OIG Analysis....................................................................8 

Better Defined Performance Measures Are Needed .......................................................8 

Recommendation ..........................................................................................................10 

Management Comments and OIG Analysis..................................................................10 

Expanded Incident Reporting Analysis Will Improve Identification and Response 

to Cyber Incidents .........................................................................................................11 

Recommendations.........................................................................................................13 

Management Comments and OIG Analysis..................................................................13 

Improved Information Sharing and Communications Will Enhance Cyber 

Infrastructure Security....................................................................................................14 

Recommendations..............................................................................................................17

Management Comments and OIG Analysis ......................................................................18 

Revised Certification and Accreditation Will Satisfy FISMA Requirements............19 

Recommendations..............................................................................................................21

Management Comments and OIG Analysis ......................................................................22 

Appendices 

 Appendix A: Purpose, Scope, and Methodology .................................................................24 

 Appendix B: Management Comments to the Draft Report .................................................26 

 Appendix C: NCSD Major Functions and Responsibilities ................................................35 

 Appendix D: Federal Agency Incident Categories ..............................................................37 

 Appendix E: Major Contributors to this Report ..................................................................38 

 Appendix F: Report Distribution.........................................................................................39 
Table of Contents/Abbreviations 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Abbreviations 

 CS&C Cyber Security and Communications 

 CS&T Cyber Security and Telecommunications 

 DHS Department of Homeland Security 

 FISMA Federal Information Security Management Act 

 IT Information Technology 

 NCSD National Cyber Security Division 

 NIST National Institute of Standards and Technology 

 OIG Office of Inspector General 

 OMB Office of Management and Budget 

 PART Program Assessment Rating Tool 

 US-CERT United States Computer Emergency Readiness Team 
OIG 

Department of Homeland Security 

Office of Inspector General

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 1 

Executive Summary 

We audited the National Cyber Security Division to determine whether: (1) it 

is working collaboratively with the public, private, and international entities to 

secure cyberspace and cyber assets; (2) it is effectively managing the 

implementation of The National Strategy to Secure Cyberspace; and 

(3) security controls were effectively implemented on two mission support 

systems (Einstein and Cybercop Portal). 

Since our last review in 2004, the National Cyber Security Division has taken 

actions to further implement The National Strategy to Secure Cyberspace. For 

example, the division has established a fully operational incident handling 

center (United States Computer Emergency Readiness Team). The National 

Cyber Security Division has put into action programs that promote cyber 

security awareness among the public and private sectors; improve vendor 

software development and reduce vulnerabilities; develop and promote sound 

practices and standards that enhance cyber security; promote a global culture 

of security through international outreach awareness; promote and facilitate the 

development of adequately trained information technology professionals; and 

plan, coordinate, and conduct cyber exercises with the public and private 

sectors to improve cyber security readiness, protection, and incident response 

capabilities. The National Cyber Security Division has established working 

groups and participated with public and private sector organizations to share 

information and protect cyberspace and cyber assets. 

While the National Cyber Security Division has made progress in meeting its 

mission, it can improve its efforts to secure the nation’s cyber infrastructure. 

Specifically, the division has not (1) established priorities to ensure that its 

mission-critical tasks supporting its programs are completed timely; 

(2) developed enhanced performance measures that can be used to evaluate the 

effectiveness in meeting its mission; (3) fully developed its information 

sharing and communications programs with the private sector; (4) developed 

and implemented enhanced procedures to ensure that all known cyber incidents 

from across the federal government are being reported; and (5) ensured that its 

support systems comply with all Federal Information Security Management 

Act requirements, including testing of contingency plans. 

We are making 14 recommendations to the Assistant Secretary for Cyber 

Security and Communications (CS&C). CS&C has already begun to take 

actions to implement 13 of the recommendations. CS&C’s response is 

summarized and evaluated in the body of this report and included, in its 

entirety, as Appendix B. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 2 

Background 

The Department of Homeland Security (DHS) established the National Cyber 

Security Division (NCSD) in June 2003 to serve as a national focal point for 

addressing cyber security issues and to coordinate implementation of the 

cyber security strategy in the United States. NCSD’s mission is to work 

collaboratively with public, private, and international entities to secure 

cyberspace and cyber assets, and to implement the actions and 

recommendations of The National Strategy to Secure Cyberspace.

1

 

In September 2006, the first Assistant Secretary for Cyber Security and 

Telecommunications (CS&T) was appointed within the Preparedness 

Directorate, which includes NCSD.2 NCSD reports to the Assistant Secretary 

and is headed by the Office of the Director and comprises four branches: 

United States Computer Emergency Readiness Team (US-CERT) Operations, 

Law Enforcement and Intelligence, Outreach and Awareness, and Strategic 

Initiatives. In October 2006, a new NCSD Acting Director was named and 

who later became the Director in January 2007. As of December 2006, NCSD 

had a staff of 107 (31 federal employees, 4 detailees, and 72 contractors). See 

Figure 1 for an organization chart and Appendix C for the major functions and 

responsibilities of each branch. 

Figure 1 – NCSD Organization Chart 

 

1

 The White House issued The National Strategy to Secure Cyberspace in February 2003. 2

 On March 31, 2007, CS&T was renamed Office of Cyber Security and Communications and the Preparedness 

Directorate was renamed the National Protection and Programs Directorate. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 3 

US-CERT is the operational arm of NCSD and is charged with protecting the 

nation's Internet infrastructure by coordinating defense against and response to 

cyber attacks. In addition, US-CERT is responsible for analyzing and 

reducing cyber threats and vulnerabilities, disseminating cyber trend and 

analysis information, and coordinating incident response activities. US-CERT 

interacts with federal agencies, the information technology (IT) industry, the 

research community, state and local governments, and others to disseminate 

reasoned and actionable cyber security information to the public. 

NCSD has worked collaboratively with the private and public sector partners 

to develop the IT Sector-Specific Plan. The final draft of this document was 

completed in December 2006. The IT Sector-Specific Plan provides a 

foundation for sector planning activities to manage risk and to improve 

situational awareness, response, recovery, and reconstitution of the nation’s IT 

infrastructure. These improvements, which can be accomplished through the 

public and private sectors, will enhance national capabilities. The IT 

Sector-Specific Plan identifies near-term (less than 1 year) and long-term 

(1 to 3 years) actions that require collaborative efforts between and among the 

private sector, state and local governments, non-governmental organizations, 

and the federal government. 

CS&T produced a strategy report, in December 2006, for the DHS Secretary 

that provides a comprehensive overview of its organization. The report lists 

stakeholders’ expectations, CS&T’s capabilities, current initiatives, gaps 

between stakeholder expectations and the organization’s capabilities and 

programs, strategic priorities, and actions to support the mission over the next 

2 years. 

In July 2004, we reported that NCSD had begun to implement the actions and 

recommendations detailed in The National Strategy to Secure Cyberspace.

3

 

We noted that NCSD faced a number of challenges to address long-term cyber 

threats and vulnerabilities to the nation’s critical infrastructure. Specifically, 

we found that NCSD had not: 

• Prioritized its initiatives to address the recommendations in The

National Strategy to Secure Cyberspace. 

• Identified the resources needed to ensure that it could identify, 

analyze, and reduce long-term cyber threats and vulnerabilities. 

• Developed strategic implementation plans, including performance 

measures and milestones, focusing on the division’s priorities, 

initiatives, and tasks. 

 

3 Progress and Challenges in Securing the Nation’s Cyberspace (OIG-04-29, July 2004).
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 4 

• Instituted a formal communications process within DHS, as well as the 

public, private, and international sectors. 

• Initiated and implemented a process to oversee and coordinate efforts 

to develop best practices and create cyber security policies with other 

government agencies and the private sector. 

NCSD has taken corrective actions to develop strategic implementation plans, 

institute a formal communications process, and oversee and coordinate efforts 

to develop best practices and security policies. However, other 

recommendations remain open and are addressed in this report. 

Results of Audit 

Progress Made In Securing Cyberspace

NCSD has made progress in its cyberspace security efforts. Specifically, 

NCSD established a fully operational, incident-handling center (US-CERT) to 

facilitate information sharing across all infrastructure sectors and to help 

protect and maintain the continuity of the nation’s cyber infrastructure. In 

addition, NCSD has established working groups and participated with public 

and private sector organizations to share information and protect cyberspace 

and cyber assets. 

NCSD also has put into action programs to address the recommendations in 

The National Strategy to Secure Cyberspace. For example: 

• The Outreach and Awareness program was established to promote 

cyber security awareness among and within the public and private 

sectors, maintain relationships with governmental cyber security 

professionals to coordinate and share information about cyber security 

initiatives, and develop partnerships to promote coordination and 

collaboration on cyber security activities. 

• The Software Assurance program seeks to reduce software 

vulnerabilities, minimize exploitation, and address ways to improve 

capabilities to develop and deploy trustworthy software products. 

• The Standards and Best Practices program promotes and sponsors the 

development of standards, guidance documents, metrics, and tools that 

raise awareness and encourage the implementation of cyber security 

practices and processes. 

• The International Affairs program was established for NCSD to 

engage in international outreach activities to build awareness about the 

global cyber risk, share information about the role and activities of 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 5 

computer security incident response teams, and build relationships 

among governments toward global cooperation on cyber security. 

• The Training and Education program promotes and facilitates the 

development of adequately trained information technology 

professionals to support the Nation’s cyber security needs via effective 

training, education, and certification programs. 

• The Cyber Exercise program plans, coordinates, and conducts cyber 

exercises with the public and private sectors as a mechanism for 

NCSD to improve cyber security readiness, protection, and incident 

response capabilities. 

The implementation of these programs helps NCSD fulfill its mission to 

partner with the public and private sectors to prevent, minimize, prepare for, 

and respond to threats to critical IT infrastructure. However, as discussed in 

the following sections, improvements are needed in NCSD’s efforts to 

implement The National Strategy to Secure Cyberspace. 

Better Management of Strategic Plan Is Needed

NCSD has not effectively managed its strategic plan to ensure that critical 

tasks are completed timely. Specifically, NCSD has not prioritized key 

activities for the division or established effective performance measures to 

monitor the division’s progress in accomplishing its mission and goals. In 

addition, NCSD needs to improve its incident reporting analysis in order to 

identify and reduce underreporting by federal agencies. Until NCSD 

implements its critical strategic initiatives, including the reporting of all cyber 

incidents and monitoring its progress with effective performance measures, 

the division cannot ensure that it can partner successfully with the public and 

private sector to prevent, minimize, prepare for, and respond to threats to 

critical IT infrastructure. 

 

Establish Priorities to Ensure Critical Tasks Are Completed

NCSD has undertaken steps to address the recommendations proposed in The 

National Strategy to Secure Cyberspace. However, in addressing the 

recommendations, NCSD has not prioritized specific initiatives, taking into 

consideration the required resources to ensure the timely completion of its 

initiatives. As a result, many of NCSD’s initiatives are not complete and the 

progress to date has been limited. 

While NCSD has begun many initiatives to address the recommendations 

proposed in The National Strategy to Secure Cyberspace, many of its efforts 

lacked specific timeframes for completion. NCSD personnel said that each of 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 6 

the actions outlined in the division’s draft Strategic Plan, dated May 2006, and 

all of the program plans developed by its branches that support it, are a 

“priority.” We determined that these actions had not been prioritized based on 

NCSD management goals or available resources. 

Certain critical programs that were established to implement recommendations 

from The National Strategy to Secure Cyberspace will not be completed for 

another 2 or more years. For example, the Control Systems Security Program, 

the goal of which is to guide a cohesive effort between government and industry 

and reduce the risk to critical infrastructure control systems, is not expected to 

complete its roll out until FY 2009. Additionally, the computer forensics 

laboratory, that will train government security personnel to become 

specialized in multiple forensic areas, is not expected to be fully operational 

until FY 2009, but will begin initial training in the fall of 2007. 

In addition, some members of the National Cyber Response Coordination 

Group, the IT-Information Sharing and Analysis Center, and the IT-Sector 

Coordinating Council expressed their concerns that NCSD has too many 

priorities for the available resources provided to the division. Some group 

members also expressed concerns that NCSD was not working on initiatives 

that are currently needed. For example, members cited the need for smaller 

cyber exercises in addition to the larger scaled one previously held or the 

creation of operational recovery plans in the event of a major cyber incident. 

Group members believed that these concerns became more critical with the 

issuance in December 2006 of the IT-Sector Specific Plan. This plan listed 

over 70 short and long-term actions, many needing NCSD assistance, which 

need to be completed within the next 3 years. In addition, the CS&T Strategy 

report, issued in December 2006, identified strategic priorities and actions that 

are to be accomplished within the next 2 years. Taken in total, the number of 

existing and new actions required of NCSD is significant and could 

overextend the capabilities of the division if not prioritized and properly 

resourced. 

NCSD Needs to Improve the Monitoring of Its Milestones 

NCSD’s monitoring of its program initiatives needs improvement. NCSD has 

not established a process to monitor and manage effectively the division’s 

current milestones and deliverables. As a result, it is difficult for management 

to monitor efforts of the division in completing its initiatives and determining 

whether they are on schedule, ahead of schedule, or behind schedule. 

NCSD has not consolidated tracking all of the division’s milestones and 

deliverables. NCSD produced a draft Strategic Plan in May 2006, which 

details its actions and milestones to accomplish the division’s goals. To 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 7 

support the Strategic Plan, NCSD staff developed ten program plans that 

include milestones for major deliverables and actions. Program plans list the 

overview of the program, goals, milestones, performance measures, resources, 

and challenges. In addition, NCSD management uses quarterly Program 

Assessment Rating Tool (PART) reports to track milestones that have been 

completed during the quarter. PART is a diagnostic tool that the Office of 

Management and Budget (OMB) uses to assess the performance of programs 

and to drive improvement in program performance. OMB defines three 

categories of performance measures: (1) outcome-the intended result of 

carrying out a program or activity (value added), (2) output-the level of 

activity that will be provided over a period of time, and (3) efficiency-the ratio 

of the outcome or output to the input of any program. The PART report only 

identifies tasks or milestones that have been met. It does not identify or 

explain delays. 

The Strategic Plan and the individual program plans do not associate resource 

requirements to the achievement of deliverables and milestones. In addition, 

four of the ten program plans reviewed did not contain information as to 

interim milestones. For example, some milestones were listed as “ongoing,” 

“FY 2007,” “FY06-FY10,” or “FY06-FY10 Planned.” These types of 

milestones do not allow NCSD management to monitor effectively the 

incremental progress of its initiatives and evaluate whether the program goals 

have been achieved. Interim milestone dates are often needed to manage 

incremental progress for activities that can take years to complete in their 

entirety. We also found instances where some milestones were reported in the 

program plans as complete, but were not reported on the PART report. 

The sophistication and effectiveness of cyber attacks have steadily advanced 

in recent years. Therefore, to improve its ability to address these threats, it is 

imperative that NCSD prioritize its initiatives based upon available resources 

and criticality as determined by management. Without establishing priorities, 

NCSD cannot ensure that critical initiatives and milestones are accomplished 

on schedule and in a timely manner. Improving the monitoring of its 

milestones will allow NCSD the ability to better guide and track the division’s 

activities in implementing the recommendations outlined in The National 

Strategy to Secure Cyberspace.

Recommendations 

We recommend that the Assistant Secretary for CS&C direct the NCSD 

Director to: 

Recommendation #1: Establish priorities and milestones (short term and long 

term) for critical tasks using input from the CS&T Strategy Summary report, 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 8 

IT-Sector Specific Plan, and The National Strategy to Secure Cyberspace. 

Milestones should be based on available funding and resources. 

Recommendation #2: Consolidate the tracking of all current and future 

initiatives and milestones, and review at least quarterly. 

Management Comments and OIG Analysis 

CS&C concurred with recommendation 1. CS&C plans to address its 

strategic priorities through short and long term actions over the course of the 

next 2 years and beyond. NCSD’s implementation plan and detailed program 

plans, which are mapped to CS&C’s priorities, contain its goals and priorities. 

To improve performance management within its new organizational structure, 

CS&C is in the process of defining additional performance measures and 

milestones that tie into NCSD’s mission, priorities, processes, and available 

resources. In addition, CS&C has scheduled its first quarterly review to 

evaluate NCSD’s progress in terms of priorities developed by the Assistant 

Secretary for CS&C, current funding, and anticipated funding for 

Fiscal Year 2008. 

We agree that the steps that CS&C has taken, and plans to take, satisfy this 

recommendation. 

CS&C did not concur with the finding that resulted in recommendation 2. In 

disagreeing with the finding that NCSD needs to improve the monitoring of its 

milestones, CS&C indicated that NCSD monitors its milestones and 

performance based upon its strategic plan and updates its program plans each 

quarter. In addressing our recommendation, NCSD is revising its strategic 

plan with a comprehensive implementation plan and updated program plans to 

track actions, milestones, and resources. NCSD will report quarterly to CS&C 

on accomplishments and deliverables mapped to programmatic actions, 

milestones and resource allocations, identifying anticipated potential shortfalls 

and remedial actions. 

We believe that the steps that NCSD has taken, and plans to take, satisfy this 

recommendation to consolidate the tracking of all current and future initiatives 

and milestones and review them quarterly. 

Better Defined Performance Measures Are Needed

NCSD has not developed effective performance measures needed to monitor 

the division’s success in accomplishing its mission and goals. Without such 

measures, NCSD will have difficulty in determining how effective its 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 9 

initiatives are in achieving significant results in strengthening the nation’s 

cyber security. 

Performance measures are the indicators or metrics that are used to gauge 

program performance. Furthermore, performance measures should address 

the direct products and services delivered by a program (outputs), and the 

results of those products and services (outcomes). Outcomes are important, as 

they often describe the intended result or consequence that will occur from 

carrying out a program or activity. For example, an outcome goal might be to 

reduce the time it takes to disseminate cyber threat warning by FY 2008. 

Therefore, outcomes are of direct importance to beneficiaries and the public. 

While performance measures must distinguish between outcomes and outputs, 

there must be a reasonable connection between them, with outputs supporting 

(i.e., leading to) outcomes in a logical fashion. 

NCSD has developed performance measures based on OMB’s PART metrics: 

output, outcome, and efficiency. NCSD branches report to OMB the status of 

their metrics on a quarterly basis. To monitor branch progress, NCSD 

identified three performance measures: 

• Number of cyber security products delivered to key stakeholders 

(output). 

• Percent of targeted stakeholders who participate in or obtain cyber 

security products and services (outcome). 

• Percent of completion of key milestones and accomplishments 

(efficiency). 

We determined that NCSD has not developed sufficient outcome performance 

measures to ensure that its programs are achieving the intended results and 

impact, i.e., secure cyberspace and reduce security vulnerabilities. For 

instance, NCSD developed performance measures that emphasize “quantity” 

(number of newsletters issued, number of people attending conferences or 

workshops), but has not developed measures to evaluate the “quality” of its 

products and services. For example, outcomes that measure the effect 

newsletters or conferences have on the cyber community, the number of 

vulnerabilities reduced, or the usefulness of the information or service to the 

public have not been established. 

In addition, since one of NCSD’s performance measures is the percent of 

completion of key milestones, it is important that target completion dates be 

established for all milestones and deliverables in order to monitor the 

performance and progress of the branches. For example, some of the 

milestones in the individual program plans were listed as “ongoing” or 

“FY06-FY10.” Additionally, one of the goals of the Outreach and Awareness 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 10 

branch is to maintain relationships with governmental cyber security 

professionals in order to share information about cyber security initiatives, 

and develop partnerships to promote collaboration on cyber security 

preparedness issues. Performance measures and milestones that would 

evaluate its success have not been developed.

According to OMB guidance, performance measures are developed to monitor 

a program’s accomplishments and determine whether results are being 

achieved. Establishing performance measures also keeps program partners 

focused on the key goals of a particular initiative. Performance measures 

must reflect a program’s mission and priorities, be few in number, and should 

reflect direct outcomes. In some cases where the outcome of a program may 

not be realized for many years, a program should define specific short- and 

medium-term steps or milestones to accomplish the long-term outcome 

performance goals. Appropriate performance goals should include 

performance measures and targets, outcomes, and annual and long-term 

measures and targets. 

Recommendation 

We recommend that the Assistant Secretary for CS&C direct the NCSD 

Director to: 

Recommendation #3: Develop additional performance measures for each 

branch that can be used to review and periodically evaluate the outcome or 

success of the division’s programs. 

Management Comments and OIG Analysis 

CS&C did not concur with the finding that resulted in recommendation 3. In 

disagreeing with the finding that better defined performance measures are 

needed, CS&C indicated that NCSD developed and began collecting internal 

program level measures to improve the ability to assess overall programmatic 

progress in the third quarter of Fiscal Year 2006. In addressing our 

recommendation, NCSD recently developed revised PART measures to cover 

all of its programs. NCSD is regularly evaluating its metrics and strives to 

improve them. 

We believe that the steps that NCSD has taken, and plans to take, satisfy this 

recommendation to develop additional performance measures. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 11 

Expanded Incident Reporting Analysis Will Improve Identification and 

Response to Cyber Incidents

US-CERT is not performing detailed analysis to ensure that all incidents are 

being received from all federal agencies.4

 Less than complete reporting 

hampers the government’s ability to know whether an incident is isolated at 

one agency or is part of a larger event (widespread propagation of an Internet 

worm) and thus complicates and delays appropriate response, such as 

distributing security patches or other compensating controls. 

Security incidents are reported to US-CERT either through OMB-mandated 

incident reports from federal agencies or incident reports submitted 

voluntarily by state, local, and tribal governments, as well as private sector 

organizations. In addition, US-CERT monitors real-time network traffic from 

selected Internet access points to detect suspicious activities for those federal 

agencies that are participating in the Einstein program. The Einstein program 

is an initiative that builds cyber-related situational awareness across the 

participating federal agencies. Specifically, the Einstein program employs an 

automated tool to monitor government agencies’ network traffic to facilitate 

the identification and response to cyber threats and attacks, improve network 

security, increase the resiliency of critical electronically delivered government 

services, and enhance the survivability of the Internet. The Einstein program 

helps agencies identify baseline network traffic patterns, configuration 

problems, unauthorized network traffic, network backdoors, routing 

anomalies, and network scanning activities. 

Federal agencies are required to follow US-CERT’s Concept of Operations for 

Federal Cyber Security Incident Handling when analyzing and reporting 

security incidents. US-CERT leveraged National Institute of Standards and 

Technology (NIST) guidance to identify incident and event categories and 

reporting timeframes for federal civilian agencies when submitting security 

incidents to US-CERT (see Appendix D). 

In OMB’s 2004 and 2005 reports to Congress, the accuracy, timeliness, and 

completeness of incident reporting was identified as a concern.5

 In addition, 

OMB cited that the number of incidents reported indicated sporadic reporting 

by some agencies and unusually low levels of reported malicious activity at 

other agencies. 

 

4

 An incident is a violation or imminent threat of violation of computer security policies, acceptable use policies, or 

standard computer security practices. Examples of the incidents are (1) denial of service attacks, (2) malicious codes, 

(3) unauthorized access, and (4) improper usage. 

5 Federal Information Security Management Act (FISMA) 2004 Report to Congress, dated March 1, 2005, and FY 2005 

Report to Congress on Implementation of The Federal Information Security Management Act of 2002, dated 

March 1, 2006. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 12 

Security incidents reported by federal agencies are analyzed for emerging 

threats and ranked by severity. US-CERT does not determine whether all 

security incidents and events are being reported by all federal agencies. 

Furthermore, US-CERT has not developed a process to use potential cyber 

incidents identified by Einstein along with other analyses to identify possible 

incident underreporting. 

We obtained a summary report prepared by US-CERT for the period of 

October 1, 2004, to September 30, 2006. The number of incidents reported to 

US-CERT by federal agencies has increased each of the last 2 years 

(FY 2005: 3,631; FY 2006: 5,237). See Figure 2 below for the number of 

incidents and events reported during this period. 

Figure 2-Total Incidents and Events by Month

There is a wide variation in the number of incidents and events reported to 

US-CERT by federal agencies of comparable size. For example, over the 

2-year period, one agency with more than 56,000 employees reported 726 

incidents, whereas another comparable agency with more than 67,000 

employees reported only 17 incidents. In an effort to address this problem, 

OMB reported to Congress in FY 2005 that DHS had begun deploying 

Einstein’s automated tool to monitor network traffic at three agencies and had 

funding to install it at an additional six agencies.6 OMB further reported that 

the use of this and similar tools should considerably improve the 

government’s ability to identify and respond to potential incidents in a timely 

manner. While DHS is the lead federal agency in securing the nation’s 

cyberspace, it is not one of the federal agencies using Einstein departmentwide to monitor its network traffic. 

 

6

 As of December 2006, eight agencies were participating in the Einstein program. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 13 

As a service to federal agencies, US-CERT summarizes all security incidents 

and events, and issues a high-level snapshot report (quarterly and annually) to 

the White House and federal agencies. According to US-CERT personnel, 

these reports can be used by agencies to compare their individual security 

incidents and events with those reported by other agencies. 

NCSD management said that while the Federal Information Security 

Management Act (FISMA) requires federal agencies to report all incidents to 

US-CERT, NCSD has not been given the authority to enforce individual 

agency reporting. To rectify sporadic incident reporting by federal agencies, 

NCSD staff have discussed the issue with OMB. NCSD has also met with the 

Chief Information Officers Council to make them aware of the need and 

benefits of reporting all incidents to US-CERT and have performed outreach 

to agencies that are reporting few if any incidents. NCSD management said 

that the annual FISMA reviews performed by the agency Inspectors General 

are used to identify and report instances of underreporting. 

An effective incident response capability is critical to a government-wide 

security program as well as individual agency programs. In order for 

US-CERT to successfully perform its duties, it must have an accurate 

depiction of incidents across all agency bureaus and operating divisions. 

Additionally, incident reports can provide Chief Information Officers and 

other agency senior managers with valuable input for risk assessments, help 

prioritize security improvements, and illustrate risk and related trends. 

Recommendations 

We recommend that the Assistant Secretary for CS&C direct the NCSD 

Director to: 

Recommendation #4: Develop and implement procedures to review and 

analyze agencies’ incidents submissions to identify underreporting of 

incidents by federal agencies. 

Recommendation #5: Work with OMB and federal agencies to eliminate 

underreporting of cyber security incidents to US-CERT and complete the 

deployment of Einstein to all federal agencies. 

Management Comments and OIG Analysis 

CS&C did not concur with recommendation 4. While NCSD continues to 

work collaboratively with OMB and federal agencies to encourage more 

robust reporting, compliance with FISMA regulations and underreporting 

issues fall outside the scope and authority of US-CERT. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 14 

While we agree that OMB is responsible for directing federal agencies to 

report all cyber incidents to US-CERT under FISMA guidance, we believe 

that NCSD, as the lead in securing cyberspace for the federal government, 

should develop and implement procedures to identify underreporting of 

incidents. The identification of underreporting would be based on analysis 

and situational monitoring currently being performed by US-CERT. The 

procedures we are recommending could be used in conjunction with 

recommendation 5 (eliminating underreporting of cyber security incidents), 

which CS&C agreed to implement. 

CS&C concurred with recommendation 5. NCSD continues to work 

collaboratively with OMB and federal agencies to encourage more robust 

reporting but does not have the authority to mandate reporting. NCSD is 

working to expand the Einstein program to additional federal agencies. The 

program is planning an aggressive rollout of additional Einstein installations 

by the end of Fiscal Year 2008.

We agree that the steps that CS&C has taken, and plans to take, satisfy this 

recommendation. 

Recommendation 

We recommend that the DHS Chief Information Officer: 

Recommendation #6: Participate in the Einstein program in order to detect 

and identify potential security incidents. 

Management Comments and OIG Analysis 

CS&C concurred with recommendation 6. NCSD is working closely with the 

Office of the Chief Information Officer to deploy Einstein across the 

department. A memorandum of agreement has been drafted between the 

parties and they will continue to collaborate. 

We agree that the steps that management has taken, and plans to take, satisfy 

this recommendation. 

Improved Information Sharing and Communications Will Enhance Cyber 

Infrastructure Security

NCSD needs to improve its information sharing and communications 

programs with its private sector partners. Some private sector partners 

expressed concern over the focus and clarity of NCSD’s communications. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 15 

Concern was also expressed with the sharing of classified information with 

the private sector. Without the public and private sectors working together in 

identifying and sharing critical cyber information, there is little assurance that 

all critical data is made available to key personnel in order to prevent or 

recover from a major cyber incident. 

Communications with Private Sector

NCSD interacts with the IT sector (comprised of the producers and providers 

of hardware, software, and IT services) through working groups and incident 

response organizations. NCSD also collaborates with groups from the public 

and private sectors to facilitate the collection and sharing of computer security 

information and incidents. 

While members of the groups interviewed said that there have been 

improvements at NCSD in the past year, including the sharing of threats and 

vulnerabilities and better interaction with NCSD staff, many in the private 

sector remain resistant to sharing information with the federal government. 

Group members expressed concern with the number of NCSD’s ongoing 

priorities (noted in our previous finding), communications with the private 

sector, and the sharing of information. 

NCSD helped to create the National Cyber Response Coordination Group, the 

principal interagency forum to coordinate intra-governmental and 

public/private preparedness efforts to respond to and recover from large-scale 

cyber attacks. NCSD established the Government Forum of Incident 

Response and Security Teams, a group of technical and tactical practitioners 

of security response teams responsible for securing government information 

technology systems. NCSD also interacts with the IT-Sector Coordinating 

Council and the IT-Information Sharing and Analysis Center. The 

IT-Information Sharing and Analysis Center is recognized by DHS as the 

information sharing organization for private industry within the IT sector. 

Some of the IT information sharing organization members expressed concern 

that NCSD is not focusing its communications program toward the right 

people. While there has been an extensive amount of outreach and 

communications with the public and private sector cyber stakeholders, some 

members believe that there is a lack of communications by NCSD with senior 

executives at key IT sector companies. The members believe that NCSD 

senior management should meet with the executives individually to convince 

them of the value of cyber security improvements and collaboration with the 

federal government. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 16 

Members in the private sector also had concerns when receiving information 

requests from NCSD. At times, it was unclear what specific information DHS 

wanted, why the information was needed, what NCSD would do with it, and 

how the information would be protected. Per group members, there remains a 

hesitancy to share information by the private sector due to a lack of trust of 

the government. 

Communications with the private sector is a challenge for NCSD. NCSD’s 

message to the private sector concerning the need for stronger cyber security 

and information sharing must be communicated to many people at different 

levels within organizations who are responsible for securing their own portion 

of cyberspace. Without the support of senior management in the private 

sector, there is little assurance that employees at these companies will 

implement the security improvements needed to secure their cyber assets. 

Further, without a clear understanding by the private sector of how 

information is used and secured, there is little assurance that NCSD will 

obtain the necessary information needed to protect the cyber infrastructure. 

Sharing of Classified and Sensitive Information

Some private sector partners also expressed concern over sharing of classified 

information. Specifically, incident and vulnerability information that the 

private sector believes that they need to secure their own cyber assets is 

deemed classified, and thus prevents the sharing of critical information. In 

addition, some members of the private sector said that they are not provided 

with clear guidance on how sensitive (For Official Use Only [FOUO]) or 

classified information that they do receive can be shared with other people 

within their organizations. 

US-CERT personnel said that classified information is received from many 

sources including the intelligence community, Department of Defense, and 

law enforcement agencies. US-CERT personnel will send an 

unclassified/FOUO version only to those organizations or individuals that 

they believe need the information. 

The originating organization of classified information is responsible for 

determining the classification of the documents being sent to US-CERT. Per 

US-CERT personnel, the originator is to include an unclassified portion or 

version so that the critical information can be distributed to the appropriate 

organizations. At times, US-CERT personnel must request the unclassified 

version from the originator if it was not sent originally. In addition, 

US-CERT must ask the originator of the information for their approval before 

sending the information out to the recipients. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 17 

According to NCSD, US-CERT’s capability to disseminate unclassified 

information to its public and private sector constituents continues to evolve as 

NCSD works more closely with the originating organizations to improve the 

process. In many cases, the originator of the classified information will not 

allow NCSD to share unclassified versions of the information with the private 

sector. The marking of unclassified documents as FOUO limits NCSD’s 

ability to share information with the private sector. 

In an attempt to keep the private sector informed on cyber matters, NCSD 

staff has held meetings with several Information Sharing and Analysis 

Centers, participated in daily IT-Information Sharing and Analysis Center 

Operations Center conference calls, and are available for individual inquiries 

concerning the sharing of classified or sensitive information. NCSD has also 

discussed classified information with cleared private sector individuals. 

Homeland Security Presidential Directive 7 requires DHS and federal 

agencies to collaborate with the private sector to facilitate sharing of 

information about physical and cyber threats, vulnerabilities, incidents, 

potential protective measures, and best practices. The National Strategy to 

Secure Cyberspace also calls for the removal of impediments to information 

sharing between the public and private sectors. 

It is essential that people and organizations receive and are able to share 

critical information in order to take the appropriate steps to reduce the effects 

of a cyber incident. In its December 2006 Strategy report, CS&T 

acknowledged that information sharing, including the sharing of classified 

information, is a gap between stakeholder expectations and DHS’ capabilities 

and programs. 

Recommendations 

We recommend that the Assistant Secretary for CS&C direct the NCSD 

Director to: 

Recommendation #7: Develop clearer communications with key personnel 

and organizations in the private sector to explain the processes being used to 

capture, share, use, and secure cyber information. 

Recommendation #8: Expand its communications program with a focus on 

key private sector executives to encourage corporations to more actively 

participate in the information sharing program and to better secure its systems. 

Recommendation #9: Develop and implement formal procedures for 

receiving, reviewing, and distributing sensitive and classified information. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 18 

The procedures should include the types of information that can be shared, 

and the timing of receiving an unclassified (or reduced level) and redacted 

version or portion of the information that can be shared with cyber partners in 

the private sector. These procedures should be reviewed and agreed to by all 

organizations providing sensitive and classified information. 

Management Comments and OIG Analysis 

CS&C concurred with recommendation 7. NCSD will continue to build and 

maintain strong working relationships with the IT sector. In October 2006, 

US-CERT, in collaboration with the IT-Information Sharing and Analysis 

Center, developed a draft Concept of Operations for private industry cyber 

security incident handling that addresses information sharing, communication, 

and coordination with the private sector, including how the public and private 

sectors will work together and how information will be shared. In addition, 

the NCSD Internet Disruption Working Group completed a draft of the 

information sharing assessment in January 2007 and findings will be briefed 

to the Internet community at the next Internet Disruption Working Group 

forum. CS&C will continue to enhance analysis and information aggregation 

functions to provide timely and actionable dissemination of information. 

We agree that the steps that CS&C has taken, and plans to take, satisfy this 

recommendation. 

CS&C concurred with recommendation 8. CS&C agreed that more can and 

should be done to expand communications to key private sector stakeholders. 

The Assistant Secretary for CS&C created a permanent External Affairs 

position within the CS&C front office working closely with NCSD to engage 

with private sector executives in a variety of forums. Recent activities include 

meetings with key private sector representatives and major industry groups. 

We agree that the steps that CS&C has taken, and plans to take, satisfy this 

recommendation. 

CS&C concurred with recommendation 9. CS&C agreed that more should be 

done to improve information sharing with public and private sector 

organizations. US-CERT is working with its intelligence community partners 

to develop procedures for sharing of information between organizations. In 

addition, NCSD and its partners have shared classified information with 

cleared private sector stakeholders and continue to make every effort to 

develop unclassified versions of key documents to ensure broad dissemination 

of actionable information. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 19 

We agree that the steps that CS&C has taken, and plans to take, begin to 

satisfy this recommendation. 

Revised Certification and Accreditation Will Satisfy FISMA Requirements 

NCSD’s certification and accreditation documentation requires updating in 

order to satisfy FISMA requirements. We noted deficiencies with the security 

artifacts7

 contained in the accreditation packages for two NCSD systems, 

Einstein and Cybercop Portal, which were authorized to operate in FY 2006.8 

Lacking key information, agency officials cannot make credible risk-based 

decisions on whether to authorize systems to operate or ensure that systems 

are adequately secure. Both systems were authorized to operate for 1 year; 

therefore they are scheduled to be recertified and accredited in 2007. 

NCSD has implemented adequate security controls over the systems reviewed. 

However, some security documents were missing key information that is 

required to meet applicable OMB, NIST, and DHS guidelines.9

 

Einstein Documentation

NCSD uses data captured from its Einstein system sensors to analyze 

suspicious network traffic from across the federal government network 

infrastructure. A number of improvements are needed in the system 

documentation. Specifically: 

• The system security plan does not identify each of the systems that are 

connected to Einstein or contain an accurate description of the 

hardware installed. Agency officials require this detailed information 

in order to make credible, risk-based decisions on whether adequate 

controls have been implemented and to determine whether to authorize 

the system to operate. 

 

7

 DHS requires 11 artifacts be developed during the certification and accreditation process. The 11 artifacts are: 

Authority To Operate letter, system security plan, risk assessment, security test and evaluation, security assessment 

report, contingency plan, contingency plan test results, e-authentication determination, Federal Information Processing 

Standards Publication 199 determination, privacy threshold analysis, and annual self assessment. 8

 The Cybercop Portal is a secure Internet-based information sharing mechanism for more than 8,700 law enforcement 

members involved in the field of electronic crimes investigations. The law enforcement community, including 

investigators from private industry, e.g., banks and the network security community, is tied together and supported by 

this secure, Internet-based, collaboration portal. Members represent all 50 states, effectively all government agencies, 

and more than 40 countries. 

9

 According to OMB policy, certification and accreditation requires documentation of security planning, including risk 

assessments, contingency plans, incident response plans, security awareness and training plans, information systems 

rules of behavior, configuration management plans, security configuration checklists, privacy impact assessments, and 

system interconnection agreements. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 20 

• Einstein’s contingency plan does not address backup and recovery 

procedures necessary to restore operations in the event of an 

emergency or system failure. 

• NCSD has not tested the contingency plan to ensure that business and 

computer operations can be maintained or restored, possibly at an 

alternate location, in the event of an emergency, system failure, or 

disaster. 

• The risk assessment does not evaluate the likelihood of vulnerabilities 

identified that may be exploited or the potential impact and magnitude 

of harm to the system by exploiting the vulnerabilities identified. 

• Physical controls implemented at the contractor's location were not 

evaluated when completing the NIST 800-26 self-assessment to ensure 

that physical access to computer resources is restricted to authorized 

personnel. 

Cybercop Portal Documentation

NCSD supports the Cybercop Portal, which is used by law enforcement 

entities at all levels of government to communicate and share information. A 

number of improvements are needed in the system’s security documentation. 

Specifically: 

• The risk assessment does not stipulate the use of security testing to 

identify vulnerabilities or analyze the effectiveness of controls 

implemented. 

• The system test and evaluation plan does not include procedures to 

assess the effectiveness of controls implemented. 

• NCSD has not performed annual security testing to evaluate the 

system controls implemented. 

• The contingency plan has not been tested to determine whether 

specific aspects of the plan such as the data backup and recovery 

procedures remain valid. 

• Security awareness training was not provided to contractors who are 

responsible for maintaining the system. 

Vulnerability Assessments of Einstein and Cybercop Portal

Overall, NCSD has implemented adequate security controls over each of the 

two systems reviewed. To assess system security, we (1) interviewed 

information technology personnel responsible for Einstein and Cybercop 

Portal; (2) performed vulnerability scans at NCSD headquarters, contractor 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 21 

sites, and the Einstein collection systems at one federal agency; (3) manually 

reviewed selected routers, switches, and firewalls for Einstein system; and, 

(4) performed vulnerability scans on selected servers and firewalls, and 

password analysis for the Cybercop Portal. Our testing did not include an 

assessment of the Cybercop Portal application software. 

No high-risk vulnerabilities were detected on the Cybercop Portal devices 

tested. We found only two medium-security vulnerabilities on the Einstein 

system. When devices are not properly configured, the vulnerabilities could 

be exploited to gain inappropriate access to sensitive information. Subsequent 

to the completion of our audit work, NCSD personnel indicated that 

appropriate actions have been taken to address the vulnerabilities. As 

fieldwork had already been completed, we did not verify whether the 

vulnerabilities had been addressed. 

FISMA requires federal agencies to provide adequate controls for the data and 

information systems under their controls by (1) periodically assessing the risk 

and magnitude of the harm that could result from the unauthorized access, use, 

disclosure, disruption, modification, or destruction; (2) performing annual 

security testing to evaluate the effectiveness of security controls implemented; 

(3) providing security awareness training to inform personnel, including 

contractors and other users; and (4) planning and developing procedures to 

ensure the continuity of operations for the systems and data that support the 

operations and assets of the agency. 

Contingency planning is designed to mitigate the risk of system and service 

unavailability by focusing effective and efficient recovery solutions. Testing 

of contingency plans is performed to validate specific aspects of the plan, 

policies, procedures, systems, and facilities that will be used in the event of an 

emergency. Testing the plan identifies planning gaps and is also a training 

exercise to prepare recovery personnel for plan activation, which can improve 

plan effectiveness and overall agency preparedness. 

Recommendations 

 

We recommend that the Assistant Secretary for CS&C direct the NCSD 

Director to develop and implement procedures for all systems to: 

Recommendation #10: Ensure that certification and accreditation documents 

contain complete and accurate information to reflect the security postures of 

the system. In addition, security documents should be reviewed periodically 

and revised if necessary to ensure that agency officials are provided with the 

most accurate information to make credible, risk-based decisions on whether 

to authorize a system to operate. 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 22 

Recommendation #11: Test contingency plans, at least annually, to ensure 

business and computer operations can be maintained or restored in the event 

of an emergency, system failure, or disaster. 

Recommendation #12: Perform security testing annually to evaluate the 

effectiveness of controls implemented. 

Recommendation #13: Provide security awareness training to all contractors. 

Recommendation #14: Remedy all vulnerabilities identified for which risks 

have not been assumed. 

Management Comments and OIG Analysis 

CS&C concurred with recommendation 10. NCSD has a dedicated full-time 

Information Systems Security Officer to ensure that US-CERT systems, 

including Einstein and Cybercop Portal, meet FISMA requirements and 

continue to maintain the proper authority to operate. Based on feedback from 

our review, US-CERT is in the process of updating documentation for the 

2007 Einstein and Cybercop Portal re-accreditations. 

We agree that the steps that CS&C plans to take satisfy this recommendation. 

CS&C concurred with recommendation 11. US-CERT has successfully 

implemented a backup system for Einstein. In March 2007, Einstein’s 

back-up system was successfully tested, with additional testing planned in 

upcoming months. Additionally, US-CERT has requested resources to build 

and maintain an alternate site location for the Einstein program. A Continuity 

of Operations Plan has been developed and is operational for the Cybercop 

Portal to include off-site backup capability. 

We agree that the steps that CS&C has taken, and plans to take, begin to 

satisfy this recommendation. However, CS&C did not specifically address 

whether it has or will test the Cybercop Portal’s off-site backup capability. 

We maintain that contingency plans should be tested for all systems, at least 

annually. 

CS&C concurred with recommendation 12. US-CERT performs security 

testing annually in compliance with DHS policy. US-CERT is currently 

submitting documentation and conducting testing so that an additional 

one-year authority to operate the Einstein system may be granted. For the 

Cybercop Portal, annual security testing is performed in accordance with 

standard certification and accreditation procedures. The administrator of the 
Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 23 

Cybercop Portal conducts additional security testing to evaluate the 

effectiveness of the controls implemented. 

We agree that the steps that CS&C has taken, and plans to take, satisfy this 

recommendation. 

CS&C concurred with recommendation 13. NCSD provides and complies 

with all security awareness training required by DHS for all staff, including 

contractors. CS&C conducted its latest security awareness training for all its 

components on April 25, 2007. 

We agree that the steps that CS&C has taken satisfy this recommendation. 

 

CS&C concurred with recommendation 14. NCSD and US-CERT have been 

proactive in addressing identified vulnerabilities within Einstein and have 

resolved the necessary items. 

We agree that the steps that CS&C has taken begin to satisfy this 

recommendation. However, while CS&C addressed the vulnerabilities we 

identified during the audit, it did not specifically address whether it will 

remedy all vulnerabilities identified for which risks have not been assumed as 

a result of future security testing. 
Appendix A 

Purpose, Scope, and Methodology

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 24 

Purpose, Scope, and Methodology 

Our objective was to determine whether NCSD is working collaboratively 

with the public, private, and international entities to secure cyberspace and 

cyber assets and has effectively managed the implementation of The National 

Strategy to Secure Cyberspace. Specifically, we determined whether: 

(1) NCSD has adequately addressed the actions and recommendations in The 

National Strategy to Secure Cyberspace; (2) implementation plans are 

meeting NCSD’s strategic goals and priorities; (3) US-CERT is adequately 

performing its mission; (4) partnerships and coordination with other 

government agencies and the private sector are effective in securing 

cyberspace and cyber assets; and, (5) operational systems are in compliance 

with Federal Information Security Management Act requirements. 

To accomplish our audit, we interviewed selected NCSD personnel and 

contractors at its headquarters and contractor facilities. We interviewed 

selected members of the Government Forum of Incident Response and 

Security Teams, IT-Information Sharing and Analysis Center, IT-Sector 

Coordinating Council, and the National Cyber Response Coordination Group 

to obtain their perspective on how well NCSD is managing its mission and 

working with its cyber partners. These groups were selected since the 

members work with NCSD on a regular basis. We asked for feedback about 

NCSD’s priorities, communications and information sharing, frustrations, and 

positive aspects of NCSD. 

During the audit, we reviewed applicable security policies, procedures, and 

other appropriate documentation. In addition, we evaluated the quality of the 

certification process for two of its mission support systems (Einstein and 

Cybercop Portal) and determined whether the systems were accredited per 

OMB and DHS guidance. We further tested security controls over the two 

systems to ensure that effective controls have been implemented to protect the 

information stored and processed by the systems. We used network 

vulnerability assessment software (Tenable Network Security’s Nessus 

Vulnerability Scanner, Internet Security Systems’ Internet Scanner) to detect 

and analyze vulnerabilities on devices for the two systems at NCSD 

headquarters, one federal agency, and contractor sites. Upon completion of 

the assessments, we provided NCSD with the technical reports detailing 

vulnerabilities detected and remediation actions needed. 

We conducted our audit between October 2006 and January 2007 under the 

authority of the Inspector General Act of 1978, as amended, and according to 

generally accepted government auditing standards. Major OIG contributors to 

the audit are identified in Appendix E. 
Appendix A 

Purpose, Scope, and Methodology

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 25 

The principal OIG points of contact for the audit are Frank W. Deffer, 

Assistant Inspector General, IT Audits at (202) 254-4100 and 

Edward G. Coleman, Director, Information Security Audit Division at 

(202) 254-5444. 
Appendix B 

Management Comments to the Draft Report 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 26 
Appendix B 

Management Comments to the Draft Report 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 27 
Appendix B 

Management Comments to the Draft Report 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 28 
Appendix B 

Management Comments to the Draft Report 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 29 
Appendix B 

Management Comments to the Draft Report 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 30 
Appendix B 

Management Comments to the Draft Report 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 31 
Appendix B 

Management Comments to the Draft Report 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 32 
Appendix B 

Management Comments to the Draft Report 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 33 
Appendix B 

Management Comments to the Draft Report 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 34 
Appendix C 

NCSD Major Functions and Responsibilities 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 35 

NCSD Director 

The NCSD Director is responsible for issues related to the operation of the NCSD, such as human resources, 

policy, and budget, as well as participation in international initiatives. The director develops the overall 

strategic direction and priorities for the division, in line with CS&C goals and objectives. The director is 

responsible for managing US-CERT—which is a partnership between NCSD and the public and private 

sectors to make cyber security a coordinated, national effort; increase public awareness of cyber threats and 

vulnerabilities; and improve computer security preparedness and response to cyber threats. 

US-CERT Operations Branch 

NCSD’s US-CERT Operations branch focuses on situational awareness, analytical cells, and federal 

coordination. US-CERT is charged with protecting the nation’s Internet infrastructure by coordinating 

defense against and response to cyber attacks. US-CERT is responsible for analyzing and reducing cyber 

threats and vulnerabilities, disseminating cyber threat warning information, and coordinating incident 

response activities. A key component of US-CERT is the National Cyber Security Response System 

(Response System), which provides a nationwide, real-time collaborative information-sharing network to 

enable communication and collaboration among DHS and federal, state, local, and international government 

and law enforcement entities. Components of the Response System include the following: 

• The US-CERT Operations Center serves as a 24-hour-a-day/7-day-a week, real-time focal point for 

cyber security, conducting daily conference calls with U.S.-based watch and warning centers to share 

classified and unclassified security information. 

• The US-CERT Portal provides a Web-based collaborative system that allows US-CERT to share 

sensitive cyber-related information with members of government and industry. 

• The US-CERT Control Systems Security Center serves as an operational and strategic component of 

US-CERT’s capability to address the complex security issues associated with the use of control 

systems. 

• The US-CERT public web site provides government, the private sector, and the public with 

information they need to improve their ability to protect their information systems and infrastructures. 

• The National Cyber Alert System is to deliver targeted, timely, and actionable information to 

Americans to allow them to secure their computer systems.

• The National Cyber Response Coordination Group brings together officials from federal agencies to 

coordinate public/private cyber preparedness and incident response.

• The Government Forum of Incident Response and Security Teams is a community of government 

response teams that are responsible for securing government information technology systems. This 

forum works to understand and handle computer security incidents and to encourage proactive and 

preventative security practices. 

Outreach and Awareness Branch 

NCSD’s Outreach and Awareness branch is responsible for outreach, awareness, and messaging. The branch 

promotes cyber security awareness among the general public and within key communities, maintains 

relationships with governmental cyber security professionals to coordinate and share information about cyber 

security initiatives, and develops partnerships to promote public/private coordination and collaboration on 

cyber security issues. 

The branch is organized into three functional areas: Stakeholder Outreach, Communications and Messaging, 

and Coordination. The Stakeholder Outreach team serves to build and maintain relationships among and 

between industry, government, and academia in order to raise cyber security awareness and secure 

cyberspace. The Communications and Messaging team focuses on coordination of internal and external 

communications. 
Appendix C 

NCSD Major Functions and Responsibilities 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 36 

The Coordination team works to ensure collaboration on events and activities across NCSD and with other 

DHS entities, including the public affairs, legislative affairs, and private-sector offices and others, as 

appropriate. The team works to foster the department’s role as a focal point and coordinator for securing 

cyberspace and implementing The National Strategy to Secure Cyberspace. The International Affairs 

program engages in international outreach activities to build awareness about the global cyber risk, secure the 

critical information infrastructure, establish information sharing relationships, communications mechanisms, 

and collaborative arrangements, and institute collaborative arrangements for addressing critical information 

infrastructure protection issues. 

Law Enforcement and Intelligence Branch 

The Law Enforcement and Intelligence branch of NCSD has two primary responsibilities: managing the 

National Cyber Response Coordination Group and facilitating the coordination of law enforcement and 

intelligence cyber-related efforts for NCSD. This branch serves as a liaison to the law enforcement and 

intelligence communities and provides a mechanism for information sharing among the components 

concerned with cyber issues of law enforcement, intelligence, and the private sector. This information 

sharing includes all levels of information (classified, law enforcement sensitive, and unclassified). 

Strategic Initiatives Branch 

NCSD’s Strategic Initiatives branch is organized into seven programs with different responsibilities, as 

follows: 

• The Critical Infrastructure Protection Cyber Security program is responsible for developing a critical 

infrastructure protection plan for the IT Sector, including the Internet, that will identify critical assets 

and vulnerabilities, map interdependencies, and promote cyber awareness throughout other sector 

specific plans. 

• The Control Systems Security program is responsible for facilitating control system incident 

management and security awareness, establishing an assessment capability for vulnerability reduction 

and incident response, creating a self-sustaining security culture within the control systems 

community, focusing attention on the protection of legacy control systems, and making strategic 

recommendations for the future of control systems and security products. 

• The Software Assurance program presents a framework for promoting and coordinating efforts to 

improve the security, reliability, and safety of software. 

• The Training and Education program is responsible for promoting the development of an adequate 

number of effective cyber security professionals, enhancing cyber security capabilities within the 

federal workforce by identifying the skills and abilities necessary for specific job tasks, and working 

with other organizations to develop content standards for training products and for certifications. 

• The Cyber Exercise program is charged with improving the nation’s ability to respond to cyber 

incidents by creating, sponsoring, and learning from international, national, regional, and interagency 

exercises. The team is responsible for planning and coordinating cyber security exercises with 

internal and external DHS stakeholders. 

• The Standards and Best Practices/Research and Development Requirements program works to 

encourage technology innovation efforts. The team is responsible for identifying cyber security 

research and development requirements and cyber security standards issues, and for assembling and 

distributing information on best practices. 

• The Information Systems Security Line of Business program provides leadership and direction for 

improving information systems security services across the federal government. The programs works 

to achieve more consistent security management processes and controls across government through 

the reuse of proven best practices, and by promoting savings through reduced duplication and 

economies of scale for common hardware, software, and shared services.
Appendix D 

Federal Agency Incident Categories 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 37 

Category Name Description Reporting Timeframe 

CAT 0 Exercise/Network 

Defense Testing 

This category is used during state, federal, 

national, international exercises and approved 

activity testing of internal/external network 

defenses or responses. 

Not Applicable; this category is 

for each agency's internal use 

during exercises. 

CAT 1 Unauthorized Access In this category an individual gains logical or 

physical access without permission to a federal 

agency network, system, application, data, or 

other resource 

Within one (1) hour of 

discovery/detection. 

CAT 2 Denial of Service An attack that successfully prevents or impairs 

the normal authorized functionality of 

networks, systems or applications by 

exhausting resources. This activity includes 

being the victim or participating in the Denial 

of Service. 

Within two (2) hours of 

discovery/detection if the 

successful attack is still ongoing 

and the agency is unable to 

successfully mitigate activity. 

CAT 3 Malicious Code Successful installation of malicious software 

(e.g., virus, worm, Trojan horse, or other codebased malicious entity) that infects an operating 

system or application. Agencies are NOT 

required to report malicious logic that has been 

successfully quarantined by antivirus software. 

Daily 

Note: Within one (1) hour of 

discovery/detection if widespread 

across agency. 

CAT 4 Improper Usage A person violates acceptable computing use 

policies. 

Weekly 

CAT 5 Scans/Probes/Attempted 

Access 

This category includes any activity that seeks to 

access or identify a federal agency computer, 

open ports, protocols, service, or any 

combination for later exploit. This activity 

does not directly result in a compromise or 

denial of service. 

Monthly 

Note: If system is classified, 

report within one (1) hour of 

discovery. 

CAT 6 Investigation Unconfirmed incidents that are potentially 

malicious or anomalous activity deemed by the 

reporting entity to warrant further review. 

Not Applicable; this category is 

for each agency's use to categorize 

a potential incident that is 

currently being investigated. 
Appendix E 

Major Contributors to this Report 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 38 

Information Security Audits Division

Edward G. Coleman, Director 

Jeff Arman, Audit Manager 

Chiu-Tong Tsang, Audit Team Leader 

Jason Bakelar, Information Technology Specialist 

Charles Twitty, Auditor 

Tarsha Ross, Referencer 

Advanced Technology Division

David Hawkins, Senior Security Engineer 
Appendix F 

Report Distribution 

Challenges Remain in Securing the Nation’s Cyber Infrastructure 

Page 39 

Department of Homeland Security

Secretary 

Deputy Secretary 

Chief of Staff 

Deputy Chief of Staff 

General Counsel 

Executive Secretary 

Assistant Secretary for Policy 

Assistant Secretary for Public Affairs 

Assistant Secretary for Legislative and Intergovernmental Affairs 

Under Secretary for National Protection and Programs 

Assistant Secretary for Office of Cyber Security and Communications 

Chief Information Officer 

Deputy Chief Information Officer 

Chief Information Security Officer 

Director, Compliance and Oversight Program 

Director, National Cyber Security Division 

Director, DHS GAO/OIG Liaison Office 

Chief Information Officer Audit Liaison 

National Protection and Programs Audit Liaison 

National Protection and Programs Information Systems Security Manager 

Director, OIG Information Security Audit Division 

Chief Privacy Officer 

Office of Management and Budget

Chief, Homeland Security Branch 

DHS OIG Budget Examiner 

Congress

Congressional Oversight and Appropriations Committees, as appropriate 
Additional Information and Copies 

To obtain additional copies of this report, call the Office of Inspector General 

(OIG) at (202) 254-4100, fax your request to (202) 254-4285, or visit the OIG 

web site at www.dhs.gov/oig.

OIG Hotline 

To report alleged fraud, waste, abuse or mismanagement, or any other kind 

of criminal or noncriminal misconduct relative to department programs or 

operations, call the OIG Hotline at 1-800-323-8603; write to DHS Office of 

Inspector General/MAIL STOP 2600, Attention: Office of Investigations – 

Hotline, 245 Murray Drive, SW, Building 410, Washington, DC 20528; fax 

the complaint to (202) 254-4292; or email DHSOIGHOTLINE@dhs.gov. The 

OIG seeks to protect the identity of each writer and caller.