# Network Device and Hardware Security - Do you know who's logged on to your machines?

*By Don S. Hutcheson,*

Chief Engineer and Chan S. Lim, Senior Security Architect, IBM Business
Consulting Services, IRS Web Hosting Project.

* *

# Introduction

 

This is the story of how one large Federal organization maneuvered
through the masses of confusing and often contradictory security
requirements to implement a secure Internet transaction system for its
constituency. It describes how the IRS is centralizing privileged user,
system and application logs in an attempt to hunt, deter and prosecute
unauthorized access to their new modernized infrastructure. In the
thousand words we have here, we will show you a clear, deliberate and
short path to success. Step1: clean functional audit requirements. Step
2: log centralization.

 

In early 2001, the IRS completed the logical design of its new
modernized infrastructure. Over 700 requirements were documented.
Approximately half were security related. The IRS decided that the best
way to implement the infrastructure was to outsource its Web Portal (the
front-end) and keep the large-scale processing internal to the IRS (the
back-end). This is how over 300 mostly government-related security
requirements came to be implemented in a commercial hosting facility.

 

# What are the goals?

 

IT professionals know that all systems have vulnerabilities. The best we
can do is limit the vulnerabilities and vigilantly watch for attacks.
Attacks may be against one device or many devices simultaneously.
Cumulative attacks can form patterns. Identify the pattern and you have
identified an attack. There are products that look for attack
patterns: 1) across a network and 2) on a machine. But what about
identifying patterns on other devices? Aren't internal attacks more
likely than external ones? In some cases, one undetected internal
security breach is all it takes to lose valuable information to a
terrorist, a fraud or a thief. Several undetected breaches may form the
pattern of a slow deliberate attack on a high value system, a system
that holds "The Crown Jewels."

 

So, along with traditional security tools, the IRS decided that it must
implement a rigorous set of "audit" requirements for every device in the
infrastructure. And, additionally, the IRS wanted to collect the audit
data for further analysis (and prosecutions). The bottom line:
***discover known attack scenarios, combinations of suspicious
activities, and patterns of events that identify malicious behavior as
early as possible***.

 

# Step 1: Clean Functional Audit Requirements

 

Technology continues to outpace written law enforcement and security
guidelines. Of the 300 or so security requirements, many came directly
from law enforcement manuals and other government sources. Detached from
their original source, requirements were used to create test plans for
all devices instead of only the devices that were intended by the source
document. In this confusion, we took the opportunity to analyze the
requirements.

 

A group of distinguished engineers from the IRS, IBM, CSC, Booz Allen
Hamilton, Mitre and Northrop Grumman took part in a study of the
functional audit requirements. The results were surprising in their
clarity.

 

Table 1 shows the functional audit requirements along the left hand
column and the type of device as column headings. Each device type was
assessed on a four-point scale. You might recognize this as the
"Consumer Reports" scale. For functional audit requirements we asked,
"What are the audit events we think need to be logged?" There were: 1)
operating events, 2) program execution events and 3) application events.
In addition, we asked, "What are the data elements that we want to
capture from these devices?" "Are there any impediments to
implementation (e.g., file reductions due filtering of data volume,
performance or device log settings)?" And are there any impediments to
using the data collected?" For example, "is the format encrypted or
binary requiring a special reader or analysis tool?"

 

Our goal with Table 1 is simple: to take a picture of the infrastructure
"as-is" and in subsequent generations of the infrastructure (the
"to-bes"), we want to fill the \"Consumer Reports\" dots in solid (i.e.
improve the audit security of the infrastructure).

 

**TABLE 1 AS-IS Functional Audit Requirements For FICTItIOUS
Implementation**

## Audit Events

![](media/image1.jpeg){width="3.3777777777777778in"
height="2.579861111111111in"}[\[SDB1\]](../../../../../../../../scox-cluster1/xshare$/XC%20-%20Citizen%20Services/XCI%20-%20Intergovernmental%20Solutions%20(OLD%20MG)/WWWSITE/INTERGOV/letter13/#_msocom_1)

 

# Step 2: Log Collection

 

For a system that, at peak, could generate 18 million sessions in a
30-day period, the log collection problem could be immense. Log
aggregation is always a problem in large networked systems. Device
management can be fragmented. Many administrators maintain many
different devices. Generally, the administrators control the device's
audit subsystem. The audit subsystem is responsible for recording events
such as starting a program, ending a program, rebooting a system, adding
a user, changing a password, and upgrading hardware and software. When
there are hundred of servers, network devices, and applications in a
large environment and each of them creates 10 -- 100 megabytes of audit
log data in several different formats, aggregation gets complicated. Our
job was to create an understandable and implementable log collection
design.

 

Although we had determined the number and type of log files to be
collected and determined their locations (Figure 1), we still had to
determine how the log files would get automatically routed to the
centralized server without breaking any of the segmented, segregated or
layered security defenses we had put in place.

 

Figure 1 Log Collection from Any Device Anywhere

 

![](media/image2.jpeg){width="3.36875in"
height="2.873611111111111in"}[\[SDB2\]](../../../../../../../../scox-cluster1/xshare$/XC%20-%20Citizen%20Services/XCI%20-%20Intergovernmental%20Solutions%20(OLD%20MG)/WWWSITE/INTERGOV/letter13/#_msocom_2)

 

First, we analyzed the log collection problem by making some assumptions
about the design: 1) all device and system logs will be stored in
central log servers, 2) the log servers are clustered for 24/7/365
availability and 3) all device clocks will be synchronized (i.e., NTP).

 

Second, we had to determine the size of an audit record generated for an
audit event and we had to determine the frequency of that audit event.
We created scenarios. The scenarios typified the actions of a user
during a session. The numbers of inbound and outbound events per session
per scenario were estimated. Multiplying log size per device times the
number of inbound and outbound events yields the log file increment per
session for the scenario. Multiplying by the expected number of sessions
per month yields file size by month for the scenario. These requirements
were then used to specify the delivery and storage systems necessary to
do the job.

 

# Step 3: Analysis and Prosecution

 

Analysis of the logs could lead to improved threat signatures and an
increase in prosecution. Remember, current threat detection tools don't
notice slow and methodical attacks. Clean audit requirements and
centralized storage are the first two steps. Analyzing the volumes of
data is the next step. If you are familiar with data warehousing
techniques, you know the next step and hopefully the next article.

 

*Contact Don S. Hutcheson at <dhutches@us.ibm.com> or Chan S. Lim at
<cslim@us.ibm.com>, if you'd like to learn more about audit
requirements, centralization of audit logs or e-Government on Demand.*