# Presentation: 417820

## Host and User Level Security

**Host and User Level Security**
**Once again the battleground...**
- Stephen Lau / Scott Campbell
- NERSC Center Division, LBNL
- April 20, 2005

## Overview

**Changing Security Environment**
**Design Patterns**
**Classic Security Model**
**Evolving Security Threat Model**
**Real World Incident**
**Credential Theft / One Time Passwords**
**Kernel Rootkits**
**Possible solutions**
**The Future**
**Summary**
**Contact Information**

## Changing Security Environment

**Events of the past year reinforces the concept of evolving security threats.**
**Countermeasures need to evolve to keep pace.**
**Systems are becoming more complex**
**Increase in interdependencies**
**Access methods changing**
**More mobility**
**Different types of platforms**

## Design Patterns

**Common solutions for similar problems**
**Arises out of need for elegant, but simple, reusable solutions.**
**Determine commonalities and devise solution to fit similar problems**
**Classic example: **
**Reusable object oriented code.**

## Classic Security Model

**Design pattern for computer security**
**Service offered at DMZ**
**Internal network is protected via firewall or other perimeter protection**

## Classic Security Model

**Classic Design Pattern that addresses *some* well defined problems **
**Widely deployed**
**Based on key assumption:**
**Threat model is fairly static**
**But:**
**Attackers have adapted their attacks to counter this model**

## Attackers are Adapting

**Evolving security threat model**
**Nature of the opponent is changing**
**Becoming difficult to determine users from attackers**
**Users have become a vector**
**Clicking on email attachments**
**‘****Phishing’ attacks**
**Attackers have become persistent**
**Target specific users / systems repeatedly**
**Wait for vulnerability to appear**
**Attackers look and behave like users**

## Stolen Credentials

**One time passwords are *supposed* to minimize this problem**
**#1 is legitimate, #2 is an attacker**
**Mar 10 18:39:41 2.3.4.5 Message forwarded from host609: sshd[74334]: Accepted password for user1 from 1.2.3.XXX port 45674 ssh2**
**Mar 11 11:00:50 2.3.4.6 Message forwarded from host013: sshd[117314]: Accepted password for user1 from 1.2.3.XXX port 46570 ssh2**** **
**Difficult to distinguish between the authorized and unauthorized**
**Network filtering/analysis has limited affect**
**Widely used in recent attacks.****	**

## Cross Site Issues

**Users at Science Labs depend upon collaboration.**
**Most often have access to multiple systems at multiple sites.**
**Many of these outside of our control**
**“****Single site” incidents can quickly blossom into complex multi-site incidents.**

## Real World Example

**DOE**
**host**

**.edu**
**host**

**.net**
**host**

**.edu**
**host**

**[1]**

**[4]**

**[6]**
**3/11/4 7:16**

**.edu**
**host**

**[2]**

**Other**
**host**

**[3]**
**3/4/416:02**

**[5]**
**3/10/4 10:23**

**Incident from 2004**
**Crossed many sites**
**Initial Analysis**

## Real World Part II

**DOE**
**host**

**.edu**
**host**

**.net**
**host**

**.edu**
**host**

**[1]**

**[4]**

**[6]**
**3/11/4 7:16**

**.edu**
**host**

**[2]**

**Other**
**host**

**[3]**
**3/4/416:02**

**[5]**
**3/10/4**
**10:23**

**NASA**
**host**

**.mil**
**host**

**.mil**
**host**

**.mil**
**host**

**.edu**
**host**

**DOE**
**host**

**DOE**
**host**

**[14]**
**3/12/4**
**11:08**

**[8]**
**3/11/4 12:29**

**[17]**
**04/26/04**

**[13]**
**3/11/4**

**[7]**
**3/11/4, 7:23**

**[9]**

**[10] **=<03/11/4

**[11]**
**File marked 3/10/04 10:52**

**[12]**
**3/13/4, 11:30**

**[15]**
**3/14/4**
**12:27**

**[16]**
**3/18/4**

## Real World Part III

**Even more hosts were involved.**
**Too many to show on a slide!**
**Attackers were persistent.**
**Kept trying same accounts / systems**
**We only saw one part of the puzzle**
**Many more that we don’t know about**

## Scan Detection Data

**Scans detected at LBNL/NERSC**
**Over a 12 hour period**
**Number of unique IP addresses seen**
**Almost 10% of the scanning addresses seen by both address ranges**

## Credential Theft and One Time Passwords

**OTP solutions address traditional problem of credential theft**
**Sniffed passwords / Readable SSH Key**
**Attackers are adapting**
**No longer stealing and reusing credentials**
**Accessing authenticated user sessions as a *****service***
**Attacker rides in on user authenticated session**
**Session hijacking**

**Host**

**Compromised**
**Host**

**Attacker**

**User**

## Credential Theft and One Time Passwords

**OTP solutions address traditional problem of credential theft**
**Sniffed passwords / Readable SSH Key**
**Attackers are adapting**
**No longer stealing and reusing credentials**
**Accessing authenticated user sessions as a *****service***
**Attacker rides in on user authenticated session**
**Session hijacking**

**Host**

**Compromised**
**Host**

**Attacker**

**User**

**Unsuspecting user**
**authenticates**
**to Host with OTP**

## Credential Theft and One Time Passwords

**OTP solutions address traditional problem of credential theft**
**Sniffed passwords / Readable SSH Key**
**Attackers are adapting**
**No longer stealing and reusing credentials**
**Accessing authenticated user sessions as a *****service***
**Attacker rides in on user authenticated session**
**Session hijacking**

**Host**

**Compromised**
**Host**

**Attacker**

**User**

**Attacker hijacks**
**or duplicates**
**existing session.**
**User suspects nothing. **

## Credential Theft and One Time Passwords

**OTP solutions address traditional problem of credential theft**
**Sniffed passwords / Readable SSH Key**
**Attackers are adapting**
**No longer stealing and reusing credentials**
**Accessing authenticated user sessions as a *****service***
**Attacker rides in on user authenticated session**
**Session hijacking**

**Host**

**Compromised**
**Host**

**Attacker**

**User**

**Attacker launches local **
**attack against end host.**

## Credential Theft and One Time Passwords

**OTP solutions address traditional problem of credential theft**
**Sniffed passwords / Readable SSH Key**
**Attackers are adapting**
**No longer stealing and reusing credentials**
**Accessing authenticated user sessions as a *****service***
**Attacker rides in on user authenticated session**
**Session hijacking**

**Host**

**Compromised**
**Host**

**Attacker**

**Lather, rinse, **
**repeat.**

**User2**

## Credential Theft and One Time Passwords

**OTP address a well defined problem.**
**Problem:**
**Attackers are quickly creating new class of attacks that are harder to deal with!**
**Once again, no magic solutions!**

## Kernel Rootkits

**Ability to control a system without administrator being aware**
**Defeats most local monitoring systems**
**You can not trust your operating system**
**Many perform process hiding**
**Example from SK Rootkit:**
- bash-2.05$ ps -aux | grep vi
- scottc 13779 1.6 0.0 5140 2336 pts/2 S 16:54 0:00 vim x
- scottc 13781 0.0 0.0 1736 576 pts/3 S 16:54 0:00 grep vi
- bash-2.05$ ./sk i 13779
- /dev/null
- Detected version: 1.3b
- Pid 13779 is hidden now!
- bash-2.05$ ps -aux | grep vi
- scottc 13785 0.0 0.0 1736 580 pts/3 S 16:54 0:00 grep vi
- bash-2.05$

## Kernel Rootkits

**Some rootkits enable remote access**
**Example:**
**Rootkit can listen on port 22 for specifically crafted packet to arrive.**
**When it arrives, rootkit establishes outbound connection back to host with root shell.**
- bash-2.05$ ./login -h localhost -d 22
- /dev/null
- Listening to port 32774
- password:
- Trying 127.0.0.1:22...
- Trying...Et voila
- Server connected. Escape character is '^K'
- /dev/null
- [root@arnor .sk12]#

## Kernel Rootkits Example

**Target**
**Host**

**Attacker**

**Host**

**Attacker installs rootkit on Target Host **

## Kernel Rootkits Example

**Attacker**

**Host**

**Target Host now looking**
**for crafted packet to arrive on port 22 **

**Target**
**Host**

**22**

## Kernel Rootkits Example

**Attacker**

**Host**

**Attacker sends crafted packet to Target Host on port 22.**
**From Target host’s point of view looks legitimate.**

**Target**
**Host**

**22**

## Kernel Rootkits Example

**Attacker**

**Host**

**Target Host connects back with a**
**root shell on a different port.**
**Target host is now compromised.**

**Target**
**Host**

**22**

## Kernel Rootkits

**Widely used by attackers**
**Great way to credential mine or attack user sessions!**
**Not enough time to go into detail.**
**Exercise left to viewer to explore in more detail.**
**http://www.phrack.org/phrack/50/P50-05 **
**http://www.phrack.org/phrack/51/P51-05**

## What To Do

**Expect attackers to quickly devise ways around any “solution”**
**Expect that some of your systems / accounts are ******already compromised**********
**How does one detect them and how do you minimize the damage?**
**Security posture needs to be flexible**
**Attackers are constantly changing, we need to be too.**
**Network monitoring / perimeter defense not enough**
**Need to combine network data with host level data**
**Get the “whole” picture**
**Grow relationships with other sites**
**Cross site incidents are becoming quickly commonplace**
**Other sites’ compromise affect you!**

## Some solutions

**Some solutions we’re pursuing:**
**Deployment of center wide syslogging**
**Correlated with network IDS**
**Instrumenting SSH**
**Cross Site IDS correlation**
**Development / deployment of rootkit monitoring**

## NERSC

**High performance computational center**
**http://www.nersc.gov**
**Part of LBNL / unclassified facility**
**Direct peering with ESNet**
**Large global user base**
**Almost no users on site**
**Mix of citizens and foreign nationals**
**Network perimeter**
**Bro intrusion detect system**
**http://www.bro-ids.org/ **
**Snort IDS**
**Heterogeneous systems**
**Many systems unique**
**Clusters, workstations, HPC systems, Mass Storage, etc.**
**Ranges from Windows to HPSS to AIX**

## Syslogging

**Goal for all systems within NERSC to centrally log**
**Unix, windows systems, etc**
**Correlate network data captured from Bro IDS with syslog data.**
**Allows one to see initial connection all the way to host level activity.**
**Ability to flag data of interest for more detailed analysis.**
**Has been key in forensic analysis of incidents.**
**Problems:**
**Not all syslog messages are created equal**
**Lots of noise in data.**
**Tools to analyze syslog data are not well developed (yet).**

## Instrument SSH

**Problem with SSH is can’t see what user is doing.**
**Stupid encryption! ******
**Solution:**
**Do what attackers are already doing.**
**Backdoor SSH.**
**All user is activity is mirrored out to a central server.**
**Data can be used for real time analysis or forensics.**
**Correlate with existing syslog and network data.**
**Need to address:**
**Privacy issue**
**Potential security implications**

## Cross Site Correlation

**Most of our incidents are cross site**
**External host is compromised**
**User uses external compromised host to access NERSC.**
**D’oh!**
**Bro to Bro communication**
**Many attackers will follow legitimate usage patterns**
**Ride in on compromised account.**
**Goal is to correlate data across Bro systems monitoring different sites.**
**Alert other sites to activity seen at one site.**
**Act as an early warning system.**
**Track attacker activity across different sites.**

## Summary

**Attackers quickly adapt to any “solution”.**
**Becoming harder to distinguish between legitimate users and attackers.**
**Network protection no longer enough!**
**Need to consider entire process from external host all the way to local host.**
**Need to approach problem with the knowledge that some of your systems / accounts are and will ****always**** be compromised. **

## Contact

**Stephen Lau**
**NERSC/LBNL****Email: slau@lbl.gov****Phone: +1 (510) 486-7178****After Hours: +1 (510) 486-8600****NERSC Security Contact: security@nersc.gov****PGP Key Fingerprint:****44C8 C9CB C15E 2AE1 7B0A 544E 9A04 AB2B F63F 748B**
**Scott Campbell**
**NERSC/LBNL****Email: scampbell@lbl.gov****Phone: +1 (510) 486-6986****After Hours: +1 (510) 486-8600****NERSC Security Contact: security@nersc.gov****PGP Key Fingerprint:****89F9 593E 2381 88B2 B270  78B2 2B63 E5AB C07B 641C**** **