We present a payload-based anomaly detector, we call PAYL, for intrusion detection. PAYL models the normal application payload of network traffic in a fully automatic, unsupervised fashion. The method we choose is very efficient; our goal is to deploy the detector in high bandwidth environments either on a firewall, a network appliance, a proxy server or on the target hosts. We first compute during a training phase a profile byte frequency distribution and their standard deviation of the application payload flowing to a single host and port. We then use Mahalanobis distance during the detection phase to calculate the similarity of new data against the pre-computed profile. The detector compares this measure against a threshold and generates an alert when the distance of the new input exceeds this threshold. The model is host-and port-specific and is conditioned on the payload length. Thus, a set of profiles are computed for every possible length payload. A second phase clusters the profiles to increase accuracy and decrease resource consumption. The method has the advantage of being very fast to compute, is state-less and does not parse the input stream, generates a small model, and can be easily modified to an incremental online learning algorithm so that the model is continuously updated to maintain an accurate normal model in real-time. The modeling method is completely unsupervised, and is tolerant to noise in the training data. Furthermore, the method is also resistant to mimicry-attack; attackers would need to model the site's normal payload distributions in order to pad their poisoned payload to go unnoticed by the detector. We demonstrate the surprising effectiveness of the method on the 1999 DARPA IDS dataset and a live dataset we collected on the Columbia CS department network. In once case nearly 100% accuracy is achieved with .1% false positive rate for port 80 traffic. The payload anomaly detector is designed to be a component in an integrated and correlated detection system including other detectors to mitigate false positives, and to enrich the information available from all detectors to defend against zero-day worms and new attacks exploits.
Wang et al. present PAYL, a system that searches for anomalies in the payload of network packets in order to detect attack traffic REF .
Anomalous Payload-Based Network Intrusion Detection
RAID
Computer Science