# Breaking Alert Fatigue: AI-Assisted SIEM Framework for Effective Incident Response
## 4 Methodology
### Correlation Problem
#### 4.2.1 Implementing Alert Correlation  
A two-step method can be employed to address the correlation problem efficiently. The first step utilizes a clustering algorithm to group alerts in \(\mathbf{X}\) based on their similarity. This algorithm partitions alerts into \(k\) (\(k\leq N\)) clusters using the numerical representations in \(\mathbf{X}\), thereby ensuring that alerts within each cluster are more similar to each other than to alerts in other clusters. In the second step, a segmenting algorithm is employed to divide alerts in each cluster into sets of candidate events (denoted \(E_{j}\)) that satisfy the three-tuple and timeout conditions. This procedure ensures that each candidate event comprises alerts exhibiting compactness in behavioral, spatial, and temporal aspects. Finally, to enable effective incident investigation, an event profile is generated for the identified group of alerts. The profile includes various information, such as attack type, attack time, affected assets, and attack methods. When clustering the filtered alerts into events, selection of the similarity threshold has a critical impact on the resulting groups, where a lower threshold reduces the number of events to be investigated but increases the complexity of alerts within each event, thereby making incident handling more challenging. Conversely, a higher threshold reduces the complexity of alerts in each event but increases the number of events that require investigation. Practically, events with high purity are preferable because they do not involve dealing with multiple types of alerts within a single event, which makes the investigation more efficient. Using numerical representations provided by one-hot encoding, which can result in a large number of replicated alerts, is equivalent to setting the similarity threshold to 1 in a hierarchical clustering algorithm. In our implementation, we leveraged this feature by requiring alerts in the same cluster to be homogeneous in terms of the one-hot encoding features for the selected categorical fields. This ensures that the resulting events comprise alerts with consistent attributes, thereby improving the efficiency and accuracy of the incident investigation. The segmentation algorithm used to divide a group of alerts into events ensures that alerts in the same event exhibit spatial compactness. Many existing NIDSs group alerts into events based on a four-tuple [source IP, destination IP, source port, destination port] or a three-tuple [source IP, destination IP, destination port] for ease of implementation. After carefully observing a substantial number of alerts and using the ATG as a visualization aid, we discovered that using a three-tuple consisting of [source IP, destination IP, service port] combined with a strict similarity criterion resulted in a noteworthy improvement on the grouping of security alerts into separate events. Additionally, our choice of the service port over the source or destination port corresponds to the incident handling expertise of SOC operators, as evidenced by the escalation set utilized during the assessment. Further details can be found in Appendix A.2. Here, the service port refers to the port whose vulnerability is exploited in the attack campaign. The source port on the attacker side is typically selected randomly, which makes it less useful for correlating alerts from the same attack campaign. For example, in a DoS attack against a web server, the source port may differ, but the service port (e.g., port 80 for HTTP services and port 8080 for HTTPS services) remains the same. Grouping alerts based on the proposed three-tuple provides an appropriate level of granularity for segmenting alerts into events. We propose a simple technique to identify the service port in a group of alert messages produced by an IDS. Here, we assume that the alerts originate from a pair of IP addresses and that they occur within a short time span, which allows them to be merged into a single event.