## 4 Dataset
### Data Acquisition
The models we chose are well-suited for classification tasks and have shown effectiveness in cybersecurity domains, especially when dealing with large datasets and multiple features. The following models were evaluated:

1. Random Forest (RF): A robust ensemble learning method that builds multiple decision trees and aggregates their results. RF well suited for this system due to its ability to handle large datasets with a mixture of features types and its strength in estimating feature importance. 2. XGBoost: Similar to RF, XGBoost is an ensemble method; however, it uses a gradient boosting framework in which it builds trees sequentially to improve model accuracy. It is known for its high performance, speed, and ability to handle complex patterns, which is crucial for accurately classifying user risk. 3. Support Vector Machine (SVM): A powerful model for classification problems, particularly when data points are not linearly separable, SVM works well in high-dimensional spaces, making it effective for identifying risky behavior based on a variety of input features. 4. Logistic Regression (LR): An interpretable model that provides clear probabilities for classification. Given its simplicity and ease of implementation, it serves as a good baseline for comparison with more complex models such as RF and XGBoost. ### Evaluation Metrics

To assess the performance of the ML models, we employed several key metrics, including the accuracy, precision, recall, F1-score, and confusion matrix. Each of these metrics provides insight into different aspects of a model's classification ability. Accuracy is a general measure of how well the model classifies all instances, defined as the ratio of correctly predicted cases (true positives and true negatives) to the total number of predictions. Mathematically, accuracy can be expressed as follows:

\[\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}\]

where \(TP\) denotes true positives, \(TN\) denotes true negatives, \(FP\) denotes false positives, and \(FN\) denotes false negatives. Precision focuses on the reliability of positive predictions, measuring the proportion of true positives among all predicted positives. This metric is particularly important when the cost of false positives is high. Precision is provided by

\[\text{Precision}=\frac{TP}{TP+FP}.\]

Recall, also known as sensitivity, quantifies a model's ability to capture all relevant instances within a particular class. It is the ratio of true positives to the sum of true positives and false negatives, and can be formulated as follows:

\[\text{Recall}=\frac{TP}{TP+FN}.\]

The F1-score balances precision and recall by computing their harmonic mean, offering a single metric that considers both false positives and false negatives. This is particularly useful when there is an uneven class distribution. The F1-score is represented as follows:

\[\text{F1-Score}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{ Precision}+\text{Recall}}\]

Finally, the confusion matrix provides a comprehensive view of a model's performance by displaying the average distribution of true positives, false positives, true negatives, and false negatives across all anomalous behaviors. This matrix enables a more granular understanding of the model's behavior in detecting different behaviors. ### ML Results

In this study, the dataset was divided into training (70%), validation (15%), and testing (15%) sets to ensure robust model evaluation, prevent overfitting, and provide reliable results. The training set, comprising 70% of the data, was allocated a larger proportion to ensure that the machine learning models had sufficient data to effectively learn patterns and relationships. A large training set is crucial for models to generalize well to unseen data, as it allows them to capture complex behaviors and reduces the likelihood of underfitting. The validation set, accounting for 15% of the data, was used to tune the model parameters and select the best-performing model during the training process.