# Improving Threat Detection in Wazuh Using Machine Learning Techniques
## 3 Materials and Methods
### Data Collection, Preprocessing, and Feature Engineering  
The dataset consists of log data, predominantly stored in plain text or CSV formats. Logstash is utilized to parse these logs, converting them into JSON format for structured processing. The processed logs are then archived in Elasticsearch, a NoSQL database, which serves as a core component of the ELK Stack. Furthermore, the log data is transmitted to ML models as JSON objects through API integrations, enabling advanced analytical processing. The alerts.json dataset was selected due to its wide range of data points, essential for training ML models aimed at identifying patterns and distinguishing between normal and anomalous behaviors [16]. This helps in developing tools that can enhance threat detection accuracy and reduce false positives. Using the alerts.json data, machine learning models were built and refined to identify patterns indicative of false positives and improve detection criteria and overall system accuracy. This process involved not only data collection but also its comprehensive analysis and preparation for use in model training [17]. Our dataset was derived from Wazuh alerts.json files collected over a three-month period in a controlled environment where various attack scenarios were simulated. The dataset comprises 15,427 security events collected from Wazuh agents deployed on 12 different endpoints (8 Windows, 3 Linux, and1 macOS). The network environment was designed to simulate a mid-sized enterprise with segmented networks, including DMZ, internal networks, and a cloud component. The network environment was meticulously designed to simulate a realistic enterprise architecture with segmented networks, including a demilitarized zone (DMZ) hosting web services and email servers, internal corporate networks with workstations and file servers, and cloud components integrated through VPN connections. Security events were generated through systematic simulation of attack scenarios based on the MITRE ATT&CK framework, ensuring comprehensive coverage of contemporary threat techniques and tactics. Each attack scenario was executed multiple times under varying conditions to capture behavioral variations and ensure robust model training, while legitimate user activities were simultaneously conducted to provide realistic baseline behavior patterns. The dataset consists of log data, predominantly stored in plain text or CSV formats. Logstash is utilized to parse these logs, converting them into JSON format for structured processing. The processed logs are then archived in Elasticsearch, a NoSQL database, which serves as a core component of the ELK Stack. Furthermore, the log data is transmitted to ML models as JSON objects through API integrations, enabling advanced analytical processing. The alerts.json dataset was selected due to its wide range of data points, essential for training ML models aimed at identifying patterns and distinguishing between normal and anomalous behaviors. Feature engineering is a pivotal step in our machine learning pipeline, involving the transformation of raw security logs into structured and meaningful representations suitable for model training. Leveraging domain knowledge, we identified, extracted, and encoded the most informative attributes from Wazuh logs to enhance model performance and detection accuracy. The goal was to convert log data--often in textual or semi-structured formats--into numerical features without losing the semantic context critical for accurate threat classification. We extracted and processed the following key features from Wazuh logs:  
1. Day, Hour, Minute (numerical): Temporal components normalized from timestamps, enabling modeling of event time patterns;
2. Event Type (categorical): Event category encoded using label encoding or one-hot encoding, depending on cardinality;
3. Event Name (categorical): Event subtype or name, encoded similarly to Event Type;
4.