# Median Absolute Deviation for BGP Anomaly Detection
## 2 Materials and Methods
### Methodology
#### 2.1.2 Data Processing: Collection and Preprocessing  
The data processing phase of the study encompasses the collection and preprocessing of BGP data. The primary source of data for this research comes from the Reseaux IP Europeens Network Coordination Centre (RIPEC NCC) [24], which offers extensive public Internet routing data through the Routing Information Service (RIS). The data was obtained in the Multi-threaded Routing Toolkit (MRT) binary format [25], a standardized format developed by the Internet Engineering Task Force (IETF) for exporting various BGP protocol information, including messages, state changes, and Routing Information Base (RIB) contents [26]. Specifically, the study extracted routing data from Remote Route Collectors (RRC), focusing on RRC04 located at the CERN Internet Exchange Point (CIXP) in Geneva, Switzerland. For details on the extraction dates, total number of observation, and route collector information for each event, please refer to Table 2. The use of a robust and diverse dataset ensures that our research findings are substantiated with solid empirical evidence, thereby enhancing the reliability and validity of the results.  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Anomaly Event** & **Category** & **Occurrence (UTC)** \\ \hline Code Red 1 v2 & Indirect Anomaly & 2001-07-19 13:20:00–2001-07-20 00:00:00 \\ Slammer & Indirect Anomaly & 2003-01-25 05:31:00–2003-01-25 19:59:00 \\ Nimda & Indirect Anomaly & 2001-09-18 13:19:00–2001-09-19 10:59:00 \\ Moscow Blackout & Link Failure & 2005-05-25 04:40:00–2005-05-25 10:33:00 \\ TMnet & Direct Unintended Anomaly & 2015-06-12 08:43:00–2015-06-12 11:53:00 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summary of anomaly types and occurrence period.  
To construct the dataset for this study, a specialized Python script was developed. This script was specifically designed to parse and extract relevant attributes (see Table 3) from the raw BGP data, utilizing the Python API _PyBGPStream_, a well known tool for BGP data processing [27].  
The extraction phase primarily focuses on the type of UPDATE messages, specifically targeting a critical period during which these anomaly events were prevalent, ranging from 1 to 3 days. During these periods, both normal and anomalous behaviors were expected to be detected through the volume of announcements and withdrawals--these being the primary features extracted. By processing the data in 3-min batches, we calculated the exact number of announcements and withdrawals for each batch, with each batch representing a single row in our dataset, as illustrated in Figure 1.  
To enhance the dataset further, we included timestamps and record counts as additional parameters. Timestamps were converted from Unix format to a human-readable form. This conversion was important not only for facilitating the interpretation of the data but also for enabling a detailed analysis of temporal patterns. Furthermore, a record count was implemented to track each entry within the 3-min intervals, thereby organizing and simplifying the analysis of the BGP data's extensive volume.  
To facilitate a comprehensive understanding of the dataset structure and to guide researchers in accurately reproducing the analysis, Table 2 provides details regarding the anomalies event, including the specific extracted dates, the route collector used, and length of dataset per event.