## 4 Dataset Description and Preprocessing
### Preprocessing  
Following data collection, preprocessing steps were undertaken to enhance the dataset's quality. This process includes data cleaning to address missing values and duplicates. Feature engineering also uses correlation feature selection to determine variables that could improve model performance, such as volume and market capitalization, as presented in Figure 2. Additionally, the standard scaler preprocessing technique was employed to normalize all cryptocurrency datasets, adjusting them to have a mean of 0 and a standard deviation of 1. This step is essential for enhancing the effectiveness of machine learning algorithms [40]. The threshold value for labeling abnormal values is derived from the Z-Score, a statistical measure that quantifies the number of standard deviations a data point is from the mean of a dataset. The Z-Score is calculated using a rolling window approach, where a rolling mean and standard deviation are computed over a specified period (in this case, 30 days). This ensures that the Z-Score dynamically adjusts to local trends and variations in the data, making it suitable for time-series analysis. The decision to use a 30-day rolling window aims to balance the identification of medium-term trends and outliers while reducing the noise from short-term fluctuations. This time frame corresponds with the typical expectation that financial market patterns may display cyclical trends over monthly periods. We acknowledge that this window size might not adequately represent sudden changes. Consequently, additional tests were performed using rolling windows of 15 and 7 days to assess their effects on the results. We recalculated the rolling mean and standard deviation for these new window sizes and analyzed how these changes impacted the Z-Score, anomaly detection, and overall model performance. Algorithm 1 identifies abnormal data points based on their deviation from recent trends using a financial instrument's closing prices \(C(t)\). The analysis was carried out by computing rolling statistics and normalizing the data using the Z-Score. The algorithm relies on three key components: the rolling mean \(\mu(t)\), rolling standard deviation \(\sigma(t)\), and Z-Score \(Z(t)\). The rolling mean \(\mu(t)\) represents the local average of the time-series data over a specified window size \(w\), smoothing short-term fluctuations to highlight longer-term trends. Similarly, the rolling standard deviation \(\sigma(t)\) quantifies the dispersion of the data around the rolling mean, capturing the variability within the same window. Using these rolling statistics, the algorithm adapts to local trends and avoids the influence of earlier data, which might no longer be relevant. The Z-Score \(Z(t)\) is a normalized measure that quantifies the deviation of a data point from the rolling mean in units of the rolling standard deviation. This normalization enables a consistent comparison of deviations across different points in time, even when the scale or variability of the data changes. The algorithm detects abnormalities by evaluating whether the absolute value of the Z-Score exceeds a predefined threshold \(z_{\text{threshold}}\). Data points for which \(|Z(t)|>z_{\text{threshold}}\) are flagged as abnormal, and an abnormality indicator \(A(t)\) is assigned a value of 1, while normal points are assigned a value of 0. This approach allows for the detection of significant deviations, regardless of whether they are above or below the rolling mean. The rolling window approach ensures that the algorithm is sensitive to recent data, making it well suited for applications where trends and patterns evolve over time. Common use cases include financial data analysis, where sudden price movements can signal market anomalies, or sensor data monitoring, in which deviations may indicate faults or unusual events.