## 6 Streaming Machine-Learning Models
### Model Selection  
During model selection, the objective is to select the model with the best prediction accuracy to performance ratio. For the regression problems, we are looking for an algorithm that minimizes the Mean Absolute Error (MAE), a commonly used metric in regression, while being sufficiently fast. We selected a set of popular streaming regression algorithms available in the MOA framework [20] (v23.04.0) that are representative of different classes of regressors:  
1. K-Nearest Neighbors (KNN) predicts the target value for a new data point by finding the \(k\) (e.g., 10) most similar (nearest) instances from the latest \(W\) (e.g., 1000) instances of the incoming data stream based on a distance metric (e.g., Euclidean distance). 2. Adaptive Random Forest (ARF) is an ensemble learning method that consists of multiple streaming decision trees (e.g., Hoeffding Trees), each trained on different subsets of the data stream. 3. Support Vector Regression (SVR) relies on a subset of the streaming training data, called support vectors, to incrementally define the decision function that minimizes the error within a predefined margin (epsilon). 4. Linear Regression with Stochastic Gradient Descent (LR-SGD) fits a linear relationship between input features and a target variable by minimizing the difference between predicted and actual values. The model parameters are updated incrementally using SGD. 5. Linear Regression with Adaptive Gradient Algorithm (LR-ADA) works the same as LR-SGD, but the model parameters are updated incrementally using ADA instead of SGD. All of the models mentioned above are designed to adapt to concept drift, i.e., changes in the underlying data distribution over time, by incorporating a variety of mechanisms, such as using a fixed-size sliding window of recent data points (KNN), replacing a tree when its performance degrades (ARF), adjusting the support vectors incrementally (SVR), and incorporating techniques like learning rate decay or forgetting mechanisms (LR-SGD, LR-ADA). We executed these regression algorithms to predict the Next Offset and the Next File Hotness targets. Table 3 presents the MAE and execution time for Next Offset, while the results for Next File Hotness are very similar. Regarding prediction performance, KNN has the lowest MAE because it performs well in common, low-variance cases such as sequential workloads (which appear in many traces). On the other hand, ARF ranks second lowest for both targets. ARF handles outliers and complex relationships better, even with a slightly higher MAE. LR-SGD and LR-ADA have a slightly higher MAE, whereas SVR has the highest MAE and clearly fails to make good predictions. On the other hand, the execution time is much higher for KNN compared to the other algorithms, while ARF had the second highest execution time, revealing a clear tradeoff between prediction performance and execution time. Based on the above results, ARF was selected as the algorithm to use for the two regression problems. The decision was driven by two key factors. First, KNN has a significantly higher overhead (1.6-2.5\(\times\)) compared to ARF. The second, and perhaps the most important reason, is related to the inner workings of KNN. Specifically, KNN computes the distance between the current sample and the available centroids, matching the sample to the nearest centroid within a time window. This drives KNN to always under-predict values for the Next Offset. For Trace 1, it under-predicts by about 16k, but for Trace 4 by typically 1-2 MB (because of larger files). For classification, the Hoeffding Tree (HT) [20] was compared against the Adaptive Random Forest of Hoeffding Trees (ARF-HT). HT is a state-of-the-art, memory efficient decision tree designed for streaming data, capable of learning from large datasets. It exploits the fact that small samples can be enough to choose an optimal splitting attribute.