## 7 Experimental Evaluation
### Streaming ML Scalability and Adaptability  
This section aims to provide an in-depth evaluation of the proposed streaming ML models regarding scalability and adaptability. For this purpose, we concatenated the 17 traces into one extensive trace containing 27.2 million file requests (testing scalability) and sudden changes to the workload access patterns at the trace boundaries (testing adaptability). The file requests are submitted by almost 50 k different applications with  
Figure 14: F1 score for rule-based approach against streaming ML for _Next File Hotness Class_. Figure 13: F1 score for rule-based approach against streaming ML for _Next Offset Class_. varying intensity, with the underlying storage system yielding up to 20.3 k IOPS and 3.5 GB/s throughput. We extracted the selected features described in Section 4 from the trace and generated 14.5 million training instances for the Next Offset case and 9.1 million for the Next File Hotness case. We used prequential evaluation to evaluate the streaming ML models in terms of F1 score and MAE (for classification and regression, respectively), model size, and execution runtime. For comparison purposes, we also evaluated the corresponding batch-based ML models using the 3 rd scenario (as it yielded the highest predictive performance among the batch scenarios; recall Section 7.2), where we retrain the model every one million training instances using all the instances up to that point. Figure 15 shows the F1 score, model size, and runtime (for testing and training per instance) for the Next Offset Class case. Regarding the F1 score, we observe that the streaming ML model is able to quickly achieve and maintain a high score of around 99.5%, with two notable exceptions: near 4 million instances (when Trace 15 begins) and near 5.4 million instances (when Trace 16 begins). As these traces have some different access patterns not seen previously, the performance of the model drops to 93% but recovers quickly (as seen in Figure 15a), highlighting the strong adaptability of the streaming ML model. On the other hand, the batch model experiences multiple performance drops down to 90-92% F1 score for longer durations and recovers after retraining the model every one million instances to achieve an F1 score of up to 98%. Figure 15: The (**a**) F1 score, (**b**) model size, and (**c**) runtime for the _Next Offset Class_ case using all the training instances from the 17 traces consecutively. The red points show when a trace ends and the next one begins. The size of the ML streaming model shown in Figure 15b increases linearly to 600 KB during the first 4 million training instances and sublinearly thereafter as the model grows more slowly to 800 KB. This observation applies to the batch ML model as well. Similarly, we observe some variations in the runtime in Figure 15c that are more intense at the beginning of the trace or near the trace boundaries due to the more active training and growth of the model. Afterward, the runtime stabilizes to around 0.021 ms per record. However, what is not shown in these graphs are the additional overheads of the batch-based approach: (1) 350 MB of memory is required for storing one million records between each retraining and (2) the first training takes 16.8 s but the time increases linearly and reaches 231 s for training with 14 million instances. As the streaming ML model trains whenever a new training instance is generated, there is no need for storing the instances or retraining the model periodically; hence, it is more scalable than the batch-based approach. Figure 16 shows the MAE, model size, and runtime for the Next Offset regression case. Unlike the classification case, the batch-based regression model exhibits a very unstable predictive performance characterized by high oscillations of MAE between 0.02 and 0.93 (recall the target is normalized in the range 0 to 1), as seen in Figure 16a. Occasionally, the MAE reduces sharply after some retraining sessions, but shortly after the oscillations resume.