Employing Streaming Machine Learning for Modeling Workload Patterns in Multi-Tiered Data Storage Systems +
Footnote †: This paper is an extended version of our paper published in [PERSON] [PERSON]; [PERSON]; [PERSON]; [PERSON] Streaming Machine Learning for Supporting Data Prefetching in Modern Data Storage Systems. In Proceedings of the First Workshop on AI for Systems (AI&Sys 23), ACM, 2023, pp. 7-12. [PERSON]  
1 Department of Electrical Engineering and Computer Engineering and Informatics, Cyprus University of Technology, Limassol 3036, Cyprus, [EMAIL_ADDRESS] (E.R.L.F. ); [EMAIL_ADDRESS] (G.S.) [PERSON]  
1 Department of Electrical Engineering and Computer Engineering and Informatics, Cyprus University of Technology, Limassol 3036, Cyprus, [EMAIL_ADDRESS] (E.R.L.F. ); [EMAIL_ADDRESS] (G.S.) [PERSON]  
2  
[PERSON]  
2  
[PERSON]  
2  
[PERSON]DRESS]  
Footnote 2: [[https://www.sciencedirect.com/science/article/pii/article/](https://www.sciencedirect.com/science/article/pii/article/)]([https://www.sciencedirect.com/science/article/pii/article/](https://www.sciencedirect.com/science/article/pii/article/))  
performance of such policies is susceptible to the storage workload, often generated by numerous applications accessing thousands of files in parallel. While there have been several efforts to understand and exploit file access patterns on data storage systems to benefit caching and tiering policies [1; 2; 3; 4; 5; 6; 7; 8], the growing number of storage workloads with crescent complexity, such as machine learning, scientific workflows, and data science, poses a challenge to keep these policies up-to-date. One solution to this challenge is to enable data storage systems to learn workload patterns. In recent years, there have been many efforts to support caching and tiering policies using different machine-learning algorithms. Even though such policies can be used at various levels of the storage's software stack and operate on different data abstractions (e.g., memory pages, disk blocks, files), the employment of Machine Learning (ML) is similar across them. For instance, algorithms such as Neural Networks, Decision Trees, Random Forests, and Support Vector Machines are commonly trained with a set of data storage I/O requests to learn the workload patterns and then predict the value of a certain attribute of a page, block, file, or object (e.g., access frequency, next offset) that is required by a policy to make its decisions [9; 10; 11; 12; 13; 14; 15]. Even though modern data storage workloads reflect continuous and dynamic streaming of data requests, current approaches learn workload patterns by training their models offline in a batch-based fashion. This drives the need for systems to retrain their models regularly; a process that demands extra space to store new training data as well as additional processing to update the model, thereby hurting system performance. Deep Learning (DL) [16; 17; 18] further exacerbates the issue as it requires significant computational resources at scheduled intervals, leading to inefficient resource utilization. In addition, the models can become outdated between training cycles, reducing predictive accuracy for dynamic environments [19]. In this work, we investigate the use of streaming machine learning (as opposed to traditional batch-based models) to learn workload patterns that are useful for caching and tiering policies in multi-tiered, file-based storage systems. Streaming ML involves training models online by processing one training instance at a time and introduces various advantages, including (i) high efficiency during both training and using the models, (ii) high accuracy in predictions, and (iii) high adaptability to changing workload patterns [20]. We perform our analysis over a set of production traces provided by Huawei Technologies.