## 4 Experimental Results  
To validate the effectiveness of the suggested data shuffling scheme, we conduct experiments with data reference traces collected while running neural network workloads introduced in Section 2. Specifically, we capture block reference traces while conducting the classification problem of ImageNet100 and Food101 with the ResNet50 model [23]. For the eviction algorithm of the buffer cache, we use the least recently used (LRU), as it is the most commonly adopted in the buffer cache [12]. Figure 5 shows the cache hit rate of the buffer cache when the proposed shuffling scheme is used in comparison with the original shuffling used in PyTorch. In the graph, the \(x\)-axis is the size of the buffer cache in a relative scale. The 100% cache size implies the capacity that can load all data blocks simultaneously during the full training periods. As can be seen from this figure, the proposed shuffling scheme performs the best regardless of the cache size and workload situation. When the size of the buffer cache is too small, even hot data cannot be accommodated, causing the low cache hit rate regardless of the shuffling method. On the other end, if the size of the buffer cache is very large, all data blocks can be simultaneously loaded into the buffer cache, resulting in the same cache hit rate no matter which shuffling method is used. When considering the conventional system configurations, the results with the cache size of 30-50% represents the most practical situations, and we can observe significant improvement of the proposed scheme in this range. For the ImageNet100 dataset, the proposed shuffling scheme exhibits the improvement of 40.7% on average and up to 74.1% in terms of the cache hit rate. For the Food101 dataset, the improvement is 33.8% on average and up to 59.6%. When considering the two dataset cases together, the average improvement in hit rate is 37.2%. To quantify how the enhancement in the cache hit rate affects the actual I/O time of workloads, we conduct measurement studies. Figure 6 plots the data loading time resulting from the measurement while executing each workload with the proposed shuffling scheme compared to the original data shuffling method used in PyTorch. To see the pure time of data loading, we measure the I/O latency to load the datasets, convert them in Tensor, and resize them, and exclude the time to compute in cores. To see the performance trend while training in neural networks, we measure the loading time as the number of epochs increases. As we see, the proposed shuffling scheme outperforms the original shuffling method of PyTorch throughout the epochs. Specifically, the improvement of the loading time is 8.96% and 13.76% in ImageNet100 and Food101 datasets, respectively, and their average is 11.4%. Figure 4: Data references after adopting bundle-based shuffling and bundle alternating. Figure 5: Performance comparison with respect to the cache hit rate. Figure 6: Performance comparison with respect to the data loading time. By applying the proposed shuffling scheme, we showed that the hit rate of the buffer cache and the data loading time can be improved. However, unlike the original method that pursues complete randomness, the proposed scheme maintains the randomness of data references only within a bundle, which may degrade the model's training accuracy when the same training process is performed. In the worst case, the model might fail to converge because of biased batches. To validate this, we compare the training accuracy of the two models trained by using the proposed shuffling scheme and the original method used in PyTorch as time progresses. Figure 7 shows the training accuracy of the two methods as the number of epochs increases. As shown in the figure, the training accuracy of the proposed scheme is almost the same as that of the original method. Meanwhile, the actual time spent in each epoch will be shortened by applying the proposed scheme since it improves buffer cache performance.