# Memory-Efficient Batching for Time Series Transformer Training: A Systematic Evaluation
## 5 Experimental Setup and Evaluation
### Results and Analysis
#### 5.3.2 Runtime Performance Analysis  
Modifications made to the data sample preparation and batch formation directly affect not only memory usage in both CPU and GPU devices but also data transfer time, and batch processing time during the model's forward pass. By offloading pre-batched data to the GPU, we can reduce the amount of data typically transferred between the devices, therefore reducing the transfer time. We can also leverage the GPU's computing power to perform the sliding-window operation for the batch formation process. As a result, we conducted several runtime analyses to measure the impact factor and understand the efficacy of our approach. To evaluate our hypotheses, we divide this set of evaluations into two parts. First, we benchmark the average time duration for the system to prepare each batch of tensor data and have it available in the GPU memory for the training process. We conducted three main experiments to understand the influence of our approach on different batch sizes, sequence lengths, and increasing number of features. This part of the experiment  
Figure 10: Memory Usage—Increasing Sequence Length. (**a**) Memory Usage per Batch; (**b**) Memory Reduction Percentage. would demonstrate the runtime effect of the data size reduction. Second, we measure the end-to-end runtime to load and process each batch of data during the models' training iteration. To understand the effect of our approach on different time series models, we also measured the average epoch training time for each different model architecture to process the data. This later part of the evaluation would illustrate the effect of utilizing the GPU-optimized sliding-window operation for the batch formation and how different model complexity could benefit from this optimization. **(1) Number of features:** In this experiment, we compare the average time duration for loading in each batch of data consisting of various numbers of features. We used the same seven datasets as the experiment in Section 5.3.1 with varying numbers of features ranging from 10 to 5000 to measure the average per-batch runtime. The experiment results and runtime reduction percentages are displayed in Figure 11. Figure 11 displays the average time duration for each batch of tensors to be available in the GPU memory for training. This includes the data transfer time from CPU to GPU and applying the GPU-optimized sliding-window operation on the data residing in the GPU memory. The \(x\)-axis represents each dataset with an increasing number of features. The \(y\)-axis shows the time duration measured in seconds (s). For visualization, the values are plotted on a logarithmic scale. As the number of features increases across datasets, the runtime for both approaches (all variants) also increases accordingly due to transferring increasing amounts of pre-batched data from CPU to GPU, and the sliding-window operation had to be applied to datasets with an increased number of input features. However, our GPU-optimized sliding-window approach (both variants) has a much lower runtime across all dataset sizes. On average, across seven datasets, our \"per-batch\" optimization approach labeled \"Optimized\" and the \"preload\" variant labeled \"Optimized in memory\" reduced the runtime by 99% on average as shown in Figure 11b. The significant runtime reduction is a result of avoiding unnecessary materialization of windowed data across the time series dataset. The runtime reduction also increases with the number of features in the dataset. When applying our optimization approach under the assumption that the dataset size is smaller than the GPU memory, the entire dataset was loaded into the GPU memory prior to the batching process, resulting in a much lower runtime. **(2) Batch size:** For this experiment, we increased the number of samples per batch to explore the effect of runtime reduction on different batch sizes. The experiment results and runtime reduction percentages are displayed in Figure 12.  
Figure 11: Runtime—Increasing Number of Features.