# Memory-Efficient Batching for Time Series Transformer Training: A Systematic Evaluation
## 5 Experimental Setup and Evaluation
### Results and Analysis
#### 5.3.2 Runtime Performance Analysis
(**a**) Runtime per Batch; (**b**) Runtime Reduction Percentage. In this experiment, we used the same datasets as the experiment in Section 5.3.1 with varied batch size ranging from 100 to 1000 samples. The average runtime results are displayed in Figure 12a. The \(x\)-axis represents the dataset with an increasing number of batch sizes. The \(y\)-axis shows the time duration per batch measured in seconds (s). Note that the values are plotted on a logarithmic scale for visualization. As the number of samples per batch increases, the runtime for the baseline (labeled \"Original\") increases linearly. This is because the system has to transfer and assemble an increasing amount of windowed data per batch. On the other hand, both of our optimized variants have a much lower increase in runtimes. For our \"preload\" optimization variant, when all the data are pre-loaded into the GPU memory (labeled \"Optimized in memory\") there is no increase in the time duration even when the batch size increases. This is because the entire dataset was already available in the GPU memory and the same sliding-window settings were applied to the dataset, making the batch size increase trivially with the overall runtime. The sliding windows that were applied to the dataset only resulted in views with no additional data copies. For our \"per-batch\" optimized variant (labeled \"Optimized\"), the increase in the time duration across different batch sizes is still significantly less than that of the baseline. On average, across the five different batch sizes, both of our optimized variants reduced the runtime by 99%, as shown in Figure 12b. **(3) Sequence length:** For this experiment, we increased the number of time steps for each sample to explore the effect of memory reduction on different sample sequence lengths (window sizes). The experiment results and runtime reduction percentages are displayed in Figure 13.  
In this experiment, we used the same datasets as the experiment in Section 5.3.1 with varied sequence lengths ranging from 100 to 5000 time steps to measure the average per-batch runtime. The results are displayed in Figure 13a. The \(x\)-axis represents the dataset with an increasing number of time steps (window size). The \(y\)-axis shows the time duration per batch measured in seconds (s). Note that the values are plotted on a logarithmic scale for visualization purposes. Similar to the batch size evaluation results, as the number of time steps per window increases, the runtime for the baseline (labeled \"Original\") increases linearly. This is because the system has to assemble an increase in the data volume due to the increase in the sequence length per sample even with the fixed number of features and batch size. On the other hand, our \"per-batch\" optimized variant (labeled \"Optimized\"), has a significantly lower increase in runtimes than that of the baseline. For our \"preload\" optimized variant (labeled \"Optimized in memory\"), there was no increase in the time duration even when  
Figure 12: Runtime—increasing batch size. (**a**) Runtime per batch; (**b**) runtime reduction percentage. the window size increased. This is because the entire dataset was already available in the GPU memory. The changes in the window size only affect the sliding-window settings. Since the sliding-window operation in our implementation only creates views, there were no additional data copies with the changes in the sequence length. On average, across five different sequence lengths, both of our optimized variants reduced the runtime by 99% as shown in Figure 13b. **(4) Time-per-batch:** In the second part of our runtime experiment, we measure the end-to-end runtime across multiple batches of data. The purpose of this experiment is to give us an insight into the performance of each data-loading pipeline when it is operating. In this evaluation, we compare the average runtime between our two variants of GPU-optimized sliding-window techniques and the default PyTorch's tensor stacking workflow (labeled 'Original').