# Memory-Efficient Batching for Time Series Transformer Training: A Systematic Evaluation
## 5 Experimental Setup and Evaluation
### Results and Analysis
#### 5.3.3 End-to-End Training and Accuracy Evaluation  
To evaluate the practical impact of our memory-efficient batching strategy on the full training workflow, we conducted end-to-end runtime measurements across several model architectures. In addition to performance, we validated that our proposed optimization method preserves model accuracy using real-world benchmarks.  
**(1) End-to-End Training:** For this experiment, we apply our optimization approaches to four different transformer-based model architectures and one linear model to explore the effect of memory reduction on different architecture complexity. In order to demonstrate both the practicality and scalability of our approach, we applied our optimizations in a distributed training mode. PyTorch's Distributed Data Parallel (DDP) allows parallel training of the model by partitioning the data and replicating the model architecture across the GPU devices. We applied the DDP training on 4 GPUs and measured the average runtime per training iteration (epoch). The runtime reporting was conducted only on rank 0 of the training processes.  
Figure 15 displays the average time duration for each training iteration (epoch). The \(x\)-axis represents the different model architecture names. The \(y\)-axis shows the time duration per batch measured in seconds (s). Each training iteration includes the processes of loading each batch of data, applying the model, computing the loss and gradients, and adjusting the learning weights. The operation sequence was repeated for all batches of data. We utilized our generated dataset from the previous experiment (Section 5.3.1) with 100 time series features. The sequence length was fixed to 300 time steps, the batch size was 100 samples, and the total number of time steps was 10,000. For each evaluated model architecture, we trained the model on a fixed multivariate time series dataset using both our optimized data-loading pipeline (both variants) and the standard PyTorch pipeline. We monitored and recorded the end-to-end runtime for each training epoch across multiple training runs, ensuring that all hyperparameters remained consistent across both the optimized and baseline configurations.  
Our experimental result indicates a consistent trend across all evaluated architectures that our GPU-optimized sliding-window approach (both variants) outperforms the PyTorch baseline in terms of runtime efficiency. The magnitude of the increase in speed varied slightly between architectures, but in all cases, the reduction in runtime was statistically significant. We summarize the percentage of runtime reduction for each model architecture in Table 2. For the transformer-based architectures, except PatchTST, our \"Optimized\" and \"Optimized in memory\" variants reduced the runtime on average by 20% and 33%, respectively. For PatchTST, the runtime reductions were 7% and 8% for the two optimized variants. This is due to the default extra pre-processing (pat  
Figure 14: Multiple batches runtime comparison.  
tensor data had to be materialized and the model itself consumed more memory and computational power than the other evaluated transformer-based models for time series forecasting. For DLinear, the runtime reduction were 15% for the \"Optimized\" variant and 95% for the \"Optimized in memory\" variant.  
**(2) Accuracy Evaluation:** To verify that our batching optimizations do not impact model quality, we ran additional accuracy tests using the PatchTST/42 model (42 input patches with sequence length = 336) across four benchmark ETT datasets (ETTh1, ETTh2, ETTm1, ETTm2). These datasets have been extensively utilized for benchmarking and publicly available in the Autoformer's online repository [26]. The results are shown in Table 3. Our accuracy metrics are consistent with those reported in prior work [30], confirming that the optimization is transparent to model performance.