# Memory-Efficient Batching for Time Series Transformer Training: A Systematic Evaluation
## 4 System Design
### Zero-Copy Sliding Window
Our approach extracts overlapping sliding windows directly from the raw tensor residing on the GPU, thereby minimizing memory copies and preserving data locality. By employing views, we avoid the creation of intermediate tensors for each window. Furthermore, by operating directly on the GPU, we take advantage of the computational benefits of hardware acceleration. This view-based, GPU-centric collate_fn has the potential to streamline the training process, particularly for large-scale multivariate time series datasets. The reduction  
Figure 4: Modified sliding-window data preparation. Figure 5: Custom dataset example. in memory footprint and enhanced device-side computations can lead to improved resource utilization and faster training iterations as demonstrated later in our experimental section. ```
Data: Tensor of time series samples \(B=\{(\mathbf{x}_{1},y_{1}),(\mathbf{x}_{2},y_{2}),...,(\mathbf{x}_{n},y_{n})\}\) where \(\mathbf{x}_{i}\in\mathbb{R}^{T\times F}\), window size \(w\), stride \(s\) Result: Tensor of batched windows \(\mathbf{X}\in\mathbb{R}^{n\times w\times F}\), corresponding labels \(\mathbf{y}\in\mathbb{R}^{n}\) \(\mathbf{T}\leftarrow\texttt{move\_to\_device}(B,\texttt{'cuda'})\)// Move input batch to GPU \(\mathbf{X}\leftarrow\texttt{generate\_sliding\_windows}(\mathbf{T},w,s)\)// Apply windowing using  view-based transformation \(\mathbf{y}\leftarrow\texttt{extract\_labels}(B)\)// Collect labels from original batch return\(\mathbf{X}\), \(\mathbf{y}\)
```  
**Algorithm 1**GPU-optimized sliding window collate_fn. For the input data, R describes the space of matrices or tensors with real-valued entries that have the following dimensions: T corresponds to the number of time steps, and F represents the number of features measured at each time step. In terms of the result, R has the following dimensions N X W X F: N corresponds to number of samples per batch and W represents the window size. Based on a given batch size value, we assemble the corresponding number of sliding-window views with a predefined look-back window and a stride. Since the raw data reside in GPU memory, we can apply a time-based zero-copy sliding window on the tensors. This method creates sliding-window views with memory pointers to the actual raw data. In this way, sliding-windowed data are only a view, and data duplication across windows is greatly reduced as we only use pointers and not the actual data. Figure 6 displays the modified multivariate time series data batch formation process. This approach differs from traditional windowed data accumulation within the CPU, followed by replication to GPU memory. Instead, our method prioritizes efficiency by preparing only a batch's worth of raw data on the CPU. These pre-batched data are subsequently tensorized and transferred to the GPU. The critical process of our approach lies in the zero-copy sliding-window operation executed directly on the pre-batched tensor within the GPU. This strategy significantly reduces memory usage, a known bottleneck in time series processing. Furthermore, by streamlining data handling and minimizing redundant data transfers, we achieve computational performance improvements as shown later in Section 5. **Theoretical Justification:** The substantial memory savings from our zero-copy sliding-window technique stem directly from a fundamental reduction in memory complexity. Traditional CPU-based batching and tensor stacking operations replicate tensor data, incurring a memory complexity of O(B x S x F), where B is batch size, S is sequence length, and F is the number of features. In contrast, our view-based sliding windows avoid actual data replication, using tensor views with pointer references. This effectively reduces the  
Figure 6: Improved batch formation. memory complexity to O((B + S) \(\times\) F), representing only minor overhead for pointers.