# Memory-Efficient Batching for Time Series Transformer Training: A Systematic Evaluation
## 4 System Design
### Zero-Copy Sliding Window  
In a default PyTorch workflow, even if view-based windowing operations are applied on the CPU side to generate sliding windows, those windowed tensors become re-materialized during batching due to PyTorch's default batching (e.g., stacking individual samples with collate_fn) ends up copying or concatenating the windowed data into a new tensor. In other words, the result of the window-based operations on CPU does not inherently remain a \"zero-copy\" view once you start collecting individual samples into a batch. Instead, PyTorch's standard batching code (the default collate_fn) creates new contiguous blocks of memory that hold the stacked batch. Compared to the default data preparation method depicted in Figure 1, we propose a modified sample preparation workflow. Instead of applying the sliding-window operation on the time series data residing on the CPU, we transfer a batch of data over to the GPU and apply the sliding-window operation there, as shown in Figure 4. By contrast, our framework delays re-materializing until the data are on the GPU and performs the operation via views, thus forming batches without triggering a second memory copy step. Overlap redundancy is efficiently addressed through the use of tensor views, where overlapping windows reference the same underlying data in GPU memory, avoiding explicit duplication of data and significantly reducing memory consumption. The size of the pre-batched data is determined by the sum of the batch size and the sequence length (B + S\(-\)1). For instance, in the example of preparing batches of five samples with a sequence length of six and a stride value of one, we only need to copy pre-batched data containing ten time steps (B + S\(-\)1) to the GPU. This contrasts with the original batch preparation workflow, which requires copying five samples, each with a sequence length of six. As indicated by the dotted line on the GPU side in Figure 4, the view-based sliding-window operation employs pointers instead of replicating data for each window, resulting in reduced memory usage and computational time for batch formation. Our batching strategy accommodates variable-length time series while maintaining fixed input shapes. For sequences longer than or equal to the window size, we apply view-based sliding-window generation directly using GPU-resident tensor views, without any padding. For sequences shorter than the window size, we apply minimal zero-padding to extend the sequence and yield one valid window. This selective padding ensures that no data are discarded. In order to effectively utilize a view-based sliding window as part of the sample preparation workflow, we employ a customized Dataset class with predefined batch size calculation. An example code snippet for such a custom dataset is displayed in Figure 5. The data-partitioning logic to calculate pre-batched sequence length (line 13) is part of the class initialization step ensuring sufficient samples to create each batch of training data. In the code snippet, the variable \"seq_len\" stores the sequence length value. The_len_method also has to be updated to return the number of batches instead of returning the number of samples (line 24). If padding is enabled on the sequences, this method will have to be updated to reflect an additional last batch (if any). In order to avoid redundant data copies,the automatic batching mode has to be disabled by setting a corresponding flag in the DataLoader class. In addition to offloading sample preparation to the GPU, another equally important modification to the data pipeline is the custom sliding-window collation function. The default behavior of PyTorch's collate_fn, which involves stacking individual tensors into batches, can introduce memory overheads and redundant data transfers from host to device memory. To address this, we implemented a custom collate_fn, Algorithm 1 shown below, that leverages sliding-window view mechanism on GPU-resident data.