# Walking Back the Data Quantity Assumption to Improve Time Series Prediction in Deep Learning
## 1 Introduction  
Deep learning has made remarkable strides in time series prediction, largely due to its ability to model complex temporal dependencies that traditional methods often struggle with. The evolution of deep learning architectures for time series forecasting began with multilayer perceptrons (MLPs), which laid the groundwork for later developments in the field. MLPs [1; 2; 3; 4] enabled the modeling of static relationships in data, but as the need for capturing sequential information arose, more sophisticated techniques were developed. This led to the advent of recurrent neural networks (RNNs) [5; 6] which incorporate feedback loops to retain information from past states, significantly enhancing predictive capabilities. The field continues to progress with the introduction of transformer networks [7; 8] which have shown exceptional performance in a variety of tasks, including time series forecasting, by allowing for long-range dependencies without the limitations of sequential processing inherent in RNNs. Modern deep learning models, particularly large language models (LLMs) [9], have architectures that can include billions of parameters, reinforcing the principle of \"more data, better model performance\". This principle posits that increasing the amount of training data generally leads to improvements in model accuracy, yet this can be challenging in practice. Despite these advancements in architecture, a significant research gap remains regarding the optimal use of data volume in training deep learning models for time series forecasting. Although the prevailing assumption in the field is that increasing the amount of training data improves model accuracy, this approach has limitations. Large datasets introduce challenges such as noise, missing values, and computational complexity, which may hinder performance and create inefficiencies. Furthermore, traditional learning frameworks like the probably approximately correct (PAC) learning theory [10] suggest thatwhile large sample sizes improve the likelihood of accurate learning, unique challenges arise in time series data, including autocorrelation, non-stationarity, and seasonality [11]. Time series data often exhibit behavioral shifts, where only specific subsets of data may be relevant for making accurate forecasts. Innovative strategies have emerged to mitigate the limitations associated with small datasets while leveraging the potential of large models. For example, recent findings indicate that thoughtful optimization of model architecture and training processes can yield high accuracy, even with limited data availability [12]. These insights suggest that while the volume of data typically enhances model performance, effective model tuning and the utilization of pre-existing knowledge can reduce the dependence on extensive datasets, making deep learning techniques more applicable across domains where data may be scarce. The existing literature largely overlooks the possibility that effective training could be achieved with subsets of data, especially when historical data from previous horizons is strategically incorporated. This study addresses this gap by questioning whether the principle of \"the more data, the better model performance\" holds in the context of time series forecasting. Specifically, it explores whether training deep learning models on selectively chosen subsets provides improved performance metrics compared to training on complete datasets. Although models are studied in which research focuses on estimating model parameters by using a partial window of available observations [13], if the first available data follow a process unrelated to the current reality, then the use of such data in the estimation can lead to biased predictions and report higher mean square errors. Excessive reduction, however, can negatively impact the variance, reflecting higher errors.