# Short-Term and Long-Term Travel Time Prediction Using Transformer-Based Techniques
###### Abstract
In our study, we define short-term predictions as those within an hour's time frame and long-term predictions as those that exceed this duration. Significant advancements have been made in the area of short-term traffic prediction, largely driven by developments in deep learning technologies that focus on immediate or upcoming traffic conditions. Research has explored the use of Long Short-Term Memory Networks combined with Deep Neural Networks (LSTMs-DNNs) for analyzing highway travel time data [6]. Similarly, another study merged LSTM techniques with ensemble learning and optimization strategies to enhance short-term traffic flow predictions [7]. Moreover, the application of Gated Recurrent Units (GRUs) coupled with eXtreme Gradient Boosting (XGBoost) in a separate study has provided deeper insights into traffic behaviors, thus enhancing the precision of traffic predictions [8]. These studies highlight various methods employed in short-term travel time prediction, each with unique benefits. However, they also share common obstacles such as the complexity of multi-step forecasting and the accumulation of errors [4]. Innovative modeling techniques have significantly impacted the field of long-term traffic prediction, notably improving strategic planning and decision making within urban traffic management. Research employing Recurrent Neural Networks (RNNs) has demonstrated the ability to forecast traffic flows over extended periods by incorporating meteorological and contextual data [9]. Additionally, a blend of wavelet decomposition, Convolutional Neural Network (CNN), and LSTM has been applied to predict traffic flow for the subsequent day, emphasizing the analysis of long-term temporal characteristics [10]. Moreover, recent progress has showcased the utility of Transformer-based methods in effectively capturing long-term dependencies, with one study introducing a Transformer model equipped with multi-head attention designed for this task [11]. Another study presented a Multi-Size Patched Spatiotemporal Transformer Network (MSP-STTN), a model that utilizes a patched Transformer structure to predict traffic flow in both the short and long term [4]. These technological advancements highlight the importance of long-term traffic prediction for real-world applications, illustrating how these models provide valuable insights that aid in traffic management and urban planning. The continuous development of these methods to capture complex traffic patterns plays a vital role in advancing ITS. To address the challenges and evolving requirements in short-term and long-term traffic prediction, this study introduces the Short-Term and Long-Term Integrated Transformer (SLIT). SLIT is designed to leverage the strengths of Transformer models for effectively handling the complexities of traffic data, enabling reliable predictions for continuous periods within the short term (up to 1 h) and for specific days in the long-term duration (e.g., 1, 2,... days ahead). Unlike existing methods, which often struggle to simultaneously capture the dynamic nature of short-term and long-term traffic patterns, SLIT introduces a novel architecture that ensures a comprehensive approach to traffic forecasting. The key contributions of this study are as follows:  
* The proposed SLIT, which incorporates the Enhanced Data Preprocessing (EDP) and the Short-Term and Long-Term Integrated Encoder-Decoder (SLIED), represents a significant advancement in traffic prediction. SLIT demonstrates the capability to predict travel times across various periods. * The EDP module effectively utilizes periodic segments and temporal attributes in data processing, significantly enhancing the model's predictive capabilities for both short-term and long-term traffic scenarios. * The SLIED module, consisting of Short-Term and Long-Term Integrated Encoding (SLI-E) and Decoding (SLI-D), captures dependencies over different time frames. This design mitigates error accumulation commonly observed in autoregressive models.