# Short-Term and Long-Term Travel Time Prediction Using Transformer-Based Techniques
## 2 Related Work
### Sequence-to-Sequence Models in Long-Term Prediction  
Sequence-to-sequence methods are commonly used in the domain of travel time prediction, representing a significant evolution in the application of deep learning within ITS. These methods have led to the development of innovative models that provide substantial improvements over traditional machine learning approaches. For instance, the deep ensemble stacked LSTM model (DE-SLSTM) [22] integrates weather conditions into long-term forecasts covering several hours. One study established a network of Long Short-Term Memory units paired with Deep Neural Networks (LSTMs-DNNs) for analyzing highway travel time data [6]. In another effort, the synergistic use of GRU and XGBoost was utilized to uncover hidden features in traffic data, leading to improved short-term forecasts [8]. Furthermore, MTSMFF introduces a multivariate time series forecasting framework that employs an attention-based encoder-decoder model, utilizing BiLSTM for encoding data's hidden states before making predictions [23]. Continuing this trend, the temporal fusion Transformer (TFT) [24] combines short-term and long-term temporal dependencies, utilizing diverse inputs for predictions ranging from 5 to 150 min. The fusion network [25] captures both long- and short-term dependencies by integrating spatial and temporal information. Additionally, the study in [5] introduces an LSTM-based encoder-decoder architecture with attention mechanisms, designed for multi-step, long-term travel time predictions. These advancements highlight the role of deep learning networks as foundational elements in traffic prediction, enhancing the capabilities of ITS.  
These approaches generally focus on predicting traffic conditions for the upcoming few hours. However, when forecasts are required for more extended periods, such as multiple days, the task becomes significantly more challenging. The complexity of traffic variables and the necessity for longer forecasting intervals highlight the critical need for accurate long-term forecasts in practical scenarios. One common difficulty with extended forecasts is the progressive accumulation of errors in multi-step forecasting. To tackle this, the research highlighted in [5] presents an approach that employs an LSTM-based encoder-decoder model with attention mechanisms designed to each of the periodic patterns. This model, referred to as PASS2S, processes each periodic segment individually, generating multiple forecasts that are then integrated through a fusion mechanism to produce the final prediction. By effectively leveraging periodic patterns, this strategy enables the model to focus on essential information at each forecasting step, thereby mitigating the issue of error accumulation common in long-term predictions. While this demonstrates advantagesin handling periodic data and employing attention mechanisms, it does not specifically focus on simultaneously managing short-term and long-term predictions, a critical area for comprehensive traffic forecasting.  
Despite these advancements, the domain of both long-term and short-term prediction remains challenging, particularly when extending the forecasting horizon while ensuring reliable performance. Capturing complex traffic patterns, along with the need to manage sequential data and long-range dependencies, demonstrates the intricacies involved in traffic forecasting. These challenges motivate the exploration of Transformer-based methods, known for their capabilities in capturing long-range dependencies effectively. In this study, the proposed Short-Term and Long-Term Integrated Transformer (SLIT) effectively addresses both the need for accurate short-term forecasts and the complexities of long-term traffic prediction.