# Short-Term and Long-Term Travel Time Prediction Using Transformer-Based Techniques
## 3 Proposed Methodology
### Enhanced Data Preprocessing  
Inspired by the collaborative use of attention mechanisms with periodic segment generation for travel time prediction [5], the Enhanced Data Preprocessing (EDP) is devised to enhance the quality of historical traffic data \(\mathbf{X}\). This procedure aims to facilitate the generation of attribute-encoded periodic segments \(\hat{S}\), which are further divided into long-term and short-term segments. The procedure is illustrated in Figure 2 and is detailed in Algorithm A1, as presented in Appendix A. For additional clarity, a detailed explanation of the symbols utilized throughout the process is provided in Table 1.  
In Figure 2, EDP incorporates several procedures to refine the input data for subsequent predictive modeling. Initially, missing data points are tackled through the Missing Data Filling (MDF) using linear interpolation. By averaging the data from equivalent time slots on adjacent days where information is present, MDF ensures the continuity and reliability of the dataset, thus preparing it for further analysis. Following this, the Data Scaling (DS) step is introduced, where the data are normalized by Z-score standardization. This normalization mitigates the influence of outliers and ensures consistency in data scales. This standardization is necessary for enhancing the predictive model's ability to generalize and perform consistently, regardless of the varying scales of the input data. Furthermore, the Temporal Attribute Extraction (TAE) transforms each data point, denoted as \(X_{i}\), into an attribute-encoded vector \(\hat{X}_{i}\) in \(\mathbb{R}^{m}\), where \(m=316\)[5]. The first two dimensions of this vector represent essential traffic information, including travel time and speed. The subsequent 314 dimensions are encoded as one-hot vectors, representing a range of temporal attributes. These include holidays, the day of the week, month, peak hours, and specific time slots, providing a comprehensive view of the traffic patterns influenced by temporal patterns and societal behaviors, as detailed in Table 2. \begin{table}
\begin{tabular}{c l} \hline \hline
**Symbol** & **Description** \\ \hline \(S^{L}\) & Collection of long short-term segments \(S^{L}_{d}\) in \(\mathbb{R}^{(D\times T^{l})\times m}\). \\ \(S^{S}\) & Short-term segment in \(\mathbb{R}^{(1\times T^{l})\times m}\). \\ \(T^{\prime}\) & Number of data points per periodic segment, 12 time slots per hour. \\ \(t_{d^{\prime}}\), \(t_{k}\) & Starting time step for \(S^{L}_{d}\) and \(S^{S}\), respectively. \\ \(S\) & Combined periodic segments in \(\mathbb{R}^{(D\times T^{l})\times m}\), with \(m=316\). \\ \(D\) & Number of long short-term segments, set to 7 in this study. \\ \(D^{\prime}\) & Represents \(D+1\), denoting the combined periodic segments \(S\). \\ _interval_ & Duration of each period’s time step, with 288 time slots per day. \\ _pos_ & Position within \(S\), \(i\) indexes the feature dimensions. \\ \(d_{model}\) & Size of the feature vector. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Symbol descriptions. Figure 2: Enhanced data preprocessing. The core part of EDP is known as Periodic Segmentation (PS). Building upon the segmentation approach outlined in [5], PS is designed specifically for SLIT's requirements. To effectively capture temporal dependencies in traffic data, PS systematically segments \(\hat{X}\) into periodic segments \(S\), which include long short-term segments \(S^{L}\) and long short-term segments \(S^{S}\), as presented in Figure 3. For example, considering the timestamp of January 8 th at 8:00 AM (\(\tau\)), the objective is to forecast travel times for the following hour. As depicted in Figure 3a, \(S^{L}\) includes traffic data from the same hour but on the \(d\)-th day before, with each day defined by an interval of 1 day. Thus, \(S^{L}_{\gamma}\) contains traffic data from January 1 st (the 7 th day before January 8 th), represented as twelve consecutive time slots, each denoted by \(s^{L}_{7,l}\) (\(l=1,\dots,12\)).