# Travel Time Prediction on Long-Distance Road Segments in Thailand
## 3 Materials and Methods
### Our Approach
#### 3.3.4 Architecture of Our Travel Time Prediction Model  
This subsection describes the architecture of our travel time prediction model. The model is demonstrated in Figure 6, after all the experiments were completed and with all the adjusted parameters. Our model has six layers: input data, a Butterworth low-pass filter, a self-attention layer, an LSTM layer, a fully connected layer, and an output. It was assumed that the model had already been trained from historical data. Thus, some variables, such as the weight vector of query (WQ), the weight vector of key (WK), the weight vector of value (WV), the hidden state input vector (H0), the cell state input vector (C0), the weight tensor of the LSTM (WLSTM), and the weight matrix (WFC) of the artificial neural network in the fully connected layer, were already assigned. The input data were the sequences of previous travel times over a period of 24 h that were selected from TTR (Equation (4)) with a given road segment. This is a 24-dimension (24-dim) vector. In the case of predicting road segment A at 07:00 on one day, the average hourly travel time along road segment A between 07:00 and 24:00 of the previous day and between 00:00 and 06:00 of the current day needs to be collected. Butterworth Low-Pass Filter  
The task of this layer is to process the input vector by the Butterworth low-pass filtering. This resulted in the 24-dim vector. The self-attention layer has an attention function that multiples the input travel time vector with the trained weight vectors: WQ, WK, and WV, becoming vectors Q, K, and V,  
Figure 6: The architecture of our travel time prediction model. respectively. Then, it calculates the vectors Q, K, and V by the \(\text{SoftMax}\left(\frac{\text{Q}\text{K}^{\text{T}}}{\sqrt{24}}\right)\cdot\text {V}\) equation, where the SoftMax is a function expressed by \(\text{SoftMax}(\text{z})_{\text{i}}=\frac{\text{e}^{\text{z}_{\text{i}}}}{ \sum_{\text{j}}-1\text{e}^{\text{z}_{\text{i}}}}\text{for i}=1,\dots,\text{ J}\). This resulted in the 24-dim vector of the self-attention feature map. Then, the input vector from the previous layer is added by the self-attention feature map vector, so it becomes a 24-dim input feature vector. The LSTM layer consists of 24 LSTM units. All of the LSTM units are constructed by the trained weight tensor (\(\text{W}_{\text{LSTM}}\)). In addition, the trained 256-dim vectors of the hidden state input vector (\(\text{H}_{0}\)) and the cell state input vector (\(\text{C}_{0}\)) are inputs of the first LSTM unit. Then, the result of the last LSTM unit is used as an output feature, which is a 256-dim vector. Output  
The output from the fully connected layer is the predicted travel time of the following hour along a given road segment. In addition, we wrote the following pseudocode to demonstrate our model in detail, corresponding to Figure 6. The input was a 24-dim input vector of the travel time (input_ttr) and the trained variables; the output was a predicted travel time of the following hour. The trained variables were selected according to the road segment from input_ttr. This means that it needed to prepare trained variables for each road segment. In the pseudocode, BF() is a function of the Butterworth low-pass filter; SoftMax() is a SoftMax function; LSTM() was used to construct an LSTM model based on an input cell state, an input hidden state, and a weight tensor; and ANN() was used to construct an ANN model based on the input weight matrix with the linear activation function (Algorithm 1).