# Highway Speed Prediction Using Gated Recurrent Unit Neural Networks
## 3 Materials and Methods
### Methods  
RNN refers to a neural network with one or more recurrent layers in a hidden layer. The RNN differs from other networks in its reception of input data. It is mainly used for sequence data, closely related to time-series data, including weather, stock prices, and language. The RNN has a feedback structure that receives and uses the output of a layer again as its input. Thereby, the previous data affect the results. Figure 4 presents the unfolded structure of an RNN. The RNN is well-designed to process time-series data; it uses weights and biases repeatedly by passing information via cyclic vectors. Figure 5 shows an RNN architecture. The RNN receives the input vector X and generates the output vector y. RNNs are characterized by no limit on the length of input and output and have fully connected structures. Networks of various shapes and styles can be formed by modifying their architecture. However, existing RNNs have a long-term dependency problem [31]. The weights converge to 0 or diverge to infinity as the time lag increases. Figure 4: Unfolded structure of recurrent neural network. Figure 3: Traffic speed for one month between Singal junction and Seoul toll gate. Thus, the LSTM was proposed to solve the long-term dependence problem of the existing RNN. It is a modified architecture of the RNN and is characterized by the existence of a cell state (Figure 6). The LSTM is composed of three gates, namely the input, forget, and output gates. Each gate controls its value through the sigmoid layer and pointwise multiplication. The input gate (\(i_{t}\)) utilizes a sigmoid function (\(\sigma\)). The sigmoid function determines which of the new information is to be stored in the cell input state. Next, the tanh function creates new data candidate values as the cell candidate state, \(\bar{C}_{t}\). Figure 5: Recurrent neural network architecture. Figure 6: Long short-term neural network architecture. The forget gate determines what is forgotten using the weights of how much to forget and updates the state by multiplying the previous state, \(C_{t-1}\), by the output value of the forget gate. In addition, the output values of the input gate are multiplied and added to the cell state \(C_{t}\). The output gate (\(O_{t}\)) determines the output value and uses a sigmoid function. The final output value is calculated by multiplying the value obtained by applying tanh to the cell state (\(C_{t}\)) and the sigmoid output value (\(O_{t}\)). Although the LSTM has an excellent ability to memorize long-term dependencies, it takes a long time to train the model owing to its complex structure. Thus, the GRU was proposed to accelerate the training process [32]. The GRU is a type of RNN framework with a gate mechanism applied, inspired by LSTM, and with a simpler structure [33]. The GRU architecture is described in Figure 7.  
The reset gate (\(r_{t}\)) uses the sigmoid function as an output to properly reset the past information and multiplies the (0.1) value by the previously hidden layer. The output value can be obtained by multiplying the hidden layer's value at the previous point and the information at the present point by weight. The update gate is a combination of the forget and input gates of the LSTM model. The update gate determines the rate of update of past and present information. In the update gate, the result \(u_{t}\) output as sigmoid determines the amount of information at present, and the value subtracted from \(1(1-u_{t})\) is multiplied by the information of the hidden layer at the most recent time. Each update gate is similar to the input and forget gates of the LSTM. This study employs the GRU to predict traffic speed on highway sections. Figure 8 presents a flow chart of the study design. We first conducted data preprocessing and hyperparameter tuning. It is essential to conduct hyperparameter tuning for better prediction. We considered batch size, two factors (i.e., speed and volume), and the number of hidden layers. The details are discussed in Section 4.1.