# Augmentation of Deep Learning Models for Multistep Traffic Speed Prediction
## 3 Methodology
### AttBDLSTM-FCN  
The proposed model, AttBDLSTM-FCN, augments an attention-based bidirectional LSTM with a fully convolutional network to predict multistep network-wide traffic speed. To encapsulate the bidirectional spatial features and temporal dependencies, augmentation of AttBDLSTM and FCN are exploited. In traffic speed prediction, to compute the backward dependency of data, this is the first time that augmentations of AttBDLSTM and FCN are used as a building block for a deep architecture model. Figure 4 describes the architecture of the proposed methodology. Several hidden layers of deep LSTM architectures are built up and thus work more effectively than described in existing studies [25; 26; 39]. AttBDLSTM-FCN comprises two blocks, as depicted in Figure 4. The FCN block consists of temporal convolutions that are applied for feature extraction generally. The global average pooling [40] is used to diminish the number of parameters.  
The FCN block augments through the AttBDLSTM block. The FC block contains a temporal block having a filter size of 128. The structure of the FC block is described as a temporal convolutional layer accompanied by batch normalization [40] and followed by a sigmoid activation function. Finally, we used the global average pooling after the convolutional block.  
We used the self-attention mechanism in the BDLSTM module as attention-based RNNs are very efficient in many fields, e.g., machine translation, image captioning, speech emotion recognition, etc. Thus, we strengthened the contribution of the self-attention mechanism in our study. The AttBDLSTM and the FCN block are concatenated and fed into the sigmoid layer. We used the sigmoid function in our proposed model as the last layer of the model. Further, it also serves as an activation function in machine learning. The AttBDLSTM-FCN inputs spatial time series data and predicts the expected future values for the multistep traffic speed prediction. When the time series spatial-temporal traffic data are fed to the AttBDLSTMs, the spatial correlation of the speed at different locations and the temporal dependencies of the speed values are captured during the process of feature learning, as bidirectional LSTMs can utilize both backward and forward dependencies, so the BDLSTM is an entirely reasonable model to take in more valuable information  
Figure 3: Example of causal convolution.  
from spatial time series data. In the FCN model, there is no fully connected layer, so learning filters are everywhere. It is faster than CNN. An FCN learns representations and makes decisions based on local spatial input. The proposed model, AttBDLSTM-FCN, uses the augmentation of AttBDLSTM and FCN. It is evident that when applying FCN and AttBDLSTM blocks parallelly, the blocks augment and force each other to find a set of features when combined, resulting in a model with better overall performance. Our proposed model significantly enhances the performance of the FCN network with minimal increase in the model size and requires minimal processing of the dataset. One of the major advantages of using the attention BDLSTM cell is that it provides a visual representation of the attention vector. Utilization of the attention mechanism allows one to visualize the decision process of the LSTM cell. A major advantage of the proposed model is that it does not require heavy preprocessing or feature engineering.  
The mechanism used to make short-long-term traffic speed predictions in the present study is depicted in Figure 4. The one-time-step prediction model property is shown in  
Figure 4: The proposed model architecture.