# Multistep Prediction of Bus Arrival Time with the Recurrent Neural Network

Authors: Zhi-Ying Xie; Yuan-Rong He; Chih-Cheng Chen; Qing-Quan Li; Chia-Chun Wu
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

Accurate predictions of bus arrival times help passengers arrange their trips easily and flexibly and improve travel efficiency. Thus, it is important to manage and schedule the arrival times of buses for the efficient deployment of buses and to ease traffic congestion, which improves the service quality of the public transport system. However, due to many variables disturbing the scheduled transportation, accurate prediction is challenging. For accurate prediction of the arrival time of a bus, this research adopted a recurrent neural network (RNN). For the prediction, the variables affecting the bus arrival time were investigated from the data set containing the route, a driver, weather, and the schedule. Then, a stacked multilayer RNN model was created with the variables that were categorized into four groups. The RNN model with a separate multi-input and spatiotemporal sequence model was applied to the data of the arrival and leaving times of a bus from all of a Shandong Linyi bus route. The result of the model simulation revealed that the convolutional long short-term memory (ConvLSTM) model showed the highest accuracy among the tested models. The propagation of error and the number of prediction steps influenced the prediction accuracy.


---

## Body

## 1. Introduction

The rapid and continuous development of China has led to an increase in the number of vehicles. The National Bureau of Statistics of China announced that the number of privately owned vehicles reached 261.5 million in 2019 with 21.22 million vehicles increased in a year. 96 cities in China had more than one million registered vehicles [1]. The rapid increase of vehicles causes traffic congestion, parking problems, and environmental pollution. Public transportation affords a larger number of passengers and alleviates such problems. Mass transportation consumes less energy and emits less amount of pollutants than private transport. Therefore, urban planning puts a priority on public transportation. New technologies such as bus rapid transit (BRT) and driverless bus have been developed significantly with huge investment to support the public transportation system. However, a trip by bus takes a relatively long time and is not punctual, which makes people avoid it. Encouraging people to use buses more often requires optimized bus routes and punctuality of bus operation [2, 3]. However, the absence of an accurate operation schedule often causes long waiting times and bus bunching on the same route. For the punctual operation of the public buses, the bus schedule needs to be optimized, which needs an accurate prediction of the arrival time of buses on a route accurately. This not only meets the demand of ordinary passengers who want to know the arrival times of a bus at boarding stations but also optimizes the intelligent bus scheduling system and improves the operation efficiency of the bus company.Several neural networks have been used to predict the arrival time of a bus: non-RNN network, RNN with the time series, and temporal and spatial RNN network. Several studies adopted non-RNN networks for predicting bus arrival and operation times using (1) MapReduce-based clustering withK-means [4], (2) a backpropagation (BP) neural network model [5], (3) a particle swarm algorithm [6], (4) a wide-depth recursive (WDR) learning model [7], and (5) RNN with the time series such as long short-term memory (LSTM) [8]. Models with LSTM processed the historical data of the global position system (GPS) and bus stop locations with the influence of different routes, drivers, weather conditions, time distribution [9], heterogeneous traffic flow, and real-time data [10–12]. The temporal and spatial RNN network with ConvLSTM or a spatiotemporal property model (STPM) was originally used to predict the precipitation [13]. However, it was also used for predicting bus arrival times based on the total operation time of a bus on a route, waiting and on-board times, transfer location wait times [14–16], and multilane short-term traffic flow [17] and for creating the multitime step deep neural network [18].The bus is running on fixed lines with fixed stations. The spatial relationship between its stations determines the arrival times in the time series. Thus, this study used an RNN to predict the arrival time of a bus. A route of a bus has 30–40 bus stations in general. Arrival time prediction includes the time prediction of each station along the way from the starting to the finishing stop, the arrival times at subsequent stations, and the arrival time of the nearest vehicle to a station. This study first analyzed the bus arrival time. Based on the analysis, the input eigenvectors of a neural network were defined, and then, seven RNN models for predicting the arrival time from four categories were tested. Then, the proposed model was trained by the measured data of arrival and departure times of the buses in a route of Linyi, Shandong Province. Then, the multistep prediction of the arrival time was carried out.This paper is organized as follows. Section2 describes the theoretical background and introduces the recurrent neural network. Section 3 describes the pretreatment and analysis of data. Section 4 discusses the analysis result of the RNN model. Finally, Section 5 concludes this study.


## 2. Theoretical Background

A recurrent neural network (RNN) [19] has a feedback structure that processes sequential data for time-series prediction or classification. RNN is widely used in various applications, and new models using it have been suggested such as LSTM, GRU, and ConvLSTM. According to the data in this study, we divided the prediction into four categories and adopted a multistep prediction for bus arrival times. The time-series input data is essential for the prediction with optimal feature extraction and memory efficiency. The data is processed in an RNN with internal feedback and feedforward connection, which retain and reflect the state or memory of a long context window [20]. The RNN suffers from a common disadvantage of the gradient disappearance (gradient vanishing) and gradient explosion problem [21–23], which results in limited applications due to training problems. To solve the problems, Hochreiter et al. [24] proposed and continued improving LSTM for different applications [25, 26]. LSTM specializes in memorizing long sequences and effectively avoiding the problem of gradient disappearance. Hidden layers of LSTM use memory blocks that store the previous sequence information, while increasing the performance of three gates: input, output, and forget gates. These control the sequence information for memory. The gated recurrent unit (GRU) [27] is a modestly simplified LSTM. GRU combines the forget and input gate into an update gate and the cell and hidden state. A model with GRU is simpler and has less activation function and output computation than the standard LSTM model.


### 2.1. Pure LSTM and Pure GRU Model

Figure1 shows the hidden units of LSTM which are replaced by memory blocks.Figure 1
LSTM memory cell structure.Calculatingct and ht requires the following equations:(1)it=σWiXt+Uiht−1+biInput gate,ft=σWfXt+Ufht−1+bfForget gate,ot=σWoXt+Uoht−1+boOutput gate,c˜t=tanhWcXt+Ucht−1+bcNew memory cell,ct=ft⊙ct−1+it⊙c˜tMemory cell,ht=ot⊙tanhct.In these equations, ⊙ Hadamard product is the multiplication of the corresponding elements in the operation matrix,Wi, Wf, Wo, and Wc are the weights of Xt, Ui, Uf, Uo, and Uc are the weights of ht−1, bi, bf, bo, and bc are the bias conditions, σ is the sigmoid function, and tanh is the hyperbolic tangent function.Figure2 shows the GRU. There is only one hidden state ht in GRU. Through the linear transformation of the input tensor and hidden state, the weighted sum of the hidden state inflow is calculated with equations (2) and (3). The linear transformation for rt, ht−1, and the input tensor is combined with the activation function of equation (4) to calculate the updated value of the hidden state. The mixed weight for calculation of the implicit state in the previous step is shown in equation (5). The final output htis the same as LSTM. Compared with LSTM, there is one less activation function calculation and output calculation as well as the final hidden state update, so the calculation is relatively simple.Figure 2
GRU unit structure.(2)zt=σWzXt+Uzht−1+bzUpdate gate,(3)rt=σWrXt+Urht−1+brReset gate,(4)h˜t=tanhWhXt+rt⊙Uhht−1+bhNew memory cell,(5)ht=1−zt⊙h˜t+zt⊙ht−1.Figure3 illustrates a pure network model of LSTM and GRU. The input layer has the sequence of the arrival time series input, and the other two layers use a fully connected prediction network. Input and output data are 3D tensors with a shape [?, 41, 1] (“? ”means that a dimension can have any length).Figure 3
A pure LSTM and pure GRU network model.These models use a single layer of LSTM and GRU. In the input layer, variables are, such as route, direction, vehicle model, and driver, also regarded as a part of the time sequence.


### 2.2. Multi-Input Model Separated by Time Series

As the variable is not sensitive to any specific ordering, the RNN cannot process it alone. However, a BP network can process through a connection layer. Thus, the integration of RNN and BP was used for the prediction network (Figure4).Figure 4
An LSTM-BP integrated network model.The integrated network was in accordance with the characteristics of the input data. A two-part network used the time series-related input data such as route number, driver, departure time, and route length for LSTM processing. Through a connection layer, the prediction layer was processed. Since time series input data became shorter even with the addition of LSTM, the total trainable parameters were not significantly increased compared with pure LSTM.


### 2.3. LSTM Stacking Model

To achieve better accuracy of the prediction than a single layer, a multilayer LSTM was employed. Stacking four LSTMs had hidden units in 256, 128, 64, and 32 layers, respectively. Figure5(a) shows the diagram of the stacking models. There is also a two-way LSTM composition, in which the forward and backward connections also employ a reverse projection function, which is suitable in our case to verify arrival time predictions. Figure 5(b) shows the diagram of the bidirectional network models.Figure 5
(a) A LSTM stacking model. (b) A bidirectional LSTM model.


### 2.4. Spatiotemporal Time-Series Model

The bus operation is in a space-time domain although there are little changes in the spatial dimension for an operation in the fixed route. As ConvLSTM processes the data of time and space, it integrates a convolution of time and space into calculating each gate of LSTM. The following equations are used for the calculation:(6)it=σWi∗Xt+Ui∗ht−1+biInput gate,(7)ft=σWf∗Xt+Uf∗ht−1+bfForget gate,(8)ot=σWo∗Xt+Uo∗ht−1+boOutput gate,(9)c˜t=tanhWc∗Xt+Uc∗ht−1+bcNew memory cell,(10)ct=ft⊙ct−1+it⊙c˜tMemory cell,(11)ht=ot⊙tanhct.A ConvLSTM network with batch normalization (BN) consists of a specification and flattening layer and a prediction network. Figure6 shows the diagram of the network that has a long training time and many parameters in more than five dimensions. This network is appropriate to process time-series data with spatial properties such as bus arrival times with high accuracy.Figure 6
ConvLSTM network model.


## 2.1. Pure LSTM and Pure GRU Model

Figure1 shows the hidden units of LSTM which are replaced by memory blocks.Figure 1
LSTM memory cell structure.Calculatingct and ht requires the following equations:(1)it=σWiXt+Uiht−1+biInput gate,ft=σWfXt+Ufht−1+bfForget gate,ot=σWoXt+Uoht−1+boOutput gate,c˜t=tanhWcXt+Ucht−1+bcNew memory cell,ct=ft⊙ct−1+it⊙c˜tMemory cell,ht=ot⊙tanhct.In these equations, ⊙ Hadamard product is the multiplication of the corresponding elements in the operation matrix,Wi, Wf, Wo, and Wc are the weights of Xt, Ui, Uf, Uo, and Uc are the weights of ht−1, bi, bf, bo, and bc are the bias conditions, σ is the sigmoid function, and tanh is the hyperbolic tangent function.Figure2 shows the GRU. There is only one hidden state ht in GRU. Through the linear transformation of the input tensor and hidden state, the weighted sum of the hidden state inflow is calculated with equations (2) and (3). The linear transformation for rt, ht−1, and the input tensor is combined with the activation function of equation (4) to calculate the updated value of the hidden state. The mixed weight for calculation of the implicit state in the previous step is shown in equation (5). The final output htis the same as LSTM. Compared with LSTM, there is one less activation function calculation and output calculation as well as the final hidden state update, so the calculation is relatively simple.Figure 2
GRU unit structure.(2)zt=σWzXt+Uzht−1+bzUpdate gate,(3)rt=σWrXt+Urht−1+brReset gate,(4)h˜t=tanhWhXt+rt⊙Uhht−1+bhNew memory cell,(5)ht=1−zt⊙h˜t+zt⊙ht−1.Figure3 illustrates a pure network model of LSTM and GRU. The input layer has the sequence of the arrival time series input, and the other two layers use a fully connected prediction network. Input and output data are 3D tensors with a shape [?, 41, 1] (“? ”means that a dimension can have any length).Figure 3
A pure LSTM and pure GRU network model.These models use a single layer of LSTM and GRU. In the input layer, variables are, such as route, direction, vehicle model, and driver, also regarded as a part of the time sequence.


## 2.2. Multi-Input Model Separated by Time Series

As the variable is not sensitive to any specific ordering, the RNN cannot process it alone. However, a BP network can process through a connection layer. Thus, the integration of RNN and BP was used for the prediction network (Figure4).Figure 4
An LSTM-BP integrated network model.The integrated network was in accordance with the characteristics of the input data. A two-part network used the time series-related input data such as route number, driver, departure time, and route length for LSTM processing. Through a connection layer, the prediction layer was processed. Since time series input data became shorter even with the addition of LSTM, the total trainable parameters were not significantly increased compared with pure LSTM.


## 2.3. LSTM Stacking Model

To achieve better accuracy of the prediction than a single layer, a multilayer LSTM was employed. Stacking four LSTMs had hidden units in 256, 128, 64, and 32 layers, respectively. Figure5(a) shows the diagram of the stacking models. There is also a two-way LSTM composition, in which the forward and backward connections also employ a reverse projection function, which is suitable in our case to verify arrival time predictions. Figure 5(b) shows the diagram of the bidirectional network models.Figure 5
(a) A LSTM stacking model. (b) A bidirectional LSTM model.


## 2.4. Spatiotemporal Time-Series Model

The bus operation is in a space-time domain although there are little changes in the spatial dimension for an operation in the fixed route. As ConvLSTM processes the data of time and space, it integrates a convolution of time and space into calculating each gate of LSTM. The following equations are used for the calculation:(6)it=σWi∗Xt+Ui∗ht−1+biInput gate,(7)ft=σWf∗Xt+Uf∗ht−1+bfForget gate,(8)ot=σWo∗Xt+Uo∗ht−1+boOutput gate,(9)c˜t=tanhWc∗Xt+Uc∗ht−1+bcNew memory cell,(10)ct=ft⊙ct−1+it⊙c˜tMemory cell,(11)ht=ot⊙tanhct.A ConvLSTM network with batch normalization (BN) consists of a specification and flattening layer and a prediction network. Figure6 shows the diagram of the network that has a long training time and many parameters in more than five dimensions. This network is appropriate to process time-series data with spatial properties such as bus arrival times with high accuracy.Figure 6
ConvLSTM network model.


## 3. Pretreatment and Analysis of Data

### 3.1. Data Characteristics

A bus was equipped with a device that included a GPS and data communication module. The device transmitted data to a bus scheduling system. Table1 shows the data structure of the reporting system.Table 1
The data structure from a bus.
No.NameTypesRemarks1Route_noLongRoute number2Route_nameStringRoute name3Route_subnoLongRun method no.4Up downIntDirection on route5Bus_noLongVehicle number6SpeedFloatSpeed7datatime_inStringEntry time8driver_noLongDriver number9busstop_noLongSite number10busstop_nameStringSite name11busstop_lngDoubleSite longitude12busstop_latDoubleSite latitude13busstop_serialIntSite serial number14busstop_typeIntSite type15pack_datetimeStringOutbound time16inform_typeIntInform mode17netpack_typeIntEntry modeThe data of arrival and departure of a bus at a bus stop consists of route, speed, arrival and departure time, coordination, and driver’s number. For obtaining the Lasso variable correlation [8], the bus number, number of bus stops, days in the week, distances between bus stops, arrival and departure times, and weather were included, too. The variables were grouped into two: dynamic and static variables [14]. The dynamic variables include driving times between bus stops, staying times, and weather, while the static variables include a route, direction, vehicle model, driver, arrival and departure times, days of the week, holidays, and working days. We selected variables related to the route and the arrival times at the previous stops as the input features to predict the arrival time at a bus stop. Input and output features are as follows:Input feature: [route_no, direction, bus_no, driver_no, departure hours, departure minutes, days of the week, holidays, distance, and weatherxt−k,…,xt]Output prediction series: (−xt−k,…,0,xt+1,…,xt+n)wherext is the difference of the arrival time between the current station and the previous station.


### 3.2. Data Preprocessing

Step 1.
Generating a Sample Dataset
According to the data in Section3.1, an arrival time series was obtained from the data of arrival and departure times of a bus at a bus stop. For the convenience of calculation, the difference of the arrival times between two bus stops was calculated in seconds. Table 2 shows the example of the dataset. The existing sequence data are 120, 220, 250, and 260 which correspond to four bus stops A, B, C, and D. This means that 120 s is needed for a bus to drive from the starting location to A, 220 s from A to B, 250 s from B to C, and 260 s from C to D. When the bus arrives at C, the prediction of the arrival time to D is only needed. The input sequence is the sequence of all arrival times from the starting location to C, and the output sequence includes 260 s from C to D and the backward sequence from C to the starting location. The length of the input sequence is shorter than that of the output sequence as it only needs to predict the time to the finishing location of a bus. When predicting the time to the finishing location, it only needs to know the sequences before it. When the bus arrives at a bus stop between the starting and finishing location, for a consistent sequence length, the time to the previous bus stop is input as 0.
Figure7 shows the time-series data of real arrival times. The blue and orange line is for the input and output sequence, respectively. The sequence has the predicted times of 0 at the current bus stop. An output sequence has negative numbers to maintain the correctness of the inverted time from the starting location to the bus stop.Table 2
Examples of the arrival time-series dataset format.
Input sequenceOutput sequence(120, 220, and 250)(−120, −220, 0, and 260)(120, 220, and 0)(−120, 0, 250, and 260)(120, 0, and 0)(0, 220, 250, and 260)Figure 7
A diagram of the input and output sequence generated by the time-series data of arrival times.Step 2.
Dataset Normalization
The variables had different dimensions and units which affected the results of data analysis. Thus, normalization was necessary to eliminate the differences. Standardizing with theZ-score and the minimum-maximum values were used so that the final values were ranged between 0-1. The equation for standardization is as follows:(12)xnorm=x−minxmaxx−minx.
Z-score standardization uses the mean and standard deviation of the data and is calculated as follows:(13)xnorm=x−μσ,where µ is the mean and σ is the standard deviation of the data.
In this paper, after sorting the vehicle number and driver number on the route, the sorted sequence was used as the input data and normalized by equation (7). The route number, route direction, departure time (hh:mm), days of the week, holidays, distances from the starting location, weather, and other information were normalized, and their arrival time series were processed by equation (8).


## 3.1. Data Characteristics

A bus was equipped with a device that included a GPS and data communication module. The device transmitted data to a bus scheduling system. Table1 shows the data structure of the reporting system.Table 1
The data structure from a bus.
No.NameTypesRemarks1Route_noLongRoute number2Route_nameStringRoute name3Route_subnoLongRun method no.4Up downIntDirection on route5Bus_noLongVehicle number6SpeedFloatSpeed7datatime_inStringEntry time8driver_noLongDriver number9busstop_noLongSite number10busstop_nameStringSite name11busstop_lngDoubleSite longitude12busstop_latDoubleSite latitude13busstop_serialIntSite serial number14busstop_typeIntSite type15pack_datetimeStringOutbound time16inform_typeIntInform mode17netpack_typeIntEntry modeThe data of arrival and departure of a bus at a bus stop consists of route, speed, arrival and departure time, coordination, and driver’s number. For obtaining the Lasso variable correlation [8], the bus number, number of bus stops, days in the week, distances between bus stops, arrival and departure times, and weather were included, too. The variables were grouped into two: dynamic and static variables [14]. The dynamic variables include driving times between bus stops, staying times, and weather, while the static variables include a route, direction, vehicle model, driver, arrival and departure times, days of the week, holidays, and working days. We selected variables related to the route and the arrival times at the previous stops as the input features to predict the arrival time at a bus stop. Input and output features are as follows:Input feature: [route_no, direction, bus_no, driver_no, departure hours, departure minutes, days of the week, holidays, distance, and weatherxt−k,…,xt]Output prediction series: (−xt−k,…,0,xt+1,…,xt+n)wherext is the difference of the arrival time between the current station and the previous station.


## 3.2. Data Preprocessing

Step 1.
Generating a Sample Dataset
According to the data in Section3.1, an arrival time series was obtained from the data of arrival and departure times of a bus at a bus stop. For the convenience of calculation, the difference of the arrival times between two bus stops was calculated in seconds. Table 2 shows the example of the dataset. The existing sequence data are 120, 220, 250, and 260 which correspond to four bus stops A, B, C, and D. This means that 120 s is needed for a bus to drive from the starting location to A, 220 s from A to B, 250 s from B to C, and 260 s from C to D. When the bus arrives at C, the prediction of the arrival time to D is only needed. The input sequence is the sequence of all arrival times from the starting location to C, and the output sequence includes 260 s from C to D and the backward sequence from C to the starting location. The length of the input sequence is shorter than that of the output sequence as it only needs to predict the time to the finishing location of a bus. When predicting the time to the finishing location, it only needs to know the sequences before it. When the bus arrives at a bus stop between the starting and finishing location, for a consistent sequence length, the time to the previous bus stop is input as 0.
Figure7 shows the time-series data of real arrival times. The blue and orange line is for the input and output sequence, respectively. The sequence has the predicted times of 0 at the current bus stop. An output sequence has negative numbers to maintain the correctness of the inverted time from the starting location to the bus stop.Table 2
Examples of the arrival time-series dataset format.
Input sequenceOutput sequence(120, 220, and 250)(−120, −220, 0, and 260)(120, 220, and 0)(−120, 0, 250, and 260)(120, 0, and 0)(0, 220, 250, and 260)Figure 7
A diagram of the input and output sequence generated by the time-series data of arrival times.Step 2.
Dataset Normalization
The variables had different dimensions and units which affected the results of data analysis. Thus, normalization was necessary to eliminate the differences. Standardizing with theZ-score and the minimum-maximum values were used so that the final values were ranged between 0-1. The equation for standardization is as follows:(12)xnorm=x−minxmaxx−minx.
Z-score standardization uses the mean and standard deviation of the data and is calculated as follows:(13)xnorm=x−μσ,where µ is the mean and σ is the standard deviation of the data.
In this paper, after sorting the vehicle number and driver number on the route, the sorted sequence was used as the input data and normalized by equation (7). The route number, route direction, departure time (hh:mm), days of the week, holidays, distances from the starting location, weather, and other information were normalized, and their arrival time series were processed by equation (8).


## 4. Analysis Result of the RNN Model

### 4.1. Dataset

The experiment was based on the data collected from March 28 to June 28, 2020, in Linyi, Shandong Province. The data was obtained from buses that ran on route no. 30 which had 36 bus stops (Figure8). Tensorflow-GPU 2.0 was used for data processing and algorithm creation. The numbers of the dataset were 122,336 after pretreatment, 78,303 in the training set, 19,590 in the verification set, and 24,443 in the test set.Figure 8
Linyi city’s 30 bus routes and site distribution.


### 4.2. Training the RNN Prediction Model

Seven network models were designed, trained, verified, and tested by using the preprocessed dataset. Pure LSTM and GRU were RNN models. LSTM-BP and GRU-BP were multiple input models with variable features separated from the time series. Bidirectional LSTM (LSTM-Bi) and LTSM-Stack were LSTM stack models. ConvLSTM was a spatiotemporal sequence model. Table3 shows a model structure and a comparison of the parameters of the RNN network. Pure GRU had the smallest number of the parameter, while the stack model had the largest number.Table 3
Seven RNN network model structures and their parameters.
ClassificationType of networkType of layerOutput sequenceNumber of parametersTotal number of parametersPure RNNPure GRUGRU(None, 41, 64)∗1286415,009Dense(None, 32)1056Dense(None, 33)1089Pure LSTMLSTM(None, 64)1689620,065Dense(None, 32)2080Dense(None, 33)1089Multi-input hybrid modelGRU-BPInput layer[(None, 32, 1)]025,089Input layer[(None, 9)]0GRU(None, 32, 64)12864Dense(None, 16)160GRU(None, 32)9408Concatenate(None, 48)0Dense(None, 32)1568Dense(None, 33)1089LSTM-BPInput layer[(None, 32, 1)]032,129Input layer[(None, 9)]0LSTM(None, 32, 64)16896Dense(None, 16)160LSTM(None, 32)12416Concatenate(None, 48)0Dense(None, 32)1568Dense(None, 33)1089Stacking modelsLSTM-BiBidirectional (LSTM (64))(None, 41, 128)3379278,177Bidirectional (LSTM (32))(None, 64)41216Dense(None, 32)2080Dense(None, 33)1089LTSM-StackLSTM(None, 41, 256)264192525,281LSTM(None, 41, 128)197120LSTM(None, 41, 64)49408LSTM(None, 32)12416Dense(None, 32)1056Dense(None, 33)1089Space-time modelConvLSTMConvLSTM2D(None, 41, 1, 1, 41)62156117,233BN(None, 41, 1, 1, 41)164Flatten(None, 1681)0Dense(None, 32)53824Dense(None, 33)1089The loss function selected the average absolute error (MAE) which was the difference between the prediction and real value. All network parameters were updated using the Adam optimization algorithm. The Adam algorithm performs first-order optimization. The first- and second-order optimizations were used for a dynamic design of independent adaptive learning rates for different parameters. The estimations were better than the traditional gradient descent method. Since the output was a time series, cosine similarity was used to determine the accuracy.Figure9 shows the training loss and accuracy when an epoch is 1000, a batch is 100, and a verification set is 20% of the training set. As the seven models show similar MAEs, only the training output of the ConvLSTM model was selected. In the process of training, the trend of training and validation data is consistent, and there is no fitting case. The super parameters of the selected model are suitable, too. As seen from Figure 10, the pure GRU had fewer training parameters and less training time than pure LSTM. The LSTM-Bi doubled the number of parameters and training time than the pure LSTM. The LSTM-Stack had five times more parameters than other models, but less training time than the ConvLSTM. The ConvLSTM had the longest training time, 5.8 times more parameters, and 12 times longer training time than the pure LSTM.Figure 9
Loss function and accuracy diagram during training.Figure 10
Comparison of training parameters and times of seven RNN models.


### 4.3. Analysis of Results

The test set was used to predict the training model, and the predicted arrival times are shown in Table4.Table 4
List of arrival times predicted by the seven models.
Network classificationNetwork modelPredicted arrival time seriesPure RNNPure LSTMPure GRUMulti-input hybrid modelLSTM-BPGRU-BPStacking modelsLSTM-BiLTSM-StackSpace-time modelConvLSTMThe fitting degree of the real and the predicted value in Table4 shows that the ConvLSTM provides the best prediction. The multi-input hybrid model, which separates the parameters from the time series, not only increased the network complexity but also reduced the prediction accuracy. Table 5 shows the statistics of the prediction results by the seven models. MAE, RMSE, MAE, COS, number of training parameters, and time were used to quantitatively evaluate the seven network models. The prediction accuracy was improved from the pure LSTM to the ConvLSTM, as shown in Figure 11.Table 5
Statistics of the prediction results of seven models.
ModelRMSEMSEMAECOSTrainable parametersTraining time (s)Pure LSTM37.78921428.02923.13000.949220,0655120Pure GRU37.75751425.630223.00650.949415,0095820LSTM-BP37.68181419.919722.89060.949832,1296812GRU-BP37.87381434.428222.88100.949625,0897886LSTM-Bi37.46151403.364522.55480.950778,1779126LTSM-Stack34.62791199.096318.97670.9585525,28113172ConvLSTM34.38121182.073417.08760.9595117,23361773MAE: mean absolute error; RMSE: root mean square error; COS: cosign similarity.Figure 11
Prediction accuracies of the seven models.The results reveal the following:(1)
The GRU was more efficient than the LSTM model with fewer parameters and considerable accuracy(2)
The LSTM models except the ConvLSTM had more parameters and higher network accuracy than other models(3)
The dataset property did not influence the results of the models but the complexity of the models(4)
The ConvLSTM showed the highest accuracy as it processed the data of time and space, which indicated the need to include the space-related propertiesIn the process of arrival time-series prediction, the arrival times at subsequent bus stops were based on those at the previous bus stops. The ConvLSTM network model was selected to analyze the prediction accuracy through one- and two-step prediction and total time prediction.Figure12 shows the test sample set on the x-axis and the difference between the predicted and real values on the y-axis. The mean and RMSE were calculated from the mean values and mean square deviation of the differences. The one-step prediction had the highest accuracy, and the total time prediction (multistep prediction) showed the lowest accuracy. The regularity in the histogram of Figure 12 reveals that the one-step prediction has the smallest deviation and the highest error, which is related to the accumulation and propagation of errors in the prediction of the arrival times of the subsequent bus stops.Figure 12
Multistep prediction comparison chart. (a) One step. (b) Two step. (c) Total. (d) Histogram.
(a)(b)(c)(d)


## 4.1. Dataset

The experiment was based on the data collected from March 28 to June 28, 2020, in Linyi, Shandong Province. The data was obtained from buses that ran on route no. 30 which had 36 bus stops (Figure8). Tensorflow-GPU 2.0 was used for data processing and algorithm creation. The numbers of the dataset were 122,336 after pretreatment, 78,303 in the training set, 19,590 in the verification set, and 24,443 in the test set.Figure 8
Linyi city’s 30 bus routes and site distribution.


## 4.2. Training the RNN Prediction Model

Seven network models were designed, trained, verified, and tested by using the preprocessed dataset. Pure LSTM and GRU were RNN models. LSTM-BP and GRU-BP were multiple input models with variable features separated from the time series. Bidirectional LSTM (LSTM-Bi) and LTSM-Stack were LSTM stack models. ConvLSTM was a spatiotemporal sequence model. Table3 shows a model structure and a comparison of the parameters of the RNN network. Pure GRU had the smallest number of the parameter, while the stack model had the largest number.Table 3
Seven RNN network model structures and their parameters.
ClassificationType of networkType of layerOutput sequenceNumber of parametersTotal number of parametersPure RNNPure GRUGRU(None, 41, 64)∗1286415,009Dense(None, 32)1056Dense(None, 33)1089Pure LSTMLSTM(None, 64)1689620,065Dense(None, 32)2080Dense(None, 33)1089Multi-input hybrid modelGRU-BPInput layer[(None, 32, 1)]025,089Input layer[(None, 9)]0GRU(None, 32, 64)12864Dense(None, 16)160GRU(None, 32)9408Concatenate(None, 48)0Dense(None, 32)1568Dense(None, 33)1089LSTM-BPInput layer[(None, 32, 1)]032,129Input layer[(None, 9)]0LSTM(None, 32, 64)16896Dense(None, 16)160LSTM(None, 32)12416Concatenate(None, 48)0Dense(None, 32)1568Dense(None, 33)1089Stacking modelsLSTM-BiBidirectional (LSTM (64))(None, 41, 128)3379278,177Bidirectional (LSTM (32))(None, 64)41216Dense(None, 32)2080Dense(None, 33)1089LTSM-StackLSTM(None, 41, 256)264192525,281LSTM(None, 41, 128)197120LSTM(None, 41, 64)49408LSTM(None, 32)12416Dense(None, 32)1056Dense(None, 33)1089Space-time modelConvLSTMConvLSTM2D(None, 41, 1, 1, 41)62156117,233BN(None, 41, 1, 1, 41)164Flatten(None, 1681)0Dense(None, 32)53824Dense(None, 33)1089The loss function selected the average absolute error (MAE) which was the difference between the prediction and real value. All network parameters were updated using the Adam optimization algorithm. The Adam algorithm performs first-order optimization. The first- and second-order optimizations were used for a dynamic design of independent adaptive learning rates for different parameters. The estimations were better than the traditional gradient descent method. Since the output was a time series, cosine similarity was used to determine the accuracy.Figure9 shows the training loss and accuracy when an epoch is 1000, a batch is 100, and a verification set is 20% of the training set. As the seven models show similar MAEs, only the training output of the ConvLSTM model was selected. In the process of training, the trend of training and validation data is consistent, and there is no fitting case. The super parameters of the selected model are suitable, too. As seen from Figure 10, the pure GRU had fewer training parameters and less training time than pure LSTM. The LSTM-Bi doubled the number of parameters and training time than the pure LSTM. The LSTM-Stack had five times more parameters than other models, but less training time than the ConvLSTM. The ConvLSTM had the longest training time, 5.8 times more parameters, and 12 times longer training time than the pure LSTM.Figure 9
Loss function and accuracy diagram during training.Figure 10
Comparison of training parameters and times of seven RNN models.


## 4.3. Analysis of Results

The test set was used to predict the training model, and the predicted arrival times are shown in Table4.Table 4
List of arrival times predicted by the seven models.
Network classificationNetwork modelPredicted arrival time seriesPure RNNPure LSTMPure GRUMulti-input hybrid modelLSTM-BPGRU-BPStacking modelsLSTM-BiLTSM-StackSpace-time modelConvLSTMThe fitting degree of the real and the predicted value in Table4 shows that the ConvLSTM provides the best prediction. The multi-input hybrid model, which separates the parameters from the time series, not only increased the network complexity but also reduced the prediction accuracy. Table 5 shows the statistics of the prediction results by the seven models. MAE, RMSE, MAE, COS, number of training parameters, and time were used to quantitatively evaluate the seven network models. The prediction accuracy was improved from the pure LSTM to the ConvLSTM, as shown in Figure 11.Table 5
Statistics of the prediction results of seven models.
ModelRMSEMSEMAECOSTrainable parametersTraining time (s)Pure LSTM37.78921428.02923.13000.949220,0655120Pure GRU37.75751425.630223.00650.949415,0095820LSTM-BP37.68181419.919722.89060.949832,1296812GRU-BP37.87381434.428222.88100.949625,0897886LSTM-Bi37.46151403.364522.55480.950778,1779126LTSM-Stack34.62791199.096318.97670.9585525,28113172ConvLSTM34.38121182.073417.08760.9595117,23361773MAE: mean absolute error; RMSE: root mean square error; COS: cosign similarity.Figure 11
Prediction accuracies of the seven models.The results reveal the following:(1)
The GRU was more efficient than the LSTM model with fewer parameters and considerable accuracy(2)
The LSTM models except the ConvLSTM had more parameters and higher network accuracy than other models(3)
The dataset property did not influence the results of the models but the complexity of the models(4)
The ConvLSTM showed the highest accuracy as it processed the data of time and space, which indicated the need to include the space-related propertiesIn the process of arrival time-series prediction, the arrival times at subsequent bus stops were based on those at the previous bus stops. The ConvLSTM network model was selected to analyze the prediction accuracy through one- and two-step prediction and total time prediction.Figure12 shows the test sample set on the x-axis and the difference between the predicted and real values on the y-axis. The mean and RMSE were calculated from the mean values and mean square deviation of the differences. The one-step prediction had the highest accuracy, and the total time prediction (multistep prediction) showed the lowest accuracy. The regularity in the histogram of Figure 12 reveals that the one-step prediction has the smallest deviation and the highest error, which is related to the accumulation and propagation of errors in the prediction of the arrival times of the subsequent bus stops.Figure 12
Multistep prediction comparison chart. (a) One step. (b) Two step. (c) Total. (d) Histogram.
(a)(b)(c)(d)


## 5. Conclusion

The public transport system is a complex system with a high degree of uncertainty. The system is understood as a multistep prediction problem in which uncertainty leads to poor prediction accuracy. This paper first analyzed the main variables affecting this uncertainty, and then, the variables such as route, direction, vehicle, driver, departure hour, departure minute, day of the week, holiday, distance from the starting location, and weather were selected. The arrival time series before the current bus stops was also selected. These variables fully reflected the impact on the arrival time-series prediction. Among RNN networks for time-series analysis, we processed the data by using seven different network models in four different types of networks.We analyzed and compared the predictive power of the seven RNN models with the variables and parameters in the measured dataset. We noticed an improvement in prediction accuracy by adding variables in one- and two-step prediction models, but not in the multistep (total time prediction) model. The multistep model increased the network complexity only. The ConvLSTM showed the highest prediction accuracy with spatiotemporal data. The statistics of one-, two-, and multistep prediction showed that the accumulation and propagation of the sequence prediction error caused more steps and a large deviation of the predicted time. The accurate bus arrival time prediction encourages more people to use buses for transportation and allows operating companies to optimize bus schedules for increasing the efficiency of their operation. This also improves the traffic condition in cities.Accurate bus arrival information also relieves the anxiety of users by decreasing waiting time and helps to provide passengers with an improved service. The accurate prediction of bus arrival times can be integrated into an intelligent bus scheduling system in a smart transportation system. Such a system improves the management of a public transport system, increases the economic benefits of the system, and ultimately brings social benefits.



---

*Source: 6636367-2021-03-12.xml*