# Visual Analysis of Spatiotemporal Data Predictions with Deep Learning Models
## 4 Spatiotemporal Prediction Models
### Deep Learning Models for Temporal Prediction  
We compare temporal prediction and spatiotemporal prediction algorithms to see how the prediction performance changes with and without spatial information. Deep learning for temporal forecast is examined, focusing on RNN, and the representative algorithms are LSTM and GRU. We construct LSTM and GRU architectures as temporal prediction algorithms and convLSTM as a spatiotemporal prediction algorithm.  
LSTM is a type of RNN that is a recurrent neural network designed to resolve the long-term dependencies in RNNs and to achieve faster convergence in training. Time-series training is performed by adding memory cell and a forget gate to the RNN structure. The LSTM cell is largely composed of a forget gate \(f\), input gate \(i\), and output gate \(o\). The input of the LSTM cell consists of a vector \(h_{t}\) for a short-term state state and a vector \(c_{t}\) for a long-term state. In LSTM, the output vector \(y_{t}\), according to the previous state \(h_{t-1}\), \(c_{t-1}\) and input vector \(x_{t}\), is presented as follows [50].  
\[f_{t}=\sigma(W_{xf}\cdot x_{t}+W_{hf}\cdot h_{t-1}+b_{f}) \tag{1}\]  
\[i_{t}=\sigma(W_{xi}\cdot x_{t}+W_{hi}\cdot h_{t-1}+b_{i}) \tag{2}\]  
\[o_{t}=\sigma(W_{xo}\cdot x_{t}+W_{ho}\cdot h_{t-1}+b_{o}) \tag{3}\]  
\[g_{t}=tanh(W_{xg}\cdot x_{t}+W_{hg}\cdot h_{t-1}+b_{g}) \tag{4}\]  
\[c_{t}=f_{t}\bigotimes c_{t-1}+i_{t}\bigotimes g_{t} \tag{5}\]  
\[y_{t},h_{t}=o_{t}\bigotimes tanh(c_{t}), \tag{6}\]  
where \(W_{xf}\), \(W_{xi}\), \(W_{xo}\), \(W_{xg}\) are weight matrices for the layers connected to the input vector \(x_{t}\), and \(W_{hf}\), \(W_{hi}\), \(W_{ho}\), \(W_{hg}\) are weight matrices for the layers connected to the short-termstate \(h_{t-1}\). Additionally, \(b_{f}\), \(b_{i}\), \(b_{o}\), and \(b_{g}\) are biases for four layers. The \(\bigotimes\) is an element-wise matrix multiplication. The current short-term state \(h_{t}\) is affected by the long-term state \(c_{t-1}\) and the current long-term state \(c_{t}\) is calculated based on the long-term state \(c_{t-1}\) at the previous time and the input gate \(i_{t}\) at the present time. LSTM resolves the long-term dependence problem in RNN by transmitting the long-term state and prevents the vanishing of the gradient, using \(tanh\) as a cell activation function.  
The GRU algorithm utilizes only one state vector \(h_{t}\) and controls both the forget gate and input gate with one gate controller, \(z_{t}\). The GRU is presented as follows [51].  
\[r_{t}=\sigma(W_{xx}\cdot x_{t}+W_{hr}\cdot h_{t-1}+b_{r}) \tag{7}\]  
\[z_{t}=\sigma(W_{xz}\cdot x_{t}+W_{hz}\cdot h_{t-1}+b_{z}) \tag{8}\]  
\[g_{t}=tanh(W_{xg}\cdot x_{t}+W_{hg}\cdot(r_{t}\bigotimes h_{t-1})+b_{g}) \tag{9}\]  
\[h_{t}=z_{t}\bigotimes h_{t-1}+(1-z_{t})\bigotimes g_{t}. \tag{10}\]  
The GRU algorithm works similar to LSTM and can perform time-series training with fewer parameters. However, since only one state is stored, it is difficult to analyze the state value of each cell. In this paper, we choose LSTM and GRU as temporal prediction algorithms and train the data to compare model performances.