# Animal Movement Prediction Based on Predictive Recurrent Neural Network
## 3 Methods
### Movement Prediction  
In this section, we describe how to construct a movement prediction model using predictive RNNs. The predictive RNN structure is known to give better accuracy for sequence-based prediction than single or shallow structures. Predictive RNN structure such as convolutional long short-term memory (LSTM) has a stack of learning and memory units, which is effective in solving sequence prediction problems [38; 39]. For instance, [PERSON] et al. introduced a convolutional LSTM network approach for precipitation nowcasting. Using the Radar Echo dataset, which has a sequence of weather satellite images, they constructed a prediction model using the continuous stack of convolution LSTMs [40]. Likewise, we consider the movement prediction problem as a sequence-to-sequence prediction. For the animal movement observation period \(k\), we can define a sequence of movement densities of size \(T\) defined by the input size and \(k\) by \(X^{k}=\left\{X^{k}_{1},X^{k}_{2}\ldots,\ X^{k}_{T}\right\}\). Usually, each \(X^{k}_{i}\) has a centroid of movement ranges, and the centroid shows the maximum density value of KDE. Generally, for a given input sequence \(\left\{X^{k}_{T-\tau},X^{k}_{T-\tau+1},\ldots,\ X^{k}_{T}\right\}\), our model predicts its movement density sequence \(\left\{X^{k}_{T+1},X^{k}_{T+2}\ldots,\ X^{k}_{T+\tau}\right\}\), Figure 5 illustrates an example of \(X^{k}_{T}\) where the observation period \(k\) is one month. To predict animal movements, we use the PredRNN++ model [41]. RNN architecture has been widely used in the sequence data prediction. However, standard RNN can't handle long-term temporal dependency because its loss gradient deteriorates exponentially over time. Despite convolutional LSTM being suggested as an alternative to standard RNN, it showed the vanishing gradient problem. PredRNN++ differs from the convolutional LSTM in terms of internal structure and mechanism of gradient transition to solve the back propagation problem. Figure 6 compares the structural differences of convolutional stacking LSTM, PredRNN [42], and PredRNN++. Figure 5: Examples of movement density sequence–(**a**) 12 Dec. 2013 to 12 Jan. 2014; (**b**) 12 Jan. 2013 to 12 Feb. 2014; (**c**) 12 Feb. 2014 to 12 Mar. 2014; (**d**) 12 Apr. 2014 to 12 May 2014; (**e**) 12 May 2014 to 12 Jun. 2014. PredRNN and PredRNN++ have in common that they use a spatiotemporal memory transient concept, but PredRNN++ differs from PredRNN in that it has unique structures of increasing recurrent depth. PredRNN++ has a cascade spatiotemporal memory that is effective in analyzing spatial correlation. Its cascade LSTM unit has a dual memory form of temporal memory and spatial memory. The spatial memory, \(M_{t}^{k}\), depends on \(M_{t-1}^{k}\) from the transient path, which is represented by dotted lines in Figure 6c. The temporal memory \(C_{t}^{k}\) depends on previous state \(C_{t-1}^{k}\) and is adjusted by current forget, input and input modulation gate. Here, \(k\) indicates the vertical depth of the layer. Another difference is the existence of the gradient highway unit (GHU), which prevents quick vanishing and can send the gradient information to the deeper layer. The advantages of GHU were confirmed through various experiments [41]. Figure 7 shows the structural comparison of convolutional LSTM, stacked LSTM, and cascade LSTM. The green, blue, and red boxes indicate standard convolutional LSTM, stacked LSTM, and cascade LSTM, respectively. The element notations of structures are as follows: The sequence input at time \(t\) (which denotes \(X_{1},\ \dots\ X_{t}\)), output state \(C_{1}^{k},\ \dots\ C_{t}^{k}\), hidden state \(H_{1}^{k}\ \dots\ H_{t}^{k}\), spatiotemporal memory \(M_{t}^{k}\) input gates \(i_{t}\), forget gates \(f_{t}\), input modulation gate \(g_{t}\), and output gate \(o_{t}\). As shown in Figure 7, to obtain effectiveness of sequence modeling, the cascade LSTM adds non-linear hyperbolic tangent layers to recurrent transitions more than convolutional LSTM and stacked LSTM.