# Deep Learning Resolves Representative Movement Patterns in a Marine Predator Species
## 3 Approach
### Recurrent Neural Networks with Confidence Measure  
Some of their trajectory segments may differ significantly when the animals are from different age or gender groups. We therefore propose the problem that provided a trajectory segment, identifying relevant group identities for the owner of the segment. Without loss of generality, we consider that our animal dataset has \(K\) groups according to data labels. Our neural network model is composed of two parts. The first part is an RNN that can extract features from the input trajectory segments. The second part includes two single-layer neural networks, each of which is fed by the output of the aforementioned RNN. These two single-layer networks are used for predicting group labels and for estimating the confidence of the predictions, respectively. Thus, the overall network is a recurrent neural network with a confidence measure. For the first part, we use the long short-term memory (LSTM) network [27; 28] as the basic element for analysis. An LSTM cell for time step \(t\) takes a tuple (\(d_{t}\) and \(\theta_{t}\)) as the input. Each LSTM cell is in its basic form [27] except that the state variable \(h_{t}\) is a vector [28]. In our model, we connect these LSTM cells in accordance with their represented time steps, so that variables at different time steps are not isolated:  
\[h_{t}=\mathbf{f}([d_{t},\theta_{t}],h_{t-1}),\text{for }t=1,2,\cdots,T. \tag{3}\]  
where \(h_{0}\) is initialized randomly. In the second part, we use the hidden state of the last LSTM cell as the features for group prediction. If in total there are \(K\) groups, we define a binary vector \(c\) with length \(K\) as the ground-truth group indicator. Only one entry in \(c\) can be one and the index of that entry corresponds to the group label. We use vector \(\hat{c}\) of the same length to represent an estimator of \(c\). With the value of the last hidden state \(h_{T}\), we use a fully connected layer to compute the animal group estimator \(\hat{c}\) for each trajectory segment. We also use another fully connected layer to find the confidence \(\rho\) of the estimator. The vector of \(\rho\) is of length two. These two layers can be expressed by two equations as below:  
\[\hat{c} =W_{c}h_{T}+b_{c}, \tag{4}\] \[\hat{\rho} =W_{\rho}h_{T}+b_{\rho}, \tag{5}\]  
where \(W_{c}\in\mathbb{R}^{K\times H}\) and \(W_{\rho}\in\mathbb{R}^{2\times H}\) are weight matrices, and \(b_{c}\in\mathbb{R}^{K\times 1}\) and \(b_{\rho}\in\mathbb{R}^{2\times 1}\) are bias vectors. Without loss of generality, we consider the state vector \(h_{T}\) as a column vector of size \(H\). An illustration of the network architecture is shown in Figure 2, in which the two fully connected networks for computing \(\hat{c}\) and \(\hat{\rho}\) can be considered as a single network for computing variable \(y\). Variable \(\hat{c}\) and \(\hat{\rho}\) are then normalized respectively for the robustness of the approach. Since only a scalar is needed as a confidence indicator, we use a softmax function, \(h(\cdot)\), to normalize \(\hat{\rho}\) and then take the leading entry as the indicator. \[\tilde{\rho}=h(\hat{\rho})[0]=\frac{e^{\hat{\rho}[0]}}{\sum_{k=0}^{1}e^{\hat{ \rho}[k]}}. \tag{6}\]  
In practice, trajectory segments corresponding to certain behaviours (e.g., migration) may be quite similar for animals from different groups. In this case, classifiers are most likely to make incorrect or random predictions. We therefore consider these segments unpredictable. On the other hand, some trajectory segments associated with other behaviours may be quite different between different animal groups. The classifiers from these segments can therefore more readily predict the group membership of the animals from these segments, and we consider them as representative segments for the group. Our aim is to design a classifier that makes good predictions on predictable segments while ignoring unpredictable ones.