# Roll Motion Prediction of Unmanned Surface Vehicle Based on Coupled CNN and LSTM
## 2 Methodology
### Convolution-Based USV Sensor Data Feature Extraction  
A typical CNN is alternately performed by several convolution layers, pooling layers, and fully connected layers, as shown in Figure 2. The convolution layer extract features from the input data by performing a convolution operation. This is the core of the CNN. For one-dimensional input data, the convolution operation with a one-dimensional kernel is performed as follows:  
\[s(t)=(x*\omega)(t)=\sum_{a}x(a)(t-a), \tag{4}\]  
Figure 1: Framework of the proposed coupled convolutional neural network (CNN) and long short-term memory (LSTM) prediction model. where \(x\) is the input data, \(t\) represents time, \(\omega\) denotes weighting function (also known as the convolution kernel), \(a\) is the position where the convolution kernel is currently sliding, and \(*\) denotes the convolution operation. Feature mapping is implemented by sliding several different convolution kernels. The parameters are shared with each convolution kernel on each convolution layer. The weight-sharing structure of the CNN reduces the network complexity and reduces the number of weights. The pooling layer reduces the size of the output dimension by max pooling or average pooling. After several rounds of convolution and pooling, the multidimensional data will be flattened by the fully connected layer. At present, the typical CNN has been improved according to different tasks. In our proposed prediction model, the input is the sensor data of the USV. These sensor data are different features of USV movements at a certain point in time. These features constitute a multidimensional space of the USV's movements. Similar to the color images, which are a three-dimensional space composed of three channels (RGB), these sensor data can be regarded as a multi-dimensional space composed of multi-channel movement features. Inspired by the CNN feature extraction of color images, in the proposed model, we automatically extract features of these sensor data through the convolution operation. At the CNN layer, the input data are converted to several three-dimensional matrices of \(1\times D\times m\) (also known as M-dimensional row vector). \(D\) denotes the length of the setting time window; \(m\) is the number of channels (also known as the number of features). Macroscopically, it can be described that the input data are one-dimensional time series data of \(m\) channels. The multi-channel one-dimensional convolution operation is used to process the input data. The sliding convolution kernels move vertically to extract features from the \(m\) channels of the input data. The size of the convolution kernel is \(1\times k\times m\), where \(m\) is the number of channels. As shown in Figure 1, there are \(N\) types of convolution kernels, corresponding to \(N\) feature maps. The operation processing of one type convolution kernel is shown in Figure 3. On each channel of input data, the convolution operation is performed with convolution kernels of size \(1\times k\). The resulting matrix of the dot product of the input data and the convolution kernel is summed, and then a basic term is added. The convolution operation can be calculated as follows:  
\[M_{j}=f(\sum_{i=1}^{m}X_{i}\times K_{i}^{j}+b_{j}), \tag{5}\]  
where \(j\) and \(i\) denote the feature map and the channel, respectively; \(M\) is the feature map; \(m\) is the number of channels; \(X\) denotes the input time series data on a channel; \(K\) represents the convolution kernel; \(b_{j}\) means the basic matrices. Note that the parameters of the convolution kernels on the \(m\) channels are not shared, and the bias \(b_{j}\) is shared. With the translation of the convolution kernels in the vertical direction, the convolution operation is repeated to obtain feature maps. The input data are mapped into higher-dimensional feature maps by the convolution layer. The feature extraction of the USV sensor data is realized. The primary purpose of the pooling layer is to simplify the computational  
Figure 2: The typical CNN structure.