# Multi-Feature Fusion with Convolutional Neural Network for Ship Classification in Optical Images
## 1 Introduction
[29] introduced a method based on CNN to extract features from ship images. Their method combined the HOG and hue, saturation, and value (HSV) algorithms to extract edge and color features, respectively, from images, and demonstrated attainment of a classification accuracy of 93.55% when applied to the visible ship dataset. [PERSON] et al. [30] proposed a ship recognition method combined with image segmentation and deep learning feature extraction in video surveillance, the method can effectively identify three types of ships, with an average detection accuracy of 87%. [PERSON] et al. [31] designed a multi-feature structure fusion method based on spectral regression discriminant analysis (SF-SRDA). [PERSON] et al. [32] proposed a classification framework consists of a multi-feature ensemble based on the convolutional neural network (ME-CNN) for optical remote sensing images. According to the current relevant researches, the feature characterization ability and classification accuracy of the ship classification method needs to be further improved. The proposed study aims at attainment of high performance and ship-classification accuracies. Although the CNN can automatically capture structural information and has performed encouragingly, it still has some limitations. Firstly, in the CNN model, low-level features extracted from the first convolutional layer are fed into the top layer via layer-by-layer propagation to generate high-level features. However, the low-level features obtained in the first convolution layer may cause some important low-level information lost, such as edges and contours. Secondly, CNN requires large amounts of labeled data to train models. Network architectures, such as the VGGNet and ResNet, are relatively complex, and their direct application to ship datasets containing few samples may result in significant overfitting, thereby affecting algorithm performance. In view of this, a multi-feature fusion with CNN has been proposed for ship classification. First, an improved shallow layers CNN-based method has been proposed to facilitate the learning of more useful, robust CNN features. Additionally, because there are many mature handcrafted feature extraction methods, such as HOG that can extract the edge of an image, and LBP can extract the local structure and texture information of an image [13]. Therefore, HOG and LBP are selected to describe the ship edge features and texture features more accurately. Subsequently, these three types of features are fused, and the Softmax function is used to classify different types of ship images in the output layer. Major contributions of this paper can be summarized as follows. (1) An improved CNN with shallow layers and relatively few convolution kernels has been proposed to facilitate ship classification. The proposed method automatically extracts and learns features from ship images, thereby avoidingoccurrence of overfitting problems caused by a lack of training samples. Few network parameters can improve the efficiency of the algorithm. (2) The HOG and LBP features are fused with the high-level features extracted by the improved CNN network to further supplement the edge, profile information and spatial texture information of the ship images. The fused features also take the advantages of both high-level and handcrafted features to obtain more comprehensive ship features and improve the ability to describe and identify ships. Moreover, handcrafted features are not affected by the number of labeled samples. Namely, they can be extracted even when the amount of dataset is very small. (3) To facilitate comparison of the proposed method against other deep CNN models, such as AlexNet, VGG-16, and ResNet-18, to validate its effectiveness, the said three networks are separately fine-tune into network architectures suitable for an image size of 64 pixels \(\times\) 64 pixels. (4) We construct a self-built ship dataset, including 9000 images.