UW-NET: AN INCEPTION-ATTENTION NETWORK FOR UNDERWATER IMAGE CLASSIFICATION

1 INTRODUCTION . Underwater images and videos contain a lot of valuable information for many underwater scientific researches ( Klausner & Azimi-Sadjadi , 2019 ; Peng et al. , 2018 ) . However , the image analysis systems and classification algorithms designed for natural images ( Redmon & Farhadi , 2018 ; He et al. , 2017 ) can not be directly applied to the underwater images due to the complex distortions existed in underwater images ( e.g. , low contrast , blurring , non-uniform brightness , non-uniform color casting and noises ) and there is , to the best of our knowledge , no model for underwater image classification . Except for the inevitable distortions exhibited in underwater images , there are other three key problems for the classification of underwater images : ( 1 ) the background in underwater images taken in different environments are various ; ( 2 ) the salient objects such as ruins , fish , diver exist not only in underwater environment , but also in air . The features extracted from the salient objects can not be relied on primarily in the classification of underwater images ; and ( 3 ) since the classification of underwater images is only a dualistic classification task , the structure of the designed network should be simple to avoid the over-fitting . Increasing the depth and width of a CNN can usually improve the performance of the model , but is more prone to cause over-fitting when the training dataset is limited , and needs more computational resource ( LeCun et al. , 2015 ; Srivastava et al. , 2014 ) . To remit this issue , ( Szegedy et al. , 2015 ) proposed the inception module , which simultaneously performs the multi-scale convolution and pooling on a level of CNN to output multi-scale features . In addition , the attention mechanism ( Chikkerur et al. , 2010 ; Borji & Itti , 2012 ) is proposed and applied in the recent deep models which takes the advantage that human vision pays attention to different parts of the image depending on the recognition tasks ( Mnih et al. , 2014 ; Zhu et al. , 2018 ; Ba et al. , 2014 ) . Although these strategies play an important role in advancing the field of image classifications , we find that the large-scale features such as the background area play a more important role in the visual attention mechanism when people understanding of underwater images , which is unlike the attention mechanism applied in natural scene image classification ( Xiao et al. , 2015 ; Fu et al. , 2017 ) . In this paper , we propose an underwater image classification network , called UW-Net . The overview network structure is shown in Fig . 1 . Unlike other models , the UW-Net pays more attention to the background features of images by construct the inception-attention ( I-A ) modules and thus achieves better performance . The contributions of this paper are as follows : ( i ) to the best of our knowledge , it is the first CNN-based model for underwater image classification ; ( ii ) an inception-attention module is proposed , which joints the multi-dimensional inception module with the attention module to realize the multiple weighting of the output of various scales of features ; ( iii ) this work is a first attempt to simulate the visual correlation between understanding images and background areas through I-A modules . The rest of the paper is organized as follows : Section 2 introduces the related work . The proposed UW-Net is described in Section 3 . Section 4 illustrates the experimental results and analysis , and we summarize this paper in Section 5 . 2 RELATED WORK . As mentioned before , there is less work focusing on underwater image classification . Thus , we mainly introduce the classification models designed for natural scenes and the recent attention mechanism which is incorporated in our network in this section . 2.1 IMAGE CLASSIFICATION MODELS . Since Krizhevsky et al . ( 2012 ) won the ImageNet Large Scale Visual Recognition Competition ( ILSVRC ) ( Deng et al. , 2009 ) , CNNs become more and more popular in the application of image recognition tasks . Many CNNs pursue better performance by means of superimposing more convolution layers , but the number of the parameters is concomitantly increased . As the depth increases ( Simonyan & Zisserman , 2014 ) , the gradient of the network will disappear or explode in the training process . Particularly , Szegedy et al . ( 2015 ) proposed the ” GoogleNet ” model , in which the inception architecture is first proposed . The 1 × 1 convolution is also applied as a dimension reduction technique ( Lin et al. , 2013 ) and to extract more nonlinear characteristics of the features . By fusing the features extracted from multi-scale convolutions , better image representation is obtained in the deep layers of the network . Based on the initial inception structure , multiple network structures such as Inception V3 ( Szegedy et al. , 2016 ) and Inception V4 ( Szegedy et al. , 2017 ) are further proposed . He et al . ( 2016 ) proposed the Resnet , in which the residual modules are proved to be effective in solving the gradient disappearance problem of deep convolution networks . An important feature of human vision is that people usually focus more on a certain area of the whole scene while ignoring other areas ( Mnih et al. , 2014 ; Treisman & Gelade , 1980 ) , which is called the attention mechanism ( AM ) . Hu et al . ( 2018 ) proposed the SeNet by constructing the attention mechanism of feature channels , and won the championship of ILSVRC2017 . Wang et al . ( 2017 ) proposed the residual attention network ( RAN ) , which also combined the attention mechanism with the residual modules , and achieves 4.8 % top-5 error rate on ILSVRC 2012 dataset . These works are all evidences of the effectiveness of the attention mechanism in deep learning models on one hand , and on the other , none of these studies on attention mechanism focus on background features . 2.2 UNDERWATER IMAGING . Although the performance of the existing image classification models has exceeded that of human beings in some classification tasks ( Huang et al. , 2017 ; He et al. , 2016 ) , most of the existing classification models assume that images have legible texture and uniform features . However , when light propagating through the water , the absorption and scattering determined by the internal optical property ( IOP ) of the water affect the process of underwater imaging . Not only the water body , but also the dissolved organic matter and small floating particles ( called sea snow ) , whose concentration and species vary greatly , also affect the underwater image quality ( Kjerstad , 2014 ; Johnsen et al. , 2009 ) . With the depth increases in water , the color of light disappears according to their wavelengths . Artificial lighting often results in uneven lighting , creates bright spots in the image , and makes the scattering of suspended matter worse ( Yang & Sowmya , 2015 ) . These challenges make the design of an effective underwater image classification algorithm difficult . Moreover , human recognition of underwater images is often based on the background features of the images . In the next section , we use the inception module with multiple sizes of receptive fields to extract features and simulate the human attention mechanism by combining an attention module emphasizing the background features of underwater images . 3 PROPOSED APPROACH . The network structure of UW-Net is constructed based on multiple I-A modules , as shown in Fig . 2 . The network can be extended with more I-A modules for other complex visual classification tasks . An underwater image is first forwarded to a convolutional layer with 7× 7 size of kernels to obtain large reception fields . An auxiliary classification branch is added after two I-A modules , and the output of the auxiliary branch will be used to optimize the network as part of the loss function . As follows , we will introduce the key components in the UW-Net , i.e. , I-A module including inception module and attention module , and classification branch in details . 3.1 INCEPTION-ATTENTION MODULE . The classical inception models ( Szegedy et al. , 2015 ; 2016 ; 2017 ) are constructed by multiple sizes of convolutional kernels . The feature maps are processed by different convolution kernels in one inception module , and then merged and forwarded to the next inception module directly . However , not all the extracted features of every convolution kernel are positively related to the current image classification task . For example , the features extracted by a convolution kernel with large size tend to describe the global information , which has little effect on a fine-grained image classification task even though they can be transferred to the deeper layers , and will result in a certain degree of waste of computational resource . 3.1.1 INCEPTION MODULE . Underwater images exhibit the characteristics of large intra-class differences . Furthermore , the positions and proportions of the background areas vary in underwater images , and the recognition of underwater image is based more on the global features of the image . In addition , the quality of most underwater images is poor due to the effects of lighting absorption and scattering . In view of these characteristics of the underwater images , we adopt convolution kernels with larger size and the average pooling to reduce the impact of local features of the image on the final classification ( Boureau et al. , 2010 ) . In the experiment , we find that the best classification results can be obtained by using convolution kernel sizes of 1× 1 , 5× 5 and 7× 7 in the inception module . 3.1.2 ATTENTION MODULE . We construct the attention module based on soft attention ( Xu et al. , 2015 ) , as shown in Fig . 3 , which consists of a trunk branch and a mask branch to simulate the recognition of underwater image by humans beings ( Wang et al. , 2017 ; Hu et al. , 2018 ) . Inspired by the residual network ( He et al. , 2016 ) , the trunk branch takes the output of the previous layer as input directly so that the basic features of the image can be transmitted to the deep layers of the network , and the gradient disappearance and the gradient explosion can be remitted . On the other hand , the down-sampling operation is firstly performed in the mask branch , and then up-sampling by bilinear interpolation is used at the last step to keep the same size of the feature map with the input . The activation function of the first and second convolutional layers are Relu ( Nair & Hinton , 2010 ) and Sigmoid ( Han & Moraga , 1995 ) , respectively . The adaptive weight N ( x ) for a point x of the original features map P ( x ) , in the range of [ 0,1 ] , can be learned after the mask branch . The output of the attention module F ( x ) can be expressed as : F ( x ) = ( 1 +N ( x ) ) × p ( x ) . ( 1 ) For a N ( x ) approximating 1 , F ( x ) will be near twice the value of the original feature P ( x ) , which means that for the features that are valid for the current classification , more attention will be given . On the contrary , for a N ( x ) approximating 0 , the output of the attention module will approximate the original feature P ( x ) .

Summary: This paper shows a method to classify between underwater and non-underwater images (binary). For this, they proposed a new module in a CNN network, called 'Inception-Attention' module. It combines the inception module (multiple sizes of kernels) with an attention module (learn jointly some masks in order to focus the network to part(s) on the image). Also, their network proposes 2 classifications, by separating the network in 2 branches at the end - with the goal of less overfitting. Their method was compared against other state-of-the-art networks, usually designed for more complex tasks (multiclass classification). Lastly, they showed how their new 'IA module' could be useful in other tasks/networks, by replacing the standard inception network with theirs.