UW-NET: AN INCEPTION-ATTENTION NETWORK FOR UNDERWATER IMAGE CLASSIFICATION
Authors: Miao Yang and Ke Hu
Chongyi Li
Zhiqiang Wei
Venue:  (2020)
Keywords: Underwater image
Convolutional neural network
Image classification
Inception module
Attention module

Abstract: The classification of images taken in special imaging environments except air is the first challenge in extending the applications of deep learning. We report on an UW-Net (Underwater Network), a new convolutional neural network (CNN) based network for underwater image classification. In this model, we simulate the visual correlation of background attention with image understanding for special environments, such as fog and underwater by constructing an inception-attention (I-A) module. The experimental results demonstrate that the proposed UW-Net achieves an accuracy of 99.3% on underwater image classification, which is significantly better than other image classification networks, such as AlexNet, InceptionV3, ResNet and Se-ResNet. Moreover, we demonstrate the proposed IA module can be used to boost the performance of the existing object recognition networks. By substituting the inception module with the I-A module, the Inception-ResnetV2 network achieves a 10.7% top1 error rate and a 0% top5 error rate on the subset of ILSVRC-2012, which further illustrates the function of the background attention in the image classifications.

Under review as a conference paper at ICLR 2020
UW-NET: AN INCEPTIO--ATTENTION NETWORK FOR UNDERWATER IMAGE CLASSIFICATION
Anonymous authors Paper under double-blind review
ABSTRACT
The classification of images taken in special imaging environments except air is the first challenge in extending the applications of deep learning. We report on an UW-Net (Underwater Network), new convolutional neural network (CNN) based network for underwater image classification. this model, we simulate the visual correlation of background attention with image understanding for special environments, such as fog and underwater by constructing an inception-atten -attioninn (I-A) module. The experimental results demonstrate that the proposed UW-Net achieves an accuracy of 99.3% on underwater image classification, which is sig- nificantly better than other image classification networks, such AlexNet, In- C3ptionV3, ResNet and Se-ResNet. Moreover, we demonstrate the proposed I- A module be used to boost the performance of the existing object recogni- tion networks. By substituting the inception module with the I-A module. the Enception-Resnetiie
network achieves a 10.7% topl error rate and a 0% top5 er- ror rate on the subset of ILSVRC-2012, which further illustrates the function of the background attention in the image classifications.
INTRODUCTION
Underwater images and videos contain a lot of valuable information for many underwater scientific researches (Klausner & Azimi-Sadjadd [2119] Peng et al.) (2018). However, the image analysis sys- tems and classification algorithms designed for natural images (Redmon & Farbadil (2018) He al.] 2017) cannot be directly applied to the underwater images due to the complex distortions existed in underwater images (e.g., low contrast, blurring, non-uniform brightness, non-uniform color casting and noises) and there is, to the best of our knowledge, model for underwater image classification. Except for the inevitable distortions exhibited in underwater images, there are other three key prob- lems for the classification of underwater images: (1) the background in underwater images taken in different environments are various; (2) the salient objects such as ruins, fish, diver exist not only in underwater environment, but also in air. The features extracted from the salient objects cannot be relied on primarily in the classification of underwater images; and (3) since the classification of un- derwater images is only dualistic classification task, the structure of the designed network should be simple to avoid the over-fitting
Increasing the depth and width of CNN can usually improve the performance of the model, but is more prone to cause over-fitting when the training dataset is limited, and needs more computational resource (LeCun al., 2013] Srivastava al. 2014). To remit this issue, (Szegedy al.) 2013) proposed the inception module, which simultaneously performs the multi-scale convolution and pooling on level of CNN to output multi-scale features. In addition, the attention mechanism (Chikkerur et al.) 2010 Borji & Itii 2012) is proposed and applied in the recent deep models which takes the advantage that human vision pays attention to different parts of the image depending on the recognition tasks (Mnih et al. [2014] Zhu et al.) 2018 Ba et al.) 2014). Although these strategies play an important role in advancing the field image classifications, we find that the large-scale features such the background area play more important role in the visual attention mechanism when people understanding of underwater images, which is unlike the attention mechanism applied in natural scene image classification (Xiao et al., 2015 Fu al., 2017).
In this paper, we propose an underwater image classification network, called UW-Net. The overview network structure is shown in Fig. T Unlike other models, the UW-Net pays more attention to the

---

Under review as a conference paper at ICLR 2020
Ingut image
Conv layers
Inception-A Anteation modules
Output
Inception Attention Inception-Attentinn
Calemhap
Figure The structure of the UW-Net. The bottom part the output of the eighth layer in I-A module. The red area represents higher response of features for the underwater image clas- sification. As shown, our I-A module concerns more about the background regions of underwater images.
background features of images by construct the creption-attention (I-A) modules and thus achieves better performance. The contributions of this paper are as follows: (i) to the best of our knowledge, it attention module
the first CNN based model for underwater image classification; (ii) an proposed, which joints the multi-dimensional inception module with the attention module to real- ize the multiple weighting of the output various scales features; (iii) this work is first attempt to simulate the visual correlation between understanding images and background areas through I-A modules.
The rest of the paper is organized as follows: Section 2 introduces the related work. The proposed UW-Net described in Section Section illustrates the experimental results and analysis, and summarize this paper in Section
RELATED WORK
As mentioned before, there is less work focusing on underwater image classification. Thus, we mainly introduce the classification models designed for natural scenes and the recent attention mech- anism which is incorporated in our network in this section.
2.1 IMAGE CLASSIFICATION MODELS
Since Krizhevsky et al. (012)) won the ImageNet Large Scale Visual Recognition Competition (ILSVRC) (Deng et 2009), CNNs become more and more popular in the application of image
recognition tasks. Many CNNs pursue better performance by means of superimposing more convo- lution layers, but the number of the parameters concomitantly increased. As the depth increases (Simonyan & Zisserman 2014), the gradient of the network will disappear or explode in the training process. Particularly, (Szegedy et (2015) proposed the "GoogleNet" model, which the incep- tion architecture is first proposed. The x convolution is also applied as dimension reduction technique (Lin al. 2013) and extract more nonlinear characteristics the features. By fusing the features extracted from multi-scale convolutions, better image representation is obtained in the deep layers of the network. Based on the initial inception structure, multiple network structures such Inception V3 (Szegedy et al. 2016) and Inception V4 (Szegedy et al. (017)) further proposed. He et al. (2016) proposed the Resnet, in which the residual modules are proved to be effective in solving the gradient disappearance problem of deep convolution networks.
An important feature of human vision is that people usually focus more on a certain area of the whole scene while ignoring other areas (Mnih et al. 2014 Treisman & Geladel 1980), which is called
2

---

Under review as a conference paper at ICLR 2020
the attention mechanism (AM). Hu al. (2018) proposed the SeNet by constructing the attention mechanism of feature channels, and won the championship of ILSVRC2017. Wang et al. (2017) proposed the residual attention network (RAN), which also combined the attention mechanism with the residual modules, and achieves 4.8% top-5 error rate on HLSVRC 2012 dataset. These works are all evidences of the effectiveness of the attention mechanism in deep learning models one hand, and on the other, none of these studies on attention mechanism focus on background features.
2.2 UNDERWATER IMAGING
Although the performance of the existing image classification models has exceeded that of human beings in some classification tasks (Huang et al., 2017 He et al. (016), most of the existing classi- fication models assume that images have legible texture and uniform features. However, when light propagating through the water, the absorption and scattering determined by the internal optical prop- erty (IOP) of the water affect the process of underwater imaging. Not only the water body, but also the dissolved organic matter and small floating particles (called sea snow), whose concentration and species vary greatly, also affect the underwater image quality (Kjerstad [214] Johnsen al. 2009). With the depth increases in water, the color of light disappears according to their wavelengths. Ar- tificial lighting often results in uneven lighting, creates bright spots in the image, and makes the scattering of suspended matter worse (Yang& & Sowmya 2013). These challenges make the design of effective underwater image classification algorithm difficult.
Moreover, human recognition of underwater images is often based on the background features of the images. In the next section, we use the inception module with multiple sizes of receptive fields to extract features and simulate the human attention mechanism by combining an attention module emphasizing the background features of underwater images.
PROPOSED APPROACH
The network structure of UW-Net is constructed based on multiple I-A modules, shown in Fig.
The network can be extended with more I-A modules for other complex visual classification tasks. An underwater image is first forwarded to a convolutional layer with x 7 size of kernels to obtain large reception fields. An auxiliary classification branch is added after two I-A modules, and the output of the auxiliary branch will be used to optimize the network part of the loss function. As follows, we will introduce the key components in the UW-Net, i.e., I-A module including inception module and attention module, and classification branch in details.
3.1 INCEPTION-ATTENTIOO MODULE
The classical inception models (Szgedy et al. 2015 2016 2017) are constructed by multiple sizes of convolutional kernels. The feature maps are processed by different convolution kernels in one inception module, and then merged and forwarded to the next inception module directly. However, not all the extracted features of every convolution kernel are positively related to the current image classification task. For example, the features extracted by convolution kernel with large size tend to describe the global information, which has little effect a fine-gra fine-grained image classification task even though they can be transferred to the deeper layers, and will result in a certain degree of waste of computational resource.
3.1.1 INCEPTION MODULE
Underwater images exhibit the characteristics of large intra-class differences. Furthermore, the po- sitions and proportions of the background areas vary in underwater images, and the recognition of underwater image is based more on the global features of the image. In addition, the quality of most underwater images is poor due to the effects of lighting absorption and scattering. In view of these characteristics of the underwater images, we adopt convolution kernels with larger size and the av- erage pooling to reduce the impact of local features of the image the final classification (Boureau jet al. 2010). In the experiment, we find that the best classification results can be obtained by using convolution kemel sizes of
x 5 and 7 in the inception module.
3

---

Under review as a conference paper at ICLR 2020
I-A Module
I-A Module
Inception
Attentiuon
Inception
Attentiuon
Inpul
Stem
Auxiliary Classification Branch
Classification Branch
Outpul (Au))
Output(main)
Maxpool
Avgpool
Inception
Avgpool
Softmax
conv
conv
conv
conv
FC
Figure 2: The structure of the UW-Net. After two I-A modules, one branch serves as an auxiliary classification branch and one branch serves the backbone of the network. The final model has two outputs, in which the output of the auxiliary branch is used to compute the loss function.
3.1.2 ATTENTION MODULE
We construct the attention module based on soft attention (Xu et al. 2015), shown in Fig. which consists of a trunk branch and a mask branch to simulate the recognition of underwater im- age by humans beings (Wang al., 2017: Hu et al., 2018). Inspired by the residual network (He et al. 2016), the trunk branch takes the output of the previous layer as input directly so that the basic features of the image can be transmitted to the deep layers of the network, and the gradient disappearance and the gradient explosion can be remitted. On the other hand, the nown-sampling operation firstly performed in the mask branch, and then up-sampling by bilinear interpolation is used at the last step to keep the same size of the feature map with the input. The activation function of the first and second convolutional layers are Relu (Nair & Hinton 2010) and Sigmoid (Han & Moraga 1995). respectively. The adaptive weight N (z) for a point x of the original features map P(x), in the range of [0,1], can be learned after the mask branch. The output of the attention module F(x) can be expressed as:
F F()) = -1 (1+ (z)) x (x).
(1)
For (xx) approximating 1, F(x) will be near twice the value the original feature P(x), which means that for the features that are valid for the current classification, more attention will be given. On the contrary, for a (xx) approximating 0, the output of the attention module will approximate the original feature P(x).
3.2 AUXILIARY CLASSIFICATION BRANCH AND LOSS FUNCTION
In the proposed UW-Net, an auxiliary classification branch is introduced the output of the second I-A module to reduce the risk of over-fitting. The convergence curve of the UW-Net shown in It is obvious that adding auxiliary classifier can not only accelerate the convergence, but also improve the accuracy on the test set.
Fig. 4
The loss function J of the UW-Net can be expressed follows:
J= Jo + J1 tax
(2)
where Jo is the entropy of the final output of the model and the real label of the image, J1 is the cross entropy of the output of the model's auxiliary classification branch and the actual label, a the weight attenuation coefficient of the network, and L. is the 1.2 regularization term.
4

---

Under review as a conference paper at ICLR 2020
branch
F(6)-(11N)) (N))) Xptx)
Trunk branch
sample
convolution
8
IX convolution
Down sample
pix)
Figure 3: The architecture of the attention module.
0.99
0.98
Accuracy
0.97
Auxiliary dassilier
No cowuliaarydassifii
100
150
200
750
300
Iteration number
Figure 4: The accuracy curves of models on the test set.
3.3 THE UW-NET NETWORK
The UW-Net is constructed by two I-A modules proposed above, and the detailed blocks and param- eter settings are reported in Table | Before the first I-A module, the size the image is reduced to 36 x 36, and the channel of the input image is increased to 32 by two convolutional layers and one pooling operation. After the I-A module, the size of the feature map is reduced and the channel of the feature map increased. The final prediction of the model is obtained after average pooling (Avgpool) and fully connected (FC) layer.
EXPERIMENTS
In this section, series of experiments are conducted to demonstrate the performance of the UW-Net in underwater image classification. Additionally, the effectiveness of the proposed I-A module in improving the performance existing inception based networks is investigated. We compare the UW-Net with several typical image classification models and inception model based networks. The

---

Under review as a conference paper at ICLR 2020
Table 1: The network structure of UW-Net.
Type
Patch size/stride or remarks
Input size
Conv
7 x 7/2
299 299 3
Conv
3 x 3/1
147 147 x 16
Pool
3 x 3/2
73 x 73 x 32
1xI-A module
As in figure? 2
36 x 36 x 32
1 1xI-A module
As in figure 2
36 x 36 x 128
1 x Inception
As in Section 3.1.1
17 x 17 x 512
Avgpool
8 x 8/1
x 8 X 1024
Dropout
0.5
X 1 1024
FC
logits
1 X 1 X 512
Softmaxl
classifier
x 1 X
models for comparison are re-trained in the same dataset without changing the structure, only the parameters are optimized.
4.1 BENCIIMARK DATASET
To ensure the diversity of underwater images, we collect more than 4,000 underwater images from ImageNet dataset JAMSTEC dataset underwater rock image dataser and online underwater im- ages. These images are labeled as underwater images. In addition, more than 5,000 non-ume -on-underwater images from the ImageNet including birds, cars, food, airplanes, cats, etc. selected and labeled as non-underwater images.
4.2 TRAINING AND ANALYSIS
The proportion of the samples used in training and testing is 70% and 30%, respectively. To reduce the risk of over-fitting, we augment the data by random clipping and flipping. We use the initializa- tion method proposed by He et al. (2015) to initialize weights and train the UW-Net by using the SGD (Stochastic Gradient Descent) (Ketkar (014)) with a mini-batch size of 32. The weight decay, momentum and initial learning rate set to 0.001, 0.9 and 0.001, respectively. The learning rate decreased ten times of its original value at 1k and 2k iterations. The training end at 3k iterations. The loss curve is shown in Fig.
loss
tag: loss/loss
250
200
150
50.0
0.00
1.00kk
2.000k
3.000k
Figure The curve of the classification training loss of the UW-Net.
'[Online]. Available:
http://www.immggneet..cc
2[Online]. Available:
[[Online] Available: http::///thhhooomm/saaaaa/aaEtee

---

Under review as a conference paper at ICLR 2020
Table 2: The comparisons with the competing models underwater image classification.
Network
Mult-Adds(Million)
(arameters(Milion)
Test Accuracy
AlexNet
720
60
96.5%
VGG16
15300
138
97.0%
Googlenet
1550
6.8
98. %
ResNer-50
3860
25.6
97.8%
RE-ResNet-50
3870
28
98.0%
Our model
750
6.6
99.3%
Original
image
Alexnet
VGG16
Googlenet
Resnet50
SE Resnet50
model
Figure 6: Visualized examples of regional contributions to the final classification. From up to down are: the output of the fifth layer in Alexnet, the output of the thirteenth layer in VGGI6, the output of the twelfth layer in Googlenet, the output of the seventeenth layer in Resnet-50, the output of the seventeenth layer in SE-Resnet-50 50 and the output of the eighth layer in the UW-Net.
4.3 EXPERIMENTAL RESULTS
The UW-Net achieves 100% and 99.3% accuracy on the training and testing datasets, respectively. We also report the comparisons with the AlexNet, VGG16, Inception Resnet-50 and Se-Resnet- 50 on the testing dataset in Table 2 and class activation maps (CAMs) (Selvaraju et al. (017) produced by these models for underwater image classification are shown in Fig. 6 The darker red color in the CAMs represent more importance of the regions to the final classification. As shown in Fig. 6 the interesting areas the UW-Net locate more in the background areas compared to the competing models. The data in Table 2 shows that the UW-Net is superior to the competing models for the task underwater image classification by fewer computation units and parameters, and higher accuracy. By adopting the I-A model in the UW-Net. we achieve higher accuracy with smaller depth of network. At the same time, the UW-Net requires lower computation than the other inception based models.

---

Under review as a conference paper at ICLR 2020
Table 3: The performance of the I-A modules in other networks.
Network
Top-I err.
Top-5 err.
Inception
I-A
Inception
I-A
GoogleNet
18.3%
17.8%
1.2%
1.0%
Inception V3
13.1%
12.6%
0.0%
0.0%
Inception V4
11.2%
11.0%
0.0%
0.0%
Inception
Inception-ResnetV2 ResnetV2
11.0%
10.7%
0.0%
0.0%
4.4 THE PERFORMANCE OF THE I-A MODULE
The I-A module designed in this work can not only be applied in the UW-Net, but also to other common image classification networks. To further verify the generalization ability of the I-A mod- ule in boosting the performance of related models, we embed the proposed I-A module with the max-pooling in the down-sampling of the mask branch into several image classification models in- cluding GoogleNet, InceptionV3, Inception V4 and Sxception-Resnet/y of images are selected from the ILSVRC-2012 dataset, including ships, sharks, dogs, cocks, etc. Each category contains about 1300 images, and total number of 130,000 images is included. The training and testing sets consist of 125,000, and 5000 images respectively. The size of the images is 299 x 299. All the networks are tested and tuned on this dataset in the same way. The experimental results are shown Table 3 It can be seen that the error rates of Top-1 obtained by substituting the inception module with the I-A module are significantly decreased. Such result indicates the gen- cralization and effectiveness of the proposed I-A module, also gives another evidence that attention weighted large-scale image features simulate better the visual understanding mechanism in image classifications.
One hundred categories
CONCLUSION
A new underwater image classification network UW-Net is proposed in this work, wherein an inception-attention module is constructed. In this model, we simulate the visual correlation be- tween understanding images and background areas through I-A modules, which joint the multi- dimensional inception module with the attention module to realize the multiple weighting of the output of various scales of features. The 100% accuracy on the training set and 99.3% accuracy on the testing set of the UW-Net is achieved benefiting from the refinement of the usefulness of multi- scale features by the I-A module. In the future, we will try to improve the performance other underwater image visual analysis models by introducing the proposed I-A module.
REFERENCES
Jimmy Ba, Volodymyr Mnih, and Koray Kavukcuoglu. Multiple object recognition with visual attention. arXiv preprint arXh::1412.7755, 2014.
Ali Borji and Laurent Itti. State-of-the-art in visual attention modeling. IEEE transactions pattern analysis and machine intelligence, 35(1):185-207, 2012.
Y-Lan Boureau, Francis Bach, Yann LeCun, and Jean Ponce. Learning mid-level features for recog- nition. In 2010 JEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2559-2566. Citeseer, 2010.
Sharat Chikkerur, Thomas Serre, Cheston Tan, and Tomaso Poggio. What and where: A bayesian inference theory of attention. Vision research, 50((2):2233-222 22223,222722222 2247,2011 2010.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- crarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, leee, 2009.
Jianlong Fu, Heliang Zheng, and Tao Mei. Look closer to see better: Recurrent attention convolu- tional neural network for fine- ane-grained image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 4438 4446, 2017.
8

---

Under review as a conference paper at ICLR 2020
Jun Han and Claudio Moraga. The influence of the sigmoid function parameters the speed of backpropagation learning. In International Workshop on Artificial Neural Networks, pp. 195- 201. Springer, 1995.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance imagenet classification. In Proceedings of the IEFE international conference on computer vision, pp. 1026-1034, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969. 2017.
Jie Hu, Li Shen, and Gang Sun. Squeere-and-exciin conference on computer vision and pattern recognition, pp. 7132-7141, 2018.
networks. In Proceedings of the IEEE
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2017.
Geir Johnsen, Zsolt Volent, Egil Sakshaug, Fred Sigernes, Lasse H Pettersson, and K Kovacs. Re- mote sensing the barents sea. Ecosystem Barents Sea, 2:139-166, 2009.
N. Ketkar. Stochastic gradient descent. Optimization, 2014.
Ingrid Kjerstad. Underwater imaging and the effect of inherent optical properties on image quality. Master's 's thesis, Institutt for biologi. 2014.
Nick H Klausner and Mahmood Azimi-Sadjall Performance prediction and estimation for un- derwater target detection using multichannel sonar. IEEE Journal of Oceanic Engineering, 2019.
Alex Krizhevsky, Iya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521 2015.
Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. arXiv preprint arXiv:1312.4400, 2013.
Volodymyr Mnih, Nicolas Heess, Alex Graves, et al. Recurrent models of visual attention. In Advances in neural information processing systems, pp. 2204 2212, 2014.
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference machine learning (ICML-10), pp. 807-814, 2010.
Yan- Yan-Tsung Peng, Keming Cao, and Pamela C Cosman. Generalization of the dark channel prior for single image restoration. IEEE Transactions on Image Processing, 27(6):2856-25) 2868, 2018.
Joseph Redmon and Ali Farhadi. Yolov3: arXiv:180..02767, 2018.
An incremental improvement.
arXiv preprint
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 618-626, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409. 1556, 2014.

---

Under review as a conference paper at ICLR 2020
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, 1lya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-1958, 2014.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9, 2015.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2818-2826, 2016.
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thistty-Fistt AAAI Confer- ence on Artificial Intelligence, 2017.
Anne M Treisman and Garry Gelade. A feature-integration theory of attention. Cognitive psychol- ogy, 12(1):97-136, 1980.
Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Cheng Li, Honggang Zhang, Xiaogang Wang, and Xiaoou Tang. Residual attention network for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156-3164, 2017.
Tianjun Xiao, Yichong Xu, Kuiyuan Yang. Jiaxing Zhang, Yuxin Peng, and Zheng Zhang. The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 842-850, 2015.
Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning, pp. 2048-2057, 2015.
Miao Yang and Arcot Sowmya. An underwater color image quality evaluation metric. IEEE Trans- actions Image Processing, 24(12):6062-6011, 2015.
Yousong Zhu, Chaoyang Zhao, Haiyun Guo, Jinqiao Wang, Xu Zhao, and Hanqing Lu. Attention couplenet: Fully convolutional attention coupling network for object detection. IEEE Transac- tions on Image Processing, 28(1):113 13-126, 2018.
10


Reviews:
[{"review_id": "S1ggA9GTFB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "- This works presents yet another incremental modification of well-known architectures (i.e., inception network). Particularly, authors propose to add an attention mechanism in the inception module to pay more attention to a given set of features. To evaluate the proposed network, authors resort to the task of underwater image classification. \n- The technical contribution of this work is rather limited, being the main contribution the application of deep classification models to underwater image classification. \n- Furthermore, authors mention that this work is a first attempt to simulate the visual correlation between understanding images and background areas through I-A modules. Nevertheless, this was never shown, beyond some classification activation maps in Figure 6. If I understood correctly, the classification task basically reduces to predict whether an image is taken underwater or not. Focusing on the background, however, may introduce errors, since I believe that pictures containing mainly sky may trigger the same activations. It would be more interesting to see also the classification activation maps for the negative class (non-underwater images).\n- Additionally, looking at Fig 6, GoogleNet and ResNet seem to provide more meaningful regions than the proposed network in some cases. Again, showing results on non-underwater images would help to better understand how the proposed method works.\n- Authors mislead some messages: ‘classification algorithms designed for natural images cannot be directly applied to the underwater images due to the complex distortions existed in underwater images’. Later in the manuscript, they show that those standard classification algorithms achieve almost the same performance as the propose method (1-2% of difference). Does it mean that this 1-2% of improvement makes these algorithms applicable on this task?? In another example: ‘The classification of images taken in special imaging environments except air is the first challenge in extending the applications of deep learning.’ \n- Some results on Table 3 are useless (e.g., those 5-top error equal to 0%). \n- Authors split the dataset into training and evaluation. Nevertheless, they should also use a validation set to stop the training and pick the best model (based on the validation images) to generate the predictions on the testing set. Otherwise, they may be overfitting the model on the training set. \n- Overall, this paper presents an incremental contribution with respect to existing networks, just to improve 1% the classification performance on an easy task (baseline performance around 98%). Thus, I do not feel that this work may attract the interest of the ICLR attendees.  ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "Hyeii7JAtr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed an underwater image classification network. The current manuscript missed some very important information (see below). Besides, the experimental results are also weak.  \nThe paper mentioned \"complex distortions existed in underwater images (e.g., low contrast, blurring, non-uniform brightness, non-uniform color casting and noises)\" many times. But when the paper introduced the UW-Net structure, it does not explain how the network over-comes these difficulties. The UW-Net structure only considers the factors of the background and attention. Thus I think the proposed network structure is not convincing.\nFor the experimental part, I am afraid the results are also weak. For example, please notice that many network structures have proposed to improve the classification. I think authors should compare more existing works to demonstrate the superiority of the proposed one. \n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "HJeZZLZsYS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper shows a method to classify between underwater and non-underwater images (binary). For this, they proposed a new module in a CNN network, called 'Inception-Attention' module. It combines the inception module (multiple sizes of kernels) with an attention module (learn jointly some masks in order to focus the network to part(s) on the image). Also, their network proposes 2 classifications, by separating the network in 2 branches at the end - with the goal of less overfitting. Their method was compared against other state-of-the-art networks, usually designed for more complex tasks (multiclass classification). Lastly, they showed how their new 'IA module' could be useful in other tasks/networks, by replacing the standard inception network with theirs.\n\nOverall, I found the paper confusing: while some technical aspects are interesting, I still don't get the motivation of this new network for this task. Moreover, I did not understand the goal of the paper until the middle of the paper: I thought it was to do a multi-class classification from underwater images, while it is just a binary classification of underwater/ non-underwater images. I still think there are some valuable arguments, and the last part, 4.4, showing the usefulness of the method for more complex classifications, is to me the most interesting part. I could change my grade if a better motivation or a reorienting of the paper was made.\n\nRemarks/ questions:\n- The writing should be improved as it is even hard to understand some sentences. Some particular help will be given below.\n- My main concern is: have you tested easier classifications methods? It seems that classifying underwater vs. not underwater images would be easy. In fact, even the mean color should be identifiable... do you have a simple baseline to compare to? You are saying that one of the main problems is that salient objects are less visible underwater, there is blurring, ect. : these are for me all arguments why it would be easy to detect underwater images, because standard convolutions will behave differently.\n- Have you tested a simpler CNN? You are right in saying that the state-of-the art methods for multi-class classification are too large for this task. So why to you want to complexify it, and why not use a simpler network?\n- There is no related work on background classification (or 'context' classification); but I am sure that there might be works on this. It would be more interesting than the general image classification models.\n- in 3.1 and 3.2, you are saying 2 opposite things. a) 'the features extracted by a conv. kernel with large size tend to describe the global information [..] and will result in a certain degree of waste of computational resource' and b) 'we adopt convolution kernels and the average pooling to reduce the impact of local features of the image'. Do you want or not want large kernels?\n- 'Moreover, human recognition of underwater images is often based on the background..'. Why? Are you sure? I think it is more based on the color of the image, the texture, ect.\n- Auxiliary classification branch: If you have overfitting problems; why don't you use standard methods for treating overfitting (the first one is to have a smaller and simpler network..)?\n-I don't see a real improvement between the auxiliary classification branch and no auxiliary classifier in the Figure 4. Such a small difference, compared to the large oscillations, is not enough. Why don't you use different runs of your model with different initializations, in order to take the mean?\n- 3.3: not clear: 'the channel of the input image is increased to 32'. I think you want to talk about the number of channels, and not 'the channel'. Same error few lines below.\n- How did you select the 5000 non-underwater images? Randomly? Or just outside views?\n- 4.2: I don't see a validation set. It is important to have 3 sets, one training, one validation, one test. All the tuning of parameters/architectures must be done on the validation, with the test kept hidden until the publication.\n- 4.3: results on the training set should not even be shown...if you want, you can show the validation error, but not the training. An accuracy of 99.3% means that it is an easy task. Yes, it's true that your method works, but i) since you are not using a validation set, you could have just tuned your model until you have a good accuracy on the test set; ii) it might be better to prove the usefulness of your method on a more complex task.\n- 4.4 : this is the important part of the paper I think, you should develop it :)!", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers have issues with the lack of enough experimental results as well as with novelty of the solution proposed. I recommend rejection.
Decision: Reject