# A Hybrid Architecture Based on CNN for Image Semantic Annotation

Yongzhe Zheng, Zhixin Li, Canlong Zhang

► To cite this version:

Yongzhe Zheng, Zhixin Li, Canlong Zhang. A Hybrid Architecture Based on CNN for Image Semantic Annotation. 9th International Conference on Intelligent Information Processing (IIP), Nov 2016, Melbourne, VIC, Australia. pp.81-90, 10.1007/978-3-319-48390-0_9. hal-01615004

HAL Id: hal-01615004

https://inria.hal.science/hal-01615004

Submitted on 11 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
A hybrid architecture based on CNN for image semantic
annotation

Yongzhe Zheng¹, Zhixin Li¹,²,*, Canlong Zhang¹,²

1. Guangxi Key Lab of Multi-source Information Mining & Security, Guangxi Normal University, Guilin, 541004, China
2. Guangxi Experiment Center of Information Science, Guilin 541004, China

zhengyzms@163.com, lizx@gxnu.edu.cn, clzhang@gxnu.edu.cn

**Abstract.** Due to semantic gap, some image annotation models are not ideal in semantic learning. In order to bridge the gap between cross-modal data and improve the performance of image annotation, automatic image annotation has became an important research hotspots. In this paper, a hybrid approach is proposed to learn automatically semantic concepts of images, which is called Deep-CC. First we utilize the convolutional neural network for feature learning, instead of traditional methods of feature learning. Secondly, the ensembles of classifier chains (ECC) is trained based on obtained visual feature for semantic learning. The Deep-CC corresponds to generative model and discriminative model, respectively, which are trained individually. Deep-CC not only can learn better visual features, but also integrates correlations between labels, when it classifies images. The experimental results show that this approach performs for image semantic annotation more effectively and accurately.

**Keywords:** semantic learning; image auto-annotation; convolutional neural network

# 1 Introduction

In the past decades, several state-of-the-art approaches have been proposed to solve the problems of automatic image annotation, which can be roughly categorized into two different models. The first one is based on generative model. The auto-annotation is first defined as a traditional supervised classification problem [8] and [7], which mainly depends on similarity between visual features and predefined tags to model the classifier, then a unknown image is annotated relevant tags by computing similarity of visual level. The other is based on discriminative model, which are treat image and text as equivalent data. These methods try to mine the correlation between visual features and labels on an unsupervised basis by estimating the joint distribution of multi-instance features and words of each image [8] and [17]. In brief, these methods

* Corresponding author.
extract various low-level visual features. These approaches greatly reduces the ability
of feature presentation, therefore it makes the semantic gap become more serious
between image and semantic.

Furthermore, the performances of image annotation are highly dependent on the
representation of visual feature and semantic mapping. In view of the fact that deep
convolutional neural networks (CNNs) has been demonstrated a outstanding performa-
nce in computer vision recently. Besides, Mahendran and Vedaldi [4], [5], [10] and [12]
have demonstrated that CNN has a better effect over existing methods of hand-crafted
features in many applications, such as object classification, face recognition, and
image annotation. Inspired these articles, this paper proposes a hybrid architecture
based on CNN for image semantic annotation to improve the performances of image
annotation.

In this paper, our main contributions are the following. Firstly, we use redesigned
CNN model to learn high-level visual features. Secondly, we employ the ensembles of
classifier chains (ECC) to train model on visual features and predefined tags. Finally,
we propose a hybrid framework to learn semantic concepts of images based CNN
(Deep-CC). Deep-CC not only can learn better visual features, but also integrates
correlations between labels when it classifies images. The experimental results show
that our approach performs more effectively and accurately.

2 CNN visual feature extraction

In the past few years, some recent articles [15] and [18] have demonstrated that the
CNN models pre-trained on large datasets with data diversity, e.g., AlexNet [5] which
can be directly transferred to extract CNN visual features for various visual recognition
tasks such as image classification and object detection. CNN is a special form of
neural network that consists of different types of layers, such as convolutional layers,
spatial pooling layers, local response normalization layers and fully connected layers.
Different network structures will show different ability of visual features
representation. Krizhev et al. [5] have proved that the Rectified Linear Units (ReLUs)
not only saves the computing time, but also implements the features of sparse
representation, and ReLU also increases the sample characteristic diversity. So in
order to improve the generalization ability of the feature representation, we extract fc7
visual vectors after ReLU. As shown in the top of the Fig. 1, our CNN model has the
similar network structure to the AlexNet. As reflected in Fig. 1, which contains five
convolutional layers (short as conv) and three fully-connected layers (short as fc). The
CNN model is pre-trained in 1.2 million images of 1000 categories from ImageNet
[15].

2.1 Extracting Visual Features From pre-trained CNN Model

G Li et al. [6] and Razavian et al. [13] have demonstrated the outstanding performance
of the off-the-shelf CNN visual features in various recognition tasks, so we utilize the
pre-trained CNN model to extract visual features. Particularly, each image is resized to
227 * 227 and fed into the CNN model. As shown in Fig. 1, it represents the feature flow extracted from the convolutional neural network. The fc7 features are extracted from the secondly convolution layer after ReLU. The fc7 denote the 4096 dimensional features of the last two fully-connected layers after the rectified linear units (ReLU) [5].

## 2.2 Exacting visual feature from fine-tuned CNN model

Taking into account the different categories (and the number of categories) between the target dataset and ImageNet, if we directly utilize the pre-trained model on the ImageNet to exact image visual features, it may not be the optimum strategy. To make the model fit the parameters better, we redesign the last hidden layer for visual feature learning task, later re-designed CNN model by fine-tuning parameters with each of images in the target dataset. Considering the rationality of the design of the convolutional neural networks, our CNN model has the similar network structure to the AlexNet. As show in the mid of Fig. 1, the overall architecture of our CNN model still contains five conv layers, followed by a pooling layer and three fully-connected layers. We redesign the last hidden layer for feature learning task. The number of neural units of the last fully-connected layer is modified from 1000 to m, where m is the number of the target dataset's categories. The output of the last fully-connected layer is then fed into a m-way softmax which produces a probability distribution over m categories.

Given one training sample x, the network extracts layer-wise representations from the first conv layer to the output of the last fully connected layer fc8 ∈ ℝ^m, which can be viewed as high level features of the input image. Followed by a softmax layer, fc8 is transformed into a probability distribution p∈ℝ^m for objects of m categories, and cross entropy is used to measure the prediction loss of the network. Specifically, we define the following formula.

$$p_i = \frac{\exp(v_i^\hat{)}}{\sum_i \exp(v_i^\hat{)}} \quad \text{and} \quad L = t_i \log(p_i) \qquad (1)$$

In formula (1), L is the loss of cross entropy. The gradients of the deep convolutional neural network is calculated via back propagation

$$\frac{\partial L}{\partial \hat{h}_i} = p_i - t_i \qquad (2)$$

In formula (2), t = {t_i|t_i∈{0, 1}, i = 1, ..., m, Σ_{k=1}^m t_i = 1} denotes the true label of the sample x_j, where the {x_j|j=1,2,..., n} is a bag of instances. t = {t_i|t_i∈{0, 1}, i = 1, ..., m} is the label of the bag; Convolutional neural network extracts representations of the bag,
it can get a feature vector $v = \{v_{ij}\} \in \mathbb{R}^{m \times n}$, in which each column is the representation of an instance. The aggregated representation of the bag for visual vectors are defined as follows.

$$ \hat{v}_i = f(v_{i1}, v_{i2}, \dots, v_{in}) \qquad (3) $$

In the training phase, similarly back propagation algorithm is used to optimize the loss function $L$. Suppose that we have a set of training images $I = \{M_i\}$. The trained instances of traditional supervised learning in which training instances are given as pairs $\{(m_i, l_i)\}$, where $m_i \in \mathbb{R}^m$ is a feature vector and $l_i \in \{0,1\}$ is the corresponding label. In visual feature learning, trained sample is regarded as bags $\{I_i\}$, and there are a number of instances $x_{ij}$ in each bag. Finally, the network extracts layer-wise representations from the first conv layer to the output of the last fully connected layer visual vectors $v_i$, which can be viewed as high level features of the input image. By fine-tuning like this, the parameters can better adapt to the target dataset by rectifying the transferred parameters. For the task of visual feature learning, we first employ existing model to fine-tune the parameters in the target dataset, then we apply the fine-tuned CNN model to learn image visual features. Similarly, the FT-fc7 denotes the 4096 dimensional features of the last two fully-connected layers after the rectified linear units (ReLU).

## 3 Ensembles of classification classifiers for semantic learning

In the discriminative learning phase, the ensemble of classification classifiers (ECC) [14] are used to accomplish the task of multi label classification, and each of the binary classifier is implemented by SVM. Taking into account the semantic correlations between tags, ECC can classify images into multiple semantic classes, with a high degree of confidence and acceptable computational complexity. Furthermore, by learning the semantic relevance between labels, classifier chain can effectively overcome the problems of label independence in image binary classification.

The classifier chain model consists of $|L|$ binary classifiers, where $L$ denotes the truth label set. Classifiers are linked along a chain where each classifier deals with the binary relevance problem associated with label $l_i \in L$. The feature space of each linked in the chain is extended with the $\{0,1\}$ label associations of all previous links. The training procedure is outlined in Algorithm 1 in the left of Table 1. Lastly, we can note the notation for a training example $(x, S)$, where $S \subseteq L$ and $x$ is an instance feature vector.

Hence a chain $C_1, C_2, \dots, C_i$ of binary classifier is formed. Each classifier $C_j$ in the chain is responsible for learning and predicting the binary association of label $l_j$, which is given in the feature space and is augmented by all prior binary relevance predictions in the chain $\{l_1, l_2, \dots, l_{j-1}\}$. The classification procedure begins at $C_1$ and propagates along the chain $C_1$ determines $Pr(l_1|x)$ and every following classifier $C_2, \dots, C_j$ predicts
Pr($l_j$|$x_i, l_1, l_2, ..., l_{j-1}$). This classification procedure is described in Algorithm 2 in the right of Table 1.

<table><caption>**Tabel 1.** Training and prediction procedures of ensembles of classifier chains for multi-label learning</caption><thead><tr><th>Processing</th><th>Algorithm 1.<br>Training steps of classifier chain</th><th>Algorithm 2.<br>Classifying procedure ECC</th></tr></thead><tbody><tr><td>Input</td><td>Training set I = {(x<sub>1</sub>,S<sub>1</sub>),(x<sub>2</sub>,S<sub>2</sub>),...,(x<sub>n</sub>,S<sub>n</sub>)}</td><td>Test example x.</td></tr><tr><td>Output<br>procedures</td><td>Classifier chains {C<sub>1</sub>,C<sub>2</sub>,...C<sub>|L|</sub>}</td><td>Y = {l<sub>1</sub>,l<sub>2</sub>,...l<sub>|L|</sub>},</td></tr><tr><td>1</td><td>For i∈1,2,...,|L|</td><td>Y ← {}</td></tr><tr><td>2</td><td>Semantic learning</td><td>For i∈1,2,...,|L|</td></tr><tr><td>3</td><td>I' ← {}</td><td>Do Y ← Y ∪ (l'<i ← C<sub>i</sub>:(x<sub>i</sub>,l<sub>1</sub>, l<sub>2</sub>,...,l<sub>i-1</sub>))</td></tr><tr><td>4</td><td>For (x,S)∈I</td><td>Return (x,Y)</td></tr><tr><td>5</td><td>Do I' ← I' ∪ ((x,l<sub>1</sub>,l<sub>2</sub>,...,l<sub>i-1</sub>),l<sub>i</sub>)</td><td></td></tr><tr><td>6</td><td>Train C<sub>i</sub> to predict binary relevance of l<sub>i</sub></td><td></td></tr><tr><td>7</td><td>C<sub>i</sub> : I' → l<sub>i</sub> ∈ {0,1}</td><td></td></tr></tbody></table>

This training method passes label information between classifiers, with classifier chain taken into account label correlations, so it overcomes the label independence problem of binary relevance method. However, classifier chain still remains advantages of binary relevance method including low memory and runtime complexity. Although $|L|/2$ features are added to each instance on an average, this item is negligible in computational complexity because $|L|$ is invariably limited in practice.

Different order of the chain clearly has a different effect on accuracy. This problem can be solved by using an ensemble framework with a different random train ordering for each iteration. Ensembles of classifier chains train m classifier chains $C_1, C_2, ..., C_m$. Each $C_k$ model is trained with a random chain which can order the L outputs and get a random subset of D. Hence each $C_k$ model is likely to be unique and able to give different multi-label predictions. These predictions are summed by label so that each label receives a number of votes. A threshold is used to select the most popular labels, which form the final predicted multi-label set. These predictions are summed by label so that each label receives a number of votes. A threshold is used to select the most popular labels which form the final predicted multi-label set.

Each kth individual model predicts vector $y_k = (l_1, l_2, ..., l_{|L|}) \in \{0,1\}^{|L|}$. The sums are stored in a vector $W = (\lambda_1, \lambda_2, ..., \lambda_{|L|}) \in \mathbb{R}^{|L|}$, where $\lambda_j$ is defined as $\lambda_j = \sum_{k=1}^{m} l_{jk} y_k$. Hence each $\lambda_j \in W$ represents the sum of the votes for label $l_j \in L$. We then normalize $W$ to $W_{norm}$, which represents a distribution of scores for each label in [0,1]. A threshold is used to choose the final multi-label set $Y$ such that $l_j \in Y$ where $\lambda_j \ge t$ for threshold $t$. Hence the relevant labels in $Y$ represent the final multi-label prediction.

# 4 Hybrid framework for image annotation

On the deep model and ensembles of classifier chains, we propose a hybrid learning framework to address cross-modal semantic annotation problem between images and text with Multi-label. Fig. 2 shows two setups of the hybrid architecture approach for
semantic learning based on deep learning. The first path (generative learning) feeds
training image to the fine-tune pre-trained CNN step which is also called the feature
learning phase, then in the discriminative learning phase, we utilize ensemble of
classifier chains to model the visual vectors which are co-occurrence matrix consisting
of texture and exacted visual features by pre-trained CNN model. This hybrid pipeline
model is called Deep-CC image annotation system.

<figure><img src="image_2.png" /><figcaption>Fig. 1. The Pipeline of Image Annotation</figcaption></figure>

Bases on the learning feature, the trained CNN model output visual features after ReLU. Suppose that we have a set of images M = {m₁,m₂,...,mᵢ}, this model extracts visual vectors by pre-trained CNN model and we denote the space of visual vectors as V={v₁,v₂,...,vᵢ}, where vᵢ denotes the visual vector of ith image. Noting the notation for a training example (vᵢ,S), where S∈L, L denotes the label set and v is a feature vector. Then, by making use of the aspect distribution and original labels of each training image, we build a series of classifiers in which every word in the vocabulary is treated as an independent class. The classifier chain model implements the feature classification task and it can effectively learn the semantic correlation between labels in discriminative step. Finally, given a test image, the Deep-CC system will return a correlative label subset l∈L.

As a comparison, we evaluate the deep feature's performance from the AlexNet
CNN on those same benchmarks. Following by [11], we choose 5 words with highest
confidence as annotations of the test image. After each image in the database is
annotated, the retrieval algorithm ranks the images labeled with the query word by
decreasing confidence.

5 Experiments and results
In this section, we conduct experiments of our Deep-CC learning framework on both image classification and image auto-annotation. We choose a dataset Corel5K which is widely used in image classification and annotation. In order to make the experimental result more convinced, we simultaneously compare the experimental results with the existing traditional model and deep model.

## 5.1 Datasets and Evaluation measures

In order to test the effectiveness and accuracy of the proposed approach, we conduct our experiments on a baseline annotated image datasets Corel5K [3]. Corel5k is a basic comparative dataset for recent research works on image annotation. The dataset contains 5000 images from 50 Corel Stock Photo cds. We divided this dataset into 3 parts: a training set of 4000 images, a validation set of 500 images and a test set of 500 images. Like the Duygulu et al. [3], We divide separately the training set of 4500 images and the test set of 500 images.

Image annotation performance is evaluated by comparing the captions automatically generated for the test set with the human-produced ground truth. It is essential to include several evaluation measures in multi-label evaluation. Similar to Monay et al. [11], we use mAP as evaluation measures. Naturally, we define the automatic annotation as the top 5 semantic words of largest posterior probability, and compute the recall and precision of every word in the test set.

<table><caption>**Table 2.** Performance (mAP in %) comparison in terms of different methods and visual features algorithms on Corel5k. (The "-" means to use their method)</caption><thead><tr><th rowspan="2">Method</th><th rowspan="2">Visual features</th><th colspan="2">Result on all words</th><th rowspan="2">mAP</th></tr><tr><th>Precision</th><th>Recall</th></tr></thead><tbody><tr><td rowspan="3">PLSA-WORDS</td><td>-</td><td>22.1</td><td>12.1</td><td>19.1</td></tr><tr><td>Fc7</td><td>27.5</td><td>21.7</td><td>26.9</td></tr><tr><td>FT-fc7</td><td>29.3</td><td>22.6</td><td>27.3</td></tr><tr><td rowspan="3">HGDM</td><td>-</td><td>32.1</td><td>29.3</td><td>26.3</td></tr><tr><td>Fc7</td><td>36.4</td><td>30.5</td><td>29.7</td></tr><tr><td>FT-fc7</td><td>37.6</td><td>32.9</td><td>30.9</td></tr><tr><td>DNN</td><td>-</td><td>37.5</td><td>40.5</td><td>32.7</td></tr><tr><td>Deep-CC(our)</td><td>-</td><td>39.7</td><td>37.6</td><td>35.2</td></tr></tbody></table>

## 5.2 Results for image annotation on Corel5K

In this section, we demonstrate the performance of our model on the corel5k data set for image multi-label annotation, and compare the results with some existing image annotation methods, e.g. PLSA-WORDS [11], HGMD [9] and DNN [16]. We evaluate the returned keywords in a class-wise manner. The performance of image annotation is evaluated by comparing the captions automatically generated with the original manual annotations. Similar to Monay F et al. [11], we compute the recall and precision of
every word in the test set and use the mean of these values to summarize the system performance.

<table><caption>Fig. 2. Comparison of annotations made by HGDM and Deep-CC on Corel5k</caption><thead><tr><th>Image</th><th></th><th></th><th></th><th></th><th></th></tr></thead><tbody><tr><td>Ground Truth</td><td>temple, sky, buddhist, mountains</td><td>elephant, trees, planes, sky</td><td>cabin, trees, autumn, field</td><td>trees, sky, road, park</td><td>people, water, trees, sand</td></tr><tr><td>HGDM annotations</td><td>house, sky, water,cloud, mountains</td><td>Africa, sky, land,animal, beach</td><td>field, grass, land, trees, mountains</td><td>trees, road, sky, pant, sea</td><td>sand, water, tress,people, cloud</td></tr><tr><td>Deep-CC annotations</td><td>temple, sky, palace, land, mountains</td><td>elephant, land, plane, trees, sky</td><td>chair, trees, field, land, mountains</td><td>park, road, trees, sky, mountains</td><td>sand, coast, trees, water, people</td></tr></tbody></table>

Table 2 reports results of several models on the set of all 260 words which occur in the training set. Data in precision and recall columns denotes mean precision and mean recall of each word. The off-the-shelf CNN features (i.e. fc7 and FT-fc7) obtain significant improvements (7.8% based on PLSA-WORDS, 3.4% based on HGDM) compared with these traditional feature learning methods. After fine-tuning, a further improvement (8.2% based on PLSA-WORDS, 4.6% based on HGDM) can be achieved with the best performance of the CNN visual features FT-fc7.

Annotations of several images obtained by our Deep-CC annotation system are shown in Fig. 2. We can see that annotations generated by Deep-CC are more accurate than HGDM in most cases. In order to be more intuitive to observe different precision-recall in various methods, the Fig. 3 presents the precision-recall curves of several annotation models on the Corel5k data set. As is shown in Fig. 3, Deep-CC performs consistently better than other models. Where the precision and recall values are the mean values calculated over all words.

<figure><img src="image_3.png" /><figcaption>Fig. 3. Precision-recall curves of several models for image annotation on Corel5K</figcaption></figure>
## 5.3 Result analysis

In summary, the experimental results on Corel5k show that Deep-CC outperforms many state-of-the-art approaches, which proves that the redesigned CNN and the hybrid framework is effective in learning visual features and semantic concepts of images. We compare the CNN visual features with traditional visual features for learning semantic concepts of images over two traditional learning approaches and a deep model. Especially, the comparison in terms of rigid and articulated visual features among Corel5k is shown in Table 2, from which it can be seen that CNN feature outperforms almost all the original hand-crafted features. To verify this assumption, different visual features between traditional models (also from the authors of this paper) and CNN mode, and FT-fc7 is executed to make an enhanced prediction for Corel5k. Incredibly, the mAP score on Corel5k can surge to 35.2% as shown in Table 2, which demonstrates the great dominance in the deep networks. To sum up, based on the above reported experimental results, we can see that CNN visual features are very effective for semantic image annotation.

# 6 Conclusion

In this paper, we utilize CNN model to learn deep visual features, and we redesign the last hidden layer for feature learning task, and in order to obtain high performance of feature representation, we first train our deep model on ImageNet, then the pre-trained parameters are fine-tuned on target dataset. We showed under what conditions each visual feature can perform better, and propose a hybrid architecture. We demonstrated that re-designed CNN model and ensembles of classifier chains can effectively improve annotation accuracy.

In comparison to many state-of-the-art approaches, experimental results show that our method achieves superior results in the tasks of image classification and annotation on Corel5K. However, in the process of learning visual features, Deep-CC only employ single convolution neural network not fully understanding multiple instance in the image, and how to excavate the high-level semantic relevance between the tags, it can be deeply studied. In future research, we aim to take semi-supervised learning based on a large number of unlabeled data to improve its effectiveness.

# 7 Acknowledgement

This work is supported by the National Natural Science Foundation of China (Nos. 61663004, 61262005, 61363035), the Guangxi Natural Science Foundation (2013GXNSFAA019345, 2014GXNSFAA118368), the Guangxi "Bagui Scholar" Teams for Innovation and Research Project, Guangxi Collaborative Innovation Center of Multi-source Information Integration and Intelligent Processing.

# 8 References
1. Cusano C, Ciocca G, Schettini R. Image annotation using SVM[J]. Proceedings of SPIE - The International Society for Optical Engineering, 2003, 5304:330-338.

2. Deng J, Dong W, Socher R, et al. ImageNet: A large-scale hierarchical image database[C]// 2009:248-255.

3. Duygulu P, Barnard K, Freitas J F G D, et al. Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary[C]// European Conference on Computer Vision. Springer-Verlag, 2002:97-112.

4. He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition[J]. Computer Science, 2015.

5. Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks[J]. Advances in Neural Information Processing Systems, 2012, 25(2):2012.

6. Li G, Yu Y. Visual saliency based on multiscale deep features[C]// Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on. IEEE, 2015.

7. Li J, Wang J Z. Automatic Linguistic Indexing of Pictures by a statistical modeling approach[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2003, 25(9):1075-1088.

8. Liu Y, Zhang D, Lu G, et al. A survey of content-based image retrieval with high-level semantics[J]. Pattern Recognition, 2007, 40(1):262-282.

9. Li Z, Shi Z, Zhao W, et al. Learning semantic concepts from image database with hybrid generative/discriminative approach[J]. Engineering Applications of Artificial Intelligence, 2013, 26(9):2143-2152.

10. Mahendran A, Vedaldi A. Understanding deep image representations by inverting them[J]. Computer Science, 2014.

11. Monay F, Gatica-Perez D. Modeling Semantic Aspects for Cross-Media Image Indexing[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2007, 29(10):1802-17.

12. Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning.[J]. Nature, 2015, 518(7540):529-33.

13. Razavian, Ali Sharif, Azizpour, Hossein, Sullivan, Josephine, et al. CNN Features Off-the-Shelf: An Astounding Baseline for Recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition Workshops. IEEE Computer Society, 2014:512-519.

14. Read J, Pfahringer B, Holmes G, et al. Classifier chains for multi-label classification[J]. Machine Learning, 2011, 85(3):254-269.

15. Russakovsky O, Deng J, Su H, et al. ImageNet Large Scale Visual Recognition Challenge[J]. International Journal of Computer Vision, 2015, 115(3):211-252.

16. Sermanet P, Eigen D, Zhang X, et al. OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks[J]. Eprint Arxiv, 2013.

17. Smeulders A W M, Worring M, Santini S, et al. Content-based image retrieval at the end of the early[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2000, 22(12):1349-1380.

18. Wu J, Yu Y, Chang H, et al. Deep multiple instance learning for image classification and auto-annotation[C]// Computer Vision and Pattern Recognition. IEEE, 2015:3460-3469.