# Bilinear CNN Model for Fine-Grained Classification Based on Subcategory-Similarity Measurement
## 2 Related Work  
The key of image classification is to extract robust features of objects and form better feature representations. Previous work on fine-grained classification usually focused on part detection to establish correspondence between object instances and reduce the impact of object-pose variations under a strongly supervised setting. In order to apply fine-grained classification methods to practical applications, many researchers turn to studying how to accurately locate discriminative regions under weakly supervised conditions, and then use CNN to extract features from these regions. [PERSON] et al. designed the first two-level attention model [16] of a weakly supervised classification algorithm, where object-level attention was adopted to select a relevant bounding box of a certain object, while part-level attention was used to locate discriminative components of the object, which achieved 69.7% accuracy on the CUB-200-2011 dataset [17]. [PERSON] et al. proposed the recurrent attention convolutional neural network (RA-CNN) [18], which recursively learns discriminative-region attention and feature representation of this region on multiple scales in a mutually reinforcing way, but this method adds computational overhead. These approaches take CNN as a part detector but ignore the global information of the object. In order to avoid the direct detection of discriminative regions, [PERSON] et al. proposed the end-to-end B-CNN, using two feature extractors based on CNNs [19] to extract the global features of object, which collects second-order statistics of local features over a whole image to form a holistic descriptor for classification after pooling across locations. But the computational complexity of  
Figure 1: Sample examples on Stanford Cars-196.  
B-CNN is too large. Based on B-CNN, [PERSON] et al. adopted low-rank approximation to the covariance matrix to avoid direct calculation of the outer product, and further reduced the feature dimension [20]. However, the original B-CNN work of [PERSON] et al. and the low-rank models of [PERSON] et al. directly use the original image as input, while the original image has a lot of background noises, especially when the target is very small. Inspired by the work of SCDA [21], we propose to add weakly supervised localization into a traditional B-CNN, which can eliminate the influence of background noise and extract features more accurately.  
Since it is a common phenomenon for there to be differences between different categories in image classification, several similarity constraints have been proposed for feature representation learning. [PERSON] et al. introduced the Siamese network [22], which defines similar and dissimilar image pairs and requires that the distance between dissimilar pairs be larger than a certain margin, while the distance between similar pairs should be smaller. [PERSON] et al. proposed to improve the performance of traditional CNNs by combining softmax and contrastive loss through joint optimization [23], because contrastive constraints might augment information for training the network. Previous image-similarity learning mainly focused on the similarity of two images, while this paper studies the similarity between two subcategories. The main difference between newly proposed bilinear methods [24; 25; 26] and our method is that we add subcategory-similarity measurement and obtain the fuzzing similarity matrix to measure the fuzzy degrees among different subcategories in the training set. Driven by the fuzzing similarity matrix, triplet loss-based deep metric learning adaptively sets the sample ratios to form the triplets. Finally, triplet loss and weighted softmax loss are combined to restrict interclass distance and increase intraclass distance. The overall architecture of the deep-learning model is shown in Figure 2.