# A Study on the Channel Expansion VAE for Content-Based Image Retrieval
## 2 Related Work  
Since the CNN features of general visual recognition work effectively, broad vision tasks have been studied by neural networks [6]. Following this trend, image retrieval was also studied by CNN layer output as a feature descriptor. The paper \"Neural Codes for Image Retrieval\" [7] was the first to apply a pre-trained classification model for CBIR. The output of the network consists of types of codes, and these codes are a descriptor of the images. Afterward, off-the-shelf feature-based studies proceeded with image retrieval.  
In 2015, sum pooling of convolution (SPoC) [8] was proposed as a spatial-wise weighting method; prior to that, the region of interest was generally placed in the center. Thus, a Gaussian filter was used as a weighting function and the descriptor as a sum-pool of the convolution layer. Cross-dimensional weighting (CroW) [9] considered not only spatial-wise but also channel-wise weighting in 2016. Spatial weight was based on the activation map, and channel weight was based on the sparsity given by the non-zero response of the k-channel. The sparsity was inversely proportional to the max response. In the following year, selective convolutional descriptor aggregation (SCDA) was used [10], which mainly deals with the selective region. Each channel of the layer can obtain a gradient-based response derived from the model's output, which indicates a class. The method is to sum the channel-wise figure and generate a regional mask. Finally, the authors concatenated the global max-pooling and average-pooling as a descriptor. Part-based weighting aggregation (PWA) [11] introduces the concept of \"Probabilistic Proposal\" by selecting N kernels that have a larger variance in the off-line stage. Then, the probabilistic proposal spotlights the discriminative region in the image. Generalized-mean pooling (GeM) [12] generalizes pooling with the coefficient that could be both max-pooling and average pooling. The region activation varies in size depending on the coefficient and includes the means of extraction and representation. Unlike this, the most representative descriptor RMAC [13] uses classification models and extracts an appropriate descriptor, which aggregates regional maximum activation values. In a recent work, [PERSON] [14] proposed a framework to combine multiple global descriptors that utilize different features' properties. A study by [PERSON] [15] dealt with local descriptors that effectively outperformed global descriptors on a large scale.  
The abovementioned papers are based on fine-tuning and feature aggregation, which means a supervised methodology to shift domains. Like other flows, generative models, such as the generative adversarial network [16] and VAE [17], are used in unsupervised methods. In 2019, unsupervised adversarial image retrieval UAIR [18] was proposed to  
Figure 1: General content-based image retrieval framework.  
learn data distribution via an adversarial feedback structure for CBIR. UAIR takes extracted features from various types of representation, such as hand-crafted or learning-based features. Then, the framework generates a rank of similarity and discrimination. On the other hand, some papers applied VAE for CBIR. [PERSON] [19] designed an image retrieval model with VAE and analyzed it with the MNIST dataset [20], which had 16% mean average precision (mAP). [PERSON] [21] extracted latent codes through VAE and used them for image retrieval, recommendations, and the generation of a fashion product dataset. [PERSON] [22] studied an auto-encoder implementation on MNIST in detail. Moreover, [PERSON] [23] dealt with the search for binary diagram images with a VAE-based model. We proposed a new channel expansion with a bottleneck effect and regional masking in convolutional networks to improve VAE for CBIR.