Learning Product Codebooks Using VectorQuantized Autoencoders for Image Retrieval VectorQuantized Variational Autoencoders (VQVAE)1 provide an unsupervised model for learning discrete representations by combining vector quantization and autoencoders. In this paper, we study the use of VQVAE for representation learning of downstream tasks, such as image retrieval. First, we describe the VQVAE in the context of an informationtheoretic framework. Then, we show that the regularization effect on the learned representation is determined by the size of the embedded codebook before the training. As a result, we introduce a hyperparameter to balance the strength of the vector quantizer and the reconstruction error. By tuning the hyperparameter, the embedded bottleneck quantizer is used as a regularizer that forces the output of the encoder to share a constrained coding space. With that, the learned latent features better preserve the similarity relations of the data space. Finally, we incorporate the product quantizer into the bottleneck stage of VQVAE and use it as an endtoend unsupervised learning model for image retrieval tasks. The product quantizer has the advantage of generating large and structured codebooks. Fast retrieval can be achieved by using lookup tables that store the distance between any pair of subcodewords. Stateoftheart retrieval results are achieved by the proposed codebooks.

EndtoEnd Supervised Product Quantization for Image Search and Retrieval. Product Quantization, a dictionary based hashing method, is one of the leading unsupervised hashing techniques. While it ignores the labels, it harnesses the features to construct look up tables that can approximate the feature space. In recent years, several works have achieved state of the art results on hashing benchmarks by learning binary representations in a supervised manner. This work presents Deep Product Quantization (DPQ), a technique that leads to more accurate retrieval and classification than the latest state of the art methods, while having similar computational complexity and memory footprint as the Product Quantization method. To our knowledge, this is the first work to introduce a dictionarybased representation that is inspired by Product Quantization and which is learned endtoend, and thus benefits from the supervised signal. DPQ explicitly learns soft and hard representations to enable an efficient and accurate asymmetric search, by using a straightthrough estimator. Our method obtains state of the art results on an extensive array of retrieval and classification experiments.