Revisiting Multi-Codebook Quantization

Multi-Codebook Quantization ( MCQ ) is a generalized version of existing codebook-1 based quantizations for Approximate Nearest Neighbor ( ANN ) search . Therefore,2 MCQ theoretically has the potential to achieve the best performance because so-3 lutions of other codebook-based quantization methods are all covered by MCQ ’ s4 solution space under the same codebook size setting . However , finding the opti-5 mal solution to MCQ is proved to be NP-hard due to its encoding process , i.e.,6 converting an input vector to a binary code . To tackle this , researchers apply7 constraints to it to find near-optimal solutions , or employ heuristic algorithms8 which are still time-consuming for encoding . Different from previous approaches,9 this paper takes the first attempt to find a deep solution to MCQ . The encoding10 network is designed to be as simple as possible , so the very complex encoding11 problem becomes simply a feed-forward . Compared with other methods on three12 datasets , our method shows state-of-the-art performance . Notably , our method13 is 11×-38× faster than heuristic algorithms for encoding , which makes it more14 practical for real scenery of large-scale retrieval . Our code is publicly available:15 https : //github.com/DeepMCQ/DeepQ.16 1 Introduction17 Rapidly increasing multimedia contents in recent years raise an urgent request for retrieval in a18 short time . Unlike the exhaustive routine [ 31 , 20 ] , Approximate Nearest Neighbor ( ANN ) search19 significantly reduces retrieval time while preserving high recall . It has been widely applied to various20 scenarios , such as database indexing , fast image retrieval , and recommender systems.21 As a typical approach , vector quantization ( VQ ) [ 7 ] is at first developed as a compression technique,22 which uses a codebook to approximate vectors . People further find the power of VQ to preserve23 similarities between quantized features and enable VQ to perform ANN search . In order to achieve24 low quantization errors with limited codebook size , a multi-codebook structure is introduced . The25 proposal of the Multi-Codebook Quantization ( MCQ ) [ 2 ] describes the approach as a combination26 of one codeword for each sub-codebook , and previous methods [ 9 , 6 , 19 , 30 , 10 , 3 ] are summarized27 as exceptional cases of MCQ or constrained MCQs . The quantization codes are designed to be28 compacted , which results in negligible storage cost and high-quality results.29 However , the optimization of MCQ without any constraints is formally NP-hard . [ 14 ] models30 it as the minimization on several fully-connected Markov Random Fields ( MRFs ) . As a result,31 current researches aim at solving MCQ under acceptable computational costs . Other than applying32 constraints on it [ 34 , 4 , 15 ] , another approach designs algorithms in a heuristic way [ 2 , 14 , 16 ] . The33 latter achieves better performance but suffers from slow encoding.34 There are chances to employ neural networks ’ power to solve MCQ , where people expect to obtain35 higher performance and encoding efficiency than previous methods . [ 11 , 5 , 28 , 33 , 27 ] already give36 the way to treat codebook as network parameter and update it by gradient-descent , but they are37 Submitted to 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Do not distribute . all still under constraints that hinder performance . Morozov and Babenko [ 18 ] and Sablayrolles et38 al . [ 22 ] map datapoints to learned space , which are not flexible , especially when performing the39 reconstruction . Therefore in this paper , we give our first attempt to solve MCQ in a deep learning40 approach , without constraints and work-arounds . Our contributions can be summarized as three-folds:41 • Our novel approach , Deep Multi-Codebook Quantization ( DeepQ ) , fully considers encoding42 difficulty and time complexity in MCQ . With the high efficient and parallelized encoding networks,43 our method significantly reduces encoding time.44 • To tackle the NP-hard encoding problem and non-differentiable gradient estimation , we employ and45 further revise a policy gradient method . Value-Corrected Proximal Policy Optimization ( VC-PPO ) 46 is proposed to speed up convergence in the training phase.47 • Experiments conducted on a benchmark dataset validate our proposed method . Furthermore , to48 evaluate the scalability of the method , it is tested on million-scale datasets to show the effectiveness49 of our proposed algorithm.50 2 Related Works51 Vector quantization is a routine to approximate vectors by a codebook . Typical applications include52 clustering , compression , and Approximate Nearest Neighbor ( ANN ) search . The famous proposal53 k-means [ 7 ] , also known as Lloyd ’ s algorithm [ 13 ] , clusters the dataset into uniformly sized convex54 cells . When it is applied to ANN search , datapoints from the base set are quantized into their55 nearest centriods and represented by indices . The distance from a given query to any datapoint56 is approximated by the distance from the query to the datapoint ’ s centriod , which is effectively57 pre-computed and stored in a lookup table . To perform fine-grained clustering as well as reducing the58 space and time complexity , they [ 9 , 6 , 19 , 10 , 30 ] divide the feature space orthogonally by performing59 k-means in each subspace concurrently . Meanwhile , the introduced sub-codebook structure reveals60 the prototype of MCQ . Formally , [ 2 ] gives a well definition of MCQ , and previous works are all61 summarized into constrained MCQs . Specifically , subspace k-means must keep orthogonality among62 sub-codebooks . Zhang et al . [ 34 ] loosens the orthogonality constraint , but sub-codebooks are still63 weakly-orthogonal . Chen et al . [ 4 ] and Martinez et al . [ 15 ] propose hierarchical k-means , where64 vectors are quantized coarse-to-fine . If constraints are moved , MCQ is not easy to solve . Current65 state-of-the-art methods develop heuristic algorithms to help to encode . Specifically , Babenko and66 Lempitsky [ 2 ] employs beam search , Martinez et al . [ 14 , 16 ] give algorithm based on Iterated67 Conditional Modes ( ICM ) . However , the above methods do not achieve satisfied time complexity in68 encoding yet.69 When neural networks and gradient descent become a fashion , a few attempts to integrate quantization70 into deep retrieval networks are proposed . Klein and Wolf [ 11 ] and Song et al . [ 5 ] propose Deep71 Product Quantization ( DPQ ) and Deep Progressive Quantization ( DPgQ ) which update codebook by72 soft relaxation , but they are still under the same constraints as [ 9 , 15 ] . Sablayrolles et al . [ 22 ] and73 Morozov and Babenko [ 18 ] give pipelines to encode compact representations for compressed-domain74 search , but they do not strictly follow the paradigm of MCQ.75 3 Preliminaries76 Given a vector x ∈ RD , its quantized vector x̃ are composed by several codewords in a codebook77 C. More Specifically , C = ( Cm ) , Cm ∈ RK×D , 1 ≤ m ≤M contains M sub-codebooks and K78 codewords for each . Quantization codes are formed by b = ( bm ) , bm ∈ { 1 , 2 , · · · , K } , 1 ≤ m ≤79 M , which indicates the picked codeword in each sub-codebook . For the whole training setX = { x } 80 with N datapoints , MCQ aims at finding the optimal quantization codesB = { b } and codebook C81 to minimize following objective:82 min C , B E x∈X b∈B Q ( x , b , C ) = min C , B E x∈X b∈B ∥∥∥∥∥x− M∑ m=1 Cmbm ∥∥∥∥∥ 2 ( 1 ) whereCmbm ∈ RD is the bm-th codeword of the m-th sub-codebook . The sum of picked codewords83 ∑ Cmbm tries to approximate x . C and b are stored for further retrieval . Some of the previously84 mentioned methods [ 9 , 6 , 4 , 15 , 34 ] are treated as constrained MCQs , as they are all represented85 as special cases of ( 1 ) . Specifically , when M = 1 , ( 1 ) becomes VQ . Or if any two sub-codebooks86 Ci , Cj are orthogonal , it will be PQ or OPQ.87 The optimization of ( 1 ) without any constraints is proved to be NP-hard [ 14 ] . To tackle this , we88 propose a Expectation-Maximization style solution . Following sections will explain the deep neural89 network for encoding b ( Section 4.1 ) , the way to solve C ( Section 4.2 ) , and how to conduct retrieval90 ( Section 4.3 ) , respectively.91 4 Methodology92 4.1 Expectation : EncodingB with neural networks93 Our first step , is to find a potential code b by given x and94 a fixed C. A policy π parameterized by θ is employed to95 take possible solution of b by feeding x:96 π = ( πm ) = π ( x | θm ) , 1 ≤ m ≤M . ( 2 ) More specifically , π produces M Categorical distribu-97 tions Categorical ( K , pm1 , · · · , pmK ) , where pmj is98 the probability to pick the j-th codeword in the m-th99 sub-codebook . A potential encoding bm is generated by100 drawing samples from πm , which then helps us to pick101 codeword Cmbm . Therefore:102 bm ∼ πm ( x | θm ) = Categorical ( K , pm ) . ( 3 ) Since the independence among different sub-codebooks103 is a prerequisite of MCQ , bm should be drawn from πm104 independently . Intuitively , the probability of b to be a105 specific b ? is derived by conditional independence:106 Pr ( b=b ? ) = M∏ m=1 Pr ( bm=b ? m ) = M∏ m=1 pmb ? m . ( 4 ) We adopt the power of neural networks to model107 πm . Specifically , θm produces K unnormalized log-108 probabilities ` m and pmj is obtained by Softmax . To109 keep the independence , θm will not share parameters with110 each other.111 Therefore , θ , or our proposed IndepNet is illustrated in112 Figure 1 . We first build a basic structure called IndepBlock and duplicate this block for M times as113 θ1 , θ2 , · · · , θM . We try to keep the basic structure really simple to achieve high efficiency during114 training and encoding . As the figure shows , IndepBlock is an hourglass network contains 6 layer-115 groups ( consists of a linear layer with ReLU activation and layer-normalization ) with skip-connections.116 The last three outputs are concatenated and further fed into a final linear layer with K outputs as117 ` m = ( ` m1 , · · · , ` mK ) , and therefore:118 pmj = Softmax ( ` m ) j , where ` m = θm ( x ) . ( 5 ) 4.1.1 Gradient estimation119 The objective of training θ is formed as:120 min π E x∈X b∼π ( x|θ ) Q ( x , b , C ) . ( 6 ) However , the optimization faces two problems : 1 ) The encoding of b involves sampling from discrete121 distributions , which is non-differentiable , 2 ) All possible encoding of b is O ( KM ) . Exhaustive122 search becomes impracticable.123 Therefore , gradient estimation over discrete , stochastic computation graph is required to train θ.124 Mainstream methods [ 23 , 32 , 17 ] include score function gradient estimator , pathwise gradient125 estimator , etc . Meanwhile , minimizing ( 6 ) is also faced with the high-variance problem during126 gradient estimation . To tackle this , the advantage function is introduced [ 12 , 25 ] . Specifically in127 our work , a value network called QENet parameterized by τ is proposed to model a value function128 v = V ( · | τ ) . It performs a regression task to minimize the following objectives:129 min τ E x∈X b∼π ( x|θ ) ‖Q ( x , b , C ) −V ( x , b , C | τ ) ‖2 . ( 7 ) Advantages Â is then estimated by130 Â = Q ( x , b , C ) −V ( x , b , C | τ ) . ( 8 ) The detailed architecture of QENet is shown in Figure 2.131 We reuse the IndepBlock to generate v by M + 1 blocks:132 τ = ( τ1 , · · · , τM , τx ) . Specifically , latent representation133 for each selected-codeword Cmbm is obtained by:134 ιm = τm ( Cmbm ) . ( 9 ) The last IndepBlock τx is introduced to transform x. Then,135 all the outputs from IndepBlocks are summed up to get136 scalar value v ( denoted as “ reduce-sum ” ) :137 v = sum ( ι1 , · · · , ιM , ιx ) . ( 10 ) Value-corrected proximal policy optimization We138 propose a variant of score function gradient estimator139 called Value Corrected Proximal Policy Optimization ( VC-140 PPO ) based on PPO to get simple but efficient Trust Re-141 gion updates [ 26 , 24 ] . In the real scenario of large-scale142 ANN search , the training size N is usually larger than 10k . Conventional PPO still does not satisfy143 us due to the speed of convergence . Therefore , we revise and propose the Value-Corrected PPO144 ( VC-PPO ) to achieve fast training . Firstly in the sampling stage , bo and vo is produced from datapoint145 x over whole training setX by freezing current policy network and value network as θo , τo:146 bo ∼ π ( x | θo ) , vo = V ( x , bo , C | τo ) . ( 11 ) The probability of producing the sampled bo is denoted as po = Pr ( bo | x , θo ) , calculated by147 equation ( 4 ) . Finally , our surrogate objectives of VC-PPO is defined as [ 8 ] :148 Lθ = min ( Pr ( bo | x , θ ) Pr ( bo | x , θo ) Â , clip1+ 1− ( Pr ( bo | x , θ ) Pr ( bo | x , θo ) ) Â ) , ( 12 ) 149 Lτ = max ( ( Q ( x , bo , C ) −V ( x , bo , C | τ ) ) 2 , ( Q ( x , bo , C ) − vo − clip+ − ( V ( x , bo , C | τ ) − vo ) ) 2 ) . ( 13 ) Here , The clip ( · ) forces the policy and value to be not too far from old ones and is the clip-range.150 In both equations , it prevents a large update ratio leading to an unstable policy . The key difference151 between the original PPO and our VC-PPO is , we use V ( x , bo , C | τ ) other than the recorded old152 value vo from sampling stage to estimate advantage . This modification is treated as a value-correction153 process . Correcting value leads to a precise estimation on advantage , which is based on two reasons:154 a ) Biases are introduced into advantage estimation if we use vo , since the policy is getting better and155 better during training but τo is froze , and 2 ) The calculation of V ( x , bo , C | τ ) can be done instantly156 without introducing significant computational overhead . To further encourage the network choose157 codewords uniformly , a regularization is applied to θ to maximize the entropy of π:158 eθ = − M∑ m=1 K∑ j=1 pmj log pmj ( 14 ) which forces network to try more codeword combinations.159 4.2 Maximization : Solve C by least-squares160 To give the closed-form derivation of solving C by givenX andB , We will firstly rewrite Equation161 ( 1 ) to a matrix formulation . Since b = ( b1 , b2 , · · · , bM ) and bm ∈ { 1 , 2 , · · ·K } is the index of162 selected codeword in the i-th sub-codebook , a one-hot encoding and a concatenation on each bm:163 b′m = one-hot ( bm ) , b ′ = ( b′1 , · · · , b ′ m ) will convert the quantization code to a M -hot vector i.e . a164 vector that contains M segments , and each segment contains exactly one 1 and remaining 0 , where165 1 is the entry of picked codeword . Correspondingly , a reshape is applied to C : C ′ = C1 C2 ... CM ∈166 R ( M×K ) ×D . ( 1 ) will become:167 min C′ ∥∥X −B′C ′∥∥2 2 . ( 15 ) This equation is formally a linear least-squares regression , where B′ ∈ { 0 , 1 } N× ( M×K ) is known168 and X is target . Although there is a bunch of algorithms to solve it , we finally choose gelsy [ 1 ] ,169 which in our experiments shows the best results . The solution is to first apply a QR factorization with170 column permutation onB′ :171 B′ = Q ( R11 R12 0 R22 ) P ᵀ ( 16 ) where Q and R = ( R11 R12 0 R22 ) is the factorization matrix and P is an orthogonal matrix that172 permutes columns ofB′ untilR11 is well-conditioned ( its estimated condition number approaches173 0 ) . With the permutation , R22 becomes negligible . Moreover , R12 is erased by another orthogonal174 transformation:175 ( R11 R12 0 R22 ) → ( R11 R12 0 0 ) = ( T 11 0 0 0 ) Z ( 17 ) where T and Z are from the orthogonal transformation ofR . Then , C ′ is derived by:176 B′ = Q ( T 11 0 0 0 ) ZP ᵀ , C ← C ′ ← PZᵀ ( T−111 Q ᵀ 1X 0 ) ( 18 ) whereQ1 is the top rank ( B ′ ) columns ofQ.177 In brief , our overall training approach is summarized into algorithm 1.178 4.3 Fast retrieval179 After training , we are able to encode the base set for retrieval . Other than sampling from π , codewords180 are simply rolled out by greedy assignments:181 bgm = arg max θm ( x ) . ( 19 ) We firstly use the greedy roll-out strategy to obtain B in the training set in order to solve the final182 codebook . Then , we employ the same strategy to encode the base set.183 To further refine assignments , we add an extra step that randomly selects and alters bi while fixing184 others:185 bgi ← arg min bgi Q ( x , bg , C ) , i ∼ U [ 1 , M ] . ( 20 ) Since this refinement only causes negligible overhead referred to the implementation by [ 14 ] , in186 practice , we benefit from it not only to get lower quantization error but also to obtain acceptable187 performance from a fast training , i.e. , training within a very few steps before the network is converged.188 The encoded and refined base set , combined with the codebook , is finally employed for retrieval . The189 LSQ-style lookup table [ 14 ] is utilized to speed up similarity search.190 4.4 Discussion191 Algorithm 1 : VC-PPO for Training Inputs : Training setX , max step T , hyperparameters α , , learning rates η1 , η2 . Outputs : Policy π. Initialize codebook C , parameters θ and τ ; i← 0 ; while i < T do / * Training loop * / for x inX do / * Sampling stage * / Sample bo ∼ π ( x | θo ) intoB ; Compute vo , po into V , Pr ; end for x , bo , vo , po inX , B , V , Pr do / * Updating stage * / τ ← τ − η1∇τLτ ; Compute Â by ( 8 ) ; θ ← θ + η2∇θ ( Lθ + α · eθ ) ; end C ← Solved by ( 15 ) ∼ ( 18 ) ; i← i+ 1 ; end return π ( · | θ ) Our work aims at solving Multi-Codebook Quanti-192 zation via neural networks . Similar works include193 Unsupervised Neural Quantization ( UNQ ) [ 18 ] and194 Spreading Vectors [ 22 ] . But ours has several key195 advantages compared to previous works : 1 ) Unlike196 UNQ , which reconstructs features by an encoder-197 decoder structure , we follow the paradigm of MCQ198 to directly give binary codes and codebooks for the199 benefit of speed and storage , for UNQ needs an200 extra decoding stage during retrieval . 2 ) UNQ and201 Spreading Vectors both project original features202 into a learned space . Although similarities between203 features are preserved , they still have biases in quan-204 tized results . This causes several issues , especially205 when we want to perform a reconstruction to ap-206 proximate original features , e.g . data compression.207 Compared to LSQ [ 14 ] , the state-of-the-art heuris-208 tic algorithm , our work is the first to tackle MCQ in209 a deep learning fashion . The policy network is de-210 signed to be very simple to get fast encoding speed211 and comparable retrieval performance.212 5 Experiments213 Our proposed Deep Multi-Codebook Quantization ( DeepQ ) is compared against the state-of-the-arts214 on a visual-feature dataset ( LabelMe22K ) to evaluate retrieval performance and encoding speed.215 Then , we scale up to make comparisons on commonly used large-scale datasets ( SIFT1M and216 DEEP1M ) , whose base sets include 1 million vectors for retrieval . Furthermore , ablation study on217 SIFT1M investigates the effectiveness of each component in our proposed pipeline.218 5.1 Datasets and evaluation metrics219 LabelMe22K [ 29 ] : This dataset collects images by the LabelMe annotation tool1 and uses Convolu-220 tional Neural Network ( CNN ) to extract them into 512-d features . It has 22 , 019 vectors for training221 and 2 , 000 vectors for test.222 SIFT1M2 and DEEP1M3 : Both datasets contain 104 , 105 , 106 vectors in query , training and base223 set , respectively . Vectors from SIFT1M is extracted by Scale-Invariant Feature Transform ( 128-d ) 224 while DEEP1M contains 96-d vectors from outputs of a CNN.225 1https : //github.com/CSAILVision/LabelMeAnnotationTool 2http : //corpus-texmex.irisa.fr/ 3http : //sites.skoltech.ru/compvision/noimi/ Recall @ { 1 , 10 , 100 } and quantization error are adopted as evaluation metrics . These two metrics226 indicate not only the retrieval performance but also the reconstruction accuracy . Because LabelMe22K227 does not have a base set , its training set is adopted as a base set . We train on the training set , and then228 encode the base set for evaluations with queries . When calculating recall , groundtruth is defined as229 the nearest neighbor of each query in the base set ( sorted by l2 distance ) . As for quantization error,230 the average value of Q ( x , b , C ) is reported over all x in the base set.231 We compare our proposal with both shallow and deep methods , including three classic quantiza-232 tion : OPQ [ 6 ] , SQ [ 15 ] and LSQ++ [ 14 , 16 ] ( denoted as LSQ for simplicity . Also , these two in233 our experiments have similar performance ) , as well as three graident-based methods : DPQ [ 11 ] ,234 DPgQ [ 5 ] and DRQ [ 28 ] . DPQ and PQNet [ 33 ] have basically the same architecture that extend235 PQ with gradient-descent , so we only report the performance of DPQ . Additionally , UNQ [ 18 ] is236 also included , although they introduce an extra decoder and re-ranking trick for retrieval.237 5.2 Implementation details238 Our method is implemented with Py-239 Torch,4 the popular deep learning240 package in Python . Codebook C is241 solved by Intel MKL that has been242 fully optimized for speed . As for net-243 work training , we adopt Adam opti-244 mizer with AMSGrad [ 21 ] and hyper-245 parameters are tuned by grid search.246 Specifically , learning rates η1 = η2 =247 2× 10−4 , with an exponetial learning248 rate decay γ = 0.9999 . Batch-size249 in updating stage is 2000 , while other250 hyper-parameters = 0.2 , α = 0.05.251 Additionally , during training , we in-252 sert dropout layers after every layer-253 normalization in all layer-groups to254 tackle overfitting . More detailed set-255 tings as well as specifications of In-256 depNet θ and QENet τ on each dataset257 ( LabelMe22K , SIFT1M , DEEP1M ) can be found in supplementary material.258 As for quantization code-lengths , K = 256 codewords for each sub-codebook and M = { 4 , 8 } 259 sub-codebooks are employed in total . We follow [ 2 ] to report “ effective ” code-lengths ( additional260 code-length for storing ‖x‖ for lookup table is ignored ) . Therefore code-lengths become { 32 , 64 } 261 bits , respectively.262 For a fair comparison , experiments are conducted on a single machine , equipped with Intel Xeon263 E5-2678v3 CPU , 256 GiB RAM , and NVIDIA RTX 3090 GPU . For other methods , we re-run on all264 datasets under unified settings with implementations provided by the authors.265 5.3 Comparisons with state-of-the-arts266 Under the small training set and base set settings on LabelMe22K , we get the results placed in267 Table 1 . Our method takes the highest recall on this dataset , outperforming the state-of-the-art268 by 2.20 % , 7.85 % , 3.30 % on 32 bits for R @ 1 , R @ 10 and R @ 100 . It also outperforms the best269 competitor by 2.70 % , 1.45 % , 0.65 % on 64 bits . In brief , All methods except for UNQ are generally270 split into three styles : 1 ) PQ-like : OPQ and DPQ . 2 ) SQ-like : SQ , DPgQ and DRQ . 3 ) MCQ : LSQ271 and ours . Generally , DPQ , DPgQ , and DRQ achieve similar results compared to their shallow272 versions . However , since they are still constrained MCQs , they show worse performances than 3 ) .273 The performance of LSQ is worse than ours , shows the effectiveness of neural networks for modeling274 the MCQ encoding problem . As for UNQ , it takes several extra tricks i.e. , another network for275 decoding and re-ranking in retrieval . Although it beats LSQ , our network still shows the power of276 MCQ to win the competition.277 4https : //pytorch.org/ 5.3.1 Large-scale retrieval performance278 Our evaluations on SIFT1M and DEEP1M datasets is presented in Table 2 . The training set and279 base set are scaled up , and retrievals on these datasets become more difficult . We observe expected280 results on two datasets . Compared to our main competitor , LSQ , our method achieves comparable281 performance on SIFT1M , and outperforms LSQ on DEEP1M in most cases . Our method achieves282 higher recall on DEEP1M than SIFT1M . A potential reason is that DEEP1M is under a nearly normal283 distribution that , in practice , is easier to converge than SIFT1M , which has a larger variance between284 datapoints . The performance of UNQ in our experiments is lower than expected , possibly due to285 different dataset settings.286 Another key advantage of our method is that , different from shallow methods , which are hand-crafted287 algorithms that find possible solutions manually or with constraints , our DeepQ encodes vectors by288 only a feed-forward.289 5.3.2 Encoding efficiency290 In order to verify the encoding efficiency of291 our method , evaluations of encoding time on292 SIFT1M with the 106 base set are conducted293 by checking the total time spent . All of them294 are run under GPU-acceleration . Additionally,295 we evaluate the time with and without the extra296 codewords refinement that introduced in section297 4.3 ( 128 bits results are simulated ) . As Figure298 3 shows , our network is significantly faster than299 LSQ since it needs to perform local search it-300 eratively for 25 or even 100 rounds . Specifi-301 cally , to encode SIFT1M base set , LSQ takes302 52.84s , 96.99s , 256.86s and 639.18s for 16 , 32,303 64 and 128 bits respectively . By contrast , our304 method takes 4.46s , 5.46s , 8.26s and 16.64s,305 which is 11.8× , 17.8× , 31.1× and 38.4× faster306 than LSQ . Moreover , our method is even faster307 than most of the constrained MCQs . We also308 notice that the refinement takes negligible over-309 head . Although UNQ takes the fastest encoding310 speed , it still needs to decode and re-rank during311 retrieval , which slows down its retrieval speed.312 5.3.3 Reconstruction accuracy313 The comparisons of quantization error on three314 datasets are stated in Table 3 . Basically , when the quantization error gets lower , recall will be higher.315 Ours get the 2nd place on SIFT1M , and the lowest on remaining datasets in most cases . Quantization316 error indicates reconstruction accuracy and further shows the quality of codebook generation and317 quantization codes selection . Notably , ours significantly outperforms UNQ , which has a strong bias318 on the reconstruction task . This is because they focus more on ranking , not the quantization error.319 The result shows that our method can be applied to other areas , e.g . vector compression.320 5.4 Ablation study321 Method SIFT1M @ 32 bitsQE R @ 1 R @ 10 R @ 100 w/o regularization 3.38× 104 7.60 29.96 68.73 w/o return-norm 3.06× 104 10.57 36.44 76.04 w/o correction 3.10× 104 10.09 35.30 75.16 w/o refinement 3.17× 104 9.91 30.39 68.28 DeepQ 2.92× 104 11.02 37.73 76.79 Table 4 : Ablation study conducted on SIFT1M with 32 bits code-length . Entropy regularization forces network to try more codeword combinations , which help to jump out of local-optima . Return normalization and value correction help for fast convergence . The extra refinement leads to low quantization error and high recall with negligible costs . Our ablation study is conducted on the322 SIFT1M dataset , with the code-length of323 32 bits , which in our experiments is suf-324 ficient to show how does each component325 affects our model . We choose the following326 variants to perform ablation:327 w/o regularization : which removes eθ in328 the losses , and the output distributions will329 not be forced to be uniform.330 w/o return-norm : which does not normal-331 ize R , and therefor advantage is computed332 by R other than R̄.333 w/o correction : which removes value cor-334 rection . So our VC-PPO falls back to the original PPO.335 w/o refinement : which directly encode the base set without extra refinement.336 Quantization error and recall are evaluated and placed in Table 4 . We report the best value they337 ever met during the training procedure . Specifically , when regularization is removed , it seems that338 the network is trapped in local-optima and the performance drops . Meanwhile , although return339 normalization and value correction give us only subtle improvements , we find they help the network340 to converge quickly . The extra refinement gives us lower quantization error and higher recall , specially341 when we want to perform fast training before the network is converged.342 6 Conclusion and Future Work343 In this paper , we first review previous works of constrained MCQs , and investigate solutions to344 unconstrained ones . Since finding the global-optima of MCQ is NP-hard , researchers apply constraints345 to find near-optimal solutions or employ heuristic algorithms that are still time-consuming . This346 paper takes the first attempt to find a deep solution to MCQ . The proposed IndepNet is designed to be347 simple enough to encode vectors extremely fast . Furthermore , our network shows state-of-the-art348 performance in retrieval and reconstruction tasks . Our method is slow to converge in a large dataset,349 which hinders our performance . So , our future work will focus on training speedup.350 References351 [ 1 ] Anderson , E. , Bai , Z. , Dongarra , J. J. , Greenbaum , A. , McKenney , A. , Croz , J. D. , Hammarling , S.,352 Demmel , J. , Bischof , C. H. , and Sorensen , D. C. ( 1990 ) . LAPACK : a portable linear algebra library for353 high-performance computers . In SC , pages 2–11.354 [ 2 ] Babenko , A. and Lempitsky , V. ( 2014 ) . Additive quantization for extreme vector compression . In CVPR,355 pages 931–938.356 [ 3 ] Babenko , A. and Lempitsky , V. ( 2015 ) . Tree quantization for large-scale similarity search and classification.357 In CVPR , pages 4240–4248.358 [ 4 ] Chen , Y. , Guan , T. , and Wang , C. ( 2010 ) . Approximate nearest neighbor search by residual vector359 quantization . Sensors , 10 ( 12 ) :11259–11273.360 [ 5 ] Gao , L. , Zhu , X. , Song , J. , Zhao , Z. , and Shen , H. T. ( 2019 ) . Beyond product quantization : Deep progressive361 quantization for image retrieval . In IJCAI , pages 723–729.362 [ 6 ] Ge , T. , He , K. , Ke , Q. , and Sun , J . ( 2013 ) . Optimized product quantization for approximate nearest neighbor363 search . In CVPR , pages 2946–2953.364 [ 7 ] Gray , R. ( 1984 ) . Vector quantization . IEEE Assp Magazine , 1 ( 2 ) :4–29.365 [ 8 ] Ilyas , A. , Engstrom , L. , Santurkar , S. , Tsipras , D. , Janoos , F. , Rudolph , L. , and Madry , A . ( 2020 ) . A closer366 look at deep policy gradients . In ICLR.367 [ 9 ] Jégou , H. , Douze , M. , and Schmid , C. ( 2010 ) . Product quantization for nearest neighbor search . IEEE Trans.368 Pattern Anal . Mach . Intell. , 33 ( 1 ) :117–128.369 [ 10 ] Kalantidis , Y. and Avrithis , Y . ( 2014 ) . Locally optimized product quantization for approximate nearest370 neighbor search . In CVPR , pages 2329–2336.371 [ 11 ] Klein , B. and Wolf , L. ( 2019 ) . End-to-end supervised product quantization for image search and retrieval.372 In CVPR , pages 5041–5050.373 [ 12 ] Konda , V. R. and Tsitsiklis , J. N. ( 2000 ) . Actor-critic algorithms . In NeurIPS , pages 1008–1014.374 [ 13 ] Lloyd , S. ( 1982 ) . Least squares quantization in pcm . IEEE transactions on information theory , 28 ( 2 ) :129–375 137.376 [ 14 ] Martinez , J. , Clement , J. , Hoos , H. H. , and Little , J. J . ( 2016 ) . Revisiting additive quantization . In ECCV,377 pages 137–153 . Springer.378 [ 15 ] Martinez , J. , Hoos , H. H. , and Little , J. J . ( 2014 ) . Stacked quantizers for compositional vector compression.379 arXiv preprint arXiv:1411.2173.380 [ 16 ] Martinez , J. , Zakhmi , S. , Hoos , H. H. , and Little , J. J . ( 2018 ) . Lsq++ : Lower running time and higher381 recall in multi-codebook quantization . In ECCV , pages 491–506.382 [ 17 ] Mohamed , S. , Rosca , M. , Figurnov , M. , and Mnih , A . ( 2020 ) . Monte carlo gradient estimation in machine383 learning . J. Mach . Learn . Res. , 21:132:1–132:62.384 [ 18 ] Morozov , S. and Babenko , A . ( 2019 ) . Unsupervised neural quantization for compressed-domain similarity385 search . In ICCV , pages 3036–3045.386 [ 19 ] Norouzi , M. and Fleet , D. J . ( 2013 ) . Cartesian k-means . In CVPR , pages 3017–3024.387 [ 20 ] Radenovic , F. , Iscen , A. , Tolias , G. , Avrithis , Y. , and Chum , O . ( 2018 ) . Revisiting oxford and paris:388 Large-scale image retrieval benchmarking . In CVPR , pages 5706–5715.389 [ 21 ] Reddi , S. J. , Kale , S. , and Kumar , S. ( 2018 ) . On the convergence of adam and beyond . In ICLR.390 [ 22 ] Sablayrolles , A. , Douze , M. , Schmid , C. , and Jégou , H. ( 2019 ) . Spreading vectors for similarity search . In391 ICLR.392 [ 23 ] Schulman , J. , Heess , N. , Weber , T. , and Abbeel , P. ( 2015a ) . Gradient estimation using stochastic computa-393 tion graphs . In NeurIPS , pages 3528–3536.394 [ 24 ] Schulman , J. , Levine , S. , Abbeel , P. , Jordan , M. I. , and Moritz , P. ( 2015b ) . Trust region policy optimization.395 In ICML , pages 1889–1897.396 [ 25 ] Schulman , J. , Moritz , P. , Levine , S. , Jordan , M. I. , and Abbeel , P. ( 2016 ) . High-dimensional continuous397 control using generalized advantage estimation . In ICLR.398 [ 26 ] Schulman , J. , Wolski , F. , Dhariwal , P. , Radford , A. , and Klimov , O . ( 2017 ) . Proximal policy optimization399 algorithms . arXiv preprint arXiv:1707.06347.400 [ 27 ] Song , J. , Lang , R. , Zhu , X. , Xu , X. , Gao , L. , and Shen , H. T. ( 2020 ) . 3d self-attention for unsupervised401 video quantization . In ACM SIGIR , pages 1061–1070.402 [ 28 ] Song , J. , Zhu , X. , Gao , L. , Xu , X.-S. , Liu , W. , and Shen , H. T. ( 2019 ) . Deep recurrent quantization for403 generating sequential binary codes . In IJCAI , pages 912–918.404 [ 29 ] Torralba , A. , Fergus , R. , and Weiss , Y . ( 2008 ) . Small codes and large image databases for recognition . In405 CVPR , pages 1–8 . IEEE.406 [ 30 ] Wang , J. , Wang , J. , Song , J. , Xu , X. , Shen , H. T. , and Li , S. ( 2015 ) . Optimized cartesian k-means . IEEE407 Transactions on Knowledge and Data Engineering , 27 ( 1 ) :180–192.408 [ 31 ] Weyand , T. , Araujo , A. , Cao , B. , and Sim , J . ( 2020 ) . Google landmarks dataset v2 - A large-scale409 benchmark for instance-level recognition and retrieval . In CVPR , pages 2572–2581.410 [ 32 ] Williams , R. J . ( 1992 ) . Simple statistical gradient-following algorithms for connectionist reinforcement411 learning . Machine learning , 8 ( 3-4 ) :229–256.412 [ 33 ] Yu , T. , Yuan , J. , Fang , C. , and Jin , H. ( 2018 ) . Product quantization network for fast image retrieval . In413 ECCV , pages 186–201.414 [ 34 ] Zhang , T. , Du , C. , and Wang , J . ( 2014 ) . Composite quantization for approximate nearest neighbor search.415 In ICML , volume 2 , page 3.416 Checklist417 The checklist follows the references . Please read the checklist guidelines carefully for information on418 how to answer these questions . For each question , change the default [ TODO ] to [ Yes ] , [ No ] , or419 [ N/A ] . You are strongly encouraged to include a justification to your answer , either by referencing420 the appropriate section of your paper or providing a brief inline description . For example:421 • Did you include the license to the code and datasets ? [ Yes ] See Section ? ? .422 • Did you include the license to the code and datasets ? [ No ] The code and the data are proprietary.423 • Did you include the license to the code and datasets ? [ N/A ] 424 Please do not modify the questions and only use the provided macros for your answers . Note that the425 Checklist section does not count towards the page limit . In your paper , please delete this instructions426 block and only keep the Checklist section heading above along with the questions/answers below.427 1 . For all authors ... 428 ( a ) Do the main claims made in the abstract and introduction accurately reflect the paper ’ s429 contributions and scope ? [ Yes ] 430 ( b ) Did you describe the limitations of your work ? [ Yes ] See Section 6.431 ( c ) Did you discuss any potential negative societal impacts of your work ? [ N/A ] 432 ( d ) Have you read the ethics review guidelines and ensured that your paper conforms to them ? 433 [ Yes ] 434 2 . If you are including theoretical results ... 435 ( a ) Did you state the full set of assumptions of all theoretical results ? [ N/A ] 436 ( b ) Did you include complete proofs of all theoretical results ? [ N/A ] 437 3 . If you ran experiments ... 438 ( a ) Did you include the code , data , and instructions needed to reproduce the main experimental439 results ( either in the supplemental material or as a URL ) ? [ Yes ] See https : //github.com/440 DeepMCQ/DeepQ.441 ( b ) Did you specify all the training details ( e.g. , data splits , hyperparameters , how they were442 chosen ) ? [ Yes ] See Section 5 and supplementary materials.443 ( c ) Did you report error bars ( e.g. , with respect to the random seed after running experiments444 multiple times ) ? [ No ] 445 ( d ) Did you include the total amount of compute and the type of resources used ( e.g. , type of446 GPUs , internal cluster , or cloud provider ) ? [ Yes ] See Section 5.447 4 . If you are using existing assets ( e.g. , code , data , models ) or curating/releasing new assets ... 448 ( a ) If your work uses existing assets , did you cite the creators ? [ Yes ] See Section 5.449 ( b ) Did you mention the license of the assets ? [ N/A ] 450 ( c ) Did you include any new assets either in the supplemental material or as a URL ? [ Yes ] See451 https : //github.com/DeepMCQ/DeepQ.452 ( d ) Did you discuss whether and how consent was obtained from people whose data you ’ re453 using/curating ? [ N/A ] 454 ( e ) Did you discuss whether the data you are using/curating contains personally identifiable455 information or offensive content ? [ N/A ] 456 5 . If you used crowdsourcing or conducted research with human subjects ... 457 ( a ) Did you include the full text of instructions given to participants and screenshots , if applica-458 ble ? [ N/A ] 459 ( b ) Did you describe any potential participant risks , with links to Institutional Review Board460 ( IRB ) approvals , if applicable ? [ N/A ] 461 ( c ) Did you include the estimated hourly wage paid to participants and the total amount spent on462 participant compensation ? [ N/A ] 463

Summary: This paper proposes a multi-codebook quantization method (aka additive quantization).   In this setting vectors are decoded as x = sum_i C(i, b_i) with C a codebook matrix.  There is no clear optimal algorithm to train the codebook and optimize the b_i (the strongest competitor that this paper compares with is LSQ [16]).   This paper proposes to train a neural net that computes the b_i's in a forward pass together with the codebook matrices.  At encoding time, the neural net outputs a probability distribution for the code components (a policy) from which several codes are sampled randomly, and the best one is kept (after computing the quantization error).  The training is based on a policy optimization, that is able to handle noisy gradients that result from the code sampling procedure.   The experiments show promising results, that sometimes outperform the SOTA LSQ at a much lower encoding cost.