Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The paper proposes new loss functions for quantization when the task of interest is maximum inner product search (MIPS).
The paper is well written with clear descriptions, fairly comprehensive analysis and empirical exploration, and good results, and in general I agree that learning quantization so as to minimize quantization related errors on task at hand is a good strategy.  
Specific comments and suggestions for strengthening the paper are:
a) The proposed loss function in (2) includes a weight function that serves as a proxy for the task objective of giving more emphasis to quantization errors on samples with larger inner product.  Instead, why not use the true task objective which for the MIPS task is stated in the Introduction section?  If this was considered please comment on reasons for not including / discussing this in the paper, otherwise perhaps this’ll be good to discuss.
b) Did the authors consider using a task dependent training data set which will capture both ‘q’ and ‘x’ distributions and potentially lead to even further improved quantization?  This has the disadvantage of making quantization dependent on query distribution, but in cases where such data is available it will be very valuable to know if incorporating data distributions in quantization process helps performance and to what extent.
c) It will also be valuable to consider the closely related task of cosine distance based retrieval and comment on how that impacts the modifications of loss functions.
d) The idea of learning quantization under objective of interest using observed data distribution has been studied earlier (e.g. see Marcheret et al., “Optimal quantization and bit allocation for compressing large discriminative feature space transforms,” ASRU 2009), perhaps worth citing as related work.

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: The paper proposes a novel quantization algorithm for maximum inner product search. Instead of imposing equal weights on all possible queries, the proposed framework considers giving larger weights on queries having larger inner product values. The authors shows that this derives a loss function having different weights on parallel and orthogonal component of residual of quantization. The paper shows the performance of the algorithm mainly by using the Glove1.2M dataset.

The basic idea and the weighting approach would be reasonable. Empirical evaluation might be slightly weak.

In practice, how can the hyper-parameter (T and b) be determined? T = 0.2 shows the best performance on Fig. 2 (b), but its seems sensitive, and generality of this setting is not clear.

In (7), only <q,x> larger than T is penalized, but larger <q,\tidel{x}> seems important as well. If the encoded \tilde{x} has a large inner product value wrongly, it would deteriorate performance.

The tightness of the the upper bound of the inequality in the end of page 4 would be unclear. Replacing norm with its possible maximum value seemingly has large effect.

Minor comment:
Figure 1 is not referred from the main text.

In Fig. 2 (b), reconstruction and proposed have the save value at T = 0.1. Is this just a coincident?
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: Summary:
The authors propose a new loss function for solving large-scale inner product search that rely on quantization, based on the intuition that all pairs of (query, database vector) are not equally important for a given query. In particular, the authors weight the reconstruction error so that the pairs with a higher scalar product are more precisely quantized (as they lie among the most plausible candidates).

Strengths of the paper:
- The paper is well written and easy to follow. In particular, the intuition of the method is well explained in Figure 1 and the setup in Section 3 is well formulated.
- The proposed method works with a variety of quantization approaches, such as binary, simple PQ or LSQ (even if the authors aren't able to report results for this last method due to technical issues as explained in Appendix 7.7)

Weaknesses of the paper:
- The related work could be more detailed, see for example: "Spreading vectors for Similarity Search", Sablayrolles et al. ; "Pairwise Quantization",  Babenko et al ; "Unsupervised Neural Quantization for Compressed-Domain Similarity Search", Morozov et al.

Justification of rating:
The paper proposes a new loss function that weights the scalar products differently according to their importance than can be applied to a wide range of existing quantization methods. However, the strength of experimental results (in particular the fact that LSQ or other cited references above are missing) remains unclear. 
review_title: Official Blind Review #1

Meta-review: While there was some support for the ideas presented, the majority of reviewers felt that this submission is not ready for publication at ICLR in its present form.

Concerns were raised as to the generality of the approach, thoroughness of experiments, and clarity of the exposition.
Decision: Reject