Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The authors present an approach to optimize determinantal point processes directly (by gradient descent, instead of . via approximations), so that diversity could be modeled in objective functions for deep learning systems. The approach taken is to express the DPP term as an L-ensemble in the spectral domain over the gram matrix. They also generate sub-gradients for cases where the gradient does not exist (when the gram matrix is not invertible).

The approach is interesting, and the problem (modeling of diversity constraints) seems important. They have experimental results (on metric learning and image learning tasks) to show that optimization with the DPP + wasserstein  gan constraint (to ensure features lie in a bounded space) result in better quality. However, they do not discuss the performance (comptuational) impact, or compare their approach to other approximation based systems (such as the approach of Elfeki et al).
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: Summary: the authors introduce a method to learn a deep-learning model whose loss function is augmented with a DPP-like regularization term to enforce diversity within the feature embeddings. 

Decision: I recommend that this paper be rejected. At a high level, this paper is experimentally focused, but I am not convinced that the experiments are sufficient for acceptance.

****************************
My main concerns are as follows:

- Many mathematical claims should be more carefully stated. For example, the authors extend the discrete DPP formulation to continuous space. It is not clear to me, based on the choice of the kernel function embedding, that the resulting P_k(X) is a probability (Eq. 1). If it is not (using a DPP-based formulation as a regularizer does not require a distribution), the authors should clarify that fact; more generally, the authors should be more careful throughout the paper (for example, det=0 if features are proportional, not necessarily equal; the authors inconsistently switch between DPP kernel L and marginal kernel K throughout computations.)

- The authors do not describe their baselines for several experiments. In tables 1, 2, 3, the baseline is never described (I assume it's the same setup without regularization); I did not find a description of DCH (Tab 4) in the paper (Deep Cauchy Hashing?). The mAP-k metric should also be defined. Furthermore, the authors do not report standard deviations for their experiments.

- A key consideration when using DPPs is their compulational cost: most operations involving them require SVD (which seems to be used in this work), matrix inversion, and often both. This, unsurprisingly, limits the applications of DPPs, and has driven a lot of research focused on improving DPP overhead. I would like to see more discussion in this paper focused on to which extent the DPP's computational overhead remains tractable, and which methods were used (if any) to alleviate the computational burden.

- Finally, the paper itself appears to be somewhat incomplete: sentences are missing or incomplete (Section 4), and numbers are missing in some tables (Table 5).


***********************
Questions and comments for the authors:

- When computing the proper sub-gradient, are you computing the subgradient as inv(L + \hat L)?

- You state that by avoiding matrix inversion, your method is more feasible. However, it seems like your method requires SVD, which is also O(N^3); could you please provide more detail for this?

- Could you report number of trials and standard deviations for your experiments?

- Do you have any insight into why DPPs do more poorly than the DCH baseline in Table 4 for mAP-k metrics?

- You might be able to save space by decreasing the space allocated to the exponentiated quadratic kernel.

review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: Determinantal Point Processes (DPPs) are statistical models that allow efficient sampling of diverse solutions - a problem that is hard with most machine learning modeling frameworks.  However, learning diverse features via DPPs with deep learning frameworks is challenging due the instability in computing the gradient which involves a matrix inverse operation. Better marriage between deep learning and DPPs is an important problem as diversity is crucial in many machine learning problems (even beyond computer vision - the experimental domain of this paper - e.g. in machine translation and document summarization). 

The authors of this paper make several important contribution to this problem. First, they develop an effective sub gradient procedure, proper spectral subgradient generation,  that can replace the true gradient and empirically demonstrate its effectiveness in computer vision applications.  Then, they describe a method for constraining the DPP features into a bounded space to facilitate network predictability. When incorporating Wasserstein GAN into their framework they show performance gains in computer vision tasks: metric learning, image hashing and local descriptor
retrieval task based on HardNet.

The paper is overall clearly written, but in some places it is not well edited (to the level that is should be carefully proofread). For example, "For the first test." just before 4.1, and "showed significance performance" in the last sentence of the paper. I strongly encourage the authors to carefully edit the paper before publication so that their nice work will be properly presented.


review_title: Official Blind Review #1

Meta-review: Most reviewers seems in favour of accepting this paper, with the borderline rejection being satisfied with acceptance if the authors take special heed of their comments to improve the clarity of the paper when preparing the final version. From examination of the reviews, the paper achieves enough to warrant publication. Accept.
Decision: Accept (Poster)