Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes to use the variational estimation of the (lower bound of the) skew renyi divergence for contrastive representation learning. The authors compare the reynyi divergence against traditional divergence such as KL divergence and skew KL divergence, and argue the proposed skew renyi divergence is beneficial for contrastive representation learning empirically.
strengths_and_weaknesses: While it is nice to have a new divergence and understand some properties of it, I am not quite convinced why the proposed skew Renyi divergence is better than the traditional ones, for example, skew KL divergence. Besides the empirical results, is there any explanation why skew Renyi divergence is preferred? 

Another thing that bothers me is the paper entangles the analysis of of the skew Renyi divergence with data augmentation. I am curious if it is possible to decouple the two. One can do contrastive learning using the estimator of the skew Renyi divergence without data augmentation. Coupling the two makes the analysis harder to read. In particular, I don't feel the transition probability A(v|x) is well explained. Why do authors need to use the transition probability to define the positive and negative pairs? Can we define them just using the data distribution without the transition probability?


questions: 1. equation below line 86: g is not defined
2. issues with using Donsker style variational estimation: The sup is over all possible functions. However in practice one has to assume some function class which is more restrictive. In the end the estimation is only a lower bound.
3. line 212: can we have some details on how to estimate the constant Z using Monte-Carlo methods? how complex is that? usually this is a headache for a lot of problems


limitations: NA
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper revisits the derivations of the InfoNCE objective from the original *Contrastive Predictive Coding* paper, replacing the KL divergence in the mutual mutual information with the Renyi divergence. As with the KL divergence, the Renyi divergence is known to enjoy a variational representation, which in both cases has a naive estimator that suffers from high variance. They then note a variance reduction idea, which I do not recognize from the contrastive literature, that shows that for D \in \{KL, Renyi\}, replacing D(P|Q) with D(P | alpha*P + (1-alpha)*Q) for some small alpha yields an estimator with much improved variance. The paper proposes to use this variance reduction with the Renyi divergence as a plug-in replacement for InfoNCE as a training objective. The final theoretical component is to show that the resulting objective has gradients that naturally place high importance weights on hard negative and easy positives. The authors suggest that the easy positive in particular lend themselves to using more aggressive data augmentations in practice. Experiments cover visual, tabular, and graph structured data. 
strengths_and_weaknesses: Congratulations to the authors on a well written, tidy piece of work. There are certainly some interesting ideas in this work, and I enjoyed reading this paper. First I outline some of the positive aspects I saw in this work, before turning to some weaker points and a few concerns I have, particularly with some of the experimental comparisons. 

It is good to see that the thought process behind the mutual information —> InfoNCE derivation can be replicated for other divergences. Perhaps the same could be done with all sorts of other divergences, leading to a variety of different behaviors in objective functions. I also liked the presentation of the skew-divergence connection. The skew-divergence seems like it has useful low-variance, high-bias properties that could be useful in other contrastive learning contexts. 

One of the premier benefits of RenyiCL is touted to be able to gracefully handle, and make use of, aggressive positive sampling thanks to its weighting biassing towards easy positives. As such the authors use what is said to be “harder data augmentations” for their method.  One of the ingredients constituting “harder augmentations” is multi-crop sampling à la SwAV. This raises a major concern of a lack of apples to apples vision comparisons, since the multi-crop method has, in all cases I have ever seen, always boosted performance, and increase the memory and runtime requirements. So the results reported in Tables {1,2,3} all compare a method using multi-crop to methods that do not (except for SwAV, which does and performs similarly to RenyiCL). It would be much fairer to remove the multi-crop sampling from RenyiCL. Table 4 has this number, but it is only for a 200 epoch run so again it is not really fair to compare. Finally, Table 5 compares   RenyiCL to other methods using the hard data augmentations. An important bright-spot here is that RenyiCL does see the biggest boost of the three methods, which is great to see. However RenyiCL is only compared to CPC and a variant of CPC; it remains unclear, for instance, how MoCo-v3 would fare in comparison using multi-crop. In all, it remains hard to say exactly how RenyiCL compares to other methods, besides to say that it is likely broadly comparable.

The inclusion of experiments on tabular and graph data is welcome, and I am glad to see promising results on other domains besides vision. 

I end with a much more philosophical point. The motivation for “why Renyi” divergence is mostly post hoc—instead of any intrinsic explanation for why Renyi may be a better estimator than KL, the authors first go through the process of deriving their low-variance estimator, and analyzing its gradients. They then find a number of natural, potentially desirable, attributes of the final objective such as easy positive/hard negative sample importance weights.  This is, again, very similar to process of producing the InfoNCE objective, starting from mutual information. It was convincingly argued by Tschannen & Djolonga et al. (2020), that it is the form of the InfoNCE loss itself, rather than any relation to mutual information, that explains its success. I have to say that I very much expect the same to hold for the-based Renyi objective presented in this work. From this point of view, it is worth putting to one side for a moment all of the scaffolding built on top of the idea of the Renyi divergence, and simply focus on the form of the final RenyiCL objective. As noted by the authors, this objective is of similar form to prior idea, but with an importance sampling re-weighting to emphasize hard negatives and easy positives. The main distinction here from prior works, which have considered hard negatives and hard positives using similar importance weightings, is to keep using hard negatives, but instead to reverse the focus to be on *easy positives*, and note that actually this may buy some benefits in terms of being able to use more aggressive data augmentations. This is an interesting proposal, and may well be at the core of the effectiveness of the proposed method; the Renyi divergence may mostly serve to obfuscate this.  


To conclude, while I have some gripes with the vision experimental setup, and suspect that the main benefit of RenyiCL of easy positive sampling could similarly be implemented in any other contrastive framework using importance weights, I am still broadly supportive of this work. To me this core recipe of easy positive weights + aggressive data augmentations (and its justification in Table 5) is already an interesting contribution, since how to produce good positive samples is perhaps the single most important question in contrastive learning. On a different note, the paper also offers other ideas, such as the skew-divergences, and their applicability to both KL and Renyi divergences seem interesting and may be of independent interest for producing low-variance contrastive learning objectives. 

---- 

Post rebuttal: increased score by one.

questions: See previous box, particularly regarding experimental comparisons. 
limitations: Yes.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper investigates a novel alternative formulation of contrastive learning that leverages the Renyi divergence. Motivated by the sensitivity of augmentation schemes for the learned representations, the author(s) argues using Renyi divergence can better manage hard augmentations. To overcome the variance issue in the naive formulation, a more stable version based on skewed statistics is advocated, with theoretical justifications. The author(s) showed that using the proposed RenyiRL can spared the need for additional hacks or extra computations to get decent performance. Strong empirical evidence supports the claims made in the paper.
strengths_and_weaknesses: ### Strength

* *(Quality, Originality)* The application of Renyi contrastive learning for self-supervised learning is novel. Although I have seen other recent works also attempting to Renyi for contrastive learning, I believe this manuscript is the only one that well balances theoretical and practical aspects.
*  *(Clarity, Significance)* This paper is written with exceptional clarity and I enjoyed reading it. It provides an extensive coverage on related work, and I am genuinely surprised to see some research threads not well known to the general contrastive learning community. Nice work. I added some additional thread which should make the literature review more comprehensive, and will hopefully inspire future research ideas.
* *(Quality)* Strong experimental results, well curated experimental code. 

### Weakness

* It is not clear to me how the variance in Theorem 3.2 turns into the variance in Theorem 3.1 in the limiting case $\alpha=0$, could the author(s) elaborate on that?

* Variance is not the only reason causing InfoNCE to perform sub-optimally. Discussions from [5-7] have presented diverse perspectives on how contrastive learning can be improved based numerical or optimization views, also backed up strong mathematical arguments. These works are complementary to the approach adopted here and should be discussed. 

### Misc

* Theorem 3.1 is related to the result from [1] on the exponential variance issue for MI estimation, which should be  discussed.
* The objective function for skewed $\chi^2$ divergence presented in the [Tsai et al. 2021] (which this work is based on) is related to the spectrum contrastive learning objective [5], which is simpler and also demonstrated strong performance gains over the vanilla InfoNCE objective. The Renyi divergence and $\chi^2$ divergence have also been successfully applied in the context of generative modeling [2-4], where the goal of minimizing $D_{\alpha}(P_{XY}\parallel P_XP_Y)$ or $D_{\alpha}(P_X\parallel Q_X)$ is shared. 
* Potential typos: $\alpha$-CPC is actually proposed by [B Poole et al. ICML 2019], not [J Song, et al. NeurIPS 2020].

[1] McAllester, D. and Stratos, K. Formal limitations on the measurement of mutual information. AISTATS 2020
[2] Y Li, et al. Rényi Divergence Variational Inference. NeurIPS 2016
[3] L Chen, et al. Variational inference and model selection with generalized evidence bounds. ICML 2018
[4] C Tao. Chi-square Generative Adversarial Network. ICML 2018
[5] J HaoChen, et al. Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss. NeurIPS 2021
[6] Q Guo, et al. Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization. 2021
[7] J Chen, et al. Simpler, Faster, Stronger: Breaking The log-K Curse On Contrastive Learners With FlatNCE. 2021

questions: See my first comment in the Weakness section.
limitations: I don't have any concerns on limitations and potential negative societal impact.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposed the new robust contrastive learning scheme called RényiCL. By using the Rényi divergence instead of KL divergence as the objective function, the proposed method becomes robust against harder data augmentations. In order to keep the variance of the objective function small, this paper proposed the use of a variational lower bound for skewed Rényi divergence, which allows for training stabilization. The use of such skewed divergence is based on the observation that the original contrastive learning corresponds to the variational lower bound of the skewed KL divergence.
strengths_and_weaknesses: ## Strength
- The flow of the text is clear, and the purpose, background, and proposed method of this study can be easily understood.
- The proposed method is simple to implement and can be easily used as a plug-in to existing SSL methods.
- Numerical experiments clearly demonstrate the effectiveness of the proposed method. The proposed method achieves comparable or slightly better performance than existing SSL methods when using normal data augmentation. In the situation with strong data augmentation, there was significant performance improvement over the existing methods, InfoMin and CLSA, for the two conditions of CUB200 and the Plant dataset.

## Weakness
- The significance of the theorem 3.2 is not adequately presented. This paper argues that skewed divergence allows the variance of the objective function to be bounded from above using some constants. However, this argument is not sufficiently convincing because there is not enough reference to the actual magnitude of the constant. More details are given in the Questions section
- Discussion about the gradient analysis of RényiCL is not fully convincing. The interpretation of the effect of the change in divergence is discussed by comparing the gradient of the objective function of RényiCL with the baseline gradient, but the conclusion is not convincing due to its logical development. I will discuss in detail in the Question section.
- Lack of ablation studies on robustness. Since the main concern of this paper is robustness to various data augmentation, the paper should directly demonstrate the effectiveness of the proposed method by comparing the performance by ablation of the contrastive objective under strong data augmentation.

questions: ## Implications of Theorem 3.2 for this study
The claim that motivates the technical contribution of this paper is Theorem 3.1, and the claim is roughly as follows: The estimated variance around the true solution of the Rényi divergence can be bounded from below by an exponential function on the difference between the KL divergence and the Rényi divergence. This observation indicates the variance of the naive objective could explode exponentially. Against this, Theorem 3.2 and additional theorem in the appendix argue that by introducing the skewed divergence instead of naive divergence, the estimated variance can be bounded from above by a constant. Although it attempts to show that exponential explosion of variance can be prevented because the constant appearing in the upper bound is independent of the number of samples, I believe this argument is inadequate for the following reasons

To begin with, the exponential function itself, which appears in the lower bound of the estimated variance, is also not a sample size dependent issue. The lower bound on the estimated variance in Theorem 3.1 depends essentially on $\gamma$ and the magnitude of divergence between distributions $P$ and $Q$, not on the number of samples. In other words, even if the constants appearing in the upper bound are independent of sample size, if they have exponential dependence on these variables, the upper bound is loose and meaningless in current context. The authors should explicitly mention in the text the order these constants have.

## Logical developments in the discussion of the gradient analysis for RényiCL
I have several concerns on the logical developments in the discussion on the gradient analysis. This paper discussed the effect of using $\alpha$-skewed Rényi divergence by investigating the gradient of the corresponding gradient. The brief logical flow is as follows:
1. Consider the case of $\alpha = 0$ for simplicity (the paper states it can be done without loss of generality)
2. Take the gradients for both baseline objective (MLCPC) and proposed Rényi objective (RMLCPC)
3. Construct the alternative objective functions of which gradient is equivalent to the original one
4. Interpret the implication of each term appears in the alternative objective function

I have two concerns about the above logical flow. The first one is a major and the second one is a minor concern.

### The simplification of $\alpha=0$ could mislead the discussion
By interpreting the positive first term for positive pairs and the negative second term for negative pairs, respectively, the authors state that the proposed method with $\gamma>1$ achieves easy positive sampling & hard negative sampling simultaneously compared to existing methods. However, it is only in the special case where $\alpha=0$ that terms for positive/negative pairs appear corresponding to exactly positive first/negative second terms, respectively. If we set α > 0 (which is done in practical experiment), the negative term which include positive samples will appears. For this reason, I believe that the claim that easy positive sampling occurs by using Rényi divergence with $\gamma>1$ somewhat ignores the real mechanism.

### The gradient of the alternative objective functions presented is not equivalent with original one
The paper states that the original objective function "is equivalent" to the alternative objective function in terms of gradient (e.g., p.6, l.184). However, this statement is not true **unless we introduce the stop-gradient operator for $q_{\theta}$**. This logical development itself is not a problem as long as the stop-gradient is mentioned, so the authors should clearly state how each equations related.

## Minor comments
- p.5, l.174: "$\alpha$-skew $\alpha$-skew Rényi divergence between ..." -> "$\alpha$-skew Rényi divergence between ..."
limitations: The authors have adequately addressed the limitations and potential negative societal impact of their work in conclusion section. In particular, the paper properly mentioned that it is difficult to identify what kind of data augmentation would be effective as a limitation of the proposed method. Yet, since the introduction of the paper motivates that the choice of data extension is the key in contrastive learning, there seems to be a slight discrepancy between the research objective and the actual proposed method.
review_title: null

Meta-review: metareview: The reviewers reached a consensus that this paper is a nice addition to NeuRIPS. Please refers to the reviews and author's responses for reviewers' opinions on the strength and weakness of the paper.


confidence: Certain
Decision: Accept