Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This interesting paper on an important topic; however, its readability could be dramatically improved, especially for the reader less familiar with the problem. 

In order to make the paper more accessible, the authors should reorganize the introduction by breaking it down into two parts:
1) a more traditional introduction 
- one intuitive paragraph about multi-view coding
- one intuitive paragraph with an illustrative example on how the proposed approach will help solve a problem; at the same intuitive level, compare-and-contrast it with existing approaches 
- one intuitive, in-detail paragraph on how the proposed approach works
- one paragraph summarizing the main findings/results 
2) a second, new section, that will turn the current Figures 1 & 2 into a complete description of an illustrative example (the current, detailed "captions" are a good start, but they should be fleshed out into a full, detailed section of the paper)  

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper proposed a new self-supervised learning methods by utilizing contrastive predictive coding technique.  The proposed algorithm is more effective than existing self-supervised learning algorithm.  The presented results are encouraging.  
1. In section 3.2,  the authors show that  a large number of views would improve the representation quality. However,  multi-views may provide redundancy information. What is the core information that affect  the representation quality?

In fact,  I am not an expert on self-supervised learning and  contrastive predictive coding,  so my reviewer confidence is low.
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: The paper presented a multi-view learning method that is based on negative sampling in contrastive learning. The core idea is to set an anchor view and the sample positive and negative data points from the other view and maximise the agreement between positive pairs in learning from two views. When more than two views are presented, the learning objective is a sum over all possible combinations of two views. The performance of the proposed model is good, and the ablation study is interesting. 

Comments:

1. The core concept, or at least one of the core concepts, in multi-view learning is the conditional independence.

Normally, the underlying assumption in multi-view learning is that, given the class label, the samples from multiple views are conditionally independent from each other. Therefore, the goal is to learn distinctive representations from different data sources/disjoint populations, so then after learning, the ensemble of them is able to capture a set of diverse aspects of the data. A "side-effect" of learning from multiple views is that individual views indeed get improved by learning from others. Meanwhile, self-supervised learning is the case when the input data to the designed learning system is also the target of the system. 

The paper presented an idea for self-supervised learning from multiple views, which is not exactly the same, but still in the same regime. This concept could be used to explain some empirical findings in this paper. Since it is expected, there is even no need in conducting experiments. 




2. My main concern of this paper is the novelty, however, the empirical results are strong.

The paper mainly presented a simple yet effective method for self-supervised learning from two views, and the generalisation is a sum over all possible combinations of two views. The method itself has already been proposed many years ago as mentioned in the related work section in the paper, and the generalisation was also described in prior work, which makes me doubt the novelty of the paper. 

The earliest work to the best of my knowledge is [1], and later on there are a couple workshops [2,3] on multi-view learning which largely settled the field of learning from multiple views from neural networks', kernels', and bayesian perspectives. Many things mentioned in this paper have already been discovered at that time. 

3. The theoretical justification is not as strong as the generalised CCA.

CCA has been applied in the field of multi-view learning and self-supervised learning for long, and it was initially proposed for comparing the correlation between two sets of samples of two random variables. A successful generalisation is the generalised CCA which is capable of learning from multiple views. The formula of GCCA as referred in [4] is simple and elegant, and then the extension of using neural networks is also straightforward. Since people has relatively clearer understanding of CCA itself, the generalised version or the kernel version of it is also well-understood. 

A nice theoretical understanding of contrastive unsupervised learning is provided in [5], and I recommend the authors to study.


[1] de Sa, Virginia R. "Learning classification with unlabeled data." Advances in neural information processing systems. 1994.
[2] ICML Workshop, "Learning With Multiple Views". 2005
[3] NIPS workshop, "Learning from multiple sources". 2008
[4] Benton, Adrian, et al. "Deep generalized canonical correlation analysis." ICLR workshop 2017.
[5] Arora, Sanjeev, et al. "A theoretical analysis of contrastive unsupervised representation learning." ICML 2019.
review_title: Official Blind Review #3

Meta-review: This paper proposes to use contrastive predictive coding for self-supervised learning.  The proposed approach is shown empirically to be  more effective than existing self-supervised learning algorithms.  While the reviewers found the experimental results encouraging, there were some questions about the contribution as a whole, in particular the lack of theoretical justification.
Decision: Reject