Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper proposes a ’Cross View training’ approach to semi-supervised learning. In the teacher-student framework for semi-supervised learning, it introduces a new cross view consistency loss that includes auxiliary softmax layers (linear layers followed by softmax) on lower levels of the student model. The auxiliary softmax layers take different views of the input for prediction.

Pros:
1. A simple approach to encourage better representations learned from unlabeled examples. 

2. Experiments are comprehensive.

Cons:

0. The whole paper just presented strategies and empirical results. There are no discussions of insights and why the proposed strategy work, for what cases it will work, and for what cases it will not work? Why? 

1. The addition of auxiliary layers improves Sequence Tagging results marginally. 

2. The claim of cross-view for sequence tagging setting is problematic. Because the task is per-position tagging, those added signals are essentially not part of the examples, but the signals of its neighbors. 

3. Adding n^2 linear layers for image classification essentially makes the model much larger. It is unfair to compare to the baseline models with much fewer parameters. 

4. The "CVT, no noise" should be compared to "CVT, random noise", then to "CVT, adversarial noise". The current results show that the improvements are mostly from VAT, instead of CVT. 



review_title: Experimental results are not impressive

final_score: rating: 2: Strong rejection
review_content: This paper presents a so-called cross-view training for semi-supervised deep models. Experiments were conducted on various data sets and experimental results were reported.

Pros:
* Studying semi-supervised learning techniques for deep models is of practical significance.

Cons:
* The novelty of this paper is marginal. The use of unlabeled data is in fact a self-training process. Leveraging the sub-regions of the image to improve performance is not new and has been widely-studied in image classification and retrieval. 
* The proposed approach suffers from a technical weakness or flaw. For the self-labeled data, the prediction of each view is enforced to be same as the assigned self-labeling. However, since each view related to a sub-region of the image (especially when the model is not so deep), it is less likely for this region to contain the representation of the concepts (e.g., some local region of an image with a horse may exhibit only grass); enforcing the prediction of this view to be the same self-labeled concepts (e.g,“horse”) may drive the prediction away from what it should be ( e..g, it will make the network to predict grass as horse). Such a flaw may affect the final performance of the proposed approach.
* The word “view” in this paper is misleading. The “view” in this paper is corresponding to actually sub-regions in the images
* The experimental results indicate that the proposed approach fails to perform better than the compared baselines in table 2, which reduces the practical significance of the proposed approach. 

review_title: This paper presents a so-called cross-view training for semi-supervised deep models. This paper suffers from several weaknesses, e.g., lack of novelty, technical flaw and no significant improvement over the existing approaches.

final_score: rating: 7: Good paper, accept
review_content: This paper proposes a multi-view semi-supervised method. For the unlabelled data, a single input (e.g., a picture) is partitioned into k new inputs permitting overlap. Then a new objective is to obtain k predictions as close as possible to the prediction from the model learned from mere labeled data.

To be more precise, as seen from the last formula in section 3.1, the most important factor is the D function (or KL distance used here). As the author said, we could set the noisy parameter in the first part to zero, but have to leave this parameter non-zero in the second term. Otherwise, the model can't learn anything.

My understanding is that the key factor is not the so called k views (as in the first sight, this method resembles conventional ensemble learning very much), but the smoothing distribution around some input x (consistency related loss). In another word, we set the k for unlabeled data as 1, but use unlabeled data k times in the scale (assuming no duplicate unlabeled data), keeping the same training (consistency objective) method, would this new method obtain a similar performance? If my understanding is correct, the authors should further discuss the key novelty compared to the previous work stated in the second paragraph of section 1. One obvious merit is that the unlabeled data is utilized more efficiently, k times better.



review_title: This paper proposes a multi-view semi-supervised method

Meta-review: This paper combines ideas from student-teacher training and multi-view learning in a simple but clever way.  There is not much novelty in the methods, but promising results are given across several tasks, including realistic NLP tasks.  The improvements are not huge but are consistent.  Considering the limited novelty, the paper should include some more convincing analysis and insight on why/when the approach works. Given the intersting results, the committee recommends this for workshop track.
Decision: Invite to Workshop Track