Paper:
Recent semi-supervised learning (SSL) methods often have a teacher to train a student in order to propagate labels from labeled data to unlabeled data. We argue that a weakness of these methods is that the teacher does not learn from the student's mistakes during the course of student's learning. To address this weakness, we introduce Coaching, a framework where a teacher generates pseudo labels for unlabeled data, from which a student will learn and the student's performance on labeled data will be used as reward to train the teacher using policy gradient. Our experiments show that Coaching significantly improves over state-of-the-art SSL baselines. For instance, on CIFAR-10, with only 4,000 labeled examples, a WideResNet-28-2 trained by Coaching achieves 96.11% accuracy, which is better than 94.9% achieved by the same architecture trained with 45,000 labeled. On ImageNet with 10% labeled examples, Coaching trains a ResNet-50 to 72.94% top-1 accuracy, comfortably outperforming the existing state-of-the-art by more than 4%. Coaching also scales successfully to the high data regime with full ImageNet. Specifically, with additional 9 million unlabeled images from OpenImages, Coaching trains a ResNet-50 to 82.34% top-1 accuracy, setting a new state-of-the-art for the architecture on ImageNet without using extra labeled data.

Review:
This paper provides a simple but novel coaching method for teacher-student based semi-supervised learning framework. The coaching method consists of a two-stage update: first update Student network according to the pseudo label produced by Teacher network on the unlabeled dataset; second update Teacher network according to the Student network's performance on the labeled dataset. The authors propose a novel policy gradient update for the Teacher network. The authors evaluate the coaching method on several different semi-supervised learning dataset CIFAR-10, SVHN and ImageNet. The comparison with baselines are thorough. I appreciate that the authors explain the design of the experiments and tuning in details.
I vote for acceptance but I still have some questions. I would be willing to increase my score if the authors address my questions in the rebuttal.
1. The motivation of coaching is not accurate. In the second sentence of Introduction, the authors mention "Although coaches do not play as well as the players". However, we are training neural networks. There is no such thing as "coaches do not play as well as the players". They are the same parametric neural networks. (The authors also use the same architectures for both teacher and student neural networks.) I would remove this analogy sentence.
2. The authors mention that their coaching method beats the fully supervised learning on the CIFAR-10. I am not convinced by this result. The author explained that this is due to the less overfitting in the student network. However, besides coaching, there are many other regularization method we can use to avoid overfitting. Using far less labels in training strictly reduces the amount of information we have. There is a simple test the authors can do. We can use the full labelled data set and use sampled results from Teacher network to train a Student network on it. We can perform coaching on this regime. This coaching on full dataset should do better than coaching on partially-labeled dataset (4000 labels).
3. The state of art for CIFAR-10 is 99% now [1]. It would be nice to see the performance of author's approach applies to the state-of-art network/structure.
References:
[1] Huang, Yanping, et al. "Gpipe: Efficient training of giant neural networks using pipeline parallelism." arXiv preprint arXiv:1811.06965 (2018).
-----
The authors' response does not answer the questions about motivation and why the method works, which is also questioned by the two other reviewers. For example, why the authors need to have two networks is unclear; I guess that true labels should be more helpful to guide a student network to learn than a teacher network. Even though the authors seem to have state-of-art semi-supervised learning results, some extra explanations are needed.