Venue: ICLR 2018 conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: (Summary)
This paper is about learning a representation with curriculum learning style minibatch selection in an end-to-end framework. The authors experiment the classification accuracy on MNIST, FashionMNIST, and CIFAR-10 datasets.

(Pros)
The references to the deep metric learning methods seem up to date and nicely summarizes the recent literatures.

(Cons)
1. The method lacks algorithmic novelty and the exposition of the method severely inhibits the reader from understand the proposed idea. Essentially, the method is described in section 3. First of all, it's not clear what the actual loss the authors are trying to minimize. Also, \min_v E(\theta, v; \lambda, \gamma) is incorrect. It looks to me like it should be E \ell (...) where \ell is the loss function. 

2. The experiments show almost no discernable practical gains over 'random' baseline which is the baseline for random minibatch selection.

(Assessment)
Clear rejection. The method is poorly written, severely lacks algorithmic novelty, and the proposed approach shows no empirical gains over random mini batch sampling.
review_title: Review 

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors purpose a method for creating mini batches for a student network by using a second learned representation space to dynamically selecting  examples by their 'easiness and true diverseness'. The framework is detailed and results on MNIST, cifar10 and fashion-MNIST are presented. The work presented is novel but there are some notable omissions: 
 - there are no specific numbers presented to back up the improvement claims; graphs are presented but not specific numeric results
- there is limited discussion of the computational cost of the framework presented 
- there is no comparison to a baseline in which the additional learning cycles used for learning the embedding are used for training the student model.
- only small data sets are evaluated. This is unfortunate because if there are to be large gains from this approach, it seems that they are more likely to be found in the domain of large scale problems, than toy data sets like mnist. 

**edit
In light of the changes made, and in particular the performance gains achieved on CIFAR-100, i have increased my ratting from a 4 to a 6
review_title: Review

final_score: rating: 4: Ok but not good enough - rejection
review_content: While the idea is novel and I do agree that I have not seen other works along these lines there are a few things that are missing and hinder this paper significantly.

1. There are no quantitative numbers in terms of accuracy improvements, overhead in computation in having two networks.
2. The experiments are still at the toy level, the authors can tackle more challenging datasets where sampling goes from easy to hard examples like birdsnap. MNIST, FashionMNIST and CIFAR-10 are all small datasets where the true utility of sampling is not realized. Authors should be motivated to run the large scale experiments.


review_title: The authors propose a method that uses an embedding network trained with magnet loss for adaptively sampling and feeding the student network that is being trained for the actual task

Meta-review: Although paper has been improved with new quantitative results and additional clarity, the reviewers agree though that larger-scale experiments would better highlight the utility of the method. There are some concerns with computational cost, despite the fact that the two networks are trained asynchronously. A baseline against a single, asynchronously trained network (multiple GPUs) would help strengthen this point. Some reviewers expressed concerns with novelty.
Decision: Reject