Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: Summary: This paper introduces a simple idea to optimize the weights of a weighted empirical training distributions. The goal is to optimize the population risk, and the idea is to optimize a distribution over the training examples to maximize the cosine similarity between training set gradients and validation set gradients. The distribution over the training set is parameterized by a neural network taking as arguments the

Strengths:
- The method is quite simple.
- The results appear to be strong, although I am less familiar with the NMT baselines. The imagenet results seem quite strong to me.

Weaknesses:
- I couldn't find a particularly clear description of the scoring networks architecture. Given that it observes the whole dataset, this seems like a critical choice that could have a big impact on the complexity of this approach. At the very least, this should be clearly reported, and I recommend a more thorough investigation of this choice.
- The authors report that their method takes 1.5x to 2x longer to run than the uniform baseline. Yet, they ran all methods for the same number of steps / epochs. It seems to me that a fairer comparison might be letting all methods enjoy the same total budget measure roughly by wall time.

Questions:
- I didn't follow why the computation of the per example gradient grad l(x_i, y_i, theta_t-1) is so onerous. Isn't that computed on line 5 already?
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper presents a reinforcement learning approach towards using data that present best correlation with a validation set’s gradient signal. The broader point of this paper is that there is inevitably some distribution shift going from train to test set - and the validation set can be a small curated set whose distribution is closer to the testing distribution than what the training dataset's distribution is. 

The problem setup bears relationship to several areas including domain adaptation/covariate shift problems, curriculum learning based approaches amongst others. One assumption that I see which needs to be understood more is equation (6) - wherein, somehow, there is a Markov assumption used to zero out the contribution of the scoring network on parameters unto previous time step. Trying to understand the implications of this assumption (how the performance varies with/without this assumption) would be instructive for understanding potential shortcomings of this framework.

I think the paper is well written, handles an important question. That said, I am not too aware of recent work in this area to make a decisive judgement on this paper’s novelty/contributions. 
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: The paper proposes an iterative method that jointly trains the model and a scorer network that places a non-uniform distribution over data sets.  The paper proposes a gradient method to learn the scorer network based on reinforcement learning, which is novel as to what the reviewer knows.

There are several concerns/questions:

1) The paper doesn’t define the D_{dev} clearly. How is D_{dev} chosen? Is it a subset of D_{train}? 

2) In section 2.1, why “smaller development set D_{dev} is much closer to the P_{test}(X,Y)”? P_{test}(X,Y) is supposed to be not observed during training?

3) In Eq (5), if D_{dev} is s subset of D_{train}, if \theta* is the minimal of J, it means the gradient 
at  \theta* is 0. To calculate the gradient of J with respect to \psi, by chain rule, it need to calculate gradient to \theta* first then \theta* to \psi. If gradient of \theta* is 0, the product is also 0? So the \psi will not be updated if D_{dev}  is sufficiently similar to D_{train} ?

4) In Section 2.3, it omits the second order Hessian term. How does that influence the performance? 

5) it mentions “without significant computing overhead“ in abstract, which is not demonstrated elsewhere.

6) In the experiments, table 1, it seems the major improvement comes from retrain and TCS rather than DDS? In figure 3, it is better to show the weights of an image without DDS and comparing that with DDS.

7) The paper contains many typos such as Eqn.11 is not defined in the main paper, the “Eqn ??” Appears in the appendix, “tha minimizes” etc.

In general, the idea of the paper is natural and the results seem promising. I am looking forward to the reply to my questions/concerns. 

#############

I have read the author's feedback. I think the clarity of both methodology and experiment does not reach the acceptance level and would maintain my current rating. 

review_title: Official Blind Review #1

Meta-review: The paper proposes an iterative learning method that jointly trains both a model and a scorer network that places a non-uniform weights on data points, which estimates the importance of each data point for training.  This leads to significant improvement on several benchmarks.  The reviewers mostly agreed that the approach is novel and that the benchmark results were impressive, especially on Imagenet.  There were both clarity issues about methodology and experiments, as well as concerns about several technical issues.  The reviewers felt that the rebuttal resolved the majority of minor technical issues, but did not sufficiently clarify the more significant methodological concerns. Thus, I recommend rejection at this time.
Decision: Reject