Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper improves on the idea of mixup by selecting samples for mixup via a model that selects suitable pairs found using knn in a batch. In order to provide a learning signal to this discrete process, the authors apply REINFORCE, using the loss of the downstream regressor (not applied to classification tasks). They show promising results on a number of regression tasks.
main_review: Strengths:
The idea is simple and appears to be implemented correctly, and the experiments appear to be justifiable given mixup is a useful regularization technique for supervised learning, and has been shown to be very useful in classification tasks. The application of REINFORCE as a stochastic gradient approximator is well known in other context that involve NNs with discrete processes (see DLGMs, e.g., Rezende 2014). The experiments are straightforward and support the value of their work.

Weaknesses:
I have deep concerns about this paper being marketed as "RL for mixup". RL comes packaged with a great deal more than just the REINFORCE algorithm, e.g., the policy gradient, and rephrasing a lot of what is a gradient estimator for a discrete process as "RL" not only may potentially confuses the reader on what RL is, but also doesn't connect with an active area of research on doing backprop through discrete processes. Yes, I understand that the cited Yoon 2020 paper did this as well, but I don't think they should have either. One of the most famous examples of using REINFORCE is with discrete VAEs or DLGMs (Rezende 2014), and they are careful not to call it "RL"

But there are a number of other works or baselines that this paper should have compared to, and I think in part this is because the related works talk about other RL algorithms: for instance A3C or even PPO aren't really relatable to the problem being addressed here as straight-through estimators (Bengio 2013) or Gumbel Softmax (Jang 2017) (Yoon does a better job at connecting to these types of works). 

For instance, it seems possible to backprop through to the selection probabilities using Gumbel Softmax. Variance is usually much better than REINFORCE, even with the baseline removal.

These are important baselines and it is important this paper places itself correctly among related works.

Next, there are many claims about why classification doesn't need this sort of mixup, but I didn't follow the arguments entirely (P1-P2 in Section 2). I don't see any classification experiments to back up this claim, at least in the main text.  For instance "Linear assumption (in classification) turns out to be "reasonable" because the label difference..." Why is it reasonable to leave the label space in classification mixup: with regression I have a better chance of hitting a "label" in the distribution.

Other notes / concerns:
So the motivation was on some physical systems, and we're using REINFORCE to learn how to mix data. But if we're in the physical systems such as you describe, usually we have some sort of (closed-form solution) model that describes the physical process (maybe poorly). Even if this model has error, wouldn't this be a good place to design a prior that tells you how to sample? e.g., if the masses are some set distance apart, then use, else don't.

I see meta-learning as a description of the model, but not in the algorithm 1.

How do you sample k when you decide how many neighbors to mix? Is this sampled from a prior?

Rezende 2014: Stochastic Backpropagation and Approximate Inference in Deep Generative Models
Bengio 2013: Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Jang 2017: Categorical Reparameterization with Gumbel-Softmax
summary_of_the_review: I recommend reject, as the paper's core story isn't well placed compared to related works. The paper is placed as "RL" so we're missing important baselines and connections to other gradient estimators through discrete processes.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose MixRL to improve upon mixup in regression settings. MixRL is used to impose a proximity constraint on the input/output pairs that are mixed during mixup-based data augmentation, by predicting how many nearest neighbors to utilize, from a small set of pre-specified options, based on feedback from evaluating the validation set. Consistent but small gains over mixup and manifold mixup are realized on several datasets.
main_review: Strengths:

-Data-dependent input/output proximity constraints on mixup are learned.

-Consistent but small gains over mixup and manifold mixup are realized on several regression datasets.

Limitations:

-The set of nearest neighbors considered for use in mixup, while dynamic, are highly restricted (e.g. {4, 16, 64, 128} for the NO2 dataset in table 3.

-More generally, the lack of a local input/output kernel to prioritize more local data in a continuous manner feels like a significant limitation.

-Optimized input/output kernels that restrict and more generally re-weight mixup pairs or construct locally non-constant regressions to sample from feel like important, missing baselines. The lack of a locally non-constant regression model in MixRL (beyond mixing data pairs) is likely limiting performance significantly.
summary_of_the_review: The technique while sound is currently quite limited in scope in that important baselines and algorithmic elements (e.g. locally non-constant models and local kernels) that need to be considered in the context of doing high fidelity data augmentation for regression problems have not been adequately investigated or discussed.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: To apply Mixup for regression tasks, the paper first utilizes the stricter assumption that linearity only holds within specific data or label distances for regression.
Then this paper proposes a data mixing augmentation method called MixRL. The goal of MixRL is to identify which examples to mix with which nearest neighbors. MixRL employs a meta-learning framework that estimates how important mixing a sample is to minimize the model loss on a validation set using policy gradient reinforcement learning.
main_review: MixRL is inspired by the problem of measuring how individual examples contribute to model performance. The idea is reasonable for regression tasks.
The experiments verify its effectiveness by improving regression performance by carefully mixing examples.
The paper is well written and organized.

I notice MixRL cannot improve previous Mixup methods by a clear margin. Since I am hesitant about the importance of the improvement, I want to hear about other reviewers’ opinions to make the decision.
summary_of_the_review: I'd like to see the other reviewers' opinions about the empirical results.
review_title: null

Meta-review: This paper generalize the idea of Mixup-based data augmentation for regression. Compared to classification for which Mixup was used, the paper argues that in regression the linearity assumption only holds within specific data or label distances. The paper thus proposes MixRL to select suitable pairs using k-nearest neighbor in a batch for mixup. The selection policy is trained with meta-learning by minimizing the validation-set loss. The approach provides consistent but small improvement over mixup on several datasets. Reviewers have also suggested discussion and comparison with more baselines, such as respective method using other (lower-variant) gradient estimators (e.g., gumbel-softmax), and using local input/output kernels for data selection, etc.
Decision: Reject