Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies a class of problems known as the Bank Loan Problem (BLP), where the learner only observes whether a customer will repay a loan if the loan is accepted. The labeled training data in this problem is biased since it is affected by previous decisions. The authors propose adversarial optimism (AdOpt) to address this problem utilizing adversarial domain adaptation. First, they train a debiased classifier using adversarial training and incorporate this classifier with the pseudo-label optimism (PLOT) method to increase the rate of correct decisions. The numerical results demonstrate the advantages of AdOpt in the standard BLP benchmark.

strength_and_weaknesses: Strengths:
-The paper concentrates on an interesting and significant problem characterized as the Bank Loan Problem.
-The paper is easy to follow.

Weaknesses:
1. The use of adversarial domain adaptation in the paper is not conceivable. The adversarial domain adaptation aims to equalize the probability of 1s from both domains (Equation 5). However, it inherently contradicts the PLOT method (Step 5, Algorithm 1).
2. It is unclear how the debiased classifier's performance impacts the learners' final performance compared to the previous method (PLOT). It would be valuable if the authors could provide a visualization of the proposed method (see Figure 2 in the PLOT paper).
3. The proposed approach requires training an additional debiased classifier in comparison to PLOT. However, I am not convinced that the experimental results in Figure 2 represent a noticeable improvement in the learners' performance over PLOT (see results in Bank, Crime, and German datasets).
4. The paper argues that using the debiased classifier for assigning pseudo-labels enables AdOpt to explore faster. Why do the authors not consider other RL methods(such as Double Deep-Q) as the baselines?
5. Why is MNIST selected as the comparative dataset while the paper examines the Bank Loan Problem?

clarity,_quality,_novelty_and_reproducibility: I find that the paper is easy to follow. 

The submission includes python codes but due to time constraints, I could not verify whether the results are reproducible.


Minor comments:
-Section 1, Paragraph 8: mitigate - remove “for”
-Section 2, Paragraph 6: remove “in”
-Section 4, Paragraph 1: lables -> labels
-Section 4, Paragraph 3: approximating -> approximate
-Section 4.1, Paragraph 1: posses -> possess
-Section 4.2, Paragraph 2: picking -> pick
-Equation 3, Line 1: (\x_0, 1)
-Equation 4: not specify the set of x for the third term

summary_of_the_review: Overall, I find the paper interesting, and I appreciate this approach to solving an important but often neglected problem in ML.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: PLOT [Pacchiano 2021] leverages a adversarial-training-like scheme to train NN for online binary classification problem: at each time step, retrain the model M while pretending that the new data are of the positive class, and give prediction on the new data using the retrained model M.

This paper proposes to add a second classifier C to this training process. C is trained at each time step using adversarial domain adaptation; it classifies the embedding of the data. Now, only those that are rejected by M and accepted by C enters the PLOT training for this time step.


strength_and_weaknesses: === Strength ===

According to the authors and the best of my knowledge, this paper is the first attempt to utilize adversarial domain adaptation to tackle bias in online learning. The paper is well-written and easy to follow. 

=== Weaknesses ===

The paper refers to G as a de-biased generator and C a de-biased classifier, but there is no discussion in 3.3 on what "de-biased" means, or how exactly adversarial domain adaptation achieves it. 

If the role of the de-biased classifier, as expected, turns out to be trading off precision and recall (compared to the biased classifier), why don't we just do thresholding on the biased classifier? That is, go with PLOT, but simply choose a good threshold (<.5) in the biased classifier to select nodes to assign pseudo-labels. In any case, a precision-recall-curve comparison is necessary to establish that the de-biased classifier is trading off precision and recall at a larger area-under-the-precision-recall-curve (compared to the biased classifier) in the threshold region of interest, but this is missing from the paper.

The paper also lacks an evaluation of the de-biased classifier: how unbiased is it actually, over the time steps?
clarity,_quality,_novelty_and_reproducibility: Clarity and Quality: The writing is excellent and easy to follow. The literature review is cut short compared to PLOT. I personally believe that this paper can attract a larger audience by including the discussion on its parallel to strategic classification and performative prediction.

Novelty: Half of the paper (adversarial domain adaptation) is new contribution compared to PLOT.

Reproducibility: I briefly read the code. It appears to be well-written and documented.
summary_of_the_review: The paper is well-written, the contribution is non-trivial, and should be of interest to a broad online learning / strategic learning / domain adaptation audience. However, due to missing important ablation studies, I recommend reject.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper studies the bank-loan problem: a binary classification setting where the learner only receives feedback when it assigns an example the positive label. In this setting, training on a dataset consisting only of the accepted examples naturally leads to bias, since the learner has no ability to correct mislabelled rejections. A recent work addresses this by adding rejected samples to the dataset with an optimistic (positive) label. This naturally leads to a higher false acceptance rate. To help alleviate this issue, this paper proposes modifying this approach by using a second classifier with high recall to decide which points to add to the dataset, so that fewer false acceptances make their way into the dataset. This high-recall classifier is trained using an existing adversarial domain adaptation technique.
strength_and_weaknesses: **Strengths** The core idea seems like a reasonable way to to address a shortcoming of an existing method

**Weaknesses** I found the paper hard to follow in general. It felt like the relevant background explaining PLOT and where the domain adaptation technique fits into the picture was scattered around the paper in an unintuitive way;  I had to read some of the sections several times and out of order to understand what was being proposed. Essentially, the difference between the proposed algorithm (AdaOpt) and PLOT is just that AdaOpt uses a different mechanism to decide whether to include a rejected sample in the dataset. 

The idea of using a high-recall classifier to decide which examples to optimistically add to the dataset to help mitigate high false acceptance rate of PLOT seems reasonable. However, as far as I can tell there are no concrete performance guarantees here, so the only way we can evaluate the proposed approach is via the experimental results. Unfortunately, I don't think that the results manage to demonstrate the claims of the paper. Since AdaOpt is a modification of PLOT, the main baseline to outperform is PLOT, yet Figure 2 implies that there is no statistically significant difference between PLOT and AdaOpt on any of the datasets except for "Adult", wherein AdaOpt finishes with slightly lower regret in the end. 

**Misc** I think Perdomo et. al. (2020) is an relevant citation to discuss in this paper, as it studies risk minimization in settings where the decision-making policy effects the training data distribution, as it does in the bank loan problem.


References
---
- Perdomo, J., Zrnic, T., Mendler-D\"unner, Celestine, & Hardt, M. (2020).
  Performative Prediction. In H. D. III, & A. Singh, Proceedings of the 37th
  International Conference on Machine Learning (pp. 7599–7609). : PMLR.

clarity,_quality,_novelty_and_reproducibility: As noted above, I think the writing could use some work to be more clear. The approach is a modification of an existing approach, PLOT, but uses a novel mechanism to decide which rejected samples will be added to the dataset with an optimistic pseudolabel. The adversarial domain adaptation approach used to train the classifier used by this mechanism is also borrowed from existing works, though hasn't been used in this setting before. The works seems sufficiently reproducible, assuming the provided code is released.
summary_of_the_review: Overall the paper proposes a reasonable solution to an issue with a state-of-the-art method for this setting, but fails to provide convincing evidence of improvement over the algorithm it is based on (in theory nor experimentally), and so do not recommend this paper for acceptance.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper studies the bank-loan problem, a binary online problem where the learner only receive positive or negative reward when it classifies the sample as positive and 0 reward when it classifies it as negative. The aim of the learner is the maximize the reward obtained.

While the problem can be modeled as a contextual bandit problem, there is a line of recent papers proposing heuristic solutions based on deep learning. This submission falls in this category presenting a heuristic method that builds on top of Pacchiano et al. (2021). The scientific claim of the submission is that the proposed algorithms improves previous work. This claim is validated through an empirical evaluation over 5 datasets.

This submission has been thoroughly discussed by the reviewers and the authors, without reaching an agreement on the significance of the results. None of the reviewers fully accepted this paper, but the discussion focused on the statistical significance of the gains over the previous algorithms.
On one hand, it is (almost*) safe to claim that the t-values calculated by the authors show a statistical significant result on 2 datasets out of 5, while the results on the other 3 are less clear. However, there is a difference between "statistically significant results" and the "significance" that the reviewers are asked to evaluate in their forms. A result can be statistically significant, as in the results in this paper, yet not significative enough to warrant publication in a selective venue as ICLR. In this view, unfortunately I agree with Reviewer cz4a who points out that the gains over the other algorithms might be too small to be interesting for this community. Note that even other communities would have the same standard distinguishing the statistical significance from the size of the effect. Moreover, the incremental nature of the proposed solution and the lack of any theoretical justification make the contribution even smaller. For these reasons, the paper cannot be accepted in the current form at ICLR.

* the "almost" above refers to the fact that comparing multiple algorithms on multiple datasets is a tricky business. I would encourage the authors to read Demšar, "Statistical Comparisons of Classifiers over Multiple Data Sets", JMLR 2006 for an actual principled way to do it.
justification_for_why_not_higher_score: The proposed method is incremental with respect to prior work and it only shows small gains over previous baselines.
Decision: Reject