Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper proposes a new approach to improve the performance of non-linear predictors (DNN models) on binary labels when the observations are endogenously selected. The issue arising in traditional setups is that the algorithm will only select observations with high positive outcome (reward). This may lead to sub-optimal and unfair models generating a large proportion of false negatives. The solution proposed in this paper is to add optimistic labels (with positive rewards) to a subset of points that the model would normally reject. Doing so optimistically bias the model, and stimulate the exploration of the more uncertain regions. The suggested algorithm and paper focus is on online learning methods. Several experiments highlight the competitiveness of this approach compared to similar ones.
main_review: * The main contribution of the paper appears to be described in Section 5, which is on pages 6-7. The previous three sections (setting, related work, and background) occupy a lot of space without necessary conveying crucial information about this work. The paper would benefit from some restructuring.
* There is a large literature on self-selection bias beyond the online learning literature which is nowhere discussed.
* Being familiar but not a specialist in online learning (OL), I have the following questions about the approach:
  * OL is typically applied to problems with high data velocity, when model re-training should be made online because a batch approach would be too costly and/or slow. The examples discussed do not convey the necessity of online updates. Similarly, there is no adversarial game-like sequential interaction. Why would one prefer OL as opposed to, for example, an offline approach that would look at the entire historical sample to quantify model uncertainty and then setup an exploration strategy for the next $n$ observations.
  * It seems that the DNN is fully recalibrated after a new data batch is created (e.g. Algorithm 2). Please comment.
* The numerical experiments in section 6 illustrate that the approach is in line with the selected OL benchmarks, but do not highlight a noticeable performance gain or computational advantage.
limitations_and_societal_impact: * The approach limitations regarding the data independence and separability should be further discussed. Observations and labels in a bank-loan problem would typically be exposed to correlated and time-varying factors depending, for example, on the current health of the economy.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: Authors propose neural pseudo-label optimism for solving the bank loan problem. This is an interesting setting and I judge the work in the paper (theory & experiments) to be mostly correct. I consider this paper to be sufficiently novel, although there are certainly a large number of papers on the bank loan problem. I am not an expert on the bank loan problem in particular, so I hope another review can provide more clarity as to this paper’s novelty. I am tentatively voting for a rating of 7, but I would like the authors to address some of my concerns below.
main_review: Pros:
- Interesting topic & sufficient novelty (as far as I am aware)
- Complementary experiments & theory
- Acceptable writing quality

Cons:
- Some parts of the theory are slightly contrived (but this could be fixed)
- Experiments could use additional details and polish

I strongly suggest the authors rewrite the “neural realizability” aspect of their theoretical framework (or at least provide more details). I understand the motivation behind it, but it seems like a bit of a stretch. For example, consider Assumption 2. Based on the universal approximation property of DNNs, I don’t doubt Assumption 2, but in practice we have no way of knowing whether or not the architecture of choice actually meets this Assumption (at least not before training on a similar dataset). The authors could go a similar route using something like NTK theory which is the limit of wide-networks and is theoretically sound in terms of training and generalization. I understand there is work show a wider class of overparameterized networks can interpolate data due to local convexity, but still this thread of research is not mentioned either. 

Related to the above, I am confused if Theorem 2 relies on the assumption of a convex loss function or also convexity of the loss with respect to the model parameters. The authors should clarify these details or at least include a reference to the supplement in which this is discussed.

EXPERIMENTS:
 - Most details are present but the description could use a bit more clarity. The code is provided in the supplement. I did not attempt to run it.
- Need more details concerning the DNN architectures (unless these are in the supplement and I just missed it)
- For the Bank dataset (which is the most interesting for this topic) does not seem to have major improvement between the proposed method and the baselines. The authors might rescale the y-axis so that we can see the differentiation between the curves for the bank dataset
- Error bars/regions are barely visible
- 3 datasets are ok but MNIST in particular does not seem like a great candidate for this topic. I like the use of “Adult” and “Bank” and encourage the authors to move MNIST to the supplement in the place of 1 or 2 more “realistic” datasets.


Minor:
-typo in line 57: “bey”

limitations_and_societal_impact: Sure
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper first motivates the "bank loan problem", which is a scenario where the learner only observes whether a customer will repay a loan if the loan is issued to begin with. The paper moves on to discussions of this problem and introduces contextual linear bandit formulations. This is extended to the case where the model is a deep neural network, and a Theorem shows theoretical guarantees under the case where data is separable. Experiments show improvement over baseline methods, including the recent NeuralUCB method.
main_review: ### Strengths

- The paper focuses on an important problem characterized as the "bank loan problem" and provides a simple way to use neural networks for this problem.
- Experiments show improvement over baselines.
- Code is provided in the supplementary materials and should be helpful for the research community to reproduce results.

### Weaknesses and questions

- Some of the equations seem to be weird. For example, in the equations after line 225 and 232, the first term has $\log f(x)$, but should this be $\log \mu (f(x))$ since $f:\mathbb{R}^d \rightarrow \mathbb{R}$?
- Notations are confusing. $\mathcal{L}^\lambda(\bf{\theta}|\mathcal{D}_t)$ are used as the ordinary cross entropy loss in Appendix C but is used as the negative cross entropy loss after line 225.
- Line 271 points to Appendix E but I couldn't find Appendix E in the supplementary material. 
- In the code, I can see that the neural network is a fully-connected 1 hidden layer neural network. There may be a space issue, but I believe this is basic info and should be included in the main paper.
- In Theorem 1: Is $1-2\delta$ a valid probability when $\delta \in (0,1)$?

#### Minor suggestions
- It would be interesting to see experiments where the separability is violated and how it affects the results.
- line 277: Should $R^d$ be $\mathbb{R}^d$?
- Eq. after line 161: Are the parenthesis located in wrong places?

---

Thank you for the response. I found Appendix C corresponds to the proof of Theorem 2, instead of Appendix E. The author responses in the other reviews about the details of this proof was also helpful. Since my concerns were addressed, I would like to raise my score to 6.
limitations_and_societal_impact: yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper studies class of problems, called bank loan problem (BLP), where the learner only observes whether a customer will repay a loan if the loan is issued to begin with.
To solve this problem, the paper proposes a simple and efficient algorithm, Pseudo-Labels for Optimism (PLOT), which adds an optimistic label to a subset of data points rejected by the model and trains the model on all the data seen so far including the optimistically added points. Empirical evidence shows that PLOT achieves competitive performance in comparison to known storng baselines such as NeuralUCB.
main_review: 
Strengths:
---------------

- The proposed algorithm PLOT, for bank loan problem, is a simple and intuitive algorithm to enable optimism in the bank loan problem.

- Theorem 1, provides regret analysis for the linear logistic models.

- Theorem 2, states that under some assumptions, PLOT converges to a classifier with perfect accuracy.


Weakness:
---------------

- Some of the assumptions are hard to justify in the Theorem 2 (see Assumptions 4 and 5 in the appendix). I would appreciate any intuition as to why the authors feel this connectedness assumption would be satisfied in the neural setting.

- Proof of the Theorem 2 in the appendix feels more like sketch than any rigorous mathematical argument. 

Questions for Authors:
---------------
- What is Algorithm 3 on line 270?

- Your proof of Theorem 2, requires additional assumptions (4, and 5 listed in the appendix). Is there a reason why these assumptions would be satisfied in a Neural setting?

- The bank loan problem is also related to the Selective Classification / Learning with abstention literature, where a network has an extra option to either predict on a new example or reject/abstain the example (implying that the learner is not confident in its prediction on this new example). In this literature, a very simple and strong baseline is thresholding based on the entropy of the prediction ( entropy of the prediction is something your counterfactual loss anyways incorporates). This could be a simple and strong baseline in this problem as well.

- Can you confirm if the dependence on d in the Theorem 1 is linear or \sqrt{d}? The proof in the appendix seems to suggest otherwise.

Writing Clarity:
---------------
Paper is easy to follow except for some of the typos listed below and the proof of Theorem 2 in the appendix.

Line 57: model bey training..
Line 92: the learner has decided to accept or not datapoint xt....
Line 515 (appendix): lemmaconfidenceboundrussacappendix ..



limitations_and_societal_impact: N/A
review_title: null

Meta-review: The scores from four reviews are 4, 5, 5, and 7. The authors have a persuasive reply to the review with the lowest score, so this score is too harsh. As the authors say, the criticisms in one of the reviews with a 5 score are rather superficial. Overall, the paper is on an important topic and is correct and novel, so I lean towards publication.

One issue not mentioned in the paper is delayed feedback. When a bank grants a loan, a lot of time passes before the true label becomes available, and during this time the function to be learned p(y|x) may change. A related issue mentioned by a reviewer is correlation between labels of different examples, as caused for example by a recession. The last section should discuss these issues briefly, and also discuss fairness, as opposed to merely mentioning it. Lack of exploration can lead to never giving loans to members of a group that has historically faced prejudice. Of course, the three issues just mentioned apply more broadly than just specifically to granting of credit.
Decision: Accept (Poster)