Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors address the problem that models trained for the same purpose may achieve similar accuracy on consistent test data but differently on individual predictions. Such inconsistent behavior is highly problematic, especially in the medical and financial domains. To address this problem, the authors introduce a selective ensemble that mitigates such inconsistencies by applying hypothesis testing to the predictions of a set of models trained on randomly selected starting conditions. First, they proved that the inconsistency of predictions between the selective ensembles is bounded. Additionally,  they empirically showed that the selective ensembles achieve consistent predictions and feature attributions while maintaining a low abstention rate. On seven benchmark datasets, the selective ensembles have zero points with inconsistent predictions. And an abstention rate as low as 1.5%. Thus, the selective ensemble proposed by the authors may present a more reliable method for using deep models in environments where high model complexity and stability are required.
main_review: The authors address the problem, where models trained for the same purpose achieve similar accuracy on consistent test data but behave very differently in individual predictions, is highly problematic and essential in domains such as medicine and finance where a high degree of reliability and stability is required.
To address this problem, the authors introduce a selective ensemble that mitigates such inconsistencies by applying hypothesis testing to the predictions of a set of models trained with randomly selected starting conditions.
In the selective ensemble, predictions are aborted when the constituent models disagree sufficiently. To demonstrate the effectiveness of the proposed selective ensemble, we first prove theoretically that the prediction discrepancies between the selective ensembles are bounded. Furthermore, using seven benchmark datasets, we show that the selective ensemble has zero points with inconsistent predictions and a low abstention rate of 1.5%.
The proposed idea is simple but appears to work well for the problem being tackled.
summary_of_the_review: This paper addresses an important problem in machine learning that is of great interest to many researchers and practitioners. The authors propose a method called selective ensembles and evaluate its effectiveness from both theoretical and empirical perspectives to solve this problem. Experiments on benchmark data show that the proposed method seems to work well for the problem. In conclusion, we believe that this paper is worthy of acceptance.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a selective ensemble method. Given a set of models, it performs a statistical test to determine if the mode prediction was selected by a statistically significant majority of the constituent models. If the test succeeds, it returns the mode prediction, Otherwise, it abstains from prediction. the authors demonstrated the effectiveness of their method on improving classification accuracy by abstaining from those that are difficult to classify.
main_review: Although the idea is interesting and theoretically sound, it is unclear in which aspect the proposed method is superior to existing methods. 
- If it's classification accuracy, the method should be compared with such methods as hard-voting prediction, soft-voting prediction, and other basic ensemble methods that aggregate individual predictions, and also individual model's prediction and MC-dropout.
- If it's uncertainty quantification performance (or selective classification performance - trade-off between classification accuracy and abstention rate), various uncertainty quantificaiton measures can be compared, such as entropy, BALD, .... Deep ensemble can also be compared.
- Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016). Simple and scalable predictive uncertainty estimation using deep ensembles. arXiv preprint arXiv:1612.01474.

Another contribution the authors mentioned is that the proposed method achives consistent feature attribution. i'm wondering what's the practical meaning of it. how does it relate to the use of a model in classifying data?

For the proposed method, the absention rate may be adjustable by using different criteria for the statistical testing. It would be interesting if the proposed method is evaluated in terms of accuracy-rejection curve to see the trade-off between classificaiton accuracy and abstention rate)
- Nadeem, M. S. A., Zucker, J. D., & Hanczar, B. (2009, March). Accuracy-rejection curves (ARCs) for comparing classification methods with a reject option. In Machine Learning in Systems Biology (pp. 65-81). PMLR.

Minor comments
please elaborate on what "binom_p_value", "top_2" function, n_A, and n_B are in Algorithm 2 to help readers to comprehend.

summary_of_the_review: It's unclear how the proposed method is beneficial in classification problems. The authors should provide the practical meaning of their method and compare their method with some proper baselines if necessary.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper considers the problem of dealing with inconsistencies when forming (binary) classifier ensembles. The authors propose a mechanism whereby an ensemble can abstain from making a prediction based on a statistical test on the predictions from its constituents.
main_review: The topic is interesting and the algorithms nice and simple. However, the empirical section is quite difficult to read due to insufficient descriptive details in the captions, as well as out of order referencing in the main text. 

strengths:

- Theoretically sound idea which is also simple in practice.

weaknesses:

- The authors demonstrate that selective ensembles remove disagreements by contrasting with standard ensembles, but don't show accuracy comparisons against standard ensembles. Table 1 could do with comparitive numbers from a standard ensemble.
- Captions in general don't explain the plots/tables sufficiently. Table 4 in particular is uninterpretable as it is unclear what the two numbers
in the cells actuall are. This is a significant issue for interpreting the feature attribution results.
- Resampling can also involve features not just samples, like the random forest (an ensemble method) has demonstrated. This is not explored in the paper.

minor:

- I'm used to captions above tables and below figure, so I kept reading the wrong thing with consecuitive tables
- Table 3/fig 3 seem to be discussed and referenced before table 1/fig 1; maybe swap the order
- figure 4 as a bar chart is visually misleading due to overplotting of many colours; why not just plot the points with some x displacement to prevent overlaps?
summary_of_the_review: While an interesting method the lack of clarity in the empirical experiments reduces the impact. Furthermore, the method reduces disagreement by essentially avoiding the problem through abstention; it's not really demonstrated that this has a practical advantage other than through highlighting specific cases for manual intervention or inspection.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In the paper the authors proposed a selective ensemble whose constituents are classifiers with reject (abstention) option. They theoretically exposed the problem that differentiable models yielding identical predictions might remotely share identical gradients, biasing the feature attribution based on (integrated) gradient approaches. They theoretically demonstrated that the selective ensemble can contribute to model stability, and empirically showed that both the model stability and feature attribution could be improved as a result of the proposed method.
main_review: The paper motivates a very practical problem of model and feature attribution instability, and proposed a solution with a plethora of theoretical and empirical guarantees. The theoretical development is sound, and the empirical study section is comprehensive and convincing. The paper is also well written in a clear and concise manner.

The reviewer's major concerns here are based on the novelty and the feasibility of the proposed approach. It is worth pointing out that neither of these two points prevents the paper from being considered as a highly completed work, and the reviewer would gladly update the rating should the concerns be concisely addressed.

1. Regarding the novelty, it is not very surprising to see that bagging / ensembling helps with the prediction stability for classifiers, especially when equipped with reject option so that the most ambiguous (hard) examples can somehow evade the strict metric computation. What seems more novel here is the improvement of gradient stability, which is still expected (since taking gradient is still a linear operation inside the expectation) but has involved less study. The empirical evidence shown in Figure 4 is strong, and it would be even better if there were corresponding theoretical discussions.
2. The proposed method constructs multiple (20 at most as in the empirical study) models. Since the discussion seems to be heavily focused on deep neural nets, two things to catch here:
- It is unclear whether the proposed method would feasibly apply to cases involving giant model structures and a large amount of output classes. (This comment is slightly nitpicking)
- On one hand, neural net ensembles can be technically constructed as a single neural net with an aggregating final layer and multiplicative numbers of parameters. On the other hand, neural nets themselves are showing behaviors of ensembles, especially when trained with dropout or similar mechanism. The proposed method does not take advantages of these existing relationships between neural nets and neural net ensembles.

The reviewer also finds two minor theoretical issues that might deserve a bit more attention
- The power of the top_2 test is highly influenced by the data generating distribution (e.g. a fair coin flip would also trigger the reject option, and maybe the German Credit dataset), therefore \alpha and \beta are probably not two independent parameters. 
- For reject (abstention) options there is usually a penalty for saying "I'm not sure". It would influence the rates by a fraction of \beta which is safe to ignore. 

The paper is nicely written. The only editorial comments:
1. Theorem 3.1 is also referred as Theorem A.1 in multiple places.
2. Table 3, LF -> LOO
3. Section 3, line 5, "Feature attributions are a commonly used as a tool...", redundant "a". 



=====
The reviewer would like to upgrade the rating to an 8 after the author response. 

summary_of_the_review: The paper presents a highly completed study on the improvements of model stability and feature attribution thanks to the contribution from selective ensembles with abstention option. The paper could use a bit more theoretical discussions regarding the inner workings of the proposed method as well as the relationship between the work and existing ensemble studies to highlight its novelty.
review_title: null

Meta-review: This is a well-done job which combines a few ideas to reach means to identify problematic cases and indicate this when classifying. It has raised doubts about the applicability, though I can see that a abstention rule can have multiple uses. While the work seems to be well done, it has not largely excited the committee members. It initially missed to be placed well wrt existing work to highlight the novelty, and the demonstration that the approach can be generally useful is not complete. Dealing with abstention rules always brings another facet to classification and comparisons are not trivial in many situations. Not surprisingly, this has landed as a borderline case, which I place on the inside (as I like the topic and I think it is interesting work) but it could become an outsider depending on the overall view of the selected papers for the conference and other constraints.
Decision: Accept (Poster)