From: NeurIPS.cc/2022/Workshop/TSRML/Program_Chairs
title: Paper Decision
decision: Accept
comment: Interesting work on explainable AI with good empirical studies.

From: NeurIPS.cc/2022/Workshop/TSRML/Paper125/Reviewer_8sBC
title: This is an interesting work on enabling human-in-the-loop for "concept correction", and is well motivated from explainability point of view.
summary: This work proposed SIUL which is an uncertainty based strategy for selecting the intervention in a Concept bottleneck model. Such CBM models work towards creating human understandable concepts, which can then be interpreted and intervened via humans to increase performance of the NNs. The proposed method is intuitive, considers 4 common uncertainty metrics for measuring uncertainty, and subsequently enabling efficient human intervention. 

strengths: Enabling uncertainty-based concept selection for human intervention is of clear importance towards interpretable AI which can interact with humans. The paper is well written and easy to follow.
weaknesses: In terms of novelty, the work is limited, however, the idea is simple and intuitive.
The comparison against performance of other concept selection methods is lacking.
Last figure lacks legend.
overall_recommendation: The draft is well written and intuitive, and its publication and disclosure is beneficial to the trustworthy and explainable AI community. 
overall_rating: 7: Good paper, accept
review_confidence: 2: The reviewer is willing to defend the evaluation, but it is quite likely that the reviewer did not understand central parts of the paper

From: NeurIPS.cc/2022/Workshop/TSRML/Paper125/Reviewer_Qv9z
title: An interesting problem and an effective solution
summary: Summary. 

The paper is dedicated to proposing test time intervention for learning from uncertain concepts. The authors point out that the human-model interactions in CBM are prone to information leakage and can often be ineffective inappropriate communication with humans. They introduce an uncertainty-based method to select the interventions. Experiments on three vision datasets are presented.


strengths: Pros.

1. This problem setting is interesting.

2. Multiple runs of experiments are conducted.


weaknesses: Cons.

1. The arrangement is poor, which has multiple inappropriate vspace like section 4.4 and section 5.

2. Fonts in Figures 3 and 4 are too small to be recognized.

3. The experiments are limited to single network backbones. Hard to know whether it can work for other network architectures.
overall_recommendation: Learning to accept. Details are referred to previous sections.
overall_rating: 6: Marginally above acceptance threshold
review_confidence: 3: The reviewer is fairly confident that the evaluation is correct