Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper introduces a novel 2-stage classification network where the 1st stage consists of a normal feed-forward pass through a network and the 2nd pass(Introspection) creates a set of features using gradients of the penultimate layer of feed-forward network and passes it through another MLP to predict the correct class. The set of features for each sample is created by concatenating the class-conditioned cross-entropy gradients at the penultimate layer for each class in the dataset. In other words, they calculate cross-entropy gradients for each sample by exhaustively setting every target class as the sample label. These gradients are then concatenated and passed through the MLP to generate the class label. They show that this approach gives them SOTA for Cifar10C and Cifar10 Cure datasets.
strengths_and_weaknesses: STRENGTHS:
1. The paper proposes a novel approach to rectify predictions for a classification network.
2. They show theoretically that the features calculated using gradients could be useful.
3. They show results on Cifar10C and Cifar10 Cure datasets.

WEAKNESSES:
1. This applicability of this work is currently limited to classification tasks that have very few target classes and a penultimate layer of modest dimension.
2. Results are shown only for Cifar10 related dataset and raises concerns if they work for other datasets like Flowers 102,Caltech 101, Cub200 etc.
3. Introspection stage is memory and compute-intensive. The authors should find ways to address this and other scaling issues


questions: SEE Strengths and Weaknesses section
limitations: Yes
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a two-stage formulation for neural network decision making inspired by human introspection. The first stage is the regular discriminative feed-forward stage, where a deep network processes an input and generates the logit features. The introspective features for class I are then given as the change in the network parameters (up until the last layer) when a label of I is introduced for a sample x. The introspective features are then used to train a second introspective network, which tries to match the class predictions obtained from the introspective features to the ground truth predictions as well as the predictions from the first network. The authors provide theoretical derivations for these features, their approximation, sparsity, as well as how to extract them efficiently. The extensive experiments performed by the authors demonstrate that introspective learning improve neural network performance in terms of robustness, calibration, domain adaptation and active learning.
strengths_and_weaknesses: Strengths

1. While the idea of logit gradients encoding class similarity information is not new, using this information to improve robustness and calibration in neural networks is quite novel and very well motivated theoretically and empirically. The biological analogue drawn to Kahneman's Fast/Slow learning to motivate introspective learning is relevant, but somewhat tenuous.

2. The experiments conducted on robustness and calibration show that introspection helps improve both the out-of-distribution test accuracy for distorted images as well as the calibration of the neural network. The introspective neural networks tend to have very low expected calibration error in general.  

3. The results in Table 1 show consistent improvement in performance on existing robustness techniques when using introspection on top. This motivates the usage of introspection as a general purpose technique to train more robust neural network models.

Weakness 

1. This is not an issue with the work itself but against the nomenclature of the approach. The authors use a narrow definition of introspection pertinent specifically to discriminative learning : Why class X over class Y? In general, I am against the practice of co-adapting terms for ideas established in other fields (cognitive science in this case) and using them in an extremely narrow scope in machine learning. This proposed method covers one abductive reasoning based post-hoc 'introspection' approach, not all of introspection as it pertains to humans.  

2. Could the low calibration error of the introspective model be affected by the fact that the introspective model is an MLP, whereas the models used for comparison are CNNs? [1] showed that vector/matrix scaling of CNN logits leads to better calibration error, and I wonder if the introspective model ends up doing something similar to reduce the calibration error.

References

[1] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger, “On calibration of modern neural networks,” in International Conference on Machine Learning. PMLR, 2017, pp. 1321–1330

Disclaimer

I have not formally verified the proofs provided by the authors in Appendix
questions: 1. In L43, the authors say "Consider any dataset with N classes. A network trained on this dataset will have N possible introspective questions and answers." Shouldn't such a network have N^2 possible introspective questions (Why class i, rather than j? where i, j ∈ {1, N})

2. L245 Typo: Robustness
limitations: The authors have adequately discussed the limitations of their work as well as its broader societal impact in Section 7.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a novel idea by using a simple three-layer MLP, instead of fully connected layers, as the reflection stage to ask the network to reflect on its feed-forward decision by considering and evaluating all available choices/classes.
strengths_and_weaknesses: The proposed Introspection Network is very inspiring in transforming $N$ classes into $N$ possible introspective questions and answers. However, the robustness provided by the introspective networks (in Section 4) is not clearly presented.
questions: 1. How does the intuition behind getting Introspective Features different from traditional approaches with fully connected layers + softmax + cross-entropy loss? And why does the proposed approach works better?

2. Please clarify what does $\mathcal{H}(\cot)$ function stand for in equation 7.

3. What is the increased computational complexity of changing the last layer of the network into an Introspective Network?
limitations: N/A
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The paper advocates the use of a two stage process for classification to increase robustness of NN. The first net is standard. The second network leverages gradients of the first network as inputs, i.e., they compute gradients for all possible outcomes towards the last layer (at least this is what seems to be the case according to Fig. 3).

strengths_and_weaknesses: - The basic idea is not novel against the authors claim. See [1]. This work published in 2020 uses also a 2-stage process (one fast and one slow stage), it speaks of reflection, uses the Kahnemann reference, etc. However, aside from these similarities, the technical details seem to be quite different. Also the cited work does not rely on explanations of all classes. However, the authors should discuss this work.
- Theory is appreciated, but I have doubt on correctness. In the proof supplementary material from equation (15) to (16) the authors transform using log(a0*b0+a1*b1)= log(a0+a1) +log(b0+b1). But this equation is not true.
- The paper does not improve in general, i.e., for regular CIFAR-10, but only for CIFAR-10C. Thus, it seems valuable only in a few cases (e.g. the work above improves on CIFAR-10 on the test set using reflection)
- The evaluation is just on one dataset and one architecture. This is insufficient and makes the generalization very questionable.

Detailed comments:
- Robsutness  -> Robustness
[1] "Reflective-Net: Learning from Explanations" from 2020 (https://arxiv.org/abs/2011.13986)
questions: None
limitations: Limitations are ok.
review_title: null

Meta-review: metareview: The paper proposes an approach for reflecting on model predictions in a classification task. The approach is novel and empirical evaluations show significant improvement over standard predictive networks. One of the reviewers is critical of the paper because this is not the first two-stage approach proposed. I do not think that this is fair criticism given the technical details of the current paper and an existing approach the reviewer points out is significantly different, as the reviewer themselves agree. Questions about the practicality of applicability when the number of classes are large and generalization of the conclusions to other datasets were raised, which are fair points in my opinion and addressing these points would make a stronger contribution. 

confidence: Certain
Decision: Accept