Introspective Learning : A Two-Stage Approach for Inference in Neural Networks
Anonymous Author(s)
AfﬁliationAddress email
Abstract
In this paper, we advocate for two stages in a neural network’s decision makingprocess.
The ﬁrst is the existing feed-forward inference framework where patternsin given data are sensed and associated with previously learned patterns.
Thesecond stage is a slower reﬂection stage where we ask the network to reﬂect on itsfeed-forward decision by considering and evaluating all available choices.
Together,we term the two stages as introspective learning.
We use gradients of trained neuralnetworks as a measurement of this reﬂection.
A simple three-layer MLP is usedas the second stage that predicts based on all extracted gradient features.
Weperceptually visualize the post-hoc explanations from both stages to provide avisual grounding to introspection.
For the application of recognition, we showthat an introspective network is 4% more robust and 42% less prone to calibrationerrors when generalizing to noisy data.
We also illustrate the value of introspectivenetworks in downstream tasks that require generalizability and calibration includingactive learning, out-of-distribution detection, and uncertainty estimation.
Finally,we ground the proposed machine introspection to human introspection for theapplication of image quality assessment.
Introduction
Introspection is the act of looking into one’s own mind (1).
Classical introspection has its roots inphilosophy.
Locke (2), the founder of empiricism, held that all human ideas come from experience.
This experience is a result of both sensation and reﬂection.
By sensation, one receives passiveinformation using the sensory systems of sight, sound and touch.
Reﬂection is the objectiveobservation of our own mental operations.
Consider the toy example in Fig.
Given an imagex and an objective of recognizing x , we ﬁrst sense some key features in the bird.
These featuresinclude the color and shape of the body, feathers and beak.
The features are chosen based on ourexisting notion of what is required to contrast between birds.
We then associate these features withour existing knowledge of birds and make a coarse decision that x is a spoonbill.
This is the sensingstage.
Reﬂection involves questioning the coarse decision and asking why x cannot be a Flamingo,
Crane, Pig or any other class.
If the answers are satisfactory, then an introspective decision that x isindeed a spoonbill is made.
The observation of this reﬂection is introspection.
In this paper, we adopt this differentiation between sensing and reﬂection to advocate for twostage neural network architectures for perception-based applications.
Speciﬁcally, we considerclassiﬁcation.
The sensing stage is any existing feed-forward neural network including VGG (3),
ResNet (4), and DenseNet (5) architectures among others.
These networks sense patterns in x andmake a coarse feed-forward decision, ˆ y .
The second stage examines this decision by reﬂecting on theintrospective question ‘Why ˆ y , rather than y I ?’
where y I is any introspective class that the sensingnetwork has learned.
Note that there is no external intervention or new information that informsthe reﬂection stage.
Hence, the introspective features are post-hoc in the sense that they are not causal answers to the introspective questions but rather, are the network’s notion of the answers.
Thisis inline with the original deﬁnition of introspection which is that introspection is the observationof existing knowledge reﬂected upon by the network.
These post-hoc introspective features arevisualized using Grad-CAM (6) in the reﬂection stage of Fig.
The challenge that we address in this paper is in deﬁning the reﬂection stage in terms of neuralnetworks.
Consider any dataset with N classes.
A network trained on this dataset will have Npossible introspective questions and answers.
Our goal is to implicitly extract features that answerall N introspective questions without explicitly training on said features.
This involves 1) implicitlycreating introspective questions, 2) answering the posed questions to obtain introspective features,and 3) predicting the output ˜
y
from the introspective features.
We show that gradients w.r.t networkparameters store notions about the difference between classes and can be used as introspectivefeatures.
We use an MLP termed H ( · ) , as our introspective network that combines all features topredict ˜ y .
A limitation of the proposed method is the size of N which is discussed in Sections 3.
We show that the introspective prediction ˜
y is robust to noise.
An intuition for this robustness is thatnot only should the network sense the feed-forward patterns, it must also satisfy H ( · ) ’s N notions ofdifference between classes.
Hence, during inference, we extract N additional features that inform theintrospective prediction ˜ y .
The main contributions of this paper are the following:
We deﬁne implicit introspective questions that allow for reﬂection in a neural network.
This
reﬂection is measured using loss gradients w.r.t.
network parameters across all possible
introspective classes in Section 3.2.
We provide a methodology to efﬁciently extract introspective loss gradients and combinethem using a second H ( · ) MLP network in Section 4.3.
We illustrate H ( · ) ’s robust and calibrated nature in Section 6.
We validate the effectivenessof this generalization in downstream applications like active learning, out-of-distributiondetection, uncertainty estimation, and Image Quality Assessment in Appendix D.
Background
2.1 Introspective questions, features, and network
Introspective questions The choice of ‘Why ˆ y , rather than y I ’? is not arbitrary.
The authorsin (7) describe three questions that complete post-hoc explanations - correlation, counterfactual,and contrastive questions.
These questions together allow for an alternate form of reasoning withinneural networks called abductive reasoning.
The sensing network predicts based on correlations.
Since, we do not intervene within the data during the reﬂection stage, counterfactual questions cannotbe answered.
Hence, we use contrastive questions as our introspective questions.
Further detailsregarding abductive reasoning and our choice of question is provided in Appendix A.
Gradients as Introspective Features The gradients from a base network have been utilizedin diverse applications including post-hoc visual explanations (6; 8; 9), adversarial attacks (10), uncertainty estimation (11), anomaly detection (12; 13), and saliency detection (14) among others.
Fisher Vectors use gradients of generative models to characterize the change that data creates withinfeatures (15).
(16) uses gradients of parameters to characterize the change in manifolds when newdata is introduced to an already trained manifold.
Our framework uses the intuition from (16) tocharacterize changes for a datapoint that is perceived as new, due to it being assigned an introspectiveclass
y I , that is different from its predicted class
ˆ y .
In (17), the authors view the network as agraph and intervene within it to obtain holographic features.
Our introspective features are alsoholographic in the sense that they characterize the change between ˆ y and y I without changing thenetwork.
However, our features do not require interventions that become expensive with scale.
The usage of two-stage approaches to inference in neural networks is not new.
The authors in (18) propose SimCLR, a self-supervised framework where multiple data augmentationstrategies are used to contrastively train an overhead MLP.
The MLP provides features which arestored as a dictionary.
This feature dictionary is used as a look-up table for new test data.
In thispaper, we use gradients against all classes as features and an MLP H ( · ) , to predict on these features.(19) and (20) consider all classes in a conditional maximum likelihood estimate on test data to retrainthe model.
These works differ from ours in our usage of the base sensing network.
(21) uses gradientsand activations together as features and note that the validity of gradients as features is in pretrainedbase networks rather than additional parameters from the two-stage networks.
This adds to ourargument of using two-stage networks but with loss gradients against introspective classes.
2.2 Feed-forward Features
For the application of recognition, a sensing neural network f ( · ) is trained on a distribution X toclassify data into N classes.
The network learns notions about data samples when classifying them.
These notions are stored as network weights W .
Given a data sample x , f ( x ) is a projection on thenetwork weights.
Let y feat be the logits projected before the ﬁnal fully connected layer.
In the ﬁnalfully connected layer f L ( · ) , the parameters W L can be considered as N ﬁlters each of dimensionalityd L  1 ⇥ 1 .
The output of the network ˆ y is given by,
[Equation]
Here ˆ y is the feed-forward inference and y feat are the feed-forward features.
In this paper, wecompare our introspective features against feed-forward features.
Since introspection occurs afterf ( · ) , all our results are plug-in on top of existing f ( · ) .
Introspective Features
In this section, we describe introspective features and implicitly extract them using the sensingnetwork.
We then analyze them for sparsity, efﬁciency, and robustness.
Deﬁnition 3.1 (Introspection) .
Given a network f ( · ) , a datum x , and the network’s predictionf ( x ) =
ˆ y , introspection in f ( · ) is the measurement of change induced in the network parameterswhen a label y
I is introduced as the label for x .
This measurement is the gradient induced by a lossfunction J ( y I , ˆ y ) , w.r.t.
the network parameters.
This deﬁnition for introspection is in accordance with the sensing and reﬂection stages in Fig.
Thenetwork’s prediction ˆ y is the output of the sensing stage and the change induced by an introspectivelabel, y I , is the network reﬂecting on its decision
ˆ y as opposed to y I .
Note that introspection canoccur when ˆ y is contrasted against any trained label
y I , I 2 [1 , N ] .
For instance, in Fig. 1, thenetwork is asked to reﬂect on its decision of spoonbill by considering other y I
that x can takeﬂamingo, crane, or pig.
These results are extracted from ImageNet (22) and hence, I 2 [1 , 1000] .
Additional visual introspective saliency maps similar to Fig.
1 are provided in Appendix A.
Reﬂection is the empirical risk that the network has predicted x as ˆ y instead of y I .
Given the networkparameters, this risk is measured through some loss function J ( y I , ˆ y ) .
y I is a one-hot vector witha one at the I th location.
The change that is induced in the network is given by the gradient of
J ( y I , ˆ y ) w.r.t.
the network parameters.
For an N -class classiﬁer, there are N possible introspectiveclasses and hence N possible gradients each given by, r I =
r W J ( y I , ˆ y ) , I 2 [1 , N ] .
Here, r I arethe introspective features.
Since we introspect based on classes, we measure the change in networkweights in the ﬁnal fully connected layer.
Therefore, the introspective features are given by,
[Equation]
124 where W L are the network weights for the ﬁnal fully connected layer.
Note that the ﬁnal fullyconnected layer from Eq.
1 has a dimensionality of <
d L  1 ⇥ N .
For every x , Eq. 2 is applied N timesto obtain N separate r I .
We ﬁrst analyze these features.
3.1 Sparsity and Robustness of Introspective Features
Consider r I in Eq. 2.
Each r I is a d L  1 ⇥ N matrix.
Expressing gradients in r I separately w.r.t.
thedifferent ﬁlters in W L , we have a row-wise concatenated set of gradients given by,
[Equation]
where each W L,j 2 <
d L  1 ⇥ 1 and r I 2 <
d L  1 ⇥ N 2 .
For all data x 2 X the following lemma holds:
Lemma 1.
Given a unique ordered pair ( x, ˆ y ) and a well-trained network f ( · ) , the gradients for aloss function J ( y I , ˆ y ) w.r.t.
classes are pairwise orthogonal under the second-order Taylor seriesapproximation, each class paired with the predicted class.
Proof.
Provided in Appendix B.1.
Sparsity Lemma 1 states that backpropagating class
y I does not provide any information to W L,j , j 6 =
I and hence there is no need to use
[Equation]
features when considering y I .
The proofis provided in Appendix B.1.
r W J ( y I , ˆ y ) for an introspective class reduces to,
[Equation]
where y ˆ y is the logit associated with the predicted class.
We demonstrate the sparsity of Eq.
4 in
Fig.
A two-layer CNN is trained on MNIST (23) dataset with a test accuracy exceeding 99% .
Thissatisﬁes the condition for Lemma 1 that f ( · ) is well-trained.
The ﬁnal fully connected layer in thisnetwork has a size of 50 ⇥ 10 .
We provide an input image x of number 5 to a trained network asshown in Fig.
The network correctly identiﬁes the image as a 5 .
We then backpropagate theintrospective class 0 using J (5 , 0) with ˆ y =
and y I = 0 .
This answers the question ‘Why 5, ratherthan 0?’ .
The gradient features in the ﬁnal fully connected layer are the same dimensions as theﬁnal fully connected layer, 50 ⇥ 10 or more generally O ( d L  1 ⇥ N ) .
This matrix is displayed as a 146 normalized image in Fig.
Yellow scales to 1 and blue is  1 while green is 0 .
It can be seen thatthe only values present in the matrix are negative at W L, 0 , in blue, and positive in W L, 5 , in yellow.
This validates Eq.
5 that for a fully-trained network the only values, and hence the only information,required from W L for I = 0 is r W L, 0 .
We show the matrix r W L when I = 0 , 1 , 2 , 4 , 5 , 6 .
Thedifference among all matrices is the location of the negative values that exist at r W L,I for differentvalues of I .
Robustness Eq.
4 motivates the generalizable nature of introspective features.
Consider some noiseadded to x .
To change the prediction ˆ y , the noise must sufﬁciently decrease y ˆ y from Eq.
4 andincrease the closest logit value, y I , to change the prediction.
Hence, it needs to change one orthogonal relationship.
However, by constraining our ﬁnal prediction ˜ y on N such features, the to orthogonal
N
3.2 Efﬁcient Extraction of Introspective Features
From Lemma 1, the introspective feature is only dependent on the predicted class ˆ y and the introspective class
y I making their span orthogonal to all other gradients.
Hence,
[Equation]
Building on Lemma 1, we present the following theorem.
Theorem 1.
Given a unique ordered pair ( x, ˆ y ) and a well-trained network f ( · ) , the gradientsfor a loss function J ( y I , ˆ y ) , I 2 [1 , N ] w.r.t. classes when y I are N orthogonal one-hot vectors isequivalent to when y I is a vector of all ones, under the second-order Taylor series approximation.
Proof.
Provided in Appendix B.2.
The proof follows Lemma 1.
Theorem 1 states that backpropagating a vector of all ones ( 1 N ) isequivalent to backpropagating N one-hot vectors with ones at orthogonal positions.
This reduces thetime complexity from O ( N ) to a constant O (1) since we only require a single pass to backpropagate1 N .
Hence, our introspective feature is given by,
[Equation]
Note the LHS is now r x instead of r I from Eq.
The ﬁnal introspective feature is a matrix of thesame size as W L extracted in O (1) time with a space complexity of O ( d L  1 ⇥ N ) .
r x is vectorized 172 and scaled between [  1 , 1] before being used in Sections 4 and 6 as introspective features.
Thisprocedure is illustrated in Fig.
Introspective Network
Once r x are extracted using Eq. 6, the introspective label ˜ y from Fig.
3 is given by ˜ y
= H ( r x ) .
In this section, we analyze H ( · ) .
From Fig.
3, f ( · ) is any existing trained network used to obtainintrospective features r x .
It is trained to predict the ground truth y given any x .
Based on theassumption that H ( r x ) =
E ( y | f ( x )) and hence expectation of y  H ( r x ) is 0 , the loss function canbe decomposed as,
[Equation]
Note that since the goal is to predict y given x , H ( r x ) =
E ( y | f ( x )) is a fair assumption to make.
Substituting for f ( x ) in Eq. 7, and using variance decomposition of y onto f ( x ) , we have,
[Equation]
This decomposition is adopted from structured calibration techniques.
A full derivation is presentedin (24).
The ﬁrst term Var (ˆ y ) is the the variance in the prediction from f ( · ) .
This term is the precisionof f ( · ) and is low for a well trained network.
The third term is the MSE function between theintrospective network H ( · ) and the ground truth.
It is minimized while training the H ( · ) network.
The second term is the variance of the network
H ( · ) , given features r x .
Note that minimizing 188 Eq.
8 can occur by maximizing Var (
H ( r x )) .
We use a ﬁsher vector interpretation from (16) toanalyze
Var ( H ( r x )) .
If H ( · ) is a linear layer with parameters W H , the Var ( H ( r x )) term reduces to 190 W T [Equation]
Var ( r [Equation]
[Equation] ( r [Equation] ⌃  1 r x ) where ⌃ is the covariance matrix.
⌃ is a gaussian approximationfor the shape of the manifold.
Generalizing it to a higher dimensional manifold and replacing ⌃ with
F , we have,
[Equation]
The RHS of Eq.
10 is a sum of ﬁsher vectors taken across all possible labels.
When do introspective networks provide robustness?
We use Eq.
10 to analyze introspectivelearning usage.
Speciﬁcally, we consider two cases: When input x is taken from the trainingdistribution X , and when it is taken from a noisy distribution X 0 .1.
When a sample x 2 X is provided to a network f ( · ) trained on X , all r j , j
= ˆ y in Eq.
10tend to 0 .
The RHS reduces to r T ˆ y F  1 r ˆ y .
r ˆ y is a function of f ( x ) only and hence adds nonew information to the framework.
The results of H ( · ) remain the same as f ( · ) .
In otherwords, given a trained ResNet-18 on CIFAR-10, the results of feed-forward learning will bethe same as introspective learning on CIFAR-10 testset.2.
When a new sample x 0 62 X is provided to a network f ( · ) trained on X , a ﬁsher vectorbased projection across labels is more descriptive compared to a feed-forward approach.
The N gradients in Eq.
10 add new information based on how the network needs to changethe manifold shape F to accommodate the introspective gradients.
Hence, given a distortedversion of CIFAR-10 testset, our proposed introspective learning generalizes with a higheraccuracy while providing calibrated outputs from Eq.
We empirically illustrate theseclaims in Section 6.
We motivate other applications including active learning, Out-OfDistribution (OOD) detection, uncertainty estimation, and
Image Quality Assessment (IQA)that are dependent on generalizability and calibration in Section 5.
Related Works for Considered Applications
Augmentations and Robustness The considered r x features from Eq.
6 can be considered asfeature augmentations.
Augmentations, including SimCLR (18), Augmix (25), adversarial augmentation (26), and noise augmentations (27) have shown to increase robustness of neural networks.
We use introspection on top of non-augmented and augmented (Section 6) networks and show thatour proposed two-stage framework increases the robustness to create generalizable and calibratedinferences which aids active learning and out-of-distribution (OOD) detection.
The same frameworkthat robustly recognizes images despite noise can also detect noise to make an out-of-distributiondetection.
Conﬁdence and Uncertainty The existence of adversarial images (10) heuristically decouplesthe probability of neural network predictions from conﬁdence and uncertainty.
A number of worksincluding (28) and (29) use bayesian formulation to provide uncertainty.
However, in downstreamtasks like active learning and Out-Of-Distribution (OOD) detection applications, existing state-of-theart methods utilize softmax probability as conﬁdences.
This is because of the simplicity and ease ofnumerical computation of softmax.
In active learning, uncertainty is quantiﬁed by the entropy (30),least conﬁdence (30), or maximum margin (31) of predicted logits, or through extracted features in
BADGE (32), and BALD (33).
In OOD detection, (34) proposes Maximum Softmax Probability(MSP) as a baseline method by creating a threshold function on the softmax output.
(35) proposes
ODIN and improves on MSP by calibrating the network’s softmax probability using temperaturescaling (36).
In this paper, we show that the proposed introspective features are better calibrated thantheir feed-forward counterparts.
Hence existing methods in active learning and OOD detection have asuperior performance when using H ( · ) to make predictions.
Human Introspection We are unaware of any direct application that tests visual human introspection.
In its absence, we choose the application of Full-Reference Image Quality Assessment (FR-IQA)to connect machine vision with human vision.
The goal in FR-IQA is to objectively estimate thesubjective quality of an image.
Humans are shown a pristine image along with a distorted imageand asked to score the quality of the distorted image (37).
This requires reﬂection on the part ofthe observers.
We take an existing algorithm (38) and show that introspecting on top of this IQAtechnique brings its assessed scores closer to human scores.
Experiments
Across all applications except in Ablation studies in Table 7, we use a 3-layered MLP with sigmoidactivations as H ( · ) .
The structure is presented in Appendix C.1.
We ﬁrst deﬁne robustness andcalibration in the context of this paper.
Robsutness In this paper, without loss of consistency with related works, we say that the networktrained on distribution X is robust if it correctly predicts on a shifted distribution X 0 .
The differencein data distributions can be because of data acquisition setups, environmental conditions, distortionsamong others.
We use CIFAR-10 for X and two distortion datasets - CIFAR-10C (26) and CIFAR10-CURE (39) as X 0 .
Generalization is measured through performance accuracy.
Calibration Given a data distribution x 2 X , belonging to any of y 2 [1 , N ] , a neural networkprovides two outputs - the decision ˆ y and the conﬁdence associated with ˆ y , given by ˆ p .
Let p be thetrue probability empirically estimated as p
= ˆ p
i , 8 i 2 [1 , M ] .
Then calibration is given by (36),
[Equation]
Calibration measures the difference between the conﬁdence levels and the prediction accuracy.
Toshowcase calibration we use the metric of Expected Calibration Error (ECE) as described in (36).
The network predictions are placed in 10 separate bins based on their prediction conﬁdences.
Ideally,the accuracy equals the mid-point of conﬁdence bins.
The difference between accuracy and mid-pointof bins, across bins is measured by ECE.
Lower the ECE, better calibrated is the network.
Datasets and networks CIFAR-10C consists of 950 , 000 images whose purpose is to evaluate therobustness of networks trained on original CIFAR-10 trainset.
CIFAR-10C perturbs the CIFAR-10testset using 19 distortions in 5 progressive levels.
Hence, there are 95 separate X 0 distributions totest on with each X 0 consisting of 10000 images.
Note that we are not using any distortions or datafrom CIFAR-10C as a validation split during training.
The authors in (39) provide realistic distortionsthat they used to benchmark real-world recognition applications including Amazon Rekognition and
Microsoft Azure.
We use these distortions to perturb the test set of CIFAR-10.
There are 6 distortions,each with 5 progressive levels.
Of these 6 distortions - Salt and Pepper, Over Exposure, and Under
Exposure noises are new compared to CIFAR-10C.
We train four ResNet architectures - ResNet-18,34, 50, and 101 (4).
All four ResNets are evaluated as sensing networks f ( · ) .
The training procedureand hyperparameters are presented in Appendix C.1.
Testing on CIFAR-10 testset The trained networks are tested on CIFAR-10 testset with accuracies91 .
09% , and 93 .
11% respectively.
Next we extract r x on all training and testingimages in CIFAR-10.
H ( · ) is trained using r x from the trainset using the same procedure as f ( · ) .
When tested on r x of the testset, the accuracy for ResNets-18,34,50,101 is 90 .
17% ,and 93 .
Note that this is similar to the feed-forward results.
The average ECE of all feedforward and introspective networks is 0 . 04 .
Hence, when the test distribution is the same as trainingdistribution there is no change in performance.
Testing on CIFAR-10C and CIFAR-10-CURE
The results of all networks averaged acrossdistortions in both the datasets are shown in Fig.
Note that in each case, there is a shift leftwardand upward indicating that the performance improves while the calibration error decreases.
In thelarger CIFAR-10C dataset, the introspective ResNet-18 performs similar to ResNets-34 and 50 in terms of accuracy while beating them both in calibration.
A more ﬁne-grained analysis is shown in
Fig.
5 for ResNet-18.
The blue bars in Fig.
5a) represent the feed-forward accuracy.
The red bars arethe introspective accuracy gains over the feed-forward accuracy.
Among 7 of the 19 distortions, theaccuracy gains are over 5% .
In Appendix C.2.1, we see that the gains are higher when the distortionsare higher.
Introspection performs well on blur-like distortions while struggling with distortionsthat disrupt the lower level characteristics of the image like brightness, contrast, and saturate.
Thiscan be attributed to the fact that r x are derived from the last layer of f ( · ) and are missing low-levelstatistics that are ﬁltered out by network in the initial layers.
However, in Fig.
5b)
, we show ECEfor brightness and saturate distortions across all 5 distortion levels - higher the level, more is thedistortion affecting X 0 .
It can be seen that while the ECE for feed-forward networks increases across levels, the for introspective networks decrease.
Hence, even when there are gains to had, introspection
Plug-in results of
Introspection Note that there are a number of techniquesproposed to alleviate a neural network’s robustness challenges against distortions.
[Table]
The authors in (27) show that ﬁnetuning VGG-16 us-
[Table]
ing blurry training images increases the performance
[Table]
of classiﬁcation under blurry conditions.
(40) propose
[Table]
utilizing distorted virtual images to boost performance
[Table]
accuracy.
The authors in (26) use adversarial images
[Table]
to augment the training data.
All these works require
[Table]
knowledge of distortion or large amounts of new data
[Table]
during training.
Our proposed method can infer intro-
[Table]
spectively on top of any existing f ( · ) enhanced using
[Table]
existing methods.
In Table 1, we show results of in- [Table] trospection as a plug-in approach on top of existing [Table]
techniques. In all methods, introspection outperforms [Table] its feed-forward counterpart.
While introspecting on
[Table] top of Augmix (25) provides insigniﬁcant recognition [Table] accuracy gains, introspection reduces ECE of Augmix
[Table] network by 43 .
In Appendix C.2, we show performance on top of (27) and (26) of 6 . 8% on Level 5 distortions.
In Appendix C.3, we analyze SimCLR and show that introspecting on the [Table] self supervised features increases its CIFAR-10C performance by about 6% on ResNet-101.
Anumber of ablation studies including analysis of structure of H ( · ) , loss functions, distortion levels onperformance accuracy and ECE are shown in Appendix C.4.
Moreover, we examine introspectionwhen X 0 is domain shifted data from Ofﬁce (41) dataset in Appendix C.6.
Downstream Applications Usage of introspective learning on top of existing state of the artalgorithms in active learning and OOD detection is demonstrated in Appendix D. In both applications,we consider two settings.
The ﬁrst is when the test data is from the training distribution X and thesecond is when the testset is from X 0 .
We show results that conform to Eq. 10.
In active learning in
Table 10, the accuracy on X 0 increases by an average of 1 . 3% across ﬁve query strategies averagedacross 20 rounds.
In Image Quality Assessment in Table 13, introspection provides statisticallysigniﬁcant results against all algorithms in atleast one of the compared metrics.
Discussion and Conclusion
Limitations and future work
The paper illustrates the beneﬁts of utilizing the change in modelparameters as a measure of model introspection.
In Section 3.2, we accelerate the time complexity to
O (1) .
However, the space complexity is still dependent on N .
The paper uses an MLP for H ( · ) andconstructs r x by vectorizing extracted gradients.
For datasets with large N , usage of r x as a vectoris prohibitive.
Hence, a required future work is to provide a method of combining all N gradientswithout vectorization.
Also, our implementation uses serial gradient extraction across images.
This isnon-ideal since the available GPU resources are not fully utilized.
A parallel implementation withper-sample gradient extraction (42) is a pertinent acceleration technique for the future.
Broader and Societal Impact
In his seminal book in 2011 (43), Daniel Kahneman outlines twosystems of thought and reasoning in humans - a fast and instinctive ‘system 1’ that heuristicallyassociates sensed patterns followed by a more deliberate and slower ‘system 2’ that examines andanalyzes the data in context of intrinsic biases.
Our framework derives its intuition based on thesetwo systems of reasoning.
The introspective explanations can serve to examine the intrinsic notionsand biases that a network uses to categorize data.
Note that the network H ( · ) obtains its introspectiveanswers through f ( · ) .
Hence, similar to the ‘system 2’ reasoning in humans, any internal bias presentin f ( · ) only gets strengthened in H ( · ) through conﬁrmation bias.
The framework will beneﬁt froma human intervention between f ( · ) and H ( · ) in sensitive applications.
One way would be to askcounterfactual questions by providing an established counterfactual and asking the network to reﬂectbased on that.
While the introspective framework will remain the same, the features will change.
Conclusion We introduce the concept of introspection in neural networks as two separate stagesin a network’s decision process - the ﬁrst is making a quick assessment based on sensed patterns indata and the second is reﬂecting on that assessment based on all possible decisions that could havebeen taken and making a ﬁnal decision based on this reﬂection.
We show that doing so increases thegeneralization performance of neural networks as measured against distributionally shifted data whilereducing the calibration error of neural networks.
Existing state-of-the-art methods in downstreamtasks like active learning and out-of-distribution detection perform better in an introspective settingcompared to a feed-forward setting especially when the distributional difference is high. Checklist For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] .
See Section 7.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] .
SeeSection 7.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] .
See Appendix B
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] .
The code will
[Table]
be made public upon acceptance.
[Table]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
[Table]
were chosen)?
[Yes] .
See Section 6 and Appendix C.
[Table]
(c) Did you report error bars (e
g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] .
See Appendix D
[Table]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?[Yes]
in Appendix E
[Table]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
[Table]
(a) If your work uses existing assets, did you cite the creators?
[Yes] .
See Appendix E
[Table]
(b) Did you mention the license of the assets?
[Yes] .
See Appendix E
[Table]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
[Table]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
[Table]
using/curating?
[No]
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
[Table]
information or offensive content?
[No]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
[Table]