Original: V IS FIS:  Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectives
Revised: V IS FIS: Improved Visual Feature Importance Supervision with Right-for-Right-Reason Objectives

Original: Anonymous Author(s)
Revised: s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Many past works aim to improve visual reasoning in models by supervising featureimportance (estimated by model explanation techniques) with human annotationssuch as highlights of important image regions.
Revised: Recent work aims to improve Visual Question Answering (VQA) models by supervising model feature importance (estimated by model explanation techniques) withhuman annotations such as highlights of important image regions.

Original: However, recent work has shownthat performance gains from feature importance (FI) supervisionfor Visual Question Answering (VQA) tasks persist even with random supervision, suggestingthat these methods do not meaningfully align model FI with human FI.
Revised: With furtheranalysis, however, Shrestha et al. [2020] show that performance gains from featureimportance (FI) supervision persist even with random supervision, suggesting thatthese methods do not meaningfully align model FI with human FI.

Original: In thispaper, we show that model FI supervision can meaningfully improve VQA modelaccuracy as well as performance on several Right-for-the-Right-Reason (RRR)metrics by optimizing for four key model objectives: (1) accurate predictions givenlimited but sufficient information (Sufficiency); (2) max-entropy predictions givenno important information (Uncertainty); (3) invariance of predictions to changesin unimportant features (Invariance); and (4) alignment between model FI explanations and human FI explanations (Plausibility).
Revised: In this paper, weshow that model FI supervision can meaningfully improve VQA model accuracyas well as performance on Right-for-Right-Reason (RRR) metrics by optimizingfor four key model properties: (1) accurate predictions given limited but sufficientinformation (Sufficiency); (2) max-entropy predictions given no important information (Uncertainty); (3) invariance of predictions to changes in unimportant features(Invariance); and (4) alignment between model FI explanations and human FI explanations (Plausibility).

Original: Our best performing method, 
Revised: Our best performing method, Visual Feature Importance

Original: Visual Feature Importance Supervision (V IS FIS), outperforms strong baselines onbenchmark VQA datasets in terms of both in-distribution and out-of-distributionaccuracy.
Revised: Supervision (V IS FIS), outperforms strong baselines on benchmark VQA datasets.

Original: While past work suggests that the mechanism for improved accuracy isthrough improved explanation plausibility, we show that this relationship dependscrucially on explanation faithfulness (whether explanations truly represent themodel’s internal reasoning).
Revised: While past work suggests that the mechanism for improved accuracy is throughimproved explanation plausibility, we show that this relationship depends cruciallyon explanation faithfulness (whether explanations truly represent the model’s internal reasoning).

Original: Predictions are more accurate when explanations areplausible and faithful, and not when they are plausible but not faithful.
Revised: Predictions are more accurate when explanations are plausibleand faithful, and not when they are plausible but not faithful.

Original: Lastly, weshow that, surprisingly, RRR metrics are not predictive of out-of-distribution modelaccuracy when controlling for a model’s in-distribution accuracy, which calls intoquestion the value of these metrics for evaluating model reasoning. 
Revised: We also show that,surprisingly, RRR metrics are not predictive of out-of-distribution model accuracywhen controlling for a model’s in-distribution accuracy. This calls into questionthe value of these metrics for evaluating model reasoning. Yet, interestingly, FIsupervision with RRR objectives can still substantially improve in-distribution andout-of-distribution accuracy.

Original:  Introduction
Revised:  Introduction

Original: Many past works aim to teach models to ignore spurious features by making use of additionalinformation about which features in an input are important [29, 42, 55].
Revised: Many past works aim to teach models to ignore spurious features by making use of additional information about which features in an input are important [27, 39, 52].

Original: For example, individualwords can be annotated as (un)important in NLP tasks [42, 60], or regions of image pixels canbe highlighted by humans as extra supervision for vision tasks [11, 47].
Revised: For example, individualwords can be annotated as (un)important in NLP tasks [39, 57], or regions of image pixels canbe highlighted by humans as extra supervision for vision tasks [11, 44].

Original: In this broad class offeature importance (FI) supervision methods, human annotations of important features are typicallyprovided for individual datapoints, and methods often use data augmentation or gradient supervisionto encourage models to rely only on important features when making predictions.
Revised: In this broad class offeature importance (FI) supervision methods, human annotations of important features are typicallyprovided for individual datapoints, and methods often use data augmentation or gradient supervisionto encourage models to rely only on important features when making predictions.

Original: Such approacheshave seen performance improvements in image classification [46, 7], text classification [33, 60, 53],and multimodal visual question answering (VQA) tasks [43, 58, 34].
Revised: Such approacheshave seen performance improvements in image classification [43, 7], text classification [31, 57, 50],and multimodal visual question answering (VQA) tasks [40, 55, 32].

Original: One of the primary motivations behind these approaches is to improve task accuracy by makingmodels “Right for the Right Reasons” [42].
Revised: One of the primary motivations behind these approaches is to improve task accuracy by makingmodels “Right for the Right Reasons” [39].

Original: While existing FI supervision methods for VQA models [Figure] can improve accuracy [43, 58], recent work has cast doubt on whether human supervision is thesource of these improvements.
Revised: While existing FI supervision methods for VQA modelscan improve accuracy [40, 55], recent work has cast doubt on whether human supervision is thesource of these improvements.

Original: Specifically, Shrestha et al.
Revised: Specifically, Shrestha et al.

Original: [44] show that VQA improvements persisteven when random image FI annotations are used as supervision, suggesting that existing approachesmay not extract meaningful signal from the human annotations.
Revised: [41] show that VQA improvements persisteven when random image FI annotations are used as supervision, suggesting that existing approachesmay not extract meaningful signal from the human annotations.

Original: Motivated by this shortcoming, weexplore several aspects of the FI supervision problem in the context of VQA tasks:
Revised: Motivated by this shortcoming, weexplore several aspects of the FI supervision problem in the context of VQA tasks:

Original: Improving VQA accuracy with FI supervision via four key model objectives (Sec. 7.1).
Revised: Improving VQA accuracy with FI supervision via four key model objectives (Sec. 7.1).

Original: Past VQAmethods focus on data augmentation techniques [34] or  directly supervising model feature importance[43, 58].
Revised: Past VQA methods focus on data augmentation techniques [32] or ways to directly supervise model featureimportance [40, 55].

Original: We make use of four key objectives (represented in Fig.
Revised: We make use of four key objectives (represented in Fig.

Original: 1): (1) a Sufficiency objectiveencouraging the model to predict the correct label given only important input features [7]; (2) an Uncertainty objective encouraging max-entropy outputs when given only unimportant features; (3)an Invariance objective encouraging model outputs to be invariant to changes in unimportant features[42]; and (4) an Align objective that penalizes the model when its FI estimates differ from human
Revised: 1): (1) a Sufficiencyobjective encouraging the model to predict the correct label given only important input features [7]; (2)an Uncertainty objective encouraging max-entropy outputs when given only unimportant features; (3)an Invariance objective encouraging model outputs to be invariant to changes in unimportant features[39]; and (4) an Align objective that penalizes the model when its FI estimates differ from human

Original: FI annotations [43].
Revised: FI annotations [40].

Original: Our best performing method, termed Visual Feature Importance Supervision(V IS FIS), combines these strategies  in a novel manner, improving both in-distribution and out-ofdistribution accuracy.
Revised: Our best performing method, termed Visual Feature Importance Supervision(V IS FIS), combines these strategies strategies in a novel manner, improving both in-distributionand out-of-distribution accuracy.

Original: Following guidelines from Shrestha et al.
Revised: Following guidelines from Shrestha et al.

Original: [44], we show that this improvementdoes not occur with random supervision, meaning V IS FIS learns from human supervision itself.
Revised: [41], we show thatthis improvement does not occur with random supervision, meaning V IS FIS learns from humansupervision itself.

Original: Lastly,  analyzing how explanation plausibility and faithfulness [22] relate to accuracy at the datapointlevel, we suggest that FI supervision improves prediction accuracy by improving the plausibility offaithful FI explanations, rather than plausibility alone as past work has suggested [43, 7].
Revised: Lastly, after analyzing how explanation plausibility and faithfulness [20] relateto accuracy at the datapoint level, we suggest that FI supervision improves prediction accuracy byimproving the plausibility of faithful FI explanations, rather than plausibility alone as past work hassuggested [40, 7].

Original: because suggests generalizability
Revised: Evaluating models on new Right for the Right Reason (RRR) metrics (Sec. 7.4). Beyondmeasuring model accuracy, past works evaluate models on a few Right for the Right Reason metricsin order to understand whether model reasoning is correct [40, 41, 56, 7, 15].

Original:  occur e.g. when there  exponentially  we wish to test data is prohibitivelyexpensive to collect.
Revised: Correct reasoning isvaluable because it suggests that models will generalize to test data that we might not be able to verifytheir performance on, which can occur e.g. when there are exponentially many cases we wish to testor when such data is prohibitively expensive to collect.

Original: We propose a broad set of RRR metrics for model evaluation, with similarmotivation to our key model objectives above (see Fig. 1).
Revised: We propose a broad set of RRR metrics formodel evaluation, with similar motivation to our key model objectives above (see Fig. 1).

Original: Specifically, in addition to measuringexisting metrics 
Revised: Specifically,in addition to measuring existing metrics for (1) in-distribution accuracy, (2) out-of-distributionaccuracy [1, 10, 6, 8, 7], and (3) model accuracy on sufficient feature subsets (RRR-Suff) [7], wealso evaluate (4) model uncertainty given uninformative inputs (RRR-Unc), and (5) model invarianceto changes in unimportant features (RRR-Inv).

Original: model accuracy also model uncertaintygiven uninformative inputs These evaluate featuresthat actually important  invariant  addition  features  should  when  contains no meaningful evidence 
Revised: These metrics help verify that models can: arriveat correct answers relying only on features that are actually important (metric #3), are invariant tothe addition or removal of unimportant features that should not affect the label (metric #4), and areappropriately uncertain about the model class when the input contains no meaningful evidence forany class (metric #5).

Original: Predicting model generalization to OOD data with RRR metrics (Sec. 7.5).
Revised: Predicting model generalization to OOD data with RRR metrics (Sec. 7.5).

Original: The practical valueof the above RRR metrics can be considered in terms of their ability to inform us about modelperformance on data that we are not able to test on.
Revised: The practical valueof the above RRR metrics can be considered in terms of their ability to inform us about modelperformance on data that we are not able to test on.

Original: We simulate this situation by evaluating modelson in-distribution (ID) data and predicting whether the models will generalize to OOD data based on their accuracy and RRR metrics for ID data.
Revised: We simulate this situation by evaluating modelson in-distribution (ID) data and predicting whether the models will generalize to OOD data based ontheir accuracy and RRR metrics for ID data.

Original: Surprisingly, we find that both existing RRR metrics andour new ones do not better predict OOD accuracy than ID accuracy does on its own.
Revised: Surprisingly, we find that both existing RRR metrics andour new ones do not better predict OOD accuracy than ID accuracy does on its own.

Original: This findingsuggests that these metrics may not be a good evaluation of model reasoning, and that there is nogood replacement yet for evaluating model accuracy on OOD data in addition to ID data.
Revised: This findingsuggests that these metrics may not be a good evaluation of model reasoning, and that there is nogood replacement yet for evaluating model accuracy on OOD data in addition to ID data.

Original: In summary of our contributions, we show that:
Revised: In summary of our contributions, we show that:

Original: FI supervision can improve both ID and OOD model accuracy on several benchmark VQA datasets.
Revised: FI supervision can improve both ID and OOD model accuracy on several benchmark VQA datasets.

Original: In particular, V IS FIS improves over unsupervised baselines and the previous state-of-the-art onCLEVR-XAI  by up to 4.7 points on OOD data.
Revised: In particular, V IS FIS improves over unsupervised baselines and the previous state-of-the-art onCLEVR-XAI and GQA, by up to 4.7 points on OOD data.2.

Original: Explanation plausibility correlates with model accuracy only when explanations are also faithful,which sheds light on the mechanism by which FI supervision improves model accuracy.3.
Revised: Explanation plausibility correlates with model accuracy only when explanations are also faithful,a novel result shedding light on the mechanism by which FI supervision improves model accuracy.3.

Original: FI supervision improves model performance on several RRR metrics, including new invarianceand uncertainty metrics.4.
Revised: FI supervision improves model performance on several RRR metrics, including new invarianceand uncertainty metrics.4.

Original: RRR metrics do not correlate better with OOD accuracy than ID model accuracy does on its own. 
Revised: RRR metrics do not correlate better with OOD accuracy than ID model accuracy does on its own. Consequently, RRR metrics may not

Original: Consequently, RRR metrics may not be as valuable as previouslythought.
Revised: be as valuable as previously

Original:   Related Work
Revised: thought. [Figure]  Related Work

Original: Supervising FI explanations.
Revised: Supervising FI explanations.

Original: Past works primarily supervise gradient-based [42] or attention-basedmodel explanations [60, 48, 17, 9].
Revised: Past works primarily supervise gradient-based [39] or attention-basedmodel explanations [57, 45, 16, 9].

Original: For example, Ross et al.
Revised: For example, Ross et al.

Original: [42] enforce an ℓ 2 norm on the gradientof the loss w.r.t.
Revised: [39] enforce an ℓ 2 norm on the gradientof the loss w.r.t.

Original: the model input for features marked as unimportant by a human FI explanation.
Revised: the model input for features marked as unimportant by a human FI explanation.

Original: Thismethod appears in several later works [46, 18, 47].
Revised: Thismethod appears in several later works [43, 17, 44].

Original: In a VQA setting, Selvaraju et al.
Revised: In a VQA setting, Selvaraju et al.

Original: [43] andWuand Mooney [58] align the entire input gradient with human FI.
Revised: [40] and Wu and Mooney [55] align the entire input gradient with human FI.

Original: In addition to using input gradients(termed Vanilla Gradient), we consider the Expected Gradients method [15], a computationallyefficient implementation of Integrated Gradients [50].
Revised: In addition to using input gradients (termed Vanilla Gradient), we consider the Expected Gradients method [14], a computationally efficient implementation of Integrated Gradients [47].

Original: Omission or perturbation-based approacheshave seen more limited use.
Revised: Omission or perturbation-based approaches have seen morelimited use.

Revised: Kennedy et al.

Revised: [24] regularize FI scores toward 0 for group identifiers in hate speech detection.

Original: In addition to simple leave-one-out [31] and keep-one-in methods, wepropose a differentiable version of the popular linear method, SHAP [35].
Revised: In addition to simple leave-one-out [29] and keep-one-in methods, we propose a differentiableversion of the popular linear method, SHAP [33].

Revised: For a survey of methods we refer readers to Friedrichet al.

Revised: Following the analysis of Shrestha et al.

Revised: [41], we use a random supervision baseline to showthat V IS FIS succeeds by virtue of additional supervision and not simply via model regularization.

Original: Supervised data augmentation.
Revised: Supervised data augmentation.

Original: This line of work uses human explanations to guide data augmentation, sometimes in a human-in-the-loop manner.
Revised: This line of work uses human explanations to guide data augmentation, sometimes in a human-in-the-loop manner.

Original: For instance, Teney et al.
Revised: For instance, Teney et al.

Original: [53] present LIMEexplanations to people and solicit feedback that is converted into counterexamples for model training.
Revised: [50] present LIME explanations to people and solicit feedback that is converted into counterexamples for model training.

Original: Liang et al.
Revised: Lianget al.

Original: [32] use expert natural language counterfactual explanations to manufacture new labeledinputs.
Revised: [30] use expert natural language counterfactual explanations to manufacture new labeled inputs.

Original: We build on prior work for our data augmentation objectives.
Revised: We build on prior work for our data augmentation objectives.

Original: Similar to Chang et al.
Revised: Similar to Chang et al.

Original: [7] andconcurrent work from Singla et al.
Revised: [7] andconcurrent work from Singla et al.

Original: [47], our Sufficiency objective encourages models to be accurategiven inputs with sufficient features selected according to human FI.
Revised: [44], our Sufficiency objective encourages models to be accurategiven inputs with sufficient features selected according to human FI.

Original: Our Uncertainty objectivereduces model confidence when no important features are provided, while the most related objectivesfrom Chang et al.
Revised: Our Uncertainty objectivereduces model confidence when no important features are provided, while the most related objectivesfrom Chang et al.

Original: [7] and Liu et al.
Revised: [7] and Liu et al.

Original: [34] encourage a different answer rather than an uncertain output.
Revised: [32] encourage a different answer rather than an uncertain output.

Original: Right for the Right Reason metrics.
Revised: Right for the Right Reason metrics.

Original: Only a few past works explicitly evaluate RRR metrics besidestest set accuracy.
Revised: Only a few past works explicitly evaluate RRR metrics inaddition to test set accuracy.

Original: Our metrics include the existing RRR-Suff metric [7] and our novel RRR-Invand RRR-Unc metrics.
Revised: Our metrics include the existing RRR-Suff metric [7] and our new RRRInv and RRR-Unc metrics.

Original: While explanation plausibility is regularly proposed as an RRR metric[43, 44, 59, 39, 16], we show that the relationship between accuracy and plausibility is controlled byexplanation faithfulness, meaning that plausibility on its own should not be an RRR metric.
Revised: While explanation plausibility is regularly proposed as an RRR metric[40, 41, 56, 36, 15], we show that the relationship between accuracy and plausibility is controlled byexplanation faithfulness, meaning that plausibility on its own should not be an RRR metric.

Original: Newmachine learning metrics are often justified from first principles or on the basis of a strong correlationwith some other good metric, like a human rating [37].
Revised: New machine learning metrics are often justified from first principles or on the basis of a strongcorrelation with some other good metric, like a human rating [34].

Original: Here, we assess RRR metrics on the basis oftheir correlation with OOD accuracy, while controlling for model ID accuracy.
Revised: Here, we assess RRR metrics onthe basis of their correlation with OOD accuracy, while controlling for model ID accuracy.

Original:  Terminology and Notation
Revised:  Terminology and Notation

Original: FI Explanations.
Revised: FI Explanations.

Original: We distinguish between a human explanation e and a model explanation ˜ e thatis obtained algorithmically to explain how a model arrived at its prediction for some datapoint.
Revised: We distinguish between a human explanation e and a model explanation ˜ e thatis obtained via an explanation method to explain how a model arrived at its prediction for some datapoint.

Original: Inthis paper, human explanations are real-valued annotations for input features (which are boundingbox representations for each of our datasets).
Revised: In this paper, human explanations are real-valued annotations for input features (whichare bounding box representations for each of our datasets).

Original: The score for each bounding box is an indication of itsimportance to determining the datapoint label, which could roughly be thought of as an answer to thequestion, “why did data point x receive label y ”
Revised: The score for each bounding box is anindication of its importance to determining the datapoint label.

Original: For several objectives and metrics, we binarizethe explanations, selecting a threshold based on the data distribution (see Appendix E).
Revised: For several objectives and metrics, webinarize the explanations, selecting a threshold based on the data distribution (see Appendix E).

Original: Replace Functions.
Revised: Replace Functions.

Original: Both generating and evaluating model explanations often require “hiding”input features from a model.
Revised: Both generating and evaluating model explanations often require “hiding”input features from a model.

Original: In practice, we must replace features with some baseline value [49].
Revised: In practice, we must replace features with some baseline value [46].

Original: One simple and common way to replace features is to use an all-zeros feature [30, 50, 4].
Revised: Onesimple and common way to replace features is to use an all-zeros feature [28, 47, 4].

Original: We compareamong several Replace functions to find the best function for learning from FI supervision (see
Revised: We compareamong several Replace functions to find the best function for learning from FI supervision (see

Original: Appendix B).
Revised: Appendix B).

Original: We ultimately select the All-Negative-Ones function, which replaces a bounding boxfeature vector with the all negative ones vector, {− 1 } d .
Revised: We ultimately select the All-Negative-Ones function, which replaces a bounding boxfeature vector with the all negative ones vector, {− 1 } d .

Original: We use x e to denote a version of the input xwhere features where e is 0 are replaced via our Replace function. 
Revised: We use x e to denote a version of the input xwhere features where e is 0 are replaced via our Replace function. Model Notation.

Revised: We parametrize a distribution p ( y | x ) =

Revised: f θ ( x ) for classification purposes.

Revised: Here,ˆ y = arg max y f θ ( x ) y is the model prediction, f θ ( x ) ˆ y is the predicted probability, and Y is the spaceof eligible answers, which is a large set that is shared across all questions in our VQA tasks.

Original:  Methods for Learning from Human Feature Importance Supervision
Revised:  Methods for Learning from Human Feature Importance Supervision

Original: We now describe how to optimize for several key model desiderata using human FI supervision (represented in Fig. 1).
Revised: We now describe how to optimize for several key model desiderata using human FI supervision (represented in Fig. 1).

Original: In Sec. 4.5, we give the overall objective for Visual Feature Importance Supervision(V IS FIS), which combines the objective terms below in order to improve model generalization.
Revised: In Sec. 4.5, we give the overall objective for Visual Feature Importance Supervision(V IS FIS), which combines the objective terms below in order to improve model generalization.

Original: 4.1 Accuracy Given Sufficient Information
Revised: 4.1 Accuracy Given Sufficient Information

Original: Goal.
Revised: Goal.

Original: Like Chang et al.
Revised: Like Chang et al.

Original: [7], we hope for image processing models to make accurate predictions givensubsets of image features that are sufficient for arriving at the correct label, since this suggests that amodel recognizes that the important features are in fact important.
Revised: [7], we hope for image processing models to make accurate predictions givensubsets of image features that are sufficient for arriving at the correct label, since this suggests that amodel recognizes that the important features are in fact important.

Original: Method.
Revised: Method.

Original: Access to human explanations should enable us to automatically construct sufficient inputswith some amount of unimportant information removed [7].
Revised: Access to human explanations should enable us to automatically construct sufficient inputswith some amount of unimportant information removed [7].

Original: In particular, for every input we cancreate another datapoint by using the human explanation e to Replace unimportant features, whilekeeping the same label.
Revised: In particular, for every input we cancreate another datapoint by using the human explanation e to Replace unimportant features, whilekeeping the same label.

Original: The corresponding objective is given as:
Revised: The corresponding objective is given as:

Original: [Equation]
Revised: [Equation]

Original: We term the method using L Suff "Suff-Human."

Original: This objective differs from previous instantiations [7]by virtue of the Replace function used (see Sec. 4.5).
Revised: This objective differs from previous instantiations [7] by virtue of the Replace function used (see Sec. 4.5).

Original: We compare L Suff against an unsupervisedbaseline using random feature subsets.
Revised: We compare L Suff against an unsupervised baseline using random feature subsets.

Original: That is, a random distribution D s specifies how likely it isthat we Replace a feature:
Revised: That is,a random distribution D s specifies how likely it is that we Replace a feature:

Original: [Equation]
Revised: [Equation]

Original: which, when training via SGD, is estimated using one sample per datapoint per batch.
Revised: which, when training via SGD, is estimated using one sample per datapoint per batch.

Original: 4.2 Uncertainty Given Only Unimportant Information
Revised: 4.2 Uncertainty Given Only Unimportant Information

Original: Goal.
Revised: Goal.

Original: We would prefer for a model to give uncertain outputs for inputs with no important features,meaning the model should give a near-uniform distribution over classes.
Revised: We would prefer for a model to give uncertain outputs for inputs with no important features,meaning the model should give a near-uniform distribution over classes.

Original: Since there is no evidencefor any given class, the model should not be confident the input belongs in a particular class.
Revised: Since there is no evidencefor any given class, the model should not be confident the input belongs in a particular class.

Original: Method.
Revised: Method.

Original: With this goal in mind, Chang et al.
Revised: With this goal in mind, Chang et al.

Original: [7] train models to give less confident outputs forimages with important foreground features removed.
Revised: [7] train models to give less confident outputs forimages with important foreground features removed.

Original: More specifically, they encourage the modelto predict any class except the image’s true class.
Revised: More specifically, they encourage the modelto predict any class except the image’s true class.

Original: In contrast, we penalize a KL divergence betweenthe model output distribution and a uniform distribution,
Revised: In contrast, we penalize a KL divergence betweenthe model output distribution and a uniform distribution,

Original: [Equation]
Revised: [Equation]

Original:  where Unif ( |Y| ) is the uniform distribution and u = 1 − e indicates unimportant features.
Revised: [Equation] where Unif ( |Y| ) is the uniform distribution and u = 1 − e indicates unimportant features.

Original: 4.3 Invariance to Unimportant Information
Revised: 4.3 Invariance to Unimportant Information

Original: Goal.
Revised: Goal.

Original: We would like models to be invariant to changes in an image’s unimportant features.
Revised: We would like models to be invariant to changes in an image’s unimportant features.

Original: This property is desirable because it means that a model correctly treats unimportant features as unimportant.
Revised: This property is desirable because it means that a model correctly treats unimportant features as unimportant.

Original: Method.
Revised: Method.

Original: We first describe a  data augmentation approach, then describe an FI supervision approachsimilar to past work [46, 7].
Revised: We first describe a simple data augmentation approach, then describe an FI supervisionapproach similar to past work [43, 7].

Original: In the first approach, we train a model to produce the same outputs fortwo inputs that share the same important information while differing in the unimportant informationthey contain.
Revised: In a data augmentation approach, we train a model to produce the same outputs for two inputs thatshare the same important information while differing in the unimportant information they contain.

Original: Specifically, we use e to obtain an input with both important and unimportant features,denoted by x e ∪ u = Replace
Revised: Specifically, we use e to obtain an input with both important and unimportant features, denoted byx e ∪ u = Replace( x, e ∪ u ) .

Original: ( x, e ∪ u ) . Then, we penalize the KL divergence between the outputdistributions on the two inputs x e and x e ∪ u .
Revised: Then, we penalize the KL divergence between the output distributionson the two inputs x e and x e ∪ u .

Original: The resulting objective is then:
Revised: The resulting objective is then:

Original: [Equation]
Revised: [Equation]

Original: where D u is a distribution over binary vectors.
Revised: where D u is a distribution over binary vectors.

Original: L Inv-DA is estimated with one sample like L Suff-Random .
Revised: L Inv-DA is estimated with one sample like L Suff-Random .

Original: In an FI supervision approach, we first obtain model explanations at the datapoint level as ˜ e =
Revised: In an FI supervision approach, we first obtain model explanations at the datapoint level as ˜ e =

Original: Explain( f θ , x, ˆ y ) , where Explain is a differentiable explanation method (possible methods describedbelow).
Revised: Explain( f θ , x, ˆ y ) , where Explain is a differentiable explanation method (possible methods describedbelow).

Original: Then we seek to directly penalize models for treating unimportant features as important.
Revised: Then we seek to directly penalize models for treating unimportant features as important.

Original: Todo so, we encourage FI scores for unimportant features to be 0 :
Revised: Todo so, we encourage FI scores for unimportant features to be 0 :

Original: [Equation]
Revised: [Equation]

Original: (5) where ˜ e u is the subset of the explanation over features marked as unimportant by e .
Revised: where ˜ e u is the subset of the explanation over features marked as unimportant by e .

Original: Past work usesan ℓ 2 distance for this objective [46, 7], while we use an ℓ 1 penalty after normalizing explanations tounit length, since explanations from different FI methods have different scales.
Revised: Past work usesan ℓ 2 distance for this objective [43, 7], while we use an ℓ 1 penalty after normalizing explanations tounit length, since explanations from different FI methods have different scales.

Original: We consider a few options for differentiable explanation methods.
Revised: We consider a few options for differentiable explanation methods.

Original: Past work has primarily usedgradient-based [42, 43, 46, 18, 58, 7] and attention-based explanations [60, 48, 17, 9].
Revised: Past work has primarily usedgradient-based [39, 40, 43, 17, 55, 7] and attention-based explanations [57, 45, 16, 9].

Original: We adoptexisting gradient/attention methods and provide new differentiable perturbation-based methods.
Revised: We adoptexisting gradient/attention methods and provide new differentiable perturbation-based methods.1.

Original: Gradient-based explanations.
Revised: Gradient-based explanations.

Original: One can optimize objectives involving gradient-based explanations
Revised: One can optimize objectives involving gradient-based explanationsw.r.t.

Original: w.r.t. θ by computing second derivatives like ∇ θ ∇ x f θ ( x ) in a framework like PyTorch [38]. We
Revised: θ by computing second derivatives like ∇ θ ∇ x f θ ( x ) in a framework like PyTorch [35].

Original: use a simple Vanilla Gradient method and the Expected Gradients method (see Appendix A).
Revised: Weuse a simple Vanilla Gradient method and the Expected Gradients method (see Appendix A).2.

Original: Attention-based explanations.
Revised: Attention-based explanations.

Original: We supervise bounding box attention weights in the UpDn model [2], but early experiments suggest this is not an effective method and we do not explore it further.
Revised: We supervise bounding box attention weights in the UpDn model[2], but early experiments suggest this is not an effective method and we do not explore it further.3.

Original: Perturbation-based explanations.
Revised: Perturbation-based explanations.

Original: Perturbation-based methods like SHAP [35] are very popular ex- planation methods, but have seen only limited use for FI supervision [26].
Revised: Perturbation-based methods like SHAP [33] are very popularexplanation methods, but have seen only limited use for FI supervision [24].

Original: We consider a leave-one- out method (LOO), a keep-one-in method (KOI),Average Effect, and SHAP (see Appendix A).
Revised: We consider aleave-one-out method (LOO), a keep-one-in method (KOI), and SHAP (see Appendix A).

Original: In Appendix D, we discuss limits on the compute budget used for each method during 
Revised: In Appendix D, we discuss limits on the compute budget used for each method during model training.

Original: model training.

Original: 4.4 Aligned Model and Human Feature Importance
Revised: 4.4 Aligned Model and Human Feature Importance

Original: Goal.
Revised: Goal.

Original: Alignment between human and model explanations has frequently been proposed as a goal formodels [43, 44, 59, 16].
Revised: Alignment between human and model explanations has frequently been proposed as a goal formodels [40, 41, 56, 15].

Original: In general, past works assume that model explanations are faithful , meaningthey accurately communicate a model’s internal reasoning [22].
Revised: In general, past works assume that model explanations are faithful , meaningthey accurately communicate a model’s internal reasoning [20].

Original: This assumption is necessary for thealignment between model and human explanations, termed plausibility by Jacovi and Goldberg [22],to be evidence that model reasoning is similar to human reasoning.
Revised: This assumption is necessary for thealignment between model and human explanations, termed plausibility by Jacovi and Goldberg [20],to be evidence that model reasoning is similar to human reasoning.

Original: Of course, model explanations arenot guaranteed to be faithful.
Revised: Of course, model explanations arenot guaranteed to be faithful.

Original: To the extent that they are faithful, however, encouraging explanationplausibility during training may help align model reasoning with human reasoning.
Revised: To the extent that they are faithful, however, encouraging explanationplausibility during training may help align model reasoning with human reasoning.

Original: Method.
Revised: Method.

Original: We first obtain model explanations at the datapoint level as ˜ e = Explain( f θ , x, ˆ y )
Revised: We first obtain model explanations at the datapoint level as ˜ e = Explain( f θ , x, ˆ y )

Original: (see Sec.4.3 above).
Revised: (see Sec.4.3 above).

Original: Then, we can measure the difference between ˜ e and the human explanation e using an l pdistance, cosine similarity, or a differentiable ranking function [43, 58].
Revised: Then, we can measure the difference between ˜ e and the human explanation e using an l pdistance, cosine similarity, or a differentiable ranking function [40, 55].

Original: We use a cosine similaritysince model explanations and human explanations do not share the same scale.
Revised: We use a cosine similaritysince model explanations and human explanations do not share the same scale.

Original: Our objective is thus:
Revised: Our objective is thus:

Original: [Equation]
Revised: [Equation]

Original: 4.5 Overall Objective for V IS FIS: Visual Feature Importance Supervision
Revised: 4.5 Overall Objective for V IS FIS: Visual Feature Importance Supervision

Original: We combine the supervised objective terms from above to achieve the corresponding model desideratasimultaneously.
Revised: We combine the supervised objective terms from above to achieve the corresponding model desideratasimultaneously.

Original: Following objective tuning experiments showing that Inv-FI outperforms Inv-DA (see
Revised: Following objective tuning experiments showing that Inv-FI outperforms Inv-DA (see

Original: Appendix Table 10), we use Inv-FI rather than Inv-DA, and therefore our final V IS FIS objective is: 
Revised: Appendix Table 8), we use Inv-FI rather than Inv-DA, and therefore our final V IS FIS objective is: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (7) where L Task is a standard supervised cross-entropy loss.
Revised: where L Task is a standard supervised cross-entropy loss.

Original: Besides tuning the values of λ i one ata time, we also tune the Replace function and FI method used in this objective, making sure touse comparable compute budgets across FI methods.
Revised: Besides tuning the values of λ i one at a time,we also tune the Replace function and FI method used in this objective, making sure to use comparable compute budgets across FI methods.

Original: Replace functions we consider are listed in Appendix B (results in Table 6), and FI methods in Appendix A (results in Table 9).
Revised: Replace functions we consider are listed in Appendix B (resultsin Table 6), and FI methods in Appendix A (results in Table 7).

Original: Followingtuning, we find that it is preferable to Replace bounding box representations with the negative onesvector, {− 1 } d , and surprisingly, we find that  Vanilla Gradient  is the best performing FI method,surpassing all perturbation-based methods as well as the Expected Gradients method.
Revised: Following tuning, we find that itis preferable to Replace bounding box representations with the negative ones vector, {− 1 } d , andsurprisingly, we find that the Vanilla Gradient FI method is the best performing in terms of validationaccuracy, surpassing all perturbation-based methods as well as the Expected Gradients method.

Original:  Metrics
Revised:  Metrics

Original: Next, we describe the RRR and explanation metrics for each of our model desiderata outlined above.
Revised: Next, we describe the RRR and explanation metrics for each of our model desiderata outlined above.

Original: We also measure model ID and OOD accuracy (distribution shifts described in Sec. 6).
Revised: We also measure model ID and OOD accuracy (distribution shifts described in Sec. 6).

Original: As with themodel objectives, we use the All-Negative-Ones Replace function as needed.
Revised: As with themodel objectives, we use the All-Negative-Ones Replace function as relevant.

Original: RRR-Suffiency.
Revised: RRR-Suffiency.

Original: We measure model accuracy on inputs containing only features selected as importantby their respective human explanation (similar to [7]).
Revised: We measure model accuracy on inputs containing only features selected as importantby their respective human explanation (similar to [7]).

Original: The remaining features are Replaced .
Revised: The remaining features are Replaced .

Original: RRR-Uncertainty.
Revised: RRR-Uncertainty.

Original: We propose to measure how uncertain the model prediction is given only unimportant information.
Revised: We propose to measure how uncertain the model prediction is given only unimportant information.

Original: Specifically, we report the average model predicted probability when we provideonly unimportant features to the model (according to the human explanation), so lower is better.
Revised: Specifically, we report the average model predicted probability when we provideonly unimportant features to the model (according to the human explanation), so lower is better.

Original: RRR-Invariance.
Revised: RRR-Invariance.

Original: We propose to calculate the agreement between model predictions with the inputx e and three x e ∪ u that each include a random number of unimportant features.
Revised: We propose to calculate the agreement between model predictions with the inputx e and three x e ∪ u that each include a random number of unimportant features.

Original: The final metric isaveraged over three random u for each test point, then over all test points.
Revised: The final metric isaveraged over three random u for each test point, then over all test points.

Original: Explanation Plausibility.
Revised: Explanation Plausibility.

Original: Our explanation plausibility metric is the Spearman’s rank correlationbetween the human and model feature importance vectors, similar to past work [11, 43].
Revised: Our explanation plausibility metric is the Spearman’s rank correlationbetween the human and model feature importance vectors, similar to past work [11, 40].

Original: We usecontinuous FI estimates in order to calculate the rank correlation.
Revised: We usecontinuous FI estimates in order to calculate the rank correlation.

Original: A rank correlation is preferrablehere because human and model FI explanations do not lie in the same space.
Revised: A rank correlation is preferrablehere because human and model FI explanations do not lie in the same space.

Original: Explanation Faithfulness.
Revised: Explanation Faithfulness.

Original: We use two standard faithfulness metrics [13].
Revised: We use two standard faithfulness metrics [12].

Original: Sufficiency measureswhether keeping important features (according tomodel explanation ˜ e ) leads the model to retainits confidence in its original prediction:
Revised: Sufficiency measures whether keeping important features (according to ˜ e ) leads the model to retain its confidence in its original prediction:

Original: Suff ( f θ , x, ˜ e ) =
Revised: Suff ( f θ , x, ˜ e ) =

Original: f θ ( x ) ˆ y − f θ ( x ˜ e ) ˆ y .
Revised: f θ ( x ) ˆ y − f θ ( x ˜ e ) ˆ y .

Original: Comprehensivenessmeasures whether removing important features from an input leads to a decline in model confidence,
Revised: Comprehensiveness measures whether removing important features from an input leads to a decline in model confidence,

Original: Comp ( f θ , x, ˜ e ) =
Revised: Comp ( f θ , x, ˜ e ) =

Original: f θ ( x ) ˆ y − f θ ( x C ˜ e )
Revised: f θ ( x ) ˆ y − f θ ( x C ˜ e )

Original: ˆ y , where x C ˜ e
Revised: ˆ y , where x C ˜ e

Original: = Replace( x, 1 − ˜ e ) is the complement of featuresin x ˜ e .
Revised: = Replace( x, 1 − ˜ e ) is the complement of features inx ˜ e .

Original: We average these score over several sparsity levels of ˜ e , keeping or removing the top 10%,25%, or 50% of features [13].
Revised: We average these score over several sparsity levels of ˜ e , keeping or removing the top 10%, 25%, or50% of features [12].

Original: Note we compute these metrics using the best available explanationmethod per dataset, as measured by explanation faithfulness (comparison in Appendix G).
Revised: Note we compute these metrics using the KOI explanation method, which yieldsthe most faithful explanations out of all the methods in Appendix A (comparison in Appendix G).

Original:  Experiment Setup
Revised:  Experiment Setup

Original: [Table]

Original: [Table]

Original: Datasets.
Revised: Datasets.

Original: We perform experiments on threebenchmark datasets: CLEVR-XAI [5], GQA[20], and VQA-HAT [11].
Revised: We perform experiments on three benchmark datasets: CLEVR-XAI [5], GQA [18],and VQA-HAT [11].

Original: CLEVR-XAI isalgorithmically generated  based on CLEVR[25] and provides ground truth visual segmentation masks for each question.
Revised: CLEVR-XAI is an algorithmically generated dataset based on CLEVR [23]and provides ground truth visual segmentation masks for each question.

Original: CLEVR-XAIis limited in visual varieties and vocabularies, but it offers FI supervision in a controlled, low-noisesetting.
Revised: CLEVR-XAI is limited invisual varieties and vocabularies, but it offers FI supervision in a controlled, low-noise setting.

Original: GQA contains compositional reasoning questions over naturalistic images.
Revised: GQAfeatures compositional reasoning questions over naturalistic images.

Original: GQA alsoincludes the program for generating the questions and the ground-truth scene graph from the Visual Genome dataset [28].
Revised: GQA also includes the programfor generating the questions and the ground-truth scene graph from the Visual Genome dataset [26].

Original: This allows us to obtain bounding boxes of relevant objects identifiedthrough the question program, which we use as FI supervision.
Revised: This allows us to obtain bounding boxes of relevant objects identified through the question program,which we use as FI supervision.

Original: VQA-HAT is based on VQAv1 [3],including naturalistic images and questions with mouse tracking de-blurring used to collect image FI annotations from humans.
Revised: VQA-HAT is based on VQAv1 [3], including naturalistic imagesand questions with mouse tracking de-blurring used to collect image FI annotations from humans.

Original: For VQA, we report model performance on the more challenging othertype questions as recommended by Teney et al.
Revised: [Table]

Original: Distribution Shifts.
Revised: Distribution Shifts.

Original: We create both ID and OOD test sets for each dataset, so we always have fourdata splits: Train, Dev, Test-ID, and Test-OOD (see Table 1).
Revised: We create both ID and OOD test sets for each dataset, so we alwayshave four data splits: Train, Dev, Test-ID,and Test-OOD (split sizes shown in Table1).

Original: To obtain OOD data, we use distributionshifts similar to those in VQA-CP, which are intended to vary the linguistic bias between ID and OODsplits [1].
Revised: To obtain OOD data, we use distributionshifts similar to those in VQA-CP, which areintended to vary the linguistic bias between ID and OOD splits [1].

Original: In detail, we create groups of questions according to the first few words in each question(indicating the type of question), and allocate groups unevenly into ID and OOD sets, randomly assigning 80% of each group to one set and 20% to the other.
Revised: In detail, we create groups of questions according to the first few words ineach question (indicating the type of question), and allocate groups unevenly into ID and OOD sets,randomly assigning 80% of each group to one set and 20% to the other.

Original: The ID set is split into Train, Dev, and Test-ID.
Revised: The ID set is split into Train, Dev, and Test-ID.

Original: Model  is selected according to Dev set performance.
Revised: Model selection is done according to Dev set performance.

Original: We downsample the very large GQA dataset fromto about 101k for training and 20k for other splits.

Original: See Appendix Fig.

Original: 4 for training size ablation analysis.

Original: We note that we avoid several pitfalls in evaluating VQA models against distribution shifts, asoutlined by Teney et al.
Revised: We note that we avoidseveral pitfalls in evaluating VQA models against distribution shifts, as outlined by Teney et al.

Original: See Appendix Table 7 for sensitivity analysis with randomly resplit data.

Original: Human Feature Importance.
Revised: Human Feature Importance.

Original: For all datasets, we obtain human FI scores at the bounding box (BB)level for detected BBs from the Faster-RCNN detector [41].
Revised: For all datasets, we obtain human FI scores at the bounding box(BB) level for detected BBs from the Faster-RCNN detector [38].

Original: For both VQA-HAT and CLEVR-XAIwe obtain importance scores from pixel-level annotations following Selvaraju et al.
Revised: Following Selvaraju et al. [40],for both VQA-HAT and CLEVR-XAI we obtain importance scores from pixel-level annotations ass k = E ki / ( E ki + E ko ) , where s k is the score for the k th detected BB and E ki and E ko are the averagepixel-level importance score inside and outside the BB, respectively.

Original: VQA-HAThas real-valued pixel-level scores, while for CLEVR-XAI, we set the pixel-level score to 1 for pixelswithin the segmentation mask and 0 elsewhere.
Revised: VQA-HAT has real-valuedpixel-level scores, while for CLEVR-XAI, we set the pixel-level score to 1 for pixels within thesegmentation mask and 0 elsewhere.

Original: For GQA, since we have BB level annotations, wecalculate the importance score based on the intersection over union (IoU) between ground-truthimportant BBs and detected BBs.
Revised: For GQA, since we have BB level annotations, we calculate theimportance score based on the intersection over union (IoU) between ground-truth important BBsand detected BBs: s k = max l ∈G IoU ( B kd , B l gt ) where B kd is the BB of the k th detected object and B l gt is the BB of the l -th ground-truth object.

Original: We manually set a threshold on importance scores to determine important and unimportant objects (0.85, 0.55, and 0.3 for CLEVR-XAI, VQA-HAT, and GQArespectively).
Revised: With importance scores for each BB, we manually set athreshold for determining important and unimportant objects (0.85, 0.55, and 0.3 for CLEVR-XAI, VQA-HAT, and GQA respectively).

Original: See Appendix E fordetails on obtaining FI and sensitivity analysis for thresholds.
Revised: See Appendix E for sensitivity analysis for this threshold.

Original: Models.
Revised: Models.

Original: We run experiments with UpDn [2] and LXMERT [52].
Revised: We run experiments with UpDn [2] and LXMERT [49].

Original: Both models rely on bounding boxrepresentations generated by a pretrained Faster R-CNN model [41] (further details in Appendix F).
Revised: Both models rely on bounding boxrepresentations generated by a pretrained Faster R-CNN model [38] (further details in Appendix F).

Original: Hypothesis Testing.
Revised: Hypothesis Testing.

Original: We conduct hypothesis tests via a bootstrap resampling model seeds anddatapoints 10k times [14].
Revised: We conduct hypothesis tests via a bootstrap resampling model seeds anddatapoints 10k times [13].

Original: We obtain 95% confidence intervals in the same way.
Revised: We obtain 95% confidence intervals in the same way.

Original:  Experiment Results
Revised:  Experiment Results

Original: 7.1 Can FI Supervision Improve Model Accuracy for VQA?
Revised: 7.1 Can FI Supervision Improve Model Accuracy for VQA?

Original: Design.
Revised: Design.

Original: Using UpDn on our three datasets, we compare V IS FIS with previous state-of-the-art FIsupervision methods for VQA tasks [43, 58] as well as for image classification [46, 7].
Revised: Using UpDn on our three datasets, we compare V IS FIS with previous state-of-the-art FIsupervision methods for VQA tasks [40, 55] as well as for image classification [43, 7].

Original: We giveresults for LXMERT only on CLEVR-XAI, since GQA and VQA are a part of the pretrainingdata for LXMERT [52].

Original: Note we test on the more challenging other type questions only for VQA,following Teney et al.

Original: Selvaraju et al. [43] use L align with a ranking loss to align Vanilla Gradient explanations and human FI supervision.
Revised: Selvaraju et al.[40] use L align with a ranking loss to align Vanilla Gradient explanations and human FI supervision.

Original: Wu and Mooney [58] propose a relaxed version ofthe ranking loss that binarizes important and unimportant features according to human FI supervisionand encourages higher model FI for important objects than unimportant ones.
Revised: Wu and Mooney [55] propose a relaxed version of the ranking loss that binarizes important andunimportant features according to human FI supervision and encourages higher model FI forimportant objects than unimportant ones.

Original: The other methodsweconsider all use an L FI-Inv objective with an l 2 penalty on Vanilla Gradient explanations.
Revised: The image classification methods use an L FI-Inv objectivewith an l 2 penalty on Vanilla Gradient explanations.

Original: On top ofthis, Chang et al.
Revised: Additionally, Chang et al.

Original: [7] add an L Suffobjective with a Shuffle Replace function that randomly permutesfeatures rather than replacing them, to preserve the marginal data distribution, and Singla et al. [47]add an L Suff objective with a Gaussian noise Replace function.
Revised: [7] use L Suff with a Shuffle Replace function that randomly permutes features rather than replacing them, to preserve themarginal data distribution.

Original: Our unsupervised baselines aremodels trained with only label supervision or using L Suff-Random .
Revised: Our unsupervised baselines are models trained with only label supervisionor using L Suff-Random .

Revised: We also test methods on CLEVR-XAI using a LXMERT model to check forgeneralizability of our takeaways with a different base model.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Results.
Revised: Results.

Original: We show results for UpDn in Table2 and for LXMERT in Table 3.
Revised: We show results for UpDn in Table2 and for LXMERT in Table 3.

Original: First, we findthat FI supervision can meaningfully improvemodel accuracy.
Revised: First, we findthat FI supervision can meaningfully improvemodel accuracy.

Original: With UpDn on CLEVR-XAI,
Revised: With UpDn on CLEVR-XAI,

Original: V IS FIS improves ID accuracy by 1.1 points (±0.5;p = 1e-4) and OOD accuracy by 4.7 points (±1.4;p< 1e-4) over the strongest baseline withoutsupervision, Suff-Random 
Revised: V IS FIS improves ID accuracy by 1.1 points (±0.5;p = 1e-4) and OOD accuracy by 4.7 points (±1.4;p< 1e-4) over the strongest baseline withoutsupervision, Suff-Random (trends are similar with

Original: (see Appendix Fig.8 for breakdown in improvements by CLEVRquestion type). Trends are similarly positive onthe other datasets and with LXMERT, where
Revised: LXMERT). This improvement does not persistwhen using random explanations (last Next, we observe that V IS FIS is the best overall method on CLEVR-XAI and GQA-100k.

Original:  V IS FIS outperforms the baseline by 0.48 points(±0.35; p< .01) on ID data and1.07 points( p< 1e-4) on OOD data (for results on all VQA question types, see Appendix Table 14).
Revised: On GQA, V IS FIS improves ID accuracy by 2.71 points and OOD accuracy by 2.89 points over Simpson et al.[43]. On VQA, however, no supervised method outperforms Suff-Random.

Original: Theseimprovements do not persist when using random explanations (last row), meaning they are causedby the human supervision. Finally, we observe that V IS FIS is the best overall method across datasets and architectures, as other methods typically do not improve accuracy over an unsupervised baseline. The next best method is thatof Singla et al.
Revised: Since we know that

Original: [47], which improves over an unsupervised baselineonly for the GQA dataset with UpDn, but V IS FIS still outperforms Singla et al. [47] there by 2.39(±0.55; p< 1e-4) points on ID data and 2.31 points (±0.66; p< 1e-4) on OOD data.
Revised: V IS FIS can benefit from human supervision, this seems to be an artifact of the VQA dataset. It’spossible that this difference stems from the fact that QA pairs for VQA are manually written withseveral statistical biases observed in past work [25].

Original: 7.2 How Does FI Supervision Improve Accuracy?
Revised: 7.2 How Does FI Supervision Improve Accuracy?

Original: [Figure]
Revised: [Figure]

Revised: Trendlines are logistic regressions.

Revised: When explanations are more faithful, their alignment with human explanations better correlates with model accuracy.

Original: Design.
Revised: Design.

Original: Past work hypothesizes that FI supervi- sion improves accuracy by aligning model andhuman FI [43, 7].
Revised: Past work hypothesizes that FI supervi- sion improves accuracy by aligning model and human FI [40, 7].

Original: Surprisingly, we find that therelationship between model test accuracy and average explanation plausibility is fairly weak (linear correlation on UpDn+CLEVR-XAI modelsis ρ = 0.14±0.19).
Revised: Surprisingly, we find that the relationship between model test accuracy and av- erage explanation plausibility is fairly weak (linear correlation on UpDn+CLEVR-XAI modelsis ρ = 0.14±0.19).

Original: Here, we argue that plausibleexplanations alone are not evidence of correctmodel reasoning, but plausible and faithful explanations are.
Revised: Here, we argue that plausibleexplanations alone are not evidence of correctmodel reasoning, but plausible and faithful explanations are.

Original: Using 4 million ID/OOD testpredictions from UpDn+CLEVR-XAI models,we visualize trendlines from logistic regressionspredicting model accuracy based on plausibilityand faithfulness at the datapoint level, groupedinto Worst, Middle, and Best faithfulness categories based on Sufficiency/Comprehensivenessmetrics (see Appendix E).
Revised: Using 4 million ID/OOD testpredictions from UpDn+CLEVR-XAI models,we visualize trendlines from logistic regressionspredicting model accuracy based on plausibilityand faithfulness at the datapoint level, groupedinto Worst, Middle, and Best faithfulness categories based on Sufficiency/Comprehensivenessmetrics (see Appendix E).

Original: Results.
Revised: Results.

Original: Fig.
Revised: Fig.

Original: 2 shows that as an explanation for a datapoint becomes more plausible, the modelis more likely to correctly predict that point’s label, but only when the explanation is also faithful .
Revised: 2 shows that as an explanation for a datapoint becomes more plausible, the modelis more likely to correctly predict that point’s label, but only when the explanation is also faithful .

Original: Indeed, a maximally plausible and faithful explanation has about a 90% chance of being correct,while a minimally plausible but highly faithful explanation has closer to a 12.5% of being correct.
Revised: Indeed, a maximally plausible and faithful explanation has about a 90% chance of being correct,while a minimally plausible but highly faithful explanation has closer to a 12.5% of being correct.

Original: Forunfaithful explanations, plausibility has essentially no relationship with accuracy.
Revised: Forunfaithful explanations, plausibility has essentially no relationship with accuracy.

Original: Though these trendsare not necessarily causal, they are consistent with the view that when model predictions are correct,it is because their true reasoning (as revealed by faithful explanations) aligns with human reasoning.
Revised: Though these trendsare not necessarily causal, they are consistent with the view that when model predictions are correct,it is because their true reasoning (as revealed by faithful explanations) aligns with human reasoning.

Original: 7.3 Which FI Supervision Objectives Improve Accuracy?
Revised: 7.3 Which FI Supervision Objectives Improve Accuracy?

Revised: [Table] [Table]

Original: Design.
Revised: Design.

Original: We ablate across objective terms from Sec.
Revised: We ablate across objective [Table]

Original:  4 for UpDn on CLEVR-XAI.
Revised: terms from Sec. 4 for UpDn on CLEVR-XAI.

Revised: The weight for each

Original: The weight foreach objective term is tuned while using  only that objective, then kept fixed when objectives arecombined (further details in Appendix F).
Revised: [Table] objective term is tuned while using [Table] only that objective, then kept fixedwhen objectives are combined (further details in Appendix F).

Original: We consider another kind of ablation experiment where weuse random supervision for one objective at a time in V IS FIS, with results in Appendix Table 11.

Original: Results.
Revised: Results.

Original: In Table 4, we find that each individual objective is valuable on its own, and they do wellwhen combined.
Revised: In Table 4, we find thateach individual objective is valuableon its own, and they do well whencombined.

Original: Relative to the Baseline OOD accuracy, Suff-Human adds 4.1 points, Unc adds 1.54points, Inv-FI adds 2.08 points, and Align adds 4.81 points.
Revised: Relative to the Baseline OOD accuracy, Suff-Human adds 4.1points, Unc adds 1.54 points, Inv-FIadds 2.08 points, and Align adds 4.81points.

Original: When the four objectives are combinedin V IS FIS, the improvement rises to 6.98 points.7.4 Can FI Supervision Make Models Right for the Right Reasons? [Table] [Table] [Table]
Revised: When the four objectives are combined in V IS FIS, the improvement rises to 6.98 points.

Original: [Table] [Table] [Table] [Table]
Revised: 7.4 Can FI Supervision Make Models Right for

Original: [Table]

Original: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: the

Original: 27 + Align

Original: [Table]

Original: [-[Table] [Table] [Table] 73 + Unc 72.33 41.
-]
Revised: [-[Table] [Table] [Table] 73 + Unc 72.33 41.
-]Right

Original: [-[Table] [Table] [Table] 41 V IS FIS 72.82 43.78 76.
-]
Revised: [-[Table] [Table] [Table] 41 V IS FIS 72.82 43.78 76.
-]Reasons?

Original: [Table]

Original: [Table]

Original: Design.
Revised: Design.

Original: We report RRR metricsas well as explanation plausibilityfor the UpDn+CLEVR-XAI modelsfrom our objective ablation above.
Revised: We report RRR metrics as well as explanation plausibility for the UpDn+CLEVR-XAImodels from our objective ablation above.

Original: Results.
Revised: Results.

Original: In Table 4, we find that FI supervision generally improves RRR metric scores.
Revised: In Table 4, we find that FI supervision generally improves RRR metric scores.

Original: Comparedto the Baseline, V IS FIS achieves27.8 points better Sufficiency, 13.8points better Invariance, and 11.5points better Uncertainty.
Revised: Compared tothe Baseline, V IS FIS achieves 27.8 points better Sufficiency, 13.8 points better Invariance, and 11.5points better Uncertainty.

Original: The bestunsupervised method closes the gapslightly on RRR-Suff and RRR-Inv.
Revised: The best unsupervised method closes the gap slightly on RRR-Suff and RRR-Inv.

Original: Specifically, Suff-Random is only 3.97 points worse than Suff-Human on RRR-Suff, and only 0.53points worse than Inv-DA on RRR-Inv.
Revised: Specifically, Suff-Random is only 3.97 points worse than Suff-Human on RRR-Suff, andonly 0.53 points worse than Inv-DA on RRR-Inv.

Original: These results suggest that FI supervision noticeably improves RRR metrics, meaning model behavior better fulfills the theoretical desiderata from Sec. 4.
Revised: These results suggest that FI supervision noticeablyimproves RRR metrics, meaning model behavior better fulfills the theoretical desiderata from Sec. 4.

Original: 7.5 Do RRR Metrics Predict OOD Generalization?
Revised: 7.5 Do RRR Metrics Predict OOD Generalization?

Original: Design.
Revised: Design.

Original: We measure the correlation between RRR metrics (calculated with ID data) and OODaccuracy across a large set of models.
Revised: We measure the correlation between RRR metrics (calculated with ID data) and OOD accuracy across a large set of models.

Original: We report results here for all UpDn models on CLEVR-XAI,with similar results for GQA/VQAand LXMERT given in Appendix Table 15.
Revised: We report results here for all UpDn models on CLEVR-XAI, withsimilar results for GQA/VQA given in Appendix G.

Original: We consider a fewpossible model metrics, including several composite metrics that combine model-level metrics.
Revised: We consider a few possible model metrics, including several composite metrics that combine model-level metrics.

Original: To optimally weight the individual metrics, we fit statistical models to the data that predict OODaccuracy given the available metrics.
Revised: To optimally weight the individualmetrics, we fit statistical models to the data that predict OOD accuracy given the available metrics.

Original: Since this risks overfitting the composite metrics to the datawe have, we perform a cross-validation resampling model-level statistics 10k times, using 90 models’metrics as training data and 10 for testing each time.
Revised: Since this risks overfitting the composite metrics to the data we have, we perform a cross-validationresampling model-level statistics 10k times, using 90 models’ metrics as training data and 10 fortesting each time.

Original: The final metrics we consider are: (1) IDaccuracy on its own as a baseline, (2) RRR metrics on their own, (3) ID accuracy plus averagemodel confidence, (4) ID accuracy plus explanation metrics (for plausibility and faithfulness), (5) ID accuracy plus RRR metrics, and (6) All Metrics, which uses all available metrics.
Revised: The final metrics we consider are: (1) ID accuracy on its own as a baseline, (2) RRRmetrics on their own, (3) ID accuracy plus average model confidence, (4) ID accuracy plus explanationmetrics, (5) ID accuracy plus RRR metrics, and (6) All Metrics, which uses all available metrics.

Original: [Table]
Revised: [Table]

Original: Results.
Revised: Results.

Original: In Table 5, we show the average correlations between each metric and model OOD accuracyachieved in our cross-validation.
Revised: In Table 5, we show the average correlations between each metric and model OOD accuracyachieved in our cross-validation.

Original: Interestingly, wefind that RRR metrics do not achieve a better correlation with OOD accuracy than ID accuracydoes on its own .
Revised: Interestingly, wefind that RRR metrics do not achieve a better correlation with OOD accuracy than ID accuracydoes on its own .

Original: ID accuracy alone has a correlationof 0.863 with OOD accuracy, while using IDAcc +
Revised: ID accuracy alone has a correlation of 0.861 with OOD accuracy, while using ID

Original:  RRR metrics achieves a correlation of 0.852.
Revised: Acc + RRR metrics achieves a correlation of 0.85.

Original: In fact,the only additional metric that improves one’s ability to predict OOD accuracy is the average modelconfidence on ID data (more confident models haveslightly better OOD accuracy), though this does nothold for LXMERT models (see Appendix Table 15).
Revised: In fact, the only additional metric that improvesone’s ability to predict OOD accuracy is the averagemodel confidence on ID data (more confident models have slightly better OOD accuracy). A compositemetric based on ID accuracy and model confidenceachieves a correlation of 0.907 with OOD acc, improving over ID accuracy on its own by 0.046 points (±.0013; p< 1e-4).

Original: These results cast doubt on the value of RRR metrics.
Revised: These results cast doubt onthe value of RRR metrics.

Original: If ID accuracy on its own is a better predictor of OOD accuracy than RRR metrics, then RRRmetrics may not be a better measure of the quality of model reasoning than ID accuracy is.
Revised: If ID accuracy on its own is a better predictor of OOD accuracy than RRRmetrics, then RRR metrics may not be a better measure of the quality of model reasoning than IDaccuracy is.

Original: Webelieve the RRR metrics considered in this paper are still theoretically justified as model desiderata,but we cannot recommend them as measures of model generalization to OOD data.
Revised: We believe the RRR metrics considered in this paper are still theoretically justified asmodel desiderata, but we cannot recommend them as measures of model generalization to OOD data.

Original:  Discussion & Conclusion
Revised:  Discussion & Conclusion

Original: Limitations.
Revised: Limitations.

Original: Though we evaluate with two standard model architectures and three datasets, ourconclusions may be limited to settings using Faster-RCNN [41] bounding box representations as thefeature space rather than pixel space.
Revised: Though we evaluate with two standard model architectures and three datasets, ourconclusions may be limited to settings using Faster-RCNN [38] bounding box representations as thefeature space rather than pixel space.

Original: Additionally, though we follow existing guidelines with ourdistribution shifts [11, 54], we do not measure model generalization across all typical kinds of shifts[40].
Revised: Additionally, though we follow existing guidelines with ourdistribution shifts [11, 51], we do not measure model generalization across all typical kinds of shifts[37].

Original: Lastly, we note that FI supervision methods are limited by the need for additional annotations.
Revised: Lastly, we note that FI supervision methods are limited by the need for additional annotations.

Original: Ethics.
Revised: Ethics.

Original: We hope that our findings regarding model accuracy and explanation plausibility/faithfulnesswill help dispel the notion that models reason like humans (or are more grounded) simply becausemodel explanations look similar to human explanations, which can cause unwarranted trust in MLmodels [23].
Revised: We hope that our findings regarding model accuracy and explanation plausibility/faithfulnesswill help dispel the notion that models reason like humans (or are more grounded) simply becausemodel explanations look similar to human explanations, which can cause unwarranted trust in MLmodels and is therefore an ethical risk [21].

Original: We do not foresee specific ethical risks arising from this work that do not already applyto the general use of machine learning for visual question answering tasks, such as the potentialdeployment of ML models in settings where they may harm people [56, 51].
Revised: We do not foresee any other ethical risks arising from thiswork that do not already apply to the general use of machine learning for visual question answeringtasks, such as the potential deployment of ML models in settings where they may harm people [53, 48].

Original: Conclusions.
Revised: Conclusions.

Original: In this paper, we show that (1) FI supervision can improve VQA model accuracyvia our V IS FIS method, (2) accuracy improvements appear to stem from improving explanationplausibility specifically for faithfully explained data, (3) FI supervision can improve RRR metricperformance, and (4) RRR metrics do not actually correlate well with OOD accuracy. 
Revised: In this paper, we show that (1) FI supervision can improve VQA model accuracyvia our V IS FIS method, (2) accuracy improvements appear to stem from improving explanationplausibility specifically for faithfully explained data, (3) FI supervision can improve RRR metricperformance, and (4) RRR metrics do not actually correlate well with OOD accuracy. Checklist For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Revised: contributions

Revised: and scope?

Revised: [Yes] All

Revised: claims made

Revised: in the

Revised: abstract and

Revised: introduction are

Revised: properly supported by our paper’s results and contributions.

Revised: (b) Did you describe the limitations of your work?

Revised: [Yes] See Section 8.

Revised: (c) Did you discuss any potential negative societal impacts of your work?

Revised: [Yes] Also seeSection 8.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Revised: [Yes]2.

Revised: If you are including theoretical results...

Revised: (a) Did you state the full set of assumptions of all theoretical results?

Revised: [N/A](b)

Revised: Did you include complete proofs of all theoretical results?

Revised: [N/A]3.

Revised: If you ran experiments...

Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Revised: [Yes] The code andinstructions for reproducing the main results are in supplement.

Revised: The datasets are freelyavailable online (urls in Appendix E).

Revised: (b) Did you specify all the training details (e

Revised: g., data splits, hyperparameters,

Revised: how they

Revised: were chosen)?

Revised: [Yes] See Section 6 and Appendix E and F.

Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Revised: [Yes] All main experiments are conducted with 5 random seeds.

Revised: Hypothesis testing are conducted with 10k bootstrapping.

Revised: See Section 7.

Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Revised: [Yes] See Appendix F.

Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Revised: (a) If your work uses existing assets, did you cite the creators?

Revised: [Yes] All creators of

Revised: existing assets are properly cited.

Revised: (b) Did you mention the license of the assets?

Revised: [Yes] All assets are used in accordance with

Revised: their licenses (listed in Appendix E).

Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [N/A]

Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Revised: using/curating?

Revised: [N/A]

Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Revised: information or offensive content?

Revised: [Yes] See Section 8.

Revised: If you used crowdsourcing or conducted research with human subjects...

Revised: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?

Revised: [N/A]

Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Revised: Board (IRB) approvals, if applicable?

Revised: [N/A]

Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Revised: spent on participant compensation?

Revised: [N/A]