Introspective Distillation for Robust Question Answering
Authors: Yulei Niu
Hanwang Zhang
Venue: NeurIPS 2021 Poster (2021)
Keywords: VQA
extractive QA
introspection
distillation
trade-off

Abstract: Question answering (QA) models are well-known to exploit data bias, e.g., the language prior in visual QA and the position bias in reading comprehension. Recent debiasing methods achieve good out-of-distribution (OOD) generalizability with a considerable sacrifice of the in-distribution (ID) performance. Therefore, they are only applicable in domains where the test distribution is known in advance. In this paper, we present a novel debiasing method called Introspective Distillation (IntroD) to make the best of both worlds for QA. Our key technical contribution is to blend the inductive bias of OOD and ID by introspecting whether a training sample fits in the factual ID world or the counterfactual OOD one. Experiments on visual QA datasets VQA v2, VQA-CP, and reading comprehension dataset SQuAD demonstrate that our proposed IntroD maintains the competitive OOD performance compared to other debiasing methods, while sacrificing little or even achieving better ID performance compared to the non-debiasing ones.

Introspective Distillation for Robust Question Answering
Yulei Niu Nanyang Technological University yn mp.yyleeniuuggail.co com
Hanwang Zhang Nanyang Technological University
hanwangzhangöntu.c edu.
Abstract
Question answering (QA) models are well-known to exploit data bias, e.g., the language prior in visual QA and the position bias in reading comprehension. Recent debiasing methods achieve good out-of-distribution (OOD) generalizability with considerable sacrifice of the in-distribution (ID) performance. Therefore, they are only applicable in domains where the test distribution is known in advance. In this paper, we present a novel debiasing method called Introspective Distillation (IntroD) to make the best of both worlds for QA. Our key technical contribution is to blend the inductive bias OOD and ID by introspecting whether a training sample fits in the factual ID world the counterfactual OOD one. Experiments visual QA datasets VQA v2, VQA-CP, and reading comprehension dataset SQuAD demonstrate that our proposed IntroD maintains the competitive OOD performance compared to other debiasing methods, while sacrificing little or even achieving better ID performance compared to the non-debiasing ones.
Introduction
+
Bracine
Debining Melscds
# Oun
Accuracy (VDA vo)
Question answering (QA), which requires machines to answer questions given a context, is one of the most fundamental AI tasks. Popular contexts are vision (e.g.. image for VQA 55) and natural language (e.g., pas- sage for extractive QA (27). A common observation is that QA models prefer to over-exploit the training bias, which bypasses the context comprehension for short- answer. For example, by only using the linguistic correlations between questions and answers, VQA mod- els can answer most questions correctly [6 2 20].
CFVOx-faiND
Iserob
S-MRL
LMIH
CSS 171
50
000 Acturacy (VGA-CP v2 test)
Figure 1: Recent debiasing methods achieve high OOD accuracy with the sacri- fice of ID accuracy. Our proposed IntroD makes the best of both worlds.
Similarly, extractive QA models may use the spurious positional cues to locate the answer in the passage [22]. result, QA models that have already achieved strong in in-distribution (ID) performance may inevitably fail in out-of-distribution (OOD)
test scenarios, regardless of the scale of training data and models 14
22 37
Recently, several debiasing methods aim to close the gap between the ID and OOD performances [6] 25). However, many of them hold the assumption that the training and test distributions are very different or even reversed, e.g., if there are more "yes" answers in training, there must be more "no" answers testing. As result, these methods encounter severe performance drop under the ID evaluation, although they significantly outperform non-debiasing baselines in terms of OOD performance. An interesting observation from Figure iss that non-debiasing methods (circles) obtain high ID but low OOD performance, while debiasing methods (squares) achieve high OOD but low ID performance. This observation motivates us to ask: can we make the best of both worlds?
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Question type
Answer Distribution
Question type
Distribution
Question type
Answer Distribution
"ss...
many
"Whot color
whiee porc
Training sample
Intraspection
Training somple
Introspection
Training sample
Intraspection
000
000
OOD
Ratio
000
Ratio
10
DO0
Ratio
000
Q: is that electric oven? (G7: Yes.)
Q: How many skiers? (GT: 3.)
Q: What color the older mon's shirt? (GT: Blue.)
(a)
(b)
(c)
Figure 2: Illustration of our proposed introspection. (a) When ID inductive bias dominates the learning, the student model should listen more to the OOD-aware knowledge. inductive bias dominates the learning, the student model listens more the ID-aware knowledge. (b) When learning is fair, the student model listens to both teachers equally. The areas in the "ID" and "OOD" bars represent the proportion of the predicted probability. The areas the "Ratio" bars represent the proportion of the introspective weights (Eg.
(c) When OOD
In this paper, we take a step forward to building robust QA models that achieve strong performances in both ID and ODD evaluations. We point out that if the model is over-exploiting the bias in one world, the performance in the other one would be significantly degraded. Therefore, the "best of both" model should be fair with the inductive bias in either world. To this end, present simple yet effective training paradigm- Introspective Distillation (IntroD) to blend the inductive bias of both worlds fairly. Suppose that have two expert teacher models: MD-teacher and DOD-teacher, each of which captures the ID or OOD inductive bias and represents the corresponding world. Figure 2 illustrates three cases about how introspective student learns from the two very different teachers.
Case ID-bias OOD-bias, then ID-teacher < OOD-teacher. ID inductive bias dominates the learning, and the student should listen more to OOD-teacher. This case occurs when ID-teacher has low training loss while OOD-teacher has a high one. As shown in Figure 2 (a), hard for QA models to conclude whether the oven is electric not without additional context. Due to the inductive bias in the training data, i.e., most questions starting with "is" are answered by "yes", ID-teacher concludes with over-confidence while OOD-teacher does not.
Case 2: ID-bias < 0OD-bias, then ID-teacher>> > 00D-teacher. OOD inductive bias dominates the learning, and the student should listen more to ID-teacher. This case occurs when ID-teacher has high training loss while OOD-teacher has low one. As shown in Figure 2 (c), there are at least two older men, one in blue shirt selling fruits and one in white shirt walking in the crowd. Therefore, both "blue" and "white" should be correct. However, as most training questions starting with "what color" are labeled by "white" answer, the bias of "OOD should be different from ID" enforces OOD-teacher to downplay "white" unfairly while ID-teacher does not.
Case 3: if ID x OOD, then ID-teacher = O0D-teacher. Learning is fair and the student should listen both teachers equally. This case occurs when the training losses of the two are close. As shown in Figure 2 (b), the ID-teacher and OOD-teacher produce similar predictions.
The above introspection can be represented distilled to the student model 18] Yet, an unsolved challenge is how to obtain the "oracle" teachers, especially the OOD-teacher, because the OOD distribution unseen training, not mentioning to train a teacher model. Thanks to the recent causality- -bsed approach (25]. we can approximate the OOD-teacher using a causal model that imagines the unseen world by counterfactual reasoming.
blended knowledge of the two teachers, which is
Without loss of generality, we take visual QA and extractive QA as case studies. Experiments on VQA-CP 2 VQA v2 (16), and SQuAD (27] validate the effectiveness of our proposed IntroD. Interestingly, extensive ablations demonstrate that the success of IntroD indeed from the causal introspection but not from the simple ensemble.
2 Related Work
Visual Question Answering (VQA) 5
16 is to answer the question given a visual context, i.e. image. Traditional VQA models are found to exploit the language priors in the training data [16]
2, 20)]. For example, in the first version of the VQA dataset VQA v1.0, about 40% of the sports-

---

related questions are answered as "tennis". Although utilizing the shortcut bias helps with the in-distribution (ID) performance, the out-of-distribution (OOD) one is severely hurt 2. In order to mitigate the language bias, recent methods proposed to utilize extra annotations for accurate
visual grounding 28 [33, generate synthetic data for data augmentation 7 14 30
31,, modifying
language modules [19 23 or explicitly formulate and exclude the language prior [6] 1 225 These
methods obtain significant OOD improvement on the VQA-CP [2] dataset whose answer distributions
in training and testing are reversed. However, the OOD improvement is achieved with the cost of
severe ID performance drop. Therefore, it is still a challenge to achieve strong performances in both ID and OOD evaluations.
Extractive Question Answering (extractive QA) is to answer the question given a natural language context, i.e., passage (27]. Extractive QA assumes that the answer always locates in the passage, and further reduces the generative QA task to a classification task, i..,, position prediction. Recent years
have witness many influential works R5| 29
10 39
12
38 [9. However, directly predicting the
answer positions has a severe side effect, i.e,, correlating answers with positions [22]. For example, a language model is trained on biased dataset where answers always locate in the first sentence of the passage, the model will tend to ground the answer in the first sentence. Recently, a new variant of the reading comprehensive dataset SQuAD [2]] is proposed to evaluate whether language models are robust to the position bias [22]. Similar to VQA. the answer position distribution is skewed in the training set. In this paper, we follow Ko et al. [2 to evaluate the robustness for extractive QA.
Ensemble- based methods for debiasing explicitly formulate and exclude the shortcut bias in the training data 6 7
25 g The shortcut bias can be captured by separate branch 6 or statistical priors [1. These methods are further interpreted as sussality-basedd approaches (25) However, most of these methods achieve promising performance under the out- outto--distriuution (OOD) evaluation but sacrifice the performance under the in-distribution (ID) evaluation. The reason is that these methods hold an assumption that the training and test distribution are very different or even reversed. In this paper, we implement our ID-teacher and OOD- using the causality-basel methods, and further achieve good trade-off between ID and OOD evaluations. Previous OOD-teachers, ie., causality- based methods, only generate the OOD- -oD-prrdiction for debiased inference and ignore the role of ID- ID-prediction. We further point out that the ID- ID-prediction is crucial in introspecting the training process and achieving a good trade-off between ID performance and OOD performance.
Knowledge Distillation is first proposed for model compression by transfering the teacher's knowl- edge small student model [8 [.]. The idea of knowledge distillation has been further extended to establish debiasing models in natural language understanding (NLU) tasks [32 [3] and long-tail
classification 34 42 [7 The idea of "introspection" related to "self distillation" " which considers
student model itself as the teacher for the next training epoch or stage (24 41 21 36 40) Although
our introspection and self distillation both share the similar idea of "self-teaching" they are funda- mentally different: the latter is still in-distribution and has comparative reasoning about the factual and unseen counterfactual. This difference reveals the key reason why introspection introduces new blended knowledge rather than just an old copy. Also, different from traditional knowledge distillation methods that use a fixed weight as hyper-parameter our IntroD weights the models based on the introspective weights, which does not require careful selection of dyper-parameters.
3 Introspective Distillation
We present a simple yet effective training paradigm, Introspective Distillation (IntroD), to achieve good trade-off between the in-distribution (ID) and outtof-distribution (OOD) performances for robust QA. Given a visual or natural language context C'=c c and a question Q Q=q as input, the QA model generates answer A= Generally, the model is usually not prototyped as generation but a nulti-classification for prediction space reduction, i.e., a E A. For VQA (5), the context refers to an image, and the answers are selected from pre-defined candidate set. For extractive QA (27) the context refers to a passage, and the answers are locations in it.
Our IntroD aims to blend the ID and OOD inductive bias fairly. As illustrated in Figure 3 it consists of three key parts: 1) causal teacher for capturing the ID and OOD inductive bias, 2) introspection for blending the two different inductive biases, and 3) distillation for robust student model.
3

---

10 i--rediction pid
ID-tnowledge
Introspection
pid
Causal Teacher
Student Model
Pst
PT
ps
pood
OOD-Prediction
OOD Knowledge
Factual & Counterfaccuul Reasoning
Knowledge Blending
Knowledge Distillation
Figure 3: Our Introspective Distillation training paradigm. Given the input question and context c, the causal teacher outputs the ID-aware and OOD-aware predictions. After introspecting whether the training sample suffers from the inductive bias, the and OOD knowledge adaptively blended. Finally, the blended knowledge is distilled to student model.
3.1 ID-Teacher and OOD-Teacher
We expect ID-teacher and OOD-teacher to delineate the ID and OOD worlds, respectively. However, without access the OOD distribution. it is difficult to obtain the "oracle" OOD-teacher. Thanks the recently proposed causality-baie -base method [25], OOD-teacher can be approximated by counterfactual reasoning. Also, ID-teacher can approximated using the same causal model by factual reasoning. We briefly introduce the key concepts of the causal method below, and encourage readers to refer to Niu er al. 25 for more details.
The causal QA models formulate the causal relations between the input {Q, C} and the output A. The ID inductive bias is formulated the direct effect of inputs the output, egg., the language prior in VQA as Q + A and the position bias in extractive QA as C A. Compared to traditional QA models that can only conduct factual reasoning formulate the seen ID world, the causal QA models can also imagine the unseen OOD world by counterfactual reasoning. Therefore, we can implement ID-teacher and OOD-teacher using the same causal model. By factual reasoning, the causal QA model predicts the answers piD that include the ID inductive bias into total causal effect. By counterfactual reasoning, the causal QA model explicitly estimates the direct causal effect to exclude the inductive bias, and generate the counterfactual predictions pood i.e., total indirect effect [25) natural indirect effect 66 17], that reflect the unseen OOD world. The training of ID and OOD teachers strictly follows their corresponding methods. The teacher model is trained with standard cross-entropy loss on the ID data, and do not separately train the ID and OOD teachers.
3.2 Introspection of Inductive Bias
Introspection first examines whether the model over-exploits the inductive bias in either or OOD world, and then blends the ID and OOD inductive bias fairly. If the ID inductive bias in world dominates the learning, we expect the student model learn more from the other world for debiasing. This raises questions, how to define "dominate" and "more". In other words, how introspect and weight the inductive bias.
Introspecting the bias. We introspect the effect of inductive bias by comparing the predictions of ID-teacher and OOD-teacher. the inductive bias dominates the learning a sample, ID-teacher's confidence (i.e., predicted probability) on the ground-truth answers would be much larger than that of OOD-teacher. We denote the confidence as:
Σ GE
PDD (a).
OOD
Σ DOOD (a).
(1)
aE
where GT denotes the set of ground-truth answers These scores reflect how well the training sample is matched with the inductive bias. The introspection is realized by comparing ID and S0D
'The number of answers be for single-label classification multiple for multi-label classification.

---

If S > s000,, we think the sample's learning is dominated by the ID inductive bias (see Figure (a)). and vice versa (see Figure (c))
Note that the cross entropy between the ground-truth answers and predictions, XF, is inversely proportional to the confidence. Therefore, we can also use the standard cross-entropy loss to denote the matching scores SnD and s g0DD;
8
XE (GT PDD) EnEA= - pot (a) log pid (a)
(2)
1
G0DD XE(P poon)
EPEA- -p (a) log (a)
where PGT denotes the ground-truth labels. We empirically found that the cross-entropy loss achieves more stable improvements compared to the confidence in the implementation (see Table 3)
Weighting the bias. We blend the II) and OOD knowledge by weighted sum of their knowledge. The purpose of knowledge blending is to mix the ID and OOD inductive bias fairly. the learning is biased to one world, the model may suffer from over-exploiting the corresponding inductive bias. As illustrated Figure (a). is difficult to judge whether the oven electric or not without external knowledge. However, ID-teacher is over-confident in its prediction due to the over-exploitation of the training answer distribution, ie.. ST > s g00D In this case, the model should learn less from ID-teacher. We realize this by increasing the weight of OOD DOD-knowledge w woD and decreasing the weight of ID-knowledge wn i.e., wn < ooD Similarly, for the training samples that is overconfident by OOD-teacher (see Figure F i.e., S!D < 200 we w > OOD We determine the knowledge weights by setting the weights inversely proportional to the matching scores, i.e., w x s-1 The weights are normalized by scaling it between O and 1:
(S (SID) + (SOOD)-
OOD
s
w 1D
0.0
1
W ID
(3)
3ID + gOOD
5ID
GOD
We take VQA as an example to show how the dis- tribution of knowledge weights reflect the effect of inductive bias, i.e., language prior. Recall that VQA v2 [16] is proposed balance the answer distribu- tion to remove the language bias, while VQA-CP v2 2 is proposed to evaluate whether VQA models memorize the language priors. As a result, the VQA v2 train split contains little language bias, while the bias in VQA-CP v2 is artificially severe. Figure 4 illustrates the distribution of wil on the two training sets using CF-VQA (25 as the causal teacher. It can be clearly observed that the distributions of wil totally different, which exactly reflects how the data bias affects the training process. Note that a small "D indicates a high ID-bias. Here are three interesting observations:
VIA-CP V2
percentage (16)
0.0
02
0.4
WDID
0.6
0.8
1.0
Figure 4: The distribution of will on VQA-CP v2 and VQA v2 training sets.
The nD of most samples is around 0.5 for both of the datasets. This indicates that most of the samples are learned unbiasedly and predicted fairly (e.g., Figure (b)).
Both of the distributions are left-skewed. In particular, only 4% of the samples have whl that is larger than 0.6, while the ratio for wD < 0.4 40% on VQA-CP v2 and 25% on VQA v2. The reason is that ID-teacher is directly optimized on the ID data, while OOD-teacher is indirectly approximated. Therefore, ID-teacher outperforms OOD-teacher on the seen ID data in most cases, i.e., "D < 0.5.
A spike lies at the left side of the VQA-CP v2 distribution. In particular, 9.6% of the samples have wilh that lower than 0.05, while the ratio is only 0.4% on VQA v2. Also, the difference between the percentages becomes larger with a decreasing wh and TD < 0.5. This observation indicates that VQA models tend to exploit the training bias on the imbalanced VQA-CP v2 dataset while not on the balanced one. Recall that the VQA-CP training set is artificially modified to "encourage"
5

---

the models to learn from the language prior. Without the memorized priors, VQA models cannot answer the questions confidently or correctly in a few extreme cases (e.g., Figure 2 (a)).
We also define a stochastic hard variant to weigh the bias:
1
if D S S0D,
TD
(4)
otherwise.
The hard weighting forces the student to entirely learn from the OOD teacher for most of the training samples to maintain its OOD performance. In practice, one may choose soft hard variants based on the trade-off between ID and OOD performances. We empirically use the soft variant for strong OOD-teachers and the hard variant for weak ones that achieve relatively lower OOD performance.
Based on the knowledge weights, the ID-knowledge and OOD-knowledge are blended as:
pT wD ID-Knowledge + OOD OOD-Knowledge.
(5)
Considering that the ID ground-truth labels pot are more accurate than the ID-predictions pID, we use pot as the "oracle" ID-Knowledge. Since the OOD distribution is unobserved in training, is impossible to obtain the oracle OOD-Knowledge. Thanks to the causal teacher, we can use the OOD-prediction pood to approximate the OOD-knowledge.
3.3 Distillation of Fair Knowledge
After obtaining the blended fair knowledge from the causal teacher, we train a student model using knowledge distillation manner [18]
Σ (a log (a) ps (a)
L= K -K(PP
(6)
acAA
where p$ denotes the output of the student model. The difference between the teacher model and the student model their architectures. The student model is simply the baseline model, e.g.. UpDn for VQA and BERT [12] for extractive QA. Besides the baseline model, the teacher model ensembles a separate branch formulate the shortcut bias, e.g., Q-> for VQA and C A for extractive QA. Therefore, the student is more efficient in both parameters and inference speed compared to the causal teacher model. We fix the causal teacher and only update the student model during distillation.
Experiments
We take visual QA and extractive QA, two representative QA tasks, as examples to evaluate our proposed Introspective Distillation (IntroD
4.1 Visual QA
Dataset. We conducted experiments the benchmark datasets VQA v2 [16] and VQA-CP v2 (2) VQA v2 is balanced VQA dataset that significantly reduces the language bias. For each question in the dataset, VQA v2 has two different answers for two different images. VQA-CP is variant of VQA v2 to evaluate whether the model answers the questions by simply memorizing the language priors. VQA-CP v2 reverses the priors the training and validation splits. For example, most of "what sports" questions are answered as "tennis" in the training set while "baseball" in the test set.
Metric and setting. The standard evaluation metric for VQA is accuracy. In order to evaluate the robustness of VQA methods, we conducted experiments on two settings: in-distribution (ID) setting and out-o--distribution (OOD) setting. For the IDD setting, reported the results VQA v2 val set. For the OOD setting, we report the results on VQA-CP v2 test set. For the VQA-CP dataset, we
Code available at httptp:://ggttab....o////lltt///

---

Table 1: Comparisons on VQA. Methods in gray denote the baseline models. We reimplement the
methods using their released codes for fair comparisons.
VQA-CP v2 test (OOD)
VQA v2 val (ID)
Methods
All
Y/N
Num.
Other
All
Y/N
Num.
Other
HM
UpDn
39.79
43.23
12.28
45.54
63.42
81.19
42.43
55.47
48.90
LMH [1]
52.01
72.58
31.12
46.97
56.35
65.06
37.63
54.69
34,09
+ IntroD
51.31 4.70
71.39
27.13
47.41
62.05
+5.70
77.65
40.25
55.97
56.17 +2.08
CCS (
58.95
84.37
49,42
48.21
56.98
65.90
38.19
55.18
37,95
+ IInrroD
60.17 +1.22
89.17
46.91
48.62
62.57
45.59
78.57
41.42
56.00
61.35 +3.40
S-MRI 6
37.09
41.39
12.46
41.60
63.12
81.83
45.95
53.43
46.72
RUBi [6
47.60
70.48
20.33
43.09
61.16
81.97
44.86
49.65
53,53
+ -IntroD
48.54 +0.96
73.94
19.43
43.21
61.86
+0.70
82.40
45.40
50.58
54.40 +0.87
RUBB-CP [23T
54.90
90.26
34.33
42.01
60.33
81.39
42.87
49.34
37.58
+ -IntroD
54.92 +0.02
90.84
25.17
44.26
63.15
2262
82.44
45.12
53.25
58.75 +1.17
CF-VQA [25]]
55.05
90.61
21.50
45.61
60.94
81.13
43.86
50.11
37.85
+ IIntroD
55.17 +0.12
90.79
17.92
46.73
63.40
+2.46
82.48
46.60
54.05
58.99 +1.14
also followed Teney el al. 31 and held out 8k samples from the training set as the val set for ID evaluation. We further reported the harmonic mean (HM) of the accuracies on VQA-CP v2 test and VQA v2 val set. We use this metric to evaluate the trade-off between ID and OOD evaluations.
we implemented the counterfactual teacher 2.5 In particular, the earlier works RUBi and LMH used natural indirect effect (NIE) (66 for inference. CSS is variant of LMH that generates counterfactual training samples for data augmentation. CF-VQA proposed to use total indirect effect
Methods. According to the causal explanation 25
as RUBi ( LMH [1I. CSS (7] and CF-VQA
(TIE) [26] for debiasing. and improved RUBi by replacing NIE with TIE. We denote this variant as RUBi-CF. Following previous works, we used UpDn [4] and S-MRL 6 as the backbone. Based on the debiasing ability. we used the soft variant of weights for LMH, CSS, RUBi-CF and CF-VQA, and the hard variant for RUBi (see Table 5p. More training details are in the appendix.
Overall results. Table D and 2 show how our proposed IntroD strengthens the exist- ing causal models. First, according to the
Table 2:
Comparisons on the VQA-CP v2 val set for the in-distribution (ID) evaluation.
HM metric, IntroD improves the trade-off ability of all the causal teachers. In partic- ular, CSS+oD achieves an accuracy of over 60% under both ID and OOD settings, which is the only among all the combina- tions. Second, with a deep look at the OOD evaluation, IntroD shows its competitive de- biasing ability. Surprisingly, IntroD even slightly increases the OOD performance of causal teachers except for LMH. Third, with deep look at the ID evaluation, IntroD outperforms RUBi by 0.7% and other teachers by over 2.4%. The biggest winners are LMH and CSS which suffer from significant drop in the ID performance. Their increases in ID performance are over 5.5%. Similar conclusions can be obtained based on Table Furthermore, IntroD with CF-VQA obtains higher ID performance (63.40%) than the baseline M-RLL (63.12% which achieves the best of both ID and OOD worlds. These results demonstrate the effectiveness of our proposed IntroD on top of different causal VQA models.
Methods
Y/N
Num.
Other
LMH [
58.38
67.17
31.16
57.16
+ IntroD
63.68 45.30
77.33
35.92
58.10
CSS 77]
53.89
55.72
33.98
57.22
+ IntroD
58.63 +4.74
64.83
36.01
58.62
CF-VQA [25
$7.86
66.24
44.98
53.38
+ Introo
59.96 +2.10
66.81
47.65
56.75
Also, the results indicate that the OOD approximation has an impact on the OOD performance of students. Overall, the OOD performance of the student is proportional to that of the teacher, while there is no clue whether the student's ID performance is correlated to that of the OOD-teacher. As shown in Table I CSS+IntroD with the best OOD teacher CSS (58.95%) achieves the highest accuracy (60.17%) compared to other students on VQA-CP v2 test set. Also, IntroD increases the OOD performance of CSS by 1.22%, while the improvement over CF-VQA is much slighter (0.12%) The student achieves even decreased accuracy over the comparatively weakest LMH (-0.70%).
Ablation studies. We further conducted ablation studies to evaluate the introspection and distillation strategy. We compared the alternatives with ID-teacher and OOD-teacher, i.e., factual and counterfac- tual predictions of the same causal model. The ablations aimed to answer the following questions.
7

---

Table Effects of matching scores on VQA. "Prob." denotes using the predicted probability as the matching score (Eq. "XE" denotes using the cross entropy (Eq. (2).
Measurement Prob.
LMH
[11
VQA
25
CSS 7
Notes
XB
00D
ID
HM
00D
ID
HM
00D
ID
HM
ID- Teacher
38.74
63.46
48.11
37.10
63,22
46.76
38.20
63.30
47.65
OOD-Teacher
52.01
56.35
54.09
55.05
60.94
57.85
58.95
56.98
57.95
45.02
63.80
52.79
54.82
63.26
58.74
54.45
63.83
58.76
IntroD
51.31
62.05
56.17
55.17
63.40
58.99
60.17
62.57
61.35
Table 4: Effects of knowledge weights on VQA. "Fixed" denotes fixed wD. the weighted average ensemble. "Simple Avg." denotes the average ensemble. "CFD", Counterfactual Distillation, denotes that the student only learns from DOD-Teached
"Weight Avg." denotes
Weight
LMH [I]
CF-VQA
25
CSS
Notes
Fixed wxs
wx'
OOD
ID
HM
00D
ID
HM
00D
ID
HM
ID-Teacher
38.74
63.46
48.11
37.10
63.22
46.76
38,20
63.30
47.65
OOD-Teacher
52,01
56.35
54.09
55.05
60.94
57.85
58,95
56.98
57.95
Weight Avg.
43.04
64.09
51.50
39.25
64.12
48.69
43.59
64.01
51.86
Simple Avg.
0.5
45.69
64.06
53,33
50.71
63.95
56.57
50.69
63.84
56.51
CFD
0.0
52.88
59.09
55.81
55.40
62.62
58.79
59.21
59.14
59.17
IntroD
51.31
62.05
56.17
55.17
63.40
58,99
60.17
62.57
61.35
Table 5:
Effects of weight variants on VQA.
RUBi [
LMH [1]]
CF-VQA [25]
CSS
Notes
OOD
ID
HM
00D
ID
HM
0OD
ID
HM
0OD
ID
HM
ID-Teacher
36.92
63.21
46.61
38.74
63.46
48.11
37.10
63.22
46.76
38.20
63.30
47.65
OOD-Teacher
47.60
61.16
53.53
52.01
56.35
54.09
55.05
60.94
57.85
58,95
56.98
57.95
Hard Variant
48.54
61.85
54.39
52.91
59.38
55.96
55,48
60.43
57.85
59.39
59.22
59.30
Soft Variant
45.95
62.72
53.04
$1.31
62.05
56.17
55.17
63.40
58.99
60.17
62.57
61.35
Note that Q1 is for "introspecting the bias". Q2-Q5 are for "weighing the bias", and Q6 and Q7 are for "distillation of fair knowledge" in Section
Q1: Can we use the predicted probability of the ground-truth answer ("Prob." for short) as the matching scores? Better not. As shown in Table although using "Prob." achieves even better ID performance than D-teacheer, the rOD-performance drops by ~7% compared to LMH and 4.5% compared to CSS. As a result, the trade-off metric HM decreases with LMH, and increases marginally (<1%) with CF-VQA and CSS.
Q2: Can the student learn more from the more accurate teacher, i.e., setling W 8? No. This is a natural question because we hope to learn the best from the best. Unfortunately, this alternative ("Weight Avg." for short) enhances the inductive bias rather than reduces it. As shown in Table 4 the alternative "Weight Avg." achieves the best ID performance on top of different causal teachers, even beat ID-teacher. However, the students fail to learn the debiasing ability from OOD-teachers and achieves much lower OOD performance compared to OOD-teachers. This observation verifies that the "best" here should be the debiasing ability to the inductive bias rather than the fitting ability.
Q3: Can the student equally learn from ID and OOD teachers, i.e., setting w = 00D = =0.5? No. This alternative can be regarded as a simple average ensemble ("Simple Avg." for short) of ID and OOD teachers. As shown in Table 4 similar to Q2, the students outperform ID-teachers on the ID evaluation with the sacrifice of DOD-performance compared to OOD-teacher
Besides, there is a large gap between "Simple Avg." " and our IntroD with difference causal models, e.g., >2% for LMH and CF-VQA, and ~5% for CSS. This observation indicates that our IntroD is not just simple ensemble method that combines two teacher models into a bigger one.
Q4: Can the student only learn from O0D-teacher? Yes, but worse than IntroD. This alternative can be called counterfactual distillation ("CFD" for short) as the student model only learns from the counterfactual teacher. As shown in Table 4 CFD also achieves better trade-off on top of different causal teachers, especially promote all of the OOD performance compared to OOD-teacher. However, there is large gap between IntroD's and CFD's ID performances because the ID- ID-knowledge is not utilized. As a result, for the HM metric, IntroD outperforms CFD by a small margin (<0.4%) on LMH and CF-VQA and a large margin (> 2%) on CSS.
8

---

Table Effects of ID-Knowledge on VQA.
ID- D--ooweeegge
LMH [
CF-VQA 25
CSS 7
Notes
pib
pGr
OOD
ID
HM
00D
ID
HM
OOD
ID
HM
ID- Teacher
38.74
63.46
48.11
37.10
63.22
46.76
38.20
63.30
47.65
O0D-Teacher
52.01
56.35
54.09
55.05
60.94
57.85
58.95
56.98
57.95
48.37
61.48
54.14
50.72
63.81
56.52
59.82
62.00
60.89
Introl
/
51.31
62.05
56.17
55.17
63.40
58.99
60.17
62.57
61.35
Table 7: Ensemble vs. IntroD using LMH [1]] VQA. "OOD" represents VQA-CP v2 test set,
and "ID" represents VQA v2 val set.
Ensemble
Introl>
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
OOD
52.01
47.35
44.43
42.69
41.53
40.70
40.08
39.56
39.18
38.82
38.74
51.31
56.35
59.34
61.20
62.47
63.14
63.33
63.37
63.39
63.41
63.43
63.46
HM
54.09
52.67
51.48
50.72
50.10
49.55
49.10
48.72
48.43
48.16
48.11
56.17
Table 8: Ensemble vs. IntroD using CSS
7 on VQA. "OOD" represents VQA-CP v2 test set, and
IID" represents VQA v2 val
Ensemble
Introl)
w
0.1
0.2
0.3
0.4
0.6
0.8
1.0
OOD
58.95
54.92
53.30
52.57
51.17
46.75
41.84
39.39
38.31
37.86
38,20
60.17
ID
56.98
59.80
61.92
62.88
63.14
63.22
63.26
63.27
63.26
63.26
63.30
62.05
HM
57.95
57.26
57.29
57.26
56.53
53.75
53.75
50.37
48.55
47.37
47.65
61.35
Q5: Should we use the hard or soft variant to calculate the knowledge weights? It depends on the debiasing ability of the causal teacher. There are some interesting observations from Table 5 First, the OOD performance is proportional to OOD-teachers' debiasing ability. Second, the hard variants marginally improve OOD-teacher's 's OOD performances in all cases. Third, the hard variants cannot fully overcome the sacrifice of degrading ID performance compared to the ID teacher. Empirically, we use the hard variant for the weaker OOD-teacher, e.g., RUBi, and the soft variant for the stronger OOD-teachers, e.g., LMH. CF-VQA, and CSS.
Q6: Can we use the ID-Prediction piD as the ID- ID-Knowledge? No. As shown in Table 6 using pID as the ID-Knowledge significantly degrades the OOD performance for LMH and CF-VQA. This observation indicates that it is better to use the oracle knowledge available.
Q7: Can we ensemble the two teacher models and directly use that without distillation? In other words, is IntroD just an ensemble method? No. Recall that our goal is to achieve the best of both ID and OOD worlds, ie.,, high OOD performance with less or no sacrifice of ID performance. However, the naive ensemble strategy simply combines two models' predictions using a fixed weight without figuring out whether a sample comes from ID or OOD distribution. As result, the ensemble method only inherits the disadvantages of the two teacher models rather than their advantages. Empirical results in Table 7 and 8 further verify our analysis. Here we report the results of ensembling two teachers with different w ID, the weight of teacher. In particular, "D denotes the OOD teacher and wh =1 denotes the ID teacher. We can see that (1) with wt increasing. the ID performance keeps improving, but the OOD performance is gradually decreasing, (2) all of the ensemble alternatives achieve lower HM compared to the OOD teacher. These results indicate that (1) simple ensemble of the two teacher models fails to achieve good trade-off between ID and OOD performances, (2) IntroD is not simply ensemble method.
4.2 Extractive QA
Dataset and settings. We conducted experiments on the reading comprehension benchmark dataset SQuAD 27 SQuAD requires QA models to extract the answer from a passage. Recently, new setting 22 was proposed evaluate whether the extractive QA models suffer from the position bias. This setting divided subset from the training set SQuADuuin based on the position of answers. For

---

Table 9: Comparisons on extractive QA with iQuAD
as the biased training set.
Train
(ID)
SQuAD 1
(OOD)
SQuADaev
(All)
dev
dev
Methods
EM
F1
EM
F1
EM
F1
XLNet
79.65
87.48
30.17
35.91
47.20
53.65
[M [1]]
78.31
85.97
61.04
69.49
66.98
75.16
+ IItroD
81.08 +2.77
88.55 +2.58
61.52 +0.48
68.84
0.65
68.25 +1.27
75.62 +0.46
BERT
77.87
86.41
10.95
16.17
33.95
40.34
[M [II
77.18
85.15
71.31
79.79
73.33
81.64
IntroD
79.21
87.04
+1.89
72.14
79.97
74.58
82.40
+0.76
Table 10: Comparisons on extractive QA trained on different biased training subsets and unbiased training set and tested on the origin evaluation set SQuADdev
Biased Training
Unbiased Training
SQuAD
2
SQuAD
SQuAD
k=4
SQuAD
22
SQuAD
min F1
Imin
Irain
min
Methods
EM
EM
F1
EM
F1
EM
F1
EM
F1
XLNNt
46.40
53.76
47.40
55.29
47.27
55.19
51.38
59.45
72.76
80.58
[M [1]]
67.24
73.05
66.36
74.32
61.86
70.88
53.63
62.40
72.34
80.31
-IntroD
69.38
76.79
68.25
75.72
64.46
72.71
59.58
67.19
73.02
81.01
BERT
32.50
39.23
43.06
51.03
41.38
49.95
59,51
68.31
81.32
88.65
[LM [11]
71.61
80036
69.04
77.91
64.31
73.72
62.82
72.30
"8..12
88.44
+ IIntroD
73.66
82.01
71.69
80.07
66.66
75.41
64.48
73.48
81.39
88,79
example, SQuAD train denotes the subset where all answers are in the first sentences. The test set is divided into two subsets: SQuAD for II) evaluation and SQuAD *1 for OOD evaluation. dev
Metrics and method. The standard evaluation metrics are exact match (EM) and Fl score 27 Following Ko et al. (22), we used XLNet (38) and BERT [1] the backbone models, and LM III as the causal teacher. We empirically used the hard variant for the knowledge weights calculation.
train as the biased training set. The results are reproduced based on the released codef] Overall, LM increases the OOD performance by a large margin but slightly sacrifices the ID performance. As a comparison, our IntroD achieves the best of both ID and OOD performances. Table [00lc/I0fuur further shows that IntroD can promote LM with different answer position bias and different numbers of training samples. In particular, when trained on the less biased training subset SQuAD four, LM achieves less improvement on the overall performance, while IntroD stably promotes LM. Furthermore, using the origin training set SQuADrrain for unbiased training, LM slightly degrades the performance, while IntroD can still beat the baseline models. This observation indicates that IntroD does not over-correct the inductive bias.
Results. Table 9 shows the main analysis with SQuAD
train where the answers locate in sentences except the first
5 Conclusion
In this paper, we proposed novel training paradigm, Introspective Distillation (IntroD), to achieve a fair trade-off between in-distribution (ID) and out-of-distribtiion (OOD) evaluations for question answering tasks, e.g., visual QA and extractive QA. IntroD uses a causal teacher to estimate the ID and OOD inductive bias, introspects whether one of the inductive biases dominates the learning, blends the inductive bias fairly, and distills the knowledge to the student model. Experiments on VQA v2, VQA-CP v2, and SQuAD demonstrated that our IntroD is able to achieve the best of both ID and OOD worlds. The main limitation of our IntroD is that its OOD performance heavily relies on the OODtteacher. In the future, will explore how to establish stronger OOD OOD-teacher.
Acknowledgement
We thank anonymous ACs and reviewers for their valuable discussion and insightful suggestions. This work was supported in part by NTU-Alibaba JRI and MOE AcRF Tier grant.
10

---

References
[1] Ehsan Abbasnejad. Damien Teney, Amin Parvanch, Javen Shi, and Anton den Hengel. Counterfactual vision and language learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10044-10054, 2020.
[2] Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. Don't t just assume; look and answer: Overcoming priors for visual question answering. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pages 4971-4980, 2018.
[3] Aishwarya Agrawal, Jiasen Lu, Stanislaw Antol, Margaret Mitchell, C Lawrence Zitmick, Devi Parikh, and Dhruv Batra. Vqa: Visual question answering. International Journal of Computer Vision, 123(1):4-3) 31, 2017.
Peter Anderson, Xiandong He, Chris Buchler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6077-6086. 2018.
[5] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision, pages 2425-2433, 2015.
[6] Remi Cadene. Corentin Dancette, Matthieu Cord, Devi Parikh, et al. Rubi: Reducing unimodal biases for visual question answering. Advances Neural Information Prucessing Systems, 32:841-852, 2019.
Long Chen, Xin Yan, Jun Xiao, Hanwang Zhang, Shiliang Pu, and Yueting Zhuang. Counterfactual samples synthesizing for robust visual question answering. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 10800-10809, 2020.
[8] Long Chen, Yuhang Zheng, Yulei Niu, Hanwang Zhang, and Jun Xiao. Counterfactual samples synthesizing and training for robust visual question answering. arXiv preprint arXiv:2110.01013, 2021.
[9] Hao Cheng, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. Probabilistic assumptions matter: Improved models for distantly-uperrvised document-level question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5657-5667, 2020.
[10] Christopher Clark and Matt Gardner. Simple and effective multi marti-paragraph reading comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pages 845 845-855, 855, 2018.
[11] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. Don't 't take the easy way out: Ensemble based methods for avoiding known dataset biases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JJCNL)), pages 4060-4075 2019.
2019.
[12] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidi- rectional transformers for language understanding. In Proceedings of the 2019 Conference the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages 4171-4186, 2019.
[13] Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, and Xia Hu. Towards interpreting and mitigating shortcut learning behavior of nlu models. arXiv preprint arXiv:2103.0691 2021.
[14] Tejas Gokhale, Pratyay Banerjee, Chitta Baral, and Yezhou Yang. Mutant: A training paradigm for out-of-distriuution generalization in visual question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 878-892, 2020.
[15] Jianping Gou, Baosheng Yu, Stephen Maybank, Dacheng Tao. Knowledge distillation: survey. International Journal of Computer Vision, pages 1 31, 2021.
[16] Yash Goyal, Tejas Khot, Douglas Summers-Stay Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6904-6913, 2017.
[17] Yin- Yin He, Jianxin Wu, and Xiu-Shen Wei. Distilling virtual examples for loigetailed recognition. arXiv preprint arXiv:2103.15044 15042, 2021.

---

[18] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. stat, 1050:9. 2015.
[19] Chenchen Jing, Yuwei Wu, Xiaoxun Zhang. Yunde Jia, and Qi Wu. Overcoming language priors in vqa via decomposed linguistic representations. In AAAI, pages 11181-111111 2020.
[20] Kushal Kafle and Christopher Kanan. An analysis of visual question answering algorithms. Proceedings of the IEEE International Conference on Computer Vision, pages 1965-1973, 2017.
[21] Kyungyul Kim. MyeongMoon Ji, Doyoung Yoon, and Sangheum Hwang. Self saff-knowledge distillation with progressive refinement of targets. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6367-6576, 2021.
[22] Miyoung Ko, Jinhyuk Lec, Hyunjae Kim, Gangwoo Kim, and Jacwoo Kang. Look at the first sentence: Position bias in question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 110--1121, 2020.
[23] Gouthaman KV and Anurag Mittal. Reducing language biases in visual question answering with visually- grounded question encoder. arXiv preprint (rXiv:2007.06/99 2020.
[24] Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang. Haotang Deng. and Qi Ju. Fastbert: self-distiling bert with adaptive inference time, arXiv preprint arXiv:2004.02/75. 2020.
[25] Yulci Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen. Counterfactual vqu: A cause-effect look at language bias. arXiv preprint wxXiv:206.04311 2020.
[26] Judea Pearl. Direct and indirect effects. In Proceedings of the seventeenth conference on uncertainty in artificial intelligence, pages 111-420. Morgan Kaufmann Publishers Inc., 2001.
[27] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprchension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383-2392,200 2016.
[28] Ramprasaath R Selvaraju. Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry Heck, Dhruv Batra, and Devi Parikh. Taking hint: Leveraging explanations to make vision and language models more grounded. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2591-2600, 2019.
[29] Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. Bidirectional attention flow for machine comprehension. arXiv preprint weXiv:1611.0160 2016.
[30] Damien Teney, Ehsan Abbasnejad, and Anton van den Hengel. Unshufling data for improved generaliza- tion. arXiv preprint arXiv:2002.1184/ 2020.
[31] Damien Teney, Kushal Kafle, Robik Shrestha, Ehsan Abbasnejad, Christopher Kanan, and Anton den Hengel. On the value of out-of-distiibutiin testing: example of goodhart's law. arXiv preprint MwXiv:2005.09241 2020.
[32] Prasetya Ajie Utama, Nafise Sadat Moosavi, and Iryna Gurevych. Mind the trade-off: Debiasing nlu models without degrading the in-distribution performance. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8717-8729, 2020.
[33] Jialin Wu and Raymond J Mooney. Self-critica reasoning for robust visual question answering. arXiv preprint arxiv: 0tt:::9900 190.00998, 2019.
[34] Liuyu Xiang, Guiguang Ding. and Jungong Han. Learning from multiple experts: Self-paced knowledge distillation for long-tailes tailed classification. In European Conference Computer Vision, pages 247-263. Springer, 2020.
[35] Caiming Xiong, Victor Zhong, Richard Socher. Dynamic coattention networks for question answering. arXiv preprint arXiv: /61 I. 1006044 2016.
[36] Chenglin Yang. Lingxi Xie, Chi Su, and Alan L Yuille. Snapshot distillation: Teacher-student optimiza- tion in one generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2859-2868, 2019.
[37] Xu Yang, Hanwang Zhang, Guojun Qi, and Jianfei Cai. Causal attention for vision-language tasks, arXiv preprint 21XX:21113304995 2021.
12

---

[38] Zhilin Yang, Zihang Dai. Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint wrXiv:1906.08237, 2019.
[39] Adams Wei Yu, David Dohan, Minh- Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoe V Le. Qanet: Combining local convolution with global self-attention for reading comprehension. arXiv preprint
2018.
[40] Li Yuan, Francis EH Tay, Guilin Li, Tao Wang, and Jiashi Feng. Revisiting knowledge distillation label smoothing regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3903-399 3911, 2020.
[41] Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages
2019.
[42] Shaoyu Zhang, Chen Chen, Xiyuan Hu, and Silong Peng. Balanced knowledge distillation for long-tailed learning. arXiv preprint arXiv: 104, 105 2021.
13


Reviews:
[{"review_id": "y6xFbzuPFUK", "reviewer": "Reviewer_CgJy", "summary": "This paper proposed Introspective Distillation (IntroD) to achieve good OOD generalizability without sacrificing ID performance. Their training paradigm has three key components (1) factual reasoning ID teader model and counterfactual reasoning OOD model to capture ID and OOD inductive bias, (2) intersection between two inductive biases by comparing the predictions between ID teacher and OOD teader, and (3) knowledge distillation for a strong student model. They conduct experiments on visual QA and extractive QA tasks. On VQA-CP v2 and VQA v2, built upon four counterfactual teacher models (i.e., RUBi, LMH, CSS and CF-VQA0, their introD have shown large ID improvements, leading to considerably large improvements in harmonic mean of ID and OOD accuracies. They also conduct detailed ablation studies to identify the best introspection and distillation strategy. For Extractive QA experiments, they follow the experimental setting from prior work (Ko et al., 2020) where the dataset is divided into subsets based on the position of answers. IntroD has shown its effectiveness on the extractive QA experiments as well.  ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality:**\nThe ID performance deterioration by debiasing has been pointed out, and this work addresses the important problem with a simple yet effective approach. By combining prior counterfactual models with their proposed introD, they significantly improve the ID performance while maintaining or improving the original OOD performance. \n\n**Quality:**\nI think the proposed approach is technically sound and is well-motivated and the rigorous experimental results support its effectiveness. The detailed analysis helps us to understand the best training strategy to address the aforementioned issue. Probably more experiments on extractive QA. For example, testing on another extractive QA dataset having a similar positioning bias, or studying another annotation bias seen in extractive QA (e.g., lexical overlap) would make this paper even stronger. \n\n**Clarity:**\nI think this paper is generally well-written and easy to read. \n\n**Significance:**\nI think this paper addresses an important issue and provides a simple yet effective solution which can be combined with current or new debiasing techniques. The experimental results show its effectiveness on two tasks, namely VQA and extractive QA. As mentioned above, the experimental results on extractive QA may be weak, and more experiments on that task would be helpful. Position issue is one of the core issues in extractive QA, but there are several major annotation biases in the task, and I wonder if the proposed approach would be useful or not for those (more complex) inductive biases. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors acknowledge that the OOD performance is proportional to OOD-teachers’ debiasing ability and future work would study how to establish a stronger OOD-teacher. \nI cannot find any major discussions on the **negative** social implications of this work in the submitted manuscript.  \n", "review_text": "", "extra_scores": {"ethical_concerns": "I don't think there are any major ethical issues. ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "j4ocZY-uA-Y", "reviewer": "Reviewer_75Tq", "summary": "The paper claims to provide a better balance of in-domain and out-of-distribution settings. Normally, algorithms optimized for one hurt the other. The paper proposes to use a weighting mechanism to to balance a model's reliance on in-domain world (modeled as causal factual world) and out-of distribution world (modeled as causal counterfactual world). Experiments on VQA-CP and SQUAD datasets show that the proposed method does indeed help improve ID performance of several algorithms optimized of OOD accuracy while maintaining/slightly improving its OOD performance. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths** \n\n\n**[S1]** Interesting and well-motivated method: The proposed weighting scheme by dividing the training samples into belonging to \"ID world\" and \"OOD world\" is well-motivated and interesting. Despite its simplicity, it is quite effective (S2).\n\n**[S2]** The paper boasts really good results: The paper consistently improves ID performance of several OOD-oriented algorithms without sacrificing its \"debiasing\" abilities. This is, furthermore, achieved with a well-motivated approach, which is more than can be said for several recent works. \n\n**[S3]** Clearly written *except for* Section 3.1: The paper, generally, raises, addresses and answers all of the most relevant questions adequately and clearly. It is easy to understand the motivations, implementations, and results of those choices.  The ablations experiments also show a clear picture of different model choices. The only caveat is the details about the ID and OOD teacher which is sorely lacking (even with supplemental materials). More on the below\n\n\n**Weaknesses**\n\n**[W1]** Very unclear about how the ID and OOD teachers are trained/modeled (aka Section 3.1 + supplementary): I am assuming the paper simply followed the implementation of [23]. If that is the case, then there are no problems and I am familiar with [23]. However, it is super unclear what is happening with regards to ID and OOD teachers, even after reading the supplemental. E.g., the notations used for TIE and NIE are not consistent with the paper -- e.g., compare paper's presentation with Figure 3 from [23] which shows that counterfactual VQA used only Q to predict the answer, whereas the current paper shows links from inputs X which contains both V and Q to answer Y, which is not correct. Please clarify the exact formulation of the ID and OOD teachers in the revised version.\n\n**[W2]** Some concerns about OOD/ID testing setup :\n\n*W2.1* Use of soft/hard weighting for strong vs weak teachers is not very well motivated. Why is Rubi (or by extension, NIE methods) a weak teacher? Why does using hard weighing help \"weak models\"?\n\n*W2.2* Are ID and OOD performance measured for the same model/data choices? As mentioned in [30], retraining before reporting ID performance can inflate the results on ID evaluation and effectively create two different versions of the model. Upon reading the paper, I don't think that this has been addressed. \n\n**Overall:** Overall, I think this is a good paper and I currently recommend a weak acceptance. However, it could be slightly higher if the clarity issues were resolved.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "INLB5vHvb-", "reviewer": "Reviewer_Rti1", "summary": "This paper studies learning models for visual and text-only question answering (VQA and SQUAD) that do well on both in-distribution test sets, as well as out-of-distribution ones.\n\nThe paper takes a knowledge distillation approach for this and designs two \"teacher\" modules: \"ID-teacher\" that captures in-domain bias, and \"OOD-teacher\" in which the in-domain bias is reduced using [23] (Niu et al 2021)'s method. The models are then ensembled and the aggregate probabilities are used to supervise training of a student model.\n\n\nThe paper evaluates this approach on VQA and SQUAD, with a variety of different methods as teachers. The method seems to increase performance on both OOD and ID sets (which is perhaps surprising, since one might hypothesize that the ID teacher ought to do best by itself on ID data, and likewise for the OOD teacher).", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "To this reviewer, this paper looks promising overall, however there are a few key concerns that hold me back from wanting to accept it at this moment. If they are addressed, I would be willing to increase my score in the rebuttal.\n\n\nStrengths: To this reviewer, the pitch of the paper seems interesting, that of using knowledge distillation as a way to effectively ensemble a model that is robust on out-of-distribution data, with one that is robust on in-distribution data. It could be helpful for people working on VQA.\n\nThe ablaion study answers a lot of reasonable questions about model performance, e.g. that knowledge distillation from just the OOD teacher isn't as good, and about which ways of ensembling knowledge work best for which models.\n\nWeaknesses: The novelty of this paper is a bit unclear to this reviewer. One argument made in this paper is that prior work on ensemble-based methods (like 10; Clark et al 2019 Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases\") explicitly formulates what the language prior is. However, it is not clear to me how the causal approach proposed in this work (which seems to be from [23]; Niu et al 2021) does not explicitly formulate what the language prior is. \n\nAdding onto this point, to this reviewer, it wasn't super clear how the ID teacher and OOD teacher are trained, or what aspects of [23] (Niu et al 2021) are being used. I looked through the appendix as well but was still confused.\n\nRelatedly, one concern that comes to mind is whether knowlege distillation helps in this dataset, or whether for the \"harmonic mean\" evaluation on both VQA-v2-CP and VQA-v2, using the proposed weighting method is what is important. In other words, could one ensemble the two teacher models, and directly use that (without knowledge distillation)?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "There was no substantive discussion of limitations and societal impact of this work. The paper could be improved by adding such a section.", "review_text": "", "extra_scores": {"ethical_concerns": "Nothing that stands out to this reviewer.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "1"}, "extra_text": {}}, {"review_id": "90XIPyxkGv6", "reviewer": "Reviewer_hVVm", "summary": "This paper propose a novel method on training VQA system that can achieve competitive performance both in-distribution and out-of-distribution. Specifically, the proposed system leverage a recent causality-based QA model to estimate the OOD distribution (via counterfactual reasoning), thus building two sub-modules to handle ID and OOD settings respectively. The two modules are then blended via knowledge distillation to train a single student model that can better handle the inductive bias. The authors perform extensive experiments on VQA and text-only question answering, with multiple baseline models and show the proposed training paradigm can consistently improve the overall performance (measured by harmonic mean over ID and OOD settings).\n\n========================================================================================\n\nThanks for the response from the authors. I've updated my rating after reading the rebuttal as well as the comments from other reviewers.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. Originality\n- The main idea of the paper can be summarized as \"proportionally distilling two teacher models into a single one to handle both ID and OOD setting\" in question answering, which is quite neat and novel. The authors cleverly leveraged a recent study that approximates the OOD performance via causal QA and counterfactual reasoning. \n- The other parts of the paradigm are quite intuitive and straightforward, including estimating the relative weights based on cross-entropy or predicted probability, and to transfer knowledge to a new model via distillation. The combination of these techniques are shown to be effective for building a more robust QA system.\n\n2. Quality\n- The proposed system is technically sound, and authors have done extensive experiments on VQA and reading comprehension to validate its effectiveness, and important ablation studies are provided.\n- From the experiments the proposed system consistently show improvements over different baseline models.\n\n3. Clarity\n- In general, the paper is well-organized and quite easy to follow, but there still exists some minor issues in the writing. For example, in L151 and L165 the authors claim \"setting the weights disproportionate to the matching scores\". My guess here (based on the equations) is that the authors meant to say \"inversely proportional\" rather than \"disproportionate\". Otherwise I could not follow the logic.\n\n4. Significance\n- This paper achieves quite significant improvements over baselines and the proposed system seems compatible with a wide range of systems.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "* My major concern about this work is the lack of evidence about the approximation of OOD setting, in other words how accurate/reliable the OOD-teacher is. As the authors mention in L64, since the OOD distribution is unseen in training the teacher model is actually an approximation using counterfactual reasoning. Therefore the quality of the approximation is important. In particular, in L294 Q6, the authors also mention that using ID-Prediction as ID-Knowledge is a bad idea, showing the importance of oracle when available. I would expect that a more careful (hopefully quantitative) analysis on the OOD approximation would be beneficial. Since the VQA setting does not have the OOD annotations, probably a proxy can be used, or using reading comprehension (SQuAD) as the testbed to measure the quality of \"approximated OOD\" vs. ground-truth OOD.\n\n* As for the novelty, the most important component in this system (i.e., approximated OOD-teacher) is from a recent work, and other techniques (model weighing and knowledge distillation) are not new as well. Admittedly combining multiple existing ideas holistically is still a contribution but individual components are not novel inventions.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers appreciated the author response and recommend to accept the paper after the author response.

I agree this work provides a solid idea and approach to robust question answering. Adding the proposed approach on top of existing methods shows consistent significant improvements on both, visual question answering as well as reading comprehension (additional results for NLI are also added in the author response). 

I recommend acceptance under the expectations that the authors will address the concerns of the reviewers in the camera ready version as discussed in the author response, including but not limited to
1) include clarifications provided in author response and improve discussion of novelty
2) ablation study ensembeling two teachers
3) additional results on NLI
[if any results don't fit in the main paper, please add them to supplement]
Decision: Accept (Poster)