Introspective Distillation for Robust Question Answering
Year: 2021

Abstract:
Question answering (QA) models are well-known to exploit data bias, e.g., the language prior in visual QA and the position bias in reading comprehension. Recent debiasing methods achieve good out-of-distribution (OOD) generalizability with a considerable sacrifice of the in-distribution (ID) performance. Therefore, they are only applicable in domains where the test distribution is known in advance. In this paper, we present a novel debiasing method called Introspective Distillation (IntroD) to make the best of both worlds for QA. Our key technical contribution is to blend the inductive bias of OOD and ID by introspecting whether a training sample fits in the factual ID world or the counterfactual OOD one. Experiments on visual QA datasets VQA v2, VQA-CP, and reading comprehension dataset SQuAD demonstrate that our proposed IntroD maintains the competitive OOD performance compared to other debiasing methods, while sacrificing little or even achieving better ID performance compared to the non-debiasing ones.

text: 1 Introduction UpDn [4]
S-MRL [6]
CFVQA+IntroD CSS+IntroD LMH+IntroD
Ensemble-based Methods Ours
LMH [11]
CFVQA [25]
CSS [7]
Baseline Debiasing Methods Ours
OOD Accuracy (VQA-CP v2 test)
ID Ac
cu ra
cy (V
Q A
v2 va
l)
Figure 1: Recent debiasing methods achieve high OOD accuracy with the sacrifice of ID accuracy. Our proposed IntroD makes the best of both worlds.
Question answering (QA), which requires machines to answer questions given a context, is one of the most fundamental AI tasks. Popular contexts are vision (e.g., image for VQA [5]) and natural language (e.g., passage for extractive QA [27]). A common observation is that QA models prefer to over-exploit the training bias, which bypasses the context comprehension for a shortcut answer. For example, by only using the linguistic correlations between questions and answers, VQA models can answer most questions correctly [16, 2, 5, 20]. Similarly, extractive QA models may use the spurious
positional cues to locate the answer in the passage [22]. As a result, QA models that have already achieved strong in-distribution (ID) performance may inevitably fail in out-of-distribution (OOD) test scenarios, regardless of the scale of training data and models [14, 22, 37].
Recently, several debiasing methods aim to close the gap between the ID and OOD performances [6, 11, 7, 25]. However, many of them hold the assumption that the training and test distributions are very different or even reversed, e.g., if there are more “yes” answers in training, there must be more “no” answers in testing. As a result, these methods encounter a severe performance drop under the ID evaluation, although they significantly outperform non-debiasing baselines in terms of OOD performance. An interesting observation from Figure 1 is that non-debiasing methods (circles) obtain high ID but low OOD performance, while debiasing methods (squares) achieve high OOD but low ID performance. This observation motivates us to ask: can we make the best of both worlds?
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
Question type Answer Distribution Question type Answer Distribution Question type Answer Distribution
In this paper, we take a step forward to building robust QA models that achieve strong performances in both ID and ODD evaluations. We point out that if the model is over-exploiting the bias in one world, the performance in the other one would be significantly degraded. Therefore, the “best of both” model should be fair with the inductive bias in either world. To this end, we present a simple yet effective training paradigm—Introspective Distillation (IntroD)—to blend the inductive bias of both worlds fairly. Suppose that we have two expert teacher models: ID-teacher and OOD-teacher, each of which captures the ID or OOD inductive bias and represents the corresponding world. Figure 2 illustrates three cases about how an introspective student learns from the two very different teachers.
Case 1: if ID-bias > OOD-bias, then ID-teacher < OOD-teacher. ID inductive bias dominates the learning, and the student should listen more to OOD-teacher. This case occurs when ID-teacher has a low training loss while OOD-teacher has a high one. As shown in Figure 2 (a), it is hard for QA models to conclude whether the oven is electric or not without additional context. Due to the inductive bias in the training data, i.e., most questions starting with “is” are answered by “yes”, ID-teacher concludes with over-confidence while OOD-teacher does not.
Case 2: if ID-bias < OOD-bias, then ID-teacher > OOD-teacher. OOD inductive bias dominates the learning, and the student should listen more to ID-teacher. This case occurs when ID-teacher has a high training loss while OOD-teacher has a low one. As shown in Figure 2 (c), there are at least two older men, one in a blue shirt selling fruits and one in a white shirt walking in the crowd. Therefore, both “blue” and “white” should be correct. However, as most training questions starting with “what color” are labeled by “white” answer, the bias of “OOD should be different from ID” enforces OOD-teacher to downplay “white” unfairly while ID-teacher does not.
Case 3: if ID ≈ OOD, then ID-teacher ≈ OOD-teacher. Learning is fair and the student should listen to both teachers equally. This case occurs when the training losses of the two are close. As shown in Figure 2 (b), the ID-teacher and OOD-teacher produce similar predictions.
The above introspection can be represented as a blended knowledge of the two teachers, which is distilled to the student model [18]. Yet, an unsolved challenge is how to obtain the “oracle” teachers, especially the OOD-teacher, because the OOD distribution is unseen in training, not mentioning to train a teacher model. Thanks to the recent causality-based approach [25], we can approximate the OOD-teacher using a causal model that imagines the unseen world by counterfactual reasoning.
Without loss of generality, we take visual QA and extractive QA as case studies. Experiments on VQA-CP [2], VQA v2 [16], and SQuAD [27] validate the effectiveness of our proposed IntroD. Interestingly, extensive ablations demonstrate that the success of IntroD is indeed from the causal introspection but not from the simple ensemble.

heading: 2 Related Work
text: Visual Question Answering (VQA) [5, 3, 16] is to answer the question given a visual context, i.e., image. Traditional VQA models are found to exploit the language priors in the training data [16, 2, 20]. For example, in the first version of the VQA dataset VQA v1.0, about 40% of the sports-
related questions are answered as “tennis”. Although utilizing the shortcut bias helps with the in-distribution (ID) performance, the out-of-distribution (OOD) one is severely hurt [2]. In order to mitigate the language bias, recent methods proposed to utilize extra annotations for accurate visual grounding [28, 33], generate synthetic data for data augmentation [7, 1, 14, 30, 31], modifying language modules [19, 23], or explicitly formulate and exclude the language prior [6, 11, 25]. These methods obtain significant OOD improvement on the VQA-CP [2] dataset whose answer distributions in training and testing are reversed. However, the OOD improvement is achieved with the cost of a severe ID performance drop. Therefore, it is still a challenge to achieve strong performances in both ID and OOD evaluations.
Extractive Question Answering (extractive QA) is to answer the question given a natural language context, i.e., passage [27]. Extractive QA assumes that the answer always locates in the passage, and further reduces the generative QA task to a classification task, i.e., position prediction. Recent years have witness many influential works [35, 29, 10, 39, 12, 38, 9]. However, directly predicting the answer positions has a severe side effect, i.e., correlating answers with positions [22]. For example, if a language model is trained on a biased dataset where answers always locate in the first sentence of the passage, the model will tend to ground the answer in the first sentence. Recently, a new variant of the reading comprehensive dataset SQuAD [27] is proposed to evaluate whether language models are robust to the position bias [22]. Similar to VQA, the answer position distribution is skewed in the training set. In this paper, we follow Ko et al. [22] to evaluate the robustness for extractive QA.
Ensemble-based methods for debiasing explicitly formulate and exclude the shortcut bias in the training data [6, 11, 7, 25, 8]. The shortcut bias can be captured by a separate branch [6] or statistical priors [11]. These methods are further interpreted as causality-based approaches [25]. However, most of these methods achieve promising performance under the out-of-distribution (OOD) evaluation but sacrifice the performance under the in-distribution (ID) evaluation. The reason is that these methods hold an assumption that the training and test distribution are very different or even reversed. In this paper, we implement our ID-teacher and OOD-teacher using the causality-based methods, and further achieve a good trade-off between ID and OOD evaluations. Previous OOD-teachers, i.e., causality-based methods, only generate the OOD-prediction for debiased inference and ignore the role of ID-prediction. We further point out that the ID-prediction is crucial in introspecting the training process and achieving a good trade-off between ID performance and OOD performance.
Knowledge Distillation is first proposed for model compression by transfering the teacher’s knowledge to a small student model [18, 15]. The idea of knowledge distillation has been further extended to establish debiasing models in natural language understanding (NLU) tasks [32, 13] and long-tail classification [34, 42, 17]. The idea of “introspection” is related to “self distillation”, which considers a student model itself as the teacher for the next training epoch or stage [24, 41, 21, 36, 40]. Although our introspection and self distillation both share the similar idea of “self-teaching”, they are fundamentally different: the latter is still in-distribution and has no comparative reasoning about the seen factual and unseen counterfactual. This difference reveals the key reason why introspection introduces new blended knowledge rather than just an old copy. Also, different from traditional knowledge distillation methods that use a fixed weight as hyper-parameter, our IntroD weights the models based on the introspective weights, which does not require a careful selection of hyper-parameters.

heading: 3 Introspective Distillation
text: We present a simple yet effective training paradigm, Introspective Distillation (IntroD), to achieve a good trade-off between the in-distribution (ID) and out-of-distribution (OOD) performances for robust QA. Given a visual or natural language context C=c and a question Q=q as input, the QA model generates an answer A=a. Generally, the model is usually not prototyped as a generation but a multi-classification for prediction space reduction, i.e., a ∈ A. For VQA [5], the context refers to an image, and the answers are selected from a pre-defined candidate set. For extractive QA [27], the context refers to a passage, and the answers are locations in it.
Our IntroD aims to blend the ID and OOD inductive bias fairly. As illustrated in Figure 3, it consists of three key parts: 1) causal teacher for capturing the ID and OOD inductive bias, 2) introspection for blending the two different inductive biases, and 3) distillation for a robust student model.

heading: 3.1 ID-Teacher and OOD-Teacher
text: We expect ID-teacher and OOD-teacher to delineate the ID and OOD worlds, respectively. However, without access to the OOD distribution, it is difficult to obtain the “oracle” OOD-teacher. Thanks to the recently proposed causality-based method [25], OOD-teacher can be approximated by counterfactual reasoning. Also, ID-teacher can be approximated using the same causal model by factual reasoning. We briefly introduce the key concepts of the causal method below, and encourage readers to refer to Niu et al. [25] for more details.
The causal QA models formulate the causal relations between the input {Q,C} and the output A. The ID inductive bias is formulated as the direct effect of inputs on the output, e.g., the language prior in VQA as Q→A and the position bias in extractive QA as C→A. Compared to traditional QA models that can only conduct factual reasoning to formulate the seen ID world, the causal QA models can also imagine the unseen OOD world by counterfactual reasoning. Therefore, we can implement ID-teacher and OOD-teacher using the same causal model. By factual reasoning, the causal QA model predicts the answers as P ID that include the ID inductive bias into total causal effect. By counterfactual reasoning, the causal QA model explicitly estimates the direct causal effect to exclude the inductive bias, and generate the counterfactual predictions P OOD, i.e., total indirect effect [25] or natural indirect effect [6, 11], that reflect the unseen OOD world. The training of ID and OOD teachers strictly follows their corresponding methods. The teacher model is trained with standard cross-entropy loss on the ID data, and we do not separately train the ID and OOD teachers.

heading: 3.2 Introspection of Inductive Bias
text: Introspection first examines whether the model over-exploits the inductive bias in either ID or OOD world, and then blends the ID and OOD inductive bias fairly. If the ID inductive bias in one world dominates the learning, we expect the student model to learn more from the other world for debiasing. This raises two questions, how to define “dominate” and “more”. In other words, how to introspect and weight the inductive bias.
Introspecting the bias. We introspect the effect of inductive bias by comparing the predictions of ID-teacher and OOD-teacher. If the inductive bias dominates the learning of a sample, ID-teacher’s confidence (i.e., predicted probability) on the ground-truth answers would be much larger than that of OOD-teacher. We denote the confidence as:
sID = ∑
a∈AGT P ID(a), sOOD = ∑ a∈AGT P OOD(a), (1)
where AGT denotes the set of ground-truth answers1. These scores reflect how well the training sample is matched with the inductive bias. The introspection is realized by comparing sID and sOOD.
1The number of answers can be one for single-label classification or multiple for multi-label classification.
If sID>sOOD, we think the sample’s learning is dominated by the ID inductive bias (see Figure 2 (a)), and vice versa (see Figure 2 (c)).
Note that the cross entropy between the ground-truth answers and predictions, XE, is inversely proportional to the confidence. Therefore, we can also use the standard cross-entropy loss to denote the matching scores sID and sOOD:
sID = 1
XE(P GT,P ID) = 1∑ a∈A −P GT(a) logP ID(a) ,
sOOD = 1
XE(P GT,P OOD) = 1∑ a∈A −P GT(a) logP OOD(a) ,
(2)
where P GT denotes the ground-truth labels. We empirically found that the cross-entropy loss achieves more stable improvements compared to the confidence in the implementation (see Table 3).
Weighting the bias. We blend the ID and OOD knowledge by a weighted sum of their knowledge. The purpose of knowledge blending is to mix the ID and OOD inductive bias fairly. If the learning is biased to one world, the model may suffer from over-exploiting the corresponding inductive bias. As illustrated in Figure 2 (a), it is difficult to judge whether the oven is electric or not without external knowledge. However, ID-teacher is over-confident in its prediction due to the over-exploitation of the training answer distribution, i.e., sID>sOOD. In this case, the model should learn less from ID-teacher. We realize this by increasing the weight of OOD-knowledge wOOD and decreasing the weight of ID-knowledge wID, i.e., wID <wOOD. Similarly, for the training samples that is overconfident by OOD-teacher (see Figure 2 (c)), i.e., sID<sOOD, we set wID>wOOD. We determine the knowledge weights by setting the weights inversely proportional to the matching scores, i.e., w ∝ s−1. The weights are normalized by scaling it between 0 and 1:
wID = (sID)−1
(sID)−1 + (sOOD)−1 =
sOOD
sID + sOOD , wOOD = 1− wID = s
ID
sID + sOOD . (3)
We take VQA as an example to show how the distribution of knowledge weights reflect the effect of inductive bias, i.e., language prior. Recall that VQA v2 [16] is proposed to balance the answer distribution to remove the language bias, while VQA-CP v2 [2] is proposed to evaluate whether VQA models memorize the language priors. As a result, the VQA v2 train split contains little language bias, while the bias in VQA-CP v2 is artificially severe. Figure 4 illustrates the distribution of wID on the two training sets using CF-VQA [25] as the causal teacher. It can be clearly observed that the distributions of wID are totally different, which exactly reflects how the data bias affects the training process. Note that a small
wID indicates a high ID-bias. Here are three interesting observations:
• The wID of most samples is around 0.5 for both of the datasets. This indicates that most of the samples are learned unbiasedly and predicted fairly (e.g., Figure 2 (b)).
• Both of the distributions are left-skewed. In particular, only 4% of the samples have wID that is larger than 0.6, while the ratio for wID < 0.4 is 40% on VQA-CP v2 and 25% on VQA v2. The reason is that ID-teacher is directly optimized on the ID data, while OOD-teacher is indirectly approximated. Therefore, ID-teacher outperforms OOD-teacher on the seen ID data in most cases, i.e., wID < 0.5.
• A spike lies at the left side of the VQA-CP v2 distribution. In particular, 9.6% of the samples have wID that is lower than 0.05, while the ratio is only 0.4% on VQA v2. Also, the difference between the percentages becomes larger with a decreasing wID and wID < 0.5. This observation indicates that VQA models tend to exploit the training bias on the imbalanced VQA-CP v2 dataset while not on the balanced one. Recall that the VQA-CP training set is artificially modified to “encourage”
the models to learn from the language prior. Without the memorized priors, VQA models cannot answer the questions confidently or correctly in a few extreme cases (e.g., Figure 2 (a)).
We also define a stochastic hard variant to weigh the bias:
wID = { 1 , if sID ≤ sOOD, 0 , otherwise.
(4)
The hard weighting forces the student to entirely learn from the OOD teacher for most of the training samples to maintain its OOD performance. In practice, one may choose soft or hard variants based on the trade-off between ID and OOD performances. We empirically use the soft variant for strong OOD-teachers and the hard variant for weak ones that achieve relatively lower OOD performance.
Based on the knowledge weights, the ID-knowledge and OOD-knowledge are blended as:
P T = wID · ID-Knowledge + wOOD · OOD-Knowledge. (5)
Considering that the ID ground-truth labels P GT are more accurate than the ID-predictions P ID, we use P GT as the “oracle” ID-Knowledge. Since the OOD distribution is unobserved in training, it is impossible to obtain the oracle OOD-Knowledge. Thanks to the causal teacher, we can use the OOD-prediction P OOD to approximate the OOD-knowledge.

heading: 3.3 Distillation of Fair Knowledge
text: After obtaining the blended fair knowledge from the causal teacher, we train a student model using a knowledge distillation manner [18]:
L = KL(P T,P S) = ∑ a∈A P T(a) log P T(a) P S(a) , (6)
where P S denotes the output of the student model. The difference between the teacher model and the student model is their architectures. The student model is simply the baseline model, e.g., UpDn [4] for VQA and BERT [12] for extractive QA. Besides the baseline model, the teacher model ensembles a separate branch to formulate the shortcut bias, e.g., Q→A for VQA and C→A for extractive QA. Therefore, the student is more efficient in both parameters and inference speed compared to the causal teacher model. We fix the causal teacher and only update the student model during distillation.

heading: 4 Experiments
text: We take visual QA and extractive QA, two representative QA tasks, as examples to evaluate our proposed Introspective Distillation (IntroD)2.

heading: 4.1 Visual QA
text: Dataset. We conducted experiments on the benchmark datasets VQA v2 [16] and VQA-CP v2 [2]. VQA v2 is a balanced VQA dataset that significantly reduces the language bias. For each question in the dataset, VQA v2 has two different answers for two different images. VQA-CP v2 is a variant of VQA v2 to evaluate whether the model answers the questions by simply memorizing the language priors. VQA-CP v2 reverses the priors in the training and validation splits. For example, most of “what sports” questions are answered as “tennis” in the training set while “baseball” in the test set.
Metric and setting. The standard evaluation metric for VQA is accuracy. In order to evaluate the robustness of VQA methods, we conducted experiments on two settings: in-distribution (ID) setting and out-of-distribution (OOD) setting. For the ID setting, we reported the results on VQA v2 val set. For the OOD setting, we report the results on VQA-CP v2 test set. For the VQA-CP dataset, we
2Code are available at https://github.com/yuleiniu/introd.
also followed Teney et al. [31] and held out 8k samples from the training set as the val set for ID evaluation. We further reported the harmonic mean (HM) of the accuracies on VQA-CP v2 test and VQA v2 val set. We use this metric to evaluate the trade-off between ID and OOD evaluations.
Methods. According to the causal explanation [25], we implemented the counterfactual teacher as RUBi [6], LMH [11], CSS [7] and CF-VQA [25]. In particular, the earlier works RUBi and LMH used natural indirect effect (NIE) [26] for inference. CSS is a variant of LMH that generates counterfactual training samples for data augmentation. CF-VQA proposed to use total indirect effect (TIE) [26] for debiasing, and improved RUBi by replacing NIE with TIE. We denote this variant as RUBi-CF. Following previous works, we used UpDn [4] and S-MRL [6] as the backbone. Based on the debiasing ability, we used the soft variant of weights for LMH, CSS, RUBi-CF and CF-VQA, and the hard variant for RUBi (see Table 5). More training details are in the appendix.
Overall results. Table 1 and 2 show how our proposed IntroD strengthens the existing causal models. First, according to the HM metric, IntroD improves the trade-off ability of all the causal teachers. In particular, CSS+IntroD achieves an accuracy of over 60% under both ID and OOD settings, which is the only among all the combinations. Second, with a deep look at the OOD evaluation, IntroD shows its competitive debiasing ability. Surprisingly, IntroD even
slightly increases the OOD performance of causal teachers except for LMH. Third, with a deep look at the ID evaluation, IntroD outperforms RUBi by 0.7% and other teachers by over 2.4%. The biggest winners are LMH and CSS which suffer from a significant drop in the ID performance. Their increases in ID performance are over 5.5%. Similar conclusions can be obtained based on Table 2. Furthermore, IntroD with CF-VQA obtains higher ID performance (63.40%) than the baseline S-MRL (63.12%), which achieves the best of both ID and OOD worlds. These results demonstrate the effectiveness of our proposed IntroD on top of different causal VQA models.
Also, the results indicate that the OOD approximation has an impact on the OOD performance of students. Overall, the OOD performance of the student is proportional to that of the teacher, while there is no clue whether the student’s ID performance is correlated to that of the OOD-teacher. As shown in Table 1, CSS+IntroD with the best OOD teacher CSS (58.95%) achieves the highest accuracy (60.17%) compared to other students on VQA-CP v2 test set. Also, IntroD increases the OOD performance of CSS by 1.22%, while the improvement over CF-VQA is much slighter (0.12%). The student achieves even decreased accuracy over the comparatively weakest LMH (-0.70%).
Ablation studies. We further conducted ablation studies to evaluate the introspection and distillation strategy. We compared the alternatives with ID-teacher and OOD-teacher, i.e., factual and counterfactual predictions of the same causal model. The ablations aimed to answer the following questions.
Note that Q1 is for “introspecting the bias”, Q2-Q5 are for “weighing the bias”, and Q6 and Q7 are for “distillation of fair knowledge” in Section 3.
Q1: Can we use the predicted probability of the ground-truth answer (“Prob.” for short) as the matching scores? Better not. As shown in Table 3, although using “Prob.” achieves even better ID performance than ID-teacher, the OOD-performance drops by ∼7% compared to LMH and 4.5% compared to CSS. As a result, the trade-off metric HM decreases with LMH, and increases marginally (<1%) with CF-VQA and CSS.
Q2: Can the student learn more from the more accurate teacher, i.e., setting w∝ s? No. This is a natural question because we hope to learn the best from the best. Unfortunately, this alternative (“Weight Avg.” for short) enhances the inductive bias rather than reduces it. As shown in Table 4, the alternative “Weight Avg.” achieves the best ID performance on top of different causal teachers, even beat ID-teacher. However, the students fail to learn the debiasing ability from OOD-teachers and achieves much lower OOD performance compared to OOD-teachers. This observation verifies that the “best” here should be the debiasing ability to the inductive bias rather than the fitting ability.
Q3: Can the student equally learn from ID and OOD teachers, i.e., setting wID=wOOD=0.5? No. This alternative can be regarded as a simple average ensemble (“Simple Avg.” for short) of ID and OOD teachers. As shown in Table 4, similar to Q2, the students outperform ID-teachers on the ID evaluation with the sacrifice of OOD-performance compared to OOD-teachers. Besides, there is a large gap between “Simple Avg.” and our IntroD with difference causal models, e.g., >2% for LMH and CF-VQA, and ∼5% for CSS. This observation indicates that our IntroD is not just a simple ensemble method that combines two teacher models into a bigger one.
Q4: Can the student only learn from OOD-teacher? Yes, but worse than IntroD. This alternative can be called counterfactual distillation (“CFD” for short) as the student model only learns from the counterfactual teacher. As shown in Table 4, CFD also achieves a better trade-off on top of different causal teachers, especially promote all of the OOD performance compared to OOD-teacher. However, there is a large gap between IntroD’s and CFD’s ID performances because the ID-knowledge is not utilized. As a result, for the HM metric, IntroD outperforms CFD by a small margin (<0.4%) on LMH and CF-VQA and a large margin (> 2%) on CSS.
Q5: Should we use the hard or soft variant to calculate the knowledge weights? It depends on the debiasing ability of the causal teacher. There are some interesting observations from Table 5. First, the OOD performance is proportional to OOD-teachers’ debiasing ability. Second, the hard variants marginally improve OOD-teacher’s OOD performances in all cases. Third, the hard variants cannot fully overcome the sacrifice of degrading ID performance compared to the ID teacher. Empirically, we use the hard variant for the weaker OOD-teacher, e.g., RUBi, and the soft variant for the stronger OOD-teachers, e.g., LMH, CF-VQA, and CSS.
Q6: Can we use the ID-Prediction P ID as the ID-Knowledge? No. As shown in Table 6, using P ID as the ID-Knowledge significantly degrades the OOD performance for LMH and CF-VQA. This observation indicates that it is better to use the oracle knowledge if available.
Q7: Can we ensemble the two teacher models and directly use that without distillation? In other words, is IntroD just an ensemble method? No. Recall that our goal is to achieve the best of both ID and OOD worlds, i.e., a high OOD performance with less or no sacrifice of ID performance. However, the naive ensemble strategy simply combines two models’ predictions using a fixed weight without figuring out whether a sample comes from ID or OOD distribution. As a result, the ensemble method only inherits the disadvantages of the two teacher models rather than their advantages. Empirical results in Table 7 and 8 further verify our analysis. Here we report the results of ensembling two teachers with different wID, the weight of ID teacher. In particular, wID=0 denotes the OOD teacher and wID=1 denotes the ID teacher. We can see that (1) with wID increasing, the ID performance keeps improving, but the OOD performance is gradually decreasing, (2) all of the ensemble alternatives achieve a lower HM compared to the OOD teacher. These results indicate that (1) a simple ensemble of the two teacher models fails to achieve a good trade-off between ID and OOD performances, (2) our IntroD is not simply an ensemble method.

heading: 4.2 Extractive QA
text: Dataset and settings. We conducted experiments on the reading comprehension benchmark dataset SQuAD [27]. SQuAD requires QA models to extract the answer from a passage. Recently, a new setting[22] was proposed to evaluate whether the extractive QA models suffer from the position bias. This setting divided a subset from the training set SQuADtrain based on the position of answers. For
example, SQuADk=1train denotes the subset where all answers are in the first sentences. The test set is divided into two subsets: SQuADk=1dev for ID evaluation and SQuAD k ̸=1 dev for OOD evaluation.
Metrics and method. The standard evaluation metrics are exact match (EM) and F1 score [27]. Following Ko et al. [22], we used XLNet [38] and BERT [12] as the backbone models, and LM [11] as the causal teacher. We empirically used the hard variant for the knowledge weights calculation.
Results. Table 9 shows the main analysis with SQuADk=1train as the biased training set. The results are reproduced based on the released code3. Overall, LM increases the OOD performance by a large margin but slightly sacrifices the ID performance. As a comparison, our IntroD achieves the best of both ID and OOD performances. Table 10 further shows that IntroD can promote LM with different answer position bias and different numbers of training samples. In particular, when trained on the less biased training subset SQuADk≤5train where the answers locate in sentences except the first four, LM achieves less improvement on the overall performance, while IntroD stably promotes LM. Furthermore, using the origin training set SQuADtrain for unbiased training, LM slightly degrades the performance, while IntroD can still beat the baseline models. This observation indicates that IntroD does not over-correct the inductive bias.

heading: 5 Conclusion
text: In this paper, we proposed a novel training paradigm, Introspective Distillation (IntroD), to achieve a fair trade-off between in-distribution (ID) and out-of-distribution (OOD) evaluations for question answering tasks, e.g., visual QA and extractive QA. IntroD uses a causal teacher to estimate the ID and OOD inductive bias, introspects whether one of the inductive biases dominates the learning, blends the inductive bias fairly, and distills the knowledge to the student model. Experiments on VQA v2, VQA-CP v2, and SQuAD demonstrated that our IntroD is able to achieve the best of both ID and OOD worlds. The main limitation of our IntroD is that its OOD performance heavily relies on the OOD-teacher. In the future, we will explore how to establish a stronger OOD-teacher.

heading: Acknowledgement
text: We thank anonymous ACs and reviewers for their valuable discussion and insightful suggestions. This work was supported in part by NTU-Alibaba JRI and MOE AcRF Tier 2 grant.
3https://github.com/dmis-lab/position-bias

Review summaries:
This paper propose a novel method on training VQA system that can achieve competitive performance both in-distribution and out-of-distribution. Specifically, the proposed system leverage a recent causality-based QA model to estimate the OOD distribution (via counterfactual reasoning), thus building two sub-modules to handle ID and OOD settings respectively. The two modules are then blended via knowledge distillation to train a single student model that can better handle the inductive bias. The authors perform extensive experiments on VQA and text-only question answering, with multiple baseline models and show the proposed training paradigm can consistently improve the overall performance (measured by harmonic mean over ID and OOD settings).  ========================================================================================  Thanks for the response from the authors. I've updated my rating after reading the rebuttal as well as the comments from other reviewers.

This paper studies learning models for visual and text-only question answering (VQA and SQUAD) that do well on both in-distribution test sets, as well as out-of-distribution ones.  The paper takes a knowledge distillation approach for this and designs two "teacher" modules: "ID-teacher" that captures in-domain bias, and "OOD-teacher" in which the in-domain bias is reduced using [23] (Niu et al 2021)'s method. The models are then ensembled and the aggregate probabilities are used to supervise training of a student model.   The paper evaluates this approach on VQA and SQUAD, with a variety of different methods as teachers. The method seems to increase performance on both OOD and ID sets (which is perhaps surprising, since one might hypothesize that the ID teacher ought to do best by itself on ID data, and likewise for the OOD teacher).

The paper claims to provide a better balance of in-domain and out-of-distribution settings. Normally, algorithms optimized for one hurt the other. The paper proposes to use a weighting mechanism to to balance a model's reliance on in-domain world (modeled as causal factual world) and out-of distribution world (modeled as causal counterfactual world). Experiments on VQA-CP and SQUAD datasets show that the proposed method does indeed help improve ID performance of several algorithms optimized of OOD accuracy while maintaining/slightly improving its OOD performance. 

This paper proposed Introspective Distillation (IntroD) to achieve good OOD generalizability without sacrificing ID performance. Their training paradigm has three key components (1) factual reasoning ID teader model and counterfactual reasoning OOD model to capture ID and OOD inductive bias, (2) intersection between two inductive biases by comparing the predictions between ID teacher and OOD teader, and (3) knowledge distillation for a strong student model. They conduct experiments on visual QA and extractive QA tasks. On VQA-CP v2 and VQA v2, built upon four counterfactual teacher models (i.e., RUBi, LMH, CSS and CF-VQA0, their introD have shown large ID improvements, leading to considerably large improvements in harmonic mean of ID and OOD accuracies. They also conduct detailed ablation studies to identify the best introspection and distillation strategy. For Extractive QA experiments, they follow the experimental setting from prior work (Ko et al., 2020) where the dataset is divided into subsets based on the position of answers. IntroD has shown its effectiveness on the extractive QA experiments as well.