Counterfactual Multihop QA: A Cause-Effect Approach for Reducing Disconnected Reasoning
Year: 2023

Abstract:
Multi-hop QA requires reasoning over multiple supporting facts to answer the question. However, the existing QA models always rely on shortcuts, e.g., providing the true answer by only one fact, rather than multi-hop reasoning, which is referred to as disconnected reasoning problem. To alleviate this issue, we propose a novel counterfactual multihop QA, a causaleffect approach that enables to reduce the disconnected reasoning. It builds upon explicitly modeling of causality: 1) the direct causal effects of disconnected reasoning and 2) the causal effect of true multi-hop reasoning from the total causal effect. With the causal graph, a counterfactual inference is proposed to disentangle the disconnected reasoning from the total causal effect, which provides us a new perspective and technology to learn a QA model that exploits the true multi-hop reasoning instead of shortcuts. Extensive experiments have been conducted on the benchmark HotpotQA dataset, which demonstrate that the proposed method can achieve notable improvement on reducing disconnected reasoning. For example, our method achieves 5.8% higher points of its Supps score on HotpotQA through true multihop reasoning. The code is available at https://github.com/guowzh/CFMQA.

heading: 1 Introduction
text: Multi-hop question answering (QA) (Groeneveld et al., 2020; Ding et al., 2019; Asai et al., 2019; Shao et al., 2020) requires the model to reason over multiple supporting facts to correctly answer a complex question. It is a more challenging task than the single-hop QA since not only the correct answer but the explicit reasoning across multiple evidences should be provided.
Hence, recent work (Groeneveld et al., 2020; Fang et al., 2019) has shown that the multi-hop QA is always formulated as two sub-tasks: question
∗*Corresponding Author
answering and support identification. For example, Groeneveld et al. (2020) solved the multi-hop QA via 1) question answering which uses a BERT (Devlin et al., 2018) span prediction model to answer the questions, and 2) support identification which aims to identify the supporting sentences. Yavuz et al. (2022) proposed PATHFID based on fusionin-decoder (FID) (Izacard and Grave, 2020) for question answering and support identification.
However, dividing the multi-hop QA into question answering and support identification doesn’t mean that QA models answer the questions according to what the multi-hop QA wants. These QA models are all based on the large pre-training language models, e.g., BERT (Devlin et al., 2018) and XLNet (Yang et al., 2019), these black-box language models are rather opaque models in terms of the reasoning processes. It may results in one main problem of multihop QA models: disconnected reasoning (Trivedi et al., 2020), which allows the QA models to exploit the reasoning shortcuts (Jiang and Bansal, 2019; Lee et al., 2021) instead of multihop reasoning to cheat and obtain the right answer. Taking Fig. 1 as an example, to answer the question “until when in the U.S. Senate", multi-hop QA requires to answer the question with a true reasoning path (e.g., the second paragraph → the third paragraph in Fig. 1). However, the black-box QA models can also infer the correct answer by just utilizing the types of problems, e.g., we can find the corresponding fact “from 2005 to 2008" in the contexts to answer this type of question “until when" without reasoning.
To address the above problem, two issues should be considered: 1) how to measure the disconnected reasoning? To the best of our knowledge,Trivedi et al. (2020) firstly defined an evaluation measure, DiRe in short, to measure how much the QA model can cheat via disconnected reasoning. A probing dataset should be constructed to measure the DiRe. And 2) how to reduce the disconnected reasoning?
4214
One possible solution is to strengthen the training dataset via extra annotations or adversarial examples, which makes it cannot find the correct answers by only one supporting fact. For example, Jiang and Bansal (2019) constructed the adversarial examples to generate better distractor facts. Besides, counterfactual intervention (Lee et al., 2021; Ye et al., 2021) had also been explored to change the distribution of the training dataset. However, when these existing approaches decrease the disconnected reasoning, the original performance also drops significantly. It is still challenging to reduce disconnected reasoning while maintaining the same accuracy on the original test set.
Motivated by causal inference (Pearl and Mackenzie, 2018; Pearl, 2022; Niu et al., 2021), we utilize the counterfactual reasoning to reduce the disconnected reasoning in multi-hop QA and also obtain the robust performance on the original dataset. We formalize a causal graph to reflect the causal relationships between question (Q), contexts, and answer (Y ). To evaluate the disconnected reasoning, contexts are further divided into two subsets: S is a supporting fact and C are the remaining supporting facts. Hence, we can formulate the disconnected reasoning as two natural direct causal effects of (Q,S) and (Q,C) on Y as shown in Fig. 1. With the proposed causal graph, we can relieve the disconnected reasoning by disentangling the two natural direct effects and the true multihop reasoning from the total causal effect. A novel counterfactual multihop QA is proposed to disentangle them from the total causal effect. We utilize the generated probing dataset proposed by (Trivedi et al., 2020) and DiRe to measure how much the proposed multi-hop QA model can reduce the disconnected reasoning. Experiment results show that our approach can substantially decrease the disconnected reasoning while guaranteeing the strong performance on the original test set. The results indicate that the proposed approach can improve the true multi-hop reasoning capability.
The main contribution of this paper is threefold. Firstly, our counterfactual multi-hop QA model formulates disconnected reasoning as two direct causal effects on the answer, which is a new perspective and technology to learn true multi-hop reasoning. Secondly, our approach achieves notable improvement on reducing disconnected reasoning compared to various baselines. Thirdly, our causaleffect approach is model-agnostic and can be used
for reducing disconnected reasoning in many multihop QA architectures.

heading: 2 Related Work
text: Multi-hop question answering (QA) requires the model to retrieve the supporting facts to predict the answer. Many approaches and datasets have been proposed to train QA systems. For example, HotpotQA (Yang et al., 2018) dataset is a widely used dataset for multi-hop QA, which consists of fullwiki setting (Das et al., 2019; Nie et al., 2019; Qi et al., 2019; Chen et al., 2019; Li et al., 2021; Xiong et al., 2020) and distractor setting (Min et al., 2019b; Nishida et al., 2019; Qiu et al., 2019; Jiang and Bansal, 2019; Trivedi et al., 2020).
In fullwiki setting, it first finds relevant facts from all Wikipedia articles and then answers the multi-hop QA with the found facts. The retrieval model is important in this setting. For instance, SMRS (Nie et al., 2019) and DPR (Karpukhin et al., 2020) found the implicit importance of retrieving relevant information in the semantic space. Entitycentric (Das et al., 2019), CogQA (Ding et al., 2019) and Golden Retriever (Qi et al., 2019) explicitly used the entity that is mentioned or reformed in query key words to retrieve the next hop document. Furthermore, PathRetriever (Asai et al., 2019) and HopRetriever (Li et al., 2021) can iteratively select the documents to form a paragraph-level reason path using RNN. MDPR (Xiong et al., 2020) retrieved passages only using dense query vectors many times. These methods hardly discuss the QA model’s disconnected reasoning problem.
In the distractor setting, 10 paragraphs, two gold paragraphs and eight distractors, are given. Many methods have been proposed to strengthen the model’s capability of multi-hop reasoning, using graph neural network (Qiu et al., 2019; Fang et al., 2019; Shao et al., 2020) or adversarial examples or counterfactual examples (Jiang and Bansal, 2019; Lee et al., 2021) or the sufficiency of the supporting evidences (Trivedi et al., 2020) or make use of the pre-trained language models (Zhao et al., 2020; Zaheer et al., 2020).
However, Min et al. (2019a) demonstrated that many compositional questions in HotpotQA can be answered with a single hop. It means that QA models can take shortcuts instead of multi-hop reasoning to produce the corrected answer. To relieve the issue, Jiang and Bansal (2019) added adversarial examples as hard distractors during training. Re-
cently, Trivedi et al. (2020) proposed an approach, DiRe, to measure the model’s disconnected reasoning behavior and used the supporting sufficiency label to reduce the disconnected reasoning. Lee et al. (2021) selected the supporting evidence according to the sentence causality to the predicted answer, which guarantees the explainability of the behavior of the model. While the original performance also drops when reducing the disconnected reasoning.
Causal Inference. Recently, causal inference (Pearl and Mackenzie, 2018; Pearl, 2022) has been applied to many tasks of natural language processing and computer vision, and it shows promising results and provides strong interpretability and generalizability. The representative works include counterfactual intervention for visual attention (Rao et al., 2021), causal effect disentanglement for VQA (Niu et al., 2021), the back-door and frontdoor adjustments (Zhu et al., 2021; Wang et al., 2021; Yang et al., 2021). Our method can be viewed as a complement of the recent approaches that utilize the counterfactual inference (Lee et al., 2021; Ye et al., 2021) to identify the supporting facts and predict the answer.

heading: 3 Preliminaries
text: In this section, we use the theory of causal inference (Pearl and Mackenzie, 2018; Pearl, 2022) to formalize our multi-hop reasoning method. Suppose that we have a multi-hop dataset D and each instance has the form of (Q,P ;Y ), where Q is a question and P = {s1, s2, · · · , sn} is a context consisting of a set of n paragraphs. And Y is the ground-truth label. Given a question Q with multiple paragraphs as a context P , the multi-hop QA models are required to identify which paragraphs are the supporting facts and predict an answer using the supporting facts.
Causal graph. In multi-hop QA, multiple supporting facts are required to predict the answer. While QA models may use only one fact to give the answer, which is referred to as disconnected reasoning. For example, given a question q and only a paragraph s, the disconnected reasoning model can predict the correct answer or correctly determine whether the paragraph s is the supporting fact. Hence, to define the causal graph of disconnected reasoning, the context P is further divided into a paragraph S and the remaining paragraphs C. Now the (Q,P ;Y ) becomes (Q,S,C;Y ). That
is each instance (Q,P ;Y ) is converted into n examples, i.e., (q, s1, C = {s2, · · · , sn};Y (s1)), · · · , (q, sn, C = {s1, · · · , sn−1};Y (sn)), where Y (si) = {Fsi ;A} includes the supporting fact and answer, where A is the answer and Fsi = 1 means the paragraph s is the supporting fact otherwise it is not. For each example, we consider the disconnected reasoning with the fact S (not C).
The causal graph for multi-hop QA is shown in Figure 1 (a), where nodes denote the variables and directed edges represent the causal-and-effect relationships between variables. The paths in Figure 1 (a) are as follows.
(Q,S) → K1 → Y : (Q,S) → K1 denotes that the feature/knowledge K1 is extracted from the question (Q) and the paragraph (S) via the QA model backbone, e.g., BERT. K1 → Y represents the process that the label Y is predicted by only using the K1. (Q,C) → K2 → Y : Similarly, the feature/knowledge K2 extracted from the question (Q) and the remaining paragraphs (C) is used to predict the label Y . (Q,S,C) → (K1,K2) → K → Y : This path indicates that the QA model predicts the label Y based on both the K1 and K2.
Based on the above, the effect of Q,S,C on Y can be divided into: 1) shortcut impacts, e.g., (Q,S) → K1 → Y and (Q,C) → K2 → Y , and 2) reasoning impact, e.g., (Q,S,C) → (K1,K2) → K → Y . The shortcut impacts capture the direct effect of (Q,S) or (Q,C) on Y via K1 → Y or K2 → Y . The reasoning impact captures the indirect effect of (Q,S,C) on Y via K → Y .
Hence, to reduce the multi-hop QA model’s disconnected reasoning proposed in (Trivedi et al., 2020), we should exclude shortcut impacts (K1 → Y and K2 → Y ) from the total effect.
Counterfactual definitions. Figure 1 (a) shows the causal graph. From causal graph to formula, we denote the value of Y , i.e., the answer A (e.g., 2008) or the supporting paragraph Fs (e.g., is the paragraph s supporting fact?), would be obtained when question Q is set to q, the paragraph S is set to s and the remaining paragraphs c = P − s are used as the context C, which is defined as
Yq,s,c(A) = Y (A | Q = q, S = s, C = c), Yq,s,c(s) = Y (s | Q = q, S = s, C = c).
For simplicity, we omit A, s and unify both equations as Yq,s,c = Yq,s,c(A) or Yq,s,c =
Yq,s,c(s). Since the causal effect of q, s, c on Y via K1,K2,K on Y , we have
Yk1,k2,k = Yq,s,c (1)
in the following discussion. To disentangle the shortcut impacts from the total causal effect, we use the counterfactual causal inference to block other effects. To model the K1 → Y , the counterfactual formulation
K1 → Y : Yk1,k∗2 ,k∗ , (2)
which describes the situation where K1 is set to the original value k1 and K and K2 are blocked. The k∗ and k∗2 are the counterfactual notations. The k1 and k∗, k∗2 represent the the two situations where the k1 is under treatment in the factual scenario and k∗, k∗2 are not under treatment (Pearl and Mackenzie, 2018) in the counterfactual scenario. The same definitions for other two effects as
K2 → Y : Yk∗1 ,k2,k∗ , (3)
and K → Y : Yk∗1 ,k∗2 ,k. (4)
Causal effects. According to the counterfactual definitions, the total effect of q, s, c on Y can be decomposed into the natural direct effects of K1,K2 on Y and the effect of K on Y as discussed before. The two natural direct effects cause the disconnected reasoning problem. The effect of K → Y is the desired multi-hop reasoning.
As shown in Figure 1 (b), the effect of K1 on Y with K2,K blocked and the effect of K2 on Y with K1,K blocked can be easily obtained by setting the S or C to counterfactual values (please refer to Section 4 for more details). While the effect of K on Y can not be obtained by changing the values S/C to S∗/C∗. We follow (Niu et al., 2021) and total indirect effect (TIE) is used to express the effect of K on Y , which is formulated as
Yk∗1 ,k∗2 ,k = Yk1,k2,k − Yk1,k2,k∗ . (5)

heading: 4 Counterfactual Multihop QA
text: Following the former formulations, we propose to construct the counterfactual examples to estimate the natural direct effect of K1 and K2, as well as using parameters to estimate the total indirect effect
of K. And our calculation of Y in Eq. (2), (3) and (5) is parametrized by a neural multi-hop QA model F . Please note that F can be any multi-hop QA model and our method is model-agnostic.

heading: 4.1 Disentanglement of causal effect
text: K1 → Y . Specifically, in Eq. (2), the Yk1,k∗2 ,k∗ describes the situation where K1 is set to the factual value with Q = q, S = s as inputs, and K2/K are set to the counterfactual values. Taking Figure 1 as an example, the QA model only considers the interaction between question q and a given paragraph s. The remaining paragraphs c are not given. It is the disconnected reasoning (Trivedi et al., 2020). To obtain the counterfactual values of K2 and K, we can set the context C as its counterfactual sample, and we have
Yk1,k∗2 ,k∗ = Yq,s,c∗ = F(q, s, c ∗). (6)
In this paper, we randomly sample the remaining contexts from the training set to construct the counterfactual c∗. It represents the no-treatment or the prior knowledge of the remaining context. In the implementation, we randomly sample the remaining contexts in a mini-batch to replace the c in the original triple example (q, s, c) and obtain the corresponding counterfactual triple example (q, s, c∗). With that, we can feed it into the QA model to get Yq,s,c∗ . K2 → Y . Similarly, in Eq. (3), the Yk∗1 ,k2,k∗ describes the situation where K2 is set to the factual value with the inputs Q = q and C = c. The K1 and K are set to the counterfactual values with the counterfactual sample S = s∗ as input, which is defined as
Yk∗1 ,k2,k∗ = Yq,s∗,c = F(q, s ∗, c). (7)
One may argue that how to predict the label when S is set to the counterfactual values? As the example shown in Figure 1, even without the paragraph s as input, the QA model still can infer the paragraph s is supporting fact via wrong reasoning: since all paragraphs in C do not include the words about time range, the rest paragraph s should be the supporting fact to answer the “until when" question. It is the exclusive method. The wrong reasoning is caused by an incorrect interaction between the paragraph s and the remaining context c. Hence, the S is set to the counterfactual values that can correct such incorrect interactions.
Hence, in the implementation, we use the adversarial examples as suggested in (Jiang and Bansal, 2019) to construct the counterfactual s∗, which aims to remove the incorrect interaction and perform multi-hop reasoning. Specifically, we randomly perturb the 15% tokens of the paragraph s, 80% of which will be replaced by other random tokens, 10% of which will be replaced by the mask token (e.g. [MASK]) of the tokenizer, and 10% of which will keep unchanged. After that, we obtain another counterfactual triple example (q, s∗, c), we can feed it into QA model to get Yq,s∗,c. K → Y . In Eq. (5), Yq,s,c indicates that the question q, paragraph s and remaining context c are visible to the QA model F :
Yk1,k2,k = Yq,s,c = F(q, s, c). (8)
The main problem is that Yk1,k2,k∗ is unknown. It is also hard to use the counterfactual samples of Q,S,C to obtain its value since the q, s and c should be the factual values for k1, k2. In this paper, we follow the work (Niu et al., 2021) and also assume the model will guess the output probability under the no-treatment condition of k∗, which is represented as
Yk1,k2,k∗ = C, (9)
where C is the output and a learnable parameter. Similar to Counterfactual VQA (Niu et al., 2021), we guarantee a safe estimation of Yk1,k2,k∗ in expectation.
Training objective. From the above discussion, we can estimate the total causal effect:
Y ← Yq,s,c∗ + Yq,s∗,c + Yq,s,c − C. (10)
Note that Y can be any ground-truth labels of answer span prediction, supporting facts identification or answer type prediction. See Appendix A.1 for further implementation details.

heading: 4.2 Training and Inference
text: Training. The model F is expected to disentangle the two natural direct effects and the true multi-hop effect from the total causal effect. To achieve this goal, we apply the Eq. (10) to train the QA model F . Our training strategy follows the HGN (Fang et al., 2019):
L = Lstart + Lend + λLsent + Lpara + Ltype + Lentity,
(11)
where λ is a hyper-parameter and each term of L is cross-entropy loss function. Specifically, for answer prediction, we utilize the Eq. (10) to obtain the predicted logits of the start and end position of the answer span, and respectively calculate the cross-entropy loss Lstart and Lend with corresponding ground truth labels. As for supporting facts prediction, similarly, we use the Eq. (10) to calculate the predicted logits in sentence level and paragraph level, and then calculateLsent andLpara. We also apply our counterfactual reasoning method to identify the answer type (Qiu et al., 2019; Fang et al., 2019), which consists of yes, no, span and entity. We use the [CLS] token as the global representation to predict the answer type under the Eq. (10) and calculate Ltype with the ground truth label. Entity prediction (Lentity) (Fang et al., 2019) is only a regularization term and the Eq. (10) is not applied to this term.
Inference. As illustrated in Section 3, our goal is to exclude the natural direct effect (K1 → Y,K2 → Y ) and use the true multi-hop effect (K → Y ) to reduce the multi-hop QA model’s
disconnected reasoning, so we use Eq. (5) for inference:
F(q, s, c)− C. (12)
The time consumption of our approach is equal to the existing methods.

heading: 5 Experiments
text: We extensively conduct the experiments on the HotpotQA (Yang et al., 2018) dataset in the distractor setting. See Appendix A.2 for more experimental results on the other multihop benchmark 2WikiMultihopQA (Ho et al., 2020).
Metrics and Dataset: To measure multi-hop QA models, we report two results: 1) original denotes the model’s original performance (larger is better) and 2) dire denotes how much the QA model is cheated (smaller is better). Please note original−dire denotes the model’s true multi-hop reasoning.
A probing dataset should be constructed to measure DiRe. To the best of our knowledge, only (Trivedi et al., 2020) provides the probing dataset
of the development set of HotpotQA. Thus we use the development set of HotpotQA to evaluate the original and dire. Specifically, the probing dataset for HotpotQA in the distractor setting divides each example of the original dataset into two instances, both of which only contain one of two ground truth supporting paragraphs respectively. If the multihop QA model can arrive at the correct test output on two instances, it means that the model performs disconnected reasoning on the original example. Please refer to (Trivedi et al., 2020) for more details.
Following the Dire (Trivedi et al., 2020), we report the metrics for HotpotQA: answer span (Ans), supporting paragraphs (Suppp), supporting sentences (Supps), joint metrics (Ans+Suppp,Ans+ Supps). We show both EM scores and F1 scores to compare the performance between baselines and our counterfactual multi-hop reasoning method.
Baselines: First, we simply use the BERT (Devlin et al., 2018) to predict the answer, supporting sentences and supporting paragraphs as the baseline. BERT + ours denotes that we apply our counterfactual multi-hop reasoning method based on BERT as the backbone. The proposed approach is model-agnostic and we also implement it on several multi-hop QA architectures, including DFGN (Qiu et al., 2019), HGN (Fang et al., 2019) and XLNet in Dire (Trivedi et al., 2020; Yang et al., 2019). Our proposed algorithm also can be implemented on other baselines.

heading: 5.1 Quantitative Results
text: For fairness, we conduct the experiments under the same preprocessing of the dataset following HGN (Fang et al., 2019), which select top K relevant paragraphs corresponding to each question example. And the experimental results are shown in Table 1 and Table 2. The main observation can be made as follows:
Our method can significantly reduce disconnected reasoning. Compared to BERT baseline, our proposed counterfactual multi-hop reasoning method can reduce the disconnected reasoning of answer prediction and supporting facts identification in both the paragraph level and sentence level. In particular, we can see big drops of 9.5 F1 points on Supps (from 64.1 to 54.6) and 13.6 EM points on Suppp (from 17.3 to 3.7) in disconnected reasoning (dire). Our method is better at reducing disconnected reasoning on the Exact Match (EM)
evaluation metric. This is because EM is a stricter evaluation metric. For example, EM requires both of the supporting facts should be predicted correctly, while it has F1 scores even when only one supporting fact is predicted correctly. For dire evaluation where only one supporting fact is provided, our approach punishes this situation and achieves lower scores on EM metric of disconnected reasoning. It demonstrates that our method effectively reduces disconnected reasoning when the supporting facts are insufficient.
Our method still guarantees comparable performance on the original dev set. As seen from Table 1 and Table 2, the proposed method also maintains the same accuracy on the original set. It even shows a better performance on the supporting facts prediction in the paragraph level.
Our method is model-agnostic and it demonstrates effectiveness in several multi-hop QA models. Based on our causal-effect insight, our proposed approach can easily be applied to other multi-hop QA architectures including XLNET, DFGN, HGN (Trivedi et al., 2020; Qiu et al., 2019; Fang et al., 2019). As shown in Table 1 and Table 2, our proposed counterfactual reasoning method achieves better performance. Our method can reduce disconnected reasoning by introducing the proposed counterfactual approach in the training procedure. The dire scores of HGN (Fang et al., 2019) and XLNET (Trivedi et al., 2020) in Ans,Suppp, Supps all drop to some extent. Besides, the performances on the original dev set are comparable simultaneously.
In summary, reducing the disconnected reasoning and guaranteeing the strong performance on the original development set indicate that the most progress of the model is attributed to the multi-top reasoning (K → Y ) capability. For intuitiveness, we also show the real multi-hop reasoning promoted by our proposed counterfactual reasoning approach, as shown in Fig. 2.

heading: 5.2 Ablation Study
text: As illustrated in Section 4, our goal is to exclude the shortcut impacts (K1 → Y,K2 → Y ) to reduce the disconnected reasoning. Hence, we study the ablation experiments by excluding one of the shortcut impacts. We explore removing K1 → Y or K2 → Y that reduces the disconnected reasoning, as shown in Table 3. We can see that excluding one of them can decrease the amount of discon-
nected reasoning to some extent on supporting facts identification except the answer span prediction. However, relieving the both impacts of K1 → Y and K2 → Y can achieve better performance on decreasing disconnected reasoning. Because the model can always exploit another shortcut if only one of the shortcuts is blocked.
We further conduct ablation studies to validate the distribution assumption for the counterfactual output of the parameter C. Similar to CF-VQA (Niu et al., 2021), we empirically validate the two distribution assumptions, as shown in Table 4. The "random" denotes that C are learned without constraint and it means that CAns ∈ Rn, Csupp ∈ R2, Ctype ∈ R4 respectively, and n represents the
length of the context. The "uniform" denotes that C should satisfy uniform distribution and it means that CAns, Csupp and Ctype are scalar. As shown in Table 4, the random distribution assumption performs better than the uniform distribution assumption.

heading: 6 Conclusion
text: In this work, we proposed a novel counterfactual reasoning approach to reduce disconnected reasoning in multi-hop QA. We used the causal graph to explain the existing multi-hop QA approaches’ behaviors, which consist of the shortcut impacts and reasoning impacts. The shortcut impacts capture the disconnected reasoning and they are formulated as the natural direct causal effects. Then we constructed the counterfactual examples during the training phase to estimate the both natural direct effects of question and context on answer prediction as well as supporting facts identification. The reasoning impact represents the multi-hop reasoning and is estimated by introducing learnable parameters.
During the test phase, we exclude the natural direct effect and utilize the true multi-hop effect to decrease the disconnected reasoning. Experimental results demonstrate that our proposed counterfactual reasoning method can significantly drop the disconnected reasoning on the probing dataset and guarantee the strong performance on the original dataset, which indicates the most progress of the multi-hop QA model is attributed to the true multi-
hop reasoning. Besides, our approach is modelagnostic, and can be applied to other multi-hop QA architectures to avoid exploiting the shortcuts.
Limitations
Our proposed method needs to construct counterfactual examples to estimate the natural direct effect of disconnected reasoning during the training phase, thus we need a little more GPU resources and computational time. However, the need of resource occupancy and time consumption of our approach does not increase during inference. Another limitation is that we use the learnable parameters to approximate the Yk1,k2,k∗ . In our future work, we will explore a better approach to model it.

heading: Acknowledgment
text: This work is supported by the National Natural Science Foundation of China (U1911203, U2001211, U22B2060), Guangdong Basic and Applied Basic Research Foundation (2019B1515130001, 2021A1515012172), Key-Area Research and Development Program of Guangdong Province (2020B0101100001).

heading: A Appendix
text: A.1 Implementation Details Specifically, given the question Q = q, and context P = {s1, s2, ..., sm} where m is the number of paragraphs, we denote the remaining context ci = P −{si}, 1 ≤ i ≤ m. To distinguish whether si is supporting fact and get the answer distribution on si, we construct the s∗i and c ∗ i as illustrated in the subsection 4.1. We respectively encode (q, si, ci),(q, s∗i , ci) and (q, si, c ∗ i ) to get the contextualized representation O ∈ Rn×d,Mi ∈n×d , Gi ∈ Rn×d, where n is the length of the question and context.
For supporting facts identification, similar to Dire (Trivedi et al., 2020), we use the start token sstarti of the paragraph as its representation and obtain their predicted logits under factual and counterfactual scenario:
Yq,si,ci(si) = g(O[s start i ]) Yq,s∗i ,ci(si) = g(Mi[s start i ]) Yq,si,c∗i (si) = g(Gi[s start i ]),
(13)
where g is a classifier instantiated as MLP layer in practice. And sstarti and s end i are denoted as the start and end position of the paragraph si respectively. In sentence level, we use the their start positions in paragraph si and operate in the same way.
As for answer span prediction, we concatenate the representation of si in Mi or Gi (1 ≤ i ≤ m)
to construct the entire answer span prediction on the whole context:
M̄ = [M1[s start 1 : s end 1 ]; ...;Mm[s start m : s end m ]] Ḡ = [G1[s start 1 : s end 1 ]; ...;Gm[s start m : s end m ]]
Yq,s,c(start) = g(O)
Yq,s∗,c(start) = g(M̄)
Yq,s,c∗(start) = g(Ḡ), (14) where [; ] denotes the operation of concatenation. And we can predict the end position of the answer in the same way.
We also apply our counterfactual reasoning method to identify the answer type, consisting of yes, no, span and entity:
Yq,s,c(type) = g(O[0]) Yq,s∗,c(type) = m∑
i=1
g(Mi[0])
Yq,s,c∗(type) =
m∑
i=1
g(Gi[0]).
(15)
We use the [CLS] token as the global representation to predict the answer type, following previous work (Qiu et al., 2019; Fang et al., 2019).
A.2 Extensive experiments
In this section, we extensively evaluate our method on 2WikiMultihopQA dataset(Ho et al., 2020). And following Dire (Trivedi et al., 2020), we construct the probing dataset of the corresonding development set to evaluate the metric of dire. The experimental results are shown in Table 5.
Our proposed counterfactual reasoning approach method can be generalized to other multihopQA benchmarks. Our approach can still significantly reduce the disconnected reasoning, and guanrantee the comparable preformance on the original dev set. Besides, our method is model-agnostic and
it demonstrates effectiveness in several multi-hop QA models.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
7
7 A2. Did you discuss any potential risks of your work? Left blank.
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
7 B1. Did you cite the creators of artifacts you used? Left blank.
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Left blank.
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank.
7 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
7 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.
C 7 Did you run computational experiments? Left blank.
7 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
7 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Left blank.
7 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
7 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Left blank.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
7 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Left blank.
7 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Left blank.
7 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Left blank.
7 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Left blank.
7 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Left blank.

heading: Future Work
text: In our future work, we will explore a better approach to model it