R e C o: Reliable Causal Chain Reasoning via Structural Causal Recurrent Neural Networks (2022)

Abstract: Causal chain reasoning (CCR) is an essential ability for many decision-making AI systems, which requires the model to build reliable causal chains by connecting causal pairs. However, CCR suffers from two main transitive problems: threshold effect and scene drift. In other words, the causal pairs to be spliced may have a conflicting threshold boundary or scenario.To address these issues, we propose a novel Reliable Causal chain reasoning framework (ReCo), which introduces exogenous variables to represent the threshold and scene factors of each causal pair within the causal chain, and estimates the threshold and scene contradictions across exogenous variables via structural causal recurrent neural networks (SRNN). Experiments show that ReCo outperforms a series of strong baselines on both Chinese and English CCR datasets. Moreover, by injecting reliable causal chain knowledge distilled by ReCo, BERT can achieve better performances on four downstream causal-related tasks than BERT models enhanced by other kinds of knowledge.

## introduction causal chain reasoning aims at understanding the long-distance causal dependencies of events and building reliable causal chains. here, reliable means that events in the causal chain can naturally occur in the order of causal evolution within some circumstance based on the commonsense @xcite . causal chain knowledge is of great importance for various artificial intelligence applications, such as question answering @xcite , and abductive reasoning @xcite . many studies focus on the reliability of causal pair knowledge but ignore that of causal chain knowledge, especially in the natural language processing (nlp) community. † corresponding author * this work was conducted during the internship of kai xiong at huawei cloud previous works mainly acquire causal chain knowledge by first extracting precise causal pairs from text with rule-based @xcite or neural-based @xcite ) methods, then connecting these causal pairs into causal chains based on the textual or semantic similarity between events. however, this straightforward approach may bring some transitive problems @xcite , leading to unreliable causal chains, which would hinder causal-enhanced models to get higher performances. for example, given a cause event: "playing basketball", and two candidate effect events: "gets a technical foul" and "gets a red card", an unreliable causal chain ("playing basketball"→"dispute"→"gets a red card") would mislead the model to choose the less plausible effect "gets a red card". among these transitive problems @xcite , threshold effect and scene drift are the most two salient ones. as shown in figure 1 (a), given two causal pairs (a causes b, and b causes c), the threshold effect problem is that the influence of a on b is not enough for b to cause c. we can notice that, "swimming in the sea" can only result in tens of milliliters of "salt water intake", while "dehydration" is caused by hundreds of milliliters of "salt water intake". therefore, "salt water intake" conditioned on "swimming in the sea" cannot lead to "dehydration". similarly, as shown in figure 1 (b), the scene drift problem means that @xmath0. these two "dispute" events are wrongly joined together by their surface forms. "dispute" that happened in a video game scene cannot lead to "gets a red card" in a football match scene. therefore, we find that the threshold effect and scene drift problems are caused by the contradictions between the threshold factors and between the scene factors, respectively. to address these two issues, in reco, we first build a structural causal model (scm) @xcite for each causal chain, and the scm introduces exogenous variables to represent the threshold and scene factors of the causal pairs within the causal chain. then, we conduct an exogenous-aware conditional variational autoencoder (ea-cvae) to implicitly learn the semantic representations of exogenous variables according to the contexts of the causal pairs. subsequently, we devise a novel causal recurrent neural network named srnn to estimate the contradictions between the exogenous variables by modeling the semantic distance between them. finally, we present a task-specific logic loss to better optimize reco. extensive experiments show that our method outperforms a series of baselines on both chinese and english ccr datasets. the comparative experiments on different lengths of the causal chains further illustrate the superiority of our method. moreover, bert @xcite injected with reliable casual chains distilled by reco, achieves better results on four downstream causal-related tasks, which indicates that reco could provide more effective and reliable causal knowledge. the code is available on https://github.com/waste-wood/reco . ## conclusion we explore the problem of causal chain reasoning and propose a novel framework called reco to overcome the two main transitive problems of threshold effect and scene drift. reco first constructs an scm for each causal chain, the scm introduces exogenous variables to represent the threshold and scene factor of the causal pairs, and then conducts ea-cvae to implicitly learn the representations of the exogenous variables with the contexts. finally, reco devises srnn to estimate the threshold and scene contradictions across the exogenous variables. experiments show that reco can achieve the best ccr performances on both chinese and english datasets.