CMA - R : Causal Mediation Analysis for Explaining Rumour Detection (2024)

Abstract: We apply causal mediation analysis to explain the decision-making process of neural models for rumour detection on Twitter.Interventions at the input and network level reveal the causal impacts of tweets and words in the model output.We find that our approach CMA-R – Causal Mediation Analysis for Rumour detection – identifies salient tweets that explain model predictions and show strong agreement with human judgements for critical tweets determining the truthfulness of stories.CMA-R can further highlight causally impactful words in the salient tweets, providing another layer of interpretability and transparency into these blackbox rumour detection systems. Code is available at: https://github.com/ltian678/cma-r.

## introduction there has been substantial work on understanding the inner workings of neural models via attention mechanisms @xcite , local surrogated approaches @xcite @xcite or integrated gradient based methods @xcite . existing works on explainable fake news or rumour detection by and large use attention weights to explain model decision @xcite @xcite , but @xcite found that the use of attention as explanation is problematic: removing words with high attention appears to have little effect on the final prediction, suggesting that attention doesn't explain the decision process. to address these limitations, in this paper, we propose cma-r -causal mediation analysis for rumour detection -grounded in causal mediation analysis (cma @xcite , as illustrated in figure 1 ) to interpret decisions for rumour detection models. cma-r is a significant departure from existing interpretation methods, as it provides greater explanatory power from assessing causal relations instead of correlations. different from studies @xcite that apply cma to examine the causal structure from network components to predictions, we perform intervention in the input and network to determine the tweets and words that are causally implicated in the final prediction and verify them with human expert annotations. using a rumour dataset that has been annotated by journalists to highlight critical tweets that determine the truthfulness of a story, we assess the salient tweets extracted by cma-r and other interpretation methods (e.g. attention) and found that cma-r yields better alignment with human judgements, empirically demonstrating that it is important to consider causality for explaining model decisions. cma-r also allows us to highlight impactful words in those salient tweets, providing another mechanism to interpret rumour detection models. the main contributions of this work are as follows: ## related work we briefly summarise prior studies from three related areas: explainable artificial intelligence, causal mediation analysis and rumour detection. explainable artificial intelligence aims to create a suite of techniques to produce interpretable artificial intelligence systems, which are often driven by deep learning @xcite . broadly speaking there are two approaches: model-agnostic and model-specific methods. model-agnostic approaches such as lime (local interpretable model-agnostic explanations) @xcite and shap (shapley additive explanations) @xcite build local surrogate models to approximate the predictions of the original model. model-specific techniques use feature visualisation @xcite and attention mechanisms @xcite to explain the decision-making process. additionally, rationalisation-based approaches focus on generating textual explanations that rationalise a model's decision. the explanations mimic human reasoning and provide narrative or rationale for why a model made a certain decision @xcite @xcite @xcite . it is not a way to explain a model's internal decision-making processes, but a method for rationalising the behaviour and justifying its predictions. causal mediation analysis (cma) aims to uncover cause-and-effect relationships, and its application to understanding deep learning models is emerging @xcite @xcite . cma-r goes beyond understanding the correlations between the input and output, but instead attempts to the causal structure for model decisions. in this paper, we employ cma-r to understand how intervention at both the word and neuron levels affect the model's predictions. deep learning is the dominant approach for automatic detection of online rumours and fake news @xcite @xcite . attention mechanisms have been widely used to explain model decisions @xcite @xcite , but there is emerging evidence showing that correlation does not always constitute explanation @xcite @xcite . ## preliminaries let @xmath0, x 1 , x 2 , ..., x n } be a set of events, where an event x i consists of either: (1) a source tweet and its comments (figure 2 ); or (2) a story with a set of source tweets and their comments (figsky news australia: a sixth hostage has escaped from the lindt cafe in sydney are they escaping or being released? they said they escaped not on your live feed??? anyone else reporting this? rt source tweet comments false figure 2: labelled source tweet in pheme. ## methodology cma-r allows us to analyse the change of a response variable (y) following a treatment (x) -e.g. in the biomedical domain this could mean the patient's health outcome given a treatment -and it does so by considering mediators (z), intermediate factors that produce an indirect effect. as shown in figure 1 , a mediator (z) is added to take into account its indirect effect. @xcite introduce cma as a means to explain the decision of a neural model, by viewing the model input as x, the model output (decision) as y, and the neurons in the model as z. in cma-r, x represents an event and y a rumour label, and the tweets in x are encoded using a sequence network (e.g. bert @xcite ). the tweets in x may be concatenated as a string or represented as a graphs (to model the conversation structure), depending on the rumour detection model (section 5.2). ## conclusion we employed causal mediation analysis to understand the inner workings of rumour detection models. by performing interventions at the input and network levels, we show that our approach cma-r can find tweets and words having the most causal impact for model decisions. to evaluate the "quality" of these insights, we train rumour detection models of differing complexity and compare cma-r to current interpretation methods to assess how well the extracted salient tweets align with human judgements. empirical results demonstrate that cma-r is consistently the best method, suggesting that causal relations, rather than correlations, can better interpret model decisions. cma-r provides further mechanism to hone in on the words for the most causal impact, and qualitative analysis reveals that the best rumour detection model is focusing on intuitively important words when determining the veracity of a story.