Causal Inference from Text: Unveiling Interactions between Variables (2023)

Abstract: Adjusting for latent covariates is crucial for estimating causal effects from observational textual data. Most existing methods only account for confounding covariates that affect both treatment and outcome, potentially leading to biased causal effects. This bias arises from insufficient consideration of non-confounding covariates, which are relevant only to either the treatment or the outcome. In this work, we aim to mitigate the bias by unveiling interactions between different variables to disentangle the non-confounding covariates when estimating causal effects from text. The disentangling process ensures covariates only contribute to their respective objectives, enabling independence between variables. Additionally, we impose a constraint to balance representations from the treated group and control group to alleviate selection bias. We conduct experiments on two different treatment factors under various scenarios, and the proposed model significantly outperforms recent strong baselines. Furthermore, our thorough analysis on earnings call transcripts demonstrates that our model can effectively disentangle the variables, and further investigations into real-world scenarios provide guidance for investors to make informed decisions.

## introduction causal inference @xcite @xcite @xcite aims to identify how the treatment variable affects the outcome variable. for example, to estimate the effect of "political risk" (treatment) faced by a company on its "stock movement" (outcome). early research efforts @xcite @xcite @xcite focusing on conducting randomized control trials (rcts) to estimate causal effects from structural numeric data have made significant progress. however, these methods requires extensive effort in treatment assignment mechanism @xcite and may suffer from ethical issues. natural language processing (nlp) researchers are increasingly interested in estimating causal effects from observational unstructured text. early literature @xcite @xcite largely focuses on transforming texts into high-dimensional vectors using lexical features for confounding adjustment. recent research primarily focuses on learning adequate representations through advanced nlp models. for example, @xcite fine-tuned bert @xcite to produce contextual text representations for efficient estimation of causal effects. later, @xcite introduced strategies involving treatment enhancement and text adjustment to estimate the causal effects related to linguistic properties. despite their efficacy, such approaches operate under the assumption that text solely encompasses confounding covariates. this assumption raises a potential issue due to the possible existence of unobserved non-confounding covariates that are pertain exclusively to either the treatment or the outcome. the causal estimation may be biased if we fail to differentiate non-confounding covariates from confounding ones when learning an estimation function through effective modeling of variable interactions @xcite . as illustrated in figure 1 , if we aim to accurately estimate the causal effects of treatment t (e.g, political risk) on the outcome y (e.g., stock movement), we intentionally omit the consideration of the impacts originating from z y (e.g., expected revenue). this mirrors our decision not to account for the influence of z c (e.g., geographical location) on y , as such inclusion could obfuscate our ability to discern the true effects originating from t . in this paper, we propose a framework named disentangling interaction of variables (diva), specifically tailored for causal inference from text. we assume that the text carries sufficient information to identify the causal effects and consider the existence of non-confounding covariates. drawing on the success of latent variable models for causal inference in literature @xcite , we use variational auto-encoder (vae) (kingma and welling, 2014) to infer confounding and non-confounding covariates. additionally, we design a disentanglement module to ensure that covariates only contribute to their specific objectives, enabling independence between covariates. furthermore, we propose to impose a constraint to balance representations from the treatment group and control group, which helps to mitigate selection bias. our contributions are summarized as follows: our work differs from these works in three main aspects. first, we aim to mitigate the bias that arises from insufficient consideration of non-confounding covariates in causal inference. second, we disentangle non-confounding covariates by encouraging independence among the variables, ensuring that each one contributes solely to its respective objective. third, we introduce regularization to balance representations from the treatment group and control group, which helps to mitigate selection bias. ## preliminaries causal inference from text aims to estimate the causal effects based on observed textual data. let here, x i is the observed textual data (e.g., earnings call transcript) for the i-th example (e.g., company), and t @xmath0, 1} is the binary treatment variable foot_0 . t @xmath1.g., a company faced high political risk). conversely, t @xmath2.g., a company faced low or no political risk). the causal effect τ i for the ith example is defined as the expected difference between its potential outcome y i (e.g., stock volatility) of the treatment and control groups, known as the individual treatment effect (ite): one of the most challenging problems in estimating causal effects from observational data is the impossibility of simultaneously observing both potential outcomes y i (t @xmath3. in other words, d only includes the observed outcome y i for each example, but not the unobserved counterfactual outcome, which refers to the potential outcome for the i-th example in the alternative group. nonetheless, it's feasible to identify the conditional average treatment effect (cate) and the average treatment effect (ate) from observational data under certain assumptions @xcite @xcite : assumption 1 (stable unit treatment values assumption (sutva)): the potential outcomes of one example are not influenced by the treatment assigned to other examples, and there are no varying forms or levels of the treatment that could result in different potential outcomes: assumption 2 (unconfoundedness): the potential outcomes are conditionally independent of the treatment given a set of observed covariates: (y (1), y (0)) ⊥ ⊥ t . assumption 3 (positivity): every individual has a non-zero probability of receiving treatment or control for all observed variables: in line with the potential outcome framework outlined by @xcite and @xcite , and with the above assumptions, we pre-trained language model ## diva: disentangling interaction of variables in this section, we present the proposed disentangling interaction of variables (diva) framework (figure 2 ) for causal inference from textual earnings call transcripts. although previous research @xcite has explored estimating causal effects from text, one of the core contributions of our work is that we disentangle various variables to effectively model the interactions among them. this in turn enables us to learn a more accurate estimation function q for predicting outcomes, thereby reducing the bias in the causal estimation. our proposed diva framework consists of a few steps. first, we extract the contextualized text representation from the pre-trained language model. following that, we employ a variational auto-encoder to determine the posterior distribution for various latent variables. once this distribution is obtained, we use the variable disentanglement module to encourage independence among the variables, ensuring that each one contributes solely to its respective objective. next, we utilize the disentangled variables to learn the q function via the outcome prediction task. finally, we plug the trained q into a pre-determined statistic to estimate the ate. ## experiments we conduct experiments on both semi-synthetic data and real-world application scenarios with two objectives: 1) to empirically evaluate the effectiveness of our proposed model, and 2) to investigate practical questions in the field of finance and gain insights from the application of our model to these real-world scenarios. ## conclusion in this paper, we propose diva, a novel framework designed specifically for causal inference from text. we verify its effectiveness by estimating the causal effects of treatment factor (e.g., political risk or sentiment) on a company's stock (e.g., stock volatility or movement) from the earnings conference call transcripts. the experimental results demonstrate that our model can effectively disentangle representations with different functionalities from text features by imposing constraints and utilizing multi-task learning. furthermore, our analysis of real-world applications highlights the causal relationship between political risks faced by a company and its stock prices, providing valuable insights for the finance and investment industry.