Causal Inference from Text: Unveiling Interactions between Variables
Authors: Yuxiang Zhou
Yulan He
Venue: EMNLP 2023 Findings (2023)
Keywords: Causal Inference
Natural Language Processing
Non-confounding Covariates
Disentanglement
TL;DR: We aim to mitigate biased issues arising from inadequate consideration of non-confounding covariates when estimating causal effects from text.

Abstract: Adjusting for latent covariates is crucial for estimating causal effects from observational textual data. Most existing methods only account for confounding covariates that affect both treatment and outcome, potentially leading to biased causal effects. This bias arises from insufficient consideration of non-confounding covariates, which are relevant only to either the treatment or the outcome. In this work, we aim to mitigate the bias by unveiling interactions between different variables to disentangle the non-confounding covariates when estimating causal effects from text. The disentangling process ensures covariates only contribute to their respective objectives, enabling independence between variables. Additionally, we impose a constraint to balance representations from the treated group and control group to alleviate selection bias. We conduct experiments on two different treatment factors under various scenarios, and the proposed model significantly outperforms recent strong baselines. Furthermore, our thorough analysis on earnings call transcripts demonstrates that our model can effectively disentangle the variables, and further investigations into real-world scenarios provide guidance for investors to make informed decisions.

Causal Inference from Text: Unveiling Interactions between Variables
Yuxiang Zhou and Yulan He He 96
V King's College London, Thhe Alan Turing Institute {yuxiang. zhou, yulan. he y@kcl. ac. uk
Abstract
cg., Pokay Shil
Policaltie Tbc
Adjusting for latent covariates is crucial for es- timating causal effects from observational tex- tual data. Most existing methods only account for confounding covariates that affect both treatment and outcome, potentially leading to biased causal effects. This bias arises from insufficient consideration of non-confounding covariates, which are relevant only to either the treatment the outcome. In this work, we aim mitigate the bias by unveiling interactions between different variables to disentangle the non-confounding covariates when estimating causal effects from text. The disentangling pro- cess ensures covariates only contribute to their respective objectives, enabling independence between variables. Additionally, we impose constraint balance representations from the treatment group and control group to alleviate selection bias. We conduct experiments on two difffrent treatment factors under various sce- narios, and the proposed model significantly outperforms recent strong baselines. Further- more, thorough analysis eamiins call transcripts demonstrates that our model can ef- fectively disentangle the variables, and further investigations into real-world scenarios provide guidance for investors to make informed deci- sions'.
Zz
T
Conbunaing
Trereesst
Couurancs
Covariates
X
Ze
ng. Geopettiion
Zy
Y
6gg. Expeerss Roveso
Stook Nowsree
Figure 1: The causal diagram for our proposed model. Shaded nodes denote observed variables. Transparent nodes denote latent covariates derived from the tran- seripts, among which, nodes outlined in red represent non-confounding covariates that impact only either the treatment or outcome O whereas the node outlined in black denotes the confounding covariate that influences both T and O.
causal effects from structural numeric data have made sigmificant progress. However, these methods requires extensive effort in treatment assignment mechanism (Halloran and Struchiner, 1995) and may suffer from ethical issues. Natural Language Processing (NLP) researchers are increasingly in- terested in estimating causal effects from observa- tional unstructured text. Early literature (Choud- hury al., 2016; Olteanu al., 2017; Pryzant et al., 2018) largely focuses on transforming texts into high-dimensional vectors using lexical features for confounding adjustment. Recent research pri- marily focuses on learning adequate representa- tions through advanced NLP models. For example, Veitch et al. (2020) fine-tuned BERT (Devlin et al., 2019) to produce contextual text representations for efficient estimation of causal effects. Later. Pryzant et al. (2021) introduced strategies involving treat- ment enhancement and text adjustment to estimate the causal effects related linguistic properties.
1 Introduction
Causal Inference (Holland, 1985; Pearl, 2000; Mor- gan and Winship, 2007; Imbens and Rubin, 2015; Hernan and Robins, 2020) aims identify how the trealment variable affects the outcome vari- able. For example, estimate the effect of "po- litical risk" (treatment) faced by a company on its "stock movement" (outcome). Early research ef- forts (Abadie and Imbens, 2004; Bardone-Cone and Cass, 2006; Kurth et al., 2006; Murnane and Willett, 2010; Keele, 2015) focusing conduct- ing randomized control trials (RCTs) to estimate
Despite their efficacy, such approaches operate under the assumption that text solely encompasses confounding covariates. This assumption raises
Our code and data are released at ht http:/ggtthhhhccee com/ zyxnIp/DIVA.

---

potential issue due the possible existence of unobserved non-confounding covariates that are pertain exclusively to either the treatment or the outcome. The causal estimation may be biased we fail to differentiate non-confounding covariates from confounding ones when learning estima- tion function through effective modeling of variable interactions (Pearl, 2010; Wooldridge, 2016). As illustrated in Figure 1, if we aim to accurately esti- mate the causal effects of treatment T (e-g, Political Risk) on the outcome (e.g.. Stock Movement), we intentionally omit the consideration of the im- pacts originating from Zy (e.g., Expected Revenue). This mirrors our decision not to account for the in- fluence of Ze (e.g., Geographical Location) on Y,
viding valuable insights for estimating causal
effects from text.
To the best of our knowledge, we are pioneers in addressing biased issues arising from inad- equate consideration of non-confounding co- variates when estimating causal effects from text.
2 Related Work
Causal estimation with text data Early efforts in estimating causal effects from text focused on using lexical features for confounding adjust- ment (Choudhury et al., 2016; Choudhury and KICI- man, 2017; Olteanu et al., 2017). Later studies investigating causal effects were devoted to effec- tively converting text into low-dimensional repre- sentations (Falavarjani al., 2017; Pham and Shen, 2017; Pryzant al., 2018; Weld et al., 2020; Cheng et al., 2021). Another line of work focused on us- ing causal formalisms to make NLP methods more reliable Goood-Doughty et al., 2018, 2021; Feder et al., 2021, 2022). Most recently, pre-trained lan- guage models such BERT (Devlin et al., 2019) significantly benefited causal estimation. For ex- ample, Veitch et al. (2020) fine-tuned BERT us- ing multi-task learning to produce contextual text representations for efficient estimation of causal effects. Later, Pryzant et al. (2021) introduced treatment-boosting and text-adjusting strategies to estimate the causal effects of linguistic properties. Our work differs from these works in three main as- pects. First, we aim to mitigate the bias that arises from insufficient consideration of non-confounding covariates in causal inference. Second, disen- tangle non-confounding covariates by encouraging independence among the variables, ensuring that each one contributes solely to its respective objec- tive. Third, we introduce regularization to balance representations from the treatment group and con- trol group, which helps to mitigate selection bias.
such inclusion could obfuscate our ability to discern the true effects originating from T.
In this paper, propose framework named Disentangling Interaction of VAriables (DIVA), specifically tailored for causal inference from text. We assume that the text carries sufficient informa- tion to identify the causal effects and consider the existence of non-confounding covariates. Drawing on the success of latent variable models for causal inference in literature (Louizos et al., 2017; Zhang Variational Auto-Encoder
et al., 2021), (VAE) (Kingma and Welling, 2014) to infer con- founding and non-confounding covariates. Addi- tionally, we design a disentanglement module to ensure that covariates only contribute to their spe- cific objectives, enabling independence between covariates. Furthermore, we propose to impose a constraint to balance representations from the treatment group and control group, which helps to mitigate selection bias.
Our contributions are summarized as follows:
We propose the Disentangling Interaction of VAriables (DIVA) approach, tailored to mit- igate the bias issue in causal inference from text.
Causal inference with latent variable model Latent variable models have demonstrated their effectiveness and gained significant popularity in causal inference (Fong and Grimmer, 2016; Srid- har and Getoor, 2019; Roberts et al., 2020). For example, Louizos et al. (2017) used Variational Auto-Encoder (VAE) (Kingma and Welling, 2014) to infer confounders from latent space estimate the effect of job training on employment follow- ing the training. Rakesh et al. (2018) inferred the causation that leads to spillover effects between
Our model is able effectively model inter- actions among diverse variables, ensuring that each variable primarily contributes to its spe- cific objective and promotes maximal indepen- dence.
Our experiments demonstrate state-of-the-art results in various scenarios. A detailed analy- sis shows that our model effectively disentan- gles different variables given inherently high- dimensional nature of text representation, pro-

---

pairs of units by incorporating VAE to learn the latent attributes as confounders. We follow the line of decomposing latent factors for causal in- ference (Hassanpour and Greiner, 2020; Wu et al., 2020; Vowels et al., 2020; Yang et al., 2021; Zhang et al., 2021). However, there are several key distinc- tions in our approach. Firstly, while previous stud- ies attempted to disentangle variables for causal inference in structured numeric data, we specifi- cally focus on estimating causal effects from tex- tual data. The inherently high- tigh-dimensional nature of text features presents substantial challenges in disentangling various variables within the latent space, leading to biased causal estimations. Sec- ondly, we tailor distinct constraints to effectively model interactions among diverse variables, ensur- ing that each variable primarily contributes to its specific objective and promotes maximal indepen- dence. Lastly, we optimize the maximum mean discrepancy loss to achieve a balanced representa- tion of samples from both treatment and control groups.
D (X,, Y.) represent the N observa- tional examples. Here, X; is the observed textual data (e.g., earnings call transcript) for the i-th exam- ple (e.g., company), and T; E {0, 13 is the binary treatment variable?. T; = indicates that the i- th example belongs to the treatment group (e.g., company faced high political risk). Conversely, T 0 indicates that the i-th example belongs to the control group (e.g., company faced low no political risk). The causal effect т₃ for the i- th example is defined as the expected difference between its potential outcome Y; (c.g., stock volatil- ity) of the treatment and control groups, known as the Individual Treatment Effect (ITE);
T Y(T,, = 1) Y,,T, 0)
(1)
One of the most challenging problems in esti- mating causal effects from observational data is the impossibility of simultancously observing both potential outcomes (T;; = 0) and (,; = 1) for given example (Rubin, 1974; Holland, 1985). In other words, D only includes the observed out- come Y; for each example, but not the unobserved counterfactual outcome, which refers to the poten- tial outcome for the i-th example in the alternative group. Nonetheless, it's feasible to identify the Conditional Average Treatment Effect (CATE) and the Average Treatment Effect (ATE) from obser- vational data under certain assumptions (Splawa- Neyman et al., 1990; Rubin, 1974; Pearl, 2009): Assumption / (Stable Unit Treatment Values As- sumption (SUTVA)): The potential outcomes of one example are not influenced by the treatment as- signed to other examples, and there are varying forms levels of the treatment that could result in different potential outcomes: Y1(41,...4. Yt(t)), and Y(T = 14) = Y₃(T)).
NLP for earnings call transcripts Earnings call transcripts (Frankel et al., 1997; Bowen et al., 2001; Price et al., 2011) have gained much popularity in financial analysis using NLP tools. Early work by Wang and Hua (2014) formulated financial risk prediction as a text regression task and used handcrafted features to improve SVM performance. Later, researchers (Qin and Yang, 2019; Sawhney et al., 2020; Sang and Bao, 2022; Pataci et al., 2022; Shah et al., 2022; Yang et al., 2022) focused on stock prediction by employing sophisticated neural networks with financial pragmatic features. An- other line of work focused on analyzing the content of earnings call transcripts (Sawhney et al., 2021; Alhamzeh et al., 2022). For example, Keith and Stent (2019) examined analysts' decision-making behavior as it pertains to the language content of carnings calls. More in line with our work, Hassan al. (2017) adapted linguistic tools to investigate the extent of political risk faced by firms over time and its correlation with stocks, hiring, and invest- ment. In contrast with this prior work, our primary focus lies on estimating causal effects between fi- nancial interests, such as the impact of political risk on stocks, rather than measuring their correlations.
Assumption 2 (Unconfoundeeneess) The po- tential outcomes are conditionally independent of the treatment given set of observed covari- ates: (Y Y()) 1L T.
Assumption 3 (Positivity): Every individual has a non-zero probability of receiving treatment or control for all observed variables: 0 < P(T
X x) < 1.
In line with the potential outcome framework outlined by Splawa-Neyman et al. (1990) and Ru- bin (1974), and with the above assumptions, we
3 Preliminaries
Causal inference from text aims to estimate the causal effects based on observed textual data. Let
We defer the scenarios involving multiple treatments for future exploration.

---

text representation from the pre-trained language model. Following that, we employ variational auto-encoder determine the posterior distribution for various latent variables. Once this distribution is obtained, we use the variable disentanglement module to encourage independence among the vari- ables, ensuring that each one contributes solely to its respective objective. Next, we utilize the disentangled variables learn the Q function via the outcome prediction task. Finally, we plug the trained Q into a pre-determined statistic to estimate the ATE.
Earnings
Nacinzua Nos Biergua
Pre-traised Languge Made
Oifiegool Hakid
Latent
Latent Variable Dheatanglemeet
Trument Podicees
Onree Priictie
Effire Exiraaion
Figure 2: DIVA architecture.
define the CATE as follows:
4.1 Text Encoder
Given a transcript æ = [wi.... Wm] that consists of words, we adopt the pre-trained language (PLM) model FinBERT (Araci, 2019)² to obtain the con- textual representation h for each transcript:
E = 2; E[ - Y(() z,] (2)
where (1) and (0) are the potential outcomes had the i-th individual received the treatment or control. X is the observed variable which is suffi- cient for causal estimation. The ATE can be written as:
= (z)
(5)
4.2 Latent Variable Inducer
E = X = xx]
(3)
Inspired by recent works (Louizos al., 2017: Zhang et al., 2021), we use the VAE to induce latent variables. Given the contextualized repre- sentation h. We compute the approximation vari- ational posterior (=(2)) using the inference net- work ()::)):
Problem Definition Defining Q(t,z)
B[],()) æ] the potential outcome of for an example with x, the objective to learn an estimation function (tt,x) that can accurately predict both the observed outcome and counterfactual outcome from D. Therefore, we can plug in Q to estimate CATE:
observing treatment
W W,, h b log 0² = W.h+
(6)
2 = 1
( 20)
(4)
n
where WT, Wo. by and by are parameters for two MLPs. and a define multivariate Gaus- sian distribution with diagonal covariance ma- trix, and A/(0 I). Then, sample from (=(zh) ~ N(,, o² to generate latent representation, where is the dimension of the representation. Under the assumption that a transcript contains not only the confounding co- variates, which affects both treatment and outcome, but also the non-confounding covariates specific
4 DIVA: Disentangling Interaction of VAriables
R the
In this section, present the proposed Disen- tangling Interaction of VAriables (DIVA) frame- work (Figure 2) for causal inference from textual earnings call transcripts. Although previous re- search (Veitch et al., 2020; Pryzant et al., 2021) has explored estimating causal effects from text, one of the core contributions of our work is that we disentangle various variables to effectively model the interactions among them. This in turn enables learn more accurate estimation function Q for predicting outcomes, thereby reducing the bias in the causal estimation.
to either the treatment or the outcome, we use separate inference networks db(h;; Φc) for infer- ring confounding covariates z., and 4,, Φ)) and b (h; σy) for inferring non-confounding covari- ates 2, and Zy, respectively. We use one-layer parameterized MLP @hh; 0) = (h Ze, Ze: zy)
We chose FinBERT due to its adaptability to text in fi- nance domain. However, other PL.Ms could serve as suitable replacements.
Our proposed DIVA framework consists of a few steps. First, we extract the contextualized

---

the decoder to reconstruct h. The objective of the latent variable inducer is maximize the evi- dence lower bound (ELBO):
where k, f4,C,y; + v}. Orth(zxi ze)
|| 2 - 1I and is the identity matrix.
Intuitively, we expect that the prediction of the treatment label should primarily rely on and že. rather than Zy. To ensure this holds, we introduce the treatment loss:
Lvac 0 log (h: 0)] 101-- - (L(
(7)
where k (), and p(zx) is the prior follows the Gaussian distribution (O,,1).
L log P(t|zy) - log pt ze)
(10)
4.3 Latent Variable Disentanglement
where E {0, 13 indicates whether the transcript belongs to the treatment group.
Despite the successful application of decomposing variables previous work (Zhang et al., 2021), unfortunately, the high-dimensional nature of text features presents significant obstacles in disentan- gling different variables in latent space, leading to biased causal estimation. As will be shown in Section 5.2 (e.g., TEDVAE CEVAE), consider- ing only non-confounding covariates, without the ability to effectively model interactions between different variables, fails to consistently achieve bet- ter performance in textual data.
Similarly, we expect the prediction of outcome should primarily rely on Zy and 2c. and define the outcome loss:
L. = Q11, Zyy =c)) =c)
(11)
where y E Y is the potential outcome. O is an MSE loss for real-valued outcomes and cross- entropy loss for the binary outcomes.
The overall objective function of the latent vari- able disentanglement module is formulated as:
To address this issue, we tailor various dis- tinct constraints to effectively disentangle non- confounding covariates from confounding ones, en- suring that each variable primarily contributes to its specific objective and promotes maximal inde- pendence.
La Lvae + aLe + 3Co + 7oon + 7/mmdd (12)
where a, B, 7, and 17 are papeer-prraeters..
4.4 Final Training Objective
Specifically, we first minimize the Maximum Mean Discrepancy (MMD) (Gretton et al., 2012) loss to balance representations from the treatment group and the control group:
Following Veitch al. (2020) and Pryzant et al. (2021), we introduce Masked Language Model (MLM) objective that predicts words that are ran- domly* masked, order to adapt text represen- tation, making it more efficient for treatment and outcome prediction. Our final objective function is multi-task learning objective:
Lunnd
Σ kEfcAy)
treat
M(z
treat contl
22 2x
(8)
where M(;) denotes the maximum mean discrep- ancy metric. ztreat and 2 zgont! are the representa- tions in the treatment group and the control group. respectively. The nice property of this loss is that minimizing the loss essentially reduces the discrep- ancy between different groups, encouraging the satisfaction of the positivity assumption. Concur- rently, it promotes the inference network to gener- alize from the factual to counterfactual domains, leading to better counterfactual inference (Johans- son al., 2016).
L = La + XLmlm
(13)
where À is the coefficient that balances the contri- bution of cach component in the training process.
5 Experiments
We conduct experiments on both semi-synthetic data and real-world application scenarios with two objectives: 1) empirically evaluate the effective- ness of our proposed model, and 2) investigate practical questions the field of finance and gain insights from the application our model to these real-world scenarios.
Next, we introduce an orthogonal loss to maxi- mize the independence between 2,, Ze, and zy as much as possible:
Lot
Σ Orth(zg3 2x)
FFlllowigg Devlin et al. (2019), we masked 15% of the words in each transcript.

---

5.1 Experimental Setup
Setup Details In our experimental evaluations. each model is trained for 30 epochs with linear warmup for the first 10% of the training steps. We employ Adam (Loshchilov and Hutter, 2019) as the optimizer. We set the maximum learning rate at 5c-5 and use a batch size of 86. We select the optimal model weights based either accuracy or the MSE loss of the Q function the develop- ment set We report the average results along with the mean absolute deviations across five runs with randomly initialized parameters.
Baselines The baseline models selected for com- parison can be broadly categorized into three groups: deep outcome regression models, latent variable models, and representation learning mod- els. Deep outcome regression models include:
TARNet Shalit et al. (2017) uses separate feed-forward networks to predict outcomes and counterfactuals.
CFRNet Shalit al. (2017) adds integral probability metric (IPM) regularization term to TARNet balance representation from differ- ent groups.
5.2 Experiments Synthetic Data Dataset
DragonNet Shi et al. (2019) extends TARNet with additional head adapts representation by modeling the propensity score.
Since ground truth causal effects ITE T; and ATE т, are typically inaccessible in real-world scenarios, directly training a model for causal inference is im- practical. Therefore, we follow Veitch et al. (2020) and Pryzant et al. (2021), using real text and meta- data to generate semi-synthetic data to empirically evaluate proposed model. We collect 1 115,880 transcripts from 1,438 companies across twelve dif- ferent sectors, for earnings calls held between May 2001 and October 2019. Then, we construct differ- ent datasets for two distinct treatment variables political risk (Tyr) and sentiment (T,) - under two separate scenarios: stock volatility (Yvot) and stock movement mYmow). To derive Tpr we follow Has- san et al. (2017) to calculate the political risk score for each transcript. We then select the top 15,000 transcripts with the highest scores the treatment group (Ipr = 1), indicating that the company faces high political risk. Conversely, we designate the bottom 15,000 transcripts with the lowest scores as the control group (Tpr = 0), suggesting these com- panies face lower or no political risk. To derive Te, we follow Maia et al. (2018) 8) and Araci (2019) to calculate the sentiment score? for each transcripts. We select the top 15,000 transcripts with the high- scores the treatment group (T, 1) and select the bottom 15,000 transcripts with the lowest scores as the control group (I, = 0). Finally, we simulate the outcomes by using the treatment vari- able T E (Tpr, Ts} along with observed covariates, Osize and Creat, which represent the size of the company in terms of the number of full-time em- ployees and the industrial sector that the company operates. The real-valued stock volatility Your
The latent variable based models are:
CEVAE Louizos et al. (2017) uses VAE to infer confounders from an unknown latent space to estimate causal effects.
TEDVAE Zhang et al. (2021) extends CEVAE by decomposing latent factors into three sets: instrumental, confounding, and risk factors.
The representation learning models are:
CausalBert Veitch al. (2020) develops approach adjust for confounding features of text to estimate causal effects from observa- tional data.
TextCause Pryzant al. (2021) introduces geattment-bossting and text-adjusting strategies estimate causal effects of linguistic proper- ties.
Whenever possible, generate results for basc- lines using the officially released source code. In cases where the code of models is not available at the time of writing, we independently implement those models using the optimal parammer settings reported in the respective papers. For a fair comparison, we use FinBERT (Araci, 2019) to encode text for generating contextualized feature representations for all models.
Evaluation Metric We evaluate the results using the precision in estimation of heteroge- neous effect (PEHE) (Hill, 2011), which re- flects model's individual-level estimation perfor- EN (Tr - 2 We also report the error of ATE estimation SATE T - N ا Å.. which measure the model's population-level estimation performance.
mance:
PPEEE
' Please refer Appendix for detailed haper-parameters. Phttps://ee thub. com/ mmscchedellerffimleeee hhttps://gtt thgbbub....bbbt.. thub. com/ProsusAT/FinBRR

---

Political risk
Sentiment
Model
/PHHE
SATE
v /PHHE
5ATH
Stock Volatility
TARNet
1.196:40.019
0.480=0.049
1.213
0.49140.049
DragonNet
1.173+40.022
0.450=0.048
1.190=0.021
0.463+00.046
CFRNet
1.169::0.020
0.445=00.045
1.185=0.020
0.455=+0.044
CEVAE
1.197=+0.025
0.477=0.050
1.211=000024
0.491=0.044
TEDVAE
1.212:40.056
0.447=+0.10
1.228=000056
0.459=+00099
CausalBert
1.097=+0.032
0.313=00.099
1.121=0.034
0.336=+0.080
TextCause
1.096:000099
0.114+0.042
1.100=0.019
0.1144:0.028
DIVA
1.003-+0.003
0.033-00.011
1.000
0.027=0.008
Stock Movement
TARNet
0,497.10.001
0.08610.009
0.49710.001
0.0891000100
DragonNet
0.497.10.003
0.08410.025
0.49710.004
0.088110.026
CFRNet
0,497.10.00 003
0.08310.025
0.49710.004
0.086110.025
CEVAE
0.499
0.076110.022
0.49910.004
0.07911.0000
TEDVAE
0.498 007
0.095100024
0,49710.007
0.09810.0223
CausalBert
0.496.10.002
0.083-10.020
0.000
0.088110.017
TextCause
0.526 008
0.03810.028
0.52210.009
0.030110.028
DIVA
0.483-10.001
0.009
0.481
0.015
Table I: The causal estimation results of different treatment factors on stock volatility and stock movement. Lower is better. The best results on each dataset are in bold. The second-best ones are underlined. The marker indicates that the p-value is less than 0.05 compared to the second-best results. The parameter setting used is (a=1, 3=1, 7=0.5, c=1) for Equation (14) and (15).
be simulated as follows:
ing that additional constraints indeed benefit the outcome regression model in causal estimation. For example, DragonNet improves upon the TARNet by 0.03 in terms of šATE based on political risk in the stock volatility scenario. We also observe that Causalbert and TextCause generally achieve bet- ter results than the deep outcome regression mod- els such as TARNet, DragonNet, and CFRNet, as well as latent variable models such CEVAE and TEDVAE. This suggests that the inclusion of the masked language modeling task has a positive im- pact on causal inference from text. Our model con- sistently outperforms all compared baseline models across both evaluation metrics and under both sce- narios. For instance, DIVA demonstrates a signif- icant improvement (with < 0.05) over the best- performing baseline TextCause and the CausalBert model.
Yout T.TT T + Bun (T (T(Ceet)) %oo)
(14)
Ba Yot) + Cr
The binary stock movement (Up or Down), Ymow can be simulated as:
Ymo Bernolli(( Bml (anl(T Ymo)
Bm22( (Teiiee Yml) cm))
(15)
where 7 (Ceize) and π (Ceect) are propensity socres estimated from meta data. Or and am control treat- ment strength. Be1 Pe2. Pm1 and 8m1 control con- found strength. 7el. 7e2, 7ml, and 7m are offset. a is the sigmoid function.
We split the dataset into the training, valida- tion, and test sets in an 8:1:6 ratio and conduct experiments in a cross-validated manner, follow- ing Egami et al. (2018) and Pryzant et al. (2021). We conduct experiments for the two different treat- ment variable Tpr and T, under the scenarios of stock volatility and stock movement, respectively. Detailed statistics each scenario can be found in the Appendix.
Interestingly, we observe that TEDVAE strug- gles to consistently outperform CEVAE. In par- ticular, TEDVAE achieves better results in terms of 8ATE but performs worse in terms of /PEHE compared CEVAE in the stock volatility sce- nario. We have contrary observations for TEDVAE and CEVAE under the stock movement setting. These results demonstrate that only considering non-confounding covariates, without the ability to
Main Results
As shown in Table 1, DragonNet and CFRNet gen- erally achieve better results than TARNet, suggest-

---

Latent Covariates
Political Risk
Sentiment
Political Risk
Sentiment
Av
V PEHE
SATE
YPEEE
6ATE
Model
VPEHE
6ATE
VPEHE
6ATE
Volarility
Stock Volatility
1.0107
1.0179
1.0022
1.0110
CausalBert
4.08
0.3858
4.1610
0.3904
1.0054
1.0140
TexiCause
4.4103
0.2968
4,3927
0.2926
1,0034
0.0332
1.0102
0.0273
DIVA
4.0589
0.0534
4.1378
0.0592
Stock
Stock Movement
0.4891
0.4857
0,0358
CausalBert 0.4992
0.0400
0,4999
0.0531
0,4900
0,4881
TexiCause 0.5306
0.0148
0.5337
0.0286
0,4845
0.0390
0,4841
0.0409
0,4831
0.0095
0.4814
0.0145
DIVA
0.4973
0.0072
0.4966
0.0103
Table 2: Ablation study of our proposed model consid- ering various latent covariates. Lower values are better.
Table 3: Causal estimation results (lower is better) under parameter settings (a=1, ,3110, >=0.5, c=4) in Equa- tion (14) and (15).
effectively modeling interactions among various variables, falls short of consistently devlivering sat- isfactory performance in textual data. However, our DIVA model consistently surpasses both CEVAE and TEDVAE by substantial margin across all sce- narios, which clearly demonstrates the importance of the constraints we introduced and underscores the effectiveness of our proposed model estimate causal effects more accurately from text data.
Political risk
Sentiment
Model
PEHE
SATE
PPHEE
6ATE
Stock Volarility
DIVA
1.003
0,033
1.010
0.027
wlo 1.004
0.062
1.011
0.068
who mmn
1.003
0.040
1.010
w/o ort
1.003
0.034
1.010
0.036
Stock Movement
DIVA 0.483
0.009
0,481
0.015
wlo mlm
0.487
0.057
0.485
0.044
Latent Covariates Analysis
wlo mmd
0.486
0.036
0.485
0.035
w/o orf 0.486
0.036
0.485
0.030
To further investigate the influence of various co- variates model performance, we conduct an in-depth analysis of DIVA, focusing the dis- entanglement of different covariates. As shown in Table 2, merely disentangling non-confounding covariates 2, or 2y from the confounding covariate ze fails to consistently achieve better results com- pared to considering only ze- Our model yields the best performance with the simultaneous dis- entanglement of 21, Ze, and Zy. This results un- derscore the necessity of comprehensive covariate disentanglement, specifically, disentangling both non-confounding covariates 2, and zy from the confounding covariate ze, as opposed a partial or singular focus.
Table 4: Ablation study of our proposed model on var- ious scenarios. Lower values are better. "w/o mlm' without masked language modeling objective; "w/o mmd' without the Maximum Mean Discrepancy (MMD) objective; "w/o without the orthogonal loss.
parameter setting, demonstrating the robustness or our DIVA model.
Ablation Study
We conducted experiments to examine the effec- tiveness of the major components of our proposed model. Table shows the ablation results on stock volatility and stock movement scenarios. We ob- serve that each component, namely Cmims Cuma- and Lon contributes to the overall performance the model. Specifically, with the removal of the Lmmd. the performance of the full model drops con- siderably in terms of /PEHE. Similarly, removing Cmlm results in considerable drop in performance, measured by JATE. These observations demon- strate the vital role played by the Land regulariza- tion term, which encourages closer representations of individuals from different groups in the latent space. Incorporating the Lulm term benefits the es-
Simulation Sensitivity Analysis
To evaluate the robustness of our proposed DIVA model, we have chosen to compare with the two strongest baseline CausalBert and TextCause, under different simulation settings (a=1, $=10, >=0.5, e=4) in Equation (14) and (15). As shown in Table our DIVA model consistently outper- forms both CausalBert and TexICause across both evaluation metrics and under both scenarios. These results suggest that the superior performance of our model is not sensitive to changes the simulation

---

Scrti Vice
L.I.I.11
0.30
Causs thlect
0.00
0.06
0.00
-0.05
-0.10
30
Trading Days
Figure 3: Causal effect of political risk and sentiment the actual stock over trading days.
Figure 4: Causal effect of political risk stock volatil- ity over companies in different sectors.
timation of CATE from text data, This phenomenon aligns with previous studies such as (Veitch et al., 2020;
Result As shown in Figure 3, we observe that the causal effects political risk on stock increases the short term (3 days) and begin to decline over time. Conversely, the causal effect of sentiment on stock movement decreases over time.
Pryzant et al., 2018).
5.3 Real World Scenario Application
To answer the questions of "How does political risk faced by company affect its stock?" and "How does the sentiment conveyed in the earning call transcription of a company affect its stock?", we apply our proposed model to estimate the treatment effect of political risk and sentiment actual stock volatility and stock movement.
Analysis To further investigate the effect of po- litical risks on the stock market for different types of companies, we examine the causal effect of po- litical risk faced by companies in different sectors on their stock prices. Figure 4 shows that the stock volatility of companies in Industrials Goods, Real Estate, and Energy most significantly affected by the political risk they faced, while companies in Consumer Cyclical and Technology are affected the smallest extent. The political risks faced by the Healthcare companies have no effect on their stock volatility.
Stock Volatility Following Qin and Yang (2019) and Kogan et al. (2009), we obtain the stock prices from Yahoo Finance by stock-market-scrapee" and calculate stock volatility as:
Ek (rr F)²
V((-u.) =
(16)
6 Conclusion
where Pt. 1 is the stock return between Pi the close of trading day t 1 and day f, P; is the divedend-adjuster closing stock price at t. F is the mean of r. over the period of day μ day i. We choose different μ {3, 15, 30) evaluate the short-term and long-term causal effects.
In this paper. we propose DIVA, novel framework designed specifically for causal inference from text. We verify its effectiveness by estimating the causal effects of treatment factor (e.g.. political risk or sen- timent) on company's ' stock (e.g., stock volatil- ity or movement) from the earnings conference call transcripts. The experimental results demon- strate that our model can effectively disentangle representations with different functionalities from text features by imposing constraints and utilizing multi-task learning. Furthermore, our analysis of real- world applications highlights the causal rela- tionship between political risks faced by a company and its stock prices, providing valuable insights for the finance and investment industry.
Stock Movement Following (Medya et al., 2022), we define stock movement as:
1, if ri 2 ((()))
174
(17)
0. else
where ([tppt] |] is the mean stock volatility over the period of day t f to day t.
"https://finance yahoo. com/ hhttps://githun www.lb..cc///aaeeeeeee com//unjannandy/ stock-market-scrre

---

Limitations
Mummun De Choudhury, Emre Kiciman, Mark Dredze, Glen Coppersmith, and Mrinal Kumar. 2016. Discov- ering shifts to suicidal ideation from mental health content in social media. In Proceedings of the 2016 CHI CHFCS.
Our work has a number of limitations. First, we constructed balanced dataset in which the number of transcripts in the treatment group is equal to that in the control group. While this facilitated rela- tively easier causal estimation, it does not account for the selection bias that commonly exists in real- world scenarios. Consequently, causal estimation in such scenarios becomes more challenging, Sec- ond, we modeled the relation between treatment factors and stocks as a linear relation. However, in reality, this relationship is likely to be much more complex and nonlinear. A more precise modeling of this relationship would enhance the accuracy of causal estimation.
Munmun De Choudhury and Emre Kieıman. 2017. The language of social support in social media and its effect on suicidal ideation risk. Proceedings of the AAAI
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of NAACI.
Naoki Egami, Christian Fong, Justin Grimmer, Mar- garet E. Roberts, and Brandon M Stewart. 2018. How to make causal inferences using texts. Science Ad- vances.
A Acknowledgements
Seyed Amin Mirlohi Falavarjani, Hawre Hossini Zeinab Noorian, and Ebrahim Bagheri. 2017. Es- timating the effect of exercising on users' online be- havior. Proceedings of AAAI.
We would like to thank the anonymous review- ers, our meta-reviewer, and senior area chairs for their constructive comments and support with our work. We would also like to thank Rui Qiao for helpful discussions and suggestions, and Zhanming Jie for feedback on the manuscript. This work was funded by the the UK Engineering and Physical Sci- ences Research Council (grant no. EP/T01711 2/1, EP/T017112/ EP/VO48597/1). YH is supported by a Turing AI Fellowship funded by the UK Re- search and Innovation (grant no. EP/V002079//1 EP/V020579/2).
Amir Feder, Katherine A. Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Ja- cob Eisenstein, Justin Grimmer, Roi Reichart, Mar- garet F. Roberts, Brandon M. Stewart, Victor Veitch, and Diyi Yang. 2022. Causal inference in natural lan- guage processing: Estimation, prediction, interpreta- tion and beyond. Trunsactions of the Association for Computational Linguistics.
Amir Feder, Nadav Oved, Uri Shalit, and Roi Reichart. 2021. CausaLM: Causal model explanation through counterfactual language models. Computational Lin- guistics, (2).
References
Christian Fong and Justin Grimmer. 2016. Discovery of treatments from text corpora. In Proceedings of ACL.
Alberto Abadie and Guido Imbens. 2004. Large sample properties of matching estimators for average treat- ment effects. Econometrica.
Richard Frankel, Marilyn F. Johnson, and Douglas J. Skinner. 1997. An empirical examination confer- ence calls as voluntary disclosure medium. Finan- cial Accounting.
Alaa Alhamzeh, Romain Fonck, Erwan Versmée, Elöd Egyed-Zsigmond, Harald Kosch, and Lionel Brunie. 2022. It's time reason: Annotating argumentation structures financial earnings calls: The FinArg dataset. In Proceedings of Workshop on FinNLP.
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alex Smola. 2012. A ker- nel two-sample test. J. Mach. Learn. Res.
Dogu Araci. 2019. Finbert: Financial sentiment analy- sis with pre-trained language models. ArXiv preprint.
M. Fliizbeth Halloran and Cláudio José Struchiner. 1995. Causal inference in infectious diseases. Epi- demiology.
Anna M. Bardone-Con Cone and Kamila M Cass. 2006. In- vestigating the impact of pro-anorexia websites: a pilot study. European Eating Disorders Review.
Tarek Alexander Hassan, Stephan Hollander, Laurence van Lent, and Ahmed Tahoun. 2017. Firm-level po- litical risk: Measurement and effects. Political Econ- omy: Government Expenditures & Related Policies eJournal.
Robert M. Bowen, Angela K. Davis, and Dawn Mat- sumoto. 2001. Do conference calls affect analysts' forecasts? Financial Accounting eJournal.
Lu Cheng, Ruocheng Guo, and Huan Liu. 2021. Esti- mating causal effects of multi-aspect online reviews with multi-modal proxies. Proceedings of WSDM.
Negar Hassanpour and Russell Greiner. 2020. Learn- ing disentangled representations for counterfactual regression. In Proceedings of ICLR.

---

MA Hernan and J Robins. 2020. Causal inference: What if. boca raton: Chapman & hill/cre.
Richard Murmane and John B. Willett. 2010. Methods matter: Improving causal inference in educational and social science research.
Jennifer L. Hill. 2011. Bayesian nonparametric model- ing for causal inference. Journal of Computational and Graphical Statistics.
Alexandra Olteanu, Onur Varol, and Emre Kıcıman. 2017. Distilling the outcomes of personal experi- ences: propensity-scone scoree analysis of social me- dia. Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing.
Paul Holland. 1985, Statistics and causal inference. Journal of the American Statistical Association.
Guido Imbens and Donald B. Rubin. 2015, Causal in- ference for statistics, social, and biomedical sciences; An introduction.
Hilal Pataci, Yunyao Li, Yannis Katsis, Yada Zhu, and Lucian Popa. 2022. Stock price volatility predie- tion: A case study with AutoML. In Proceedings of Workshop on FinNLP.
Fredrik D. Johansson, Uni Shalit, and David A. Sontag, 2016. Learning representations for counterfactual inference. In Proceedings of ICMI.
Judea Pearl. 2000. Causality: Models, reasoning and inference.
Luke J. Keele. 2015. The statistics of causal inference: A view from political methodology. Political Analy- sis.
Judea Pearl. 2009. Causal inference in statistics: An overview. Statistics Surveys.
Judea Pearl. 2010. On a class of bias-amplifying vari- ables that endanger effect estimates. In Proceedings of UAI.
Katherine Keith and Amanda Stent. 2019. Modeling fi- nancial analysts' decision making via the pragmatics and semantics of earnings calls. In Proceedings of ACL.
Thai Thuy Pham Pham and Yuanyuan Shen. 2017. A deep causal inference approach to measuring the ef- fects of forming group loans in online non-profit microfinance platform. arXiv: Machine Learning.
Diederik P. Kingma and Max Welling. 2014. Auto- encoding variational bayes. In Proceedings ICLR.
S. McKay Price, James S. Doran, David R. Peterson, and Barbara A. Bliss. 1. Farnings conference calls and stock returns: The incremental informa- tiveness of textual tone. ERN: Uncertainty Risk Modeling (Topic).
Shimon Kogan, Dimitry Levin, Bryan R. Routledge, Jacob Sagi, and Noah A. Smith. 2009. Predict- ing risk from financial reports with regression. In Proceedings of NAACL.
Tobias Kurth, Alexander M. Walker, Robert J. Glynn, K. Amold Chan. J. Michael Gaziano, Klaus Berger, and James M. Robins. 2006. Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nomuniform effect. American journal of epidemiology.
Reid Pryzant, Dallas Card, Dan Jurafsky, Victor Veitch, and Dhanya Sridhar. 2021. Causal effects of linguis- properties. In Proceedings of NAACL.
Reid Pryzant, Kelly Shen, Dan Jurafsky, and Stefan Wagner. 2018. Deconfounded lexicon induction for interpretable social science. In Proceedings of NAACL.
Ilya Loshchilov and Frank Hutter. 2019, Decoupled weight decay regularization. In Proceedings of ICLR.
Yu Qin and Yi Yang. 2019. What you say and how you say it matters: Predicting stock volatility using verbal and vocal cues. In Proceedings of ACL.
Christos Louizos, Uri Shalit, Joris M. Mooij. David A. Sontag, Richard S. Zemel, and Max Welling. 2017. Causal effect inference with deep latent-variable mod- els. In Proceedings of NeurlPS.
Vineeth Rakesh, Ruocheng Guo, Raha Moraffah, Nitin Agarwal, and Huan Liu. 2018 Linked causal varia- tional autoencoder for inferring paired spillover ef- fects. In Proceedings of CIKM.
Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. 2018 Www' 18 open challenge: Financial opinion mining and question answering. Companion Proceedings of The Web Conference.
Margaret E. Roberts, Brandon M Stewart, and Richard A. Nielsen. 2020. Adjusting for confound- ing with text matching. American Journal of Political Science.
Sourav Medya, Mohammad Rasoolinejad, Yang Yang, and Brian Uzzi. 2022. An exploratory study of stock price movements from earnings calls. Companion Proceedings of the Web Conference 2022.
Donald B. Rubin. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology.
Yunxin Sang and Yang Bao. 2022. DialogueGAT: A graph attention network for financial risk prediction by modeling the dialogues earnings conference calls. In Findings of EMNLP.
Stephen L. Morgan and Christopher Winship. 2007. Counterfactuals and causal inference: Methods and principles for social research.

---

Ramit Sawhney, Arshiya Aggarwal, and Rajiv Ratn Shah. 2021. An empirical investigation of bias in the multimodal analysis of financial earnings calls, In Proceedings of NAACL.
Jeftrey M. Wooldridge. 2016. Should instrumental vari- ables be used as matching variables. Research in Economics.
Anpeng Wu, Kun Kuang, Junkun Yuan, Bo Li, Pan Zhou, Jianrong Tao, Qiang Zhu, Yueting Zhuang, and Fei Wu. 2020. Learning decomposed representation for counterfactual inference. ArXiv, abs/2006.0700 070400
Ramit Sawhney, Piyush Khanna, Arshiya Aggarwal, Taru Jain, Puneet Mathur, and Rajiv Ratn Shah. 2020. VoITAGE: Volatility forecasting via text audio fusion with graph convolution networks for earnings calls. In Proceedings of EMNLP.
Linyi Yang, Jiazheng Li, Ruihai Dong, Yue Zhang, and Barry Smyth. 2022. Numhtml: Jumeric-oriente hi- erarchical transformer model for multi-task financial forecasting. In Proceedings of AAAI.
Raj Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, and Diyi Yang. 2022. When FLUE meets FLANG: Benchmarks and large pre- trained language model for financial domain. In Pro- ceedings of EMNLP.
Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. 2021. Causalvae: Disen- tangled representation learning via neural structural causal models. In JEEE Conference CVPR.
Uri Shalit, Fredrik D. Johansson, and David Sontag. 2017. Estimating individual treatment effect: gener- alization bounds and algorithms, In Proceedings of ICML.
Weijia Zhang, Lin Liu, and Jiuyong Li. 2021. Treatment effect estimation with disentangled latent factors. Proceedings of AAAI.
Claudia Shi, David M. Blei, and Victor Veitch. 2019. Adapting neural networks for the estimation of treat- ment effects. Proceedings of NewrIPS.
J. Splawa-Neyman, Dorota Dabrowska, and Terence P. Speed. 1990. On the application of probability the- ory to agricultural experiments. essay on principles. section 9. Statistical Science.
Dhanya Sridhar and Lise Getoor. 2019. Estimating causal effects of tone in online debates. In Proceed- ings IJCA.
Victor Veitch, Dhanya Sridhar, and David M. Blei. 2020. Adapting text embeddings for causal inference. In Proceedings of UAI.
Matthew James Vowels, Necati Cihan Camgoz, and R. Bowden. 2020. Targeted vae: Variational and targeted learning for causal inference. IEEE Interna- tional Conference on SMDS.
William Yang Wang and Zhenhao Hua. 2014. A semi- parametric Gaussian copula regression model for pre- dicting financial risks from earnings calls. In Pro- ceedings of ACL.
Galen Cassebeer Weld, Peter West, Maria Glen- ski, David T. Arbour, Ryan A. Rossi, and Tim Althoff. 2020. Adjusting for confounders with text: Challenges and an empirical evaluation frame- work for causal inference. Proceedings of AAAI, abs/2009.09961.
Zach Wood-Doughty, Ilya Shpitser, and Mark Dredze. 2018. Challenges of using text classifiers for causal inference. Proceedings of EMNLP.
Zach Wood-Doughty, Ilya Shpitser, and Mark Dredze. 2021. Generating synthetic text data to evaluate causal inference methods. ArXiv preprint.

---

A Data Statistics
Table Al shows the detailed statistics of each sce- nario.
Train
Dey
Test
Treatment
Treat.
Ctrl.
Treat.
Ctrl. Treat. # Ctrl.
Stock Volaniliy
Political Risk
8,000
8,000
1,000
1,000
6,000
6,000
Sentiment
8,000
8,000
1,000
1,000
6,000
6,000
Stock Movement
Political Risk
8,000
8,000
1,000
1,000
6,000
6,000
Sentiment
8,000
8,000
1,000
1,000
6,000
6,000
Table Al: Data statistics.
B Hyper-parameters
Table A2 shows the detailed hyper-panne paremeerrsseerrss set- ting of DIVA under all scenarios.
Hyper-parameter
Framework
Pytorch
GPUs
A100
Batch Size
86
Epoch
30
Warmup Steps
10%
Learning Rate
5.00E-05
Optimizer
A
Adam e
IE-08
Max Sequence Length
512
Hidden Size
798
Hidden Layer
12
Dropout probability
0.2
Latent Dimension Coefficient a
200
Coefficient /
Coefficient 7
0.1
Coefficient r
0.1
Coefficient À
0.01
Table A2:
of DIVA.


Reviews:
[{"review_id": "Z75i9kJ0cl", "reviewer": "Reviewer_EVAH", "summary": "", "questions": "Besides the above-mentioned questions, I would like to ask the following ones:\n\n1. What would be the largest challenge to further generalize the method to the nonlinear setting?\n\n2. If the hidden variables are actually dependent on the ground-truth data-generating process, why keep them independent and disentangle them?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful comments. We hope the responses below are helpful in addressing your concerns.\n\n**R(1) Regarding more discussion on the assumed causal diagram:** As mentioned in lines 66-85, our motivation stems from previous studies that estimated causal effects from text. These studies made an assumption that text only contains confounding covariates. This poses a potential bias because there may be unobserved non-confounding covariates that pertain specifically to either the treatment or the outcome. While attempts have been made to address this problem in causal inference involving *structured numeric data*, as seen in prior works [1] (e.g., the TEDVAE baseline we compared in Table 1), our aim revolves around investigating the possibility of mitigating this bias concern by more accurately estimating causal effects from *text*,  considering the presence of two types of non-confounding covariates (e.g., $z_t$ and $z_c$) .  \n\nTo achieve this, we proposed using VAE for inferring the variables and designed a disentanglement module aimed at differentiating non-confounding covariates from confounding ones. Moreover, we tailored distinct loss functions to effectively model interactions among diverse variables. This approach ensures that each variable contributes primarily to its specific objective, fostering as much  independence as possible. As we have analyzed in lines 554-562, merely considering non-confounding covariates is insufficient without effectively modelling interactions among the variables. For instance, when applied to textual data within the context of stock movement scenarios, TEDVAE struggles to consistently outperform CEVAE. However, our proposed model DIVA consistently surpasses all baseline models across various evaluation metrics and scenarios, which demonstrates the significance of the constraints we introduced and underscores the  effectiveness of our DIVA model.\n\nWe appreciate your valuable suggestion and will definitely include this discussion in the revised version to improve clarity. \n\n\n**R(2) Regarding  causal discovery methods:** Textual data presents unique characteristics that make traditional causal inference methods less straightforward to apply. For example, Fast Causal Inference (FCI) has a difficulty in dealing with high-dimensional data. Textual data has a high-dimensional nature, with a vast number of words or features, making it harder to identify causal relationships among many variables. Generalized Independent Noise (GIN) condition [2] was proposed to estimate linear non-Gaussian latent variable causal model. However, textual data might involve complex nonlinear interactions that are difficult to model accurately. Textual data might contain latent variables, such as political risks, sentiments, or unobservable context. These latent factors can influence both the observed text and the outcomes, leading to confounding and challenges in isolating causal effects. For the reasons highlighted above, we proposed our VAE-based model for disentangling latent confounders instead of directly applying traditional causal inference methods for causal structure discovery.\n\n**Q(1) Regarding the challenge of nonlinear setting:** This is an excellent question. We would like to share our thoughts here. Firstly, achieving identifiability of the causal graph requires more stringent assumptions. This might encompass the Markov assumption, causal sufficiency, acylicity, the additive noise assumption [3,4] or the post-nonlinear assumption [5]. However, these assumptions made by specific models could potentially deviate from reality when applied to  real-world application. Secondly, concisely summarizing nonlinear functions (e.g., equation (14) and (15)) is an open problem [6]. In this context, drawing upon domain expertise is crucial to ensure that these equations closely mirror real-world scenarios.\n\n[1] Zhang, Weijia, Lin Liu and Jiuyong Li. “Treatment effect estimation with disentangled latent factors.” AAAI (2020).\n\n[2] Xie, F., Cai, R., Huang, B., Glymour, C., Hao, Z. and Zhang, K., 2020. Generalized independent noise condition for estimating latent variable causal graphs. Advances in neural information processing systems, 33, pp.14891-14902.\n\n[3] Hoyer, Patrik O., Dominik Janzing, Joris M. Mooij, J. Peters and Bernhard Scholkopf. “Nonlinear causal discovery with additive noise models.” NIPS (2008).\n\n[4]Peters, J., Joris M. Mooij, Dominik Janzing and Bernhard Scholkopf. “Causal Discovery with Continuous Additive Noise Models.” arXiv: Machine Learning (2013): n. Pag.\n\n[5] Zhang, Kun and Aapo Hyvärinen. “On the Identifiability of the Post-Nonlinear Causal Model.” Conference on UAI (2009).\n\n[6] Dominik Janzing, David Balduzzi, Moritz Grosse-Wentrup, and Bernhard Schölkopf. ‘Quantifying causal inﬂuences’. In: Ann. Statist. 41.5 (Oct. 2013), pp. 2324–2358.\n\n\n**Q(2) Regarding the reason to disentangle the variables:** The reason why we disentangle these variables is that we aim to accurately model the interactions between the different variables so that we can learn a more accurate $\\hat Q$ to predict the outcomes, thereby reducing the bias in causal estimation. And the reasons why we assume the non-confounding variables please refer to the response of R(1).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. It considered latent non-confounding covariates, which, according to the authors, was ignored in previous works.\n\n2. Experiments support the claimed superiority in various scenarios.\n\n3. The writing is generally clear.", "reasons_to_reject": "1. More discussion on the assumed causal diagram would be helpful to really make the problem setting more intuitive since it is the foundation of all the following results.\n\n2. Since the existence of both confounding/non-confounding unobserved covariates have been assumed, why not consider some causal discovery methods that could deal with hidden variables to find the causal graph (e.g., FCI, GIN), instead of making it a hypothesis of the hidden data-generating process? Perhaps it could make the whole framework more complete.", "questions_to_authors": "Besides the above-mentioned questions, I would like to ask the following ones:\n\n1. What would be the largest challenge to further generalize the method to the nonlinear setting?\n\n2. If the hidden variables are actually dependent on the ground-truth data-generating process, why keep them independent and disentangle them?", "paper_topic_and_main_contributions": "The manuscript sets out to reduce bias by examining the interactions among various variables. Its goal was to separate non-confounding covariates during the estimation of causal effects from text. The method ensures that covariates solely influence their intended objectives, promoting variable independence. Notably, the method introduces a constraint to balance representations from both treated and control groups, addressing selection bias. The experiments span two distinct treatment factors across multiple contexts. The model exhibits superior performance when compared to contemporary leading benchmarks.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TvXM01jxqr", "reviewer": "Reviewer_5SCp", "summary": "", "questions": "1. What is the main technical contribution of the proposed method?\n2. How can the proposed method verifying the causal assumptions and the causal relations in real-world text data?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful comments. We hope the responses below are helpful in addressing your concerns.\n\n\n**R(1) Regarding clearly justifying the novelty of this work:**  We would like to thank the reviewer for bringing the two interesting works to our attention. There exist notable distinctions between [1,2] and our proposed method:\n\n1. While [1,2] aimed to  disentangle different variables for counterfactual regression in structured numeric dataset such as IHDP, Jobs and Twins, a central contribution of our work is the recognition of biased causal estimation inherent in textual data. This bias arises from inadequate consideration of non-confounding covariates and suboptimal modeling of interactions between these variables. As discussed in lines 554-562, exclusively focusing on non-confounding covariates, without the ability to effectively model interactions, fails to  consistently achieve superior performance in textual data (e.g., TEDVAE v.s. CEVAE on the stock movement scenario). However, our proposed model consistently outperforms all compared baseline models across various evaluation metrics and scenarios, which demonstrates the indispensability of imposing constraints and validates the effectiveness of our proposed model DIVA.\n\n2. Another key difference lies in the formulation of equation (10), which we designed to counteract the influence of the independent variable $z_o$ during the modeling of  the treatment variable. This step has been demonstrated to be essential for casual estimation, aligning with [3]’s work.\n\n3. Our approach aligns with [4] and [5], which incorporated a Masked Language Model objective in order to adapt text representation, making it more efficient for treatment and outcome prediction in textual data.\n\n4. Notably, we constructed a novel dataset which serves as a benchmark for causal inference research in the NLP community.\n\nWe will certainly acknowledge and cite [1,2] in our revised version and include a discussion highlighting both the similarities and differences between our work and theirs.\n\n\n\n\n\n\n**R(2) Regarding the evaluation under different parameter settings:** The parameter setting in Table 1 is ($\\alpha$=1, $\\beta$=1, $\\gamma$=0.5, $\\epsilon$=1); ,We conduct an experiment on different settings of ($\\alpha$=1, $\\beta$=10, $\\gamma$=0.5, $\\epsilon$=4) in eq (14) and eq (15), following your suggestion. We have chosen to compare the two strongest baseline CausalBert and TextCause in Table 1 under the stock movement scenario due to the time limit. Thank you for the valuable question, and we will complement such an experiment in our revised version.\n\n\n|            |        Political Risk         |                               |           Sentiment           |                                            |\n|:----------:|:-----------------------------:|:-----------------------------:|:-----------------------------:|:------------------------------------------:|\n|            |     $\\sqrt{\\text{PEHE}}$      |      $\\delta\\text{ATE}$       |     $\\sqrt{\\text{PEHE}}$      |             $\\delta\\text{ATE}$             |\n| CausalBert |       0.4992$\\pm$0.0007       |       0.0400$\\pm$0.0133       |       0.4999$\\pm$0.0009       |             0.0531$\\pm$0.0136              |\n| TextCause  |       0.5306$\\pm$0.0051       |       0.0148$\\pm$0.0104       |       0.5337$\\pm$0.0047       |             0.0286$\\pm$0.0195              |\n|    DIVA    | $\\textbf{0.4973$\\pm$0.0013} $ | $\\textbf{0.0072$\\pm$0.0057}$  | $\\textbf{0.4966$\\pm$0.0018}$  |        $\\textbf{0.0103$\\pm$0.0061}$        |\n\n\n**R(3) Regarding how the proposed method addresses the problem of causal assumption:** Rather than verifying the causal assumption, we aim to adjust for latent covariates for estimating causal effects from observational textual data. Our assumption (SUTVA, Unconfoundedness, and Positivity) aligns with the established norm for identifying causal effects from observational data [6-9]. Within the context of textual data, the sole atypical assumption we need to adopt to ensure the validity of the estimating procedure is to assume that the textual data carries sufficient information to estimate the causal effects which has been justified in [4,5]. Their theorems (e.g., Theorem 3.1 and Theorem 3.2 in [4]) still hold in our assumed causal graph, primarily because all the backdoor paths were blocked when we conditioned on the text (e.g., the earnings call transcript in Figure 1). \n\nWe will make this part clear in the revised version.\n\n**Q(1) Regarding the main technical contribution:**  Please refer to our response to R(1). \n\n**Q(2.1) Regarding verifying the causal assumption:** please refer to the response to R(3)\n\n**Q(2.2) Regarding the causal relation in real word-text data:** This question is indeed of great significance. We would like to share our thoughts here. Unfortunately, the “gold/true” causal effects are generally unavailable because of the fundamental problem which we have mentioned in lines 231-240. But what we can do is to establish a connection between treatment and outcome (e.g. as exemplified by equation (14) and (15)) grounded in domain expertise under appropriate assumption, and then design a model to conduct simulation experiments. If the model can achieve good results in these simulation experiments, we can reasonably assume that our designed model can effectively estimate the causal effect when the underlying hypothesis holds in the real-world scenarios. \n\nMoreover, with enhanced expertise, the formulation becomes more accurate, leading to improved model results. This, in turn, increases the likelihood of accurately estimating causal effects between variables in actual scenarios. While we may lack knowledge of the \"gold/true\" causal effects within real-world scenarios, the potency of the predicted causal effects can still serve as a valuable resource for making well-informed decisions in practical situations.\n\n[3] Pearl, Judea. “On a Class of Bias-Amplifying Variables that Endanger Effect Estimates.” ArXiv abs/1203.3503 (2010): n. Pag.\n\n[4] Shi, Claudia et al. “Adapting Neural Networks for the Estimation of Treatment Effects.” NeurIPS (2019).\n\n[5] Pryzant, Reid et al. “Causal Effects of Linguistic Properties.” NAACL (2021)\n\n[6] Rubin, Donald B.. “[On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9.] Comment: Neyman (1923) and Causal Inference in Experiments and Observational Studies.” Statistical Science 5 (1990): 472-480.\n\n[7] Rubin, Donald B.. “Estimating causal effects of treatments in randomized and nonrandomized studies.” Journal of Educational Psychology 66 (1974): 688-701.\n\n[8] Pearl, Judea. “Causal inference in statistics: An overview.” Statistics Surveys 3 (2009): 96-146.\n\n[9] Imbens, Guido and Donald B. Rubin. “Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction.” (2015).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The studied problem is important.\n2. The presentation is clear and easy to follow.\n3. The experimental results are promising.", "reasons_to_reject": "Some concerns for this paper:\n\nFirst, disentangling different factors in a causal model with neural networks has been studied in previous causal learning literature (e.g., [1,2]). Besides, the representation balancing from treatment and control group is a well-adopted technique in existing causal studies. Therefore, it is important to clearly justify the novelty of this work.\n\nSecond, to evaluate the robustness of the proposed method, it is suggested to include evaluation under different parameter settings.\n\nThird, in text data, verifying the causal assumptions and identifying the causal relations would be particularly difficult. How can the proposed method address this problem?\n", "questions_to_authors": "1. What is the main technical contribution of the proposed method?\n2. How can the proposed method verifying the causal assumptions and the causal relations in real-world text data?", "paper_topic_and_main_contributions": "This paper investigates the problem of causal inference from text. Different from most existing causal inference methods which only adjust for confounders, this work disentangles different factors by proposing a VAE-based framework DIVA. The proposed method is evaluated on both synthetic data and real-world data with promising results.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "[1] Hassanpour, Negar, and Russell Greiner. \"Learning disentangled representations for counterfactual regression.\" International Conference on Learning Representations. 2019.\n[2] Wu, Anpeng, et al. \"Learning decomposed representation for counterfactual inference.\" arXiv preprint arXiv:2006.07040 (2020).\n"}}, {"review_id": "0y3GOR7xQu", "reviewer": "Reviewer_nCyD", "summary": "", "questions": "NA", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your suggestions. We hope the responses below are helpful in addressing your concerns.\n\n**R(1.1) Regarding the motivation of considering non-confounding covariates:** Our motivation, as mentioned in lines 66-85, stems from the observation that prior works in estimating causal effects from text assume text only encompasses confounding covariates. This presumption poses a potential bias issue since there are some unobserved non-confounding covariates that are only relevant to either the treatment or the outcome. We want to explore if we can mitigate the bias and accurately estimate causal effects from text, considering the existence of such types of non-confounding covariates.\n\n**R(1.2) Regarding the example in Figure 1:** We apologize for any confusion caused. The example we want to illustrated in Figure 1 is that, when focusing on accurately estimating the causal effects of “Political Risk (treatment)” on the “Stock Movement (Outcome)”, we intentionally omit the consideration of the impacts stemming from  the “Expected Revenue” This parallels our disregard for factoring in the influence of the “Geographical Location (confounder)” on the “Stock Movement (Outcome)”, as such an inclusion could obfuscate our ability to discern the true effects originating from the \"Political Risk.\". Our objective is to learn a more accurate estimation function $\\hat Q$ in equation (4) by efficiently unveiling the interaction between these variables. \n\nWe will make this part clearer in the revised version.\n\n**R(1.3) Regarding combining the variables will create a new confounder:** As highlighted in our response to  R(1.1), our rationale behind assuming the existence of non-confounding covariates exclusively relevant to either the treatment or the outcome arises from the recognition that considering only the confounding covariates in text can lead to the bias issue, which has been demonstrated by [5,6]. A similar assumption has been made in causal inference with structured numeric data such as [7] (TEDVAE baseline we compared in Table 1). \n\nAs we have analyzed in lines 554-562, disentangling latent variables without imposing any constraints is inefficient. Considering only non-confounding covariates, without the ability to effectively model interactions between different variables fails to  consistently achieve better performance in textual data (e.g., TEDVAE v.s. CEVAE on the stock movement scenario). However, our proposed model consistently outperforms all compared baseline models across evaluation metrics and scenarios, which demonstrated the necessity of imposing constraints and the effectiveness of our proposed model DIVA.\n\n\n**R(2) Regarding the proposed model is a bit off to the studied problem:** Our intention is not to encompass ALL non-confounding covariates. As discussed in lines 66-85, previous attempts at estimating causal effects from text have presumed that text solely contains confounding covariates, potentially introducing bias. We want to explore if we can mitigate the bias and accurately estimate causal effects from text, considering the existence of such types of non-confounding covariates. To achieve this, we proposed using VAE to infer the unobserved non-confounding covariates and designed a disentanglement module to differentiate those non-confounding covariates from confounding ones. Additionally, we tailored various loss functions to accurately model interactions between different variables, ensuring that variables only contribute to their specific objectives and enabling independence from each other as possible.\n\nWe will make this part clearer in the revised version.\n\n\n**R(3.1) Regarding orthogonal module similar to prior work [1]:** The Orthogonal Module in Figure 2 corresponds to equation (9), which was not used in [1]. We want to highlight some crucial  differences between [1] and our proposed method:\n\n1. As illustrated in Fig 1 of [1], the authors regard the rich source of information gathered from user-item interaction and social network as a suitable proxy for accounting for confounders. They essentially assume that information only contains confounding covariates that affect both treatment and outcome. This diverges fundamentally  from our conceptualization of causal graphs when estimating causal effects (e.g., CATE) from text. Using Fig 1 of [1] as an example, if we aim to estimate causal effects of “Exposure”(treatment) on “Rating” (outcome) by conditioning only on the “rich source of information” (represented by the dashed box in Fig 1), the estimation may suffer from bias. This is because the backdoor path “exposure<-Confounders->Ratings” remains open, introducing a non-causal association. In contrast, within our assumed causal graph, all backdoor paths are blocked when estimating the causal effects (e.g, CATE) of “Political Risk” on “Stock Movement” using equation (4) upon conditioning on earnings call transcripts (Figure 1 in our paper), all of the association that flows from “Political Risk” to “Stock Movement” is purely causal.\n\n2. Another major difference is that we aim to address the bias problem when estimating causal effects (e.g. PEHE, ATE) from text rather than improving the performance (e.g., MAE, MSE) of some specific tasks under the causal lens. More specifically, as mentioned in lines 66-85, previous studies that estimated causal effects from text made the assumption that text only contains confounding covariates. This poses a potential bias because there may be unobserved non-confounding covariates that pertain specifically to either the treatment or the outcome.\n\n3. We formulated equation (10) to counteract the influence of independent variable $z_o$ when modeling the treatment variable. This step is deemed essential for causal estimation. \n\n4. As we clarified in our response to R(3.1) the orthogonal loss has not been used in [1]. Furthermore, our approach aligns with [8] and [9], which introduced a masked language model objective in order to adapt text representation, making it more efficient for treatment and outcome prediction.\n\nWe will certainly acknowledge and cite [1] in our revised version of the paper, along with a concise discussion outlining the similarities and differences.\n\n\n**R(3.2) Regarding the technical contribution is limited:** As discussed in our response to R(3.1), we tailored equation (10), more specifically, aiming to maximize the probability of  $z_o$ (independent with T) for predicting the treatment. In addition, while the prior works employing VAE and MMD have been explored, a notable contribution of our work is that we identified the causal bias issue arising from inadequate consideration of non-confounding covariates when estimating causal effects (e.g., PEHE, ATE) from text. As we have mentioned in lines 69 -85 and reiterated in our response to R (1.1),  existing approaches suffer from this issue as they are not able to differentiate the non-confounding covariates (e.g., Policy Shifts and Expected Revenue in Figure 1) from confounding ones (e.g., Geographical) under their assumption. To the best of our knowledge, we are pioneers in presenting  an effective solution to address this issue by unveiling interactions between variables through our carefully devised disentanglement model that encourages independence among the variables, ensuring that each variable contributes solely to its designated objective.\n\n**R(4.1) Regarding Baseline quite old:** We would like to point out that some of the experiment results can be viewed as surrogates for more recent baselines. For instance, in the Ablation study, the results for DIVA w/o mlm and DIVA w/o ort shown in Table 2 can be roughly considered as representing a more recent baseline, such as [1]. Nonetheless, we will certainly contemplate adding recent baselines like [1] in the revised version.\n \n**R(4.2) Regarding result (1.096 v.s. 1.003) and the significant results:** In Table 1, the causal estimation results of political risk on stock volatility for the baseline model TextCause, over five runs in terms of square root PEHE, are as follows: 1.079556, 1.102212, 1.121453, 1.099713, and 1.074944. For our model, DIVA, the results over five runs are: 1.000645, 1.00681, 1.000243, 1.007245, and 1.002079. The one-tailed t-test between the two samples is 2.40e-06, and the two-tailed t-test is 4.7908e-06, showing DIVA is significant superiority over TextCase. We will complement the results of the significance test in our revised version. \n\n**R(5) Regarding the more recent related works:** We will certainly mention and cite [2-4] in Related work in our updated version of the paper and include a brief discussion.\n\n\n[5] Pearl, Judea. “On a Class of Bias-Amplifying Variables that Endanger Effect Estimates.” ArXiv abs/1203.3503 (2010): n. Pag.\n\n[6] Wood-Doughty, Zach et al. “Generating Synthetic Text Data to Evaluate Causal Inference Methods.” ArXiv abs/2102.05638 (2021): n. Pag.\n\n[7] Zhang, Weijia et al. “Treatment effect estimation with disentangled latent factors.” AAAI (2020).\n\n[8] Shi, Claudia et al. “Adapting Neural Networks for the Estimation of Treatment Effects.” NeurIPS (2019).\n\n[9] Pryzant, Reid et al. “Causal Effects of Linguistic Properties.” NAACL (2021)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Understanding causal relations and estimating the effects are fundamental and challenging problems. With complex data such as text, this problem becomes even more challenging. \n2. The introduced model outperforms all baselines, appearing effective. \n3. The constructed datasets might help benchmark the task. ", "reasons_to_reject": "1. The studied problem of including non-confounding covariates lacks proper motivation. Using the example in Figure 1, why not include expected revenue will influence the effect of political risk on stock movement? Effect of expected revenue on stock movement is independent of the effect of political risk on stock movement. If you combine the variables of policy shifts and expected revenue, this will actually create a new confounder. It then becomes the same problem as before. \n2. The proposed model is bit off to the studied problem. The problem emphasizes all non-confounding covariates, while the proposed solution targets covariates directly related to treatment- and outcome-related covariates. \n3. The introduced orthogonal module is very similar to prior works such as [1], and the representation learning with VAE and representation balancing between different groups using MMD are also pretty standard. The technical contribution is limited. \n4. Baselines are quite old, dating back to 2021. While showing statistical significance, some results of the proposed method are quite close to baselines, e.g., 1.096 vs. 1.003. \n5. Related work should include more recent works using causal inference with text data, e.g., [2-4].\n\n\n[1] Sheth, P., Guo, R., Ding, K., Cheng, L., Candan, K. S., & Liu, H. (2022, September). Causal disentanglement with network information for debiased recommendations. In International Conference on Similarity Search and Applications (pp. 265-273). Cham: Springer International Publishing.\n[2] Feder, A., Keith, K. A., Manzoor, E., Pryzant, R., Sridhar, D., Wood-Doughty, Z., ... & Yang, D. (2022). Causal inference in natural language processing: Estimation, prediction, interpretation and beyond. Transactions of the Association for Computational Linguistics, 10, 1138-1158.\n[3] Cheng, L., Guo, R., & Liu, H. (2022, February). Estimating causal effects of multi-aspect online reviews with multi-modal proxies. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining (pp. 103-112).\n[4] Weld, G., West, P., Glenski, M., Arbour, D., Rossi, R. A., & Althoff, T. (2022, May). Adjusting for confounders with text: Challenges and an empirical evaluation framework for causal inference. In Proceedings of the International AAAI Conference on Web and Social Media (Vol. 16, pp. 1109-1120).", "questions_to_authors": "NA", "paper_topic_and_main_contributions": "This paper studies causal effect estimation with text data, focusing on addressing the selection bias. It introduces an orthogonal module to disentangle covariates into treatment-related, outcome-related, and confounders. The disentanglement aims to ensure that covariates only contribute to the corresponding parts. \n\nThe main contributions include the design of the orthogonal module and the new dataset of 115880 transcripts.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Summary: This paper investigates how to estimate causal effects using text data while addressing the issue of selection bias. It proposes a method that disentangles covariates into treatment-related, outcome-related, and confounder categories, ensuring that covariates only impact their relevant parts. Different from most existing causal inference methods which only adjust for confounders, this work disentangles different factors based on a VAE framework. This paper also introduces a new dataset consisting of 115,880 transcripts. The study includes experiments with two treatment factors in various contexts, and the proposed model outperforms other benchmarks. 

This paper studied a fundamental and challenging research problem. The paper is generally well-written and easy to follow. The proposed method is evaluated on different benchmarks with promising results. Most of the reviewers agree that this is a solid contribution but the excitement is relatively limited.
Decision: Accept-Findings