SlowGuess/NAACL_2022_OCR huggingface dataset:
Adel Daoud<sup>1,3,4</sup> Connor T. Jerzak<sup>1,3</sup> Richard Johansson<sup>2,4</sup>
$^{1}$ Linköping University,  $^{2}$ University of Gothenburg,
$^{3}$ Harvard University,  $^{4}$ Chalmers University of Technology
{adel.daoud, connor. jerzak}@liu.se, richard.johansson@gu.se
Causal inference methods that control for text-based confounders are becoming increasingly important in the social sciences and other disciplines where text is readily available. However, these methods rely on a critical assumption that there is no treatment leakage: that is, the text only contains information about the confounder and no information about treatment assignment. When this assumption does not hold, methods that control for text to adjust for confounders face the problem of posttreatment (collider) bias. However, the assumption that there is no treatment leakage may be unrealistic in real-world situations involving text, as human language is rich and flexible. Language appearing in a public policy document or health records may refer to the future and the past simultaneously, and thereby reveal information about the treatment assignment.
In this article, we define the treatment-leakage problem, and discuss the identification as well as the estimation challenges it raises. Second, we delineate the conditions under which leakage can be addressed by removing the treatment-related signal from the text in a preprocessing step we define as text distillation. Lastly, using simulation, we show how treatment leakage introduces a bias in estimates of the average treatment effect (ATE) and how text distillation can mitigate this bias.
In observational settings, scholars need to collect information about potential confounders in order to estimate the causal effect  $(\tau)$  of a treatment on an outcome (Daoud and Dubhashi, 2020). If we observed the set of confounders directly, we could condition on those quantities to recover unbiased causal effects. Yet, because some confounders  $U$  are difficult to measure directly, scholars are turning to alternative data sources, such as medical records, policy documents, or social media posts,
to indirectly measure (proxy) confounders (Kino et al., 2021). Recent methodological frameworks supply ways of integrating high-dimensional text data into causal estimation (Mozer et al., 2020; Roberts et al., 2020; Feder et al., 2021).
However, prior literature has primarily assumed that documents only contain information about the confounder, but not about the treatment—something we term the no-treatment-leakage assumption. Here, "contain information" means that the text is caused by the treatment (or the confounder) directly or indirectly. When treatment leakage occurs after treatment assignment, its bias is equivalent to a post-treatment bias (Pearl, 2015).
Treatment leakage leads to an identification challenge. The challenge is that  $W$  is both necessary for adjusting (as it is a proxy) yet it is also a post-treatment variable. Without treatment leakage,  $W$  would not be a post-treatment variable, as it does not harbour information about the treatment assignment. But because of leakage, scholars would have to accept bias arising from either adjusting on a post-treatment variable (arising from the part of  $W$  influenced by the treatment) or bias arising from not adjusting for unobserved confounding. Although several methodological studies develop and adapt causal-inference methods for text data (Keith et al., 2020), almost no studies examine the biasing influence of treatment leakage and how to counter this bias.
Our work investigates the treatment-leakage challenge. It shows that if  $\mathbf{W}$  is the only available text representing  $U$  and there exists a distillation method,  $f$ , that has the ability to transform (e.g. partition)  $\mathbf{W}$  into its post-treatment  $\mathbf{W}_T$  and proxy textual-components  $\mathbf{W}_U$ , then adjusting on  $\mathbf{W}_U$  is the best one can do in identifying  $\tau$ . As  $\mathbf{W}_U$  is not post-treatment, we can adjust for it to reduce the bias when estimating  $\tau$ . These  $f$  functions can represent a human annotator, identifying and removing parts of text (e.g., words, sentences) that
Human Language Technologies, pages 5638 - 5645
belong to  $\mathbf{W}_T$  and curating  $\mathbf{W}_U$ ; or, under additional assumptions,  $f$  can be based on supervised or unsupervised machine learning models that transform the text or its representation (Åkerström et al., 2019; Feder et al., 2021).
In this paper, we define key assumptions and demonstrate the mechanics of text distillation in a simulated experiment. Using a language model, we generate synthetic documents  $\mathbf{W}$  so that they contain information about the treatment assignment,  $T$ , and the unobserved confounding,  $U$ , imprinted paragraph by paragraph. Because we control which paragraph is affected by  $T$  (injecting post-treatment bias) or by  $U$  (infusing knowledge about the confounder), we have an oracle distillation function,  $f$ , that mimics human coding. This oracle method perfectly distills  $\mathbf{W}$ , and supplies  $\mathbf{W}_U$ . Then, when using  $\mathbf{W}_U$  in our causal model, we reduce bias of  $\hat{\tau}$  markedly. Although our oracle is idealized, it deepens intuition, and in future work, we will investigate the conditions under which automated methods can be applied to obtain  $f$ .
By conceptualizing the problem of treatment leakage in text data and investigating its impact, scholars developing causal methods can be better positioned to tailor their frameworks to reduce bias; domain scholars can better calibrate their data collection procedure to account for this leakage.
2 Treatment Leakage in Text Data
While the literature on dealing with confounding in observational studies is established (Rubin, 1974), recent advances have been made in the analysis of text-based causal inference. Indeed, text  $\mathbf{W}$  is widely available in the health and social sciences (Gentzkow et al., 2019; Kino et al., 2021), and can be used to proxy for some confounders,  $U$ , that would otherwise remain unobserved (Keith et al., 2020). If the text only contains information about  $U$  and no other factors, then  $\mathbf{W}$  is a faithful representation of  $U$  and we denote it as  $\mathbf{W}_U$ . However, text, by its nature as a medium of creativity, rarely has fixed boundaries, and can contain information not only about confounders, but also leak information about the treatment assignment and its effects.
The future- and backward-looking nature of text can exacerbate treatment leakage. Documents that often contain backward looking temporally (e.g. in much of journalism) or has an unknown production date, will like contain information about the treatment and its effects. Using these docu
ments directly for causal inference would inject post-treatment bias. Conversely, documents that reference the future (e.g., many public-policy documents in the economy and polity) may also lead to unfavorable RMSE if they predict the future well (see §2.1.2). As a result, a substantial amount of real-world text containing rich information about confounding factors might be affected by that language can reference the future, post-treatment state.
2.1 Characterizing Treatment Leakage
We define treatment leakage as when the text,  $W$ , is affected by treatment status,  $T$ : that is,  $W$  is conditionally dependent on  $T$  given  $U$ .
$$
\text {T r e a t m e n t} \quad W \not \perp T | U
$$
The treatment leakage can take different forms. In the most straightforward case, we can assume that a portion  $W_{T}$  is affected by  $T$  while another portion  $W_{U}$  is affected by  $U$ . However, in the general case it may be difficult to partition the document into treatment- and confounder-related passages, and we should see  $T$  and  $U$  as latent factors controlling the data-generating process. For instance,  $T$  may affect the overall tone or sentiment of a document.
We can quantify the degree of treatment leakage in different ways. If the text can be partitioned into treatment- and confounder-related passages  $\mathbf{W}_T$  and  $\mathbf{W}_U$  as described above, we can consider the fraction  $\frac{|W_T|}{|W|}$  to be a measure of the degree of treatment leakage; this also assumes that each partition carries strength equal to the number of its elements (e.g., words) and each element has the same strengths. In the general case, we may turn to information-theoretical quantities, for instance the conditional mutual information between  $\mathbf{W}$  and  $T$  given  $U$ .
In the following, we discuss a number of situations in which treatment leakage can occur.
2.1.1 Case 1: Text is Post-treatment
In one form of this phenomenon, there is a causal relationship between the treatment status  $T$  and the text. Figure 1, panel  $a_{*}$ , shows a directed acyclic graph (DAG) representing this scenario where the text affected by the treatment status. This sort of treatment leakage induces post-treatment bias: when the text is affected by the treatment, conditioning on the text (which is a collider) opens the path from  $T$  to  $Y$  through  $W$  and  $U$ , will in general yield biased estimates (in the notation of Pearl (2015),  $(Y \not\subset T|W)_{G_T}$ ).
Identification assumptions may also be hard to maintain, with the treated/control units having distinct text features (e.g. if all treated units have associated texts referring to the treatment). This lack of overlap would violate the identification assumptions of causal estimators such as Inverse Propensity Score Weighting (IPW) (Heinrich et al., 2010), and could lead to extreme estimated probabilities, something we see empirically in Figure 3.
Figure 1: A causal model consisting of observed variables (shaded): confounders  $(X)$ , treatment  $(T)$ , outcome  $(Y)$ , document  $(W)$ , and unobserved variables (unshaded): confounder  $(U)$  and residual factors  $(R)$ . The red-colored edge in  $a$ . represents the treatment leakage. In  $b$ , A distillation function  $f$  has removed the treatment information in the text, leaving only information from the confounder. A perfect intervention of  $f$  is equivalent with deleting the red arrow; a less than perfect intervention reduces at least its dependence.
2.1.2 Other Cases
Figure 1 shows a case when text is post-treatment, but in other cases the precise DAG structure may not be known. For example, text may represent a mediator if the document includes post-treatment information and also affected the outcome (if, for example, the text is congressional speech and the outcome is a roll call vote). If the proxy text is pretreatment and directly affects the treatment, conditioning on the treatment-related portion of the text could increase the variance of estimation, leading to unfavorable RMSE (Myers et al., 2011).
3 Text Distillation as Preprocessing
Text distillation is a form of text preprocessing. It has to target any text (e.g., tone, words, sentences) that belongs to  $W_{T}$ , and remove it from  $W$ . Thus, distillation ensures that the treatment signal is negated. As Figure 1, panel  $b$  shows, if distillation is perfectly successful, it results in cutting the red arrow (from  $T$  to  $W$ ). The arrow is cut, because the distillation function has removed  $W_{T}$  from  $W$ , supplying  $W_{U}$  for causal analysis.
3.1 Assumptions for Valid Distillation
Depending on how the treatment leakage is manifested in  $\pmb{W}$ , we need to introduce assumptions
to make distillation feasible. As already discussed in §2.1, in some cases we may assume that  $W$  contains treatment-related passages  $W_{T}$  and confounder-related passages  $W_{U}$ . We may further assume that the text is separable: that is,  $W_{T}$  and  $W_{U}$  do not overlap.
$$
\text {S e p a r a b i l i t y A s s u m p t i o n :} \quad \boldsymbol {W} _ {U} \cap \boldsymbol {W} _ {T} = \varnothing
$$
Assuming separability, a perfect distillator will produce  $\mathbf{W}^{*} = f(\mathbf{W})$  that is equivalent to the confounder-related portion of the text,  $\mathbf{W}_{U}$ . Perfect distillation means that the distillator  $f$  identified text that contains the same information about  $U$  as  $\mathbf{W}_{U}$  has. Thus, if  $\mathbf{W}_{U}$  is a valid adjustment set, then  $\mathbf{W}^{*}$  is that as well. The separability assumption is appealing because it implies that researchers only need to find a valid partition of the text (and do not need to consider all possible text transformations).
This separability assumption is particularly plausible for text data, which by its nature consists of a sequence of linguistic signifiers which can be decomposed into smaller units (e.g. paragraphs).
While plausible for many circumstances, in some cases separability may not hold, as when the entire tone of the text is affected by the treatment. In this more complicated setting, we need a more general assumption, that the transformed text,  $\mathbf{W}^*$ , is conditionally independent of  $T$  given  $U$ . That is, the conditional mutual information between  $\mathbf{W}^*$  and  $T$  given  $U$  is zero, while information about  $U$  in  $\mathbf{W}^*$  is maintained. Despite the benefits of this more general framing, because  $U$  is unobserved, it may be difficult for investigators to assess whether the assumption is satisfied or whether ethically problematic information has been included in the  $f$  function (e.g., race; Menon and Williamson (2018)). Unlike numerical data, as text data is readable, scholars can examine and validate whether  $\mathbf{W}^*$  still contains information about  $T$ .
We use simulation to illustrate the dynamics of text distillation and build on the framework for evaluating text-based causal inference methods introduced by Wood-Doughty et al. (2021). We generate numerical covariates from the model in Figure 1; the general procedure is described in  $\S A$ , with implementation details in  $\S B$ . Parameters are selected so that ATE estimates  $\hat{\tau}$  are biased if the estimator does not account for the unobserved confounder  $U$ .
Following Wood-Doughty et al. (2021), we generate documents,  $W$ , by sampling from an English-language GPT-2 model (Radford et al., 2019). In contrast to their approach, text generation is conditioned not only on  $U$  but also on  $T$ . As described in detail in §A, we define paragraph-level topics, where some topics are associated with  $U$ , some with  $T$ , and some with a residual topic related only to other background variables ( $R$  in Figure 1). For a given paragraph topic, we define a number of prompts and a distribution shift that increases the probability of generating topic-related keywords.
As we simulate and record which paragraphs are affected by  $T$  and by  $U$ , our distillator  $f$  has oracle properties. We can then use  $f$  to investigate three idealized distillation scenarios. The first is when a distillator was not applied or the distillator failed to do any distillation  $f(\mathbf{W}) = \mathbf{W}$ . It outputs the same corpus. The second is when it perfectly distills  $\mathbf{W}$ , excluding all paragraphs affected by  $T$ . That is, apply  $f(\mathbf{W}) = \mathbf{W}^*$  such that  $\mathbf{W}^* = \mathbf{W}_U$ . The third scenario is when  $f$  was overly aggressive and accidentally removed not only paragraphs related to  $T$  but also those related to  $U$ , resulting in  $\mathbf{W}^{**}$ . This corpus violates the proxy-faithfulness assumption that  $\mathbf{W}^{**}$  fully measures  $U$ . Then, we use the three corpora, one at a time, for causal inference. We use an Inverse Propensity Weighting (IPW) estimator, fully described in Appendix C.
Based on the setting described in §4, our analysis produces six estimates, three based on distillation and three based on facts about the data-generating process. Figure 2 shows all estimates.
Figure 2: Estimates under different distillation regimes.
The first estimate,  $\hat{\tau}_{1} = 5.5$ , is the baseline where all information is known to the outcome model, including  $U$ . Because this linear model adjusting for  $U$  and  $\mathbf{X}$  is equivalent to the data-
generating model, and the estimated effect would be equal to the true value of 5 without sampling noise. The bootstrapped  $95\%$  confidence interval (CI) is 3.4 to 7.6. The second estimate,  $\hat{\tau}_2 = -2.3$ , is obtained when  $U$  is omitted from the model to induce omitted variable bias (CI: -4.2, -0.1).
The third estimate,  $\hat{\tau}_3$ , uses IPW to estimate the ATE (see §C). Here, we use the non-disitilled documents,  $W$ , to estimate propensities. As Figure 2 shows, in the absence of distillation, the bias increases compared to conditioning on  $X$  alone, producing  $\hat{\tau}_3 = -7.0$  (CI: -9.4, -4.6). The fourth estimate,  $\hat{\tau}_4$ , applies overly aggressive distillation. This approach gives a result similar to the unadjusted estimate:  $\hat{\tau}_4 = -2.9$  (CI: -5.1, -0.6).
The fifth estimate,  $\hat{\tau}_{5}$ , applies oracle distillation by removing the paragraphs we know were affected by  $T$ . Using  $\boldsymbol{W}^{*}$ , the bias is reduced substantially, yielding an estimate  $\hat{\tau}_{5} = 3.5$  (CI: 1.2, 5.8). As the CI of this  $\hat{\tau}$  includes the true  $\tau = 5$ , we conclude that distillation successfully recovers  $\tau$ . However, we note that this recovery is not perfect and will be affected by sampling and modeling parameters.
The sixth estimate,  $\hat{\tau}_{6}$ , demonstrates the impact of model selection for the propensity estimator. Using the true (simulated) propensity, the IPW estimate is  $\hat{\tau}_{6} = 4.9$  (CI: 2.2, 7.6). This result shows that further gains could be made by careful model selection (Chernozhukov et al., 2018).
Figure 3 shows distributions of propensity values for  $\hat{\tau}_3$ ,  $\hat{\tau}_5$ , and  $\hat{\tau}_6$ . Without distillation (red), the estimated propensities cluster near 0 and 1.  $T$  is predicted almost perfectly, as mentioned in §2.1.1, causing the IPW estimate to be similar to the unweighted one. Conversely, with distillation, the predicted probabilities are now similar to the data-generating propensities, and thereby, the resulting causal estimate is improved.
Figure 3: Estimated and true assignment probabilities.
This paper shows the critical role of the no-treatment-leakage assumption when using text for
causal inference. While text is becoming an established data source, it may harbour valuable information about a confounder but also contaminating information about post-treatment effects. This issue has seen little discussion in text-based causal inference literature (Mozer et al., 2020; Roberts et al., 2020; Feder et al., 2021; Daoud and Dubhashi, 2020), but has the potential to severely bias causal estimates, potentially leading to false discoveries or invalid policy recommendations in social and health settings (Kino et al., 2021; Daoud et al., 2017; Balgi et al., 2022).
Before discussing the implication of treatment leakage, three limitations should be considered. First, more work is required to show how the no-treatment-leakage assumption operates under different covariance structures (i.e., different data-generating processes). Second, a larger simulation framework is needed to decompose estimator bias and variance. Third, all results are based on simulated data, and more research is needed to generalize our insights to real data. Although simulate data are idealized, they provide a benefit by allowing us to analyze the mechanics of treatment leakage and text distillation in a controlled environment. Based on our simulated data, our analysis shows that when the no-treatment-leakage assumption is violated, effect estimates will be severely biased. In the presence of treatment leakage, scholars may be better off abstaining from using a non-distilled text to adjust for confounding. Although, in theory, the best solution is to use a text distillation that removes all treatment leakage, in practice, using distillation can be difficult to achieve.
Therefore, one critical extension of our work is to develop methods that estimates the amount of treatment leakage in text. This estimate will enable applied researchers to make an informed decision about whether to adjusting for text-based confounding or abstain from it when leakage is high, and when text distillation is not an option.
A second extensions is to develop a generalized framework that accounts for when the adjusted text represents multiple nodes in a DAG (in combination with the confounding and the treatment or without them). While our article focuses on treatment leakage, there are other types of leakage when a single document is a function of combinations of DAG nodes such as the outcome, confounder, treatment, mediator, or instrument. Thus, a generalization of the no-treatment-leakage assumption is the
no-node-leakage assumption. Such methods will benefit from insights established in the literature on causal inference with proxies (Pena, 2020; VanderWeele, 2019; Miao et al., 2018; Rissanen and Marttinen, 2021). A third extension is to develop a variety of text distillation methods, suitable for different application settings. Researchers need alternative frameworks when human partitioning of text is not possible to achieve manually, because of corpus size or language complexities. Automatic distillation could be attempted with additional assumptions, perhaps building from the literature on removing sensitive information in text representations (Bolukbasi et al., 2016; Ravfogel et al., 2020).
Richard Johansson was supported by the projects Interpreting and Grounding Pre-trained Representations for NLP and Representation Learning for Conversational AI, both funded by Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. Adel Daoud would like to acknowledge a grant from The Royal Swedish Academy of Letters, History and Antiquities.
Joakim Åkerström, Adel Daoud, and Richard Johansson. 2019. Natural language processing in policy evaluation: Extracting policy conditions from IMF loan agreements. In Proceedings of the 22nd Nordic Conference on Computational Linguistics, pages 316-320, Turku, Finland. Linköping University Electronic Press.
Sourabh Balgi, Jose M. Pena, and Adel Daoud. 2022. Personalized Public Policy Analysis in Social Sciences using Causal-Graphical Normalizing Flows. Association for the Advancement of Artificial Intelligence: AI for Social Impact track. ArXiv: 2202.03281.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc.
Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. 2018. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1-C68.
Adel Daoud and Devdatt Dubhashi. 2020. Statistical modeling: The three cultures. arXiv:2012.04570.