Discovering influential text using convolutional neural networks (2024)

Abstract: Experimental methods for estimating the impacts of text on human evaluation have been widely used in the social sciences. However, researchers in experimental settings are usually limited to testing a small number of pre-specified text treatments. While efforts to mine unstructured texts for features that causally affect outcomes have been ongoing in recent years, these models have primarily focused on the topics or specific words of text, which may not always be the mechanism of the effect. We connect these efforts with NLP interpretability techniques and present a method for flexibly discovering clusters of similar text phrases that are predictive of human reactions to texts using convolutional neural networks. When used in an experimental setting, this method can identify text treatments and their effects under certain assumptions. We apply the method to two datasets. The first enables direct validation of the model’s ability to detect phrases known to cause the outcome. The second demonstrates its ability to flexibly discover text treatments with varying textual structures. In both cases, the model learns a greater variety of text treatments compared to benchmark methods, and these text features quantitatively meet or exceed the ability of benchmark methods to predict the outcome.

heading: 1 Introduction
text: Text impacts outcomes and decisions in many domains. For example, researchers have investigated the effects of campaign messaging on voting (Arceneaux and Nickerson, 2010), post content on censorship (King et al., 2014), clinical notes on diagnoses and treatment (Sheikhalishahi et al., 2019), and written profiles on citizenship decisions (Hainmueller and
*These authors contributed equally to this work.
Hangartner, 2013). Most experimental methods for estimating the effects of text on human evaluation randomly assign some subjects to a small number of treatment texts which are chosen ex-ante by the researcher. These treatments are often chosen subjectively, introducing the possibility that they may be ineffective or lack external validity. Recent literature in computational social science has sought to instead discover treatments that have an effect on an outcome of interest from unstructured texts (Fong and Grimmer, 2016; Pryzant et al., 2018). In this work, we bridge these efforts towards making causal inferences using text treatments with the domain of interpretable machine learning (Jacovi et al., 2018; Alvarez Melis and Jaakkola, 2018). Causal inference approaches with text as treatment aim to identify low-dimensional representations of text features that causally impact an outcome. We introduce an innovative application of contextualized word embeddings, convolutional neural networks (CNNs), and interpretability methods to this setting to both detect and interpret these latent text representations. Unlike previous approaches to identifying text treatments, these learned representations can vary in both length and structure and are not constrained to represent document-level sets of topics or sets of particular words. We apply our approach to two datasets: social media posts on Weibo where the outcome is post censorship, and complaints submitted to the Consumer Financial Protection Bureau with the outcome of whether a complainant received a timely response. In both cases, our method extracts qualitatively different treatments and meets or exceeds quantitative performance metrics compared to benchmark methods.
While this approach and those that it builds upon seek to simulate experiments that tar-
12002
get causal effects of specific latent text features, effects can only be identified directly under strong assumptions. Alternatively, this model may be used as a tool for researchers to discover text treatments to test in subsequent confirmatory analyses, as an alternative to subjectively posing treatments.

heading: 2 Related work
text: While much of the related social science work has focused on learning latent “features” of texts and using those as a treatment, most NLP work has focused on improving the interpretability of black-box predictive models. This paper bridges the two by using interpretability methods to flexibly discover latent treatments in text. Computational social science and causal inference Prior work has generated methods to both discover treatments and estimate their effects simultaneously (Fong and Grimmer, 2016; Pryzant et al., 2018; Egami et al., 2018; Fong and Grimmer, 2021; Feder et al., 2022). These models have typically focused on estimating either topics or individual words as treatments. Fong and Grimmer (2016) apply a supervised Indian buffet process to both discover features (topics) and estimate their effect on an outcome in an RCT setting. Pryzant et al. (2018) use n-gram features instead of topics and a neural architecture with a method for extracting feature importance from the weights of the network. Our model extends this work by allowing groups of generally similar phrases – instead of topics or unique words – to be identified as treatments. We expect our approach to work particularly well in instances where the outcome may be caused by flexibly expressed concepts (e.g. a sentiment that can be conveyed with interchangeable synonyms) instead of particular words or the full topical content of the document. Interpretable NLP Many methods have been proposed to interpret and explain NLP models, as well as meta-evaluations of those methods (Lei et al., 2016; Alvarez Melis and Jaakkola, 2018; Rajagopal et al., 2021; Alangari et al., 2023; Crothers et al., 2023; Lyu et al., 2023). Most of these methods focus on explaining and interpreting predictions at
the level of individual samples. In contrast, our method is designed to learn and interpret broader patterns that occur at the corpus level. In this respect, Rajagopal et al. (2021), who require their model to explain predictions using “global” concepts, and Jacovi et al. (2018), who interpret the latent features learned by CNNs specifically, are closest to our work. Individual tokens are not humaninterpretable or individually persuasive, so like Alvarez Melis and Jaakkola (2018) we require the network to have an interpretable final layer after a representation learning component. Rather than trying to understand why the network made the prediction it did, we seek representations of influential corpus-wide features whose effects scientists can test in follow-up experiments. For example, if the model identifies the presence of calendar dates as a globally influential feature for determining timely response to a complaint, researchers may design two texts differing only by the inclusion of dates and compare their effects in a controlled experiment.
Other existing NLP techniques could be adapted to this approach. For example, the differences between persuasive and unpersuasive texts (Zhong et al., 2022) could be used to identify persuasive concepts. While any method capable of learning corpus-wide and low-dimensional representations of influential text features could be utilized for identifying text treatments, a key challenge is the ability to capture complex feature representations while remaining human interpretable. This requires sophistication in representation learning but clarity in understanding the learned text treatments whose effects are being estimated. In experiments, our proposed model effectively achieves this balance.

heading: 3 Extracting influential text from latent representations
text: Our goal is to extract clusters of phrases that represent latent, generalizable treatments that affect a particular outcome. To do this, we imagine that N texts (Ti) are randomly assigned to a process through which they are mapped to an outcome (Yi). Let i also index the individual evaluating text i. We seek to identify and estimate the effect
of an m-dimensional latent representation of those texts (Zi) which summarizes clusters of phrases or concepts that are likely to influence the outcome in repeated experiments. We refer to Zi as “text treatments” for text i. For example, each element of Zi could represent the presence or absence of a certain phrase or grammatical structure, with Zi ∈ {0, 1}m. Zi could also contain real-valued elements indicating continuous text features like similarity to a certain vocabulary or concept alignment.
To simulate a sequential experimental setup, we follow Egami et al. (2018) in splitting our sample into training and test sets. We first train our model, using cross-validation within the training set for tuning and model selection. We then use the test data set to interpret the latent text treatments discovered and estimate their effects on the outcome under additional assumptions. Our main contribution concerns this first stage: the novel usage of a CNN model to discover a mapping between text data and text treatments (Zi).
Fong and Grimmer (2016, 2021) outline conditions under which this general process identifies causal effects of the text treatments on the outcome when treatments are binary. They assume that: 1) an individual’s treatment depends only on their assigned text, 2) any nontextual features or latent text features not captured by the model which influence the evaluator’s response (Yi) are independent from the model’s captured latent features, 3) there is a nonzero probability of each evaluator receiving any of the possible text treatments (Zi), given unmeasured text features1, 4) texts are randomly assigned and 5) latent treatments are not perfectly collinear. If these assumptions hold, our model can identify treatment effects of the discovered latent features. These may be estimated using linear regression under the additional assumption that the m text treatments do not interact with each other, in addition to linear modeling assumptions in the case of continuous treatment variables.2 How-
1For real-valued treatments, this assumption should be modified to require that the probability density function of the treatment vector is nonzero.
2Fong and Grimmer (2016) consider the Average Marginal Component Specific Effect (AMCE), which captures the effect of changing one text treatment while averaging over values of all others. For continuous treatments, the process would identify a similar effect
ever, since it is difficult to assess whether these assumptions hold – particularly assumption 2 – we recommend that when possible, practitioners use our method to suggest treatments for study in a controlled experimental setting.

heading: 4 Methodology
text: We propose harnessing the structure of CNNs to identify influential text treatments. Filters in convolutional layers project text phrases onto lower-dimensional representation spaces, and these representations are then max-pooled across all phrases within each sample to predict an outcome (Figure 1). By training the model to produce predictive max-pooled representations, filters are incentivized to detect influential n-gram patterns (Jacovi et al., 2018). These patterns could correspond to specific keywords or clusters of keywords with similar vocabulary, grammatical structure, or tone, for example. Researchers can then test how the presence of these patterns in texts affects the outcome.

heading: 4.1 Contextual encoder
text: We use pre-trained BERT models (Devlin et al., 2019) to tokenize our input text samples (Ti) and to obtain context-dependent embeddings of tokens. We denote these embeddings by ei,j ∈ RD, where i indexes each text sample, j indexes tokens (ui,j), and D represents the embedding dimension. With accessibility for social scientists in mind, we work with reduced-size models (Jiao et al., 2020), and do not perform fine-tuning. Researchers with fewer constraints on their computational budgets may find improved model performance from using larger or more complex models and/or fine-tuning these models on their outcome. Any model providing text embeddings could be substituted for BERT. However, we do recommend using models that encode context between tokens. We perform the embedding step just before creating a train-test split, but researchers who choose to fine-tune their embedding models should reverse these steps to fine-tune and train only on the training set.
capturing the marginal effect of incrementally increasing a text treatment.

heading: 4.2 Model architecture
text: Sequences of input text embeddings {ei,j}j are passed to a one dimensional convolutional layer C, or a series of M such layers in parallel (Cl), each with flexible kernel size Kl and F filters. The number of parallel convolutional layers is determined by the number of unique kernel sizes to be considered. A higher number of filters F corresponds to learning more latent text features. In our implementation all convolutional layers learn the same number of filters. The kernel size K determines the size of the filter window, or the length of phrases considered by each convolutional layer. Including filters of multiple kernel sizes allows the model to capture patterns of varying lengths. A filter f in a layer C with K = 5 tests the extent to which the representation learned by f is present in five-token phrases of the input text. For each phrase pi,1, . . . , pi,P ∈ RK×D in text i with P = U − K + 1 and filter f , the convolutional operation produces a new feature ai,f = g(Wf · pi + b), where Wf and b are the learned weights and bias respectively for filter f , and g is the sigmoid activation function. We refer to these features as “filter activations,” ai,f ∈ RP . These are summarized per text sample by max pooling layers, which keep only the highest activation across a text’s phrases per filter. The max-pooled activations apooledi,f ∈ R for each filter are then concatenated across the parallel convolutional layers. The concatenated max-pooled activations are then passed through a final fully connected layer. The activations from this final layer, Ŷi, correspond to the model predictions.

heading: 4.3 Training
text: The model is trained using Adam optimizer (Kingma and Ba, 2017) and the following loss function when M = 1:
L = − 1 N
∑
i
( Yi log(Ŷi) + (1− Yi) log(1− Ŷi) ) +
λconvker ∑
k,d,f
(W convk,d,f ) 2 + λconvact max(R) + λoutker
∑
f
|W outf |
where R is an F × F matrix with
Rf,g = { max(cor(ãf , ãg), 0) for f ̸= g 0 for f = g
and ãf ∈ RN ·P represents the vector of filter activations across P phrases of all N samples for filter f . The first term is the binary
cross-entropy loss with respect to the model predictions across samples i. This global loss was chosen because of the binary outcome in both applications presented here, but could easily be substituted for RMSE or another loss more suitable for continuous outcomes. The second term corresponds to a L2 regularization penalty applied to the weights of the convolutional layers, W conv ∈ RK×D×F . The third term represents an activity regularization which penalizes the maximum nonnegative correlation between two filter activations. This penalizes models that learn redundant filters (as measured by high correlation in their activations) to encourage convolutional layers to identify a larger number of distinct text features (Appendix A: Figure 4). For models with M > 1, terms two and three are repeated in the sum for each convolutional layer. The fourth term corresponds to a L1 regularization penalty applied to the weights of the final fully connected layer, W out ∈ RF ·M . The strength of each penalty is controlled by λconvker , λconvact , and λoutker.
These penalty strengths and other hyperparameters are determined according to a fivefold cross validation procedure using the training set. Because the motivation of these models is primarily interpretation of learned features, rather than prediction performance,
model selection is more subjective than simply choosing the highest accuracy parameter settings. We selected models based on a combination of accuracy, degree of correlation between filter activations (i.e. feature redundancy), and the number of “useful”3 filters learned. Parameter settings for the models selected in our applications are reported in the appendix.
The final selected model is then re-trained using the entire training set with a randomly sampled 20% serving as the validation set, and is assessed using the unseen test set.

heading: 4.4 Identifying and testing influential text features
text: To interpret the model’s learned latent representations and discover text treatments (Zi) for each text, we utilize three model components:
1. The filter activations of each text sample’s phrases for each filter f (ai,f );
2. The output layer weights (W out ∈ RF ·M );
3. The input text samples (Ti).
The filter activations represent how strongly each phrase corresponds with the representation learned by each filter. To facilitate interpretation and to assign manual labels to each filter, we examine the phrases that maximally activate each filter. The final layer weights determine how each text representation contributes to the ultimate outcome prediction. Finally, the original input text samples provide context for the phrases that activate highly on each filter. This last component is most subjective to interpretation. Because input text embeddings are contextdependent, each phrases’ embeddings contain more information than just the tokens that make up the phrase, which lack the context of the whole sample. However, due to the difficulty of interpreting text embedding dimensions, the context that human readers assign to phrases when reading an entire sample can not be confirmed to align with the encoded context.
3Some learned filter weights produce near-identical activations across samples. By not meaningfully distinguishing predictions between texts they are not useful for interpretation, so we omit filters with activation ranges less than a threshold t = 0.05 wide.
The objective of this interpretation process depends on whether the researcher wishes to directly estimate the effects of the identified latent features in the test set under assumptions described in Section 3, or if they wish to discover concrete text features to test in a follow up experiment. In the first scenario, the max-pooled filter activations (apooledi,f ) may be considered directly as the sample-level latent text treatments (Zi). Researchers could also choose to binarize these features, for example by defining Zi,f = 1[apooledi,f > ā pooled f ] where āpooledf is the median of (a pooled i,f ). This avoids the more stringent modeling assumptions needed for estimating effects of continuous treatments, though it may complicate interpretation. In either case, this interpretation process provides the researcher an understanding for what the latent text treatments represent and therefore the effects that they are estimating. In the second scenario, this process can guide the researcher’s process for obtaining concrete text treatments. Here, a second set of text treatments, Z̃i, are established which are not latent in the same sense as Zi, because researchers control their definition. For example, Z̃i could be defined as an indicator of whether the manual labels assigned to a filter appear in experimental texts, or as measures of different tones or grammatical structure identified by filters.

heading: 4.5 Evaluation methods
text: We evaluate our models by comparing them to two benchmark methods. The first is the methodology proposed in Fong and Grimmer (2016), which uses a topic modeling approach to discover and interpret latent text treatments. We abbreviate this method as F&G. The second is regularized logistic regression on the vocabulary of n-grams in the corpus, which we abbreviate as RLR. Methods are compared quantitatively by assessing the adjusted R-squared of linear models fit using the text treatments identified by each method to predict the outcome variable, and by assessing the mean-squared error of these linear models on out-of-sample texts. We make these comparisons robust to sampling variability by calculating the metrics across 1000 bootstrap samples of the data, fixing the trained mod-
i,f in the final model layer (second
column). The third column shows the average estimated treatment effect of apooledi,f across 1000 bootstrap samples of the test set (fixing the model and therefore the filters), with corresponding bootstrap confidence intervals in the fourth column. The fifth column lists filters’ unique top 3 most associated phrases from the test set. The sixth column associates each filter with a commonly reported censored phrase.
els (and therefore the learned latent features). To better understand the stability of the training process for our proposed model and benchmark models, we repeat the process but additionally retrain the models (fixing the tuned parameter settings) in 150 bootstrap samples of the training data. Methods are compared qualitatively by assessing the interpretability and variety of learned text features. In the censorship application, ground-truth information of which phrases led to censorship allow us to compare methods by their ability to recover text treatments which have known causal effects on the outcome. Details of how benchmark methods were implemented and full interpretation results are included in Appendix B.

heading: 5 Experiments
text: In order to sufficiently demonstrate the qualitative and substantive results that are achievable with our method, we focus our experiments on two datasets. The first was selected because of rare access to ground-truth rationales underlying the text-driven outcome. The second was chosen to explore a setting where influential text features likely exhibit complex and varied structures and because of its use in closely related research. We take a depth versus breadth approach here, but future work
should assess this method on a larger range of datasets. While the method can generalize to any dataset where text is thought to cause an outcome, datasets with ground-truth information about this relationship are ideal (though rare) for demonstrating successful identification of causal relationships and effects.

heading: 5.1 Weibo post censorship
text: Dataset and setup For our first application, we use a sample of 28,386 Weibo posts from the Weibo-Cov dataset (Hu et al., 2020). These are social media posts on the topic of COVID and were posted in February 2020 on Weibo.4 To obtain the censorship label for each post, we use the content review API from Baidu.5 The API is a classifier that returns the probability of censorship for each post. The API only returns a probability of 1 when a social media post includes words or phrases that are on Baidu’s blacklist. As the API also returns the flagged keywords and phrases, this enables us to validate whether our model can recover keywords and phrases that led to censorship.
We train our model to predict whether or not a post was flagged by the API to be cen-
4The data set creators anonymized identifiable information in posts to protect user privacy. Data is available from the creators upon request.
5ai.baidu.com/solution/censoring
sored with probability 1. Although this outcome is not determined by direct human decision making, we can view the blacklist as a decision maker that perfectly implements a set of human-defined preferences (which may or may not be generally representative of the broader censorship policy). To tokenize and embed these texts, we use a pre-trained BERT Chinese language model provided by the Joint Laboratory of HIT and iFLYTEK Research, MiniRBT-h288 (Yao et al., 2023).6 This model has an embedding dimension of 288 and 12.3M parameters. The embeddings from the BERT model’s last hidden state are used as the input features to our model architecture. Examples of posts in this data set, their censor probabilities, and their censor words (when applicable) with English translations are shown in Appendix A Table 3. Appendix A Table 4 shows the top 10 censor words across all censor-probability-one samples, their translations, and the proportion of censored samples corresponding to each. Results The trained model obtains an accuracy score of 0.87 on the test set. Across iterations where the model is repeatedly trained on resampled data and reevaluated, the model classification accuracy is relatively stable (with values in [0.84, 0.88] and an av-
6Model is licensed under Apache License 2.0.
erage of 0.86). This performance indicates that the model has learned useful representations of Weibo posts from the time period which are predictive of censorship. Based on parameter tuning results, this model was constructed with two convolutional layers with kernel sizes set to 5 and 7 Chinese characters. We highlight our interpretation of the most relevant representations in Table 1, with interpretation of all representations included in Appendix A Table 7. We find that the two most commonly censored phrases, “Wuhan virus” (23.9% of censored posts) and “national crisis” (4.9% of censored posts) are clearly identified by the model in the first and second model filters. The max-pooled activations for these filters also contribute the most to the model’s final prediction of censorship, as seen in the W outf column of this table. The most highlyactivating phrases for filters 3 and 9 have in common two other known censored phrases, “Provincial party secretary” and “Shincheonji Church,” and the highest activated phrases for filter 10 concentrate exactly around the same phrase, which relates to a fifth known censor phrase “Jiang Chaoliang.” The complete set of representation interpretations demonstrates that there is some redundancy in the keywords learned by filters. Their differences in sentence structure and context could be illuminating in other settings, though in this case it is known that the inclusion of these phrases solely affects the outcome. As a proof-of-concept, we include the effect estimates obtained by regressing the labels on the max-pooled filter activations of the test sample, though this setting does not follow a typical experimental design. Though the magnitude of the estimated effects differ from the output layer weights (in large part because the output layer weights correspond to a sigmoid rather than linear activation), they are in relative agreement about which text treatments are found to be most influential for censorship.7
Model validation We find that this methodology successfully recovers the phrases which
7The regression estimates βf identify causal effects under the assumptions in Section 3. Because of the much more complex way that the final layer weights W outf are learned in the neural network model, we have not proven that they identify causal effects even under the same assumptions.
cause the most posts to be censored. In a setting without oracle knowledge of the censorship rationale, we feel confident that researchers would be able to use this model to determine at least five of the most common censored phrases. In comparison, we find that the topics learned by the F&G method are not clearly aligned with any of the most common censored phrases (Appendix B: Table 12). A subset of the n-grams selected by the logistic regression model correspond to three common censored phrases: “Wuhan virus”, “Jiang Chaoliang” and “Provincial party secretary” (Appendix B: Table 13). Our model outperforms both benchmarks with respect to both metrics reported in Figure 2, which indicates that the features learned by our model explain significantly more variation in the censorship outcome and have much better predictive power compared to the topics and keywords learned by the benchmark methods.

heading: 5.2 Consumer Financial Protection Bureau complaint response
text: Dataset and setup For our second application, we use a dataset from Egami et al. (2018) of 54,816 consumer complaint narratives submitted to the Consumer Financial
Protection Bureau (CFPB) from March of 2015 to February of 2016.8 The outcome variable indicates whether or not the complainant received a timely response from the company filed against. Due to severe imbalance in the outcome variable, we proceed with a subsample of complaints which received a timely response (5136 timely and 1712 non-timely responses) combined with a class-weighted loss function. To tokenize and embed the complaint texts, we use a pre-trained BERT English language model bert-tiny trained by Google Research (Turc et al., 2019; Bhargava et al., 2021).9 This model has an embedding dimension of 128 and 4M parameters.
Results The trained model obtains an accuracy score of 0.76 and an F1 Score of 0.33 on the test set. Across iterations where the model is trained on resampled data and reevaluated, the model classification accuracy is relatively consistent with values in [0.72, 0.79] and an average of 0.75. F1 scores are observed to be more variable with values in [0.12, 0.44] and
8Data is publicly available: https:// www.consumerfinance.gov/data-research/ consumer-complaints/. The CFPB removes personal information from complaints.
9Model is licensed under Apache License 2.0.
an average of 0.31. Given the limited size of the data set used, the class imbalance, and the relative complexity of this learning task, it is unsurprising that this model achieves a lower performance, however, the learned representations still provide meaningful insight into the textual treatments in the dataset.
Table 2 summarizes interpretation of the top 8 representations (according to final layer weight) learned by this model. Interpretation for all filters can be found in Appendix A Table 10. This model has a single convolutional layer with kernel size set to 5 tokens, which was chosen by parameter tuning. We infer that references to credit disputes and banking processes may be positively associated with timely responses, and that references to attempted debt collection, phone calls or voicemails, previous interactions, or payday loans may be negatively associated with timely responses. Beyond these broader topics, we also find that using infinitive verbs in the context of describing disputed actions could increase the likelihood of timely response, while using contractions may have the opposite effect. Table 2 also includes effect estimates from regressing the test set labels against the texts’ corresponding max-pooled filter activations. Again, we believe it is unlikely that the assumptions necessary for causal interpretation of these effects are met, but the estimates could still act as a
useful tool for a researcher exploring possible text treatments for subsequent experiments.
Model evaluation In this application we do not have access to the true reasons that complaints receive or do not receive timely responses, and imagine that a variety of text features could impact this outcome. Both benchmark methods detected that certain financial topics seem to be associated with timely responses (Appendix B: Tables 15, 16). In particular, the results from all models imply that mentions of debt collection have a negative association, while mentions of banking processes and credit issues have a positive association, though the magnitude of estimates vary. Beyond these shared topics, our model is uniquely able to learn grammatical text features that are predictive of the outcome. While all models perform similarly in terms of the out-of-sample predictive power of their identified text features, our model results in a modestly higher R2adj value compared to benchmarks (Figure 3).

heading: 6 Conclusion
text: We present a new method to discover influential text features represented by clusters of phrases of flexible length. Our approach is inspired by and builds upon previous work in computational social science and interpretable NLP, and provides experimenters with a quantitative tool for identifying promising text treatments to test in follow up experiments. When researchers are willing to make stronger identification assumptions discussed in Section 3, text treatments identified by using the model can also be used to estimate causal effects on the test set directly. Our applications demonstrate the ability of our model to learn useful and diverse latent text representations and its capacity to recover known influential text features. Important directions for future work include a human interpretability evaluation to rigorously compare the qualitative aspects of our method to benchmarks. Approaches for evaluating the stability of identified latent text treatments across model runs would also be useful to practitioners performing follow-up controlled experiments.

heading: 7 Limitations
text: Small BERT models used out-of-the-box In this paper, we do not investigate how model performance could be affected by fine-tuning the pre-trained BERT models, or by using larger or more advanced models to obtain higher dimensional word embeddings. Future work investigating how benefits from these changes trade-off with reduced computational efficiency would be relevant to researchers using this method. Computational expense While these results demonstrate that the proposed CNNbased method is able to significantly outperform benchmark methods, these improvements come with a substantially higher computational cost in terms of both time and memory. We observed that the regularized logistic regression method was by far the most efficient option overall, with the F&G method also exhibiting much higher memory-efficiency but less time-efficiency. Future work comparing the computational expense of these methods across a larger range of datasets would be valuable for practitioners. Trade-off between experimental costs and less-interpretable treatments Under the assumptions discussed in Section 3, researchers may estimate causal effects by directly testing the identified latent text treatments. This simplifies the experimental pipeline, but as in Egami et al. (2018) and Fong and Grimmer (2016), comes with the drawback of requiring the researcher to somewhat subjectively interpret the identified latent text treatments that are being tested. Alternatively, researchers may use their interpretations of the discovered latent text features to inspire “manifest” text treatments (ex. specific keywords, sentence structures) to test in confirmatory settings. In this case, the text features being tested would be known and manipulated by the researcher, allowing clearer interpretation of effects and weaker assumptions. This alternative comes with the downside of requiring researchers to run follow-up experiments. Incorporating uncertainty in latent treatments Our paper does not provide guidance for incorporating the model-based uncertainty involved in identifying and estimating
the latent text treatments into causal effect estimates. Designing experimental texts We generally recommend using our model to guide the selection of text treatments for use in followup experiments. Designing experimental texts to isolate treatments of interest is a non-trivial task, and is left to the experimenter. In many cases, it is challenging to imagine altering a specific part of a text without affecting the interpretation of surrounding text that is not directly manipulated. This makes it difficult to establish causality for a specific text feature, rather than for the aggregate differences between a set of texts. This is a known challenge of making causal inferences with text, and relates to the strong ignorability assumption discussed in Section 3.

heading: 8 Ethics Statement
text: For any model designed to extract persuasive concepts, there is a risk that bad actors could use it to improve their ability to manipulate others. Many other tools exist which could presumably be used for this purpose, so we believe that the benefits of having this model open source outweigh this risk. An example of this kind of trade-off can be seen in the context of the model’s application to censorship. When governments utilize human censors, they could potentially use this model to identify new keywords to add to an automated censorship blacklist to improve efficiency. On the other hand, the model can also be used to reverse engineer the process and reveal censorship policies, as we demonstrate. Acknowledging the possibility for misuse, we believe that the opportunities for productive and socially beneficial application are greater.

heading: Acknowledgements
text: This work was supported in part by the National Science Foundation RIDIR program, award numbers 1738411. We thank Kevin Li for contributions to an early version of this work.

heading: A Supplemental Results
text: Interpretation of all learned filters by the censorship model
f W outf β̂f Confidence Interval of β̂f Top extracted phrases (translated)
Known censored phrases
1 1.42 0.23 [0.18, 0.28]
“[CLS] 武汉病毒所党”,“验武汉病毒所专”,“[CLS] 武 汉 病 毒 所’”,“？ 武 汉 病 毒 所 辟”,“。 武 汉 病毒所所” [“[CLS]Wuhan Institute of Virology Party”,“Wuhan Institute of Virology Specialty”,“[CLS]Wuhan Institute of Virology’”,“? Created by the Wuhan virus”, “. Wuhan Institute of Virology”] “Wuhan virus”
2 1.32 0.23 [0.19, 0.28]
“国难财”，如此人”,“汉病毒所说某中”,“国难财比如 某些”,“国难财也敢发，”,“国难财, 有些人” [“Profiting from national disasters, such people”, “Chinese virus said that some people”, “Profiting from national disasters, such as some people”, “Profiting from national disasters, some people dare to make money,”,“Profiting from national disasters, some people”] “Profiting from national disasters”
3 1.21 0.26 [0.21, 0.31]
“个省的省委书记”,“毒所陈全姣声明”,“任湖北 省委书记”,“毒所的 remdesi” [“Secretary of the Provincial Party Committee of a province”, “ Chen Quanjiao of the Poison Institute stated”, “ Renowned Secretary of the Hubei Provincial Party Committee”, “ Remdesi of the Poison Institute.”] “Provincial party secretary”
4 1.20 0.12 [0.07, 0.17]
“病毒所党委”,“病毒所所长”,“病毒所研究”,“病毒 所联合” [“Party Committee of the Institute of Virology”, “Director of the Institute of Virology”, “Research of the Institute of Virology”, “Union of the Institute of Virology”] “Wuhan virus” (using context of phrases within samples)
5 1.18 0.05 [0.00, 0.10]
“病毒所回应 6 大”,“病毒所所长已经”,“病毒所所 长”（正” [“The top 6 responses from the Institute of Virology”, “Director of the Institute of Virology has been”, ”Director of the Institute of Virology” (positive)] “Wuhan virus”
6 1.09 0.10 [0.06, 0.15]
“那些发国难”,“上是发国难”,“授旗。省委”,“期间 发国难”,“任湖北省委” [“Those who caused national calamity”, “The one who caused national calamity”, “granted the flag. Provincial Party Committee”, ”During the national crisis”, ”Served as Hubei Provincial Party Committee”] “National crisis”
7 1.07 0.10 [0.05, 0.15] “武汉病毒所” [“Wuhan Institute of Virology”] “Wuhan virus”
8 1.03 0.15 [0.11, 0.19] “发国难财！”,“发国难财” [“Profiting from nationaldisasters! ”,“Profiting from national disasters”]
9 0.91 0.07 [0.02, 0.11] “确诊与新天地教”,“一直等新天地教”,“不保证打 款时间” [”Diagnosis and Shincheonji Teaching”, ”Always waiting for Shincheonji Teaching”, ”No guarantee of payment time”] “Shincheonji Church”
10 0.77 0.11 [0.07, 0.16] “蒋超良在武” [”Jiang Chaoliang is in Wuhan”] “Jiang Chaoliang”
11∗ −0.38 - -
“2020 我们需要的是”,“: 辛苦啦，希望”,“！辛苦 了！抱抱”,“，东西都来不及”,“？有坚持有希望” [“What we need in 2020 is”,“:Thank you for your hard work, hope”,“! Thanks for your hard work! Hug”,“, it’s too late for anything”,“? ”Persistence and hope”
Interpretation of all learned filters by the CFPB model

heading: B Benchmarks
text: We evaluate the models developed for each application by comparing them to two benchmark methods.
1. F&G: The methodology presented in Fong and Grimmer (2016) is motivated by the same setting as this paper. Rather than neural networks with convolutional structures, they utilize a topic modeling approach via the supervised Indian Buffet process to discover latent text treatments. We implement their methodology to both of our applications using the texteffect R package produced by the same authors. For each application the input is a word-document matrix, with vocabulary restricted by excluding stop words and words that appear infrequently (< 400 times for the censorship data set and < 300 times for the CFPB data set). For the censorship application, we perform word segmentation using the jiebaR package. Using the training set, we perform a parameter search using the sibp_param_search function over alphas in {3, 5} and sigmasq.ns in {0.2, 0.4, 0.6}. For both applications, we set the number of topics to K = 16 so that we obtain the same number of latent text treatments as our models identify. The final text treatments are chosen using the sibp_rank_runs function.
2. Regularized logistic regression (RLR): We perform regularized logistic regression with a L1 penalty on 3-grams in each corpus. This method is simple relative to the model presented in this paper and to the F&G model, but offers a very clear interpretation of the text features that are selected as predictors of the outcome. Instead of requiring subjective labeling of latent features, this method simply selects 3-grams whose in-sample frequencies are predictive of the outcome. We chose 3-grams to ensure that phrase lengths would be similar to those highlighted in our model, as BERT uses sub-word tokenization. For both applications we exclude stop words and only consider n-grams with frequencies over a certain threshold to control vocabulary size (n-grams with frequencies < 50 were excluded from the CFPB analysis, and < 200 from the censorship analysis). We chose the penalty parameter to be the minimum magnitude such that at most 16 3-grams were selected by each model. For the censorship data, multiple selected 3-grams were perfectly collinear with other 3-grams and were dropped from the final regressions and comparisons. This tuning and variable selection process was performed using the training set, and final estimates were computed using the test set.
We use two metrics to quantitatively assess the extent to which our model’s learned latent text features predict the outcome compared to benchmark methods. First, we performed linear regressions of the outcome against the features identified by each model to compare Adjusted R-squared values. This assesses the degree to which latent features captured variation in the outcome. Feature representations are learned (F&G method) and n-grams are selected (regularized logistic regression method) in the training set, and evaluation of R2adj is calculated using the the test set. Second, we calculated the prediction accuracy (mean-squared error) of regression models fit to each method’s latent features in the training set when applied to the test set. These quantitative results are shown in the main body of the paper.
A key motivation for this work is to enable researchers to identify and understand text treatments from a text corpus. Therefore an important, but subjective, feature to compare between our models and benchmarks is the interpretability of identified text features. For this purpose, we list simplified versions of our convolutional filter interpretation tables, tables with the top keywords per topic and manual topic labels from the F&G method results, and tables with the selected n-grams from the regularized logistic regression method. In the censorship case, we can also compare the ability of the models to recover keywords that are known to cause censorship in the corpus.
Finally, the extent to which latent features correlate with each other across methods may also provide insights into the patterns that each method is successful or not successful at detecting.
To assess this, we include correlation plots for our method’s identified latent features paired with those identified by each of the other methods.
B.1 Censorship
B.2 CFPB

heading: Future Work
text: Future work investigating how benefits from these changes trade-off with reduced computational efficiency would be relevant to researchers using this method. Computational expense While these results demonstrate that the proposed CNNbased method is able to significantly outperform benchmark methods, these improvements come with a substantially higher computational cost in terms of both time and memory. We observed that the regularized logistic regression method was by far the most efficient option overall, with the F&G method also exhibiting much higher memory-efficiency but less time-efficiency. Future work comparing the computational expense of these methods across a larger range of datasets would be valuable for practitioners. Trade-off between experimental costs and less-interpretable treatments Under the assumptions discussed in Section 3, researchers may estimate causal effects by directly testing the identified latent text treatments. This simplifies the experimental pipeline, but as in Egami et al. (2018) and Fong and Grimmer (2016), comes with the drawback of requiring the researcher to somewhat subjectively interpret the identified latent text treatments that are being tested. Alternatively, researchers may use their interpretations of the discovered latent text features to inspire “manifest” text treatments (ex. specific keywords, sentence structures) to test in confirmatory settings. In this case, the text features being tested would be known and manipulated by the researcher, allowing clearer interpretation of effects and weaker assumptions. This alternative comes with the downside of requiring researchers to run follow-up experiments. Incorporating uncertainty in latent treatments Our paper does not provide guidance for incorporating the model-based uncertainty involved in identifying and estimating
the latent text treatments into causal effect estimates. Designing experimental texts We generally recommend using our model to guide the selection of text treatments for use in followup experiments. Designing experimental texts to isolate treatments of interest is a non-trivial task, and is left to the experimenter. In many cases, it is challenging to imagine altering a specific part of a text without affecting the interpretation of surrounding text that is not directly manipulated. This makes it difficult to establish causality for a specific text feature, rather than for the aggregate differences between a set of texts. This is a known challenge of making causal inferences with text, and relates to the strong ignorability assumption discussed in Section 3