We consider the problem of estimating the causal effects of linguistic properties on downstream outcomes . For example, does writing a complaint politely lead to a faster response time? How much will a positive product review increase sales? This paper focuses on two challenges related to the problem  . First, we formalize the causal quantity of interest as the effect of a writer's intent, and establish the assumptions necessary to identify this from observational data. Second, in practice  we only have access to noisy proxies for these linguistic properties---e .g., predictions from classifiers and lexicons. We propose an estimator for this setting and prove that its bias is bounded when we perform an adjustment for the text.  The method leverages (1)  a pre-trained language model (BERT) to adjust for the text , and (2) distant supervision to improve the quality of noisy proxies . We show that our algorithm produces better causal estimates than related methods on two datasets: predicting the effect of music review sentiment on sales, and complaint politeness on response time .
We consider the problem of using observational data to estimate the causal effects of linguistic properties . For example, does writing a complaint politely lead to a faster response time? How much will a positive product review increase sales? This paper addresses two technical challenges related to the problem before developing a practical method . First, we formalize the causal quantity of interest as the effect of a writer's intent, and establish the assumptions necessary to identify this from observational data. Second, in practice , we only have access to noisy proxies for the linguistic properties of interest -- e .g., predictions from classifiers and lexicons. We propose an estimator for this setting and prove that its bias is bounded when we perform an adjustment for the text. Based on these results, we introduce TextCause, an algorithm for estimating causal effects of linguistic properties. The method leverages (1) distant supervision to improve the quality of noisy proxies, and (2) a pre-trained language model (BERT) to adjust for the text . We show that the proposed method outperforms related approaches when estimating the effect of Amazon review sentiment on semi-simulated sales figures. Finally, we present an applied case study investigating the effects of complaint politeness on bureaucratic response times .
R
estimating
using observational data to estimate
D
on downstream outcomes
R
focuses on two
addresses two technical
A
before developing a practical method
A
,
R
these linguistic properties---e
the linguistic properties of interest -- e
A
Based on these results, we introduce TextCause, an algorithm for estimating causal effects of linguistic properties.
A
distant supervision to improve the quality of noisy proxies, and (2)
D
, and (2) distant supervision to improve the quality of noisy proxies
R
our algorithm produces better causal estimates than related methods on two datasets: predicting
the proposed method outperforms related approaches when estimating
R
music
Amazon
R
sales, and
semi-simulated sales figures. Finally, we present an applied case study investigating the effects of
R
response time
bureaucratic response times