Venue: CLeaR 2022 Conference

Reviews:

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: Summary: The paper discusses common misusage of causal tools and language in the machine learning literature, clarifying certain concepts and pointing out important flaws.
Main review: Let me preface my review by saying that I'm not an expert in NLP nor philosophy of causality (or any kind for the matter). I do however consider myself pretty well versed in machine learning and its intersection with causality, and found this paper very useful.

The paper clarifies many definitions present in the potential outcomes theory and the assumptions that they entail. Furthermore, the authors show how other papers in the ML / NLP literature misuse these terms and implicitly or subtly fail to satisfy the assumptions of the methods they employ. At the very core of it, it shows how people in the machine learning literature constantly misuse the term "causal" when making claims, particularly when no manipulation can be imagined. Of particular importance is the fact that using potential outcomes algorithms without satisfying SUTVA or strong ignorability conditions ( similar to "each unit be potentially exposable to any of the causes") can result in wrong claims, which is seen in machine learning very often. The examples are very easy to understand and it points to very good literature for further clarification.

The paper is very well written, is correct, and extremely precise. It clarifies a lot of misuses and I personally learnt a lot from it. I would love to see a journal version of this paper with more examples of misuse / false claims of causality in the ML literature.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary: This paper examines the role of textual data as study units when conducting causal inference by drawing parallels between human subjects and organized texts. This paper expounds the main concepts and principles of causal inference, and reveals some vague concepts and fallacies. This paper discusses two strategies in order to facilitate better framing a causal query. 
Main review: Advantages:
1)	The two strategies proposed sound reliable.
2)	This paper provides a wealth of cases to illustrate points.
3)	The relevant work provided in this paper is sufficient.

Disadvantages:
1)	Some concepts lack formal definitions, for instance, the "Study Unit".
2)	The strategy proposed in this paper lacks formal expressions and illustrations, which makes the strategy difficult to understand.
3)	Although the strategy proposed in this paper is claimed to be effective in some cases, there is a lack of experiments to prove its effect.
4)	Only two strategies are proposed but the specific implementation is lacking, which makes the contribution may be insufficient
5)	The presentation of this paper is not clear enough and lacks pictures to express opinions intuitively.
review_title: null

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: The authors discuss the application of causal inference methods to textual data, and propose two strategies for framing causal questions in a more precise manner: (i) shifting from immutable traits to perceptions of them, and (ii) shifting from abstract concepts to its constituent parts. They discuss how these strategies address the pecularities pertaining to textual data and how they can help in more clearly defining causal inference problems. 
Main review: I would like to start the review by mentioning that this article reads more like a perspective piece than an original research article. This is perfectly fine as far as I am concerned, but I am not sure if the submission then fits this conference's call for papers.

I find that this article is of decent quality and provides a useful perspective on how causal inference methodology can be applied to textual data. The strategies proposed are sound and have value beyond the textual domain. As far as I can tell, the authors manage to include a reasonably comprehensive overview of the current state-of-the-art causal inference methods and applications for textual data. Finally, the paper is very well written and structured, and virtually error-free.

The scope of the article is perhaps a bit limited. The authors focus on causal effect estimation via covariate adjustment, and do not touch upon tasks such as causal structure learning. It might be interesting to include more facets of causal inference into the discussion.

Overall, I think this article is an interesting perspective on the current and future use of causal inference methodology on textual applications, and would be a reasonable contribution to the conference.

Minor comments:
- page 10: "articulate treatment or causal variable" -> "articulate the treatment or causal variable"
review_title: null

Meta-review: This paper wholeheartedly adopts the potential-outcomes approach to causality and causal inference,  the Holland (1986) slogan of "no causation without manipulation", and the common notion within that school that it is nonsensical to talk about the causal effects of race or sex because those could not imaginably be manipulated while still being the "same person", and then extends these ideas by analogy to text.  On this basis, a lot of things people try to do by way of estimating causal effects of textual features (or using some aspects of texts to control for others, etc.) are indeed nonsensical.  One can also, by analogy, adopt ways people have come up with in the social-scientific literature to evade the Holland/Rubin strictures against ascribing causal effects to race or sex: one can manipulation _perceptions_ of the immutable variables, one can articulate the immutable variables into bundles of usually-associated variables and find aspects which are susceptible to manipulation, etc.

The reviewers found this mostly persuasive, and I suppose I do too, if I can bring myself to accept the original claims about race and sex. I myself find those claims unpersuasive [*], but I recognize that many people in the profession disagree with me.  The analogy on which this paper rests is convincing and well-argued, and many of the suggestions about, e.g., manipulating perceptions are good ideas _even if_ one thinks textual effects are better-defined than they do.

[*]:  These strictures rest on pronouncements by statisticians about what changes would leave a unit "the same person" and what would amount to being "a different person".  These are hard metaphysical questions, far outside our expertise, which we usually avoid, but here they are crucial, and seem to rest merely on intuition about what attributes are part of someone's essence.  (If I were cynical, or a sociologist of science, I'd observe that Holland made his career at the Educational Testing Service, that Rubin consulted for ETS, etc., and that the ETS had a real _interest_ in being able to assert that, scientifically speaking, no one gets a bad score on the SAT _because_ of their race or sex.)  Certainly there are plenty of philosophers of causality, including those who work closely on statistical issues of causal inference like Clark Glymour, who have absolutely no problem with assigning causal effects to race and sex.  (Two citations I happen to have handy: C. Glymour & M. Glymour _Epidemiology_ **25** (2014): 488-490 [https://doi.org/10.1097/EDE.0000000000000122], and  A. Marcellesi, _Philosophy of Science_ **80** (2013): 650--659 [https://doi.org/10.1086/673721].)  Whether  these philosophers' arguments could be extended, analogically, to text is an interesting question that would call for a paper of its own.
Decision: Accept (Poster)