Causal Augmentation for Causal Sentence Classification (2021)

Abstract: Scarcity of annotated causal texts leads to poor robustness when training state-of-the-art language models for causal sentence classification. In particular, we found that models misclassify on augmented sentences that have been negated or strengthened with respect to its causal meaning. This is worrying since minor linguistic differences in causal sentences can have disparate meanings. Therefore, we propose the generation of counterfactual causal sentences by creating contrast sets (Gardner et al., 2020) to be included during model training. We experimented on two model architectures and predicted on two out-of-domain corpora. While our strengthening schemes proved useful in improving model performance, for negation, regular edits were insufficient. Thus, we also introduce heuristics like shortening or multiplying root words of a sentence. By including a mixture of edits when training, we achieved performance improvements beyond the baseline across both models, and within and out of corpus’ domain, suggesting that our proposed augmentation can also help models generalize.

## introduction causality is an important concept for knowledge discovery as it conveys the idea of cause and effect. in the simplest sense, a causal relation exists between entities a and b through the statement "a causes b" or "b is caused by a". in recent years, causal relation extraction from text has garnered significant interest in natural language processing (nlp) @xcite @xcite . causal sentence classification (csc) is the task of identifying sentences that contain causal meaning @xcite @xcite . identification of causal sentences is often the first step in tasks like generating plot structures @xcite gtf is well tolerated and helps with catch-up growth and puberty. dietary advice by a dietitian and use of potentially helpful dietary supplements is indicated. ## conclusion & future work we explored the task of csc in a low-resource setting. following recent literature, we generated counterfactual sentences via rule-based edits that change sentences' causal direction and strength. we showed that sota csc models worryingly misclassifies on such augmented sentences. this concern can be mitigated by including of our edits 16 "the increase of the signal might correspond to formation of the high-density excitons, while the reduction of the signal originates from the relaxation." during training. we demonstrated that our proposal improves classification performance both on original and edit sentences, and within and outside of the corpus' domain. however, for negation, we found that the regular format was insufficient to teach effective decision boundaries given limited data size and augmentation templates. therefore, proposed heuristic edits and found performance improvements for both training and ood contexts. for future work, yu et al. (2020)'s recent corpus using scientific press statements annotated with the same four class labels of causality is a promising dataset to replicate our findings upon. additionally, we utilized rule-based augment schemes which have a finite number of working templates. thus, our augmentations might not be lexically diverse. therefore, our subsequent steps would be to explore sota nlp augmentation and generation tools, like from @xcite and @xcite . furthermore, it might be worthwhile to find alternative models that can learn directly from the augmented datasets without the need for heuristics. lastly, our work did not go beyond the "correctness" of the claims. however, in reality, one has to distinguish between causal effects as factual events of real-world or at the level of "meta-causality" @xcite . hence, grounding the claims to world knowledge will be an important research avenue to pursue. wiebke wagner. 2010. steven bird, ewan klein and edward loper: natural language processing with python, analyzing text with the natural language toolkit -o'reilly media, beijing, 2009, isbn 978-0-596-51649-9. lang. resour. evaluation, 44(4):421-424. tongshuang wu, marco tulio ribeiro, jeffrey heer, and daniel weld. 2021. polyjuice: generating counterfactuals for explaining, evaluating, and improving models. in proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), pages 6707-6723, online. association for computational linguistics. jinghang xu, wanli zuo, shining liang, and xianglin zuo. 2020. a review of dataset and labeling methods for causality extraction. in proceedings of the 28th international conference on computational linguistics, coling 2020, barcelona, spain (online), december 8-13, 2020, pages 1519-1531. international committee on computational linguistics. jie yang, soyeon caren han, and josiah poon. 2021. a survey on extraction of causal relations from natural language text. corr, abs/2101.06426. bei yu, yingya li, and jun wang. 2019. detecting causal language use in science findings. in proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, emnlp-ijcnlp 2019, hong kong, china, november 3-7, 2019, pages 4663-4673. association for computational linguistics. bei yu, jun wang, lu guo, and yingya li. 2020. measuring correlation-to-causation exaggeration in press releases. in proceedings of the 28th international conference on computational linguistics, coling 2020, barcelona, spain (online), december 8-13, 2020, pages 4860-4872. international committee on computational linguistics.