A Review of Dataset and Labeling Methods for Causality Extraction (2020)

Abstract: Causality represents the most important kind of correlation between events. Extracting causali-ty from text has become a promising hot topic in NLP. However, there is no mature research systems and datasets for public evaluation. Moreover, there is a lack of unified causal sequence label methods, which constitute the key factors that hinder the progress of causality extraction research. We survey the limitations and shortcomings of existing causality research field com-prehensively from the aspects of basic concepts, extraction methods, experimental data, and la-bel methods, so as to provide reference for future research on causality extraction. We summa-rize the existing causality datasets, explore their practicability and extensibility from multiple perspectives and create a new causal dataset ESC. Aiming at the problem of causal sequence labeling, we analyse the existing methods with a summarization of its regulation and propose a new causal label method of core word. Multiple candidate causal label sequences are put for-ward according to label controversy to explore the optimal label method through experiments, and suggestions are provided for selecting label method.

## introduction causality represents a kind of relationships between "cause" and "effect", stating that the happening of causes will trigger the happening of effects, which constitutes the basics for inference and reasoning. in early days, people found causal relations from production process and daily lives manually, slow and inaccurate. the ever growing web resources have made it feasible to automatically extracts causality from text, triggering an emerging and hot topic in nlp, with abundant downstream application tasks such as event detection and prediction @xcite , questions answering @xcite . for example, the new coronary pneumonia has drawn global wide attention recently, the causal relationship between "new coronavirus" and "eating wild animals" can be inferred by an expert system with causal knowledge, and it may be predicted that "new coronary pneumonia" can lead to "death". artificial intelligence can be used to assist medical research and strengthen prevention and treatment during early stage of infection. therefore, causal relation extraction is a basic task in text mining, which is driven by human's instinctive desire for knowledge. many researchers have devoted themselves to the research of causal relationship extraction, however, it is still a new field with some open problems to slove and publicly evaluated datasets. the existing relevant studies each has its own research system that cannot be compared horizontally, so the systematic research system is the key factors for the progress of causality extraction research. targeting at the limitations and problems in causality research, we comprehensively review the basic concepts, extraction methods, experimental data, labeling methods from multiple perspectives, for future research on causality extraction as a reference. the main contributions of this paper are listed as follows: ## dataset for causality experimental data is the basis of all research, however, there is currently no publicly evaluated dataset. which is one of the key factors that hinder the progress of causality research. according to the deficiencies for the causal dataset construction, we analyze the existing dataset for the subsequent studies. ## causal sequence labeling method for the causality extraction based on sequence labeling method, how to label the causal sequence is the key to the formation of experimental data. however, there is no fixed labeling rule, and horizontal comparison is impossible at present. we summarize the existing labeling methods and conduct multiangle analysis, as a reference for the follow-up research. dataset text classification relation extraction sequence labeling semeval remove the entity mark label process causal-timebank remove the entity mark label process altlex × × cec complex process complex process label process scifi remove the entity mark label process esc × × phrase boundary @xcite the method uses the "bio" (begin, inside, other) entity boundary tag in ner to mark the causal phrase boundary. three causality semantic tags "c"(cause), "e"(effect), and "emb(embedded-causality) are used to represent the causal semantics, , the "emb" tag is introduced to solve the problem that embedded-causality cannot give accurate labels. clause subscript (dasgupta et al., 2018) "c"(cause), "e"(effect), "cc" (causal connectives) and "n"(none) are used for causal semantics. the boundary is divided by clauses and different causal pairs are distinguished by the subscript in the form of "id". the current view is that the chronic inflammation o o o o o b-emb i-emb i-emb pylori i-c infection i-c in the distal part o o o o of the o o stomach o results in o o an b-e increased i-e acid i-e production i-e from the o o caused o by o non-infected upper corpus region the o o o o o of o stomach o . o helicobacter b-c figure 2: example for the causal labeling method of clause subscript. the tags "c1", "e1" and "cc1" belong to the first set of causal pair, and the compound tag "e1c2" represents the entity is the effect of the first causal pair, and the cause of the second causal pair, deftly solving the embedded-causality advantages: (1) using clause as the unit solves the problem that the text cannot extract the core word or phrase for causality. (2) a wide range of causal categories is extracted by means of the subscript. disadvantages: (1) in terms of clause unit, lots of irrelevant words are introduced in causal text. the sentence is only cut into two parts according to the causal conjunction in most text, the boundary is too loose and the essence of causal extraction is lost. (2) lots of new causal tags are introduced due to the subscripts, the tags of sentences are distributed unevenly, which makes it difficult to train the feature. event sequence block @xcite with tags "c", "e" and "n" to represent "cause", "effect" and "none", takes the event indicator as the representative of the overall event, without labeling other texts in the event. the "bio" tag is used to distinguish the boundary of causal event blocks which divides the different causal pairs, solving the multiple causality problem. core word @xcite : the method uses "c"(cause), "e"(effect) and "o" (other) for causal tag, and core word for labeling unit. it solves multiple causality according to the number of tags "c", "e" and the embedded-causality by the means of "make rules to specify existing tag" ## conclusions causal relationship extraction is still a new research field with no publicly evaluated dataset and fixed labeling method, which are the basis of all research and one of the important factors to hinder the progress of causality extraction. we summarize and analyzes the defects in the construction of the experimental data and the disputes over the labeling method for causality in all aspects, so as to serve as a reference for the further research on causality. we also explore the optimal labeling method of causal sequence through experiments, and puts forward suggestions for the selection of labeling methods. in addition, we explore the existing research on causality from multiple perspectives, summarizing the causality extraction related concepts (see the appendix b for the detailed table).