NLU -Co at S em E val-2020 Task 5: NLU / SVM Based Model Apply Tocharacterise and Extract Counterfactual Items on Raw Data (2020)

Abstract: In this article, we try to solve the problem of classification of counterfactual statements and extraction of antecedents/consequences in raw data, by mobilizing on one hand Support vector machine (SVMs) and on the other hand Natural Language Understanding (NLU) infrastructures available on the market for conversational agents. Our experiments allowed us to test different pipelines of two known platforms (Snips NLU and Rasa NLU). The results obtained show that a Rasa NLU pipeline, built with a well-preprocessed dataset and tuned algorithms, allows to model accurately the structure of a counterfactual event, in order to facilitate the identification and the extraction of its components.

## introduction we can define counterfactuals as thoughts about alternatives to past events (factual), that is, thoughts of what might have been (counter). these counterfactual thoughts are preventive in nature because they help us better anticipate and guide our daily actions. practically, counterfactual statements require a deep semantic analysis to identify them and extract their characteristic components (antecedent and consequent). though natural language understanding (nlu) is usually used in chatbots, digital personal assistants, or spoken dialogue systems (sds), its techniques can also be applied in information retrieval/extraction and more specifically in the semantic classification of events @xcite , @xcite . this proposal is based on the starting baseline @xcite . we want to exploit the pipelines offered by two mainstream nlu libraries to classify and extract the consequent and antecedent segments occurring in a textual sequence of counterfactual type. for subtask 1, two systems are proposed: the first based on support vector machine (svm) and the second on nlu. for subtask 2, we only use nlu pipelines. the two nlu solutions are built on the basis of snips nlu @xcite and rasa nlu @xcite . ## background: counterfactual thoughts and statements the counterfactual problem is above all a phenomenon of human thinking and reasoning. it was widely studied quite early in several disciplines, mainly in philosophy with aristotle, plato, leibniz (he describes the existence of several worlds gravitating around an event controlled by laws of logic.) @xcite . as described by @xcite , this thought can target reflections on the negativity of an event (downward @xmath0. kahneman and miller proposed a norm theory @xcite which describes counterfactual as "unusual experiences tending to result in 'if only' thoughts that recapitulate the normal state of affairs." @xcite . lewis @xcite on his side works on counterfactual laws that apply to causation, linked by conditions and dependencies between the events of the counterfactual relationship: "if a were the case, c would be the case" is true in the actual world if and only if either (i) there are no possible a-worlds; or (ii) some a-world where c holds is closer to the actual world than is any a-world where c does not hold. @xcite as we can see, the problem of counterfactuals is equally a problem of factual conditionals @xcite . according to the functional theory defined by @xcite , counterfactual thinking is a psychological process which makes it possible to solve problems. in this sense, if a problem arises, counterfactual thinking should have the effect of evoking behaviors that correct these problems. other studies arise on the responsibility that we have for a fact towards oneself or the others (self counterfactual), on the cognitive principles guiding the possibilities that people think of, when they imagine an alternative to reality, and on the rational counterfactuals @xcite which are those in the counterfactual universe (the possible worlds) corresponding to a given fact that maximizes a particular objective, according to the neuronal model (counterfactual machine). in other words, the theory of rational counterfactuals tries to identify the antecedent that gives the desired consequent. these theories have been applied in several real-life problems (decision-making, patient depression), in several fields (engineering, human science, computer science, economics, medicine), the latter being the focus in this work. concretely, in the next section 3, our question will be: how to mobilize the properties of rational counterfactuals in the svm and nlu pipelines ? our laboratory worked on a project in the space field, aiming at detecting causalities in patents. the goal was to annotate a corpus of patents with causalities using active learning, then use the annotated data to implement a causality detector. this currently completed work allowed us to assess the difficulties for this kind of annotation, but also the possibilities to learn models that can partially detect explicit causality entities and relations. ## counterfactual statements classification based on svm and nlu pipelines for the experiments below, we used the initial data (csv file) with 13000 sentences which are organised as follows: 1454 counterfactual statements and 11546 uncounterfactual statements. we noticed at first glance that these data were unevenly represented on both sides, with a dominance of uncounterfactual statements at almost 89% over counterfactual statements. as it is and without standardization, the classification of the two classes will be unequal. to confirm this hypothesis, we carried out training sessions on one hand with all the data sets and on the other hand with standardized data sets (in equal proportions). ## conclusion the aim of this work was to classify counterfactual statements according to whether they are counterfactual or uncounterfactual. we participated in this challenges within the framework of a project on causality conducted by our team. the challenge itself aimed at extracting, within a few thousand counterfactual statements, the representative segments of antecedents on one hand and consequents on the other hand. to solve these problems, we opted for the use of svm and nlu. we implemented a simple system for task 1, which obtained minimal results. the task 2 was much harder and we did not achieve a very good performance, but conducted some additional experiments. although our results are not satisfactory from a competitive point of view, our experiments have shown the advantages and disadvantages of nlu pipelines, similar to the state of the art, generally used for conversational agents. we remain convinced that further modeling and new experiments will make it possible to adapt this problem of classifying and extracting counterfactual statements to nlu pipelines.