ECNU - S ense M aker at S em E val-2020 Task 4: Leveraging Heterogeneous Knowledge Resources for Commonsense Validation and Explanation (2020)

Abstract: This paper describes our system for SemEval-2020 Task 4: Commonsense Validation and Explanation (Wang et al., 2020). We propose a novel Knowledge-enhanced Graph Attention Network (KEGAT) architecture for this task, leveraging heterogeneous knowledge from both the structured knowledge base (i.e. ConceptNet) and unstructured text to better improve the ability of a machine in commonsense understanding. This model has a powerful commonsense inference capability via utilizing suitable commonsense incorporation methods and upgraded data augmentation techniques. Besides, an internal sharing mechanism is cooperated to prohibit our model from insufficient and excessive reasoning for commonsense. As a result, this model performs quite well in both validation and explanation. For instance, it achieves state-of-the-art accuracy in the subtask called Commonsense Explanation (Multi-Choice). We officially name the system as ECNU-SenseMaker. Code is publicly available at https://github.com/ECNU-ICA/ECNU-SenseMaker .

## introduction commonsense reasoning is the process of making decisions by combining facts and beliefs with some basic knowledge appeared in daily life, which is fundamental to many natural language understanding (nlu) tasks @xcite . however, most existing models are quite weak in terms of commonsense acquisition and understanding compared with humans since commonsense is often expressed implicitly and constantly evolving @xcite . thus, the commonsense problem is considered to be an important bottleneck of modern nlu systems @xcite . many attempts have been made to empower machines with human abilities in commonsense understanding, and several benchmarks have emerged to verify the commonsense reasoning capability, such as squabu @xcite and commonsenseqa @xcite . however, these benchmarks estimate commonsense indirectly and do not include corresponding explanations. therefore, the commonsense validation and explanation (comve) benchmark is proposed which directly asks the real reasons behind decision making @xcite . it consists of three subtasks, and we propose an entire system (ecnu-sensemaker) to solve the first two subtasks, namely, commonsense validation and commonsense explanation (multi-choice). in fact, the two different subtasks correlate to each other. subtask a (validation) requires the model to select which one is invalid between two given statements. while subtask b (explanation) aims to identify one real reason from two other confusing options to explain why the statement is invalid. compared with the previous benchmarks, these subtasks are non-trivial because machines does not have the ability of children and adults in accumulating commonsense knowledge in daily life. since most questions in such subtasks require knowledge beyond the mentioned facts in text, even trained over millions of sentences, there still exist a great gap behind human performance in commonsense reasoning, not to mention explanation capability @xcite . besides, many previous methods on commonsense reasoning have some important drawbacks which remain to be unsolved. for example, the ways of combining knowledge into deep learning architectures are far from satisfaction, and it is an issue to balance the tradeoff between noise and the amount of incorporated commonsense from knowledge base such as conceptnet @xcite . in addition, researchers still consider this commonsense knowledge base to be incomplete despite it has been evolved for a long time. the large majority of commonsense is generally recognized to be implicitly expressed in unstructured text or human interactions in everyday life. furthermore, some pilot experiments have shown inference remains a challenging problem in the above subtasks @xcite . to overcome these difficulties, we propose a novel knowledge-enhanced graph attention network (kegat) architecture for both commonsense validation and explanation. different from the ordinary graph attention networks @xcite , this model could leverage heterogeneous knowledge resources to improve its commonsense reasoning ability. on one hand, commonsense knowledge in the structured knowledge base (i.e. conceptnet) is well incorporated and utilized by the system with an elegant structural design as knowledge-enhanced embedding module. on the other, an upgraded data augmentation technique is put forward to empower the model with commonsense understanding capability based on the unlimited commonsense knowledge learnt from a large amount of unstructured text. besides, an internal sharing mechanism is cooperated to prohibit our model from insufficient and excessive reasoning for commonsense. in this way, we build a novel graph neural network based architecture for both validation and explanation tasks, making more accurate inference for commonsense understanding. we officially name it as the ecnu-sensemaker system. in summary, the main contributions of our system are as follows: ## problem definition formally, each instance in the comve task consists of 5 sentences {s 1 , s 2 , r 1 , r 2 , r 3 }, where s 1 and s 2 are two statements with similar expressions but only one of them conforms to commonsense, and r 1 , r 2 , r 3 stand for three optional explanations. subtask a, called commonsense validation, requires the model to identify the against-common-sense sentence ŝfrom the above two statements s 1 and s 2 , where ŝ-∈ {s 1 ,s 2 }. while subtask b, called commonsense explanation (multi-choice), requires the model to pick the most reasonable explanation r from the three given options r 1 , r 2 , r 3 for the againstcommon-sense statement ŝ-, where @xmath0,r 2 , r 3 }. subtask c, called commonsense explanation (generation), is similar to subtask b, but it requires the model to generate a reasonable explanation r. ## methodology in this section, we describe the framework of our proposed ecnu-sensemaker system. an overview of this architecture is depicted in figure 1 . we will introduce the basic components of this architecture, and provide some novel proposed strategies to enhance the commonsense reasoning ability of our system. ## conclusion in this paper, we propose the ecnu-sensemaker system, which utilizes a novel knowledge-enhanced graph attention network architecture to solve both commonsense validation and explanation tasks. it well incorporates heterogeneous knowledge from both unstructured text and structured resources such as conceptnet, and relies on the graph attention module with an internal sharing mechanism to improve the commonsense reasoning ability of the model. our model achieves good results in the newest leaderboard of semeval-2020, solving two subtasks with an entire model. we hope our work can shed some lights to directly empower machines with human abilities in commonsense understanding and reasoning.