ECNU _ ICA _1 S em E val-2021 Task 4: Leveraging Knowledge-enhanced Graph Attention Networks for Reading Comprehension of Abstract Meaning (2021)

Abstract: This paper describes our system for SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning. To accomplish this task, we utilize the Knowledge-Enhanced Graph Attention Network (KEGAT) architecture with a novel semantic space transformation strategy. It leverages heterogeneous knowledge to learn adequate evidences, and seeks for an effective semantic space of abstract concepts to better improve the ability of a machine in understanding the abstract meaning of natural language. Experimental results show that our system achieves strong performance on this task in terms of both imperceptibility and nonspecificity.

## introduction recent years have witnessed the remarkable success of pre-trained language models in machine reading comprehension (mrc). nevertheless, new research points out that these dominant approaches rely heavily on superficial text pattern-matching heuristics to achieve shallow comprehension on natural language @xcite . for humans, the basic ability to represent abstract concepts guarantees an in-depth understanding of natural language. consequently, teaching machines to better comprehend abstract meaning is a significant and urgent step to push the frontier technique of mrc forward. if computers can understand passages as human do, we expect them to accurately predict abstract words that people can use in summaries of the given passages. thus, researchers have recently proposed a reading comprehension of abstract meaning (re-cam) task in semeval 2021. unlike some previous datasets such as cnn/daily mail @xcite that request computers to predict concrete concepts, e.g., named entities, recam requires machines to fill out abstract words removed * equal corresponding authors. from human written summaries. in recam, subtask 1 and subtask 2 respectively evaluate the performance of machines towards imperceptibility and nonspecificity, two formal definitions of abstractness in natural language understanding @xcite . specifically, concrete words refer to things, events, and properties that we can perceive directly with our senses @xcite @xcite , e.g., donut, trees, and red. in contrast, abstract words refer to the ideas and concepts that are distant from immediate perception. examples for abstract words include objective, culture, and economy. subtask 1 requires machines to perform reading comprehension of abstract meaning for imperceptible concepts, while subtask 2 concentrates on hypernyms, which is more abstract and different from the concrete concepts @xcite . to better understand the abstract meaning, we utilize the knowledge-enhanced graph attention network (kegat) architecture with a novel semantic space transformation strategy for recam. it well incorporates structured knowledge base such as conceptnet @xcite and exploits a novel representation transformation strategy to improve the ability of machines in natural language understanding. the main contributions of our system are as follows: ## methodology in this section, we describe the framework of our system and propose some strategies to enhance the reasoning ability of the model. an overview of the architecture is depicted in figure 1 . ## conclusion we utilize a knowledge-enhanced graph attention network architecture with semantic transformation strategies for machines to better comprehend the abstract meanings of natural language. it well incorporates heterogeneous knowledge and relies on graph attention networks to learn adequate evidences. the subsequent semantic transformation enables an effective representation mapping from concrete objects to abstract concepts. our system achieves strong performance on this comprehension task in terms of both imperceptibility and nonspecificity. we hope this work can shed some lights on the study of in-depth reading comprehension.