A Semantics-aware Transformer Model of Relation Linking for Knowledge Base Question Answering (2021)

Abstract: Relation linking is a crucial component of Knowledge Base Question Answering systems. Existing systems use a wide variety of heuristics, or ensembles of multiple systems, heavily relying on the surface question text. However, the explicit semantic parse of the question is a rich source of relation information that is not taken advantage of. We propose a simple transformer-based neural model for relation linking that leverages the AMR semantic parse of a sentence. Our system significantly outperforms the state-of-the-art on 4 popular benchmark datasets. These are based on either DBpedia or Wikidata, demonstrating that our approach is effective across KGs.

## introduction knowledge base question answering (kbqa) has received significant interest due to its real-world applications. kbqa is a task where a natural language question is transformed into a precise structured query, using entity linking and relation linking as necessary sub-tasks to retrieve an answer. for example, the question "who founded the city where pat vincent died?" requires mapping (a) founded and died to relations dbo:founder and dbo:deathplace, and (b) entity pat vincent to dbr:pat vincent, given dbpedia as the knowledge base. semantic parses such as abstract meaning representation (amr) have recently shown to be useful for the kbqa task @xcite . however, critical tasks for kbqa such as relation linking continue to be addressed primarily using the question text @xcite @xcite , ignoring the amr parses of the question which can introduce additional semantics. in the literature, some systems such as sling @xcite have used amr for relation linking. however, similar to other rulebased approaches @xcite , sling depends heavily on the specific target @xcite and it is based on a complex ensemble of different approaches, making portability to new knowledge bases a non-trivial task. in this work, we propose semrel; a single semantics-aware neural model for relation linking. semrel takes as input the question text annotated with its amr parse and entity information and outputs a ranked list of relations. the key contributions of this work are as follows: (a) a simple, knowledge graph agnostic neural model for relation linking over knowledge bases, (b) leveraging amr parses for better question representation, and (c) an experimental evaluation using four datasets based on dbpedia and wikidata where we show that semrel consistently outperforms existing systems on all datasets. ## semantics-aware relation linking we propose a relation linking system that exploits the semantic structure of a sentence to retrieve relevant relations from the underlying knowledge base. we hypothesise that semantic representations abstract away from lexical forms, providing structural clues that are more consistent across training examples than surface text. to this end, we use the amr graph of the sentence as its semantic structure. amrs are directed acyclic graphs that capture who is doing what to whom in a sentence. the nodes in the graph are concepts and the edges are labelled with relations between those concepts. figure 1 shows example amr graphs for the question "who founded the city where pat vincent died?". note that the amr graph for the question represents the target of the query as a special node labelled 'amr-unknown'. the inputs to our system are: the question text, its amr graph and the entities in the question marked and linked foot_0 . relation linking is performed in two steps. first, our system identifies the number of expected relations and their location both in the sentence and in the amr graph. next, for each identified slot, the most likely relation is predicted using a transformer based neural model, that ranks them using their english labels from the kg. the amr structure of the sentence is crucial in both steps. figure 2 shows a schematic diagram of overall system. in the following, we first explain the process of finding potential relation slots using amr graph. next we describe in detail our relation linking module. ## evaluation in this section, we detail our experimental setup and evaluate our approach against the state-of-the-art kbqa relation linking approaches. for fair comparisons, we replicate the same settings adopted by the systems we compare with both in terms of datasets and metrics. ## related work several relation linking systems have been proposed recently @xcite @xcite @xcite . most of these methods are rule-based and rely solely on the question text and/or its dependency parse. therefore, they try to improve their question understanding by using standard nlp tools such as pos tagging, tokenization n-gram tiling and even lexical database such as wordnet. falcon 2.0 @xcite is a joint entity and relation linking tool over wikidata. it uses a search engine indexed with wikidata, a pipeline of text processing including pos tagging, tokenization, n-gram tiling/splitting and a catalog of rules for entity and relation linking. kbpearl @xcite is another system that performs joint entity and relation linking to wikidata. it first create a semantic graph of text using openie and maps both entities and relations to a given kb. however, none of the above mentioned methods for kbqa perform relation linking on two different kbs using the same system whilst our work is the first to perform relation linking over both dbpedia and wikidata using the same system. in addition, some of these systems are kg-specific; e.g. falcon @xcite vs. falcon 2.0 @xcite , where adapting it from one kg to another requires non-trivial changes. unlike these systems, semrel leverages well-established semantic parsers such as amr to achieve out-of-thebox better question representation. similar to our approach, sling (mihindukulasooriya et al., 2020) is a relation linking framework based on dbpedia which leverages semantic parsing using amr and distant supervision. it consists of four distinct modules that capture different signals such as linguistic cues, semantic representation, and information from the knowledge base. unlike sling, semrel is a kg-agnostic, single end-to-end neural model that does not require various ensemble components and yet achieves stateof-the-art performance on dbpedia and wikidata datasets. ## conclusions and future work in this paper, we present a simple transformerbased neural model for relation linking that leverages the semantic structure of a sentence. in contrast to existing systems such as sling and falcon, which are either rule-based or ensembles of several components, our neural architecture enables us to adapt the system to multiple kgs (e.g. dbpedia and wikidata). it outperforms state-of-the-art systems on a variety of benchmarks. our ablation study shows that including the amr graph improves performance, even with the relatively simple encoding scheme (plain text). in future, we will explore modeling the graph structure explicitly. this model also relies on a deterministic slot-finding algorithm based on amr. while this identifies the correct relation slots most of the time, it is rule-based, and not always correct. in future work, we will explore learning algorithms to identify the slots from the amr graph. finally, amr parsers can be trained jointly with the relation linking objective. currently, these parsers are sensitive to small changes in the input. joint training can make them robust against text variations and more sensitive to the errors affecting slot identification and relation prediction.