Exploiting A bstract M eaning R epresentation for Open-Domain Question Answering (2023)

Abstract: The Open-Domain Question Answering (ODQA) task involves retrieving and subsequently generating answers from fine-grained relevant passages within a database. Current systems leverage Pretrained Language Models (PLMs) to model the relationship between questions and passages. However, the diversity in surface form expressions can hinder the model’s ability to capture accurate correlations, especially within complex contexts. Therefore, we utilize Abstract Meaning Representation (AMR) graphs to assist the model in understanding complex semantic information. We introduce a method known as Graph-as-Token (GST) to incorporate AMRs into PLMs. Results from Natural Questions (NQ) and TriviaQA (TQ) demonstrate that our GST method can significantly improve performance, resulting in up to 2.44/3.17 Exact Match score improvements on NQ/TQ respectively. Furthermore, our method enhances robustness and outperforms alternative Graph Neural Network (GNN) methods for integrating AMRs. To the best of our knowledge, we are the first to employ semantic graphs in ODQA.

## introduction question answering (qa) is a significant task in natural language processing (nlp) @xcite . open-domain qa (odqa) @xcite , particularly, requires models to output a singular answer in response to a given question using a set of passages that can total in the millions. odqa presents two technical challenges: the first is retrieving @xcite and reranking @xcite relevant passages from the dataset, and the second is generating an answer for the question using the selected passages. in this work, we focus on the reranking and reading processes, which necessitate fine-grained interaction between the question and passages. existing work attempts to address these challenges using pretrained language models (plms) @xcite . however, the diverse surface form expressions often make it challenging for the model to capture accurate correlations, especially when the context is lengthy and complex. we present an example from our experiments in figure 1 . in response to the question, the reranker incorrectly ranks a confusing passage first, and the reader generates the answer "2015-16". the error arises from the plms' inability to effectively handle the complex semantic structure. despite "mvp", "stephen curry" and "won the award" appearing together, they are not semantically related. in contrast, in the amr graph, it is clear that "stephen curry" wins over "international players", not the "mvp", which helps the model avoid the mistake. the baseline model may fail to associate "most valuable player" in the passage with "mvp" in the question, which may be why the baseline does not rank it in the top10. to address this issue, we adopt structured semantics (i.e., abstract meaning representation @xcite graphs shown on the right of figure 1 ) to enhance open-domain qa. while previous work has integrated graphs into neural models for nlp tasks, adding additional neural architectures to plms can be non-trivial, as training a graph network without compromising the original architecture of plms can be challenging @xcite . converting amr graphs directly into text sequences and appending them can be natural, but leads to excessively long sequences, exceeding the maximum processing length of the transformer. to integrate amr into plms without altering the transformer architecture and at a manageable cost, we treat nodes and edges of amr graphs as tokens (gst) in plms. this is achieved by projecting the embeddings of each node/edge, which consist of multiple tokens, into a single token embedding and appending them to the textual sequence embeddings. this allows for integration into plms without altering the main model architecture. this method does not need to integrate a graph neural network into the transformer architecture of plms, which is commonly used in integrating graph information into plms @xcite ; @xcite . the gst method is inspired by @xcite in the graph learning domain, who uses token embeddings to represent nodes and edges for the transformer architecture in graph learning tasks. however, their method is not tailored for nlp tasks, does not consider the textual sequence embeddings, and only handles a certain types of nodes/edges, whereas we address unlimited types of nodes/edges consisting of various tokens. specifically, we select bart and fid as baselines for the reranking and reading tasks, respectively. to integrate amr information, we initially embed each question-passage pair into text embeddings. next, we parse the pair into a single amr graph using amrbart @xcite . we then employ the gst method to embed the graph nodes and graph edges into graph token embeddings and concatenate them with the text embeddings. lastly, we feed the concatenated text-graph embeddings as the input embeddings to a bartbased @xcite reranker to rerank or a fid-based @xcite reader to generate answers. we validate the effectiveness of our gst approach using two datasets -natural question @xcite and triviaqa @xcite . results indicate that amr enhances the models' ability to understand complex semantics and improves robustness. bart-gst-reranker and fid-gst outperform bart-reranker and fid on the reranking and reading tasks, respectively, achieving up to 5.9 in top5 scores, 3.4 in top10 score improvements, and a 2.44 increase in exact match on nq. when the test questions are paraphrased, models equipped with gst prove more robust than the baselines. additionally, gst outperforms alternative gnn methods, such as graphtransformer and relational graph convolution network (rgcn) @xcite , for integrating amr. to the best of our knowledge, we are the first to incorporate semantic graphs into odqa, thereby achieving better results than the baselines. ## method we introduce the retrieval and reading of open-domain qa and their baselines in section 3.1, amr graph generation in section 3.2 and our method graph-as-token (gst) in section 3.3. ## conclusion in this study, we successfully incorporated abstract meaning representation (amr) into open-domain question answering (odqa) by innovatively employing a graph-as-token (gst) method to assimilate amrs with pretrained language models. the reranking and reading experiments conducted on the natural questions and triviaqa datasets have demonstrated that our novel approach can notably enhance the performance and resilience of pretrained language models (plms) within the realm of odqa. "leading goose" r&d program of zhejiang under grant number 2022sdxhdx0003.