TIARA : Multi-grained Retrieval for Robust Question Answering over Large Knowledge Base (2022)

Abstract: Pre-trained language models (PLMs) have shown their effectiveness in multiple scenarios. However, KBQA remains challenging, especially regarding coverage and generalization settings. This is due to two main factors: i) understanding the semantics of both questions and relevant knowledge from the KB; ii) generating executable logical forms with both semantic and syntactic correctness. In this paper, we present a new KBQA model, TIARA, which addresses those issues by applying multi-grained retrieval to help the PLM focus on the most relevant KB context, viz., entities, exemplary logical forms, and schema items. Moreover, constrained decoding is used to control the output space and reduce generation errors. Experiments over important benchmarks demonstrate the effectiveness of our approach. TIARA outperforms previous SOTA, including those using PLMs or oracle entity annotations, by at least 4.1 and 1.1 F1 points on GrailQA and WebQuestionsSP, respectively. Specifically on GrailQA, TIARA outperforms previous models in all categories, with an improvement of 4.7 F1 points in zero-shot generalization.

## introduction knowledge base question answering (kbqa) has established itself as an important and promising research area as it greatly helps the accessibility and usability of existing large-scale knowledge bases (kbs). such kbs contain abundant facts in a structured form, which can not only be accessed, but also reasoned over. kbqa bypasses the need for users to learn complex and burdensome formal query languages, and it allows users to leverage knowledge using natural language. while recent kbqa efforts have achieved interesting results, most previous studies target or assume an underlying strong correspondence between the distributions of schema items in test questions and training data, i.e., the i.i.d. assumption. however, this assumption does not hold, especially for large-scale kbs, which typically contain very large numbers of entities and schema items (classes and relations). 2 this extensive space is a critical challenge as it both allows for a myriad of novel compositions of schema items @xcite , i.e., requires compositional generalization. it also immensely increases the likelihood of user queries, including previously unseen items or domains @xcite , i.e., requires strong zero-shot generalization. meanwhile, pre-trained language models (plms), such as t5 @xcite and @xcite , have demonstrated notable success in many natural language processing (nlp) scenarios @xcite . many times they even show strong generalization capabilities. inspired by the progress of plms on unstructured text-to-text tasks, researchers have explored semantic parsing of natural language to logical form utilizing plms @xcite to tackle the generalization challenges and improve system robustness. however, differently from unstructured data in the typical plm pretraining phase, kbs represent rich semantics and complex structures, which lead to two challenges in using plms for kbqa: i) kb grounding: given a kb, how to understand the semantics of both the question and the relevant knowledge from the kb? ii) logical form generation: how to make sure syntax and semantics of generated logical forms conform to the kb specification and are executable (i.e., guarantee correctness)? kb grounding requires linking a question to relevant kb items, but the large size of kbs makes it question what napa county wine is 13.9 percent alcohol by volume? ## method to demonstrate how comprehensive kb context enhances the robustness of retrieval, a key component of our method, we retrieve kb contexts in a multi-grained manner and then generate target logical forms with constrained decoding, as shown in figure 1 . we introduce multi-grained retrieval in section 3.2, target logical form generation in section 3.3, and describe constrained decoding in section 3.4. ## conclusion we present tiara to empower plms on qa over large-scale kbs. multi-grained retrieval transcends insufficient exemplary logical forms and isolated schema items, and its results provide both semantic and syntactic contexts for a given question. in systematic stage-by-stage analysis, we show that: i) constrained decoding prunes invalid search and reduces generation errors (e.g., non-existent schema and illegal operators); ii) entity linking improvements address a key bottleneck for zero-shot generalization; iii) schema retrieval with flexible scope is an essential semantic supplement to sur though we have started exploiting the potential of plms, there is still a gap between the pretraining of plms on natural language and the downstream semantic parsing task. in the future, it is worth exploring how to build a bridge between unstructured natural language and structured kb in the pre-training phase.