FC - KBQA : A Fine-to-Coarse Composition Framework for Knowledge Base Question Answering (2023)

Abstract: The generalization problem on KBQA has drawn considerable attention. Existing research suffers from the generalization issue brought by the entanglement in the coarse-grained modeling of the logical expression, or inexecutability issues due to the fine-grained modeling of disconnected classes and relations in real KBs. We propose a Fine-to-Coarse Composition framework for KBQA (FC-KBQA) to both ensure the generalization ability and executability of the logical expression. The main idea of FC-KBQA is to extract relevant fine-grained knowledge components from KB and reformulate them into middle-grained knowledge pairs for generating the final logical expressions. FC-KBQA derives new state-of-the-art performance on GrailQA and WebQSP, and runs 4 times faster than the baseline. Our code is now available at GitHub https://github.com/RUCKBReasoning/FC-KBQA .

## introduction question answering over knowledge bases (kbqa) aims to provide a user-friendly way to access largescale knowledge bases (kbs) by natural language questions. existing kbqa methods @xcite can be roughly categorized into retrievalbased and semantic-parsing (sp) based methods. the former @xcite @xcite directly scores the relevance between the question and answer candidates, thus it is difficult to resolve the complex questions. on the contrary, some kbqa approaches, such as @xcite @xcite , are based on semantic parsing (denoted as sp-based), which can address complex questions and achieve promising results on i.i.d. datasets. sp-based methods first translate the questions into logical expressions such as sparql and then execute them against kb to yield answers. figure 2 : results of the pilot study. the coarse-grained method directly matches the question with the logical expression (i.e., the composition of components), while the fine-grained method matches the question with each component candidate and then composes them to derive the logical expression. the exact match accuracy of logical expressions on compositional generalization test data and zero-shot generalization test data is shown on the right of the figure. as illustrated in figure 1 , a logical expression consists of multiple components such as classes and relations. most existing sp-based approaches fail with logical expressions that contain unseen compositions of components (called compositional generalization) or unseen components (called zero-shot generalization). to address the above problem, grailqa-rank @xcite proposes a bert-based rank model to match the given question with each logical expression candidate, which leverages the generalization abilities of the pre-trained language models. on top of that, rng-kbqa @xcite further uses a pre-trained generation model, which takes top-5 ranked logical expressions as the additional input beyond the question to generate the target logical expression. behind these mainstream models, a logical expression is viewed as an inseparable unit during modeling. actually, logical expressions are coarse-grained because they can be decomposed into relatively fine-grained components including relations, classes, entities, and logical skeletons (see examples in figure 3 ). such coarse-grained modeling entangles representations of fine-grained components, thereby overfitting the seen compositions during the training process, which weakens the model's compositional generalization ability. meanwhile, even though pre-trained language models can deal with zero-shot components to some extent, compositional overfit reduces their ability to identify individual unseen components with zero-shot generalization. to demonstrate the above idea, we perform a pilot study (cf. the detailed settings in section 4.1) with two preliminary experiments: one calculates the similarity score between a question and each coarse-grained logical expression to obtain the most relevant one, and the other searches the most relevant fine-grained components to form the final logical expression of a question. we observe that the fine-grained modeling derives more accurate logical expressions on both the compositional task and zero-shot task (cf. figure 2 ). it could be explained that fine-grained modeling focuses exclusively on each component, avoiding overfitting of seen compositions in the training data. although some studies attempt to leverage fine-grained components, they only consider partial fine-grained components such as relations, classes, and entities @xcite , or suffer from inexecutability due to disconnected fine-grained components in real kbs @xcite . thus, to both ensure the generalization ability and executability of logical expressions, we propose a fine-to-coarse composition framework for kbqa (fc-kbqa), which contains three submodules. the overview of our model is shown in figure 4 . the first module is fine-grained component detection, which detects all kinds of finegrained component candidates from freebase by their semantic similarities with the question. such component detection guarantees the generalization ability in both compositional and zero-shot tasks. the second module is the middle-grained compo-nent constraint, which efficiently prunes and composes the fine-grained component candidates by ensuring the components' connectivity in the kb. the final module is the coarse-grained component composition, which employs a seq-to-seq generation model to generate the executable coarse-grained logical expression. in addition to encode the finegrained components, the middle-grained components are also encoded to enhance the model's reasoning capacity, so as to improve the executability of the generated logical expression. in contrast to previous work @xcite @xcite that only uses the knowledge constraints to guide the decoding process, we emphasize injecting them into the encoding process, because the encoder which learns bidirectional context could better suit natural language understanding @xcite . we conduct extensive experiments on widely used grailqa, webqsp, and cwq datasets. grailqa @xcite is a kbqa benchmark focusing on generalization problems. fc-kbqa derives new state-of-the-art performance on grailqa-dev (+7.6% f1 gain and +7.0% em gain respectively). meanwhile, fc-kbqa also obtains good performance on webqsp and cwq. moreover, fc-kbqa runs 4 times faster than the state-of-the-art baseline rng-kbqa. the ablation studies demonstrate the effect of our middlegrained encoding strategy. ## related work coarse-grained sp-based methods. many efforts are paid to solve generalization problems on sp-based kbqa. some approaches, such as @xcite , use a rank-based model that takes advantage of a coarse-level match between the question and the logical expressions or query graphs. they first enumerate numerous query graph candidates based on kbs and then they rank them according to how relevant they are to the question. another line of approaches, in addition to the rank-based ones, makes use of a generation model. kqapro @xcite leverages bart to directly convert questions into logical expressions. additionally, rng-kbqa @xcite further injects top-k ranked logical expressions as an additional input to the question. cbr-kbqa @xcite injects analogous questions and their corresponding logical expressions from the training data to increase the generalization. all of the aforementioned methods are pure coarse-level frameworks that treat each coarse-grained logical expression as a separate unit. ## problem definition knowledge base (kb). a kb is comprised by ontology {(c × r × c)} and relational facts {(e × r × (e ∪ c))}, where r, c, and e denote relation set, class set, and entity set respectively. notably, we consider literal as a special type of entity. specifically, an ontology triple (c d , r, c r ) consists of a relation @xmath0, a domain class c d which denotes the class of the subject entities, and a range class c r which denotes the class of the object entities. each class has multiple entities, thus an ontology triplet can be instantiated as several relational facts. for example, both (e 1 , r, e 2 ) and (e 3 , r, e 4 ) correspond to (c d , r, c r ), where e 1 , e 3 ∈ c d and e 2 , e 4 ∈ c r . figure 3 illustrates a kb subgraph. sp-based kbqa. given a natural question q, kbqa models aim to find a set of entities denoted by a ⊆ e from kb as the answers to q. instead of directly predicting a, sp-based kbqa models translate q to an executable logical expression denoted by s such as sparql, lambda-dcs @xcite , query graph (lan and jiang, 2020), and s-expression @xcite . we select s-expression as our used logical expression since it could provide a good trade-off on compactness, compositionality, and readability @xcite . the logical skeleton of an s-expression can be derived by removing all the relations, classes, and entities in the expression and only keeping function operators and parentheses. specifically, we replace relations, classes, entities, literals with special tokens "<@xmath1, "<@xmath2, "<@xmath3, "<@xmath4. figure 3 shows an executable logical expression on the kb and its corresponding logical skeleton. we unitedly name the relations, classes, entities, and logical skeleton in an s-expression as the fine-grained component, while the complete s-expression is the coarse-grained logical expression. ## conclusion this paper proposes fc-kbqa, a fine-to-coarse composition framework for kbqa. the core idea behind it is to solve the entanglement issue of mainstream coarse-grained modeling by the fine-grained modeling, and further improve the executability of logical expression by reformulating the finegrained knowledge into middle-grained knowledge pairs. benefiting from this, fc-kbqa achieves new state-of-the-art performance and efficiency on the compositional and zero-shot generalization kbqa tasks. this fine-to-coarse framework with middle-grained knowledge injection could be inspiring for generalization on other nlp tasks.