Improving Complex Knowledge Base Question Answering via Question-to-Action and Question-to-Question Alignment (2022)

Abstract: Complex knowledge base question answering can be achieved by converting questions into sequences of predefined actions. However, there is a significant semantic and structural gap between natural language and action sequences, which makes this conversion difficult. In this paper, we introduce an alignment-enhanced complex question answering framework, called ALCQA, which mitigates this gap through question-to-action alignment and question-to-question alignment. We train a question rewriting model to align the question and each action, and utilize a pretrained language model to implicitly align the question and KG artifacts. Moreover, considering that similar questions correspond to similar action sequences, we retrieve top-k similar question-answer pairs at the inference stage through question-to-question alignment and propose a novel reward-guided action sequence selection strategy to select from candidate action sequences. We conduct experiments on CQA and WQSP datasets, and the results show that our approach outperforms state-of-the-art methods and obtains a 9.88% improvements in the F1 metric on CQA dataset. Our source code is available at https://github.com/TTTTTTTTy/ALCQA .

## introduction complex knowledge base question answering (cqa) aims to answer various natural language questions with a large-scale knowledge graph. compared to simple questions with single or multihop of relations, complex questions have more kinds of answer types such as numeric or boolean types and require more kinds of aggregation operations like min/max or intersection/union to yield answers. semantic parsing approaches typically map questions to intermediate logical forms such as query graphs @xcite @xcite @xcite , and further transform them into queries like sparql query language. recently, many works @xcite @xcite predefine a collection of functions with constrained argument types and represent the intermediate logical form as a sequence of actions that can be generated using a seq2seq model. sequencebased methods are natural to accomplish more complex operations by simply expanding the function set, thus making some logically complex questions answerable while they're difficult to answer using query graphs. the seq2seq model has been widely used and achieved good results on many text generation tasks, such as machine translation, text summarization and style transfer. in these tasks, the source and the target sequence are both natural language texts and thus share some low-level features. however, semantic parsing aims to transform unstructured texts into structured logical forms, which requires a difficult alignment between them. this problem becomes more serious when the complexity of the question rises. some works propose to solve this problem by modelling the hierarchical structure of logical forms. @xcite introduces a sequence-to-tree model with an attention mechanism. @xcite proposes to decode a sketch of the logical forms which contain a set of functions at first and then decode low-level details like arguments. @xcite iteratively segments a span from the question by a segmentation model and parses it using a base parser until the whole query is parsed. @xcite uses a shift-reduce algorithm to obtain token sequences instead of predicting the start and end positions of the span. however, most of these works require intermediate logical forms or sub-questions to train models, which are usually difficult to obtain. @xcite and @xcite propose to pretrain a base parser firstly, and then search good segments figure 1 : an overview of the proposed approach. the question is first converted into a more structured form, then multiple candidate action sequences are generated by the seq2seq model, and finally the candidate action sequences are scored based on similar question-answer pairs. that predicted sub logical forms are part of or can be composed into the golden meaning representation. they don't necessarily require training pairs but have the limitation that decomposed utterances are continuous segments of the original question. in this paper, we propose a novel framework to boost the alignment between unstructured text and structured logical forms. we decompose the semantic parsing task into three stages: question rewriting, candidate action sequences generation and action sequence selection. in the question rewriting stage, we utilize a question rewriting model to explicitly transform a query into a set of utterances, each corresponding to a single action, thus reducing the complexity of the question. we propose a two-phase training method to train the rewriting model on the lack of training pairs. in the candidates generation stage, we build a seq2seq model to generate logical forms with beam search algorithm and consider kg artifacts like entities as candidate vocabularies in the decoding stage. to further align the question and action sequence, we concatenate a question and a kg artifact as input and encode it using a pretrained language model (plm) like bert @xcite . the cross attention mechanism of plm can effectively align between the question and kg artifacts implicitly, which makes decoding easier. moreover, we innovatively propose to improve complex knowledge base question answering via question-to-question alignment. motivated by the phenomenon that the more similar two questions are, the more similar their corresponding action sequences will be, we build a memory consisting of question-answer pairs and retrieve a set of question-answer pairs as the support set based on the similarity with the current question during action sequence selection phase. we then propose a reward-guided selection strategy that scores each candidate action sequence according to the support set. our main contributions are as follow: ## related work semantic parsing is the task of translating natural language utterances into executable meaning representations. recent semantic parsing based kbqa methods can be categorized as graph-based @xcite @xcite @xcite and sequence-based @xcite @xcite @xcite . graph-based methods build a query question does janko kroner have location of birth at peraia, pella and povazska bystrica ? w/o module select(janko kroner, place of birth, administrative territorial entity) # bool(povazska bystrica) # bool(povazska bystrica) prob: 0.6085 selection score: 0.7333 w/ module select(janko kroner, place of birth, administrative territorial entity) # bool(peraia, pella) # bool(povazska bystrica) prob: 0.3519 selection score: 1.0000 compared to graph-based methods, sequencebased methods can generate logical forms directly using the seq2seq model, which is easier to implement and can handle more question categories by simply expanding the set of action functions. however, the semantic and structural gap between natural language utterances and action sequences leads to poor performance on translation. ## conclusion in this paper, we propose an alignment-enhanced complex question answering framework, which reduces the semantic and structural gap between question and action sequence by question-to-action and question-to-question alignment. we train a question rewriting model to align question and subaction sequence in the absence of training data and employ a pretrained language model to align the question and action arguments implicitly. moreover, we utilize similar questions to help select the correct action sequence from multiple candidates. experiments show that our framework achieves state-of-the-art on the cqa dataset and performs well on various complex question categories. in the future, how to better align questions with logical forms will be considered.