Seeing the wood for the trees: a contrastive regularization method for the low-resource Knowledge Base Question Answering (2022)

Abstract: Given a context knowledge base (KB) and a corresponding question, the Knowledge Base Question Answering task aims to retrieve correct answer entities from this KB. Despite sophisticated retrieval algorithms, the impact of the low-resource (incomplete) KB is not fully exploited, where contributing components (. key entities and/or relations) may be absent for question answering. To effectively address this problem, we propose a contrastive regularization based method, which is motivated by the learn-by-analogy capability from human readers. Specifically, the proposed work includes two major modules: the knowledge extension and sMoCo module. The former aims at exploiting the latent knowledge from the context KB and generating auxiliary information in the form of question-answer pairs. The later module utilizes those additional pairs and applies the contrastive regularization to learn informative representations, that making hard positive pairs attracted and hard negative pairs separated. Empirically, we achieved the state-of-the-art performance on the WebQuestionsSP dataset and the effectiveness of proposed modules is also evaluated.

## introduction the task of knowledge base question answering (kbqa) refers to answering a question given a background knowledge base (kb). a large number of studies on kbqa can be cast into two mainstream categories: semantic parsing (sp) and information retrieval (ir) based. the former focuses on parsing questions into symbolic logic forms, such as query graph @xcite and skeleton grammar @xcite , before identifying final answer(s) from the kb. the ir approach, on the other hand, aims to perform semantic matching between topic entities from questions and candidate answers within the kb @xcite @xcite . this paper is on a novel method for information retrieval based kbqa. approaches to ir-based kbqa usually follow a three-step process, including question analysis, subgraph reasoning and answer matching. at first, the question analysis is to understand reasoning instructions behind questions for extracting topic entities and involved relations, etc. the second step is performed to retrieve relevant entities and relations (as subgraphs of the context kb) according to reasoning instructions, and further formulate candidate answers. the last step is to identify the best answer by estimating and ranking the semantic-relationship matching score between the given question and candidates. however, existing ir-based matching pays insufficient attention to the low resource (incompleteness) nature of kb, where contributing components (i.e. neighboring entities, key relations and/or reasoning path) may be absent. the limit or lack of enough information poses challenges for the subsequent effective question answering. some work have been proposed to utilize auxiliary information, such as extra question-related texts @xcite and pre-trained kb embeddings @xcite , which unfortunately could introduce noisy and misleading facts, not to mention the extra computational cost. notably, when human performs qa reasoning, one could infer other cases from one similar instance. even with a new question, experienced readers could still make a guess using similar concepts or facts from their current knowledge, and potentially discriminate candidate answers from different aspects. for instance, given a question of "what state did al gore represent?", one could infer from his working experience (i.e. the tennessean (news)) and/or the graduate university (i.e. vanderbilt university), to further predict the correct answer of tennessee. inspired from the aforementioned learn-byanalogy process, we propose a novel kbqa framework as illustrated in fig. 1 , which includes two major modules: knowledge extension and smoco. the former aims to expand knowledge from the existing kb to discover latent information. that is, hidden knowledge is exploited and manipulated in a form of question-answering pairs via making use of existing relations and entities (simulating human readers to use similar concepts/facts). the smoco adopts the contrastive learning mechanism, with the ultimate aim of capturing discriminative features from correct and misleading question-answering pairs (simulating human's inference skill). furthermore, different from traditional contrast methods, the proposed smoco is particularly designed to utilize hard positive and negative pairs, which further improves the model generalizability. 1 our contributions are summarized as follows: another line of studies aims to retrieve answers by following a more end-to-end training style, that is, to learn representations of the target question and candidate answers. kdreader from @xcite performed an attention-based fusion to combine the question and answer features. pullnet @xcite employed the question-related content as a guideline to extract supporting components (entities and relations from kb). the work of @xcite utilized the pre-trained kb embeddings. in addition, han et al. proposed a hypergraph-based reasoning strategy with dynamic relation and entity embeddings @xcite . similarly, rechypernet @xcite applied recursive hypergraphs to form groups (relations and entities with similar semantic) in the kb. with well-represented features, a matching step is usually followed to identify/rank the best candidate(s). despite some promising results from the afore-mentioned feature learning and matching, less work has been put forward to explore underlying/hidden knowledge from the context kb. we argue that making full use of those existing but latent knowledge is beneficial to the subsequent qa task. ## methodology the illustration of our framework is shown in fig 2 . the kbqa task has been formulated as searching for optimal entities, given the question q and the external kb (i.e. a set of triples (e h , r, e t ) representing head entity, semantic relationship and tail entity, respectively). ## analysis we present the sensitivity analysis hereafter to justify the choice of the screening parameter (or λ q from eq. ( 3 )). note that the traditional contrastive loss, for the i-th anchor, is defined as follows denoting s + @xmath0,+ i ,z i and s - @xmath1,z i , eq. ( 6 ) can be written more concisely as we then quantify the screened proportion @xmath2, 1) for negative samples, and note that by removing a given proportion of negative samples, it is equivalent to reduce s - i to si by a suitable value of such that s- the perturbed loss is caused by and lc (0|r i ) recovers the original loss in eq. ( 7 ). therefore the sensitivity of the function in eq. ( 8 ) at ( = 0) determines the change to the loss by introducing screening. the first order taylor expansion shows that: remark 1. the perturbation (or the difference in loss aroused by screening) is approximately -r i r i +1 , which is an apparent reduction to the original loss l c as all variables in eq. ( 9 ) are positive. furthermore, it is worth noting that normalized similarities are between [-1, 1], that is, z h,+ i , z @xmath3, and z - j , z @xmath4, and hence remark 2. the perturbation indicates that that if we screen out a small proportion of negative samples, say 1λ q (or ), the function loss is reduced by approximately that much. in other words, the sample screening connects to the loss reduction directly. although we discussed only ideal situation where the contrast reaches maximum, similar result exists in expectation sense as well because a large value of |q| outweighs less contrast in negative and positive similarities. notably, the above perturbation analysis is performed at 0 requiring to be small or even close to zero (@xmath5.01, 0.1]), and hence satisfies this condition.the loss function change caused by screening will eventually transfer to gradients to the model parameters. our results indicates the scaling factor in front of the final gradients. nonetheless, when r i is too large, it saturates the loss function to the "plateau" stage where gradients are small and hence the aforementioned analysis results hold. ## conclusion we present a novel kbqa model that particularly tackles the low-resource (incompleteness) nature of the context knowledge bases (kbs). the proposed model is characterized by a knowledge extension and a smoco module, that is motivated by the learn-by-analogy capability of human readers. precisely, the former extends existing knowledge via producing additional question-answer pairs, which are further utilized by the smoco module. the latter appropriately learns informative representations that grouping hard positives and pushing away hard negatives. empirically, in comparison to existing approaches, the proposed algorithm produces the state-of-the-art performance on the webques-tionssp benchmark, in particular with the extreme incomplete kbs. in future work, we will extend the proposed idea to explore more contrastive behavior of utilizing hard positives and negatives. more importantly, smoco is agnostic to the downstream tasks, i.e., we could incorporate it into other applications.