V is K o P : Visual Knowledge oriented Programming for Interactive Knowledge Base Question Answering (2023)

Abstract: We present Visual Knowledge oriented Programming platform ( VisKoP ), a knowledge base question answering (KBQA) system that integrates human into the loop to edit and debug the knowledge base (KB) queries. VisKoP not only provides a neural program induction module, which converts natural language questions into knowledge oriented program language (KoPL), but also maps KoPL programs into graphical elements. KoPL programs can be edited with simple graphical operators, such as ”dragging” to add knowledge operators and ”slot filling” to designate operator arguments. Moreover, VisKoP provides auto-completion for its knowledge base schema and users can easily debug the KoPL program by checking its intermediate results. To facilitate the practical KBQA on a million-entity-level KB, we design a highly efficient KoPL execution engine for the back-end. Experiment results show that VisKoP is highly efficient and user interaction can fix a large portion of wrong KoPL programs to acquire the correct answer. The VisKoP online demo, highly efficient KoPL engine, and screencast video are now publicly available.

## introduction knowledge base question answering (kbqa) aims to find answers to factoid questions with an external knowledge base (kb). researchers have fully explored the kbqa @xcite task and the most common solution is to convert userposed natural language questions into kb query programs via semantic parsing and then give a final result by executing queries on the kb, such as sparql @xcite , λ-dcs @xcite , 2021), and kopl @xcite . recently, many kbqa systems @xcite @xcite that implement those advanced semantic parsing algorithms in an online environment, have been developed. although semantic parsing methods have gained considerable achievement, there is still no guarantee to precisely parse every user-posed question given the limitations of current machine learning techniques. figure 1 demonstrates an example of semantic parsing results by the state-of-the-art kbqa model @xcite . as the posed question does not follow the identical distribution of the training dataset adopted by the semantic parsing model @xcite , it is falsely parsed with the or operator, which should be an and operator, causing the structure error of the kb query. meanwhile, it is extremely difficult for the semantic parsing model to correctly predict all the knowledge elements in a question @xcite . as shown in the example, the "shares border with" relation is falsely predicted as a "statement is subject of" relation, causing an argument error in the kb query. however, existing kbqa systems do not provide easy access to manipulating the kb query programs and thus users cannot intervene in the query execution. fortunately, several program-based kb query languages for complex reasoning consisting of modularized operators have come up, making kbqa easy to visualize @xcite . with applicable visual representation of kb queries, intended users are capable of identifying errors in the programs generated by semantic parsing and correct them. based on these observations, we raise a natural question: how to design a visualized kbqa system that eases users to inspect and debug those kb query programs? presented system. we demonstrate visual knowledge oriented programming (viskop) platform, an interactive, visualized and program-based kbqa system. viskop provides an interactive knowledge oriented programming environment, allowing users to monitor and debug the kb queries with graphical operators. in comparison with existing kbqa systems, viskop is easier to use due to its following characteristics: we conduct user study and find that with the help of the visualized programming interface, users can find the correct answer in an average 110.6 seconds, which alleviates the problem caused by error-prone semantic parsing algorithms. meanwhile, our efficiency study shows that the execution engine is significantly faster than the original kopl engine and virtuoso by 16× and 5×, respectively. contributions. (1) we design a visualized knowledge oriented programming platform for kbqa, which integrates human into the loop to write and debug kb queries. (2) we implement a high performing kopl execution engine that scales kopl to an up-to-date million-entity-level kb. the development and deployment of viskop validates the effectiveness of allowing questioners to monitor the error in the kb queries. the visual programming platform provides external human guidance on the neural program induction model, and potentially improves the robustness the system. ## the viskop platform the implementation of our viskop platform focuses on 4 designing principles: i. graphical element visualization: userposed questions should be parsed into the kopl program, and shown as graphical elements. ii. interactive programming: the system needs to enable users to edit and correct the kopl program with knowledge schema auto-completion and intermediate results demonstration. iii. highly efficient execution: the system should support large scale kbs for practical usage with low execution latency. iv. transparent execution: the execution footprint of each operator should be preserved for inspection within interactive programming. in particular, the first two principles are undertaken by the interactive programming interface in the front-end and the last two principles are undertaken by the highly efficient kopl program execution engine in the back-end. the overall architecture of viskop is shown in figure 2 . the implemented viskop is deployed as an openly available website 1 . the highly efficient kopl execution engine is also provided as an opensource python extension toolkit 2 . ## 4 how many countries share borders with both germany and france? parse relation: shares borders with relate direction: forward check intermediate result relation: statement is subject of relate direction: forward check intermediate result entity name: france find check intermediate result entity name: germany find check intermediate result concept: country filterconcept check intermediate result concept: country filterconcept check intermediate result count check intermediate result or check intermediate result and check intermediate result shares borders with code run run on wikidata select a function clear all figure 3 : screenshot of the interactive programming interface of viskop. when user tries to parse "how many countries share borders with both germany and france?", the semantic parsing algorithm falsely predict the or operator, and one of the argument inputs of the relate operator. this further results in the wrong answer "17". we marked this errors in the red box, and put the correct graphical elements in the nearby green box. relation: bord relate direction: forward check intermediate result shares border with kingdom of the netherlands belgium luxembourg switzerland intermediate result cancel ok the intermediate result of each knowledge operator is a powerful tool to diagnose the kopl program. it also serves as an interpretation to the question's answer. by expanding the intermediate result of the and operator, as shown in the right part of figure 4 , we are able to know which countries are taken into account. ## evaluation we evaluate the execution efficiency of the backend kopl engine. we also perform user study and case study to examine the limitations of viskop. ## related works in general, kbqa methods can be grouped into two categories: 1) semantic parsing @xcite @xcite @xcite , which translates natural language questions into logical forms, whose execution on the kb achieves the answer; 2) information retrieval @xcite @xcite @xcite , which ranks the entities from the retrieved question-specific sub-kb to get the answer. our viskop falls into the semantic parsing category. specifically, viskop translates a question into the multi-step program, pertaining to the neural program induction (npi) paradigm @xcite @xcite @xcite . the main challenge of npi is that questionprogram parallel data are expensive to obtain and the program's huge search space makes the learning challenging. existing works tackle this issue only by learning from question-answer pairs with various reinforcement learning techniques @xcite or synthesizing questionprogram data to alleviate the data scarcity problem @xcite . in this paper, our viskop proposes a different solution to this task by integrating humans into the program induction loop, providing external human guidance to program induction model, and potentially improving the system robustness. compared with other kbqa systems, including retrack @xcite , sempre @xcite , tranx (yin and neubig, 2018), dtqa @xcite , our viskop is the first to enable users to interact with the system via a visual platform and intermediate results checking. ## conclusion and future work we demonstrate viskop, a kbqa platform that allows users to monitor, edit, and debug kb queries. viskop is also accompanied with a highly efficient engine that scales kopl execution to a millionentity-level kb. in the future, it is intriguing to allow users to customize the kb. it is also important to provide guidance for users to recognize the true knowledge elements in the large scale kb.