How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering (2024)

Abstract: Knowledge Base Question Answering (KBQA) aims to answer natural language questions based on facts in knowledge bases. A typical approach to KBQA is semantic parsing, which translates a question into an executable logical form in a formal language. Recent works leverage the capabilities of large language models (LLMs) for logical form generation to improve performance. However, although it is validated that LLMs are capable of solving some KBQA problems, there has been little discussion on the differences in LLMs’ proficiency in formal languages used in semantic parsing. In this work, we propose to evaluate the understanding and generation ability of LLMs to deal with differently structured logical forms by examining the inter-conversion of natural and formal language through in-context learning of LLMs. Extensive experiments with models of different sizes show that state-of-the-art LLMs can understand formal languages as well as humans, but generating correct logical forms given a few examples remains a challenge. Most importantly, our results also indicate that LLMs exhibit considerable sensitivity. In general, the formal language with a lower formalization level, i.e., the more similar it is to natural language, is more friendly to LLMs. Code and data can be found at https://github.com/Matthewlliu/structure_probe.

## introduction knowledge base question answering (kbqa) is a challenging natural language processing (nlp) task to answer natural language questions based on fact triples stored in the knowledge base (kb), such as wikidata @xcite and freebase @xcite . in recent years, a typical paradigm of kbqa methods is semantic parsing @xcite @xcite , where natural language questions (nlqs) are translated into their corresponding structured logical forms (lfs), such as kopl @xcite , sparql @xcite or lambda dcs @xcite . the logical forms are capable of expressing multiple reasoning operations such as multi-hop inference and quantitative comparison, and can be executed on kbs to get accurate answers. the recent advancements of large language models (llms) @xcite have led to significant attention on utilizing llms for kbqa. previous works have validated the ability of llms to memorize, understand and apply knowledge for reasoning @xcite , and serve as agents to solve kbqa problem @xcite . additionally, others introduced many techniques to improve the performance, such as in-context learning @xcite , chain-of-thought @xcite , and instruction-tuning @xcite . however, there has been little discussion on the differences in the proficiency of llms in different formal languages that used as paring targets in semantic parsing. it is proved that llms can do well on programming language such as python @xcite with sufficient data in the pretraining, but it remains intriguing that how well do llms master other formal languages without extra data and further fine-tuning. by examining the proficiency of llms in formal languages, we can gain a better understanding of llms' upper limits. it also would be advantageous to choose appropriate models and formal languages for specific scenarios if different llms have varying levels of proficiency in different formal languages. in this paper, we propose to evaluate the inherent understanding and generation ability of the formal language in the original llms without additional fine-tuning. we define two evaluation tasks based on sub-tasks of kbqa: 1) formal language understanding, which aims to translate a lf into its corresponding nlq. the translation process can be considered as the model interpreting the provided lfs in natural language, demonstrating llms' understanding ability of formal language; 2) formal language generation, which aims to correctly convert a nlq into its corresponding lf, requiring the model to not only understand but also generate lfs, demonstrating its capability in generation. with respect to the formal languages for evaluation, according to the varying levels of formalization (may be broadly understood as the dissimilarity to natural language, i.e. the higher the level of formalization, the less similar it is to natural language) and different logical structures (e.g. tree, graph or chain), we choose lambda dcs @xcite , sparql @xcite , and kopl @xcite as representative formal languages, which are commonly used for knowledge based question answering research @xcite @xcite . for the generation methods, to reflect the inherent proficiency of llms, we combine the incontext learning ability @xcite of llms and chain-of-thought generation @xcite for both evaluation tasks, where the desired outputs are generated conditioned on the input along with a few demonstration pairs of nlqs and lfs carefully selected from a seed dataset. for demonstration selection, to ensure that the logical structure of the examples should be as similar as possible to the target, we carefully design a greedy search algorithm based on the minimum edit distance to solve a maximum coverage problem. for the quality evaluation of the generated nlq, to avoid the inaccuracy of automatic metrics and the labor-intensive human evaluation, we propose a contrastive evaluation approach. this involves separately training a semantic parser using llmgenerated data and comparing them with parser trained using manually labeled data. by comparing the performance of the parsers, we can assess the differences in quality between the data generated by the llms and the manually labeled data. our findings indicate that llms have approached the human annotators in the task of formal language understanding that generate natural language questions from logical forms. however, conversely, challenges still exist in the task of formal language generating if only a few examples are given. importantly, we observe that models exhibit the sensitivity to different logical forms. overall, the lower the level of formalization (the similar it is to natural language), the easier it is for models to understand and generate. in conclusion, this study examines the proficiency of llms in formal language understanding and generation, and helps to provide valuable insights for llms-based reasoning approaches. since the logical form can facilitate communication between the model and the kb, the latter category usually out-performance the former and also enjoy a better interpretability. therefore, our work sets out to explore the role of different formal languages in kbqa in the era of llms. knowledge base question generation. early kbqg works mainly utilize templates to generate questions. these approaches @xcite @xcite usually leverage search engines or manual design to obtain the templates. however, template-based methods are restricted by heavy manual labor. with the advancement of deep neural networks, neuralbased methods @xcite are proposed to generate questions automatically by the sequence-to-sequence model. since the semantic information in the structure of the entity graph is considered useful for kbqg, graph neural models @xcite @xcite are applied to generate multi-hop questions based on subgraphs. most recently, liang et al. utilize llms with its few-shot learning ability and chain-of-thought reasoning, proposing to generate question directly from its formal language. in the formal language understanding task of our work, we are inspired by the idea of liang et al. and customize our own chain-of-thought methods. ## evaluation task definition as shown in figure 1 , we define two probing tasks, namely the formal language understanding and formal language generation. in this section, we introduce the formalized definitions of these two tasks and how to assess llms' performance. ## formal language and datasets in this section, we will introduce the details of the formal languages and datasets tested in this work. as mentioned in section 1, we choose three representative formal languages according to the varying levels of formalization and different logical structure, and they are lambda dcs, sparql, and kopl. some examples are shown in figure 2 . lambda dcs is a tree-structured programming language developed from lambda calculus, similar to church and s-expression. lambda dcs removes the explicit variables in lambda calculus, making it similar to dependency-based compositional semantics @xcite . for this language, we use overnight dataset @xcite , which contains over 13,000 data examples in eight domains extracted from freebase. we follow the standard split used in wang et al.. sparql is a popular query language and it provides a standardized way for users to search and retrieve information stored in rdf databases and other linked open data 1 . the sparql describes the relations between entities using triples in the form of a graph structure. for this language, we language models seed set exemplar ø lf: ---ø nlq: --- ## implementation as mentioned above, we mainly leverage the incontext learning (icl) ability of llms to generate the output for the probing task. the demonstration selection is considered as the most critical part of this method. in this work, we adopt the principle to search most similar examples to the target l, and decently order the examples by the similarity @xcite in the prompt. ## experiment setup we introduce a range of popular language models that have been extensively studied in our experiments (6.1) as long as the semantic parsing models we use to evaluate the performance of the understanding task (3). ## results and analyses we first present the main result of the formal language understanding and generation in in the left blue section of understanding task, the figures are the absolute performance of the evaluation parser trained on training sets that generated by different models. the retrieved examples of the input prompt of icl is 3 for all models in the understanding task. the right green section presents the semantic parsing result of the models, where the retrieved examples are as many as the input context can take so as to improve the result. to cut down computation overehead, the test sets are randomly sampled subsets of 300, 120, 240 examples from the test sets of kqa pro, grailqa, and overnight, respectively. the parsing performance of kopl and lambda dcs are measured by answers' accuracy, and the sparql performance are measure by answers' f1 score. note that the human's performance is not applicable here, but we can compare it to the baseline results of understanding task. also, we only test the model over 1b because the small models perform poorly with meaningless results. then we present the conclusions and findings by analyzing them along with other ablation experiments. more detailed results for some dataset can be found in appendix. ## conclusion in this work, we evaluate the proficiency of different llms in understanding and generating different formal languages. our observations suggest that the ability of llms to generate structured semantics is notably inferior to their ability to understand it. more importantly, llms demonstrate the sensitivity to different formal languages. aligning with our intuition, we discover that the choice of formal language and knowledge base can exert significant influence on models' performance. in our experiment, models performing on kopl yields the best results on nearly all experiments. we believe it is because kopl employs expressions that are more similar to natural language while preserving the structure and modularity. however, sparql and lambda dcs face challenges in grounding entities to the knowledge base for their level of formalization is too high. as a result, kopl proves to be the most llms-friendly among the formal languages that we investigate in this work. in general, we want to point out that the formal language plays an important role in enhancing the power of llms. a formal language can be used as a medium between llms and the knowledge base, so that llms can use the knowledge base as a tool to enhance the performance of qa and reasoning tasks. on the other hand, the selection of a more model-friendly formal language, one that closely resembles the natural language in which models excel, should be prioritized.