Few-Shot Natural Language to First-Order Logic Translation via Code Generation (2025)

Abstract: Translation of natural language to first-order logical formula (NL-FOL) has recently gained significant attention for its critical role in logic-based NLP applications. Some studies attempt to utilize pretrained language models in a sequence-to-sequence manner for the NL-FOL task. However, these methods encounter challenges such as (1) inconsistency between the training and inference phases and (2) the data-intensive and resource-intensive finetuning process. This paper introduces a novel NL-FOL translation method, dubbed Code4Logic, which is based on in-context learning and employs code snippets to bridge the gap between natural language and first-order logic. By converting the translation task into a progressive code generation task, Code4Logic demonstrates strong generalization within a training-free manner, and enhances the performance of large language models (LLMs) to generate complex first-order logical formulas. Experimental results on NL-FOL task and downstream task datasets indicate that Code4Logic surpasses prominent training-free baselines and is comparable to supervised models trained on the full training data.

## introduction in recent years, the application of deep neural networks has achieved tremendous success, especially with the emergence of large language models @xcite @xcite , greatly driving the development of artificial intelligence. nevertheless, deep neural networks still exhibit limitations, notably in terms of interpretability and comprehensibility. therefore, the combination of interpretable symbolic logic and machine learning has garnered extensive attention @xcite . particularly noteworthy is the task of translating natural language into first-order logical formula (nl-fol) @xcite , which serves as a funda-mental component in various logic-based natural language processing (nlp) applications, such as textual entailment @xcite , natural language inference @xcite @xcite , machine reading comprehension @xcite , and natural language reasoning @xcite @xcite . this challenging task demands a comprehensive understanding of natural language representation, the extraction of essential information, and the subsequent establishment of logical connections among this information @xcite . to tackle this challenge, the community has introduced diverse methods focused on translating natural language to first-order logic. prior methods rely on handcrafted rules @xcite @xcite . as natural language intricacies pose scalability issues, these approaches struggle to extend to practical scenarios. recently, there has been a growing inclination toward neural methods for addressing this task @xcite @xcite . specifically, some researchers leverage pretrained language models @xcite @xcite @xcite @xcite to solve the nl-fol task in a sequence-to-sequence paradigm @xcite @xcite . typically, these approaches involve fine-tuning or retraining pretrained language models with logical form data to enhance their efficacy in specific scenarios. notwithstanding their remarkable performance, they still grapple with several notable challenges: (1) inconsistency between training and inference. current llms are trained via extensive unsupervised pretraining on large-scale general natural language corpora, which lacks firstorder logical form data, as with other symbolic data @xcite . conse- if a convicted criminal is found guilty, then they are sentenced to a punishment. nl: fol: figure 1 : example of natural language to first-order logic task from the foilo dataset @xcite , which requires the extraction of key information and identification of logical connectives to construct the logical form. in this paper, we propose an nl-fol translation method code4logic, which is based on in-context learning @xcite of llms. to bridge the gap between natural language and logical form, we utilize pretrained large code models which are pretrained by natural language and programming language @xcite @xcite @xcite and introduce code (e.g. python code snippets) as the intermediary @xcite . benefiting from the capabilities of llms learning from a few examples @xcite , code4logic could demonstrate strong generalization ability within a training-free framework. specifically, we transform the nl-fol translation task into a code generation task, enabling us to generate the logical form progressively and enhance the performance of llms to generate complex logical formulas @xcite . firstly, we define functions corresponding to components of first-order logic and implement them in python. these functions allow the conversion of each first-order logical formula into a code sequence (i.e., a function call sequence). for a sample of nl-fol pair, we transform the pair into a code sequence, which, upon execution, yields the first-order logical formula. to translate a query natural language, we provide the defined functions, k demonstration examples in code sequence format, and the query natural language to the llm. the llm is then tasked with completing the code sequence corresponding to the query natural language to generate the accurate first-order logic form. through extensive experiments, we demonstrate that code4logic achieves strong generalization performance in the nl-fol translation task and can also outperform salient baselines on representative downstream logic-based tasks. we outline our contributions as follows: in context learning with code-llms. incontext learning with large language models (llms) @xcite has exhibited robust few-shot performance across various natural language processing (nlp) tasks, such as question answering @xcite , information extraction @xcite , and mathematical reasoning @xcite . however, these methodologies encounter challenges in scenarios demanding complex reasoning and handling of symbolic data. recent studies indicate that llms trained with a code corpus exhibit outstanding complex and logical reasoning abilities @xcite @xcite , and the programming language is a good bridge between natural language and symbolic language. consequently, in-context learning with code-llms has been applied to tasks requiring complex reasoning, such as knowledge base question answering @xcite , information extraction @xcite , and table reasoning @xcite . typically, these methodologies transform the task format into code generation, prompting llms to accomplish the original task objective through the generation of class instances, supplementary code, or direct generation of required symbolic language (e.g., query sql). ## preliminary first-order logic. first-order logic (fol) @xcite ) is a logical system used for reasoning about the properties of objects. it involves quantified variables over non-logical objects, allowing the formation of sentences with variables. this structure facilitates statements like there exists x such that x is socrates and x is a man, which differs from simple propositions such as socrates is a man. in this context, there exists functions as a quantifier, with x representing a variable. this logic consists of two key components: syntax governs the construction of valid symbol sequences in first-order logic, while semantics clarifies the interpretations of these expressions. in this paper, we include the basic elements of first-order logic as follows: the full backus-naur form grammar to induce any first-order logical formula can be found in appendix a. ## method in this section, we will introduce how to translate the natural language to the first-order logic via code4logic. initially, the first-order logical formula is parsed into the tree structure (section 4.2). each node in the tree can be viewed as a function that generates a subformula of the first-order logical formula, which can be easily implemented in python. based on this, we can construct a code sequence given a first-order logical formula (section 4.2). subsequently, when a new natural language input necessitates translation, an llm is employed to create a code sequence of function calls based on demonstration examples (section 4.3). finally, a program interpreter is utilized to execute the generated code sequence and derive the first-order logical formula. we also discuss the implementation details and the utilization of resulting first-order logical formulas for downstream tasks in appendix d.1. ## conclusion this paper introduces code4logic, a trainingfree method that leverages the code based prompt for nl-fol translation task within an in-context learning framework. through utilizing code as a connector linking natural language and first-order logical formulas, our approach adeptly bridges the gap between training and inference in large language models, showcasing robust generalization capabilities. extensive experiments demonstrate the superiority of our model not only in nl-fol translation but also in various downstream tasks.