Venue: NAACL 2025

# Few-Shot Natural Language to First-Order Logic Translation via Code Generation

Junnan Liu

Department of Data Science & AI, Monash University

to.liujn@outlook.com

###### Abstract

Translation of natural language to first-order logical formula (NL-FOL) has recently gained significant attention for its critical role in logic-based NLP applications. Some studies attempt to utilize pretrained language models in a sequence-to-sequence manner for the NL-FOL task. However, these methods encounter challenges such as (1) inconsistency between the training and inference phases and (2) the data-intensive and resource-intensive finetuning process. This paper introduces a novel NL-FOL translation method, dubbed **Code4Logic**, which is based on in-context learning and employs code snippets to bridge the gap between natural language and first-order logic. By converting the translation task into a progressive code generation task, **code4Logic** demonstrates strong generalization within a training-free manner, and enhances the performance of large language models (LLMs) to generate complex first-order logical formulas. Experimental results on NL-FOL task and downstream task datasets indicate that **Code4Logic** surpasses prominent training-free baselines and is comparable to supervised models trained on the full training data.

## 1 Introduction

In recent years, the application of deep neural networks has achieved tremendous success, especially with the emergence of large language models [35, 67, 80, 9], greatly driving the development of artificial intelligence. Nevertheless, deep neural networks still exhibit limitations, notably in terms of interpretability and comprehensibility. Therefore, the combination of interpretable symbolic logic and machine learning has garnered extensive attention [15, 26]. Particularly noteworthy is the task of translating natural language into first-order logical formula (NL-FOL) [16], which serves as a fundamental component in various logic-based natural language processing (NLP) applications, such as textual entailment [11], natural language inference [14, 26, 27], machine reading comprehension [14], and natural language reasoning [13, 25, 26]. This challenging task demands a comprehensive understanding of natural language representation, the extraction of essential information, and the subsequent establishment of logical connections among this information [28, 29].

To tackle this challenge, the community has introduced diverse methods focused on translating natural language to first-order logic. Prior methods rely on handcrafted rules [12, 24, 25]. As natural language intricacies pose scalability issues, these approaches struggle to extend to practical scenarios. Recently, there has been a growing inclination toward neural methods for addressing this task [20, 26]. Specifically, some researchers leverage pretrained language models [25, 26, 27, 28, 29, 28, 26] to solve the NL-FOL task in a sequence-to-sequence paradigm [26, 27, 28, 29]. Typically, these approaches involve fine-tuning or retraining pretrained language models with logical form data to enhance their efficacy in specific scenarios. Notwithstanding their remarkable performance, they still grapple with several notable challenges: (1) **Inconsistency between training and inference.** Current LLMs are trained via extensive unsupervised pretraining on large-scale general natural language corpora, which lacks first-order logical form data, as with other symbolic data [26, 27]. Conse

quently, generating the accurate logical form for LLMs poses a considerable challenge (Nie et al., 2024). Moreover, finetuning or retraining large language models with logical form data may lead to catastrophic forgetting in models tailored specifically for natural language tasks (Wu et al., 2024).

(2) **Data scarcity and training cost.** Most supervised training methods are data-intensive, requiring a significant number of training examples with expert-annotated steps. Considering both the limited availability of labeled logical form data in practical situations and the computational overhead of finetuning large-scale language models, the generalization capability in the low resource setting of the translation model holds significant importance. Nevertheless, current approaches lack investigating the performance in zero-shot or few-shot scenarios (Shin et al., 2021).

In this paper, we propose an NL-FOL translation method Code4Logic, which is based on in-context learning (Brown et al., 2020; Dong et al., 2022) of LLMs. To bridge the gap between natural language and logical form, we utilize pretrained large code models which are pretrained by natural language and programming language (Chen et al., 2021; Li et al., 2023c; Roziere et al., 2023; Gao et al., 2023; Chen et al., 2023) and introduce _code_ (_e.g._ Python code snippets) as the intermediary (Wang et al., 2023). Benefiting from the capabilities of LLMs learning from a few examples (Wei et al., 2022a), Code4Logic could demonstrate strong generalization ability within a training-free framework. Specifically, we transform the NL-FOL translation task into a code generation task, enabling us to generate the logical form progressively and enhance the performance of LLMs to generate complex logical formulas (Wei et al., 2022b). Firstly, we define functions corresponding to components of first-order logic and implement them in Python. These functions allow the conversion of each first-order logical formula into a code sequence (_i.e._, a function call sequence). For a sample of NL-FOL pair, we transform the pair into a code sequence, which, upon execution, yields the first-order logical formula. To translate a query natural language, we provide the defined functions, \(K\) demonstration examples in code sequence format, and the query natural language to the LLM. The LLM is then tasked with completing the code sequence corresponding to the query natural language to generate the accurate first-order logic form. Through extensive experiments, we demonstrate that Code4Logic achieves strong generalization performance in the NL-FOL translation task and can also outperform salient baselines on representative downstream logic-based tasks.

We outline our contributions as follows:

* We propose a novel in-context learning method, named Code4Logic, for the NL-FOL translation task. In contrast to current methods, Code4Logic avoids extensive overhead for finetuning and mitigates the reliance on vast training data.
* We convert the NL-FOL translation task to a code generation task to bridge the gap between natural language and the first-order logical formula. By progressively generating the code sequence, our method can boost the generation performance of LLMs when generating complex logical formulas.
* Experimental results on two NL-FOL translation datasets demonstrate that our Code4Logic outperforms salient training-free methods and can achieve comparable results to those fully supervised training methods.
* Moreover, experiments conducted on three categories of downstream tasks validate the availability and efficacy of the first-order logic formulas produced by our method.

## 2 Related Work

Natural Language to First-Order Logic.The task of translating natural language to first-order logic has long garnered significant attention. Previous researchers attempt to address this problem using a rule-based approach (Bos and Markert, 2005; Zettlemoyer and Collins, 2005; Barker-Plummer et al., 2009; Abzianidze, 2017). Recently, rein

Figure 1: Example of natural language to first-order logic task from the FOILO dataset (Han et al., 2022), which requires the extraction of key information and identification of logical connectives to construct the logical form.

 

forcement learning (Lu et al., 2022) and dual learning (Cao et al., 2019) have been employed to generate first-order logical formulas using neural networks. Drawing inspiration from the machine translation task, some studies have tackled this task using a sequence-to-sequence model (Singh et al., 2020; Xu et al., 2024; Yang et al., 2024b), such as large language models. While large language models exhibit robust generalization capabilities and can handle diverse language structures, they still encounter difficulties in generating accurate logical forms due to their intricate nature. Moreover, certain studies have concentrated on developing datasets (Han et al., 2022; Tian et al., 2021; Yang et al., 2024b), offering substantial assistance for fine-tuning and context-based learning approaches.

**In Context Learning with Code-LLMs.** Incontext learning with large language models (LLMs) (Brown et al., 2020) has exhibited robust few-shot performance across various natural language processing (NLP) tasks, such as question answering (Li et al., 2023d; Nie et al., 2024), information extraction (Pang et al., 2023; Mo et al., 2024), and mathematical reasoning (Lewkowycz et al., 2022; Imani et al., 2023). However, these methodologies encounter challenges in scenarios demanding complex reasoning and handling of symbolic data. Recent studies indicate that LLMs trained with a code corpus exhibit outstanding complex and logical reasoning abilities (Chen et al., 2021; Nijkamp et al., 2022; Gao et al., 2023; Chen et al., 2023), and the programming language is a good bridge between natural language and symbolic language. Consequently, in-context learning with code-LLMs has been applied to tasks requiring complex reasoning, such as knowledge base question answering (Li et al., 2023d; Nie et al., 2024), information extraction (Li et al., 2023b; Wang et al., 2023), and table reasoning (Cheng et al., 2023). Typically, these methodologies transform the task format into code generation, prompting LLMs to accomplish the original task objective through the generation of class instances, supplementary code, or direct generation of required symbolic language (_e.g._, query SQL).

## 3 Preliminary

First-Order Logic.First-order logic (FOL) (Barwise, 1977) is a logical system used for reasoning about the properties of objects. It involves quantified variables over non-logical objects, allowing the formation of sentences with variables. This structure facilitates statements like _there exists \(x\) such that \(x\) is Socrates and \(x\) is a man_, which differs from simple propositions such as _Socrates is a man_. In this context, there exists functions as a quantifier, with \(x\) representing a variable. This logic consists of two key components: syntax governs the construction of valid symbol sequences in first-order logic, while semantics clarifies the interpretations of these expressions. In this paper, we include the basic elements of first-order logic as follows:

* **Constants.** Constants represent individuals in the world, such as \({\tt Boy},{\tt Socrates}\).
* **Variables.** Variables indicate variable symbol, such as \(x,y,z\).
* **Function.** Function maps individuals to individuals, for example, \({\tt FatherOf}({\tt Tom})\) means the father of Tom.
* **Predicate.** Predicate maps individuals to truth values, such as \({\tt Greater}(x,y)\).
* **Logical Connectives.** Logical connectives include \(\wedge,\vee,\oplus,\rightarrow,\leftrightarrow\).
* **Quantifier Symbols.** Quantifier symbols consist of \(\forall\) for universal quantification, and \(\exists\) for existential quantification.
* **Equality Symbols.** Equality symbols can be divided into equivalence \(=\) and non-equivalence \(\neq\).
* **Punctuation Symbols.** Punctuation symbols include brackets, dots, and etc.

The full Backus-Naur Form grammar to induce any first-order logical formula can be found in Appendix A.

## 4 Method

In this section, we will introduce how to translate the natural language to the first-order logic via Code4Logic. Initially, the first-order logical formula is parsed into the tree structure (Section 4.2). Each node in the tree can be viewed as a function that generates a subformula of the first-order logical formula, which can be easily implemented in Python. Based on this, we can construct a code sequence given a first-order logical formula (Section 4.2). Subsequently, when a new natural language input necessitates translation, an LLM is employed to create a code sequence of function calls 

based on demonstration examples (Section4.3). Finally, a program interpreter is utilized to execute the generated code sequence and derive the first-order logical formula. We also discuss the implementation details and the utilization of resulting first-order logical formulas for downstream tasks in SectionD.1.

### Task Formulation

Given a natural language statement \(x_{nl}\), the NLFOL task aims to translate it to the first-order logical formula \(x_{fol}\). In this paper, we reformulate the task as a code generation task, where \(x_{nl}\) and \(x_{fol}\) are denoted in code form as \(x^{c}_{nl}\) and \(x^{c}_{fol}\), respectively. When the translation model (e.g., LLM) is provided with \(x^{c}_{nl}\) as input, its objective is to generate \(x^{c}_{fol}\). Within the context of in-context learning, the translation model receives a limited set of annotated data comprising \(K\) demonstration examples in the format of pairs \(\left\{\left(x^{c}_{nl,i},x^{c}_{fol,i}\right)\right\}_{i=1}^{K}\).

### Parse First-Order Logic to Code Sequence

To perform in-context learning for NL-FOL translation in the code generation manner, the primary challenge lies in the conversion of a first-order logical formula into a code sequence. In practice, we parse the first-order logical formula to a tree structure according to the BNF (Backus Normal Form) grammar. The hierarchical tree structure offers an effective approach for obtaining code sequence: by conceptualizing each node in the tree as a function that processes the child node's formula and produces the formula representation of the current substructure, we can systematically reconstruct the original first-order logical formula in a bottom-up fashion. For instance, consider the following first-order logical formula:

\[\forall x\left(\text{ConvictedCriminal}(x)\wedge\text{FoundGuilty}(x)\right.\\ \to\texttt{SentencedToPunishment}(x)\rangle\,.\]

Figure2 shows the corresponding tree structure. Moreover, this procedure can be articulated directly using a programming language format. We can use the pseudo-code sequence in Algorithm1 to generate the first-order logical formula in Figure2. In light of the components of first-order logic, we have devised 15 fundamental functions for generating first-order logical formulas. We opt for Python as the implementation platform for these functions, building upon the proven success of Code-LLMs Chen et al. (2021); Li et al. (2023c) in Python. For comprehensive information on the implementation of all foundational functions, please refer to AppendixB.

```
formula_x=variable('X') formula1=predicate('Converducterminal',forula_x) formula2=Predicate('Foundguability',forula_x) formula3=Predicate('SetenceIdoptimishment',forula_x) formula=Conjunction(formula1,forula2) formula=Implication(formula1,forula3) formula=UniversalQuantification(formula,formula_x) formula3)
```

**Algorithm 1** Pseudo code sequence for the first-order logical formula \(\forall x(\text{ConvictedCriminal}(x)\land\text{FoundGuilty}(x)\allowbreak\rightarrow \texttt{SentencedToPunishment}(x))\).

Also, we provide the pseudo-code for how to convert the FOL tree structure to Python code sequence in AppendixC.

### In-Context Learning for NL-FOL Translation

In the realm of in-context learning Brown et al. (2020), a method typically involves a set of demonstration examples \(\left\{e_{i}\right\}_{i=1}^{K}\) and a query \(q\). Certain tasks may necessitate an additional task description or instruction denoted as \(p\). Then, a large language model is tasked with generating the output \(y\) defined as:

\[y=\text{LLM}\left(p;\left\{e_{i}\right\}_{i=1}^{K};q\right)\,. \tag{1}\]

Here, \(y\) may take the form of a label in classification tasks Shome and Yadav (2023); Yang et al. (2024a) or a sequence in generative tasks Li et al. (2023a); Mathur et al. (2023); Tang et al. (2023). Figure3 outlines the framework of Code4Logic and the aforementioned components, integrated within the framework of Code4Logic, will be further elaborated upon in the subsequent paragraphs.

Task Description.Task description serves to offer detailed guidance to the model regarding the 

task. In Code4Logic, the task description is segmented into two components: (1) comment-style instructions and (2) implementation of basis functions similar to previous work Wang et al. (2023); Li et al. (2023b). As previously stated, all basis functions are implemented in Python, thus the instructions are also provided in the form of Python comments, as demonstrated in Figure 4. Subsequently, the comment-style instructions and the implementation of basis functions outlined in Appendix B are merged to compose the task description \(p\).

Demonstration Examples.Demonstration examples consist of a series of NL-FOL code pairs \(\left\{\left(x_{nl,i}^{c},x_{fol,i}^{c}\right)\right\}_{i=1}^{K}\). Each demonstration example consists of (1) a natural language statement in Python and (2) a code sequence to generate the corresponding first-order logical formula. The natural language statement is transformed into a Python assignment statement, where the natural language string is assigned to the variable natural_language_statement. The code sequence is obtained from the parse tree of the first-order logical formula mentioned in Section 4.2. Referring to the first-order logical formula in Figure 1 and Figure 2, the demonstration example will be reformulated as shown in Figure 5. Following the execution of the aforementioned natural_language_statement = 'If a convicted criminal is '*' '\\ found guilty, then they are sentenced to a punishment.' formali = Variable(X') formula2 = Predicate("Convicted Criminal ', [formula1]) formula3 = Predicate("FoundEquity ', [formula1]) formula4 = Predicate("SentenceontpointShment ', [formula1]) formula5 = Conjunction("Comfortinale", [formula4]) formula6 = Implication(Formula5, [formula6, [formula8]) formula8 = UniversalContification(formula6, [formula8]) formula9 = End(Formual8))

Figure 4: Python code for comment-style instruction.

Figure 5: Code sequence for _If a convicted criminal is found guilty, then they are sentences to a punishment_.

Figure 3: Overall process of Code4Logic.

 

Experiments

### Experiments on NL-FOL Translation

#### 5.1.1 Setup

Datasets.We use two mainstream datasets for the NL-FOL translation task, FOLIO Han et al. (2022) and MALLS Yang et al. (2024b). Specifically, we use the validation set of FOLIO and the test set of MALLS to assess the performance of Code4Logic and other baselines.

Baselines.We mainly compare our model with two salient supervised training methods LogICLAMA Yang et al. (2024b) and SymbolLLM Xu et al. (2024). Both of these models stem from the fine-tuning or retraining of large language models using NL-FOL translation datasets. Additionally, we present performance results for FlanT5 Raffel et al. (2020); Chung et al. (2022) and Claude-1 Perez et al. (2023), CodeGeeX2 Zheng et al. (2023a), GPT-3.5 Ouyang et al. (2022), and GPT-4 OpenAI (2023) under few-shot settings, which is based on the text prompt. Please refer to Appendix D.3.2 for more details about baselines.

Metrics.Common translation and generation tasks often employ Rouge Lin (2004) and Bleu Papineni et al. (2002) as metrics, evaluating n-gram overlap between the reference and candidate. Nevertheless, these metrics are inadequate for NL-FOL translation tasks since the focus is on logical equivalence rather than exact word-level matches. Aligning with previous works Yang et al. (2024b); Xu et al. (2024), we utilize Logic Equivalence (LE) as the evaluation metric. LE is determined by comparing the truth tables of the reference first-order logical formula and the candidate version, followed by calculating the overlap ratio. For further details, please refer to the original paper Yang et al. (2024b).

#### 5.1.2 Results

Code4Logic v.s. Supervised Training Methods.As shown in Table 1, Code4Logic surpasses LogICLLaMA, showcasing strong capabilities in the NL-FOL translation task. Specifically, Code4Logic gt-3.5-turbo-16k leads to improvements of 7.77 and 9.58 over LogicLDAMA for the FOLIO and MALLS datasets, respectively. Moreover, despite operating in a training-free manner, Code4Logic demonstrates surpassing performance relative to Symbol-LLM, which is trained on a more extensive symbolic dataset than LogICLLaMA.

Code4Logic v.s. Text Prompt Based Methods.To illustrate the effectiveness of using code prompts, we proceed to compare Code4Logic with a text prompt-based baseline under the same LLM framework. The results presented in Table 1 indicate that employing a code prompt significantly enhances performance across both datasets, emphasizing the advantages of utilizing code as a bridge between natural and symbolic languages.

Performance w.r.t. Different Code-LLMs.We compare the performance of GPT-3.5 and CodeGeeX to measure the influence of different Code-LLMs. Notably, GPT-3.5 outperforms CodeGeeX due to its superior text understanding capabilities. CodeGeeX, on the other hand, having a smaller parameter count (6b), is hindered by its lack of alignment as a code completion model.

Performance w.r.t. Different Number of Demonstration Examples.As shown in Figure 6, we conduct the experiment concerning different numbers of demonstration examples. The model's performance improves with an increased number of demonstration examples. However, the performance gains become marginal once a certain threshold is reached. Notably, even with a limited number of demonstration examples, Code4Logic delivers competitive results.

Ablation Study.To assess the significance of each component of the prompt in Code4Logic, we conduct ablation experiments as detailed in Table 2. The findings indicate that excluding the _comment-style instruction_, _basis function definition_, or _demonstration examples_ led to a significant decline in model performance. Specifically, the absence of the _comment-style instruction_ could obscure the task definition, resulting in incorrect model outputs. Similarly, omitting the _basis function definition_ hinders the ability of Code-LLMs to comprehend essential functions, thus impeding the generation of code sequences. The removal of _demonstration examples_ complicates the CodeLLMs' understanding of task requirements solely based on instructions. Additionally, we switch components from text-based prompts to evaluate the efficacy of code-based prompts. Analysis of the experimental outcomes reveals a slight performance decline with text-based Comment-Style Instructions, while a notable performance drop occurs 

with text-based Basis Function Definitions and text-based Demonstration Examples. While text-based Comment-Style Instructions, resembling NL-based prompts, yield outcomes akin to Python comment-style instructions, text-based Basis Function Definitions and Demonstration Examples impede effective information provision due to their incongruuity with code-style prompts.

### Experiments on Downstream Tasks

First-order logical formulas derived from natural language expressions enable the completion of various downstream tasks, such as textual entailment (Bos and Markert, 2005) and natural language reasoning (Clark et al., 2020; Tafjord et al., 2021; Wang et al., 2022). These tasks are commonly tackled through logical reasoning or proving using external tools like Prover9 (McCune, 2005-2010)1 and Pyke (Frederiksen, 2008)2.

Footnote 1: [https://www.cs.unm.edu/~mccune/prover9/](https://www.cs.unm.edu/~mccune/prover9/)

Footnote 2: [https://pyke.sourceforge.net/](https://pyke.sourceforge.net/)

To verify the validity of Code4Logic in downstream tasks, we generate the code sequence using the proposed framework and subsequently execute Python code to acquire the first-order logic form, which is readily convertible and processable by external logical reasoning engines. Finally, we employ an external logical reasoning engine to derive the output. A similar process can be found in previous work (Pan et al., 2023).

However, there may be ambiguity errors due to the randomness of the generated results of large language models. For instance, the terms UK and UnitedKingdom may both refer to England, yet they are treated as distinct entities by the reasoning engine. To solve this problem, we first tokenize all the first-order logical formulas into tokens. Subsequently, we employ a merging algorithm based on semantic similarity to unify the statements that may express the same concept. Finally, the unified first-order logical formulas are fed into the reasoning engine for following computational processing.

#### 5.2.1 Setup

Datasets.We conduct experiments on three types of downstream tasks: natural language inference, logical reasoning, and fact-checking. We chose three representative datasets: LogicNLI (Tian et al., 2021) for natural language inference, RuleTaker (Clark et al., 2020) for logical reasoning, and VitaminC (Schuster et al., 2021) for fact-checking.
* **LogicNLI** LogicNLI is a NLI dataset. LogicNLI effectively separates the targeted FOL reasoning from common-sense inference and comprises 16k, 2k, and 2k samples for the training, validation, and test sets, respectively.

\begin{table}
\begin{tabular}{l l c c} \hline \hline  & \multirow{2}{*}{**Model**} & **FOOTNOTE:]Footnote [ENDFOOTNOTE]** & **POLIO** & **MALLS** \\  & & **LE** & **LE** \\ \hline \multicolumn{4}{c}{CodeGeeX} \\ CODE4Logic & 84.77 & 85.81 \\ w/o Comment-Style Instruction & 30.45 & 31.77 \\ w Text-Based Comment-Style Instruction & 82.68 & 83.71 \\ w/o Basis Function Definition & 21.78 & 20.29 \\ w Test-based Basis Function Definition & 20.58 & 22.81 \\ w/o Demonstration Examples & 24.61 & 22.78 \\ w Text-based Demonstration Examples & 27.64 & 23.11 \\ \hline \multicolumn{4}{c}{**GTP-3.5**} \\ COde4Logic & 92.67 & 90.92 \\ w/o Comment-Style Instruction & 29.33 & 29.61 \\ w/o Basic Function Definition & 22.82 & 23.47 \\ w/o Demonstration Examples & 24.91 & 25.19 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Ablation Study on FOLIO and MALLS. We only report the mean of experimental results.

 

* **RuleTaker.** RuleTaker is a large-scale dataset, which is designed for logical reasoning tasks. RuleTaker consists of 480k, 75.9k, and 152k samples for the training, validation, and test sets, respectively.
* **VitaminC.** VitaminC contains more than 450k claim-evidence pairs for fact-checking based on over 450k claim-evidence pairs sourced from over 100k revisions of popular Wikipedia pages, including "synthetic" revisions.

Baselines.We mainly evaluate Code4Logic against representative large language models (LLMs), such as Flan-T5 [36, 37], LLaMA [15], and GPT-3.5 [16], which are under few-shot settings. Additionally, we compare it with some prominent baseline methods [14, 15, 16] that have been trained on the full training set. Please refer to Appendix D.4.2 for more details about baselines.

Metrics.We approach the three task types as multi-classification tasks, with the average accuracy serving as the performance metric. Code4Logic and other few-shot methods are evaluated directly on the test set. In contrast, additional methods are trained on the training set, showcasing the best performance on the test set chosen using the validation set.

#### 5.2.2 Results

Performance on LogicNLIThe results for LogicNLI are presented in Table 3. Code4Logic significantly outperforms fully supervised training baselines. This superiority can be attributed to the specific requirement of logical reasoning imposed by the LogicNLI dataset, a criterion for which previous pre-trained models (_e.g._, BERT, and RoBERTa) exhibit limitations in their logical reasoning capabilities.

Performance on RuleTaker.RuleTaker dataset requires models to have complex logical reasoning ability compared to LogicNLI. The performance of Code4Logic and other baselines is shown in Table 4. We can find that Code4Logic surpasses salient baselines, which demonstrates the effectiveness of our Code4Logic to understand complex logical structures.

Performance on VitaminC.The experimental results on the Vitamin dataset are presented in Table 5. The fact-checking task requires the model to have a more complete understanding of the text structure and the facts contained within it than the previous task. We can find that our Code4Logic outperforms the few-shot methods reliant on large 

language models, indicating the efficacy of employing logical expressions as transitional tools for addressing the fact-checking task. Furthermore, our Code4Logic exhibits comparable performance over conventional text generation evaluation models, providing additional confirmation of our hypothesis.

Case Study.Additionally, we illustrate several cases of the code sequence generated by Code4Logic and the corresponding first-order logical formula, which are available in Figure 7 and Appendix E.2. These cases demonstrate Code4Logic's capability to produce efficient code sequences and high-quality first-order logical formulas.

## 6 Conclusion

This paper introduces **Code4Logic**, a training-free method that leverages the code based prompt for NL-FOL translation task within an in-context learning framework. Through utilizing code as a connector linking natural language and first-order logical formulas, our approach adeptly bridges the gap between training and inference in large language models, showcasing robust generalization capabilities. Extensive experiments demonstrate the superiority of our model not only in NL-FOL translation but also in various downstream tasks.

### Limitations

In this paper, we introduce a new NL-FOL translation method. We believe this method still has much room for improvement:

* Multilingual expansion. Currently, our method is constrained to translation between English and first-order logical formulas, necessitating additional investigation for other languages;
* Inference efficiency. Since our approach is based on a large language model and a long prompt, our approach is less efficient when inference, which is sensitive in some scenarios.

## References

* Abzianidze (2017) Lasha Abzianidze. 2017. Langpro: Natural language theorem prover. In _EMNLP (System Demonstrations)_, pages 115-120. Association for Computational Linguistics.
* Agrawal et al. (2019) Priyanka Agrawal, Ayushi Dalmia, Parag Jain, Abhishek Bansal, Ashish R. Mittal, and Karthik Sankaranarayanan. 2019. Unified semantic parsing with weak supervision. In _ACL_, pages 4801-4810. Association for Computational Linguistics.
* Andreas et al. (2013) Jacob Andreas, Andreas Vlachos, and Stephen Clark. 2013. Semantic parsing as machine translation. In _ACL_, pages 47-52. The Association for Computer Linguistics.
* Barker-Plummer et al. (2009) Dave Barker-Plummer, Richard Cox, and Robert Dale. 2009. Dimensions of difficulty in translating natural language into first-order logic. In _EDM_, pages 220-229.
* Barwise (1977) Jon Barwise. 1977. An introduction to first-order logic. In _Stud. Logic Found_, volume 90, pages 5-46. Elsevier.
* Bos and Markert (2005) Johan Bos and Katja Markert. 2005. Recognising textual entailment with logical inference. In _HLT/EMNLP_, pages 628-635. The Association for Computational Linguistics.
* Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Franav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In _NeurIPS_.
* Cao et al. (2019) Ruisheng Cao, Su Zhu, Chen Liu, Jieyu Li, and Kai Yu. 2019. Semantic parsing with dual learning. In _ACL_, pages 51-64. Association for Computational Linguistics.
* Chen et al. (2019) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Kllaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chintazis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebben Guass, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alce Radford, Matthew Knight, Miles Brunudge, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluating large language models trained on code. _CoRR_, abs/2107.03374 

Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. 2023. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. _Transactions on Machine Learning Research_.
* Cheng et al. (2023) Zhoujun Cheng, Tianbao Xie, Peng Shi, Chengzu Li, Rahul Nadkarni, Yushi Hu, Caiming Xiong, Dragomir Radev, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Tao Yu. 2023. Binding language models in symbolic languages. In _ICLR_. OpenReview.net.
* Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022. Scaling instruction-finetuned language models. _CoRR_, abs/2210.11416.
* Clark et al. (2020) Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2020. Transformers as soft reasoners over language. In _IJCAI_, pages 3882-3890. ijcai.org.
* Dai et al. (2019) Wang-Zhou Dai, Qiu-Ling Xu, Yang Yu, and Zhi-Hua Zhou. 2019. Bridging machine learning and logical reasoning by abductive learning. In _NeurIPS_, pages 2811-2822.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In _NAACL-HLT_, pages 4171-4186. Association for Computational Linguistics.
* Dong et al. (2022) Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. A survey on in-context learning. _arXiv preprint arXiv:2301.00234_.
* Du et al. (2022) Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. GLM: general language model pretraining with autoregressive blank infiniling. In _ACL_, pages 320-335. Association for Computational Linguistics.
* Dubey et al. (2022) Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhl Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Himsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethamby Brion, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakcomkin, Ehab Albadawy, Elena Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaec, Gabrieille Lee, Georgia Lewis Anderson, Graeme Neil, Gregorier Milalon, Guan Pang, Guilhem Cucurell, Hailey Nguyen, Hannk Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeeth Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasaden Ahvala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. 2024. The llama 3 herd of models. _CoRR_, abs/2407.21783.
* Frederiksen (2008) Bruce Frederiksen. 2008. Applying expert system technology to code reuse with pyke. _PyCon: Chicago_.
* Gao et al. (2023) Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. PAL: program-aided language models. In _ICML_, volume 202 of _Proceedings of Machine Learning Research_, pages 10764-10799. PMLR.
* Hahn et al. (2022) Christopher Hahn, Frederik Schmitt, Julia J. Tillman, Niklas Metzger, Julian Siber, and Bernd Finkbeiner. 2022. Formal specifications from natural language. _CoRR_, abs/2206.01962.
* Han et al. (2022) Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Luke Benson, Lucy Sun, Ekaterina Zubova, Yujie Qiao, Matthew Burrell, David Peng, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailot, Ansong Ni, Linyoung Nan, Jungo Kasai, Tao Yu, Rui Zhang, Shafiq R. Joty, Alexander R. Fabbri, Wojciech Kryscinski, Xi Victoria Lin, Caiming Xiong, and Dragomir Radev. 2022. FOLIO: natural language reasoning with first-order logic. _CoRR_, abs/2209.00840.
* Hu et al. (2022) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In _ICLR_. OpenReview.net.
* Imani et al. (2023)Shima Imani, Liang Du, and Harsh Shrivastava. 2023. Mathpromptter: Mathematical reasoning using large language models. In _ACL (industry)_, pages 37-42. Association for Computational Linguistics.
* Kryscinski et al. (2020) Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In _EMNLP_, pages 9332-9346. Association for Computational Linguistics.
* Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.

 

BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In _ACL_, pages 7871-7880. Association for Computational Linguistics.
* Lewkowycz et al. (2022) Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay V. Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. 2022. Solving quantitative reasoning problems with language models. In _NeurIPS_.
* Li et al. (2023a) Chunyou Li, Mingtong Liu, Hongxiao Zhang, Yufeng Chen, Jinan Xu, and Ming Zhou. 2023a. MT2: towards a multi-task machine translation model with translation-specific in-context learning. In _EMNLP_, pages 8616-8627. Association for Computational Linguistics.
* Li et al. (2023b) Peng Li, Tianxiang Sun, Qiong Tang, Hang Yan, Yuanbin Wu, Xuanjing Huang, and Xipeng Qiu. 2023b. Codeie: Large code generation models are better few-shot information extractors. In _ACL_, pages 15339-15353. Association for Computational Linguistics.
* Li et al. (2023c) Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocektov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhkisti, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadori, Joel Lamy-Poirjer, Joao Monteiro, Oleh Shiazzhko, Nicolas Gontier, Nicholas Meade, Armel Zebanez, Ming-Ho Yee, Logesh Kumar Umapathi, Jian Zhu, Benjamin Lipkin, Muthasham Oblokulov, Zhiruo Wang, Rudeva Murthy V, Jason Stillerman, Siva Sankalg Patel, Dmitry Abulkhanov, Marco Zocca, Manan Dey, Zhihan Zhang, Nour Moustafa Fahmy, Urvashi Bhattacharyya, Wenhao Yu, Swyam Singh, Sasha Luccioni, Paulo Villegas, Maxim Kunakov, Fedor Zhdanov, Manuel Romero, Tony Lee, Nadav Timp, Jennifer Ding, Claire Schlesinger, Hailey Schoekopf, Jan Ebert, Tri Dao, Mayankar Mishra, Alex Gu, Jennifer Robinson, Carolyn Jane Anderson, Brendan Dolan-Gavitt, Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jenneite, Carlos Munoz Ferrandis, Sean Hughes, Thomas Wolf, Arjun Guha, Leandro von Werra, and Harm de Vries. 2023c. Starcoder: may the source be with you! _CoRR_, abs/2305.06161.
* Li et al. (2023d) Tianle Li, Xueguang Ma, Alex Zhuang, Yu Gu, Yu Su, and Wenhu Chen. 2023d. Few-shot in-context learning on knowledge base question answering. In _ACL_, pages 6966-6980. Association for Computational Linguistics.
* Liang (2013) Percy Liang. 2013. Lambda dependency-based compositional semantics. _CoRR_, abs/1309.4408.
* Lin (2004) Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In _Text summarization branches out_, pages 74-81.
* investigating contextual reasoning over long texts. In _AAAI_, pages 13388-13396. AAAI Press.
* Liu et al. (2020) Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2020. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. In _IJCAI_, pages 3622-3628. ijcai.org.
* Liu et al. (2029) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. _CoRR_, abs/1907.11692.
* Lu et al. (2022) Xunato Lu, Jingping Liu, Zhouhong Gu, Hanwen Tong, Chenhao Xie, Junyang Huang, Yanghua Xiao, and Wenguang Wang. 2022. Parsing natural language into propositional and first-order logic with dual reinforcement learning. In _COLING_, pages 5419-5431. International Committee on Computational Linguistics.
* Mathur et al. (2023) Yash Mathur, Sanketh Rangreji, Raghaw Kapoor, Medha Palavalli, Amanda Bertsch, and Matthew R. Gormley. 2023. Summqa at mediaq-chat 2023: In-context learning with GPT-4 for medical summarization. In _ClinicalNLP@ACL_, pages 490-502. Association for Computational Linguistics.
* McCune (2005) W. McCune. 2005-2010. Prover9 and mace4. Http://www.cs.umn.edu/ mccune/prover9/.
* Mo et al. (2024) Ying Mo, Jian Yang, Jiahao Liu, Shun Zhang, Jingang Wang, and Zhoujun Li. 2024. C-ICL: contrastive in-context learning for information extraction. _CoRR_, abs/2402.11254.
* Nie et al. (2024) Zhijie Nie, Richong Zhang, Zhongyuan Wang, and Xudong Liu. 2024. Code-style in-context learning for knowledge-based question answering. In _AAAI_, pages 18833-18841. AAAI Press.
* Nijkamp et al. (2022) Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Liftu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. A conversational paradigm for program synthesis. _CoRR_, abs/2203.13474.
* Olausson et al. (2023) Theo Olausson, Alex Gu, Benjamin Lipkin, Cedegao E. Zhang, Armando Solar-Lezama, Joshua B. Tenenbaum, and Roger Levy. 2023. LINC: A neurosymbolic approach for logical reasoning by combining language models with first-order logic provers. In _EMNLP_, pages 5153-5176. Association for Computational Linguistics.
* OpenAI (2023) OpenAI. 2023. GPT-4 technical report. _CoRR_, abs/2303.08774.
* Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke 

Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In _NeurIPS_.
* Pan et al. (2023) Liangming Pan, Alon Albalak, Xinyi Wang, and William Wang. 2023. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning. In _EMNLP (Findings)_, pages 3806-3824. Association for Computational Linguistics.
* Pang et al. (2023) Chaoxu Pang, Yixuan Cao, Qiang Ding, and Ping Luo. 2023. Guideline learning for in-context information extraction. In _EMNLP_, pages 15372-15389. Association for Computational Linguistics.
* Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In _ACL_, pages 311-318. ACL.
* Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning library. In _NeurIPS_, pages 8024-8035.
* Perez et al. (2021) Ethan Perez, Sam Ringer, Kamile Lukosite, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Petit, Catherine Olsson, Sandipan Kundu, Sautav Kadacvath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kerionion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, London Goldberg, Liane Lovitt, Martin Lucas, Michael Sellitire, Miranda Zhang, Neerav Kingsland, Nelson Elhage, Nicholas Joseph, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleene-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zace HatfieldDodds, Jack Clark, Samuel R. Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, and Jared Kaplan. 2023. Discovering language model behaviors with model-written evaluations. In _ACL (Findings)_, pages 13387-13434. Association for Computational Linguistics.
* Picco et al. (2021) Gabriele Picco, Thanh Lam Hoang, Marco Luca Sbodio, and Vanessa Lopez. 2021. Neural unification for logic reasoning over natural language. In _EMNLP (Findings)_, pages 3939-3950. Association for Computational Linguistics.
* Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training.
* Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. _OpenAI blog_, 1(8):9.
* Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. _JMLR_, 21:140:1-140:67.
* Roziere et al. (2023) Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jeremy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton-Ferrer, Aaron Grattafori, Wenhan Xiong, Alexandre Defossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeeve. 2023. Code llama: Open foundation models for code. _CoRR_, abs/2308.12950.
* Schuster et al. (2021) Tal Schuster, Adam Fisch, and Regina Barzilay. 2021. Get your vitamin c! robust fact verification with contrastive evidence. In _NAACL-HLT_, pages 624-643. Association for Computational Linguistics.
* Shin et al. (2021) Richard Shin, Christopher H. Lin, Sam Thomson, Charles Chen, Subbro Roy, Emmanouil Antonios Platanios, Adam Pauls, Dan Klein, Jason Eisner, and Benjamin Van Durme. 2021. Constrained language models yield few-shot semantic parsers. In _EMNLP_, pages 7699-7715. Association for Computational Linguistics.
* Shome and Yadav (2023) Debaditya Shome and Kuldeep Yadav. 2023. Exnet: Efficient in-context learning for data-less text classification. _CoRR_, abs/2305.14622.
* Singh et al. (2020) Hirturiy Singh, Milan Aggarwal, and Balaji Krishnamurthy. 2020. Exploring neural models for parsing natural language into first-order logic. _CoRR_, abs/2002.06544.
* Suzuki et al. (2019) Riko Suzuki, Hitomi Yanaka, Masashi Yoshikawa, Koji Mineshima, and Daisuke Beki. 2019. Multimodal logical inference system for visual-textual entailment. In _ACL_, pages 386-392. Association for Computational Linguistics.
* Tafjord et al. (2021) Oyyind Tafjord, Bhavana Dalvi, and Peter Clark. 2021. Proofwriter: Generating implications, proofs, andductive statements over natural language. In _ACL/IJCNLP (Findings)_, Findings of ACL, pages 3621-3634. Association for Computational Linguistics.
* Tang et al. (2023) Xiangru Tang, Andrew Tran, Jeffrey Tan, and Mark Gerstein. 2023. ClineAl note summarization from doctor-patient conversations through fine-tuning and in-context learning. In _ClinicalNLP@ACL_, pages 546-554. Association for Computational Linguistics.
* Tian et al. (2021) Jidong Tian, Yitian Li, Wenqing Chen, Liqiang Xiao, Hao He, and Yaohui Jin. 2021. Diagnosing the first-order logical reasoning ability through logicml. In 

EMNLP_, pages 3738-3747. Association for Computational Linguistics.
* Touvron et al. (2023a) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothee Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023a. Llama: Open and efficient foundation language models. _CoRR_, abs/2302.13971.
* Touvron et al. (2022a) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiboglu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inanc, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Kouara, Marie-Anne Lachaux, Thibaut Lavril, Jens Lee, Diana Liskovich, Yinghail Lu, Yuning Mao, Xavier Martinet, Todor Mihalov, Pushkar Mishra, Igor Molbyog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Pavin Xu, Zheng Yan, IIlyaar Zarov, Yuchen Zhang, Angela Fan, Melanie Kambandur, Sharan Narang, Aurelieu Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023b. Llama 2: Open foundation and fine-tuned chat models. _CoRR_, abs/2307.09288.
* Vasilyev et al. (2020) Oleg V. Vasilyev, Vedant Dharmidsharka, and John Bohannon. 2020. Fill in the BLANC: human-free quality estimation of document summaries. In _Eval4NLP_, pages 11-20. Association for Computational Linguistics.
* Wang et al. (2019) Po-Wei Wang, Priya L. Donti, Bryan Wilder, and J. Zico Kolter. 2019. Satnet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver. In _ICML_, volume 97, pages 6545-6554. PMLR.
* Wang et al. (2022) Siyuan Wang, Wanjun Zhong, Duyu Tang, Zhongyu Wei, Zhihao Fan, Daxin Jiang, Ming Zhou, and Nan Duan. 2022. Logic-driven context extension and data augmentation for logical reasoning of text. In _ACL (Findings)_, pages 1619-1629. Association for Computational Linguistics.
* Wang et al. (2023) Xingyao Wang, Sha Li, and Heng Ji. 2023. Code4struct: Code generation for few-shot event structure prediction. In _ACL_, pages 3640-3663. Association for Computational Linguistics.
* Wei et al. (2022a) Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022a. Emergent abilities of large language models. _TMLR_.
* Wei et al. (2022b) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022b. Chain-of-thought prompting elicits reasoning in large language models. In _NeurlPS_.
* Wu et al. (2024) Chengyue Wu, Yukang Gan, Yixiao Ge, Zeyu Lu, Jiahao Wang, Ye Feng, Ping Luo, and Ying Shan. 2024. LLMA pro: Progressive llama with block expansion. _CoRR_, abs/2401.02415.
* Xu et al. (2024) Fangzhi Xu, Zhiyong Wu, Qiushi Sun, Siyu Ren, Fei Yuan, Shuai Yuan, Qika Lin, Yu Qiao, and Jun Liu. 2024. Symbol-llm: Towards foundational symbolic-centric interface for large language models. In _ACL_, pages 13091-13116. Association for Computational Linguistics.
* Yanaka et al. (2019) Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, Kentaro Inui, Satoshi Sekine, Lasha Abzianildze, and Johan Bos. 2019. HELP: A dataset for identifying shortcomings of neural models in monotonicity reasoning. In _SEM@NAACL-HLT_, pages 250-255. Association for Computational Linguistics.
* Yang et al. (2024a) Songhua Yang, Xinke Jiang, Hanjie Zhao, Wenxuan Zeng, Hongde Liu, and Yuxiang Jia. 2024a. Faima: Feature-aware in-context learning for multi-domain aspect-based sentiment analysis. _CoRR_, abs/2403.01063.
* Yang et al. (2024b) Yuan Yang, Siheng Xiong, Ali Payani, Ehsan Shareghi, and Faramarz Fekri. 2024b. Harnessing the power of large language models for natural language to first-order logic translation. In _ACL_, pages 6942-6959. Association for Computational Linguistics.
* Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized autoregressive pretraining for language understanding. In _NeurIPS_, pages 5754-5764.
* Yuan et al. (2021) Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. Bartscore: Evaluating generated text as text generation. In _NeurIPS_, pages 27263-27277.
* Zeng et al. (2023) Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Zhiyuan Liu, Peng Zhang, Yunxiao Dong, and Jie Tang. 2023. GLM-13DB: an open bilingual pre-trained model. In _ICLR_. OpenReview.net.
* Zettlemoyer and Collins (2005) Luke S. Zettlemoyer and Michael Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In _UAI_, pages 658-666. AUAI Press.
* Zheng et al. (2023a) Qingkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Lei Shen, Zihan Wang, Andi Wang, Yang Li, Teng Su, Zhilin Yang, and Jie Tang. 2023a. CodegeeX: A pre-trained model for code generation with multilingual benchmarking on humaneval-x. In _KDD_, pages 5673-5684. ACM.

 

Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, Teng Su, Zhilin Yang, and Jie Tang. 2023b. Codegex: A pre-trained model for code generation with multilingual evaluations on humaneval-x. _CoRR_, abs/2303.17568.

 

BNF Grammar of First-Order Logic

The BNF grammar of first-order logic is illustrated in Figure 8.

## Appendix B Implementation of Basis Functions

The definitions of basis functions in Python are as follows:

```
defConstant(constant,name:str);"""returnaconstasmymbol!"" returnconstant_name_lower() defVariable(variable_name:str):"returnavisiblesymbol, whichstartswith's^and return'$'+variable_name deffunction(function_name:str,terms:List{Str}):"""returnafunctionsynbol,forexample, father(x)meansfatherofx''return'\((())^format( function_name.lower(),',','join(terms)) defPredicate(predicate_name:str,terms:List{Str}):"returnanatomicformulawithapredicate, whosemamestartsuptuppercase" return'\((())^format( predicate_name.lower().capitalize(),',','.join(terms)) defEqual(term_a:str,term_b:str):""returnanatomicformulawithequaloperation""" return'\(=()^format(term_a,term_b) defNonQual(term_a:str,term_b:str):""returnanatomicformulawithequaloperation""" return'\u2260()^format(term_a,term_b) defNegation(formula:str):""returnthenegationoftheinputformulas''return'\u3pac(())^format(formula) defConjunction(formula_a:str,formula_b:str):""returntheconjunctionofthemipontromulas''return\u2227()^format(formula_a,formula_b) defDisjunction(formula_a:str,formula_b:str):""returnthedisjunctionofthemipontromulas''return\u2228()^format(formula_a,formula_b) defImplication(antecedent_formula:str, consequent_formula:str):""returntheimplicinationformulaofthen antecedentfromalsoncesuperformula'' return'\u1929()^format( antecedent_formula,consequetronula) defEquivalence(formula_a:str,formula_b:str):""returnthelogicalequivalenceformulaofthenipontromulas''return'\u2194()^format(formula_a,formula_b) defNonequivalence(formula_a:str,formula_b:str):""returnthelogicalnon-equivalenceformulaofthenipontromulas'' return'\u2295()^format(formula_a,formula_b) defExistentialQuantification(formula:str,variable_symbol:str):""returnanexistentialquantificationofthenipontromulasandtheinputvariablesymbolassertvariable_symbolinormulasreturn'\u2203()()^format(variable_symbol, formula) defUniversalQuantification(formula:str,variable_symbol:str):""returnanuniversalquantificationofthenipontromulaandtheinputvariablesymbol"== assertvariable_symbolinormulasreturn'\u2204()()^format(variable_symbol, formula) defEnd(formula:str):returnformula
```

## Appendix C Algorithm of FOL Grammar Tree to Python Code Sequence

The Python-style pseudo code of the FOL grammar tree to Python code sequence is shown in Algorithm 2.

```
classFOLFormamarTreeNode: def__init__(self,start,end,string,type):#startpositionintnaturallanguagestatemstelf.start=start self.start=end##endingexpressionself.string=string self.string=string/ type,canbe'variables','constant',e.t.c.self.type=type#childrennodelist self.children=[] #assignachsubformulauniqueindex expression2idx=()code_sequence=[] defconstruct_code_sequence(node: FOLGramarTreeNode):#storethechlidemodeinformation children.attributes=I forchildinmode.children:children.children_attributes.append( construct_code_sequence(child))
#Getthevariablethatcarriesthechildemodeformulaaccordingtodifferenthodep, andthengeneratethecode.code =get_code(node_type, childern_attributes) code_sequence.append(code) idx=len(expression2idx)+1 expressigmoidX(node_String)=idx
#returntheinformationcorrespondingtocurrent node return('type:node.type,'string':node.string,') } constructor_code_sequence(tree) code_sequence.append('expression_*End({})'.format( code_sequence[-1].split('=')[@].strip()))
```

**Algorithm 2** Pseudo code for obtaining code sequence from a first-order logic grammar tree in Python-style.

 

build tool and can readily export the tree structure for subsequent code generation.

Code-LLMs.For Code-LLMs, we aim to conduct experiments with Codex from OpenAI aligning with previous works (Wang et al., 2023; Li et al., 2023b). However, the Codex models have been deprecated from the OpenAI APIs. Consequently, we opt to use the gpt-3.5-turbo-16k model from OpenAI for our experiments. Additionally, to facilitate a comparison of the performance among various code-LLMs, we also conduct experiments on the open-source code completion model CodeGeeX2. CodeGeeX2 (Zheng et al., 2023a) is constructed on the ChatGLM2 (Du et al., 2022; Zeng et al., 2023) architecture and trained on a more extensive dataset of code. For gpt-3.5-turbo-16k, we use the official API4 to obtain model results, whereas we employ the open-source model parameters for CodeGeeX25. During the generation process, the temperature is set to 0.7, the max_tokens is set to 500, and other parameters are kept at default values.

Footnote 4: [https://openai.com/api](https://openai.com/api)

Footnote 5: [https://huggingface.co/THUMR/codgegex2-6b](https://huggingface.co/THUMR/codgegex2-6b)

Construction of Demonstration Examples.In our experiments, the construction of demonstration examples is mainly based on the public datasets FOLIO (Han et al., 2022) and MALLS (Yang et al., 2024b). FOLIO, curated by expert annotators, functions as a natural language reasoning dataset that presents new challenges in first-order logical reasoning. It offers a wide array of natural language variations, an extensive vocabulary, and diverse logic patterns, comprising 8k NL-FOL pairs. Con

Figure 8: The BNF grammar of first-order logic. An item with a string suffix represents a character (_e.g._, letters and common punctuation) terminal symbol.

 

versely, MALLS is an NL-FOL dataset generated by GPT-4 (OpenAI, 2023). Compared to FOLIO, MALLS introduces a broader range of contextually rich NL-FOL pairs, encompassing 34K instances. In practice, we merge all NL-FOL pairs from the training sets of FOLIO and MALLS, converting them into code sequence format to construct the supporting dataset \({\cal D}\). During in-context learning, we randomly sample a fixed number (3 or 5, based on the input constraints of the Code-LLMs) of samples from \({\cal D}\) for each basis function to serve as demonstration examples.

### Full Prompt of Code4Logic

We provide the detailed prompt of Code4Logic in Figure 9.

### NL-FOL Translation Baselines

#### d.3.1 LLM-Based Few-Shot Baselines

For all of the large language model based few-shot baseline models (_e.g._, Claude-1, GPT4, text prompt based CodeGeeX, and GPT3.5), we use the following prompt template as shown in Figure 10.

When generating, the temperature is set to 0.7, the max_tokens is set to 200, and other parameters are kept at default values.

Figure 10: The detailed prompt of LLM-Based few-shot baselines.

Figure 9: The detailed prompt of Code4Logic.

 



#### d.3.2 Supervised Training Baselines

We mainly compare Code4Logic with two prominent baselines:

* **LogicLLAMA**(Yang et al., 2024). LogicLLAMA is a translation model for translating natural language to first-order logic based on LLaMA Touvron et al. (2023), fine-tuned with LoRA Hu et al. (2022). The authors begin by curating a high-quality and diverse dataset consisting of NL-FOL pairs at the sentence level obtained from GPT-4. Subsequently, they generate a perturbed variation of the logical formula in each pair to establish a controlled perturbation dataset. Additionally, they introduce the innovative SFT+RLHF framework, which trains LogiCLLaMA on the artificially perturbed NL-FOL pairs.
* **Symbol-LLM**(Xu et al., 2024). Symbol-LLM comprises a series of models designed for text-to-symbol tasks. The authors undertake an extensive collection of 34 text-to-symbol generation tasks, encompassing approximately 20 standard symbolic forms introduced with instruction tuning format. Then they employ a two-stage continual tuning framework to tune the LLaMA-2-Chat Touvron et al. (2023) model. In the study, we utilized the public model parameters of LogicLDAMA and Symbol-LLM for the

Figure 11: Prompt of few-shot baselines for downstream datasets.



experiment, maintaining consistency with the generation hyperparameters as reported in the original paper.

### Downstream Tasks Baselines

#### d.4.1 LLM-Based Few-Shot Baselines

All of the large language model-based few-shot baseline models evaluated on the LogicNLI, RuleTaker, and VitaminC datasets use the prompt template illustrated in Figure 11.

Similarly, the temperature is set to 0.7, the max_tokens is set to 200, and other parameters are kept at default values.

#### d.4.2 Supervised Training Baselines

We include different supervised training based baseline models on each of the three downstream task datasets.

* **LogicNLI.** Pretrained sequence-to-sequence models like BERT Devlin et al. (2019), RoBERTa Liu et al. (2019), and XLNet Yang et al. (2019) are employed for solving the logical reasoning task. Each sample's premise and hypothesis from the dataset are concatenated and fed into the model. Then, a classifier head is utilized to make predictions.
* **RuleTaker.** In addition to RoBERTa Liu et al. (2019) as the baseline model similar to LogicNLI, we introduce Neural Unifer Picco et al. (2021), a novel architecture that emulates unification and fact-checking algorithms to enhance the generalization capability for responding to complex queries.
* **VitaminC.** We compare Code4Logic with three salient supervised training based baselines: FactCC Kryscinski et al. (2020), BLANC Vasilyev et al. (2020), and BartScore Yuan et al. (2021). FactCC is a novel weakly supervised BERT-based model designed to verify factual consistency. The model is trained using synthesized data created from source documents through a set of rule-based transformations inspired by error analysis of outputs from state-of-the-art summarization models. BLANC is a BERT-based Devlin et al. (2019) method for automatically assessing the quality of document summaries, which can also be utilized for evaluating factual consistency. BartScore assesses factual consistency by framing it as a text generation task and resolves the modeling challenge using the Bart Lewis et al. (2020) model.

For baselines with open-source code, we make direct use of the hyperparameters reported in the paper. For other baselines, we select the optimal hyperparameters on the validation set and report the results.

### Dataset Statistics

The statistics of FOLIO and MALLS are summarized in Table 6 and the statistics of three downstream task datasets are shown in Table 7.

### Hardcore Configurations

We conducted all experiments in the following hardware environment:

* Operating System: Ubuntu 22.04.3 LTS.
* CPU: Intel Xeon Gold 6148 CPU @ 2.40GHz with 384GB DDR4 of Memory.
* GPU: NVIDIA Tesla A100 SMX4 with 80GB of Memory.
* Software: CUDA 11.8, Python 3.9.14, PyTorch Paszke et al. (2019) 2.3.0.

## Appendix E Additional Experimental Results

### Performance w.r.t. Different LLMs

Table 8 illustrates the experimental results of Code4Logic using different LLms. We can observe that even with non-code-tuned LLMs, Code4Logic can achieve prominent results.

\begin{table}
\begin{tabular}{l c c c} \hline \hline
* \\ MALLS & 27,284 &
- & 1,000 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Statistics of FOLIO and MALLS.

\begin{table}
\begin{tabular}{l l l l} \hline \hline
**Dataset** & **\#Train** & **\#Valid** & **\#Test** \\ \hline LogicNLI & 16,000 & 2,000 & 2,000 \\ RuleTaker & 480,152 & 75,872 & 151,911 \\ VitaminC & 370,653 & 63,054 & 55,197 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Statistics of LogicNLI, RuleTaker, and VitaminC.

 



### Case Study of Code4Logic

The cases are shown in Figure 12. We randomly sample cases from LogicNLI, RuleTaker, and VitanC.

## Appendix F More Discussion about Studies on Semantic Parsing

Semantic parsing Zettlemoyer and Collins (2005); Liang (2013); Andreas et al. (2013); Agrawal et al. (2019); Shin et al. (2021) involves transforming natural language into formal languages like lambda expressions, SQL, and graphs, which computers can understand. These formal languages are commonly used for creating queries and commands. Although these formalisms are quite direct, first-order logic presents a wider range of potential uses. Methodologically, our study investigates the viability of utilizing large language models in logical transformation, moving away from the traditional approach of semantic representation combined with parsing models, without the need for supplementary model training.



natural_language_statement = 'If all people are not dramatic, '+\  'then Clyde is impatient and Clyde is not eager.'  formula1 = Variable('x')  formula2 = Predicate('Dramatic', [formula1])  formula3 = Negation(formula2)  formula4 = Constant('clyde')  formula5 = Predicate('Impatient, [formula4])  formula6 = Predicate('Eager', [formula4])  formula7 = Negation(Formula6)  formula8 = Conjunction(formula5, formula7)  formula9 = Implication(formula3, formula8)  formula10 = UniversalQuantification(formula9, formula1)  formula = End(formula10)

--

\(\forall x(\neg{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{ }}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{\rm{}}{\rm{}}{}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{}{\rm{}}{\rm{}}{\rm{}}{ \rm{}}{}{\rm{}}{\rm{}}{\rm{}}{}{\rm{}}{\rm{}}{\rm{}}{}{\rm{}}{\rm{}}{\rm{}}{}{\rm{}}{\rm{}}{ \rm{}}{}{\rm{}}{\rm{}}{\rm{}}{}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{\rm{}}{ 

natural_language_statement = 'Greg Kot was commented favorably on '+\  'Ride the Lightning.'  formula1 = Constant('gregkot')  formula2 = Predicate('Commentfavorably', [formula1])  formula3 = Constant('ridethelightning')  formula4 = Predicate('On, [formula1, formula3])  formula5 = Conjunction(formula2, formula4)  formula = End(formula5)

Commentfavorably(gregket) \ On({\rm gregkot},{\rm ridethelighting})

natural_language_statement = 'If Carrick is not noisy, then Arnold is not '+\  'huge and Nick is intelligent.'  formula1 = Constant('carrick')  formula2 = Predicate('Noisy', [formula1])  formula3 = Negation(formula2)  formula4 = Constant('arnold')  formula5 = Predicate('Huge', [formula4])  formula6 = Negation(formula5)  formula7 = Implication(formula3, formula6)  formula = End(formula7)

-- Noisy(carrick) \(\rightarrow\neg\)Huge(arnold)

natural_language_statement = 'If there is someone who is not crystal, then '+\  'Roger is not clean and Adley is not glamorous.'  formula1 = Variable('x')  formula2 = Predicate('Ctsral', [formula1])  formula3 = Negation(formula2)  formula4 = Constant('roger')  formula5 = Predicate('clean', [formula4])  formula6 = Negation(formula5)  formula7 = Constant('adley')  formula8 = Predicate('Glamorous', [Formula7])  formula9 = Negation(formula7)  formula10 = Conjunction(formula6, formula9)  formula11 = Implication(formula3, formula10)  formula = End(formula11)

Figure 12: Cases of Code4Logic.