P - FOLIO : Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains (2024)

Abstract: Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales, which are not sufficient for properly assessing model’s capabilities. We present P-FOLIO , a human-annotated dataset consisting of diverse and complex reasoning chains for a set of realistic logical reasoning stories also written by humans. P-FOLIO is collected with an annotation protocol that facilitates humans to annotate well-structured natural language proofs for first-order logic reasoning problems in a step-by-step manner. The number of reasoning steps in P-FOLIO span from 0 to 20. We further use P-FOLIO to evaluate and improve large-language-model (LLM) reasoning capabilities. We evaluate LLM reasoning capabilities at a fine granularity via single-step inference rule classification, with more diverse inference rules of more diverse and higher levels of complexities than previous works. Given that a single model-generated reasoning chain could take a completely different path than the human-annotated one, we sample multiple reasoning chains from a model and use pass@k metrics for evaluating the quality of model-generated reasoning chains. We show that human-written reasoning chains significantly boost the logical reasoning capabilities of LLMs via many-shot prompting and fine-tuning. Furthermore, fine-tuning Llam3-7B on P-FOLIO improves the model performance by 10% or more on three other out-of-domain logical reasoning datasets.

## introduction a logical reasoning story consists of a series of premises and one or more conclusions @xcite . the goal of logical reasoning is to determine the truth values of the conclusions based on the premises @xcite . however, it cannot be determined whether a machine is indeed equipped with capabilities of logical reasoning even if it is able to output the correct truth value. generating proofs or reasoning chains that support its decision is essential for explainability. resolution-based symbolic proofs can be generated automatically with a first-order logic solver given explicit logic structures, but they are hard to read and cannot be written in natural language in a straightforward way in most cases @xcite . although studies on natural language proof generation for logical reasoning have been conducted before, they focus on synthetically generated logical reasoning datasets @xcite @xcite . these logical reasoning chains contains much less natural language variation and challenging reasoning patterns than realistic logic stories @xcite . we present p-folio , a new dataset consisting of human-written proofs for an existing popular dataset of logical reasoning, folio @xcite which consists of realistic logical reasoning stories written by humans. p-folio is collected with an annotation protocol for annotators to write proofs in a step-by-step manner without using proof by contradiction or proof by case. ## related work natural language reasoning existing studies on natural language reasoning @xcite @xcite @xcite focus on using different prompting strategies on large language models (llms) for better results. despite significant progress on the llm prediction accuracy of the final answer, no work has been done to evaluate the correctness of the intermediate steps. in our work, we propose to evaluate intermediate steps at different levels of granularity in order to better probe into the reasoning capabilities of llms. improving llm logical reasoning capabilities @xcite ; @xcite finetuned language models with logical reasoning data to improve logical reasoning capabilities of llms. large language models have also been directly used as soft logic reasoners and a variety of prompting techniques are proposed in order to improve their performance under this paradigm @xcite @xcite . using large language models as semantic parsers has also shown improvement on the reasoning performance @xcite where natural language reasoning problems are first parsed into logical forms before being fed into an inference engine to output the final answer. natural language proof generation proofwriter @xcite and fld @xcite are logical reasoning datasets equipped with natural language proofs, however both of them are synthetically generated dataset which neither contains abundant natural language variation nor encompasses challenging logical reasoning patters. previous studies on proof generation focus on proofwriter @xcite @xcite and prontoqa @xcite . logicbench is a synthetically generated natural language qa dataset and is used for evaluating the logical reasoning ability of llms @xcite . while folio covers first-order logic and one or more inference rules are used in each example, logicbench focuses on reasoning patterns covering propositional logic, first-order logic, and non-monotonic logic and focuses on the use of a single inference rule for each example. we collect proofs for folio @xcite ) instead, a realistic expert-written logical reasoning dataset. such proofs need to be written from scratch and are hard and time-consuming to write because humans need to manage both the language and reasoning complexity in the proof-writing process and manually construct many steps of reasoning. the resulting proofs contain more diverse types of inference rules and reasoning patterns in addition to containing more natural language variation and ensured semantic richness. ## p-folio in this section we describe the construction and properties of our dataset p-folio . ## reasoning evaluation we propose three tasks to evaluating llm reasoning capabilities with p-folio at different levels of granularities. ## experimental results we conduct experiments using p-folio. the train/dev/test split is 70%/15%/15%, which is the same as the split for folio @xcite . for all experiments involved with gpt-4 (openai et al., 2023), we use gpt-4-0125-preview and the temperature was set to 0. ## conclusion we presented p-folio, a new dataset consisting of human-written proofs for folio, a set of realistic logical reasoning stories written by humans. 26% of a total of 1430 proofs in p-folio consist of more than five reasoning steps and 4% of proofs consist of more than 10 reasoning steps. we further use p-folio to evaluate and improve large-language-model reasoning capabilities. we propose tasks with different levels of granularities and use various metrics to evaluate the performance of llm reasoning capabilities. we show that human-written reasoning chains significantly boost the logical reasoning capabilities of llms via many-shot prompting and fine-tuning. fine-prompt using deductive reasoning, find out the truth values of the conclusions based on the premises. the truth value can be true, false or uncertain. first show the reasoning process, and then output the truth value in the format of "truth value: ". premises: 1. if someone has never left the united states, then they have not been to europe. 2. if someone has traveled to the iberian peninsula, then they have been to europe.