Can LLM s Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLM s (2024)

Abstract: Large language models (LLMs) have achieved impressive human-like performance across various reasoning tasks. However, their mastery of underlying inferential rules still falls short of human capabilities. To investigate this, we propose a logic scaffolding inferential rule generation framework, to construct an inferential rule base, ULogic, comprising both primitive and compositional rules across five domains. Our analysis of GPT-series models over a rule subset reveals significant gaps in LLMs’ logic understanding compared to human performance, especially in compositional and structural complex rules with certain bias patterns. We further distill these rules into a smaller-scale inference engine for flexible rule generation and enhancing downstream reasoning. Through a multi-judger evaluation, our inference engine proves effective in generating accurate, complex and abstract conclusions and premises, and improve various commonsense reasoning tasks. Overall, our work sheds light on LLMs’ limitations in grasping inferential rule and suggests ways to enhance their logical reasoning abilities .

## introduction however, collecting such inferential rules at scale presents a major challenge. previous work mainly relies on manual curation @xcite or inductive logic programming @xcite , which are either laborintensive or limited in diversity. besides, manually crafted rules often appear simple and overly specified, struggling to move beyond basic intuition or generalize across diverse situations. for example, the rule if x runs out of steam, then x becomes tired from @xcite has only one premise fact and narrowly specifies exhaustion. to this end, we introduce logic scaffolding inferential rule generation (loire), a framework to generate inferential rules of different complexities. loire operates in two stages: primitive rule generation and rule composition. initially, we define "primitive rules" to describe abstract objects like person and food, and ensure they cannot be decomposed into simpler rules, facilitating broad generalization and easy generation. we then incorporate gpt-4's generative capability and human expertise to generate primitive rules with high confidence. this process, consistently guided by symbolic logic, involves gpt-4 drafting potential conclusions in various domains, and forming premises with one or more facts. we ensure rules' logical soundness through the model's self-critique and human manual verification. in the second stage, we apply backward chaining @xcite upon these primitive logical rules to automatically construct compositional rules of varied lengths and structures at scale. using this framework, we construct ulogic, an inferential rule base with around 8, 000 primitive rules and over 6, 000 compositional rules. these rules span five key domains: object affordance, accessibility, interaction, location, and person's need. we hope ulogic will serve as a valuable resource, facilitating the assessment of llms' proficiency in underlying logic and enhancing flexible rule generation and downstream reasoning. we utilize ulogic to create an entailment probing task with a comprehensive and robust evaluation strategy, to assess llms' grasp of inferential rules against human performance. our analysis of gpt-series llms, including gpt-4, gpt-3.5-turbo and gpt-3.5-turbo-instruct, indicates they have a basic understanding of inferential rules but fall short of human proficiency, especially in rules with complex premises. specifically, all models struggle more as the compositional complexity increases. while gpt-4 performs consistently on verbalized and symbolic rules, the other models sharply degrade on symbolic rules. additionally, all models exhibit disparities on various rule structures with disjunctive-transitive rules posing the greatest challenges. moreover, these llms display notable polarity biases with gpt-4 showing a necessary bias, underscoring areas for improvement. we further distill crafted inferential rules into a smaller-scale inference engine for flexible rule generation and downstream reasoning. we design three tasks: conclusion generation, premise completion and premise generation, to construct an instructiontuning dataset for inferential rule distillation. experimental results through a multi-judger evaluation mechanism incorporating automatic metrics, llm evaluators and human preferences show that our inference engine possesses the ability for these three tasks. it outperforms gpt-3.5-turbo across all dimensions of three tasks and even surpasses gpt-4 in generating more complex and abstract rules. moreover, it can generate logical rules that enhance downstream commonsense reasoning. ## assessing llms' proficiency in capturing inferential rules we utilize ulogic for a systematic evaluation of llms' proficiency in underlying logic compared to human competence. specifically, we select a highquality probing subset with 1,104 diverse rules from our rule base foot_0 , and create a binary entailment classification task for assessing whether llms capture the entailment within inferential rules. ## conclusion this paper examines the proficiency of gpt-series llms in capturing logical inferential rules and probes their challenging reasoning space. we introduce a logic scaffolding inferential rule generation (loire) framework to create an inferential rule base ulogic, including nearly 8,000 primitive rules and over 6,000 compositional rules across five domains. our evaluations using a subset of ulogic show that even advanced models like gpt-4 struggle with compositional and structural complex rules and exhibit certain biases. furthermore, we distill ulogic into a smaller inference engine that performs well in generating inferential rules and benefit downstream reasoning tasks. our work points out where llms need to improve in logical reasoning and offers a pathway to enhance their reasoning capabilities.