Natural Logic at the Core: Dynamic Rewards for Entailment Tree Generation (2025)

Abstract: Entailment trees are essential for enhancing interpretability and transparency in tasks like question answering and natural language understanding. However, existing approaches often lack logical consistency, as they rely on static reward structures or ignore the intricate dependencies within multi-step reasoning. To address these limitations, we propose a method that integrates natural logic principles into reinforcement learning, enabling dynamic reward computation to guide entailment tree generation. Our approach ensures logical consistency across reasoning steps while improving interpretability and generalization. Experiments on EntailmentBank demonstrate significant improvements over state-of-the-art methods, highlighting the effectiveness of natural logic in structured reasoning.

## introduction in recent years, the generation of entailment trees @xcite @xcite and the application of reinforcement learning @xcite for knowledge selection have garnered considerable attention within the domain of automated reasoning. these methodologies aspire to model the structure and inference of knowledge, playing a crucial role in applications such as natural language understanding @xcite , automated theorem proving @xcite , and question answering @xcite . however, extant approaches frequently encounter a significant limitation: the lack of global logical constraints that direct the reasoning process. whether incrementally constructing the entailment tree @xcite @xcite or utilizing reinforcement learning @xcite to select pertinent knowledge, these methods * corresponding author. to address this issue, we propose a novel approach, nldr, which uses natural logic-based dynamic rewards to facilitate entailment tree construction. unlike prior methods that rely on fixed reward structures @xcite , our approach employs entailment scores derived from natural logic as dynamic rewards to guide the model's reasoning process. these dy-namic rewards enable the model to assess the logical consistency of each reasoning step, providing feedback that steers it towards the optimal reasoning path while helping to avoid local optima. one of the primary strengths of natural logic @xcite @xcite is its ability to capture a wide range of logical relationships inherent in natural language. by leveraging these relationships, our methodology provides nuanced logical guidance throughout the process of entailment tree generation. figure 1 illustrates an example of how a hypothesis and a set of facts lead to the structured explanation of a reasoning process. as illustrated in figure 1 , our method constructs an entailment tree by progressively combining relevant facts to reach the hypothesis. at each step of reasoning, the model selects a set of premises from the candidate sentences and generates an intermediate conclusion. the logical consistency of this inference is evaluated using natural logic operations (e.g., entailment, alternation, contradiction), and the resulting logical relation is fed into a deterministic finite automaton (dfa) to determine whether the step contributes to a valid reasoning path. the natural logic-based dynamic reward is computed from this evaluation and used to guide the reinforcement learning process. this mechanism enables the model to receive fine-grained feedback at each reasoning step, ensuring that the constructed entailment tree adheres to a globally consistent logical structure rather than only optimizing for the final output. employing entailment scores derived from natural logic as reward values enables the model to assess the logical consistency of each reasoning step, ensuring that the reasoning trajectory remains both precise and consistent. this dynamic reward mechanism also allows the model to evaluate the logical soundness of intermediate steps. moreover, the incorporation of natural logic-based rewards enhances the interpretability and transparency of the reasoning process, as these rewards directly reflect clear and explainable logical relationships. the principal contributions of this work are delineated as follows: ## method we formulate structured reasoning as a reinforcement learning (rl) task, aiming to learn an optimal reasoning policy. in rl, an agent makes sequential decisions to maximize the expected cuformally, we express the state as: s t " th, t t , u t u. ## conclusion in this paper, we introduced the nldr method, which leverages natural logic to compute dynamic entailment scores between intermediate nodes and the root node in an entailment tree, offering a more nuanced and adaptable reward function for reinforcement learning compared to traditional fixedvalue rewards. our experimental results demonstrate that nldr not only enhances the quality of entailment tree generation but also exhibits significant improvements in logical consistency, reasoning ability, and overall model accuracy. future work could focus on further optimizing the reward function design, improving model interpretability and scalability, and exploring the application of this approach to a broader range of natural language inference scenarios, ultimately advancing the development of intelligent reasoning systems.