Title: Structuring Reasoning for Complex Rules Beyond Flat Representations
Primary area: foundation or frontier models, including LLMs
Keywords: Large Language Models, Complex Rule Systems, Reasoning Divergence, Rule-based Reasoning, Dynamic Adjudication Template

Abstract: Large language models (LLMs) face significant challenges when processing complex rule systems, as they typically treat interdependent rules as unstructured textual data rather than as logically organized frameworks. This limitation results in reasoning divergence, where models often overlook critical rule dependencies essential for accurate interpretation. Although existing approaches such as Chain-of-Thought (CoT) reasoning have shown promise, they lack systematic methodologies for structured rule processing and are particularly susceptible to error propagation through sequential reasoning chains. To address these limitations, we propose the Dynamic Adjudication Template (DAT), a novel framework inspired by expert human reasoning processes. DAT structures the inference mechanism into three methodical stages: *qualitative analysis, evidence gathering, and adjudication*. During the *qualitative analysis* phase, the model comprehensively evaluates the contextual landscape. The subsequent *evidence gathering* phase involves the targeted extraction of pertinent information based on predefined template elements ([placeholder]), followed by systematic verification against applicable rules. Finally, in the *adjudication* phase, the model synthesizes these validated components to formulate a comprehensive judgment. Empirical results demonstrate that DAT consistently outperforms conventional CoT approaches in complex rule-based tasks. Notably, DAT enables smaller language models to match, and in some cases exceed, the performance of significantly larger LLMs, highlighting its efficiency and effectiveness in managing intricate rule systems.

TL;DR: Our paper introduces the Dynamic Adjudication Template, a novel approach that enables structured reasoning in complex rule systems, empowering smaller models to outperform traditional methods such as CoT and to rival larger state-of-the-art LLMs.