Logical DA : Enhancing Data Augmentation for Logical Reasoning via a Multi-Agent System (2025)

Abstract: Recent advancements in large language models (LLMs) have highlighted the importance of improving their reasoning capabilities. A critical challenge lies in the scarcity of high-quality reasoning data—characterized by diversity and rich supervisory signals—necessary for robust model training. While data augmentation (DA) methods have been leveraged to mitigate this scarcity, prevailing approaches often introduce noise and exhibit logical inconsistencies, thereby diminishing their utility for complex reasoning tasks. Moreover, existing DA paradigms predominantly isolate data synthesis from label validation, failing to unify these complementary processes within a cohesive architecture.To address these limitations, we introduce Logical DA, a multi-agent framework for enhancing reasoning-focused data augmentation in few-shot learning scenarios. Our system includes four agents operating through two synergistic phases: (1) diverse data generation, and (2) label verification.The system incorporates a reflection mechanism to continuously improve data quality by leveraging feedback from logical validation. We demonstrate the effectiveness of Logical DA through experiments on various tasks and datasets, achieving the highest average improvement in task accuracy in both fine-tuning and in-context learning paradigms, with an average improvement of 7.61% when applied to fine-tuning.

## introduction researchers have been focusing on the reasoning capability of large language models (llms) recently @xcite , where a key challenge is the limitation of high-quality reasoning data for model training. the ideal training data of reasoning is diverse and is rich in supervision information @xcite , which requires costly manual annotations @xcite . in view of this challenge, researchers have increasingly employed data augmentation (da) techniques that enrich the dataset to alleviate the issue of data scarcity @xcite . facilitated by the outstanding text generation capabilities of llms, researchers have employed llms for da, including data generation derived from the original dataset and prompt-based label assignment @xcite , which have been experimentally verified effective @xcite @xcite . driven by the target tasks, such augmented data is then used for performance enhancement, via model training or integration into in-context learning (icl) within the llm @xcite . however, due to the inherent hallucination of llms and the lack of verification for generated labels, methods that rely solely on llms for data generation @xcite often produce noisy data, manifested as factual inaccuracies, label mismatches, or irrelevant content @xcite , which sabotages the effectiveness of reasoning data augmentation, as shown in figure 1 . moreover, the generated data often suffers from a lack of diversity that may jeopardize the model's generalization ability, which can be partially alleviated by controlled prompting @xcite . another challenge is that existing augmentation techniques tend to focus on either data creation or data annotation, where a unified framework that effectively integrates both tasks is desired. in this paper, we argue that an effective data augmentation framework, which aims to generate high-quality and logically sound data, requires two key capabilities: generating diverse samples and verifying logical correctness. thus, inspired by the advantages of llm-based multi-agent systems in various domains @xcite , we propose a multi-agent system to enhance data augmentation in few-shot scenarios. to jointly optimize data diversity and label consistency, we architect a multi-agent framework comprising four collaborative agents, structured into two main phases: our contributions are three-fold as follows: differing from the aforementioned works that directly apply methods to enhance model reasoning capabilities, our objective is to employ symbolic reasoning to augment logical data. additionally, we integrate the model's independent symbolic reasoning with the invocation of symbolic solvers, thereby more comprehensively ensuring the logical reasoning process. ## preliminary problem definition our goal is to build a multiagent system for data augmentation in logical reasoning tasks. this system aims to enhance the generation of high-quality and diverse reasoning data to improve the model's reasoning abilities. we define an llm agent @xmath0, specified by a unique identifier label l and a mapping f : @xmath1. each agent a i is specialized in a specific subtask and is powered by an llm l i . for a text dataset @xmath2, y i )} k @xmath3, where each sample consists of an input text x i and a corresponding label y i , data augmentation (da) using large models involves utilizing prompts p to guide llms in generating novel augmented data from the given input text x i as follows: logical reasoning data for a logical reasoning sample (x i , y i ) ∈ d, the input text x i typically consists of two text pairs (p re i , hyp i ), where p re i is a set of premises p i , p 2 , ...p n with each p i represents a logical statement, and hyp i is usually a statement that we aim to evaluate based on the premises. the label y i for logical data refers to the logical relationship between the p re i and hyp i . ## conclusion in this study, we propose logical da, a multi-agent framework that enhances logical reasoning data augmentation in few-shot settings. the system combines two core functions: generating diverse reasoning data and verifying logical correctness through label validation. by coordinating four specialized agents, it resolves key challenges in existing methods-data repetition, label mismatches, and limited diversity. experiments demonstrate consistent performance gains in both fine-tuning and in-context learning tasks, advancing language models' reasoning capabilities while reducing reliance on costly manual annotation. future work will aim to explore applying our system to other multi-agent frameworks to mitigate the negative impacts caused by contextual logical conflicts in multi-agent collaboration. also, follow researches can extend this paradigm to multimodal reasoning tasks, and optimize computational efficiency for real-time deployment.