Title: MARFT: Multi-Agent Reinforcement Fine-Tuning
Primary area: reinforcement learning
Keywords: Multi-Agent Systems, Large Language Models, Reinforcement Fine-Tuning, Multi-Agent Reinforcement Learning

Abstract: The rapid rise of LLM-based agents has led to the emergence of LLM-based Multi-Agent Systems (LaMAS), which show strong potential in complex, collaborative tasks such as presentation generation and even scientific research. While Reinforcement Learning is well-established in enhancing LLM-based agent performance, its success has largely focused on single-agent settings. In contrast, applying Multi-Agent Reinforcement Learning to LaMAS remains limited. This is due to fundamental mismatches between traditional MARL assumptions and the unique dynamics of LaMAS, including action asynchronicity, dynamic organization, characteristic profiles, etc., which present significant new challenges. To address these challenges, we first formalize LaMAS optimization as a Flex-MG, capturing agent heterogeneity and interdependence, and then propose a novel paradigm termed Multi-Agent Reinforcement Fine-Tuning (MARFT), introducing a new optimization framework for LaMAS. Two naive instantiations of MARFT are implemented on the action-level and token-level, and experiments on math problem-solving and coding environments and benchmarks demonstrate MARFT’s effectiveness in improving accuracy and efficiency, establishing it as a principled and generalizable approach for tuning LaMAS. As this work establishes a new paradigm, we conclude by highlighting the limitations of current research and pinpointing promising directions for future work.