Title: MAT-Agent: Adaptive Multi-Agent Training Optimization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multi-Agent，Multi-Label，Bandit

Training-Optimization
Abstract: We propose a novel collaborative multi-agent optimization framework for adaptive training in multi-label image classification, fundamentally advancing beyond static decision rules and isolated automation. Our method deploys a set of distributed, task-specific agents, each responsible for dynamically orchestrating critical training components—including data augmentation, optimization methods, learning rate schedules, and loss functions—according to evolving visual-semantic relationships and training states. Each agent employs an advanced non-stationary multi-armed bandit algorithm, integrating both $\epsilon$-greedy and upper confidence bound strategies, to judiciously balance exploration with exploitation throughout the training lifecycle. A hierarchical composite reward mechanism synergizes overall classification accuracy, rare class recognition, and training stability, fostering both independent optimization and implicit collaborative behavior among agents. The framework further leverages refined techniques such as dual-rate exponential moving average smoothing and structured mixed-precision training to enhance robustness and computational efficiency. Extensive experiments across benchmarks including Pascal VOC, COCO, Yeast, and Mediamill demonstrate that our approach achieves superior mean average precision and rare-class F1 scores compared to state-of-the-art methods, while also exhibiting rapid convergence and remarkable cross-domain generalization. Our results indicate that collaborative multi-agent adaptive optimization offers a scalable and principled solution for self-optimizing deep learning in complex multi-label scenarios.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces MAT-Agent, a novel framework for Multi-Label Image Classification (MLIC) that treats the training process as a dynamic optimization problem solved by multiple collaborating agents.  The core idea is to move away from fixed, pre-defined training configurations. Instead, the framework deploys four autonomous agents, each responsible for dynamically selecting a strategy for a key training component: data augmentation, optimizer, learning rate schedule, and loss function. These agents are modeled as reinforcement learning agents, specifically using a value-based approach akin to Deep Q-Networks, which learn to make sequential decisions at each stage of training. Their collective goal is to maximize a shared, composite reward signal that balances accuracy (mAP), performance on rare classes, and overall training stability. The authors demonstrate through extensive experiments on Pascal VOC, COCO, and VG-256 datasets that MAT-Agent outperforms a wide range of state-of-the-art MLIC models and other automated training methods, showing superior final performance, faster convergence, and strong cross-domain generalization.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1.  **Originality & Significance:** The primary strength of this paper is its novel conceptualization of the MLIC training pipeline. Framing hyperparameter optimization as an online, multi-agent sequential decision-making problem is a significant departure from conventional static configurations or computationally expensive offline search methods like grid search and AutoML. This dynamic, adaptive approach addresses the non-stationary nature of the training process, where optimal strategies may change as the model learns. This is a promising and original direction for building more intelligent and efficient training systems.\n\n2.  **Quality (Technical & Experimental):** The proposed method is technically sound, grounding the agent decision-making process in established reinforcement learning principles like value function approximation (DQN) and the exploration-exploitation trade-off. The experimental evaluation is comprehensive and provides strong evidence for the method's effectiveness. The authors benchmark against numerous strong baselines on three distinct and widely used datasets (Pascal VOC, COCO, VG-256), consistently demonstrating state-of-the-art performance. The inclusion of convergence analysis, cross-dataset generalization experiments, and detailed ablation studies significantly strengthens the paper's claims and demonstrates the robustness of the approach.\n\n3.  **Clarity:** The paper is well-written and clearly structured. The motivation for moving beyond static training paradigms is well-articulated. The methodology section systematically breaks down the problem formulation, the MAT-Agent framework, the state-action-reward setup, and the learning algorithm. Figures are used effectively to illustrate the framework architecture, compare convergence curves, and visualize ablation results, which aids in understanding the paper's contributions.\n\n**Weaknesses:**\n\n1.  **Clarity & Reproducibility:** While generally clear, the paper defers many critical implementation details to the supplementary material. Key design choices, such as the complete action spaces for each agent, the precise construction of the state vector, the architecture of the Q-networks, and the weighting of the composite reward function, are not available in the main text. This makes it difficult to fully assess the methodology and reproduce the results without constant reference to the appendix, hindering the self-containedness of the paper.\n\n2.  **Overhead and Efficiency Claims:** The paper claims to operate \"without added search overhead\" compared to methods like AutoML. This is potentially misleading. The framework introduces four neural network-based agents that must be trained, and the system must perform state extraction, forward passes through the policy networks, and updates to these networks at each decision step. While Figure 4 shows MAT-Agent is highly efficient in terms of wall-clock time to convergence, the computational overhead per epoch is non-trivial and is not explicitly analyzed. A more transparent discussion of the computational trade-offs is needed.\n\n3.  **Multi-Agent Coordination:** The agents are described as decentralized but are coordinated via a single, global reward signal $R_{t+1}$. This setup is prone to the multi-agent credit assignment problem, where it is difficult to determine which agent's action contributed to a given reward. The paper does not discuss how this challenge is addressed. The ablation study on \"Agent Coordination\" shows its importance, but it is unclear what this mechanism entails beyond the shared reward, making the \"collaboration\" aspect under-specified. The conclusion's mention of \"structured communication channels\" is not substantiated in the methodology.\n\nQUESTIONS:\n#### **Questions**\n\n1.  **Multi-Agent Credit Assignment and Coordination:** The agents learn via a shared global reward, which can make credit assignment difficult. Your ablation study shows that disabling \"Agent Coordination\" degrades performance. Could you please elaborate on what this coordination mechanism consists of beyond the shared reward signal? For instance, does it involve value decomposition methods (e.g., VDN, QMIX, which you cite) or another form of communication between agents? Clarifying this would strengthen the multi-agent aspect of your contribution. My evaluation would increase if the coordination mechanism is technically novel and well-justified.\n\n2.  **Computational Overhead Analysis:** Could you provide a more detailed breakdown of the computational cost of MAT-Agent per training epoch? Specifically, what is the overhead (in terms of time or FLOPs) associated with (a) extracting the state representation $\\mathcal{I}_{t}$, (b) performing forward passes for the four agent policies, and (c) updating the agent networks, relative to the cost of a standard training update for the main MLIC model? A transparent analysis of this overhead would provide a more nuanced understanding of the method's efficiency and would help clarify the claim of operating \"without added search overhead\".\n\n3.  **Sensitivity to Reward Formulation:** The composite reward function (Eq. 3) uses weights ($w_{mAP}, w_{stab}, w_{conv}, w_{pen}$) to balance multiple objectives. The performance of RL agents is often sensitive to reward design and weighting. Could you provide an ablation study or analysis on how performance changes with different settings of these weights? Demonstrating robustness to these hyperparameters or providing a principled way to set them would significantly increase the practical value of your framework.\n\n4.  **State Representation Components:** The state vector $s_t$ includes performance metrics, training dynamics, and data descriptors like \"average texture richness\". The inclusion of data-specific features is interesting. Could you provide an ablation study on the contributions of the different components of your state vector ($s_t^{perf}$, $s_t^{dyn}$, $s_t^{data}$)? This would provide valuable insight into which signals are most informative for the agents when making their decisions.\n\nLIMITATIONS:\nThe authors mentioned the limitations of the work and proposed future directions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes MAT-Agent, a novel multi-agent framework that reimagines training as a collaborative, real-time optimization process.  It leverages non-stationary multi-armed bandit algorithms to balance exploration and exploitation, guided by a composite reward harmonizing accuracy, rare-class performance, and training stability by deploying autonomous agents to dynamically tune data augmentation, optimizers, learning rates, and loss functions.  It also ensures robustness and efficiency through enhancing with dual-rate exponential moving average smoothing and mixed-precision training.  Experiments demonstrate the effectiveness of the proposed method.  The main contributions of this paper are:\n\n- It reconceptualizes the training process for multi-label image classification (MLIC) as a multi-agent, continual learning and decision-making problem, where each decision stage is governed by principles rooted in the classic exploration-exploitation trade-off.\n- It introduces four autonomous and adaptive agents—each responsible for dynamically controlling one of the core training components: data augmentation, optimizer selection, learning rate scheduling, and loss function design.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n\n- The problem studied in this paper is interesting and valuable.\n- The paper is well-organized and clearly written, which is easy to follow.\n- The experimental results are somehow promising.\n\n\n\n**Weakness**\n\n- The paper does not include any experiments in which all four decision dimensions (data augmentation, optimizer choice, learning‑rate schedule, and loss‑function selection) are collapsed into a single RL agent with a joint action space.  Without this direct comparison, it is difficult to quantify how much of the observed performance gain truly stems from the multi‑agent decomposition versus simply applying RL.\n- While the authors claim that splitting into four agents reduces the per‑agent action space from exponential to linear size, they do not provide concrete numbers or plots that show how the size of the joint action space versus the individual action spaces impacts sample complexity or convergence speed.\n\nQUESTIONS:\n1.  Can the authors provide results for a single‑agent RL baseline in which the four decision axes are treated as one combined action space?\n\n2.  What are the exact sizes of the joint action space versus each individual agent’s action space?\n\nLIMITATIONS:\nThe authors could further discuss the potential societal impact of the proposed method.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes MAT-Agent, a reinforcement learning-based multi-agent system that dynamically optimizes the training process for multi-label image classificatio. Instead of relying on static configurations for components like data augmentation, optimizers, learning rate schedules, and loss functions, MAT-Agent treats training as a sequential decision-making problem. Four autonomous agents collaboratively adjust these components in real time based on the current training state. This dynamic strategy improves convergence speed, training stability, and generalization. Extensive experiments on widely-used datasets show that MAT-Agent achieves state-of-the-art performance, outperforming existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is well-written and easy to follow.\n2. The proposed MAT-Agent framework is novel and promising in its attempt to dynamically optimize the training process using reinforcement learning.\n3. The method achieves strong performance across multiple benchmarks, demonstrating its effectiveness.\n\nWeaknesses:\n1. Lack of Task-Specific Design for Multi-Label Image Classification (MLIC): Although the paper focuses on MLIC, the proposed MAT-Agent framework appears to be a general training optimization strategy that could be applied to a wide range of tasks, such as image classification, semantic segmentation, action detection, or pose estimation. It is unclear why the authors specifically target MLIC. If the framework is not tailored to the unique challenges of MLIC (e.g., label correlations or long-tail distribution), then evaluating it on more fundamental and general tasks would better demonstrate its versatility.\n2. Unclear Cooperation Mechanism Among Agents: The paper claims that the novelty lies in the cooperation between multiple agents. However, after a thorough reading, the interaction mechanisms among agents remain vague. The current description gives the impression that the four agents operate independently and are simply trained in parallel. There is no clear explanation of how the agents interact, coordinate, or enhance each other’s learning processes. Without an explicit cooperation strategy, the claimed multi-agent collaboration appears superficial.\n3. Insufficient Literature Review: The core idea of using reinforcement learning to dynamically adjust training processes has been explored in prior work. For example, [1] and [2] propose RL-based agent frameworks for active learning, which share conceptual similarities with this paper. The authors should position their work more clearly in relation to such existing literature and clarify the distinctions or advantages of their approach.\n\nReferences:\n[1] Reinforced Active Learning for Image Segmentation\n[2] Meta Agent Teaming Active Learning for Pose Estimation\n\nQUESTIONS:\nRefer to weakness\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}