Title: MAGO: Beyond Fixed Hyperparameters with Multi-Objective Pareto Optimization for Hybrid LLM Reasoning
Primary area: reinforcement learning
Keywords: Multi-objective optimization, Pareto optimization, Large language models, Hybrid reasoning, Chain-of-thought reasoning, Reinforcement learning

Abstract: Large language models (LLMs) with advanced step-by-step reasoning abilities have demonstrated exceptional performance in complex problem-solving through chain-of-thought (CoT) reasoning. However, applying elaborate reasoning processes uniformly to all queries creates significant computational inefficiency, as many problems could be solved more directly without extended reasoning chains. To address this challenge, we propose MAGO (Multi-objective Adaptive Generation Optimization), a principled framework that reformulates hybrid reasoning as a multi-objective optimization problem grounded in Pareto optimality theory. MAGO dynamically balances three competing objectives: accuracy (maintaining solution correctness), efficiency (minimizing computational costs), and calibration (ensuring mode selection aligns with model capabilities). Unlike existing approaches that rely on static hyperparameters, our framework employs adaptive weight selection through Pareto frontier maintenance with correlation-aware optimization to automatically determine optimal trade-offs based on task complexity and model performance. The system integrates seamlessly into existing hybrid reasoning training through end-to-end optimization, avoiding manual hyperparameter tuning while preventing the spatial constraints that limit single-objective approaches. Evaluation on mathematical reasoning benchmarks including AIME, Minerva Algebra, MATH-500, and GSM-8K demonstrates that MAGO reduces computational costs by 15-20\% while maintaining accuracy, providing principled trade-offs across diverse problem-solving scenarios.