Title: Preference-Driven Multi-Objective Combinatorial Optimization with Conditional Computation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multi-objective combinatorial optimization; preference optimization; conditional computation
Abstract: Recent deep reinforcement learning methods have achieved remarkable success in solving multi-objective combinatorial optimization problems (MOCOPs) by decomposing them into multiple subproblems, each associated with a specific weight vector. However, these methods typically treat all subproblems equally and solve them using a single model, hindering the effective exploration of the solution space and thus leading to suboptimal performance. To overcome the limitation, we propose POCCO, a novel plug-and-play framework that enables adaptive selection of model structures for subproblems, which are subsequently optimized based on preference signals rather than explicit reward values. Specifically, we design a conditional computation block that routes subproblems to specialized neural architectures. Moreover, we propose a preference-driven optimization algorithm that learns pairwise preferences between winning and losing solutions. We evaluate the efficacy and versatility of POCCO by applying it to two state-of-the-art neural methods for MOCOPs. Experimental results across four classic MOCOP benchmarks demonstrate its significant superiority and strong generalization.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a plug-and-play multi-objective reinforcement learning (RL) framework, POCCO. It consists of two major components.\n(1) Conditional Computation Block (CCO): It is a sparse router that employs a gated network, combined with a Top-k operator, to enable the model to decide which experts to activate. This can ensure the width and depth while remaining efficient. This aims to improve the exploration over pure REINFORCE.  \n(2) Preference-driven Learning: This method is inspired by the Bradley-Terry likelihood, where the weighted sum of rewards is replaced with the average log-likelihood gap between two randomly selected complete solutions. This aims to lower the gradient variance.\n\nThe experiments are done on bi/tri-objective TSP, CVRP, and Knapsack problems. The results look promising, and the backbone is improved when enhanced by their method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n(1) The method is a plug-and-play framework, and the two mechanisms are simple and elegant. So it would be easy to adapt to most multi-objective RL frameworks.\n\nWeaknesses:\n(1) The test problems are a bit \"easy\" since the backbone alone has already been very close to the optimal results. So the improvement seems minor. See more in the questions.\n(2) Some experimental settings are questionable. See more in the questions.\n\nQUESTIONS:\n1. It would be great if you had more experimental results on some more complex problems. The backbones are already performing quite well for now, so the improvements seem minor.\n\n2. For comparison, you have shown that with PL, your convergence is significantly faster than with pure REINFORCE. Then why not set the solving time the same for both the backbone and backbone + your method? This seems to be a better comparison.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes POCCO, a preference-driven multi-objective combinatorial optimization framework with conditional computation, which achieves performance improvements on MOCOPs. While the approach is innovative and empirically validated, several issues remain clarification.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The proposed approach is technically sound.\n- The paper introduces a CCO in the decoder that dynamically routes subproblems through specialized feed-forward or identity experts based on context. \n- The paper proposes a preference-driven optimization algorithm inspired by the Bradley–Terry model, which learns from pairwise solution comparisons rather than absolute rewards. \n- The proposed POCCO is evaluated on several multi-objective combinatorial optimization benchmarks.\n\n\nWeaknesses:\n- While related work is mentioned in the introduction and appendix, the paper lacks a dedicated literature review section. This makes it difficult to understand the development trajectory of existing MOCOP methods and POCCO’s positioning within the field.\n- The model complexity and computational cost are not clearly addressed, especially with the inclusion of multiple feed-forward experts and dynamic routing. Further clarification on training and inference efficiency—particularly in terms of parameter count, runtime, and scalability—is needed.\n- The concept of “Preference Learning” is vaguely defined.\n\nQUESTIONS:\n- Does the preference learning approach offer theoretical guarantees on better convergence or exploration efficiency compared to traditional REINFORCE?\n- Are the experimental results still valid for larger-scale problem instances? What are the primary factors contributing to this limitation\n- Does the introduction of multiple FF experts and dynamic routing significantly increase computational cost? Considering the potential need to handle more objectives, how do the authors envision scaling POCCO efficiently to accommodate high-dimensional objective spaces?\n- Can the process of generating preference pairs introduce bias?\nThe paper reports strong out-of-distribution generalization performance. Is the claimed generalization ability dependent on the training distribution?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes POCCO, a plug-and-play framework for Multi-Objective Combinatorial Optimization Problems (MOCOPs) using deep reinforcement learning. Unlike existing methods that rely on scalarized rewards and shared models across subproblems, POCCO introduces two key components: a conditional computation block that routes subproblems to specialized neural architectures, and a preference-driven optimization algorithm that leverages pairwise preferences.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. POCCO introduces a novel plug-and-play framework that effectively tackles key limitations of existing MOCOP methods, including insufficient exploration and limited model capacity. \n2. The paper provides substantial experimental verification across multiple classic MOCOP benchmarks, exhibiting strong generalization capabilities and excellent performance even when faced with unseen problem scales and out-of-distribution instances. \n\nWeaknesses:\n1. The paper lacks theoretical support for its design choices. For instance, it’s unclear how the CCO block enhances exploration or representation learning, or how preference learning mitigates gradient variance in REINFORCE.\n2. When generating preference pairs, the paper samples two candidate solutions from the policy and determines superiority based on the scalarized objective values. However, in complicated multi-objective settings, solutions can be mutually non-dominated.\n\nQUESTIONS:\n1. How is the routing mechanism in the CCO block learned? Specifically, how does the sparse gating network select between FF and ID experts, and what role does the Top-k operation play?\n\n2. When sampled solutions are non-dominated, how does the scalarized value determine the winner in preference learning? Could this bias the Pareto front exploration?\n\n3. The paper claims efficiency gains—could you break down the contributions of the CCO block and preference-driven training to the overall speedup?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces POCCO (Preference-driven multi-Objective Combinatorial optimization with Conditional COmputation), a framework that can be wrapped around existing neural solvers for multi-objective combinatorial optimization problems (MOCOPs).\nIn particular, POCCO adds two orthogonal components:\n1. Conditional Computation (CCO block): a sparse MoE / Mixture-of-Depth module inserted in the decoder that routes each scalarised sub-problem to a small, specialized subset of experts (or an identity path), efficiently scaling model capacity.\n2. Preference-driven Learning (PL): Instead of REINFORCE with scalar rewards, the policy is trained on pairwise “win/lose” comparisons via a Bradley-Terry likelihood, using normalized log-likelihood as an implicit reward.\n\nIntegrated into two SOTA baselines (CNH and WE-CA), the resulting POCCO-C and POCCO-W outperform all neural and classical baselines on four benchmarks (Bi-/Tri-TSP, Bi-CVRP, Bi-KP) and show better OOD size generalization. The authors also discuss remaining limitations, such as scaling to real-world constraints and very large instances.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n- Novelty: Simple, general mechanisms that can be “plugged in” to many existing models.\n- Experiments: 4 benchmarks × 3 sizes, 3 metric types, ablations on CCO variants and hyper-parameters, OOD tests.\n- Training efficiency: PL halves convergence time vs. REINFORCE.\n- Generalizability: Single model handles multiple sizes, beats multi-model baselines, and generalises to KroAB100-200 instances.\n- Paper clarity: Background and motivation sections relate CCO + PL to known MoE and preference-learning literature.\n\n## Weaknesses\n\n- Theoretical grounding: No formal analysis of convergence or expressivity. All results rely solely on empirical evidence (the authors acknowledge no new theory).\n- Experimental scope: Instances top out at 100 nodes for TSP/CVRP or 200 items for KP. Real-world constraints are left to future work.\n- Missing ablations: CCO adds parameters, yet the paper does not quantify the overhead of expert routing.\n- Writing: Several minor typos and grammatical errors that slightly hinder readability.\n\nQUESTIONS:\n## Questions\n\n1. How does performance vary with Top-k and the number of experts? Could very large expert pools cause over-fragmentation or collapse?\n2. Have you compared BT-likelihood with more recent objectives such as DPO?\n3. How would CCO interact with the constraint-handling modules suggested for future work? Could routing specialise not just by weight vector but by constraint pattern?\n\n## Possible typos\n\n- l.169-170 – “it combines of a mixture-of-experts…” → “combines a mixture-of-experts”\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nCurrent methods for multi-objective combinatorial optimization problems (MOCOPs) often rely on a single neural network to handle all subproblems with different weight vectors, resulting in a limited model capacity for diverse subproblems. On the other hand, current methods usually use REINFORCE and use scalarized objective values as the reward signals, introducing high variance. To address these issues, the paper introduces a novel method called POCCO, which increases model capacity by conditional computational (CCO) block and utilize preference-based reward for a better convergence. Experiments across several classic MOCOP benchmarks show that POCCO outperforms all the learning-based methods, with an increased but still acceptable computational time cost.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- The paper introduces both a new architecture and a new training method for MOCOPs, which are conceptually sound and well-motivated for MOCOPs.\n- In the experiment, the proposed method POCCO outperforms all the learning-based methods in terms of hyper-volume and gap. The time usage is longer but acceptable. The experiment is generally adequate.\n- The paper is clearly written, well-organized, and easy to follow.\n\nWeaknesses\n\n- The CCO block may introduce more computational overhead (including time and resource usage) compared to other learned-based methods. The longer computational time is evident in the reported experimental results.\n- The influence of the number of experts is not analyzed. It might be critical to the performance.\n- The source code is not yet available in the supplementary material, which may hinder the evaluation on reproducibility.\n\nQUESTIONS:\n- Could you provide a comparison on the resource usage (e.g., GPU memory usage) of the methods? This would provide a more complete understanding of the practical trade-offs.\n- Can POCCO be used in the tasks where preference among the objective cannot be presented as a linear weighted sum, but only a more complex combination (e.g. Tchebycheff scalarization)?\n\nI am willing to change my rating if all the concerns and questions is answered or addressed.\n\nLIMITATIONS:\nThe CCO block may require more computational cost (including time and resource usage) compared to other learned-based methods.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}