Preference Conditioned Neural Multi-objective Combinatorial Optimization

1 INTRODUCTION . Many real-world applications can be modeled as different multiobjective combinatorial optimization ( MOCO ) problems ( Ehrgott & Gandibleux , 2000 ) . The multiobjective traveling salesman problem ( MOTSP ) ( Lust & Teghem , 2010a ) , multiobjective vehicle routing problem ( MOVRP ) ( Jozefowiez et al. , 2008 ) and multiobjective knapsack problem ( MOKP ) ( Bazgan et al. , 2009 ) are three widely studied problems in the research community . These problems have multiple conflicting objectives to optimize , and no single solution can optimize all the objectives at the same time . Instead , there is a set of Pareto solutions with different optimal trade-offs among the objectives . It is very challenging to find all exact Pareto optimal solutions for a MOCO problem . Finding only one Pareto solution would be already NP-hard for many problems ( Ehrgott & Gandibleux , 2000 ) , and the number of Pareto solutions could be exponentially large to the problem size ( Ehrgott , 2005 ; Herzel et al. , 2021 ) . The decision-maker ’ s preference among different objectives is usually unknown in advance , making it hard to reduce the problem into a single-objective one . Over the past several decades , many methods have been developed to find an approximate Pareto set for different MOCO problems within a reasonable computational time . These methods often need carefully handcrafted and specialized heuristics for each problem , which can be very labor-intensive in practice . In many real-world applications , the practitioners need to repeatedly solve different problems with similar characteristics , of which the problem instances can be easily obtained or generated ( Bengio et al. , 2020 ) . It is desirable to learn the patterns behind these problem instances explicitly or implicitly to design efficient algorithms ( Cappart et al. , 2021a ) . Machine learning techniques can be naturally served for this purpose . Some learning-based methods have been recently proposed for solving single-objective combinatorial optimization problems ( Bengio et al. , 2020 ; Vesselinova et al. , 2020 ; Mazyavkina et al. , 2021 ; Cappart et al. , 2021a ) . In this work , we extend the learning-based method to solve MOCO problems in a principled way as shown in Figure 1 . Our main contributions include : • We propose a novel neural combinatorial optimization method to approximate the whole Pareto set with infinite trade-offs via a single model , which allows decision-makers to directly generate any preferred trade-off solution without extra search . This novel and powerful property is not supported by any other learning-based or heuristics-based methods . • We develop an efficient end-to-end reinforcement learning algorithm to train the single model for all different preferences simultaneously , and a simple yet powerful active adaption method to handle out-of-distribution problem instances . • We conduct comprehensive experiments on MOTSP , MOVR and MOKP problems with different settings . The results show that our proposed method can successfully approximate the Pareto sets for different problems in an efficient way . It also significantly outperforms other methods in solution quality , speed , and/or model efficiency . 2 BACKGROUND AND RELATED WORK . Multiobjective Combinatorial Optimization ( MOCO ) . MOCO has been attracting much research effort from different communities over the past several decades ( Sawaragi et al. , 1985 ; Wallenius et al. , 2008 ; Herzel et al. , 2021 ) . There are two main approaches to tackle the MOCO problems : the exact method and the approximation method ( Ehrgott , 2005 ) . Exact methods could be impractically costly when , as it often happens , the MOCO problem is NP-hard and the number of Pareto optimal solutions is huge ( Florios & Mavrotas , 2014 ) . For this reason , many heuristics ( Jaszkiewicz , 2002 ; Ehrgott & Gandibleux , 2008 ) and approximation methods ( Papadimitriou & Yannakakis , 2000 ; Herzel et al. , 2021 ) have been developed to find a moderately sized set of approximated Pareto solutions with a reasonable amount of computational budget . However , these methods usually depend on carefully handcrafted designs for each specific problem ( Ehrgott & Gandibleux , 2000 ) , of which the effort would be nontrivial for a new problem in real-world applications . Machine Learning for Combinatorial Optimization . According to Bengio et al . ( 2020 ) , there are three main learning-based approaches for CO : learning to configure algorithms ( Kruber et al. , 2017 ; Bonami et al. , 2018 ) , learning alongside the algorithms ( Lodi & Zarpellon , 2017 ; Gasse et al. , 2019 ; Chen & Tian , 2019 ; Kool et al. , 2021 ) , and learning to directly predict the solutions ( Nowak et al. , 2018 ; Emami & Ranka , 2018 ; Larsen et al. , 2018 ) . Neural combinatorial optimization ( NCO ) belongs to the last category where the training model directly predicts a good solution for a given problem instance . Vinyals et al . ( 2015 ) propose the pointer network to sequentially construct a solution for TSP problem . Bello et al . ( 2017 ) make a critical improvement to use reinforcement learning to train the model , eliminating the impractical optimal solutions collection for NP-hard problems . Further improvements on model structure and training procedure have been proposed in the past few years ( Nazari et al. , 2018 ; Deudon et al. , 2018 ; Kool et al. , 2019 ; Joshi et al. , 2020 ) , especially with graph neural networks ( GNNs ) ( Dai et al. , 2017 ; Li et al. , 2018 ; Joshi et al. , 2019 ; Dwivedi et al. , 2020 ; Drori et al. , 2020 ) . Some works have recently been proposed to design more powerful training strategies , such as optimization with multiple optimum ( Kwon et al. , 2020 ) , unsupervised learning ( Karalias & Loukas , 2020 ) and curriculum learning schemes ( Lisicki et al. , 2020 ) . Neural MOCO . The current learning-based methods are mostly for solving single-objective combinatorial problems . Recently , a few attempts have been made to solve MOCO problems ( Li et al. , 2020 ; Wu et al. , 2020 ; Zhang et al. , 2021b ; a ) . They first decompose a MOCO problem into multiple single-objective scalarized subproblems , and then build a set of models for each subproblem . However , since the number of Pareto solutions would be exponentially large ( Ehrgott , 2005 ) , the required number of models would be huge for finding the whole Pareto set . This work proposes a single preference-based model for solving MOCO problems , where the decision-makers can obtain arbitrary trade-off solutions . The proposed single neural MOCO solver could be much easier to use in a real-world system ( Veličković & Blundell , 2021 ) , rather than a large set of separate models . Our method ’ s model efficiency is crucial to truly extend the NCO approach to solve MOCO problems . 3 PROBLEM FORMULATION . 3.1 MULTIOBJECTIVE COMBINATORIAL OPTIMIZATION . A multiobjective combinatorial optimization ( MOCO ) problem can be defined as follows : min x∈X F ( x ) = ( f1 ( x ) , f2 ( x ) , . . . , fm ( x ) ) , ( 1 ) where X is a discrete search space , and F ( x ) = ( f1 ( x ) , . . . , fm ( x ) ) is an m-objective vector . Since the individual objectives often conflict with each other , no single solution can optimize all of them at the same time . Thus , practitioners are interested in Pareto optimal solutions , defined as follows . Definition 1 ( Pareto Dominance ) . Let xa , xb ∈ X , xa is said to dominate xb ( xa ≺ xb ) if and only if fi ( xa ) ≤ fi ( xb ) , ∀i ∈ { 1 , ... , m } and fj ( xa ) < fj ( xb ) , ∃j ∈ { 1 , ... , m } . Definition 2 ( Pareto Optimality ) . A solution x∗ ∈ X is a Pareto optimal solution if there does not exist x̂ ∈ X such that x̂ ≺ x∗ . The set of all the Pareto solutions is called the Pareto set , and the image of the Pareto set in the objective space is called the Pareto front . Each Pareto solution represents an optimal trade-off among the objectives , and it is impossible to further improve one of the objectives without deteriorating any other objectives . For many MOCO problems , the size of Pareto set would be exponentially large with respect to the input size ( e.g. , nodes in MOTSP ) , which makes finding the whole Pareto set intractable ( Herzel et al. , 2021 ) . Therefore , the current heuristic-based and learning-based methods all focus on finding a small subset of approximate Pareto solutions { x1 , x2 , · · · , xp } ( e.g. , p = 101 ) . The decision-makers can only select solutions from this fixed small set , which is unlikely to contain their preferred solutions precisely . 3.2 PREFERENCE-BASED SCALARIZATION . Preference-based scalarization is a widely-used approach to solve MOCO problems ( Ehrgott , 2005 ) . It decomposes a given multiobjective problem into several subproblems with different preferences . By solving all subproblems , we can obtain a set of approximated Pareto solutions . For an mobjective problems , we can define a preference vector λ ∈ Rm satisfies λi ≥ 0 and ∑m i=1 λi = 1 such that λi is the preference for the i-th objective . Weight-Sum Scalarization is the most straightforward approach : min x∈X gws ( x|λ ) = min x∈X ∑m i=1 λifi ( x ) , ( 2 ) where gws ( x|λ ) is a single-objective subproblem with preference λ . The decision-makers can assign their preferences among the objectives , and obtain different subproblems . However , the weightedsum method can only find solutions on the convex hull of the Pareto set ( Ehrgott , 2005 ) . Weighted-Tchebycheff ( Weighted-TCH ) Scalarization is an alternative : min x∈X gtch ( x|λ ) = min x∈X max 1≤i≤m { λi|fi ( x ) − ( z∗i − ε ) | } , ( 3 ) where z∗i is the ideal value for objective fi ( x ) ( e.g. , the lower bound ) , and u ∗ i = z ∗ i − ε is a utopia value with small positive component ε . According to Choo & Atkins ( 1983 ) , any Pareto optimal solution could be an optimal solution of problem ( 3 ) with a specific ( but unknown ) preference λ . To find the whole Pareto set , it still needs to solve an exponentially huge number of subproblems . Drawbacks and Remarks . Although the scalarization methods are straightforward , some critical drawbacks can not be ignored . First of all , not all scalarized multiobjective problem is the same as its single-objective version ( Ehrgott , 2005 ) . For MOTSP , each weight-sum scalarized subproblem is a single objective TSP so that a powerful TSP solver can solve it . However , it is not the case for MOCVRP . Secondly , although the Tchebycheff scalarization has good properties , it involves min-max optimization , which is typically different from the original problem . For example , the Tchebycheff scalarized MOTSP is not a single-objective TSP and can not be solved by the state-ofthe-art single-objective solver developed in the current years . In our reinforcement learning based framework , we use the Tchebycheff scalarization to combine the rewards of different objectives . It is much more flexible to use , and does not have to consider the problem-specific condition .

Summary: This paper proposes an approach for neural multi-objective combinatorial optimization. This approach uses a preference-agnostic encoder along with a weight-dependent decoder to generate approximate Pareto optimal solutions for any arbitrary set of weights of a weighted Tchebyshev scalarization at virtually no additional cost. This is in contrast with existing approaches, which require a significant amount of computation for every new set of weights. Several numerical experiments are conducted, showing favorable results for the proposed method when compared with several other existing evolutionary and learning-based methods.