Paper: Solving Multiobjective Combinatorial Optimization via Learn to Improve Method (ICLR-2024)

Reviewer comment (Novelty):
While the authors have commendably applied the L2I approach to MOCOPs, yielding impressive results, the novelty of the ideas underpinning this work doesn't fully meet the expectations I have for ICRL publications.

Author response:
Although there are similar methods previously proposed for single-objective combinatorial optimization problems, it is challenging to modify these methods for MOCOPs. We present the initial attempt of using the learn-to-improve framework. It supports collaboratively improvement of multiple solutions and has achieved promising results.

Reviewer comment (Novelty):
The concept of utilizing a trained neural model to bolster the local search component of a genetic algorithm isn't novel.

Author response:
DACT is one of the learn-to-improve methods for single-objective COPs, which cannot directly address MOCOPs. In principle, any advanced neural model can be aggregated in our weighted-related policy network, where DACT is just a building block. When directly training multiple DACT models to deal with the decomposed subproblems for MOCOPs, it costs more computational resources and exhibits poor performance compared with our model, as shown in Table 3.

Reviewer comment (Novelty):
The presented neural net architecture appears to be a minor variation of an existing one, specifically DACT.

Author response:
The shared baseline in L2I is derived by solving multiple subproblems for a MOCOP simultaneously and averaging their solutions. On the other hand, the shared baseline in POMO is obtained by solving a single-objective problem multiple times and averaging the results. However, applying POMO strategy to the multi-objective domain is challenging, because individually solving each scalar subproblem multiple times would consumes a significantly increasing amount of memory and runtime, making it impractical.

Reviewer comment (Novelty):
The employment of a shared baseline for REINFORCE isn't groundbreaking, as seen in POMO.

Author response:
The vanilla instance augmentation approach for single-objective COPs exploits equivalent instance transformation, which could be directly applied to the decomposed subproblems for MOCOPs, as in [1]. Different from the vanilla instance augmentation approach, our quality enhancement approach is specifically tailored for MOCOPs, further utilizing the external population of MOEAs and the Pareto dominance of solutions. The comparison results in Table 4 show our quality enhancement approach can better improve the solution quality compared with the vanilla instance augmentation approach.  

[1] Xi Lin, Zhiyuan Yang, and Qingfu Zhang. Pareto set learning for neural multi-objective combinatorial optimiza- tion. In International Conference on Learning Representations, 2022.

Reviewer comment (Novelty):
Quality enhancement via instance augmentation isn't a pioneering approach.

Author response:
The vanilla instance augmentation approach for single-objective COPs exploits equivalent instance transformation, which could be directly applied to the decomposed subproblems for MOCOPs, as in [1]. Different from the vanilla instance augmentation approach, our quality enhancement approach is specifically tailored for MOCOPs, further utilizing the external population of MOEAs and the Pareto dominance of solutions. The comparison results in Table 4 show our quality enhancement approach can better improve the solution quality compared with the vanilla instance augmentation approach.  

[1] Xi Lin, Zhiyuan Yang, and Qingfu Zhang. Pareto set learning for neural multi-objective combinatorial optimiza- tion. In International Conference on Learning Representations, 2022.

Reviewer comment (Writing):
Branding the methodology in this paper as "L2I" might lead to confusion in the future, as subsequent research that applies the L2I approach to MOCOPs could create ambiguity and make it challenging for future works to distinguish between this specific approach and other "L2I" methodologies for MOCOPs when comparing results.

Author response:
Thank you for your suggestion. We have taken it into consideration and decided to revise the name of our model to Weighted-Related Policy Network (WRPN). This change aims to prevent potential confusion with other L2I approaches.

Reviewer comment (Reproducibility):
The way the proposed weight-related policy network is embedded is not clearly described. A pseudo-code of the complete method for solving one multi-objective combinatorial optimization problem should be provided.

Author response:
Thanks for your suggestions. Initially, we presented a detailed explanation of the proposed weight-related policy network in Sections 4.2 and a pseudo-code outlining the L2I inference process in Algorithm 2 in Appendix G. In response to your feedback, we will include pseudo-code detailing the computation of the weight-related policy network when addressing an MOCOP with L2I, as demonstrated in Algorithm 2.

Reviewer comment (Reproducibility):
The proposed method contains several hyperparameters, e.g., the number of transformer-style stacked encoders and the number of attention heads. A summary of them needs to be provided.

Author response:
Thank you for your suggestion. We will include a list of hyperparameters in Appendix I, aiming to enhance the understanding of the model's specific parameters.

Reviewer comment (Experiments):
How does the algorithm perform when using complex operators?

Author response:
It is possible to employ more complex operators to further enhance the search process, but this may require additional computational resources during both training and inference. In our methodology, we utilize an ensemble operator consisting of three basic ones. The results of the ablation study are presented in Table 8 in Appendix F.1. It is evident from the results that the ensemble operator, being a more complex operator, surpasses the performance of three basic operators.

Reviewer comment (Experiments):
What are the numbers of variables for the test problems in the experiments?

Author response:
The number of decision variables is closely related to the length of the decision sequence. In the case of MOTSP, the number of decision variables equals the number of nodes. In our paper, we consider three specific numbers: 20, 50, and 100. For MOCVRP, additional dummy depots are included in the decision sequence, as depicted in Appendix B.2. In our analysis, we evaluate three sizes of decision variables: 30, 70, and 120, corresponding to 20, 50, and 100 nodes, respectively.

Reviewer comment (Novelty):
How does L2I compare with methods follows a general scheme of "learn-divide-and-conquer" or "divide-learn-and-conquer"? In a sense, there are approaches that learn how to decompose a problem and there are approaches that decompose a problem before learning. I qualitative assessment may be sufficient.

Author response:
Thank you for your valuable comments.

Many multiobjective algorithms first decompose the MOCOP into a number of scalar optimization subproblems, and then optimize them simultaneously. To the best of our knowledge, approaches that learn how to decompose a problem are rare. Exploring this concept could present an interesting direction for further investigation. On the other hand, our L2I approach focuses on learning improvement strategies for each individual subproblem. It belongs to the scheme of “divide-learn-and-conquer”.

Reviewer comment (Presentation):
In the result tables, the proposed method is not highlighted.

Author response:
According to your suggestions, we have highlighted our proposed method in italic in the tables.

Reviewer comment (Evaluation):
The results in tables 1 & 2 are not discussed on why the proposed approach is better only in the last entries.

Author response:
In terms of the improved outcomes reported in the tables, there are two points to clarify.

Firstly, various MOEA frameworks, when augmented with our L2I approach, retain their distinct original characteristics. Notably, MOGLS, designed specifically for multi-objective combinatorial optimization, achieves the best result. However, MOEA/D and NSGA-II, originally proposed for general multi-objective optimization, exhibit comparatively lower performance.

Secondly, one of the results obtained from MOGLS+L2I in the last entry benefits from a larger number of iterations and the incorporation of a quality enhancement mechanism. As a result, this particular configuration demonstrates the best performance.