Paper: PolyNet: Learning Diverse Solution Strategies for Neural Combinatorial Optimization (ICLR-2024)

Reviewer comment (Writing):
Please avoid using vague and less meaningful statements, such as 'has a wider application for ...', especially for the contribution.

Author response:
The statement is not meaningless. Many neural CO methods cannot be used on more than the TSP or some other simple CO problem without significant changes, if they can be used on other problems at all. This work seeks a problem independent method of generating diverse solutions. Even though we focused on routing problems, the mechanism itself is general and requires no modification for other types of CO problems (scheduling, assignment, etc.)

Reviewer comment (Theory):
The content for describing this new architecture seems not enough. The training process seems standard. In figure 2, it seems polynet only rely on some simple operations on the Bit Vector v. What such a operator will make such a big difference?

Author response:
Would you prefer it if our method would be more complicated and achieve the same results? We make a number of significant changes to the POMO and PolyNet method that lead to stark performance improvements. Achieving significant performance improvements while maintaining simplicity and efficiency is a commendable achievement.

Reviewer comment (Novelty):
In section 3.5, the proposed method is combined with an EAS. This makes the unique contribution of this work less clearer.

Author response:
We also show the performance of our method without EAS (where it outperforms all other ML methods). We do not see how providing additional results for PolyNet with EAS reduces our contribution. Not all neural CO methods can be combined easily with effective search techniques, thus it is an advantage of PolyNet that EAS can enhance its performance.

Reviewer comment (Writing):
Why named PolyNet, is anywhere mention it ?

Author response:
This is not mentioned in the paper. “PolyNet” uses the Greek word "poly," which means "many" or "multiple". We chose the name because PolyNet is able to generate multiple diverse solutions.

Reviewer comment (Novelty):
The improvement over POMO and others seems very small. Is 0.96% a enough contribution in this field?

Author response:
A 1% solution improvement in the area of vehicle routing is very significant as it often translates to tangible cost savings and enhanced resource utilization, contributing to overall efficiency. Even much smaller improvements can provide a competitive advantage.

Reviewer comment (Evaluation):
I'm curious whether the goal is to generate diverse solutions or to train a better-performing solver. If the aim is to obtain diverse solutions, it seems that there is a lack of mechanisms in the learning process to ensure that the K solutions have distinct characteristics.

Author response:
The goal of our approach is to generate better solutions for combinatorial optimization problems. Increasing solution diversity is only a means to an end. We now make this more clear in the paper.

Reviewer comment (Evaluation):
I believe that the approach focuses more on creating K distinct solutions and utilizing them to build a baseline and find the best solution, which is closer to an exploitation strategy for policy learning than generating diverse solutions.

Author response:
Did you mean to write “exploration strategy”? In that case, you are right.  Our approach creates diverse solutions with the intention to enable wider exploration of strategies and training and test time.

Reviewer comment (Evaluation):
What is the reason for the effectiveness of the shared baseline composed of K generated solutions?

Author response:
The baseline serves the purpose of stabilizing the training process. The adjustment of strategies is not based on the absolute costs achieved, as interpreting these costs can be challenging due to their dependency on both the solution and the instance. Instead, strategies are adjusted relative to how much better they performed compared to other evaluated strategies. This approach allows for a more meaningful and comparative assessment of the strategies, considering their performance in relation to each other rather than focusing solely on absolute cost values.

Reviewer comment (Experiments):
If K vectors are not used, and K solutions are generated using the same policy and the same approach, what would be the resulting outcome?

Author response:
Since the policy is conditioned on the additional input vectors, we can not use the same policy without providing K input vectors. Furthermore,the baseline requires a reasonable high value for K (>= 8) during training. At test time, we could sample solutions from only one of the K vectors which would result in decreased solution diversity.

Reviewer comment (Novelty):
The technical novelty is slightly limited. The idea of enhancing diversity (similar to EAS [3] and FER [4]) and the integration of additional trainable layers in the POMO decoder (similar to EAS [3] and PMOCO [5]) is not new.

Author response:
- The concept of enhancing solution diversity in a search algorithm or inserting new layers into an existing model isn't novel. Indeed, we can cite papers from far earlier than the papers provided in the ML literature. What sets our work apart is our ability to generate varied solutions using a model with a single decoder, showcasing significantly improved performance. This has a number of advantages over the approaches in the ML literature (which rely on multiple decoders) and the OR literature (which rely mostly on randomness). Thus, the novelty is absolutely sufficient for ICLR and we believe our decoder mechanism will have an influence on future autoregressive decoder designs.

Reviewer comment (Experiments):
The addressed TSP and CVRP are relatively small-scale. Many recent works can solve much larger sizes like 10,000 nodes, e.g., DIFUSCO [6].

Author response:
- For larger instances of the CVRP and CVRP with Time Windows (CVRPTW), there is currently a lack of end-to-end machine learning approaches. Existing methods, such as DIFUSCO, focus solely on the TSP due to its simpler constraints, enabling non-autoregressive solution generation. In our research, we concentrate on more intricate problems like CVRP and CVRPTW. Even with instances ranging from 100 to 500 nodes, achieving optimal solutions remains extremely challenging, emphasizing the continued importance of developing heuristics for problems of this magnitude. Note that common CVRP benchmarks from the operations research (OR) literature mostly have the same sizes as the problems we examine. This is because size is not the only criteria for the hardness of real-world problems. We encourage the reviewer to orient their thinking on the OR literature, which has been solving these problems for many years. Thus, our work serves to narrow the divide between machine learning and traditional OR methods by orienting the datasets on real problem sizes and structures.

Reviewer comment (Experiments):
It is unclear whether the good performance is attributed to EAS, additional training, or the added layers with bit vector inputs. More evaluation may be needed, e.g., testing PolyNet’s performance over longer inference without EAS, comparing the results of training from scratch and/or additional training (baselines also undergo extended training), and verifying the necessity of the proposed residual block design (e.g., can we change the bit inputs to random values? Can the hyper-network of PMOCO with additional training achieve similar performance?).

Author response:
- **Reason for the good performance**: We have already provided results for PolyNet without EAS in Table 4., affirming that its quality performance is not solely attributed to EAS. Additionally, despite Poppy undergoing identical training to PolyNet, it exhibits inferior performance on identical instances. This observation suggests that PolyNet's superior performance is not merely a result of additional training. Instead the enhanced performance of PolyNet is likely attributable to the novel model architecture and the refined training setup. Importantly, these two components are interdependent, and their effectiveness cannot be evaluated independently.\
**Comparison to PMOCO**: PMOCO is designed to solve multi-objective optimization problems and is hence not well-suited for the single objective problems considered in this paper.

Reviewer comment (Evaluation):
In Figure 4, it seems that both POMO and PolyNet display comparable levels of diversity, though PolyNet found solutions with lower costs. This appears to contradict the primary intention behind PolyNet, which is to amplify diversity. The results raise the question of whether the enhanced performance of PolyNet is due to diversity or if other underlying factors at play contribute to this outcome.

Author response:
- **Figure 4**: PolyNet does not only focus on solution diversity, but aims to increase solution diversity and solution quality alike. Figure 4 shows the trade-off between these two objectives. Overall, PolyNet finds solutions that are better than those of POMO with respect to these two objectives (also see Appendix A). To allow for a direct comparison of the solution diversity of POMO vs. PolyNet, we updated Figure 3 to include POMO.

Reviewer comment (Experiments):
The title and some claims have inaccuracies. For instance, the title and other sections mention “applicability to a wider range of CO problems”, but all experiments are conducted only on VRPs. The paper claims that it focuses on practical problems that have many constraints, but the experiments are limited to simple problems like CVRPTW and CVRP. The paper also mentions an "unlimited learnable policy", but the number of decoders is restricted by K.

Author response:
- **Considered problems**: We are currently working on including results for the flexible flow shop problem. The results will be included in the camera-ready version of the paper.
- **Practical problems**: The CVRPTW in particular is significantly more complex than most other problems considered in the neural combinatorial optimization literature. We hence see our work as an important step to more complex problems.
- **“Unlimited learnable policy”**: We rephrased this statement to make it more clear.

Reviewer comment (Evaluation):
The author already admitted the high computational complexity of the methods in the conclusion.

Author response:
**High computational complexity**: We solve problems of sizes relevant for many real-world contexts. Let us emphasize that the standard datasets in Operations Research for the CVRP, etc., also focus on problems of this size because this is what the vast majority of companies are interested in. As this work is trying to move the field of neural CO towards real-world impact, it makes sense that we would focus on these problems. Note that we are open about the scaling limitation of this work and state in the discussion that the computational complexity of our method limits its applicability to routing problems with less than 1,000 nodes.

Reviewer comment (Evaluation):
It seems to me that the PolyNet is not the fastest solver in Table 1, 2, 3.

Author response:
**Performance**: PolyNet outperforms all other state-of-the-art approaches for the routing problems we consider. In Tables 1, 2, and 3, PolyNet is faster than all other approaches except for POMO with greedy action selection (and POMO with sampling in one case). In addition to being faster, PolyNet finds significantly better solutions for all three problems.

Reviewer comment (Theory):
For part 3.4, can you explain more about how the diversity can be increased via the loss function?

Author response:
**Q3**: “For part 3.4, can you explain more about how the diversity can be increased via the loss function?”

In our approach, we implement the diversity-enhancing mechanism explained in the Poppy paper. Given the page limit constraint, we present a concise overview of this mechanism and direct readers to the detailed exposition in the Poppy paper. Our strategy for fostering diversity involves sampling K solutions per instance and subsequently updating the model weights based on the best sampled solutions. While this approach doesn't explicitly enforce diversity, it serves as an incentive for the model to acquire varied strategies in order to optimize overall performance. It's crucial to emphasize that our pursuit of diversity is fundamentally a means to enhance solution quality, a goal we have effectively achieved. To enhance clarity, we have revised accordingly.