Paper: Generalizable Deep RL-Based TSP Solver via Approximate Invariance (ICLR-2024)

Reviewer comment (Experiments):
Regarding Figure 1, are you using instances following the uniform distribution? Does the conclusion or empirical observation still hold for other cases, e.g., instances with several clusters of customer nodes?

Author response:
The observation we made about the kNN distribution in Fig.1 holds for other distributions than the uniform distribution, which may explain our good performance on TSPLib.

Reviewer comment (Evaluation):
I somewhat don't agree with the conclusion of the right panel of Figure 1. Recent studies find that the adversarial perturbations [5, 6] may significantly change the optimal solutions of TSP instances. What if the added random perturbation coincides with the adversarial one? 

> "Small perturbations introduce small gaps, which can be regarded as exploiting approximate invariance."

Author response:
In contrast to the two mentioned papers, we apply a non-adversarial noise. In that case, on average our perturbed instances have similar or the same solutions, which is the property that we want to exploit with approximate invariance.

Reviewer comment (Writing):
The problem formulation of TSP (with DRL) should be simple and concise, while the notations of this paper are wordy, making it unclear to the readers.

Author response:
We have updated our draft for the presentation issue.

Reviewer comment (Experiments):
The experimental evaluation has several shortcomings. First, the generalization is only evaluated with instances trained on the TSP of size 50. It would be valuable to explore the model's performance across a broader range of problem sizes to gain a more comprehensive understanding of its capabilities.

Author response:
(2) About experiments. We have trained our model on TSP-30 and TSP-100 (see table below). The performance increases when trained on larger instances. Interestingly, even training on TSP30 achieves very competitive generalization capabilities.

    |                     | **TSP20** | **TSP50** | **TSP100** | **TSP200** | **TSP500** | **TSP1000** |
    | ------------------- | --------- | --------- | ---------- | ---------- | ---------- | ----------- |
    | $\texttt{TS}^3$-30  |    $0.01$%       |    $0.56$%       |       $2.02$%     |      $4.55$%      |    $7.58$%        |        $10.10$%     |
    | $\texttt{TS}^3$-50  |     $0.03$%      |     $0.51$%      |     $1.82$%       |     $3.98$%       |       $6.68$%     |     $8.10$%        |
    | $\texttt{TS}^3$-100 |     $0.04$%      |    $0.52$%        |     $1.02$%      |     $2.59$%       |      $5.25$%      |    $7.76$%         |

Reviewer comment (Experiments):
The paper imposes time limits on several close competitors, which raises questions about whether the proposed method is genuinely superior or simply optimized for a specific time constraint. A more thorough examination without the time limit would be valuable.

Author response:
(3) About time limit. In our experiments, the other baselines given more time are still not competitive with our methods.

Reviewer comment (Experiments):
Most of the ablation study is done with ts3, but it should be done with ts4 (the complete approach including MCTS).

Author response:
(4) About ablation study. Since our main contribution is $\texttt{TS}^3$, we did an extensive ablation study of this architecture. Regarding $\texttt{TS}^4$, to demonstrate the quality of what is learned by $\texttt{TS}^3$, we compare MCTS using our proposed heatmap (i.e., $\texttt{TS}^4$) and MCTS with uniform heatmap. Table 3 shows that the former outperforms the latter.

Reviewer comment (Experiments):
The ability of the method to achieve SOTA performance in the tsplib evaluation remains less clear (see appendix), especially since the closest competitor in the random tsp (AttGRCN+MCTS) is missing from the comparison.

Author response:
(5) About comparisons in TSPLIB. We have done a comparison that takes the average of TSP results within a given range as PointerFormer has done. AttGCRN+MCTS results are missing since their code implementation is hard to apply on variable-sized TSPs.

Reviewer comment (Experiments):
How would be the generalization look like if the approach is not only trained on size 50?

Author response:
Please check our response to point (2).

Reviewer comment (Experiments):
How would the competitors perform without the time limit?

Author response:
Please check our response to point (3).

Reviewer comment (Experiments):
The compared approaches are too old and can not represent the current best performance for TSP. More recent methods should ought to be compared to render the results more convincing. For instance, the aforementioned two papers already tackled large routing problems with thousands of nodes and achieved small gaps. A comparison to them is recommended to show the reliability of the approach here.

Author response:
- (2) About comparison to other SOTA approaches. Among learn to construct method, we have not found any other method that can perform better in terms of cross-size generalization than $\texttt{TS}^3$.

Reviewer comment (Experiments):
What if POMO or PointerFormer are trained with data augmentation like rotations or scalings, which are not special and may be easily applied for these baselines?

Author response:
Note that POMO and PointerFormer train independently over a batch of an instance and its augmented ones (using rotations), while our training method is designed to enforce approximate invariance (using noise+rotation+scaling), i.e., the loss of the augmented instances is computed with respect to the original one. We believe that our training technique if applied in POMO or PointerFormer would improve their performance.

Reviewer comment (Reproducibility):
Since different techniques are added to enhance the generalization, would these extra techniques obviously raise the training time? How long would it take to train a TS3 model?

Author response:
Data augmentation won't cost too much time since it only involves simple operations like noise, rotation, and reflection, which can be processed in batches. The design of the local and global architecture introduces additional training and inference time. However, those costs are affordable since our model can be trained on small instances with low batch size and still achieve a competitive performance. Currently, our models are trained on a single RTX 3060 GPU card within three days.