SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In Algorithm 2, we present the basic pipeline of the self-training approach that selects the parameters  $\theta \in \Theta$  such that the inference algorithm  $\mathcal{A}^{\mathrm{CMP}_{\theta_t}}$  admits a competitive performance given as input graphs  $G$  following a graph distribution  $\mathcal{D} \subseteq \mathcal{G}$ . We further improve this basic pipeline with several tweaks that we incorporate into our training process:
Creating a graph buffer  $\mathcal{B}$ : We are given a shuffled dataset of graphs  $\mathcal{D}$ , which represents the training data for the model. The core difference between the pipeline and the training process comes from the graph buffer  $\mathcal{B}$ . In Algorithm 2, this buffer stores any graph  $G$  that is found during the recursive call of  $\mathcal{A}^{\mathrm{CMP}_{\theta_t}}(G_{\mathrm{init}})$  on  $G_{\mathrm{init}} \sim D$  (Step 6 of Algorithm 2). However, in the implementation of the graph buffer, it stores pairs of graphs  $(G, G')$  that were generated by  $\mathcal{A}^{\mathrm{CMP}_{\theta_t}}(G_{\mathrm{init}})$ , alongside a binary label that indicates which of the two graphs has a larger estimated MIS size. How this estimate is generated, will be explained further down this section.
The training process: Prior to starting training, we first set two hyperparameters: one that specifies the number of graphs used to populate the buffer before training the model, and another that determines the number of graph pairs generated from  $\mathcal{A}^{\mathrm{CMP}_{\theta_t}}(G)$  per graph  $G \sim \mathcal{D}$ . Then, a dataset is created by generating these pairs for the set number of graphs. The dataset is then added to the graph buffer,
Algorithm 2 Basic Pipeline of our Training Approach
1: Input: A distribution  $\mathcal{D}$  over graphs.
2: Initialize parameters  $\theta_0\in \Theta$
3: Initialize a graph-buffer  $\mathcal{B} \gets \emptyset$ .
4: for each epoch  $t = 0, \dots, T - 1$  do
5: Sample a graph  $G_{\mathrm{init}} \sim \mathcal{D}$ .
6: Run  $\mathcal{A}^{\mathrm{CMP}_{\theta_t}}(G_{\mathrm{init}})$  and store in  $\mathcal{B}$  all graphs produced during each recursive call of Algorithm 1.
7: Update the parameters  $\theta_{t + 1}\in \Theta$  such that
$$
\theta^ {t + 1} := \operatorname {a r g m i n} _ {\theta \in \Theta} \mathbb {E} _ {(G, G ^ {\prime}) \sim \mathcal {B}} \left[ \right. \ell \left( \right.\mathrm {C M P} _ {\theta} (G, G ^ {\prime}), \mathbb {I} \left[ \right. \mathbb {E} \left[ | \mathcal {A} ^ {\mathrm {C M P} _ {\theta_ {t}}} (G) | \right] <   \mathbb {E} \left[ | \mathcal {A} ^ {\mathrm {C M P} _ {\theta_ {t}}} (G ^ {\prime}) | ] \right]\left. \right)\left. \right]
$$
Figure 2: An example of data generation for the training. (Left) At the beginning of each training epoch (Step 5), Algorithm 2 samples  $G_{\mathrm{init}} \sim \mathcal{D}$  and computes an independent set by using the comparator  $\mathrm{CMP}_{\theta_t}$  and by following the branches in the recursion tree that are marked red, with the doubly circled one being the produced independent set. The generated graphs from this procedure are added to the buffer  $\mathcal{B}$ . (Right) Then, a dataset is created by sampling graphs from the buffer and then computing an estimate of their MIS size (based on  $\mathcal{A}^{\mathrm{CMP}_{\theta_t}}$ ). Based on this estimate, a dataset is created with graph pairs  $(G, G')$  and their corresponding binary labels denoting which MIS estimates are larger.
replacing steps 5 and 6 in Algorithm 2, which only does this with one graph per epoch. Next, training starts, and after completing a set number of epochs, a new dataset is created using the updated model, and the process is repeated iteratively.
Estimating the MIS: Finally, the loss function in Step 7 of Algorithm 2, also operates slightly differently. The main difference arises from  $\mathbb{I}\left[\mathbb{E}\left[|\mathcal{A}^{\mathrm{CMP}_{\theta_t}}(G)|\right] < \mathbb{E}\left[|\mathcal{A}^{\mathrm{CMP}_{\theta_t}}(G')|\right]\right]$ , since the estimates  $|\mathrm{MIS}(G)| \approx \mathbb{E}\left[|\mathcal{A}^{\mathrm{CMP}_{\theta_t}}(G)|\right]$  and  $|\mathrm{MIS}(G')| \approx \mathbb{E}\left[|\mathcal{A}^{\mathrm{CMP}_{\theta_t}}(G')|\right]$  are not directly utilized. Instead, we propose two other approaches, which are better approximations than the expectations used in Algorithm 2, since they use a maximizing operator.
The first approach involves performing so-called "roll-outs" on the graph pairs generated  $G$  and  $G'$  by  $\mathcal{A}^{\mathrm{CMP}_{\theta_t}}$ , in order to estimate their MIS sizes. To perform the roll-outs, we simply run  $\mathcal{A}^{\mathrm{CMP}_{\theta_t}}$  on graphs  $G$  and  $G'$ $m$  times and use the maximum size of the found independent sets as an estimate of their MIS. Formally, in a roll-out on a graph  $G$ , we sample the independent sets  $\mathrm{ISS}_1, \mathrm{ISS}_2, \ldots, \mathrm{ISS}_m \sim \mathcal{A}^{\mathrm{CMP}_{\theta_t}}(G)$ . Then, the estimate of the MIS size of  $G$  is  $\max(|\mathrm{ISS}_1|, |\mathrm{ISS}_2|, \ldots, |\mathrm{ISS}_m|)$ .
An example of the entire process of generating the dataset using roll-outs can be found in Fig. 2. In practice, we observe an increasing consistency of the model during training, as can be seen in Appendix J.
Mixed roll-out variant: We introduce a variant of the aforementioned method, which utilizes the deterministic greedy algorithm. This greedy algorithm iteratively creates an independent set by removing the node with the lowest degree and adding it to the independent set. This algorithm is often an efficient approximation to the optimum solution. Our variant is constructed as follows: we compute the maximum between the roll-outs of the model and the result of the greedy algorithm, which creates a dataset with more accurate self-supervised approximations of the MIS values. This, in turn, generates binary targets for the buffer that are more likely to be accurate. Thus, for this second variant, the estimate of the MIS size of a graph  $G$  would be max (  $|\mathrm{Greedy}(G)|,|\mathrm{ISS}_1|,|\mathrm{ISS}_2|,\ldots ,|\mathrm{ISS}_m|$  ).
In this section, we conduct an evaluation of the proposed method for the MIS problem. Let us first describe the training setup, the baselines, and the datasets. Additional details and experiments on MIS are displayed in the Appendices C and D. Our method also generalizes well in MVC, as the results in Appendix E illustrate.
6.1 Training Setup
Our model: We implement two comparator models: one using just roll-outs with the model, and another using the roll-outs together with greedy, called "mixed roll-out". We train each model using a graph embedding module with  $K = 3$  iterations, which takes in 32-dimensional initial node embeddings.
Baselines: We compare against the neural approaches  $\text{Erdos GNN}$  [Karalias and Loukas, 2020],  $\text{RUN-CSP}$  from Toenshoff et al. [2019], and a method specifically for the MIS problem: LwDMIS [Ahn et al., 2020]. Since we observe unexpected poor performances from RUN-CSP on the COLLAB and RB datasets, we have omitted those results from the table. We train every model for 300 epochs. Each experiment is performed on a single GPU with 6GB RAM.
Besides neural approaches, we use traditional baselines, such as the Greedy MIS [Wormald, 1995], Simple Local Search [Feo et al., 1994] and a Random Comparator as a sanity check. Furthermore, we implement two mixed-integer linear programming solvers: SCIP 8.0.3 and the highly optimized commercial solver Gurobi 10.0.
Datasets: We evaluate our model on three standard datasets, following Karalias and Loukas [2020]: COLLAB [Yanardag and Vishwanathan, 2015], TWITTER [Leskovec and Krevl, 2014] and RB [Xu et al., 2007, Toenshoff et al., 2019]. In addition, we introduce the SPECIAL dataset that includes challenging graphs for handcrafted approaches as we detail in Appendix C.
Table 1 reports the average approximation ratios on the test instances of the various datasets. The approximation ratio is computed by dividing a solution's independent set size by the optimum solution, which is computed using the Gurobi solver with a time limit of 1 hour per graph.
The results indicate that the greedy algorithm performs strongly in three of the four datasets, which is consistent with the observation of Angelini and Ricci-Tersenghi [2022]. However, notice that our proposed approach outperforms the greedy in both the Twitter and the SPECIAL datasets, which validates that the greedy heuristic is not optimal in every case and is prone to failing in few cases. Importantly, among the neural approaches that are the main compared methods, our proposed method performs favorably in all datasets. The performance of our method indicates that the proposed self-training scheme is able to learn from diverse data distributions and generalize reasonably well in the test sets of the respective dataset. In addition, the proposed method is faster than the rest neural approaches.
The mixed roll-out model in Table 1 outperforms the normal roll-out model in almost all datasets, indicating the effectiveness of the greedy heuristic in roll-outs. This is particularly evident in the RB dataset. However, for SPECIAL instances, the normal model performs marginally better, possibly due to the unsuitability of the greedy guiding heuristic as a baseline for this dataset.
Out of distribution: We examine the performance of the learned comparator through its generalization to new graph distributions. Concretely, we conduct an out-of-distribution analysis as follows:
Table 1: Test set approximation ratios (higher is better; best performance in bold) on four datasets for MIS. We report the average approximation ratios, along with std and duration in seconds per graph  $(\mathrm{s} / \mathrm{g})$ . Notice that the proposed method outperforms all the deep-learning-based approaches across datasets. Classical methods have a gray color, whilst neural approaches are in black.
Method (↓) Dataset (→)	RB	COLLAB	TWITTER	SPECIAL
CMP (Normal Roll-outs)	0.770 ± 0.107	0.990 ± 0.051	0.967 ± 0.083	0.996 ± 0.029
(0.43 s/g)	(0.17 s/g)	(0.35 s/g)	(0.04 s/g)
CMP (Mixed Roll-outs)	0.836 ± 0.083	0.990 ± 0.049	0.977 ± 0.031	0.994 ± 0.035
(0.36 s/g)	(0.21 s/g)	(0.21 s/g)	(0.05 s/g)
Erdos' GNN	0.813 ± 0.107	0.952 ± 0.142	0.935 ± 0.078	0.921 ± 0.218
(1.39 s/g)	(0.60 s/g)	(1.37 s/g)	(1.03 s/g)
LwDMIS	0.804 ± 0.089	0.978 ± 0.031	0.972 ± 0.032	0.828 ± 0.304
(0.42 s/g)	(0.17 s/g)	(0.19 s/g)	(0.32 s/g)
RUN-CSP (Accurate)	-	-	0.875 ± 0.053	0.946 ± 0.059
(0.57 s/g)	(0.51 s/g)
Greedy MIS	0.925 ± 0.053	0.998 ± 0.023	0.964 ± 0.048	0.131 ± 0.055
(0.01 s/g)	(0.02 s/g)	(0.04 s/g)	(0.03 s/g)
Random CMP	0.615 ± 0.155	0.817 ± 0.211	0.634 ± 0.182	0.225 ± 0.279
(0.42 s/g)	(0.30 s/g)	(0.36 s/g)	(0.41 s/g)
Simple Local Search (10s)	0.565 ± 0.237	0.860 ± 0.213	0.644 ± 0.218	0.188 ± 0.340
SCIP 8.0.3 (1s)	0.741 ± 0.351	0.999 ± 0.016	0.959 ± 0.024	1.000
SCIP 8.0.3 (5s)	0.937 ± 0.118	1.000	0.999 ± 0.024	1.000
Gurobi 10.0 (0.5s)	0.969 ± 0.070	0.981 ± 0.068	0.985 ± 0.085	0.940 ± 0.237
Gurobi 10.0 (1s)	0.983 ± 0.051	1.000	1.000	1.000
Gurobi 10.0 (5s)	0.999 ± 0.008	1.000	1.000	1.000
Table 2: Out-of-distribution approximation ratios during inference (higher is better). Every row denotes a model trained on a specific dataset. Every column considers a different test dataset. The CMP is trained using mixed roll-outs. Notice that the proposed method generalizes well in out-of-distribution structures. This is indicative of the learned comparator extracting robust patterns.
Model (↓) Dataset (→)	RB	COLLAB	TWITTER
CMP RB	-	0.903 ± 0.186	0.668 ± 0.187
CMP COLLAB	0.856 ± 0.080	-	0.906 ± 0.094
CMP TWITTER	0.773 ± 0.101	0.927 ± 0.148	-
Erdos' GNN RB	-	0.361 ± 0.334	0.752 ± 0.188
Erdos' GNN COLLAB	0.680 ± 0.071	-	0.592 ± 0.186
Erdos' GNN TWITTER	0.746 ± 0.092	0.666 ± 0.385	-
each model is trained in one graph distribution, indicated by the rows of Table 2. Then, the model is evaluated on different graph distributions, indicated by the columns of Table 2. The analysis is conducted on both our model and the approach of Erdos GNN [Karalias and Loukas, 2020].
Surprisingly, our model trained over COLLAB displays good generalization skills across different datasets, even outperforming the RB-trained model on the RB dataset. Conversely,  $\text{Erdos GNN}$  trained over RB performs poorly over the COLLAB dataset. Both models trained over the RB dataset perform more poorly in general, likely due to the highly specific graph distribution of the RB dataset. Moreover, our model, on the whole, exhibits good generalization skills over different graph distributions.
Motivated by the principles of Dynamic Programming, we develop a self-training approach for important CO problems, such as the Maximum Independent Set and the Minimum Vertex Cover. Our approach embraces the power of self-training, offering the dual benefits of data self-annotation and data generation. These inherent attributes are instrumental in providing an unlimited source of data indicating that the performance of the induced algorithms can be significantly improved with sufficient scaling on the computational resources. We firmly believe that a thorough investigation into the interplay between Dynamic Programming and self-training techniques can pave the way for new deep-learning-oriented approaches for demanding CO problems.
Limitations: Our current empirical approach lacks theoretical guarantees on the convergence or the approximate optimality of the obtained algorithm. Additionally, the implemented GNN is using simple modules, while more complex modules could result in further empirical improvements, which can be the next step in this direction. Furthermore, randomly selecting a vertex could be sub-optimal. One interesting future direction would be to explore predicting the next vertex to select.
This work was supported by the Swiss National Science Foundation (SNSF) under grant number 200021_205011, by Hasler Foundation Program: Hasler Responsible AI (project number 21043) and Innovation project supported by Innosuisse (contract agreement 100.960 IP-ICT). We further thank Elias Abad Rocamora and Nikolaos Karalias for the useful feedback on the paper draft as well as Planny for the helpful discussions.
Sungsoo Ahn, Younggyo Seo, and Jinwoo Shin. Learning what to defer for maximum independent sets. International Conference on Machine Learning (ICML), 2020.
Réka Albert and Albert-László Barabási. Statistical mechanics of complex networks. Reviews of modern physics, 74(1):47, 2002.
Maria Chiara Angelini and Federico Ricci-Tersenghi. Cracking nuts with a sledgehammer: when modern graph neural networks do worse than classical greedy algorithms. arXiv preprint arXiv:2206.13211, 2022.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016a.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016b.
Richard Bellman. The theory of dynamic programming. BULL, 60(6):503-515, 1954.
Richard Bellman. Dynamic programming treatment of the travelling salesman problem. JACM, 9(1): 61-63, 1962.
Paul Erdős, Alfréd Rényi, et al. On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci, 5(1):17-60, 1960.
Federico Errica, Marco Podda, Davide Bacciu, and Alessio Micheli. A fair comparison of graph neural networks for graph classification. arXiv preprint arXiv:1912.09893, 2019.
Thomas A Feo, Maurizio GC Resende, and Stuart H Smith. A greedy randomized adaptive search procedure for maximum independent set. Journal of Global Optimization, 42(5):860-878, 1994.
Maxime Gasse, Didier Chételat, Nicola Ferroni, Laurent Charlin, and Andrea Lodi. Exact combinatorial optimization with graph convolutional neural networks. NIPS, 32, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016.
Yu jiao Hu, Yuan Yao, and Wee Sun Lee. A reinforcement learning approach for optimizing multiple traveling salesman problems over graphs. Knowledge-Based Systems, 204:106244, 2020.
JQ James, Wen Yu, and Jiatao Gu. Online vehicle routing with neural combinatorial optimization and deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems, 20 (10):3806-3817, 2019.
Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. NeurIPS, 31, 2018.
Rongkai Zhang, Cong Zhang, Zhiguang Cao, Wen Song, Puay Siew Tan, Jie Zhang, Bihan Wen, and Justin Dauwels. Learning to solve multiple-tsp with time window and rejections via deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems, 2022.
Si Zhang, Hanghang Tong, Jiejun Xu, and Ross Maciejewski. Graph convolutional networks: a comprehensive review. Computational Social Networks, 6(1):1-23, 2019.
Zhaocheng Zhu, Zuobai Zhang, Louis-Pascal Xhonneux, and Jian Tang. Neural bellman-ford networks: A general graph neural network framework for link prediction. NeurIPS, 34:29476-29490, 2021.
Contents of the Appendix
We describe the contents of the supplementary material below:
- In Appendix A, we provide further information on the prior work and related research, including additional references and links to relevant papers.
- In Appendix B, we explore applications of our method beyond the MIS problem. Specifically, we discuss how our method can be applied to the Minimum Vertex Cover (MVC) problem.
- In Appendix C, we provide detailed experimental details used in our experiments.
- In Appendix D, we present additional experiments conducted specifically on the MIS problem.
- In Appendix E, we extend our investigations to the MVC problem. We present experimental results on benchmarks and discuss the performance and insights gained from the experiments.
- Appendix F provides additional background information on relevant concepts related to the MIS and MVC problems.
- In Appendix G, we present an ablation study where we systematically analyze the impact of different architecture configurations in our method.
- Appendix H delves into the broader impact of our work, discussing the potential implications of using our self-supervised approach for CO problems.
- In Appendix I, we provide the omitted proofs of theorems mentioned in the main paper.
- Finally, in Appendix J, we gain some insight into the decisions of the comparator through the consistency property.
A Additional Links to Prior Work
In Appendix A.1, we delve into an analysis of multiple deep learning techniques for solving CO (Combinatorial Optimization) problems, including how different works have addressed the non-continuity of the loss function. Additionally, in Appendix A.2 we provide a brief explanation of Dynamic Programming and Monte Carlo Tree Search.
A.1 Deep Learning Approaches for CO
The growing interest in employing machine learning techniques to tackle combinatorial optimization problems has led to addressing the task using different neural approaches [Karimi-Mamaghan et al., 2022, Mazyavkina et al., 2021]. Moreover, some works combine elements from classic heuristics with deep learning techniques, like branch-and-bound [Gasse et al., 2019] and local search [Li et al., 2018].
As we mention in Sec. 2, several works use a continuous relaxation of the loss function. Following the settings of Wang and Li [2023], CO problems consist of assigning values to discrete optimization variables  $X = (x_{i})_{1\leq i\leq n} \in \{0,1\}^{n}$  such that an energy (or loss) function  $F(G,X)$  is minimized or maximized in  $X$ . Since  $F(G,X)$  is not continuous, Karalias and Loukas [2020] employs a relaxed probabilistic loss function, which is later refined by Wang et al. [2022] by assuming continuous values and taking the values of  $F(G,X)$  at the discrete points  $X$ . Since some CO problems require instance-wise good solutions, Wang and Li [2023] focuses more on obtaining good initialization instances. Given that certain CO issues necessitate the acquisition of individual-level optimal solutions, Wang and Li [2023] places greater emphasis on attaining favorable initializations for subsequent instances rather than providing immediate solutions.
A.2 Dynamic Programming
Dynamic Programming is especially effective for solving problems that demonstrate the principle of optimality. This principle states that an optimal solution to a problem can be achieved by recursively finding the optimal solutions to smaller subproblems.
Xu et al. [2020] utilize DNNs to replace the function responsible for dividing a problem into subproblems and estimating the optimal decision at each step. Their results demonstrate significant
reductions in computation time for classical combinatorial optimization problems such as the 'TSP', compared to the Bellman-Held-Karp algorithm [Bellman, 1962].
The work of Yang et al. [2018] addresses the problem of the size of the lookup table, which may require exponential space, used to store solutions of sub-problems in Dynamic Programming. In their work, Yang et al. [2018] argue that it is possible to approximate a Dynamic Programming function with a much smaller neural network.
The Dynamic Programming Tree (DPT) is often employed for solving CO tasks. Each node of the tree represents a problem, and its child nodes represent two sub-problems derived from it. The DPT is designed to explore the solution space of a problem similar to Dynamic Programming, but it only considers a subset of the possible solutions due to the large size of the solution space. To construct the DPT, a Monte Carlo Tree Search (MCTS) technique [James et al., 2017] is utilized. The MCTS algorithm begins at the root node and navigates the tree to add new nodes to the structure. Upon reaching a new leaf node, MCTS employs a simulation to estimate the value of the node. The results of these simulations are subsequently backpropagated up through the tree, updating the node values accordingly.
Due to the reasons stated above, Monte Carlo Tree Search (MCTS) and other tree search methods have been utilized in the context of CO problems as a replacement for pure Dynamic Programming. MCTS is capable of significantly reducing computational complexity as it operates on a smaller search space. In Xing et al. [2020], MCTS is employed to solve the TSP problem on graphs with a maximum of 100 nodes. Additionally, Xu and Lieberherr [2020] employs MCTS to convert CO problems into Zermelo games [Schwalbe and Walker, 2001], and maps the winning strategy of the Zermelo games to provide solutions for CO problems.
B Beyond Maximum Independent Set
Our proposed method for solving CO problems using DP and GNNs with self-supervised training extends beyond the realm of the MIS problem. While our focus has primarily been on MIS, the underlying framework and techniques can be applied to various other CO settings. In this section, we explore the potential application and adaptation of our approach to address a different problem domain. We will focus on another problem, called the Minimum Vertex Cover (MVC) problem. We will benchmark against various neural approaches. The goal of the MVC problem is the following:
Definition 4. Given a graph  $G(V, E)$  find a set of nodes  $S \subseteq V$  with minimum size such that for each edge  $(v, u) \in E$  either  $v \in S$  or  $u \in S$ .
Minimum Vertex Cover asks for the smallest set of vertices  $S$  such that all edges of  $G$  are "covered" by at least one vertex in the set. Dynamic Programming provides similar optimality guarantees as that of Maximum Independent Set described in Theorem 1. The respective optimality guarantees for Minimum Vertex Cover are formally stated in Corollary 1.
Corollary 1. Let a graph  $G(V, E) \in \mathcal{G}$ . Then for any vertex  $v \in V$  with  $d(v) \geq 1$ ,
$$
\left| \operatorname {M V C} (G) \right| = \min  \left(\left| \operatorname {M V C} \left(G _ {0}\right) \right|, \left| \operatorname {M V C} \left(G _ {1} \right\}\right) \right|) ^ {2}.
$$