SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Transition For a vertex in the graph, if it is adjacent to the current set and all of its neighbors are also connected to the set, removing this vertex from the set will not violate the dominating set requirement. For other isolated vertices, we change their values to 1 to ensure they stay in the set, thus the state  $s$  consistently representing a dominating set. By making these adjustments, we maintain the integrity of the dominating set representation throughout the process.
Reward The log reward is set to be the negative dominating set size, i.e.,  $\mathcal{E}(\mathbf{x}) = |\mathbf{x}|_1$ . For intermediate state  $\mathbf{s}$ , we define its intermediate signal by setting  $\tilde{\mathcal{E}}(\mathbf{s})$  to be the number of 1 in  $\mathbf{s}$ .
B.3 Maximum cut MDP
We start with an empty solution vertex set, i.e., all the vertices are in  $\varnothing$  value.
Action Same as MIS and MC, the action is to add one vertex to the current solution set of vertices, by changing its entry value from  $\varnothing$  to 1.
Transition After each action is carried out, we perform the following check: if involving one void vertex to the solution vertex set would decrease the cut value, then we exclude this vertex by labeling it to 0 from void  $\varnothing$ . This guarantees that when the trajectory terminates (i.e., no void unspecified vertex), the cut is locally maximal.
Reward We set the negative energy to be the size of the current cut set, i.e., the number of edges between all 1 vertices and all 0 vertices. For intermediate state  $\mathbf{s}$ , we define its intermediate signal by setting  $-\tilde{\mathcal{E}}(\mathbf{s})$  to be the number of edges between all 1 vertices and other vertices (note that we think of void vertices as not in the solution set).
C GFlowNet details
We start with the proof of Proposition 1 and then describe other algorithmic details.
Proof. According to the theory of GFlowNets (Bengio et al., 2023), if GFlowNet's training loss is zero on full support, then the GFlowNet is able to sample correctly from the target distribution  $p_T^*(\mathbf{x})$ . When we gradually move temperature  $T$  from finite to  $\infty$ , the distribution  $p_T^*(\mathbf{x})$  will gradually become the uniform distribution over  $\mathcal{X}$ , thus the distribution sampled by GFlowNet will converge to this uniform distribution; when the temperature gradually approaching zero, the distribution  $p_T^*(\mathbf{x})$  will gradually become the uniform distribution over all argmax solution of  $\arg \max_{\mathbf{x}} \mathcal{E}(\mathbf{x})$ , thus the distribution sampled by GFlowNet will converge to this uniform distribution.
For the GFlowNet architecture, we use graph isomorphism network (Xu et al., 2019, GIN) with 5 hidden layers and 256 dimensional hidden size, for both the forward policy and the state flow function. We double the number of hidden layers for SATLIB experiment. The input of the GIN is a integer
Table 4: Max independent set experimental results on ER graph data in the same form as Table 1.
METHOD	TYPE	ER-700-800	ER-9000-11000
SIZE ↑	DROP ↓	TIME ↓	SIZE ↑	DROP ↓	TIME ↓
GUROBI	OR	43.47	3.33%	2:10:05	—	—	—
KAMIS	OR	44.98	0.00%	2:11:21	374.57	0.00%	7:37:21
PPO	UL	41.11	8.60%	4:30	344.67	7.98%	1:02:29
INTEL	SL	37.34	16.99%	28:32	—	—	—
DGL	SL	38.71	13.94%	29:36	—	—	—
OURS	UL	41.14	8.53%	2:55	349.42	6.98%	1:49:43
vector represented state in the form of  $\{0,1,2\}^{|V|}$  followed by an embedding layer, where we use 2 to denote the unspecified void  $\varnothing$ . For the forward policy GIN, we set a unidimensional output node feature followed by a softmax operation over all vertices, which gives a categorical distribution on all the vertices. For the state flow GIN, we use a graph pooling layer to extract a unidimensional graph-level output to serve as the flow value. We fix the backward policy to be a uniform distribution over all chosen vertices for simplicity. Theoretically, we can only use one GIN for both the flow function and the forward policy, but we find it is much less stable. We hypothesize that the flow function training stability is very important in DB-based GFlowNet methods, and that most previous GFlowNet works underfit the flow function and underestimate the performance of DB.
We use the Adam optimizer with default  $1 \times 10^{-3}$  learning rate without hyperparameter tuning. After breaking the complete trajectories into transition pieces, we combine them to many buffer with batchsize equals 64 to calculate forward-looking based detailed balance loss. The training keeps 20 epochs, although most experiments actually converge in the first 5 epochs. For the SATLIB experiment, our proposed method converges within the first epoch. During the inference phase, we simply the same evaluation protocol in Ahn et al. (2020). To be concrete, we sample 20 solutions for each graph configuration and take the best result.
We conduct a series of ablation study with small graph MIS experiments in Figure 5. Each algorithm is repeated with five different random seeds. For the temperature  $T$  used in GFlowNet's reward shaping, we try values of the inverse temperature from 1 to 5000 and find values higher or equal than 100 will obtain similar results. We thus simply use an inverse temperature of 500 for all the tasks. We also use temperature annealing, to start with temperature of 1 and terminate with the target temperature level (although the ablation study shows that GFlowNet could reach same performance without the annealing trick). During the training, GFlowNet will need to rollout (i.e., interact with the environment to collect complete trajectories) to collect training data. In this work, we do on-policy exploration, which means the policy used in the rollout stage is the current forward policy. Nonetheless, GFlowNet has the ability of doing off-policy exploration. In the third ablation study, we mix the forward policy with different levels of uniform noise distribution, and find that GFlowNet is very robust under this off-policy setups. In this fourth ablation study, we change the number of hidden layer in the GIN architecture and find GFlowNet is also robust to this hyperparameter.
D More about experiments
For BA graphs, we use the variants with 4 attaching edges. For small scale datasets, we first sample the number of vertices uniformly from 200 to 300 and generate the graph instances. For large-scale problems, it is uniformly sampled from 800 to 1200. Both the small and large synthetic graph trainsets contain 4000 data points for either RB graph or BA graph. For the SATLIB dataset, there are 40000 data points in total, where the largest graph has 1347 vertices. For all datasets, we use a validation set and a test set, both with 500 data points. We choose the hyperparameters of each algorithm based on the validation performance. Regarding hardware, we use NVIDIA Tesla V100 Volta GPUs. We repeat the experiments for five different random seeds.
We also conduct MIS experiments on Erdos-Renyi (ER) graph data shown in Table 4. We use ER data as large as  $700 \sim 800$  vertices and  $9000 \sim 11000$  vertices for a comparison with Qiu et al. (2022, DIMES) and Sun & Yang (2023, DIFUSCO). The training and evaluation protocols are not exactly the same, but we still present results from these baselines for completeness. DIMES achieves 38.24/42.06 and 320.50/332.80 (“/” means two variants of their algorithm) respectively
for these two tasks; DIFUSCO achieves 38.83/41.12 ("/") means two variants of their algorithm) for the ER-700-800 problem, and is not scalable for extremely large problems like ER-9000-11000. The difference in protocol includes: GFlowNet uses less training data samples; DIFUSCO is a supervised learning algorithm (requires a solver to provides solution), while DIMES and GFlowNet are unsupervised methods; the test data may not be the same (they generate their own fold). For GFlowNet in the largest task, we use a temperature of 1000 and a batch size of 32. In summary, we can see that GFlowNet also achieves state-of-the-art performance in this setup, especially in large problems.
In MIS benchmark, for the MIS supervised learning baselines (INTEL and DGL), we disable the usage of the graph reduction and 2-opt local search tricks, but only adopt the queue pruning and weighted queue pop technique. This is because even random initialized neural network can be used to find near optimal solutions with graph reduction and local search as pointed out by Böther et al. (2022). Furthermore, these forbidden techniques are very time-consuming, making the inference time exceeding our time limit constraint. For KAMIS, we use the redumis algorithm. For GUROBI, we formulate MIS as quadratic programming problem and then use the pygurobi python package to solve it. We also run Erdos-based probabilistic methods (ERDOS and ANNEAL) for MIS, but their results are worse than any baselines methods in the MIS benchmark, thus we do not include them in Table 1. In the max cut task, we use the cvxpy python package to implement the SDP baseline.