SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} - \mathcal {L} (\pmb {\theta}, \phi | \pmb {x}, \tilde {\pmb {x}}) = \mathbb {E} _ {\pmb {z} \sim q _ {\phi} (\pmb {z} | \pmb {x})} \left[ \log \frac {p _ {\pmb {\theta}} (\pmb {x} | \pmb {z} , \tilde {\pmb {x}}) p _ {\pmb {\theta}} (\pmb {z})}{q _ {\phi} (\pmb {z} | \pmb {x})} \right] \\ = \mathbb {E} _ {\boldsymbol {z} \sim q _ {\phi} (\boldsymbol {z} | \boldsymbol {x})} \left[ \log p _ {\boldsymbol {\theta}} (\boldsymbol {x} | \boldsymbol {z}, \tilde {\boldsymbol {x}}) \right] - \mathbb {E} _ {\boldsymbol {z} \sim q _ {\phi} (\boldsymbol {z} | \boldsymbol {x})} \left[ \log \frac {q _ {\phi} (\boldsymbol {z} | \boldsymbol {x})}{p _ {\boldsymbol {\theta}} (\boldsymbol {z})} \right] \\ = \mathbb {E} _ {\boldsymbol {z} \sim q _ {\phi} (\boldsymbol {z} | \boldsymbol {x})} \left[ \log p _ {\boldsymbol {\theta}} (\boldsymbol {x} | \boldsymbol {z}, \tilde {\boldsymbol {x}}) \right] - D _ {\mathrm {K L}} \left[ q _ {\phi} (\boldsymbol {z} | \boldsymbol {x}) \| p _ {\boldsymbol {\theta}} (\boldsymbol {z}) \right]. \tag {16} \\ \end{array}
$$
Consequently, the loss function can be formulated as follows:
$$
\mathcal {L} (\boldsymbol {\theta}, \phi) = \mathbb {E} _ {\boldsymbol {x} \sim \mathcal {D}} \mathbb {E} _ {\tilde {\boldsymbol {x}} \sim \tilde {p} (\tilde {\boldsymbol {x}} | \boldsymbol {x})} \left[ \mathcal {L} (\boldsymbol {\theta}, \phi | \boldsymbol {x}, \tilde {\boldsymbol {x}}) \right], \tag {17}
$$
$$
\mathcal {L} (\boldsymbol {\theta}, \phi | \boldsymbol {x}, \tilde {\boldsymbol {x}}) = \underbrace {\mathbb {E} _ {\boldsymbol {z} \sim q _ {\phi} (\boldsymbol {z} | \boldsymbol {x})} \left[ - \log p _ {\boldsymbol {\theta}} (\boldsymbol {x} | \boldsymbol {z} , \tilde {\boldsymbol {x}}) \right]} _ {\mathcal {L} _ {\mathrm {r e c}}} + \underbrace {D _ {\mathrm {K L}} \left[ q _ {\phi} (\boldsymbol {z} | \boldsymbol {x}) \| p _ {\boldsymbol {\theta}} (\boldsymbol {z}) \right]} _ {\mathcal {L} _ {\mathrm {p r i o r}}}. \tag {18}
$$
In the formula, the first term  $\mathcal{L}_{\mathrm{rec}}$  is referred to as the reconstruction loss, as it urges the decoder to reconstruct the input data  $\pmb{x}$ . The second term  $\mathcal{L}_{\mathrm{prior}}$  is referred to as the prior loss, as it regularizes the posterior distribution  $q_{\phi}(z|x)$  of the latent variable to approximate the prior distribution  $p_{\theta}(z)$ . In practice, the prior distribution  $p_{\theta}(z)$  is commonly taken as  $\mathcal{N}(\mathbf{0},\mathbf{I})$ , and a hyperparameter is often introduced as the coefficient for the prior loss. Consequently, the loss function can be expressed as:
$$
\mathcal {L} (\boldsymbol {\theta}, \phi) = \mathbb {E} _ {\boldsymbol {x} \sim \mathcal {D}} \mathbb {E} _ {\tilde {\boldsymbol {x}} \sim \tilde {p} (\tilde {\boldsymbol {x}} | \boldsymbol {x})} \left[ \mathcal {L} (\boldsymbol {\theta}, \phi | \boldsymbol {x}, \tilde {\boldsymbol {x}}) \right], \tag {19}
$$
$$
\mathcal {L} (\boldsymbol {\theta}, \phi | \boldsymbol {x}, \tilde {\boldsymbol {x}}) = \mathcal {L} _ {\text {r e c}} (\boldsymbol {\theta}, \phi | \boldsymbol {x}, \tilde {\boldsymbol {x}}) + \beta \cdot \mathcal {L} _ {\text {p r i o r}} (\phi | \boldsymbol {x}),
$$
$$
\mathcal {L} _ {\operatorname {r e c}} (\boldsymbol {\theta}, \phi | \boldsymbol {x}, \tilde {\boldsymbol {x}}) = \mathbb {E} _ {\boldsymbol {z} \sim q _ {\phi} (\boldsymbol {z} | \boldsymbol {x})} \left[ - \log p _ {\boldsymbol {\theta}} (\boldsymbol {x} | \boldsymbol {z}, \tilde {\boldsymbol {x}}) \right], \tag {20}
$$
$$
\mathcal {L} _ {\mathrm {p r i o r}} (\phi | \boldsymbol {x}) = D _ {\mathrm {K L}} \left[ q _ {\phi} (\boldsymbol {z} | \boldsymbol {x}) \| \mathcal {N} (\boldsymbol {0}, \boldsymbol {I}) \right].
$$
In G2MILP, the loss function is instantiated as:
$$
\mathcal {L} (\boldsymbol {\theta}, \phi) = \mathbb {E} _ {\mathcal {G} \sim \mathcal {D}} \mathbb {E} _ {\tilde {\mathcal {G}} \sim \tilde {p} (\tilde {\mathcal {G}} | \mathcal {G})} \left[ \mathcal {L} (\boldsymbol {\theta}, \phi | \mathcal {G}, \tilde {\mathcal {G}}) \right], \tag {21}
$$
$$
\mathcal {L} (\boldsymbol {\theta}, \phi | \mathcal {G}, \tilde {\mathcal {G}}) = \mathcal {L} _ {\text {r e c}} (\boldsymbol {\theta}, \phi | \mathcal {G}, \tilde {\mathcal {G}}) + \beta \cdot \mathcal {L} _ {\text {p r i o r}} (\phi | \mathcal {G}),
$$
$$
\mathcal {L} _ {\text {r e c}} (\boldsymbol {\theta}, \phi | \mathcal {G}, \tilde {\mathcal {G}}) = \mathbb {E} _ {\mathbf {Z} \sim q _ {\phi} (\mathbf {Z} | \mathcal {G})} \left[ - \log p _ {\boldsymbol {\theta}} (\mathcal {G} | \mathbf {Z}, \tilde {\mathcal {G}}) \right], \tag {22}
$$
$$
\mathcal {L} _ {\text {p r i o r}} (\phi | \mathcal {G}) = D _ {\mathrm {K L}} \left[ q _ {\phi} (\mathbf {Z} | \mathcal {G}) \| \mathcal {N} (\mathbf {0}, \mathbf {I}) \right].
$$
A.3 G2MILP Implementation
A.3.1 Encoder
The encoder implements  $q_{\phi}(\mathbf{Z}|\mathcal{G})$  in Equation 22. Given a bipartite graph  $\mathcal{G} = (\mathcal{V} \cup \mathcal{W}, \mathcal{E})$  equipped with the feature metrics  $(\mathbf{V}, \mathbf{W}, \mathbf{E})$ , we employ a GNN structure with parameters  $\phi$  to extract the representations. Specifically, we utilize MLPs as embedding layers to obtain the initial embeddings  $h_{v_i}^{(0)}, h_{w_j}^{(0)}$ , and  $h_{e_{ij}}$ , given by:
$$
\boldsymbol {h} _ {v _ {i}} ^ {(0)} = \operatorname {M L P} _ {\phi} (\boldsymbol {v} _ {i}), \quad \boldsymbol {h} _ {w _ {j}} ^ {(0)} = \operatorname {M L P} _ {\phi} (\boldsymbol {w} _ {j}), \quad \boldsymbol {h} _ {e _ {i j}} = \operatorname {M L P} _ {\phi} (\boldsymbol {e} _ {i j}). \tag {23}
$$
Next, we perform  $K$  graph convolution layers, with each layer in the form of two interleaved half-convolutions. The convolution layer is defined as follows:
$$
\boldsymbol {h} _ {v _ {i}} ^ {(k + 1)} \leftarrow \operatorname {M L P} _ {\phi} \left(\boldsymbol {h} _ {v _ {i}} ^ {(k)}, \sum_ {j: e _ {i j} \in \mathcal {E}} \operatorname {M L P} _ {\phi} \left(\boldsymbol {h} _ {v _ {i}} ^ {(k)}, \boldsymbol {h} _ {e _ {i j}}, \boldsymbol {h} _ {v _ {j}} ^ {(k)}\right)\right),
$$
$$
\boldsymbol {h} _ {w _ {j}} ^ {(k + 1)} \leftarrow \operatorname {M L P} _ {\phi} \left(\boldsymbol {h} _ {w _ {j}} ^ {(k)}, \sum_ {i: e _ {i j} \in \mathcal {E}} \operatorname {M L P} _ {\phi} \left(\boldsymbol {h} _ {v _ {i}} ^ {(k + 1)}, \boldsymbol {h} _ {e _ {i j}}, \boldsymbol {h} _ {w _ {j}} ^ {(k)}\right)\right). \tag {24}
$$
The convolution layer is followed by two GraphNorm layers, one for constraint vertices and the other for variable vertices. We employ a concatenation Jumping Knowledge layer to aggregate information from all  $K$  layers and obtain the node representations:
$$
\boldsymbol {h} _ {v _ {i}} = \operatorname {M L P} _ {\phi} \left(\underset {k = 0, \dots , K} {\text {C O N C A T}} \left(\boldsymbol {h} _ {v _ {i}} ^ {(k)}\right)\right), \quad \boldsymbol {h} _ {w _ {j}} = \operatorname {M L P} _ {\phi} \left(\underset {k = 0, \dots , K} {\text {C O N C A T}} \left(\boldsymbol {h} _ {w _ {j}} ^ {(k)}\right)\right). \tag {25}
$$
The obtained representations contain information about the instances. Subsequently, we use two MLPs to output the mean and log variance, and then sample the latent vectors for each vertex from a Gaussian distribution as follows:
$$
\boldsymbol {z} _ {v _ {i}} \sim \mathcal {N} \left(\operatorname {M L P} _ {\phi} \left(\boldsymbol {h} _ {v _ {i}}\right), \exp \operatorname {M L P} _ {\phi} \left(\boldsymbol {h} _ {v _ {i}}\right)\right),
$$
$$
\boldsymbol {z} _ {w _ {j}} \sim \mathcal {N} \left(\mathrm {M L P} _ {\phi} \left(\boldsymbol {h} _ {w _ {j}}\right), \exp \mathrm {M L P} _ {\phi} \left(\boldsymbol {h} _ {w _ {j}}\right)\right). \tag {26}
$$
A.3.2 Decoder
The decoder implements  $p_{\theta}(\mathcal{G}|\mathbf{Z},\tilde{\mathcal{G}})$  in Equation 22. It utilizes a GNN structure to obtain the representations, which has the same structure as the encoder GNN, but is with parameters  $\theta$  instead of  $\phi$ . To encode the masked graph, we assign a special [mask] token to the masked vertex  $\tilde{v}$ . Its initial embedding  $h_{\tilde{v}}^{(0)}$  is initialized as a special embedding  $h_{[\mathrm{mask}]}$ . We mask all edges between  $\tilde{v}$  and the variable vertices and add virtual edges. In each convolution layer, we apply a special update rule for  $\tilde{v}$ :
$$
\boldsymbol {h} _ {\tilde {v}} ^ {(k + 1)} \leftarrow \operatorname {M L P} _ {\boldsymbol {\theta}} \left(\boldsymbol {h} _ {\tilde {v}} ^ {(k)}, \underset {w _ {j} \in \mathcal {W}} {\text {M E A N}} \left(\boldsymbol {h} _ {w _ {j}} ^ {(k + 1)}\right)\right), \quad \boldsymbol {h} _ {w _ {j}} ^ {(k + 1)} \leftarrow \operatorname {M L P} _ {\boldsymbol {\theta}} \left(\boldsymbol {h} _ {w _ {j}} ^ {(k + 1)}, \boldsymbol {h} _ {\tilde {v}} ^ {(k + 1)}\right). \tag {27}
$$
This updating is performed after each convolution layer, allowing  $\tilde{v}$  to aggregate and propagate the information from the entire graph.
The obtained representations are used for the four networks—Bias Predictor, Degree Predictor, Logits Predictor, and Weights Predictor—to determine the generated graph. The details of these networks have been described in the main paper. Here we provide the losses for the four prediction tasks. In the following context, the node features, e.g.,  $h_{\tilde{v}}$ , refer to those from  $\tilde{\mathcal{G}}$  obtained by the decoder GNN.
$①$  Bias Prediction Loss:
$$
\mathcal {L} _ {1} (\boldsymbol {\theta}, \phi | \mathcal {G}, \tilde {\mathcal {G}}) = \operatorname {M S E} \left(\sigma \left(\operatorname {M L P} _ {\boldsymbol {\theta}} ^ {\text {b i a s}} \left([ \boldsymbol {h} _ {\tilde {v}}, \boldsymbol {z} _ {\tilde {v}} ]\right)\right), b _ {\tilde {v}} ^ {*}\right). \tag {28}
$$
$②$  Degree Prediction Loss:
$$
\mathcal {L} _ {2} (\boldsymbol {\theta}, \phi | \mathcal {G}, \tilde {\mathcal {G}}) = \operatorname {M S E} \left(\sigma \left(\operatorname {M L P} _ {\boldsymbol {\theta}} ^ {\deg} ([ \boldsymbol {h} _ {\tilde {v}}, \boldsymbol {z} _ {\tilde {v}} ])\right), d _ {\tilde {v}} ^ {*}\right). \tag {29}
$$
③ Logits Prediction Loss:
$$
\mathcal {L} _ {3} (\boldsymbol {\theta}, \phi | \mathcal {G}, \tilde {\mathcal {G}}) = - \mathbb {E} _ {(\tilde {v}, u) \sim p _ {\mathrm {p o s}}} \left[ \log \left(\hat {\delta} _ {\tilde {v}, u} ^ {\prime}\right) \right] - \mathbb {E} _ {(\tilde {v}, u) \sim p _ {\mathrm {n e g}}} \left[ \log \left(1 - \hat {\delta} _ {\tilde {v}, u} ^ {\prime}\right) \right],
$$
$$
\hat {\delta} _ {\tilde {v}, u} ^ {\prime} = \sigma \left(\mathrm {M L P} _ {\theta} ^ {\log_ {1 0} (\left[ h _ {u}, z _ {u} \right])}\right). \tag {30}
$$
$④$  Weights Prediction Loss:
$$
\mathcal {L} _ {4} (\boldsymbol {\theta}, \phi | \mathcal {G}, \tilde {\mathcal {G}}) = \operatorname {M S E} \left(\sigma \left(\operatorname {M L P} _ {\boldsymbol {\theta}} ^ {\text {w e i g h t s}} \left(\left[ \boldsymbol {h} _ {u}, \boldsymbol {z} _ {u} \right]\right)\right), e _ {\tilde {v}, u} ^ {*}\right). \tag {31}
$$
With these four prediction tasks, the reconstruction loss in Equation 22 is instantiated as:
$$
\mathcal {L} _ {\text {r e c}} (\boldsymbol {\theta}, \phi | \mathcal {G}, \tilde {\mathcal {G}}) = \sum_ {i = 1} ^ {4} \alpha_ {i} \cdot \mathcal {L} _ {i} (\boldsymbol {\theta}, \phi | \mathcal {G}, \tilde {\mathcal {G}}), \tag {32}
$$
A.3.3 Training and Inference
We describe the training and inference procedures in Algorithm 1 and Algorithm 2, respectively.
Algorithm 1: Train G2MILP
Input: Dataset  $\mathcal{D}$  , number of training steps  $N$  , batch size B. Output: Trained G2MILP, dataset statistics  $\underline{b},\overline{b},\underline{d},\overline{d},\underline{e},\overline{e}$    
1 Calculate the statistics  $\underline{b},\overline{b},\underline{d},\overline{d},\underline{e},\overline{e}$  over  $\mathcal{D}$  .   
2 for  $n = 1,\dots ,N$  do   
3  $\mathcal{B}\gets \emptyset$    
4 for  $b = 1,\dots ,B$  do   
5  $\begin{array}{r}\mathcal{G}\sim \mathcal{D},\tilde{v}\sim \mathcal{V}^{\mathcal{G}};\\ \tilde{\mathcal{G}}\leftarrow \mathcal{G}.MaskNode(\tilde{v});\\ \mathcal{B}\leftarrow \mathcal{B}\cup \{(\mathcal{G},\tilde{\mathcal{G}})\} ;\\ \mathrm{Compute} b_{\tilde{v}}^{*},d_{\tilde{v}}^{*},\delta_{\tilde{v},u},e_{\tilde{v},u}^{*};\\ \mathrm{Compute}\mathcal{L}(\boldsymbol {\theta},\boldsymbol {\phi}|\mathcal{G},\tilde{\mathcal{G}})\mathrm{in~Equation~}22;\\ \mathcal{L}(\boldsymbol {\theta},\boldsymbol {\phi})\leftarrow \frac{1}{|B|}\sum_{(\mathcal{G},\tilde{\mathcal{G}})\in \mathcal{B}}\mathcal{L}(\boldsymbol {\theta},\boldsymbol {\phi}|\mathcal{G},\tilde{\mathcal{G}});\\ \mathrm{Update}\boldsymbol {\theta},\boldsymbol {\phi}\mathrm{to~minimize}\mathcal{L}(\boldsymbol {\theta},\boldsymbol {\phi}). \end{array}$    
7   
8   
9   
10   
11
Algorithm 2: Generate a MILP instance
Input: Dataset  $\mathcal{D}$  , trained G2MILP, dataset statistics  $\underline{b},\overline{b},\underline{d},\overline{d},\underline{e},\overline{e}$  , masking ratio  $\eta$  Output: A novel instance  $\hat{\mathcal{G}}$    
1  $\mathcal{G}\sim \mathcal{D}$ $N_{\mathrm{iters}}\gets \eta \cdot |\mathcal{V}^{\mathcal{G}}|,\hat{\mathcal{G}}\gets \mathcal{G};$    
2 for  $n = 1,\dots ,N_{\mathrm{iters}}$  do   
3  $\tilde{v}\sim \mathcal{V}^{\tilde{\mathcal{G}}};$    
4 Compute  $\hat{b}_{\tilde{v}}^{*},\hat{b}_{\tilde{v}}\gets \underline{b} +(\bar{b} -\underline{b})\cdot \hat{b}_{\tilde{v}}^{*},\tilde{\mathcal{G}}.\tilde{v}.\mathrm{bias}\gets \hat{b}_{\tilde{v}};$    
5 Compute  $\hat{d}_{\tilde{v}}^{*},\hat{d}_{\tilde{v}}\gets \underline{d} +(\bar{d} -\underline{d})\cdot \hat{d}_{\tilde{v}}^{*};$    
6 for  $u\in \mathcal{W}^{\tilde{\mathcal{G}}}~\mathbf{do}$    
7 Compute  $\hat{\delta}_{\tilde{v},u}^{\prime}$    
8 for  $u\in \arg \operatorname {TopK}(\{\hat{\delta}_{\tilde{v},u}^{\prime}|u\in \mathcal{W}^{\tilde{\mathcal{G}}}\} ,\hat{d}_{\tilde{v}})$  do   
9 Compute  $\hat{e}_{\tilde{v},u}^{*},\hat{e}_{\tilde{v},u}\gets \underline{e} +(\overline{e} -\underline{e})\cdot \hat{e}_{\tilde{v},u}^{*};$    
10  $\tilde{\mathcal{G}}.\mathrm{AddEdge}(\tilde{v},u);$    
11  $\tilde{\mathcal{G}}.e_{\tilde{v},u}$  .weights  $\leftarrow \hat{e}_{\tilde{v},u}$    
12  $\hat{\mathcal{G}}\gets \tilde{\mathcal{G}}$    
13 Output  $\hat{\mathcal{G}}$
B Experimental Details
B.1 Dataset
The three commonly used datasets, namely MIS, SetCover, and MIK, are the same as those used in [9]. Nurse Scheduling contains a group of 4 instances from MIPLIB 2017: nursesched-medium04 and nursesched-sprint-hidden09 for training, and nursesched-sprint02 and nursesched-sprint-late03 for test. Table 6 summarizes some statistics of these datasets.
Table 6: Statistics of datasets. Size means the number of instances in the training set.  $|\mathcal{V}|$  and  $|\mathcal{W}|$  are the numbers of constraints and variables, respectively.
Dataset	MIS	SetCover	MIK	Nurse Scheduling
Size	1000	1000	80	2
Mean |V|	1953	500	346	8707
Mean |W|	500	1000	413	20659
B.2 Hyperparameters
We report some important hyperparameters in this section. Further details can be found in our code once the paper is accepted to be published.
We run our model on a single GeForce RTX 3090 GPU. The hidden dimension and the embedding dimension are set to 16. The depth of the GNNs is 6. Each MLP has one hidden layer and uses ReLU() as the activation function.
In this work, we simply set all  $\alpha_{i}$  to 1. We find that the choice of  $\beta$  significantly impacts the model performance. For MIS, we set  $\beta$  to 0.00045. For SetCover, MIK and Nurse Scheduling, we apply a sigmoid schedule [46] to let  $\beta$  to reach 0.0005, 0.001, and 0.001, respectively. We employ the Adam optimizer, train the model for 20,000 steps, and choose the best checkpoint based on the average error in solving time and the number of branching nodes. The learning rate is initialized to 0.001 and decays exponentially. For MIS, SetCover, and MIK, we set the batch size to 30. Specifically, to provide more challenging prediction tasks in each batch, we sample 15 graphs and use each graph to derive 2 masked ones for training. For Nurse Scheduling, we set the batch size as 1 due to the large size of each graph.
B.3 Structural Distributional Similarity
Table 7: Description of statistics used for measuring the structural distributional similarity. These statistics are calculated on the bipartite graph extracted by Ecole.
Feature	Description
coef_dens	Fraction of non-zero entries in A, i.e., |E|/(|V|·|W|).
consdegree_mean	Mean degree of constraint vertices in V.
consdegree_std	Std of degrees of constraint vertices in V.
vardegree_mean	Mean degree of variable vertices in W.
vardegree_std	Std of degrees of variance vertices in W.
lhs_mean	Mean of non-zero entries in A.
lhs_std	Std of non-zero entries in A.
rhs_mean	Mean of b.
rhs_std	Std of b.
clustering coef	Clustering coefficient of the graph.
modularity	Modularity of the graph.
Table 7 presents the 11 statistics that we use to measure the structural distributional similarity. First, we calculate the statistics for each instance. We then compute the JS divergence  $D_{\mathrm{JS},i}$  between the generated samples and the training set for each descriptor  $i \in \{1,\dots ,11\}$ . We estimate the distributions using the histogram function in numpy and the cross entropy using the entropy function in scipy. The JS divergence falls in the range [0, log 2], so we standardize it to a score  $s_i$  via:
$$
s _ {i} = \frac {1}{\log 2} (\log 2 - D _ {\mathrm {J S}, i}). \tag {33}
$$
Then we compute the mean of the 11 scores for the descriptors to obtain the final score  $s$ :
$$
s = \frac {1}{1 1} \sum_ {i = 1} ^ {1 1} s _ {i}. \tag {34}
$$
Hence the final score ranges from 0 to 1, with a higher score indicating better similarity.
We use the training set to train a G2MILP model for each dataset and generate 1000 instances to compute the similarity scores. For MIK, which has only 80 training instances, we estimated the score using put-back sampling.
Table 8: Results on the optimal value prediction task (mean±std). On each dataset and for each method, we sample 5 different sets of 20 instances for augmentation.
MIK	Nurse Scheduling
MSE	Improvement	MSE	Improvement
Dataset	0.0236	0.0%	679.75	0.0%
Bowly	-	-	663.52 (±95.33)	2.3% (±14.0%)
Random	0.0104 (±0.0023)	55.9% (±9.7%)	-	-
G2MILP	0.0073 (±0.0014)	69.1% (±5.9%)	548.70 (±44.68)	19.3% (±6.6%)
Table 9: Results on the predict-and-search framework on MIS. The training set contains 100 instances, and we generate 100 new instances. For Random and G2MILP, masking ratio is 0.01. Time means the time for Gurobi to find the optimal solution with augmenting data generated by different models. Bowly leads to the framework failing to find optimal solutions in the trust region.
Method	Training Set	Bowly	Random	G2MILP
Time	0.041	17/100 fail	0.037	0.032
(±0.006)	(±0.003)	(±0.004)
Notice that for a fair comparison, we exclude statistics that remain constant in our approach, such as problem size and objective coefficients. We implement another version of metric that involves more statistics, and the results are in Appendix C.3.
B.4 Downstream Tasks
The generated instances have the potential to enrich dataset in any downstream task. In this work, we demonstrate this potential through two application scenarios, i.e., the optimal value prediction task and the predict-and-search framework.
Optimal Value Prediction Two datasets, MIK and Nurse Scheduling, are considered, with medium and extremely small sizes, respectively. Following [18], we employ a GNN as a predictive model. The GNN structure is similar to the GNNs in G2MIL. We obtain the graph representation using mean pooling over all vertices, followed by a two-layer MLP to predict the optimal values of the instances.
For each dataset, we train a GNN predictive model on the training set. Specifically, for MIK, we use  $80\%$  of instances for training,  $20\%$  of instances for validating, and train for 1000 epochs to select the best checkpoint based on validation MSE. For Nurse Scheduling, we use both instances to train the model for 80 epochs. We use the generative models, Bowly, Random, and G2MILP, to generate 20 instances similar to the training sets. For Random and G2MILP, we mix together the instances generated by setting the masking ratio  $\eta$  to 0.01 and 0.05, respectively. Next, we use the generated instances to enrich the original training sets, and use the enriched data to train another predictive model. We test all the trained model on previously unseen test data. Table 8 presents the predictive MSE on the test sets of the models trained on different training sets. As the absolute values of MSE are less meaningful than the relative values, we report the performance improvements brought by different generative technique. The improvement of  $\mathrm{Model}_2$  relative to  $\mathrm{Model}_1$  is calculated as follows:
$$
\text {I m p r o v e m e n t} _ {2, 1} = \frac {\mathrm {M S E} _ {1} - \mathrm {M S E} _ {2}}{\mathrm {M S E} _ {1}}. \tag {35}
$$
On MIK, Bowly results in numerical issues as some generated coefficients are excessively large. G2MILP significant improves the performance and outperforms Random. On Nurse Scheduling, Random fails to generate feasible instances, and Bowly yields a minor improvement. Notably, G2MILP allows for the training of the model with even minimal data.
Predict-and-Search We conduct experiments on a neural solver, i.e., the predict-and-search framework proposed by Han et al. [31] Specifically, they propose a framework that first predicts a solution and then uses solvers to search for the optimal solutions in a trust region. We consider using generated instances to enhance the predictive model. We first train the predictive model on 100 MIS instances, and then use the generative models to generate 100 new instances to augment the dataset. The results
are in Table 9. Bowly generates low-quality data that disturbs the model training, so that there is no optimal solution in the trust region around the predicted solution. Though both Random and G2MILP can enhance the solving framework to reduce solving time, we can see G2MILP significantly outperforms Random.
**Discussions** These two downstream tasks, despite their simplicity, possess characteristics that make them representative problems that could benefit from generative models. Specifically, we identify the following features.
1. More is better. We want as many data instances as possible. This condition is satisfied when we can obtain precise labels using existing methods, e.g., prediction-based neural solvers [31], or when unlabeled data is required for RL model training, e.g., RL for cut selection [9].
2. More similar is better. We want independent identically distributed (i.i.d.) data instances for training. Thus
3. More diverse is better. We want the data to be diverse, despite being i.i.d., so that the trained model can generalize better.
Our experimental results demonstrate the potential of G2MILP in facilitating downstream tasks with these characteristics, thus enhancing the MILP solvers. We intend to explore additional application scenarios in future research.
C.1 Comparison with G2SAT
Table 10: Results of G2SAT on MIS. In the table, "sim" denotes similarity score (higher is better), "time" denotes solving time, and "#branch" denotes the number of branching nodes, respectively. Numbers in brackets denote relative errors (lower is better).
sim	time (s)	#branch
Training Set	0.998	0.349	16.09
G2SAT	0.572	0.014 (96.0%)	2.11 (86.9%)
G2MILP (η = 0.01)	0.997	0.354 (1.5%)	15.03 (6.6%)
G2MILP (η = 0.1)	0.895	0.214 (38.7%)	4.61 (71.3%)
We conduct an additional experiment that transfers G2SAT to a special MILP dataset, MIS, in which all coefficients are 1.0 and thus the instances can be modeled as homogeneous bipartite graphs. We apply G2SAT to learn to generate new graphs and convert them to MILPs. Results are in Table 10. The results show that G2MILP significantly outperforms G2SAT on the special cases.
C.2 Masking Process