SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Masking Variables In the mainbody, for simplicity, we define the masking process of uniformly sampling a constraint vertex  $\tilde{v} \sim \mathcal{U}(\mathcal{V})$  to mask, while keeping the variable vertices unchanged. We implement different versions of G2MILP that allow masking and modifying either constraints, variables, or both. The results are in Table 11.
Ablation on Masking Ratio We have conduct ablation studies on the effect of the masking ratio  $\eta$  on MIK. The results are in Figure 4. The experimental settings are the same with those in Table 2 and Figure 2. From the results we have the following conclusions. (1) though empirically a smaller leads to a relatively better performance, G2MILP maintains a high similarity performance even when  $\eta$  is large. (2) The downstream task performance does not drops significantly. This makes sense because smaller  $\eta$  leads to more similar instances, while larger  $\eta$  leads to more diverse (but still similar) instances, both of which can benefit downstream tasks. (3) G2MILP always outperforms Random,
Table 11: Results of different implementations of G2MILP on MIK. In the table,  $\eta$  denotes mask ratio, "v" denotes only modifying variables (objective coefficients and variable types), "c" denotes only modifying constraints, and "vc" denotes first modifying variables and then modifying constraints. We do not report the similarity scores for "v" models because the current similarity metrics exclude statistics that measure only variables.
sim	time (s)	#branch
Training Set	0.997	0.198	175.35
G2MILP(η = 0.01)	v	-	0.183 (7.5%)	136.68 (22.0%)
c	0.989	0.169 (17.1%)	167.44 (4.5%)
vc	0.986	0.186 (6.1%)	155.40 (11.4%)
G2MILP(η = 0.05)	v	-	0.176 (11.1%)	136.68 (22.0%)
c	0.964	0.148 (25.3%)	150.90 (13.9%)
vc	0.964	0.147 (25.3%)	142.70 (18.6%)
G2MILP(η = 0.1)	v	-	0.172 (13.1%)	136.67 (22.1%)
c	0.905	0.117 (40.9%)	169.63 (3.3%)
vc	0.908	0.115 (41.9%)	112.29 (35.9%)
(a) Distributional Similarity
(b) Relative MSE
Figure 4: (a) Distributional similarity score (higher is better) and (b) Relative MSE (lower is better) v.s. masking ratio  $\eta$ .
which demonstrates that the learning paradigm helps maintain the performance. (4) Bowly fails on this dataset because its generated instances lead to numerical issues and cannot be read by Gurobi or SCIP. Moreover, in real applications, it is reasonable and flexible to adjust the hyperparameter to achieve good performances in different scenarios.
Orders of Masked Constraints We also investigate different orders of masking constraint vertices, including uniformly sampling and sampling according to the vertex indices. Results are in Table 12. We find that uniformly sampling achieves the best performance. Sampling according to indices leads to a performance decrease, maybe because near constraints are relevant and lead to error accumulation. We think these results are interesting, and will study it in the future work.
C.3 Structural Distributional Similarity
In the mainbody, for a fair comparison, we exclude statistics that remain constant in our approach, such as problem size and objective coefficients. However, these statistics are also important features for MILPs. In this section, we incorporate three additional statistics in the computing of similarity scores: (1) mean of objective coefficients  $\mathbf{c}$ , (2) std of objective coefficients  $\mathbf{c}$ , and (3) the ratio of continuous variables. With these additional metrics, we recompute the structural similarity scores and updated the results in both Table 2 and Table 11. The new results are in Table 13 and Table 14,
Table 12: Results of different implementations of generation orders on MIK dataset. In the table, "Uni" denotes uniformly sampling from constraints. "Idx/” and "Idx\`" denote sampling constraints according to indices in ascending order and descending order, respectively.
order	model	sim	time (s)	#branch
Uni	G2MILP	0.953	0.129 (35.1%)	235.35 (34.2%)
Random	0.840	0.004 (97.9%)	0.00 (100%)
Idx\	G2MILP	0.892	0.054 (72.7%)	108.30 (38.2%)
Random	0.773	0.002 (98.9%)	0.00 (100%)
Idx\	G2MILP	0.925	0.027 (86.2%)	31.53 (82.0%)
Random	0.827	0.003 (98.6%)	0.00 (100%)
respectively. From the results, we can still conclude that G2MILP outperforms all baselines, further supporting the effectiveness of our proposed method.
Table 13: (Table 2 recomputed.) Structural distributional similarity scores between the generated instances with the training datasets. Higher is better.  $\eta$  is the masking ratio. We do not report the results of Bowly on MIK because Ecole [45] and SCIP [51] fail to read the generated instances due to large numerical values.
MIS	SetCover	MIK
Bowly	0.144	0.150	-
η = 0.01	Random	0.722	0.791	0.971
G2MILP	0.997	0.874	0.994
η = 0.05	Random	0.670	0.704	0.878
G2MILP	0.951	0.833	0.969
η = 0.1	Random	0.618	0.648	0.768
G2MILP	0.921	0.834	0.930
Table 14: (Table 11 recomputed.) Results of different implementations of G2MILP on MIK. In the table,  $\eta$  denotes mask ratio, "v" denotes only modifying variables (objective coefficients and variable types), "c" denotes only modifying constraints, and "vc" denotes first modifying variables and then modifying constraints.
v	c	cv
G2MILP (η = 0.01)	0.998	0.988	0.985
G2MILP (η = 0.05)	0.996	0.968	0.967
G2MILP (η = 0.1)	0.996	0.928	0.912
C.4 Sizes of Datasets
We conduct experiments on different sizes of the original datasets, as well as the ratio of generated instances to original ones, on MIS. The results are in Table 15. The results show that G2MILP can bring performance improvements on varying sizes of datasets.
C.5 Visualization
The t-SNE visualization for baselines are in Figure 5. G2MILP generates diverse instances around the training set, while instances generated by Random are more biased from the realistic ones.
Table 15: Results on the optimal value prediction task on MIS with different dataset sizes. In the table, "#MILPs" denotes the number of instances in the training sets, and "Augment%" denotes the ratio of generated instances to training instances.
#MILPs	Augment%	MSE	Improvement
50	0	1.318	0
25%	1.014	23.1%
50%	0.998	24.3%
100%	0.982	25.5%
100	0	0.798	0
25%	0.786	1.5%
50%	0.752	5.8%
100%	0.561	23.7%
200	0	0.294	0
25%	0.283	19.0%
50%	0.243	17.3%
100%	0.202	31.3%
500	0	0.188	0
25%	0.168	10.6%
50%	0.175	6.9%
100%	0.170	9.6%
Figure 5: The t-SNE visualization of MILP instance representations for MIK. Each point represents an instance. Red points are from the training set, blue points are instances generated by G2MILP, and green points are instances generated by Random.