SlowGuess/NeurIPS_2023_OCR huggingface dataset:
# of samples	method	upper bound	time(s)	gap(%)
3000	SOS1 + warm start	-7.41086 · 105	3600.63	18.58
3000	big-M50 + warm start	-7.41086 · 105	3601.32	17.88
3000	persp. + warm start	-7.41086 · 105	3600.59	18.35
3000	eig. persp. + warm start	-7.41086 · 105	3600.49	19.55
3000	ours	-7.41086 · 105	3600.48	8.32
4000	SOS1 + warm start	-1.00277 · 106	3600.31	15.06
4000	big-M50 + warm start	-1.00277 · 106	3600.44	14.85
4000	persp. + warm start	-1.00277 · 106	3600.06	13.03
4000	eig. persp. + warm start	-1.00277 · 106	3600.21	15.54
4000	ours	-1.00277 · 106	3600.62	1.66
5000	SOS1 + warm start	-1.24017 · 106	3600.99	12.01
5000	big-M50 + warm start	-1.24017 · 106	3600.27	11.82
5000	persp. + warm start	-1.24017 · 106	3600.75	10.76
5000	eig. persp. + warm start	-1.24017 · 106	3600.07	12.59
5000	ours	-1.24017 · 106	386.72	0.00
6000	SOS1 + warm start	-1.45406 · 106	3600.07	9.97
6000	big-M50 + warm start	-1.45406 · 106	3600.26	9.75
6000	persp. + warm start	-1.45406 · 106	3600.31	9.44
6000	eig. persp. + warm start	-1.45406 · 106	2855.45	0.00
6000	ours	-1.45406 · 106	54.13	0.00
7000	SOS1 + warm start	-1.70473 · 106	3601.18	8.32
7000	big-M50 + warm start	-1.70473 · 106	3600.20	8.32
7000	persp. + warm start	-1.70473 · 106	3600.18	7.85
7000	eig. persp. + warm start	-1.70473 · 106	1264.32	0.00
7000	ours	-1.70473 · 106	21.30	0.00
Table 59: Comparison of time and optimality gap on the synthetic datasets (p = 3000, k = 10, ρ = 0.7, λ2 = 10.0). All baselines use our beam search solution as a warm start.
# of samples	method	upper bound	time(s)	gap(%)
3000	SOS1 + warm start	-2.78507 · 10^6	3601.20	18.57
3000	big-M50 + warm start	-2.78507 · 10^6	3600.07	15.95
3000	persp. + warm start	-2.78507 · 10^6	3600.06	18.67
3000	eig. persp. + warm start	-2.78507 · 10^6	3600.36	19.55
3000	ours	-2.78507 · 10^6	3600.78	9.79
4000	SOS1 + warm start	-3.75192 · 10^6	3600.30	14.97
4000	big-M50 + warm start	-3.75192 · 10^6	3600.06	13.19
4000	persp. + warm start	-3.75192 · 10^6	3600.05	13.88
4000	eig. persp. + warm start	-3.75192 · 10^6	3600.06	15.77
4000	ours	-3.75192 · 10^6	3600.32	3.23
5000	SOS1 + warm start	-4.61867 · 10^6	3601.14	12.04
5000	big-M50 + warm start	-4.61867 · 10^6	3600.06	11.32
5000	persp. + warm start	-4.61867 · 10^6	3600.06	11.39
5000	eig. persp. + warm start	-4.61867 · 10^6	3600.07	12.85
5000	ours	-4.61867 · 10^6	3600.30	0.89
6000	SOS1 + warm start	-5.43816 · 10^6	3600.06	9.98
6000	big-M50 + warm start	-5.43816 · 10^6	3600.39	9.75
6000	persp. + warm start	-5.43816 · 10^6	3600.10	9.52
6000	eig. persp. + warm start	-5.43816 · 10^6	3600.13	11.05
6000	ours	-5.43816 · 10^6	3600.22	0.32
7000	SOS1 + warm start	-6.36757 · 10^6	3600.54	8.33
7000	big-M50 + warm start	-6.36757 · 10^6	3600.10	8.20
7000	persp. + warm start	-6.36757 · 10^6	3600.13	7.98
7000	eig. persp. + warm start	-6.36757 · 10^6	3600.25	0.34
7000	ours	-6.36757 · 10^6	3600.06	0.10
Table 60: Comparison of time and optimality gap on the synthetic datasets (p = 3000, k = 10, ρ = 0.9, λ2 = 10.0). All baselines use our beam search solution as a warm start.
G.3 Comparison with the Big-M Formulation with different  $M$  Values
G.3.1 Synthetic 1 Benchmark with  $\lambda_{2} = 0.001$  with No Warmstart
# of features	method	upper bound	time(s)	gap(%)
100	big-M50	-2.10874 · 10^6	0.39	0.00
100	big-M20	-2.10878 · 10^6	0.44	0.00
100	big-M5	-2.10878 · 10^6	0.32	0.00
100	ours	-2.10878 · 10^6	0.29	0.00
500	big-M50	-2.12828 · 10^6	36.80	0.00
500	big-M20	-2.12833 · 10^6	81.36	0.00
500	big-M5	-2.12833 · 10^6	52.95	0.00
500	ours	-2.12833 · 10^6	0.37	0.00
1000	big-M50	-2.11005 · 10^6	1911.41	0.00
1000	big-M20	-2.11005 · 10^6	1260.21	0.00
1000	big-M5	-2.11005 · 10^6	771.63	0.00
1000	ours	-2.11005 · 10^6	0.58	0.00
3000	big-M50	-7.83869 · 10^5	3601.20	169.78
3000	big-M20	-1.14989 · 10^6	3603.59	83.91
3000	big-M5	-2.10230 · 10^6	1998.40	0.00
3000	ours	-2.10230 · 10^6	1.73	0.00
5000	big-M50	-1.87965 · 10^6	3603.25	12.65
5000	big-M20	-9.85913 · 10^5	3602.87	114.78
5000	big-M5	-	3618.45	-
5000	ours	-2.09635 · 10^6	4.22	0.00
Table 61: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10,  $\rho = 0.1$ ,  $\lambda_2 = 0.001$ ).
# of features	method	upper bound	time(s)	gap(%)
100	big-M50	-5.28725 · 10^6	0.42	0.00
100	big-M20	-5.28726 · 10^6	0.45	0.00
100	big-M5	-5.28726 · 10^6	0.36	0.00
100	ours	-5.28726 · 10^6	0.29	0.00
500	big-M50	-5.31819 · 10^6	89.04	0.00
500	big-M20	-5.31845 · 10^6	75.86	0.00
500	big-M5	-5.31852 · 10^6	51.98	0.00
500	ours	-5.31852 · 10^6	0.41	0.00
1000	big-M50	-5.26974 · 10^6	1528.98	0.00
1000	big-M20	-5.26973 · 10^6	1180.33	0.00
1000	big-M5	-5.26974 · 10^6	794.14	0.00
1000	ours	-5.26974 · 10^6	0.52	0.00
3000	big-M50	-3.72978 · 10^6	3602.55	42.31
3000	big-M20	-3.97937 · 10^6	3604.47	33.38
3000	big-M5	-	3660.43	-
3000	ours	-5.27668 · 10^6	1.75	0.00
5000	big-M50	-4.16126 · 10^6	3602.53	27.95
5000	big-M20	-4.20081 · 10^6	3601.38	26.74
5000	big-M5	-5.27126 · 10^6	3600.14	1.00
5000	ours	-5.27126 · 10^6	4.32	0.00
Table 62: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.3, λ2 = 0.001).
# of features	method	upper bound	time(s)	gap(%)
100	big-M50	-1.10084 · 107	0.41	0.00
100	big-M20	-1.10084 · 107	0.37	0.00
100	big-M5	-1.10084 · 107	0.33	0.00
100	ours	-1.10084 · 107	0.29	0.00
500	big-M50	-1.10515 · 107	89.59	0.00
500	big-M20	-1.10518 · 107	79.56	0.00
500	big-M5	-1.10518 · 107	54.59	0.00
500	ours	-1.10518 · 107	0.40	0.00
1000	big-M50	-1.09506 · 107	1757.09	0.00
1000	big-M20	-1.09508 · 107	1133.80	0.00
1000	big-M5	-1.09508 · 107	744.15	0.00
1000	ours	-1.09508 · 107	0.59	0.00
3000	big-M50	-	3615.51	-
3000	big-M20	-	3635.15	-
3000	big-M5	-	3648.05	-
3000	ours	-1.10012 · 107	1.74	0.00
5000	big-M50	-	3612.62	-
5000	big-M20	-1.01062 · 107	3612.19	9.90
5000	big-M5	-	3605.20	-
5000	ours	-1.09966 · 107	4.00	0.00
Table 63: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.5, λ2 = 0.001).
# of features	method	upper bound	time(s)	gap(%)
100	big-M50	-2.43573 · 10^7	0.46	0.00
100	big-M20	-2.43575 · 10^7	0.38	0.00
100	big-M5	-2.43575 · 10^7	0.31	0.00
100	ours	-2.43575 · 10^7	0.32	0.00
500	big-M50	-2.44178 · 10^7	92.53	0.00
500	big-M20	-2.44185 · 10^7	75.08	0.00
500	big-M5	-2.44185 · 10^7	52.27	0.00
500	ours	-2.44185 · 10^7	0.46	0.00
1000	big-M50	-2.41987 · 10^7	1361.05	0.00
1000	big-M20	-2.41988 · 10^7	974.43	0.00
1000	big-M5	-2.41989 · 10^7	681.46	0.00
1000	ours	-2.41989 · 10^7	0.52	0.00
3000	big-M50	-	3640.12	-
3000	big-M20	-	3637.70	-
3000	big-M5	-2.43713 · 10^7	3600.23	0.56
3000	ours	-2.43713 · 10^7	1.77	0.00
5000	big-M50	-2.43694 · 10^7	3643.28	1.00
5000	big-M20	-2.43694 · 10^7	3602.42	1.00
5000	big-M5	-	3631.91	-
5000	ours	-2.43694 · 10^7	4.15	0.00
Table 64: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.7, λ2 = 0.001).
# of features	method	upper bound	time(s)	gap(%)
100	big-M50	-9.11026 · 107	0.13	0.00
100	big-M20	-9.11026 · 107	0.13	0.00
100	big-M5	-9.11026 · 107	0.10	0.00
100	ours	-9.11026 · 107	0.30	0.00
500	big-M50	-9.12047 · 107	94.03	0.00
500	big-M20	-9.12047 · 107	90.67	0.00
500	big-M5	-9.12047 · 107	185.31	0.00
500	ours	-9.12047 · 107	0.38	0.00
1000	big-M50	-9.04075 · 107	3600.01	0.07
1000	big-M20	-9.04075 · 107	3600.50	0.20
1000	big-M5	-9.04075 · 107	2081.96	0.00
1000	ours	-9.04075 · 107	0.52	0.00
3000	big-M50	-	3605.36	-
3000	big-M20	-	3623.06	-
3000	big-M5	-	3611.34	-
3000	ours	-9.12761 · 107	20.74	0.00
5000	big-M50	-	3607.68	-
5000	big-M20	-	3616.13	-
5000	big-M5	-	3627.90	-
5000	ours	-9.12916 · 107	58.29	0.00
Table 65: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.9, λ2 = 0.001).
G.3.2 Synthetic 1 Benchmark with  $\lambda_{2} = 0.001$  with Warmstart
# of features	method	upper bound	time(s)	gap(%)
100	big-M50 + warm start	-2.10878 · 10^6	0.08	0.00
100	big-M20 + warm start	-2.10878 · 10^6	0.07	0.00
100	big-M5 + warm start	-2.10878 · 10^6	0.05	0.00
100	ours	-2.10878 · 10^6	0.29	0.00
500	big-M50 + warm start	-2.12833 · 10^6	37.62	0.00
500	big-M20 + warm start	-2.12833 · 10^6	2.82	0.00
500	big-M5 + warm start	-2.12833 · 10^6	3.00	0.00
500	ours	-2.12833 · 10^6	0.37	0.00
1000	big-M50 + warm start	-2.11005 · 10^6	28.51	0.00
1000	big-M20 + warm start	-2.11005 · 10^6	17.68	0.00
1000	big-M5 + warm start	-2.11005 · 10^6	25.46	0.00
1000	ours	-2.11005 · 10^6	0.58	0.00
3000	big-M50 + warm start	-2.10230 · 10^6	407.92	0.00
3000	big-M20 + warm start	-2.10230 · 10^6	255.31	0.00
3000	big-M5 + warm start	-2.10230 · 10^6	400.99	0.00
3000	ours	-2.10230 · 10^6	1.73	0.00
5000	big-M50 + warm start	-2.09635 · 10^6	1380.38	0.00
5000	big-M20 + warm start	-2.09635 · 10^6	1107.68	0.00
5000	big-M5 + warm start	-2.09635 · 10^6	1673.77	0.00
5000	ours	-2.09635 · 10^6	4.22	0.00
Table 66: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.1, λ2 = 0.001). All baselines use our beam search solution as a warm start.
# of features	method	upper bound	time(s)	gap(%)
100	big-M50 + warm start	-5.28726 · 10^6	0.26	0.00
100	big-M20 + warm start	-5.28726 · 10^6	0.17	0.00
100	big-M5 + warm start	-5.28726 · 10^6	0.05	0.00
100	ours	-5.28726 · 10^6	0.29	0.00
500	big-M50 + warm start	-5.31852 · 10^6	3.00	0.00
500	big-M20 + warm start	-5.31852 · 10^6	12.76	0.00
500	big-M5 + warm start	-5.31852 · 10^6	1.94	0.00
500	ours	-5.31852 · 10^6	0.41	0.00
1000	big-M50 + warm start	-5.26974 · 10^6	13.79	0.00
1000	big-M20 + warm start	-5.26974 · 10^6	18.04	0.00
1000	big-M5 + warm start	-5.26974 · 10^6	12.19	0.00
1000	ours	-5.26974 · 10^6	0.52	0.00
3000	big-M50 + warm start	-5.27668 · 10^6	313.22	0.00
3000	big-M20 + warm start	-5.27668 · 10^6	461.86	0.00
3000	big-M5 + warm start	-5.27668 · 10^6	245.59	0.00
3000	ours	-5.27668 · 10^6	1.75	0.00
5000	big-M50 + warm start	-5.27126 · 10^6	1705.37	0.00
5000	big-M20 + warm start	-5.27126 · 10^6	1666.20	0.00
5000	big-M5 + warm start	-5.27126 · 10^6	3601.58	1.00
5000	ours	-5.27126 · 10^6	4.32	0.00
Table 67: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.3, λ2 = 0.001). All baselines use our beam search solution as a warm start.
# of features	method	upper bound	time(s)	gap(%)
100	big-M50 + warm start	-1.10084 · 107	0.06	0.00
100	big-M20 + warm start	-1.10084 · 107	0.06	0.00
100	big-M5 + warm start	-1.10084 · 107	0.04	0.00
100	ours	-1.10084 · 107	0.29	0.00
500	big-M50 + warm start	-1.10518 · 107	11.59	0.00
500	big-M20 + warm start	-1.10518 · 107	6.55	0.00
500	big-M5 + warm start	-1.10518 · 107	1.97	0.00
500	ours	-1.10518 · 107	0.40	0.00
1000	big-M50 + warm start	-1.09508 · 107	21.16	0.00
1000	big-M20 + warm start	-1.09508 · 107	14.95	0.00
1000	big-M5 + warm start	-1.09508 · 107	12.99	0.00
1000	ours	-1.09508 · 107	0.59	0.00
3000	big-M50 + warm start	-1.10012 · 107	243.42	0.00
3000	big-M20 + warm start	-1.10012 · 107	301.69	0.00
3000	big-M5 + warm start	-1.10012 · 107	242.64	0.00
3000	ours	-1.10012 · 107	1.74	0.00
5000	big-M50 + warm start	-1.09966 · 107	3600.95	1.00
5000	big-M20 + warm start	-1.09966 · 107	3600.28	1.00
5000	big-M5 + warm start	-1.09966 · 107	3602.10	570161330.64
5000	ours	-1.09966 · 107	4.00	0.00
Table 68: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.5, λ2 = 0.001). All baselines use our beam search solution as a warm start. The method big-M5 has some large optimality gaps because Gurobi couldn't finish solving the root relaxation within 1h.
# of features	method	upper bound	time(s)	gap(%)
100	big-M50 + warm start	-2.43575 · 107	0.05	0.00
100	big-M20 + warm start	-2.43575 · 107	0.26	0.00
100	big-M5 + warm start	-2.43575 · 107	0.05	0.00
100	ours	-2.43575 · 107	0.32	0.00
500	big-M50 + warm start	-2.44185 · 107	4.02	0.00
500	big-M20 + warm start	-2.44185 · 107	2.15	0.00
500	big-M5 + warm start	-2.44185 · 107	20.44	0.00
500	ours	-2.44185 · 107	0.46	0.00
1000	big-M50 + warm start	-2.41989 · 107	11.63	0.00
1000	big-M20 + warm start	-2.41989 · 107	13.11	0.00
1000	big-M5 + warm start	-2.41989 · 107	10.49	0.00
1000	ours	-2.41989 · 107	0.52	0.00
3000	big-M50 + warm start	-2.43713 · 107	3608.31	0.59
3000	big-M20 + warm start	-2.43713 · 107	3600.14	0.59
3000	big-M5 + warm start	-2.43713 · 107	3600.95	0.56
3000	ours	-2.43713 · 107	1.77	0.00
5000	big-M50 + warm start	-2.43694 · 107	3670.94	1.00
5000	big-M20 + warm start	-2.43694 · 107	3604.91	1.00
5000	big-M5 + warm start	-2.43694 · 107	3646.34	600317552.81
5000	ours	-2.43694 · 107	4.15	0.00
Table 69: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.7, λ2 = 0.001). All baselines use our beam search solution as a warm start. The method big-M5 has some large optimality gaps because Gurobi couldn't finish solving the root relaxation within 1h.
# of features	method	upper bound	time(s)	gap(%)
100	big-M50 + warm start	-9.11026 · 107	0.03	0.00
100	big-M20 + warm start	-9.11026 · 107	0.03	0.00
100	big-M5 + warm start	-9.11026 · 107	0.04	0.00
100	ours	-9.11026 · 107	0.30	0.00
500	big-M50 + warm start	-9.12047 · 107	34.86	0.00
500	big-M20 + warm start	-9.12047 · 107	28.75	0.00
500	big-M5 + warm start	-9.12047 · 107	18.67	0.00
500	ours	-9.12047 · 107	0.38	0.00
1000	big-M50 + warm start	-9.04075 · 107	3600.01	0.08
1000	big-M20 + warm start	-9.04075 · 107	3600.04	0.20
1000	big-M5 + warm start	-9.04075 · 107	1662.95	0.00
1000	ours	-9.04075 · 107	0.52	0.00
3000	big-M50 + warm start	-9.12761 · 107	3612.88	0.59
3000	big-M20 + warm start	-9.12761 · 107	3600.12	0.59
3000	big-M5 + warm start	-9.12761 · 107	3613.07	222997912.50
3000	ours	-9.12761 · 107	20.74	0.00
5000	big-M50 + warm start	-9.12916 · 107	3622.52	1.00
5000	big-M20 + warm start	-9.12916 · 107	3606.87	1.00
5000	big-M5 + warm start	-9.12916 · 107	3600.86	618133887.09
5000	ours	-9.12916 · 107	58.29	0.00
Table 70: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.9, λ2 = 0.001). All baselines use our beam search solution as a warm start. The method big-M5 has some large optimality gaps because Gurobi couldn't finish solving the root relaxation within 1h.
G.3.3 Synthetic 1 Benchmark with  $\lambda_{2} = 0.1$  with No Warmstart
# of features	method	upper bound	time(s)	gap(%)
100	big-M50	-2.10878 · 10^6	0.86	0.00
100	big-M20	-2.10878 · 10^6	0.41	0.00
100	big-M5	-2.10878 · 10^6	0.32	0.00
100	ours	-2.10878 · 10^6	0.36	0.00
500	big-M50	-2.12833 · 10^6	37.96	0.00
500	big-M20	-2.12833 · 10^6	27.57	0.00
500	big-M5	-2.12833 · 10^6	55.79	0.00
500	ours	-2.12833 · 10^6	0.37	0.00
1000	big-M50	-2.11005 · 10^6	1007.27	0.00
1000	big-M20	-2.11005 · 10^6	373.44	0.00
1000	big-M5	-2.11005 · 10^6	74.44	0.00
1000	ours	-2.11005 · 10^6	0.57	0.00
3000	big-M50	-7.99504 · 10^5	3608.36	164.51
3000	big-M20	-1.15295 · 10^6	3603.16	83.42
3000	big-M5	-2.10230 · 10^6	1289.57	0.00
3000	ours	-2.10230 · 10^6	1.79	0.00
5000	big-M50	-9.69753 · 10^5	3619.50	118.35
5000	big-M20	-9.85912 · 10^5	3611.26	114.78
5000	big-M5	-	3610.20	-
5000	ours	-2.09635 · 10^6	4.20	0.00
Table 71: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.1, λ2 = 0.1).
# of features	method	upper bound	time(s)	gap(%)
100	big-M50	-4.60269 · 10^6	1.83	0.00
100	big-M20	-5.28726 · 10^6	0.40	0.00
100	big-M5	-5.28726 · 10^6	0.34	0.00
100	ours	-5.28726 · 10^6	0.27	0.00
500	big-M50	-5.31827 · 10^6	85.52	0.00
500	big-M20	-5.31844 · 10^6	72.87	0.00
500	big-M5	-5.31852 · 10^6	52.81	0.00
500	ours	-5.31852 · 10^6	0.41	0.00
1000	big-M50	-5.26974 · 10^6	1576.75	0.00
1000	big-M20	-5.26974 · 10^6	1112.97	0.00
1000	big-M5	-5.26974 · 10^6	791.17	0.00
1000	ours	-5.26974 · 10^6	0.52	0.00
3000	big-M50	-3.73049 · 10^6	3603.09	42.28
3000	big-M20	-3.98562 · 10^6	3602.15	33.17
3000	big-M5	-	3634.94	-
3000	ours	-5.27668 · 10^6	1.84	0.00
5000	big-M50	-3.98069 · 10^6	3606.79	33.75
5000	big-M20	-4.58351 · 10^6	3615.50	16.16
5000	big-M5	-5.27126 · 10^6	3600.14	1.00
5000	ours	-5.27126 · 10^6	4.42	0.00
Table 72: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.3, λ2 = 0.1).
# of features	method	upper bound	time(s)	gap(%)
100	big-M50	-1.10084 · 107	0.44	0.00
100	big-M20	-1.10084 · 107	0.37	0.00
100	big-M5	-1.10084 · 107	0.33	0.00
100	ours	-1.10084 · 107	0.32	0.00
500	big-M50	-1.10515 · 107	86.52	0.00
500	big-M20	-1.10518 · 107	77.61	0.00
500	big-M5	-1.10518 · 107	51.24	0.00
500	ours	-1.10518 · 107	0.40	0.00
1000	big-M50	-1.09506 · 107	1539.25	0.00
1000	big-M20	-1.09508 · 107	1118.41	0.00
1000	big-M5	-1.09508 · 107	893.44	0.00
1000	ours	-1.09508 · 107	0.53	0.00
3000	big-M50	-	3608.09	-
3000	big-M20	-	3614.30	-
3000	big-M5	-	3610.50	-
3000	ours	-1.10012 · 107	1.67	0.00
5000	big-M50	-	3649.14	-
5000	big-M20	-1.01130 · 107	3600.71	9.83
5000	big-M5	-	3628.44	-
5000	ours	-1.09966 · 107	4.13	0.00
Table 73: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.5, λ2 = 0.1).
# of features	method	upper bound	time(s)	gap(%)
100	big-M50	-2.43573 · 10^7	0.41	0.00
100	big-M20	-2.43575 · 10^7	0.39	0.00
100	big-M5	-2.43575 · 10^7	0.30	0.00
100	ours	-2.43575 · 10^7	0.34	0.00
500	big-M50	-2.44167 · 10^7	96.59	0.00
500	big-M20	-2.44185 · 10^7	72.41	0.00
500	big-M5	-2.44185 · 10^7	54.87	0.00
500	ours	-2.44185 · 10^7	0.44	0.00
1000	big-M50	-2.41986 · 10^7	1608.58	0.00
1000	big-M20	-2.41988 · 10^7	984.16	0.00
1000	big-M5	-2.41989 · 10^7	639.70	0.00
1000	ours	-2.41989 · 10^7	0.53	0.00
3000	big-M50	-	3605.20	-
3000	big-M20	-	3617.57	-
3000	big-M5	-2.43713 · 10^7	3600.08	0.56
3000	ours	-2.43713 · 10^7	1.75	0.00
5000	big-M50	-2.43694 · 10^7	3650.52	1.00
5000	big-M20	-2.43694 · 10^7	3624.42	1.00
5000	big-M5	-	3650.91	-
5000	ours	-2.43694 · 10^7	4.22	0.00
Table 74: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.7, λ2 = 0.1).
# of features	method	upper bound	time(s)	gap(%)
100	big-M50	-9.11026 · 107	0.18	0.00
100	big-M20	-9.11026 · 107	0.11	0.00
100	big-M5	-9.11026 · 107	0.12	0.00
100	ours	-9.11026 · 107	0.31	0.00
500	big-M50	-9.12047 · 107	398.02	0.00
500	big-M20	-9.12047 · 107	113.34	0.00
500	big-M5	-9.12047 · 107	187.63	0.00
500	ours	-9.12047 · 107	0.38	0.00
1000	big-M50	-9.04075 · 107	3600.05	0.01
1000	big-M20	-9.04075 · 107	3600.24	0.20
1000	big-M5	-9.04075 · 107	2116.59	0.00
1000	ours	-9.04075 · 107	0.52	0.00
3000	big-M50	-	3619.63	-
3000	big-M20	-	3613.39	-
3000	big-M5	-	3609.80	-
3000	ours	-9.12761 · 107	20.88	0.00
5000	big-M50	-	3620.37	-
5000	big-M20	-	3603.08	-
5000	big-M5	-	3623.71	-
5000	ours	-9.12916 · 107	58.07	0.00
Table 75: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.9, λ2 = 0.1).
G.3.4 Synthetic 1 Benchmark with  $\lambda_{2} = 0.1$  with Warmstart
# of features	method	upper bound	time(s)	gap(%)
100	big-M50 + warm start	-2.10878 · 10^6	0.08	0.00
100	big-M20 + warm start	-2.10878 · 10^6	0.17	0.00
100	big-M5 + warm start	-2.10878 · 10^6	0.05	0.00
100	ours	-2.10878 · 10^6	0.36	0.00
500	big-M50 + warm start	-2.12833 · 10^6	3.78	0.00
500	big-M20 + warm start	-2.12833 · 10^6	2.75	0.00
500	big-M5 + warm start	-2.12833 · 10^6	3.10	0.00
500	ours	-2.12833 · 10^6	0.37	0.00
1000	big-M50 + warm start	-2.11005 · 10^6	26.94	0.00
1000	big-M20 + warm start	-2.11005 · 10^6	18.08	0.00
1000	big-M5 + warm start	-2.11005 · 10^6	16.21	0.00
1000	ours	-2.11005 · 10^6	0.57	0.00
3000	big-M50 + warm start	-2.10230 · 10^6	193.85	0.00
3000	big-M20 + warm start	-2.10230 · 10^6	255.55	0.00
3000	big-M5 + warm start	-2.10230 · 10^6	283.37	0.00
3000	ours	-2.10230 · 10^6	1.79	0.00
5000	big-M50 + warm start	-2.09635 · 10^6	1366.14	0.00
5000	big-M20 + warm start	-2.09635 · 10^6	1114.05	0.00
5000	big-M5 + warm start	-2.09635 · 10^6	1660.23	0.00
5000	ours	-2.09635 · 10^6	4.20	0.00
Table 76: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.1, λ2 = 0.1). All baselines use our beam search solution as a warm start.
# of features	method	upper bound	time(s)	gap(%)
100	big-M50 + warm start	-5.28726 · 10^6	0.07	0.00
100	big-M20 + warm start	-5.28726 · 10^6	0.07	0.00
100	big-M5 + warm start	-5.28726 · 10^6	0.05	0.00
100	ours	-5.28726 · 10^6	0.27	0.00
500	big-M50 + warm start	-5.31852 · 10^6	39.88	0.00
500	big-M20 + warm start	-5.31852 · 10^6	3.02	0.00
500	big-M5 + warm start	-5.31852 · 10^6	1.86	0.00
500	ours	-5.31852 · 10^6	0.41	0.00
1000	big-M50 + warm start	-5.26974 · 10^6	14.33	0.00
1000	big-M20 + warm start	-5.26974 · 10^6	14.92	0.00
1000	big-M5 + warm start	-5.26974 · 10^6	13.35	0.00
1000	ours	-5.26974 · 10^6	0.52	0.00
3000	big-M50 + warm start	-5.27668 · 10^6	317.41	0.00
3000	big-M20 + warm start	-5.27668 · 10^6	470.35	0.00
3000	big-M5 + warm start	-5.27668 · 10^6	249.02	0.00
3000	ours	-5.27668 · 10^6	1.84	0.00
5000	big-M50 + warm start	-5.27126 · 10^6	1658.42	0.00
5000	big-M20 + warm start	-5.27126 · 10^6	3607.98	1.00
5000	big-M5 + warm start	-5.27126 · 10^6	3600.14	1.00
5000	ours	-5.27126 · 10^6	4.42	0.00
Table 77: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.3, λ2 = 0.1). All baselines use our beam search solution as a warm start.
# of features	method	upper bound	time(s)	gap(%)
100	big-M50 + warm start	-1.10084 · 107	0.06	0.00
100	big-M20 + warm start	-1.10084 · 107	0.06	0.00
100	big-M5 + warm start	-1.10084 · 107	0.04	0.00
100	ours	-1.10084 · 107	0.32	0.00
500	big-M50 + warm start	-1.10518 · 107	2.28	0.00
500	big-M20 + warm start	-1.10518 · 107	6.61	0.00
500	big-M5 + warm start	-1.10518 · 107	13.48	0.00
500	ours	-1.10518 · 107	0.40	0.00
1000	big-M50 + warm start	-1.09508 · 107	14.11	0.00
1000	big-M20 + warm start	-1.09508 · 107	13.83	0.00
1000	big-M5 + warm start	-1.09508 · 107	12.46	0.00
1000	ours	-1.09508 · 107	0.53	0.00
3000	big-M50 + warm start	-1.10012 · 107	222.56	0.00
3000	big-M20 + warm start	-1.10012 · 107	290.65	0.00
3000	big-M5 + warm start	-1.10012 · 107	251.83	0.00
3000	ours	-1.10012 · 107	1.67	0.00
5000	big-M50 + warm start	-1.09966 · 107	3606.43	1.00
5000	big-M20 + warm start	-1.09966 · 107	3611.47	1.00
5000	big-M5 + warm start	-1.09966 · 107	3631.72	570161381.99
5000	ours	-1.09966 · 107	4.13	0.00
Table 78: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10,  $\rho = 0.5$ ,  $\lambda_2 = 0.1$ ). All baselines use our beam search solution as a warm start. The method big-M5 has some large optimality gaps because Gurobi couldn't finish solving the root relaxation within 1h.
# of features	method	upper bound	time(s)	gap(%)
100	big-M50 + warm start	-2.43575 · 107	0.06	0.00
100	big-M20 + warm start	-2.43575 · 107	0.24	0.00
100	big-M5 + warm start	-2.43575 · 107	0.04	0.00
100	ours	-2.43575 · 107	0.34	0.00
500	big-M50 + warm start	-2.44185 · 107	4.00	0.00
500	big-M20 + warm start	-2.44185 · 107	8.60	0.00
500	big-M5 + warm start	-2.44185 · 107	19.75	0.00
500	ours	-2.44185 · 107	0.44	0.00
1000	big-M50 + warm start	-2.41989 · 107	12.62	0.00
1000	big-M20 + warm start	-2.41989 · 107	12.12	0.00
1000	big-M5 + warm start	-2.41989 · 107	9.30	0.00
1000	ours	-2.41989 · 107	0.53	0.00
3000	big-M50 + warm start	-2.43713 · 107	3600.43	0.59
3000	big-M20 + warm start	-2.43713 · 107	3600.05	0.59
3000	big-M5 + warm start	-2.43713 · 107	3600.15	0.56
3000	ours	-2.43713 · 107	1.75	0.00
5000	big-M50 + warm start	-2.43694 · 107	3616.71	1.00
5000	big-M20 + warm start	-2.43694 · 107	3615.82	1.00
5000	big-M5 + warm start	-2.43694 · 107	3626.45	600317577.18
5000	ours	-2.43694 · 107	4.22	0.00
Table 79: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.7, λ2 = 0.1). All baselines use our beam search solution as a warm start. The method big-M5 has some large optimality gaps because Gurobi couldn't finish solving the root relaxation within 1h.