SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Output: a collection of solutions  $\mathcal{W} = \{\beta^t\}$  with  $\| \beta^t \|_0 = \| \beta' \|_0 + 1$  for  $\forall t$ . All of these solutions include the support of  $\beta'$  plus one more feature. None of the solutions have the same support as any element of  $\mathcal{F}$ , meaning we do not discover the same support set multiple times. We also output the updated  $\mathcal{F}$ .
1: Let  $S^c\gets \{j\mid \beta_j' = 0$  and  $j\notin \mathcal{N}_{\mathrm{avoid}}\}$  ▷Non-support of the given solution
2: Pick up to  $B$  coords  $(j$ 's) in  $\mathcal{S}^c$  with largest decreases in  $\Delta_j\mathcal{L}_{\mathrm{ridge}}(\cdot)$ , call this set  $\mathcal{J}'$ .  $\triangleright$ We will use these supports, which include the support of  $w$  plus one more.
3:  $\mathcal{W}\gets \emptyset$
4: for  $j\in \mathcal{J}'$  do >Optimize on the top  $B$  coordinates
5: if  $(\operatorname{supp}(\boldsymbol{w}) \cup \{j\}) \in \mathcal{F}$  then
6: continue  $\triangleright$  We've already seen this support, so skip.
7: end if
8:  $\mathcal{F}\gets \mathcal{F}\cup \{\operatorname {supp}(\boldsymbol {w})\cup \{j\} \} .$  Add new support to  $\mathcal{F}$
9:  $\pmb{u}^{*}\in \operatorname{argmin}_{\pmb{u}}\mathcal{L}_{\mathrm{ridge}}(\mathcal{D},\pmb{u})$  with  $\mathrm{supp}(\pmb {u}) = \mathrm{supp}(\pmb {w}^{\prime})\cup \{j\}$  . ▷Update coefficients on the newly expanded support
10:  $\mathcal{W}\gets \mathcal{W}\cup \{\pmb {u}^{*}\}$  cache the solution to the central pool
11: end for
12: return  $\mathcal{W}$  and  $\mathcal{F}$ .
E.6 Branching
Algorithm 8 Node.getBranchCoord  $(\beta)\rightarrow j^{*}$
Attributes in the Node class: dataset  $\mathcal{D} = \{(\pmb{x}_i, y_i)\}_{i=1}^n$ , sparsity constraint  $k$ ,  $\ell_2$  coefficient  $\lambda_2$ , a set of feature coordinates that must be selected  $\mathcal{N}_{\mathrm{select}}$ , and a set of feature coordinates that must be avoided  $\mathcal{N}_{\mathrm{avoid}}$ .
Output: a single coordinate  $j$  on which the node should branch in the BnB tree.
1:  $j^{*} = \operatorname{argmax}_{j \notin \mathcal{N}_{\mathrm{select}}} -2\beta^{T}(\boldsymbol{X}^{T}\boldsymbol{X} + \lambda_{2}\boldsymbol{I})\beta_{j}\boldsymbol{e}_{j} + ((\boldsymbol{X}^{T}\boldsymbol{X})_{jj} + \lambda_{2})\beta_{j}^{2} + 2\boldsymbol{y}^{T}\boldsymbol{X}\beta_{j}\boldsymbol{e}_{j} \triangleright \text{select the coordinate whose coefficient, if we set to 0, would result in the greatest increase in our ridge regression loss}$
2: return  $j^{*}$
E.7 A Note on the Connection with the Interlacing Eigenvalue Property
During the branch-and-bound procedure, finding the smallest eigenvalue of any principle submatrix of  $\mathbf{X}^T\mathbf{X}$  is computationally expensive if we do it for every node. Fortunately, we can get a lower bound for the smallest eigenvalue based on the following Interlacing Eigenvalue Property [36]:
Theorem E.1. Suppose  $\mathbf{A} \in \mathbb{R}^{n \times n}$  is symmetric. Let  $\mathbf{B} \in \mathbb{R}^{m \times m}$  with  $m < n$  be a principal submatrix (obtained by deleting both  $i$ -th row and  $i$ -th column for some values of  $i$ ). Suppose  $\mathbf{A}$  has eigenvalues  $\gamma_1 \leq \ldots \leq \gamma_n$  and  $\mathbf{B}$  has eigenvalues  $\alpha_1 \leq \ldots \leq \alpha_m$ . Then
$$
\gamma_ {k} \leq \alpha_ {k} \leq \gamma_ {k + n - m} \quad f o r k = 1, \dots , m
$$
And if  $m = n - 1$
$$
\gamma_ {1} \leq \alpha_ {1} \leq \gamma_ {2} \leq \alpha_ {2} \leq \dots \leq \alpha_ {n - 1} \leq \gamma_ {n}
$$
Using this Theorem, we can let  $\lambda = \lambda_{\min}(\mathbf{X}^T\mathbf{X})$  for every node in the branch-and-bound tree in our algorithm. This saves us a lot of computational time because we only have to calculate the smallest eigenvalue once in our root node. However, if the minimum eigenvalue were to be recalculated at each node efficiently, the lower bounds would be even tighter.
F More Experimental Setup Details
F.1 Computing Platform
All experiments were run on the 10x TensorEX TS2-673917-DPN Intel Xeon Gold 6226 Processor, 2.7Ghz. We set the memory limit to be 100GB.
F.2Baselines, Licenses, and Links
Commerical Solvers For the MIP formulations listed in Appendix D, we used Gurobi and MOSEK. We implemented the SOS1, bigM, perspective, and eigen-perspective formulations in Gurobi. The Gurobi version is 10.0, which can be installed through conda (https://anaconda.org/gurobi/gurobi). We used the Academic Site License. We implemented the perspective formulations and relaxed convex optimal perspective formulations in MOSEK. The MOSEK version is 10.0, which can be installed through conda (https://anaconda.org/MOSEK/mosek). We used the Personal Academic License.
PySINDy, STLSQ, SSR, E-STLSQ We used the PySINDy library (https://github.com/dynamicslab/pysindy) to perform the differential equation experiments. The license is MIT. The heuristic methods STLSQ, SSR, and E-STLSQ are implemented in PySINDy.
MIOSR For the differential equation experiments, the MIOSR code is available on GitHub(https://github.com/wesg52/sindy_mio_paper). The license for this code repository is MIT.
SubsetSelectionCIO The SubsetSelectionCIO code is available on GitHub (https: //github.com/jeanpauphilet/SubsetSelectionCIO.j1) with commit version 8b03d1fba9262b150d6c0f230c6bf9e8ee57f92e. The license for this code repository is MIT "Expat" License (https://github.com/jeanpauphilet/SubsetSelectionCIO.j1/blob/master/License.md). Due to package compatibility, we use Gurobi 9.0 for SubsetSelectionCIO. See the SusbetSelectionCIO package version specification at this link: https: //github.com/jeanpauphilet/SubsetSelectionCIO.j1/blob/master/Project.toml.
10bnb The 10bnb code is available on GitHub (https://github.com/alisaab/10bnb) with commit version 54375f8baeb64baf751e3d0effc86f8b05a386ce. The license for this code repository is MIT.
Sparse Simplex The Sparse Simplex code is available at the author's website (https://sites.google.com/site/amirbeck314/software?authuser=0). We ran the greedy sparse simplex algorithm. The license for this code repository is GNU General Public License 2.0.
Block Decomposition The block decomposition code is available at the author's website (https: //yuangzh.github.io). The author didn't specify a license for this code repository. However, we didn't modify the code and only used it for benchmarking purposes, which is allowed by the software community. Besides the block decomposition algorithm, the code repository also contains implementations for other algorithms, such as GP, SSP, OMP, proximal gradient l0c, QPM, ROMP. We used these implementations in the differential equation experiments as well. Please see [60] for details of these baselines.
F.3 Differential Equation Experiments
We compare the certifiable method MIOSR along with other baselines on solving dynamical system problems. In particular, for each dynamical system, we perform 20 trials with random initial conditions for 12 different trajectory lengths. Each dynamical system has observed positions every 0.002 seconds, so a larger trajectory length corresponds to a larger sample size  $n$ . Derivatives were approximated using a smoothing finite difference with a window size of 9 points. The derivatives were calculated automatically by the PySINDY library. Each trajectory has  $0.2\%$  noise added to it to demonstrate robustness to noise. For the SINDy approach, true dynamics exist in the finite-dimensional candidate functions/features. We investigate which method recovers the correct dynamics more effectively.
F.3.1 Model Selection
Model selection was performed via cross-validation with training data encompassing the first 2/3rds of a trajectory and the final third used for validation. Predicted derivatives were compared to the validation set, with the best model being the one that minimizes  $AIC_{c}$  for sparse regression in the setting of dynamical systems [45].
F.3.2 Hyperparameter Choices
With respect to each trajectory, each algorithm was trained with various combinations of hyperparameter choices. For both MIOSR and our method OKRidge, the ridge regression hyperparameter choices are  $\lambda_{2} \in \{10^{-5}, 10^{-3}, 10^{-2}, 0.05, 0.2\}$ , and the sparsity level hyperparameter choices are  $k \in \{1, 2, 3, 4, 5\}$ . As in the experiments of MIOSR [9], we set the time limit for each optimization to be 30 seconds for both MIOSR and our method OKRidge. For SSR and STLSQ, the ridge regression hyperparameter choices are  $\lambda_{2} \in \{10^{-5}, 10^{-3}, 10^{-2}, 0.05, 0.2\}$ . SSR and STLSQ may automatically pick a sparsity level, but they require a thresholding hyperparameter. We choose 50 thresholding hyperparameters with values between  $10^{-3}$  and  $10^{1.5}$ , depending on the system being fit. Together there are 250 different hyperparameter combinations we need to run on each trajectory length for our SSR and STLSQ. When fitting E-STLSQ, cross-validation is not performed as the chosen hyperparameters for STLSQ are used instead. For SSR, MIOSR, and OKRidge, we cross validate on each dimension of the data and combine the strongest results into a final model. For STLSQ and E-STLSQ, no post-processing is done and the same hyperparameters are used for each dimension.
F.3.3 Evaluation Metrics
Three metrics are considered to evaluate each algorithm: true positivity rate,  $L_{2}$  coefficient error, and root-mean-squared error (RMSE). The true positivity rate is defined as the ratio of the number of correctly chosen features divided by the sum of the number of true features and incorrectly chosen features. This metric equals 1 if and only if the correct form of the dynamical system is recovered.  $L_{2}$  coefficient error is defined as the sum of squared errors of the predicted and the true coefficients. Finally, for each model fit, we produce 10 simulations with different initial conditions of the true model, and calculate the root-mean-squared error (RMSE) of the predicted derivatives and the true derivatives. Besides these three metrics, we also report the running times, comparing OKRidge with other optimal ridge regression methods.
G More Experimental Results on Synthetic Benchmarks
G.1 Results Summarization
$\ell_2$  Perturbation Study We first compare with SOS1, Big-M (Big-M50 means that the big-M value is set to 50), Perspective, and Eigen-Perspective formulations with different  $\ell_2$  regularizations, with and without using beam search as the warm start. We have three levels of  $\ell_2$  regularizations: 0.001, 0.1, 10.0. The results are shown in Tables 1-60 in Appendix G.2. When the method has “-” in the upper bound or gap column, it means that method does not produce any nontrivial solution within the time limit.
Extensive results have shown that OKRidge outperforms all existing MIP formulations solved by the state-of-the-art MIP solver Gurobi. With the beam search solutions (usually they are already the optimal solutions) as the warm starts, Gurobi are faster than without using warm starts. However, OKRidge still outperforms all existing MIP formulations. If Gurobi can solve the problem within the time limit, OKRidge is usually usually orders of magnitude faster than all baselines. When Gurobi cannot solve the problem within the time limit, OKRidge can either solve the problem to optimality or achieve a much smaller optimality gap than all baselines, especially in the presence of high feature correlation and/or high feature dimension.
Big-M Perturbation Study We also conduct perturbation study on the big-M values. We pick three values: 50, 20, and 5. In practice, it is difficult to select the right value because setting the big-M value too small will accidentally cut off the optimal solutions. Nonetheless, we still pick a small big-M value (5; the optimal coefficients are all 1's in the synthetic benchmarks) to do a thorough comparison. The results are shown in Tables 66-120 in Appendix G.3. When the method has “-” in the upper bound or gap column, it means that method does not produce any nontrivial solution within the time limit.
The results show that although choosing a small big-M value can often lead to smaller running time, OKRidge is still orders of magnitude faster than the big-M formulation, and achieves the smallest optimality gap when all algorithms reach the time limit. Additionally, we find that having a small big-M value will sometimes lead to increased running time on solving the root relaxation. Sometimes, when the root relaxation is not finished within the 1h time limit, Gurobi will provide a loose lower bound, resulting in large optimality gaps. Aside from accidentally cutting off the optimal solution, this is another reason against choosing a very small big-M value.
Comparison with MOSEK, SubsetSelectionCIO, and l0bnb For the purpose of thorough comparison, we also solve the problem with other MIP solvers/formulations. The results are shown in Tables 121-Table 135 in Appendix G.4. When the method has “-” in the upper bound or gap column, it means that method does not produce any nontrivial solution within the time limit. For the MOSEK solver, when the upper bound and gap columns have “-” values, and the time is exactly 3600.00, this means that MOSEK runs out of memory (100GB).
First, we try to solve the problem using the commercial conic solver called MOSEK. We use MOSEK to solve the perspective formulation, with and without using beam search solutions as warm starts. The results indicate that Gurobi is in general much faster and more memory-efficient than MOSEK in solving the perspective formulation. We also use MOSEK to solve the relaxed convex optimal perspective formulation (relaxing the binary constraint  $z_{j} \in \{0,1\}$  to the interval constraint  $z_{j} \in [0,1]$ ). We only solve the relaxed convex SDP problem because MOSEK does not support MI-SDP. The results show that MOSEK has limited capacity in solving the perspective formulation. MOSEK also runs out of memory for large-scale problems ( $n = 100000$ ,  $p = 3000$  or  $p = 5000$ ). Besides, the experimental results show that convex SDP problem is not scalable to high dimensions. For the high-dimensional cases, OKRidge solving the MIP problem is usually much faster than MOSEK solving a convex SDP problem.
Second, we uses the SubsetSelectionCIO method, which solves the original sparse regression problem using branch-and-cut (lazy-constraint callbacks) in Gurobi. However, as the results indicate, SubsetSelectionCIO produces wrong optimal solutions when  $\ell_2$  regularization is small and the number of samples is large ( $n = 100000$ ). When SubsetSelectionCIO does not have errors, the running times or optimality gaps are much worse than those of OKRidge.
Lastly, we solve the sparse ridge regression problem using 10bnb. In contrast to other methods, 10bnb controls the sparsity level implicitly through the  $\ell_0$  regularization, instead of the  $\ell_0$  constraint. 10bnb needs to try several different  $\ell_0$  regularizations to find the desired sparsity level. For each  $\ell_0$  regularization coefficient, we set the time limit to be 180 seconds, and we stop the 10bnb algorithm after a time limit of 7200 seconds (this is longer than 3600 seconds because we want to give enough time for 10bnb to try different  $\ell_0$  coefficients). Moreover, 10bnb imposes a box constraint (similar to the big-M method) on all coefficients. The box constraint tightens the lower bound and potentially accelerates the pruning process, but it could also cut off the optimal solution if the "M" value is not chosen properly or large enough. To our best knowledge, there is no reliable way to calculate the "M" value. The 10bnb package internally estimates the "M" value in a heuristic way. Despite the box constraint giving 10bnb an extra advantage, we still compare this baseline with OKRidge. As the results indicate, 10bnb often takes a long time to find the desired sparsity level by trying different  $\ell_0$  regularizations. In some cases, 10bnb finds a solution with support size different than the required sparsity level (demonstrating the difficulty of finding a good  $\ell_0$  coefficient). In other cases, 10bnb still has an optimality gap after reaching the time limit while OKRidge can finish certification within seconds.
Discussion on the Case  $p \gg n$  For the application of discovering differential equations, the datasets usually have  $n \geq p$ . Therefore, the case  $p \gg n$  is not the main focus of this paper. Nonetheless, we provide additional experiments to give a complete view of our algorithm.
We set the number of features  $p = 3000$  and the number of samples  $n = 100, 200, 500$ . The feature correlation is set to be  $\rho = 0.1$  since this is already a very challenging case. The results are shown in Tables 151-153 in Appendix G.5. The results show that OKRidge is competitive with other MIP methods. 10bnb is the only method that can solve the problem when  $n = 500$  within the time limit. However, this algorithm has difficulty in finding the desired sparsity level in the case of  $n = 100$  and  $n = 200$  even with a large time limit of 7200 seconds. In some cases, MOSEK gives the best optimality gap. However, when  $n \geq p$  as we have shown in the previous subsections, MOSEK is not scalable to high dimensions. Additionally, the results in differential equations show that MOSEK and 10bnb are not competitive in discovering differential equations (datasets have  $n \geq p$ ), which is the main focus of this paper.
In the future, we plan to extend OKRidge to develop a more scalable algorithm for the case  $p \gg n$ .
Fast Solve vs. ADMM In the main paper, we proposed two ways to calculate the saddle point problem. The first approach, fast solve, is based on an analytical formula. The second approach is based on the ADMM method. Intuitively, since the fast solve method is only maximizing a lower bound of the saddle point objective and ADMM is maximizing the saddle point objective directly, we expect that ADMM would give much tighter lower bounds than the fast solve approach does. Here, we give empirical evidence to support our claim. The results can be seen in Figures 7 and 8.
G.2 Comparison with SOS1, Big-M, Perspective, and Eigen-Perspective Formulations with different  $\ell_2$  Regularizations
G.2.1 Synthetic 1 Benchmark with  $\lambda_{2} = 0.001$  with No Warmstart
# of features	method	upper bound	time(s)	gap(%)
100	SOS1	-2.10878 · 106	1.68	0.00
100	big-M50	-2.10874 · 106	0.39	0.00
100	persp.	-2.10878 · 106	0.08	0.00
100	eig. persp.	-2.10878 · 106	0.01	0.00
100	ours	-2.10878 · 106	0.29	0.00
500	SOS1	-2.12833 · 106	8.04	0.00
500	big-M50	-2.12828 · 106	36.80	0.00
500	persp.	-2.12833 · 106	20.29	0.00
500	eig. persp.	-2.12832 · 106	0.69	0.00
500	ours	-2.12833 · 106	0.37	0.00
1000	SOS1	-2.11005 · 106	82.53	0.00
1000	big-M50	-2.11005 · 106	1911.41	0.00
1000	persp.	-2.11005 · 106	1143.11	0.00
1000	eig. persp.	-2.11005 · 106	4.11	0.00
1000	ours	-2.11005 · 106	0.58	0.00
3000	SOS1	-2.10230 · 106	3600.15	0.59
3000	big-M50	-7.83869 · 105	3601.20	169.78
3000	persp.	-2.10231 · 106	2288.93	0.00
3000	eig. persp.	-2.10230 · 106	102.18	0.00
3000	ours	-2.10230 · 106	1.73	0.00
5000	SOS1	-1.98939 · 106	3600.13	6.43
5000	big-M50	-1.87965 · 106	3603.25	12.65
5000	persp.	-2.09637 · 106	3600.62	1.00
5000	eig. persp.	-	3880.65	-
5000	ours	-2.09635 · 106	4.22	0.00
Table 1: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.1, λ2 = 0.001).
# of features	method	upper bound	time(s)	gap(%)
100	SOS1	-5.28726 · 10^6	3.37	0.00
100	big-M50	-5.28725 · 10^6	0.42	0.00
100	persp.	-5.28726 · 10^6	0.07	0.00
100	eig. persp.	-5.28726 · 10^6	0.02	0.00
100	ours	-5.28726 · 10^6	0.29	0.00
500	SOS1	-5.31852 · 10^6	9.97	0.00
500	big-M50	-5.31819 · 10^6	89.04	0.00
500	persp.	-5.31852 · 10^6	8.44	0.00
500	eig. persp.	-5.31852 · 10^6	0.99	0.00
500	ours	-5.31852 · 10^6	0.41	0.00
1000	SOS1	-5.26974 · 10^6	88.42	0.00
1000	big-M50	-5.26974 · 10^6	1528.98	0.00
1000	persp.	-5.26974 · 10^6	54.96	0.00
1000	eig. persp.	-5.26974 · 10^6	300.72	0.00
1000	ours	-5.26974 · 10^6	0.52	0.00
3000	SOS1	-5.16793 · 10^6	3600.38	2.71
3000	big-M50	-3.72978 · 10^6	3602.55	42.31
3000	persp.	-5.27668 · 10^6	3600.33	0.59
3000	eig. persp.	-5.27668 · 10^6	100.58	0.00
3000	ours	-5.27668 · 10^6	1.75	0.00
5000	SOS1	-5.16088 · 10^6	3600.14	3.16
5000	big-M50	-4.16126 · 10^6	3602.53	27.95
5000	persp.	-5.27126 · 10^6	3600.72	1.00
5000	eig. persp.	-	3890.12	-
5000	ours	-5.27126 · 10^6	4.32	0.00
Table 2: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10,  $\rho = 0.3$ ,  $\lambda_2 = 0.001$ ).
# of features	method	upper bound	time(s)	gap(%)
100	SOS1	-1.10084 · 107	2.97	0.00
100	big-M50	-1.10084 · 107	0.41	0.00
100	persp.	-1.10084 · 107	0.09	0.00
100	eig. persp.	-1.10084 · 107	0.11	0.00
100	ours	-1.10084 · 107	0.29	0.00
500	SOS1	-1.10518 · 107	9.65	0.00
500	big-M50	-1.10515 · 107	89.59	0.00
500	persp.	-1.10518 · 107	3600.01	0.08
500	eig. persp.	-1.10518 · 107	25.13	0.00
500	ours	-1.10518 · 107	0.40	0.00
1000	SOS1	-1.09508 · 107	92.16	0.00
1000	big-M50	-1.09506 · 107	1757.09	0.00
1000	persp.	-1.09508 · 107	3600.04	0.20
1000	eig. persp.	-1.09508 · 107	226.82	0.00
1000	ours	-1.09508 · 107	0.59	0.00
3000	SOS1	-1.10012 · 107	3600.10	0.52
3000	big-M50	-	3615.51	-
3000	persp.	-1.10012 · 107	3600.91	0.59
3000	eig. persp.	-1.10012 · 107	100.85	0.00
3000	ours	-1.10012 · 107	1.74	0.00
5000	SOS1	-1.08838 · 107	3600.13	2.05
5000	big-M50	-	3612.62	-
5000	persp.	-1.09966 · 107	3600.17	1.00
5000	eig. persp.	-	3732.98	-
5000	ours	-1.09966 · 107	4.00	0.00
Table 3: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.5, λ2 = 0.001).
# of features	method	upper bound	time(s)	gap(%)
100	SOS1	-2.43575 · 107	2.83	0.00
100	big-M50	-2.43573 · 107	0.46	0.00
100	persp.	-2.43575 · 107	0.21	0.00
100	eig. persp.	-2.43575 · 107	0.02	0.00
100	ours	-2.43575 · 107	0.32	0.00
500	SOS1	-2.44185 · 107	10.25	0.00
500	big-M50	-2.44178 · 107	92.53	0.00
500	persp.	-2.44185 · 107	114.80	0.00
500	eig. persp.	-2.44185 · 107	26.49	0.00
500	ours	-2.44185 · 107	0.46	0.00
1000	SOS1	-2.41989 · 107	88.38	0.00
1000	big-M50	-2.41987 · 107	1361.05	0.00
1000	persp.	-2.41989 · 107	3600.08	0.20
1000	eig. persp.	-2.41989 · 107	232.96	0.00
1000	ours	-2.41989 · 107	0.52	0.00
3000	SOS1	-2.43713 · 107	3600.47	0.59
3000	big-M50	-	3640.12	-
3000	persp.	-2.43713 · 107	3600.70	0.59
3000	eig. persp.	-	3606.15	-
3000	ours	-2.43713 · 107	1.77	0.00
5000	SOS1	-2.42551 · 107	3600.13	1.48
5000	big-M50	-2.43694 · 107	3643.28	1.00
5000	persp.	-2.43694 · 107	3604.07	1.00
5000	eig. persp.	-	3711.01	-
5000	ours	-2.43694 · 107	4.15	0.00
Table 4: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.7, λ2 = 0.001).
# of features	method	upper bound	time(s)	gap(%)
100	SOS1	-9.11026 · 107	0.05	0.00
100	big-M50	-9.11026 · 107	0.13	0.00
100	persp.	-9.11026 · 107	0.27	0.00
100	eig. persp.	-9.11019 · 107	1.63	0.00
100	ours	-9.11026 · 107	0.30	0.00
500	SOS1	-9.12047 · 107	21.43	0.00
500	big-M50	-9.12047 · 107	94.03	0.00
500	persp.	-9.12047 · 107	176.04	0.00
500	eig. persp.	-9.12047 · 107	85.92	0.00
500	ours	-9.12047 · 107	0.38	0.00
1000	SOS1	-9.04075 · 107	1085.91	0.00
1000	big-M50	-9.04075 · 107	3600.01	0.07
1000	persp.	-9.04075 · 107	3600.01	0.20
1000	eig. persp.	-9.04075 · 107	341.28	0.00
1000	ours	-9.04075 · 107	0.52	0.00
3000	SOS1	-9.09247 · 107	3600.05	0.98
3000	big-M50	-	3605.36	-
3000	persp.	-9.12761 · 107	3600.05	0.59
3000	eig. persp.	-	3600.41	-
3000	ours	-9.12761 · 107	20.74	0.00
5000	SOS1	-9.00628 · 107	3600.16	2.38
5000	big-M50	-	3607.68	-
5000	persp.	-9.12274 · 107	3600.13	1.07
5000	eig. persp.	-	3810.32	-
5000	ours	-9.12916 · 107	58.29	0.00
Table 5: Comparison of time and optimality gap on the synthetic datasets (n = 100000, k = 10, ρ = 0.9, λ2 = 0.001).
G.2.2 Synthetic 1 Benchmark with  $\lambda_{2} = 0.001$  with Warmstart