SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For any iteration number  $N + k$  with  $k > 0$ , we have that  $(w_{N + k}, W_{N + k})$  is feasible for  $\langle Z_{N + 1}, W \rangle + w \leq \beta_{N + 1}$ , and thus the Euclidean distance between  $(w_{N}, W_{N})$  and  $(w_{N + k}, W_{N + k})$  is at least  $\epsilon_{N} / (1 + 4 \| W_{N} \|_{F}^{2})$ . If the gap in the algorithm does not converge to 0, then there is an  $\epsilon > 0$  for which  $\epsilon_{N} \geq \epsilon$  for all  $N$  and thus the distance between any two points in the sequence  $\{(w_{N}, W_{N})\}_{N}$  is bounded from below by  $\epsilon / (1 + 4 \max \{ \| W \|_{F}^{2} \mid W \in (8) \})$ . However, since the infinite sequence of points  $\{(w_{N}, W_{N})\}_{N}$  belong to a bounded set defined by (8), there must be a convergent subsequence, which contradicts that there is a positive lower bound on the distance between any two points.
From Theorem 1 the gap between the upper and lower bound converges to zero, and thus Algorithm 1 converges to a globally optimal solution.
4.2 Considerations when solving the relaxed problem
Problem (7) minimizes a concave function over a convex set. Thus, the solution is located in the extreme points of the convex set, i.e., the outer approximation of  $\mathcal{F}$ . The standard approach to solve such problems is by branch and bound methods. However, the low dimension and sequential generation of constraints make it viable to search among the extreme points for an optimal solution.
To simplify notation, we define  $x^{T} = \left( \begin{array}{cc}w & \operatorname {vec}(W)^{T} \end{array} \right)\in \mathbb{R}^{r}$  where  $r\coloneqq \ell_x\ell_y + 1$ . Then we can write (7b) on the form  $A_{k}x\leq b_{k}$ . Note that, by construction, none of the constraints are strongly redundant as every constraint is satisfied with equality for a permutation. As the constraints are added sequentially, it is actually easy to compute the new extreme points by keeping track of previous extreme points as described in the following proposition.
Proposition 3. Assume that the extreme points  $\{x_{k}\}_{k}$  of the convex set described by  $Ax\leq b$  are known. When adding a constraint  $A_N^T x\leq b_N$ , the additional extreme points are linear combinations of pairs of existing extreme points  $x_{k_1}$  and  $x_{k_2}$  both satisfying the same  $r - 1$  constraints with equality and  $A_N^T x_{k_1}\leq b_n$  and  $A_N^T x_{k_2} > b_n$  so that the combination satisfies  $A_N^T (\lambda x_{k_1} + (1 - \lambda)x_{k_2}) = b_N$ .
Proof. This is done by counting the number of constraints satisfied with equality. See the appendix for the proof.
Especially, in lower dimensions, e.g., with two or three dimensional data, this approach of keeping track of all extreme points and calculating new extreme points after adding a constraint can be very efficient for solving problem (7). More details of this method is provided in the appendix. In the numerical results, we present results where problem (7) is solved both by this extreme point search and the spatial branch and bound method in Gurobi.
5.1 Computational efficiency
In this section we compare the time to solve the problem up to an accuracy measured in relative error with different methods: Algorithm 1 when (7) is solved with the extreme point method as described in section 4.2, Algorithm 1 when (7) solved using Branch & bound using Gurobi 10.0 [13], MILP1 formulation in [11] implemented in Gurobi and finally when (6) is directly solved using Gurobi. The MILP1 formulation can handle a larger class of problems, but is reported to handle very few dimensions. All computations were performed using Matlab on an Intel i5 2.9 GHz PC. The linear optimal mass problem (9) was solved using the package [3] which is based on the network simplex [18]. The model problems tested are evenly distributed points in a unit disc or ball which we denote
Table 1: Computational efficiency. Computational time on the format [mean (low - high)] from 5 repeats for various problem geometries, dimensions and sizes, for the proposed extreme point method, the same method using branch and bound (B&B), the problem formulation MILP1 [11] and finally (6) implemented in Gurobi via Matlab interface. An "-" indicates that the problem timed out, in such a way being incomparable with the proposed method. An "+" indicates that the problem reached  $10^{4}$  iterations and stopped to the accuracy indicated.
Type	n	\( {\ell }_{x},{\ell }_{y} \)	Rel. error	Algorithm 1 [s] Extreme point / B&B	MILP1 [s]	(6) B&B [s]
U	10	2,2	\( {10}^{-8} \)	0.14 (0.07-0.3) / 21 (6-47)	39 (11-58)	0.15 (0.14-0.16)
U	100	2,2	\( {10}^{-8} \)	0.48 (0.3-0.7) / 86 (52-107)	-	25 (19-39)
U	500	2,2	\( {10}^{-8} \)	11 (9-16) / 408 (269-511)	-	-
U	1000	2,2	\( {10}^{-8} \)	69 (54-85) / 576 (389-1059)	-	-
U	2000	2,2	\( {10}^{-8} \)	460 (313-653) / -	-	-
U	10	2,3	\( {10}^{-8} \)	1.8 (1.2-2.4) / 133 (45-296)	105 (49-147)	2.4(1.8-3.4)
U	100	2,3	\( {10}^{-8} \)	278 (99-813) / -	-	172 (133-221)
U	500	2,3	\( {10}^{-8} \)	9568 / -	-	-
\( {\mathcal{N}}_{1} \)	10	2,3	\( {10}^{-8} \)	0.51 (0.39-0.65) / 708 (233-1184)	146 (66-227)	3 (2.6-4.0)
\( {\mathcal{N}}_{1} \)	100	2,3	\( {10}^{-8} \)	86 (20-275) / -	-	95 (73-116)
\( {\mathcal{N}}_{1} \)	500	2,3	\( {10}^{-5} \)	5310!/-	-	-
\( {\mathcal{N}}_{2} \)	10	3,3	\( {10}^{-2} \)	1.8 (0.7-3.2) / 142 (73-210)	117 (71-163)	0.2(0.1-0.3)
\( {\mathcal{N}}_{2} \)	100	3,3	\( {10}^{-2} \)	36 (22-55)/ -	-	45(36-65)
\( {\mathcal{N}}_{2} \)	500	3,3	\( {10}^{-2} \)	436 (228-862) / -	-	-
\( {\mathcal{N}}_{3} \)	10	3,3	\( {10}^{-2} \)	1.2 (0.5-2.3) / 22 (11-43)	72 (43-94)	0.2(0.1-0.3)
\( {\mathcal{N}}_{3} \)	100	3,3	\( {10}^{-2} \)	7 (5-8)/ 91 (76-111)	-	10 (9-12)
\( {\mathcal{N}}_{3} \)	500	3,3	\( {10}^{-2} \)	11 (9-16) / 161 (104-226)	-	-
\( {\mathcal{N}}_{3} \)	1000	3,3	\( {10}^{-2} \)	25 (22-29) / 176 (149-224)	-	-
\( {\mathcal{N}}_{3} \)	2000	3,3	\( {10}^{-2} \)	93 (91-100) / 578 (429-691)	-	-
$\mathcal{U}$ , and normally distributed points  $\mathcal{N}(0,\sigma)$ . We denote  $\mathcal{N}_1\coloneqq \mathcal{N}(0,I),\mathcal{N}_2\coloneqq \mathcal{N}(0,\mathrm{diag}(1,1,\frac{1}{10}))$  and  $\mathcal{N}_3\coloneqq \mathcal{N}(0,\mathrm{diag}(1,\frac{1}{2},\frac{1}{10}))$ . See Table 1 for numerical results. Some notes on the results
1. On 2-dimensional data  $(\ell_x = \ell_y = 2)$ , the extreme point method is particularly efficient.
2. For problems that need many extreme points  $(>10^{6})$ , which depends on the data itself, the handling of extreme points becomes the driver of computational cost.
3. Problems mainly containing reflections (e.g.  $\mathcal{N}_3$ ) are easier to solve than those with room for rotations.
4. Directly solving (6) with Gurobi was not feasible for problems with  $n \geq 500$ .
5.2 Comparison with local search method
We compare the results using the proposed method to a local search method provided in the GitHub repository for [20]. The entropy regularization parameter is set to 0, and the method is run with random initializations (including the first lower bound [17] used with success in [22]) until the relative error to the global optimum is less than a specific tolerance  $\epsilon$ . The problems are the same as in the previous section. Note here that in the local search methods we need an oracle in order to determine when we have reached a given performance level (which of course is not available in practice), whereas we in the proposed method computes upper and lower bounds.
Results for two and three dimensional data  $(\ell_x = \ell_y = 2,3)$  are presented in Table 2. The results show that the proposed method performs better than multi-starting the local method when  $\ell_x = \ell_y = 2$ . For the matching of 2-dimensional data to 3-dimensional data, the local search method is surprisingly fast suggesting that the problems is of a completely different nature than when 2-d data is matched to 2-d data or 3-d data is matched to 3-d data.
Note that we have chosen to compare with the method from [20] rather than [22]. This is since we have not optimized the optimal transport computations using the low rank structures in the problem. If we optimized the computations in this way, i.e., as in [21], we expect to get similar improvement as in [22] compared with [20].
Table 2: Computational efficiency compared with local search [20]. Computational time on the format [mean (low - high)] on two specific problems which contain near symmetries and the number of random initializations needed to achieve the required accuracy on the format [mean (low - high)]. The number of initializations were limited to 1000.
Type	n	\( \ell_x, \ell_y \)	Rel. error \( \epsilon \)	Algorithm 1 Exec. time [s]	Exec. time [s]	Local search Initializations	Successful runs
U	100	2,2	\( 10^{-6} \)	0.5 (0.4-0.6)	4 (0.3-12.7)	64 (4-205)	5
U	200	2,2	\( 10^{-6} \)	1.6 (1.0-1.9)	27 (13-42)	129 (59-197)	5
U	300	2,2	\( 10^{-6} \)	3.2 (2.7-3.9)	174 (33-458)	366 (69-959)	5
U	400	2,2	\( 10^{-6} \)	6 (5-8)	322 (97-685)	321 (97-781)	3
U	100	2,3	\( 10^{-6} \)	352 (99-669)	23 (8-60)	341 (110-853)	5
U	200	2,3	\( 10^{-6} \)	1238 (643-2612)	92 (11-168)	421(49-778)	3
U	300	2,3	\( 10^{-6} \)	3006 (601-4908)	131 (4-344)	270 (10-710)	5
U	400	2,3	\( 10^{-6} \)	4729 (4279-4868)	367 (5-343)	365 (1-859)	5
\( \mathcal{N}_1 \)	100	2,2	\( 10^{-6} \)	0.5 (0.3-0.7)	0.6 (0.2-1.0)	10 (3-16)	5
\( \mathcal{N}_1 \)	200	2,2	\( 10^{-6} \)	0.9 (0.7-1.1)	13.4 (1-37)	61 (6-168)	5
\( \mathcal{N}_1 \)	300	2,2	\( 10^{-6} \)	2.4 (2.0-2.9)	37.5 (0.5-90)	75 (1-182)	5
\( \mathcal{N}_1 \)	400	2,2	\( 10^{-6} \)	4.4 (3.6-5.3)	149 (22-378)	142 (21-361)	4
5.3 Application to symmetrical data for morphological analysis
In this example we investigate the impact of correctly evaluating the Gromov-Wasserstein discrepancy compared to estimating it by local search. As a test case, we examine the ability to classify Adeno Associated Viral (AAV) particles based on the Gromov-Wasserstein discrepancy on image data originating from transmission electron microscopy, hence  $\ell_{x} = \ell_{y} = 2$ . AAV particles are nearly round viral particles with multiple near-rotational symmetries as illustrated in Figure 2. By sampling  $n = 500$  positions on each AAV particle proportional to the protein density, the point sets from pairs of particles  $X_{i}$  and  $X_{j}$  can subsequently be compared using the Gromov-Wasserstein discrepancy.
Computing the Gromov-Wasserstein discrepancy between all objects in a large set  $\mathcal{X} = \{X_i\}_i^N$  is tedious. Therefore, one may consider calculating the discrepancy to a subset of the objects that are well distributed under the Gromov-Wasserstein discrepancy. To find such a subset without actually calculating all pairs of discrepancies, we use a greedy approach by defining the index subset  $S_{k} = \{s_{i}\}_{i = 1}^{k}$  by selecting the first object arbitrarily and then let the set grow by
$$
S _ {k + 1} := \left\{S _ {k}, \underset {i \notin S _ {k}} {\operatorname {a r g m a x}} \min  _ {j \in S _ {k}} d _ {G W} \left(X _ {i}, X _ {j}\right) \right\}. \tag {11}
$$
In this way all objects are closer than a tolerance to an object in the subset, and the way the subset is produced generates a monotonically decreasing tolerance. By using this procedure, every object obtains a feature vector of distances to the objects indexed by  $S_{k}$ . Next the feature vectors are used as input to a K-means clustering and classification quality in terms of purity, adjusted rand index and normalized mutual information compared to an expert evaluation is presented in Figure 3.
The example shows that if the data contains symmetries, local search methods may get stuck on permutations that are locally optimal but that are far from globally optimal, see Figure 2. Thus the discrepancy obtained from local search is inappropriate to use as a discriminating feature, as shown in the lower performance in the classification illustrated in Figure 3. In the left subfigure of Figure 2, self-similarities are visited in the proposed method, and these local optima are in fact also almost optimal when used as initiation point using local search methods e.g., [20].
This example shows that when the Gromov-Wasserstein problem is calculated accurately it provides valuable information and biological meaning as it differentiates viral particles with different cargo and
variations in capsid structure, and, at the same time, finds the optimal orientation positively revealing possible chiralities or isomerisms. It also shows that when the distance is calculated accurately, it provides better decision support than using local search methods.
Figure 2: Left: Each distance calculated with the proposed method and the same distance calculated with the local search method [20]. For near symmetrical data, the confusion of the measurements of the local method is clear. Middle: The relative error of the local method compared to the global solution. Right: Matching of the structure to itself to four different orientations visited by the algorithm where the color indicates the permutation  $\Gamma$ . The relative error of the Gromov-Wasserstein-discrepancy and the isometry to the global optimum is written near each matching. The global optimum is able to correctly match to itself, whereas the local search method may get stuck in local optima.
Figure 3: The trajectory of the increased quality of classification compared to an expert evaluation when the distance is computed from all particles to the sequence of particles suggested in the text. The gain of quality using an exact evaluation (blue) of the Gromov-Wasserstein problem is unambiguous over the local search (red).
When using distances as input for statistical analyses, the accuracy of the measurement set a bound for the information resolution. If the measurement system introduces error of a certain structure, this can produce artefacts in the result and affect decisions taken on the result. When using distances for such purposes, it is necessary to either know the measurement error, the artefacts being produced, or using an accurate measurement system. In this paper we have provided a method which computes the Gromov-Wasserstein problem accurately, which reduces the uncertainty of such considerations.
This work was funded by Vironova AB and the Swedish innovation agency Vinnova through the innovation milieu GeneNova (2021-02640).
[21] Meyer Sctbon, Marco Cuturi, and Gabriel Peyre. Low-rank Sinkhorn factorization. In International Conference on Machine Learning, pages 9344-9354. PMLR, 2021.
[22] Meyer Sctbon, Gabriel Peyre, and Marco Cuturi. Linear-time Gromov-Wasserstein distances using low rank couplings and costs. In International Conference on Machine Learning, pages 19347-19365. PMLR, 2022.
[23] Justin Solomon, Gabriel Peyre, Vladimir G Kim, and Suvrit Sra. Entropic metric alignment for correspondence problems. ACM Transactions on Graphics (TOG), 35(4):1-13, 2016.
[24] Cole Stiegler. Efficient local optimization for low-rank large-scale instances of the quadratic assignment problem. The University of Iowa, 2018.
[25] Titouan Vayer. A contribution to optimal transport on incomparable spaces. arXiv preprint arXiv:2011.04447, 2020.
[26] C. Villani. Topics in Optimal Transportation, volume 58. Graduate studies in Mathematics, AMS, 2003.
[27] Hongteng Xu, Dixin Luo, and Lawrence Carin. Scalable Gromov-Wasserstein learning for graph partitioning and matching. Advances in neural information processing systems, 32:3052-3062, 2019.
[28] Hongteng Xu, Dixin Luo, Hongyuan Zha, and Lawrence Carin Duke. Gromov-Wasserstein learning for graph matching and node embedding. In International conference on machine learning, pages 6932-6941. PMLR, 2019.
[29] Yuguang Yan, Wen Li, Hanrui Wu, Huaqing Min, Mingkui Tan, and Qingyao Wu. Semi-supervised optimal transport for heterogeneous domain adaptation. In IJCAI, volume 7, pages 2969-2975, 2018.
Proposition 4. Let  $\Gamma$  be a doubly stochastic matrix and the matrices  $C_x$  and  $C_y$  given by
$$
C _ {x} = \left(\left\| x _ {i} - x _ {j} \right\| _ {2} ^ {2}\right) _ {i, j = 1} ^ {n} = \mathbf {1} m _ {x} ^ {T} - 2 X ^ {T} X + m _ {x} \mathbf {1} ^ {T},
$$
$$
C _ {y} = (\| y _ {i} - y _ {j} \| _ {2} ^ {2}) _ {i, j = 1} ^ {n} = \mathbf {1} m _ {y} ^ {T} - 2 Y ^ {T} Y + m _ {y} \mathbf {1} ^ {T},
$$
Proof. The proposition follows by the following straightforward computations, where we just expand the expressions and use  $\mathbf{1} = \Gamma \mathbf{1} = \Gamma^T\mathbf{1}$ , thus
$$
\begin{array}{l} \operatorname {t r} \left(C _ {x} \Gamma C _ {y} \Gamma^ {T}\right) = \operatorname {t r} \left(\left(\mathbf {1} m _ {x} ^ {T} - 2 X ^ {T} X + m _ {x} \mathbf {1} ^ {T}\right) \Gamma \left(\mathbf {1} m _ {y} ^ {T} - 2 Y ^ {T} Y + m _ {y} \mathbf {1} ^ {T}\right) \Gamma^ {T}\right) \\ = m _ {x} ^ {T} \Gamma (\mathbf {1} m _ {y} ^ {T} - 2 Y ^ {T} Y + m _ {y} \mathbf {1} ^ {T}) \Gamma^ {T} \mathbf {1} \\ - 2 \operatorname {t r} \left(X ^ {T} X \Gamma \left(\mathbf {1} m _ {y} ^ {T} - 2 Y ^ {T} Y + m _ {y} \mathbf {1} ^ {T}\right) \Gamma^ {T}\right) \\ + \mathbf {1} ^ {T} \Gamma (\mathbf {1} m _ {y} ^ {T} - 2 Y ^ {T} Y + m _ {y} \mathbf {1} ^ {T}) \Gamma^ {T} m _ {x} \\ = m _ {x} ^ {T} \mathbf {1} m _ {y} ^ {T} \mathbf {1} - 2 m _ {x} ^ {T} \Gamma Y ^ {T} Y \mathbf {1} + n m _ {x} ^ {T} \Gamma m _ {y} \\ - 2 m _ {y} ^ {T} \Gamma^ {T} X ^ {T} X {\bf 1} + 4 \mathrm {t r} (X ^ {T} X \Gamma Y ^ {T} Y \Gamma^ {T}) - 2 {\bf 1} ^ {T} X ^ {T} X \Gamma m _ {y} \\ + n m _ {y} ^ {T} \Gamma^ {T} m _ {x} - 2 \mathbf {1} ^ {T} Y ^ {T} Y \Gamma^ {T} m _ {x} + \mathbf {1} ^ {T} m _ {y} \mathbf {1} ^ {T} m _ {x} \\ = 4 \operatorname {t r} \left(X ^ {T} X \Gamma Y ^ {T} Y \Gamma^ {T}\right) - 4 m _ {x} ^ {T} \Gamma Y ^ {T} Y \mathbf {1} + 2 n m _ {x} ^ {T} \Gamma m _ {y} \\ - 4 \mathbf {1} ^ {T} X ^ {T} X \Gamma m _ {y} + 2 \mathbf {1} ^ {T} m _ {y} \mathbf {1} ^ {T} m _ {x} \\ = \left\langle 2 X \Gamma Y ^ {T}, 2 X \Gamma Y ^ {T} \right\rangle \\ + \langle 2 n m _ {x} m _ {y} ^ {T} - 4 m _ {x} \mathbf {1} ^ {T} Y ^ {T} Y - 4 X ^ {T} X \mathbf {1} m _ {y} ^ {T}, \Gamma \rangle \\ + 2 \mathbf {1} ^ {T} m _ {y} \mathbf {1} ^ {T} m _ {x}. \\ \end{array}
$$
A.2 Proof of Proposition 3
Proposition 6. Assume that the extreme points  $\{x_{k}\}_{k}$  of the convex set described by  $Ax\leq b$  are known. When adding a constraint  $A_N^T x\leq b_N$ , the additional extreme points are linear combinations of pairs of existing extreme points  $x_{k_1}$  and  $x_{k_2}$  both satisfying the same  $r - 1$  constraints with equality and  $A_N^T x_{k_1}\leq b_n$  and  $A_N^T x_{k_2}\geq b_n$  so that the combination satisfies  $A_N^T (\lambda x_{k_1} + (1 - \lambda)x_{k_2}) = b_N$ .
Proof. Let  $W$  be a matrix whose columns consist of the extreme points defined by the  $N - 1$  constraints  $Ax \leq b$ . Also, let  $a_N^T x \leq b_N$  be an additional constraint,  $e_k$  be a unit vector with 1 on position  $k$ , and let  $\alpha$  parametrize the convex cone on  $W$ , i.e.  $1^T \alpha = 1$ ,  $\alpha \geq 0$  so that  $AW \alpha \leq b$  describes all points in the convex set. Suppose that  $B_k$  describes the indices of the constraints that define the k:th extreme point by letting  $A_{B_k}$  be the sub matrix of  $A$  including the rows denoted by the indices in  $B_k$ . Then  $A_{B_k} We_k = b_{B_k}$ . It then follows that  $AW(\lambda e_{k_1} + (1 - \lambda)e_{k_2})_j = b_j$  if and only if  $j \in B_{k_1}$  and  $j \in B_{k_2}$  and  $0 \leq \lambda \leq 1$ . The construction in the proposition,  $\lambda x_{k_1} + (1 - \lambda)x_{k_2}$  so that  $A_N^T (\lambda x_{k_1} + (1 - \lambda)x_{k_2}) = b_N$  then satisfies  $r$  constraints with equality and all other constraints with inequality, i.e. the point is an extreme point to the set. We also note that every multiple combination (three or more) of extreme points sharing  $r - 1$  constraints are not extreme points as they are linear combination of the pairs given by  $\alpha (\lambda_1 x_1 + (1 - \lambda_1) x_2) + (1 - \alpha)(\lambda_2 x_2 + (1 - \lambda_2) x_3)$ . Every linear combination of pairs of points sharing less than  $r - 1$  constraints will not satisfy  $r$  constraints, i.e. they are not extreme points.
A.3 A description of the implementation of the extreme point method
The handling of the extreme points in the paper is done by keeping track of the extreme points, their connection to the boundary constraints, and lookup tables for the adjacent extreme points, i.e., extreme points that satisfies the same  $r - 1$  constraints with equality, where  $r$  is the rank of the problem. Let the extreme points be described in the matrix  $E$  where each column describes an extreme point. Thus  $AE \leq b\mathbf{1}$ , if the constraints are described by the matrix  $A$  and vector  $b$  such that  $Ax \leq b$  is the constraint equations.
Let the adjacency be described in a (sparse) matrix with binary elements  $D$  where  $D_{i,j} = 1$  if extreme point  $i$  and  $j$  are adjacent. Let us also keep track of the constraints that are satisfied by an extreme point with equality. For this purpose let  $B$  be a matrix with binary elements in which the element  $B_{i,j} = 1$  if extreme point  $j$  satisfies constraint  $i$  with equality. Thus,  $D_{i,j} = 1$  if  $(B^T B)_{i,j} = r - 1$ . This is one of the computational drivers for the proposed extreme point method.
When a new constraint  $(A_{n},b_{n})$  is added, the new extreme points are generated by a linear combination of the infeasible extreme points  $\mathcal{I} := \{i:A_nE_i > b_n\}$  and their adjacent feasible extreme points  $f_{i} := \{j:D(i,j) = 1\}$  where  $i\in \mathcal{I}$ . Let  $\#$  indicate the number of elements of a finite set, then  $\sum_{i\in \mathcal{I}}\# (f_i)$  is the number of new extreme points. We place the new extreme points in the matrix  $E$  by adding them in the end as a matrix  $P$ . The new matrix containing the extreme points
$$
E _ {n} = (E P)
$$
The matrix keeping track of which extreme points satisfies which constraints with equality is extended with
$$
B _ {n} = \left( \begin{array}{c c} B & C \\ 0 & \mathbf {1} \end{array} \right)
$$
where  $C_{\cdot,k} = B_{\cdot,i} \odot B_{\cdot,(f_i)_k}$ , preferably implemented using bitwise operators. Here, the last row describes the newly added constraint and  $k$  a re-enumeration of the new extreme points.
The new adjacency matrix  $D_{n}$  can be concatenated with the old  $D$  and two additional matrices
$$
D _ {n} = \left( \begin{array}{c c} D & O \\ O ^ {T} & N \end{array} \right)
$$
where  $O_{i,j} = 1$  the old extreme point  $i$  is adjacent to the new extreme point  $j$ . This information is already available for us, since the new extreme points are adjacent to  $f_j$ . Finally,  $N_{i,j} = 1$  if  $(C^T C)_{i,j} = r - 2$ . This is by far the most computationally expensive operation in the proposed algorithm, which can be implemented with std::popcount in the standard c++ library. For 3-dimensional problems, around 120 bits needs to be compared between all new extreme points.
A.4 Additional tests
A.5 Convergence rate
The convergence proof in Proposition 3 does not include a rate of convergence. In Figure 4, we show the convergence trajectory for the problems  $\mathcal{U}$  and  $\mathcal{N}_1$  for 2-dimensional data. The tests show that the convergence rate is linear to its nature up to a number of iterations where the gap closes completely.
In comparison to the result of the proposed method we present the same convergence evaluation with the local search method in [20]. Figure5 shows the trajectory of convergence. The tests show that the rate of convergence is sublinear to its nature. The number of initializations needed to achieve a pre-determined accuracy increase with the number of points for the local search. Note here that in order to determine when to stop one needs to know the optimal value, which is not available for the local search method.
Figure 4: Top row: The trajectory of the relative error for the proposed method for the problem  $\mathcal{U}$  on 2-dimensional data. Bottom row: The trajectory of the relative error for the proposed method for the problem  $\mathcal{N}_1$  on 2-dimensional data. The solid blue line indicates the mean convergence rate for 20 runs and the dashed red line indicates 1 standard deviation from the mean.