SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \cos \angle (\mathbf {a} _ {\mathcal {G}} ^ {i j}, \mathbf {a} _ {\mathcal {G}} ^ {k l}) \\ = \frac {\phi_ {(c _ {1} ^ {\prime} , c _ {1} ^ {\prime \prime}) , (c _ {1} ^ {\prime} , c _ {2} ^ {\prime})} \cdot \phi_ {(c _ {2} ^ {\prime} , c _ {1} ^ {\prime}) , (c _ {2} ^ {\prime} , c _ {2} ^ {\prime \prime})}}{\sqrt {\sum_ {p : (i , j , p) \in T r p} \left(\phi_ {(i , j) , (i , p)} ^ {2} + \phi_ {(j , i) , (j , p)} ^ {2}\right)} \cdot \sqrt {\sum_ {q : (k , l , q) \in T r p} \left(\phi_ {(k , l) , (k , p)} ^ {2} + \phi_ {(l , k) , (l , q)} ^ {2}\right)}}, \\ \end{array}
$$
where  $(c_1', c_2') \in \mathcal{E}$  is the common edge between  $tr1$  and  $tr2$  such that  $c_1', c_2' \in \{i, j, k, l\}$  with  $c_1' \neq c_2'$ , and  $c_1'', c_2'' \in \{i, j, k, l\} \setminus \{c_1', c_2'\}$  with  $c_1'' \neq c_2''$  are the other two nodes in the triplet such that  $(c_1', c_2', c_1''), (c_1', c_2', c_2'') \in Trp$ .
It can be seen that for both parts in this case, if all the angles are sufficiently large, then  $\cos \angle (\mathbf{a}_{\mathcal{G}}^{ij},\mathbf{a}_{\mathcal{G}}^{kl})$  cannot be close to 1, and thus the two columns are not similar.
Case 3: Edges  $(i,j)$  and  $(k,l)$  belong to non-adjacent triplets  $tr1, tr2 \in Trp$ .
By the construction of the matrix using Eqn. 5, there would be no common non-zero elements in the two columns. So,
$$
\cos \angle (\mathbf {a} _ {\mathcal {G}} ^ {i j}, \mathbf {a} _ {\mathcal {G}} ^ {k l}) = 0.
$$
Thus, if all the angles in all the triangles formed by the triplets in  $\mathcal{G}$  are sufficiently large, then the angle matrix  $\mathbf{A}_{\mathcal{G}}$  is non-singular, and the columns are not similar implying that  $\mathbf{A}_{\mathcal{G}}$  is well-conditioned.
Proof. It is known that the union of parallel rigid components is parallel rigid if the components share at least one edge [2]. Each triplet forming a triangle in  $\mathcal{G}$  is parallel rigid by itself. Also, by construction, each connected component of the triplet network  $\mathcal{G}_T$  will have shared edges among triplets forming triangles, making every component parallel rigid independently. Thus, the maximal parallel rigid graph of  $\mathcal{G}$  can be determined by the edges in  $\mathcal{G}$  participating in the maximum connected component of  $\mathcal{G}_T$ .
9 Proposed Method for Dense Networks
In Sec. 5 of the main paper, we presented the algorithm to remove skewed triangles from a sparse network  $\mathcal{G}$ . Here, we consider the case when  $\mathcal{G}$  is dense (nearly all possible edges exist), making  $\mathbf{A}_{\mathcal{G}}$  dense. We provide the method for removing skewed triangles in the following steps.
Step 1: First, construct a matrix  $\mathbf{V}$  with columns as the directions in  $\mathcal{G}$ . The elements of  $\mathbf{V}^{\mathrm{T}}\mathbf{V}$  give the dot product between all possible directions. Then, construct the boundary matrix using Eqn. 6. The row and column indices of non-zero elements of the matrix  $\mathbf{B}_2\mathbf{B}_2^T$  give us the edge pairs participating in triplets which are the non-zero elements in the angle matrix  $\mathbf{A}_{\mathcal{G}}$ . So, the zero elements of the matrix  $\mathbf{B}_2\mathbf{B}_2^T$  can be used to mask the elements in  $\mathbf{V}^{\mathrm{T}}\mathbf{V}$ . The net matrix generated is  $\tilde{\mathbf{C}}_{\mathcal{G}}$ .
Step 2: We identify the correct signs for the elements in  $\tilde{\mathbf{C}}_{\mathcal{G}}$  to ensure that the dot products reflect the cosine of the angles of the triangles. We first create a matrix  $\mathbf{M}$  consisting of  $m^{ij,kl}$  (Eqn. 7) as the elements corresponding to the row for  $(i,j)^{th}$  edge and column for  $(k,l)^{th}$  edge. Then, the Hadamard product (element-wise multiplication) of  $\tilde{\mathbf{C}}_{\mathcal{G}}$  and  $\mathbf{M}$  gives  $\mathbf{C}_{\mathcal{G}}$ . Once the matrix  $\mathbf{C}_{\mathcal{G}}$  is formed, we take element-wise inverse cosine to get the angles as the entries and thus the angle matrix  $\mathbf{A}_{\mathcal{G}}$ .
Step 3: This step is the same as that provided in the main paper.
We summarize the method in Algo. 2.
10 Formulations for Solving Translation Averaging
In Sec. 6 of the main paper, we used two formulations, Revised LUD [41, 57] and BATA [57], for the experiments. Here, we provide the optimization problems of those methods for ready reference.
Algorithm 2: Removal of Skewed Triangles from Dense Networks
1 Compute the non-zero elements in angle matrix  $\mathbf{A}_{\mathcal{G}}$  using the boundary matrix  $\mathbf{B}_2$  (Eqn. 6).
2 Construct the matrix  $\mathbf{V}$ , whose columns consist of directions  $\mathbf{v}_{ij}$ ,  $(i,j) \in \mathcal{E}$ .
3 Get dot products between all possible directions using  $\mathbf{V}^T\mathbf{V}$
4 Use zero elements in  $\mathbf{B}_2^T\mathbf{B}_2$  to mask elements in  $\mathbf{V}^T\mathbf{V}$ , and denote it as  $\tilde{\mathbf{C}}_{\mathcal{G}}$ .
5 Get the signs of the dot product for non-zero elements of  $\tilde{\mathbf{C}}_{\mathcal{G}}$ , i.e.  $m^{ij,kl}$  using Eqn. 7.
6 Construct a matrix  $\mathbf{M}$  using  $m^{ij,kl}$ .
7 Perform element-wise multiplication of  $\tilde{\mathbf{C}}_{\mathcal{G}}$  with  $\mathbf{M}$  to get  $\mathbf{C}_{\mathcal{G}}$ .
8 Get the inverse cosine for every element in  $\mathbf{C}_{\mathcal{G}}$  to get  $\mathbf{A}_{\mathcal{G}}$ .
9 Extract the angles of the triplets from  $\mathbf{A}_{\mathcal{G}}$
10 Filter out the triplets with the minimum angle less than a threshold to get  $\tilde{\mathcal{G}}_F = (\tilde{\mathcal{V}}_F,\tilde{\mathcal{E}}_F)$
11 Construct the triplet network  $\mathcal{G}_T$  using the filtered network  $\tilde{\mathcal{G}}_F$ .
12 Get the largest connected component of  $\mathcal{G}_T$ .
13 Get the edges of  $\tilde{\mathcal{G}}_F$  contributing to the largest connected component of  $\mathcal{G}_T$  to get  $\mathcal{G}_F = (\mathcal{V}_F, \mathcal{E}_F)$ , which is connected and is parallel rigid.
Revised LUD [41, 57] (comprises relative displacements):
$$
\min  _ {\mathbf {T} _ {i, i \in \nu}, \lambda_ {i j, (i, j) \in \varepsilon}} \sum_ {(i, j) \in \varepsilon} \| \mathbf {T} _ {j} - \mathbf {T} _ {i} - \lambda_ {i j} \mathbf {v} _ {i j} \| _ {2} \tag {31}
$$
$$
\text {s . t .} \sum_ {i \in \mathcal {V}} \mathbf {T} _ {i} = \mathbf {0}, \sum_ {(i, j) \in \mathcal {E}} \left\langle \mathbf {T} _ {j} - \mathbf {T} _ {i}, \mathbf {v} _ {i j} \right\rangle = 1, \lambda_ {i j} \geq 0, \forall (i, j) \in \mathcal {E}
$$
BATA [57] (compares relative directions):
$$
\min  _ {\mathbf {T} _ {i, i \in \mathcal {V}}, \gamma_ {i j, (i, j) \in \mathcal {E}}} \sum_ {(i, j) \in \mathcal {E}} \rho \left(\left\| \left(\mathbf {T} _ {j} - \mathbf {T} _ {i}\right) \gamma_ {i j} - \mathbf {v} _ {i j} \right\| _ {2}\right) \tag {32}
$$
$$
\text {s . t .} \sum_ {i \in \mathcal {V}} \mathbf {T} _ {i} = \mathbf {0}, \sum_ {(i, j) \in \mathcal {E}} \left\langle \mathbf {T} _ {j} - \mathbf {T} _ {i}, \mathbf {v} _ {i j} \right\rangle = 1, \gamma_ {i j} \geq 0, \forall (i, j) \in \mathcal {E}
$$
The zero centroid and dot product constraints in Eqns. 31 and 32 fix the origin and the global scale ambiguities, respectively.  $\rho$  denotes the Cauchy loss function.  $\lambda_{ij}$  and  $\gamma_{ij}$  are non-negative variables that are ideally equal to baseline and inverse baseline for the edge  $(i,j)$ , respectively.
As mentioned in the main paper, our filtering scheme uses only input directions and does not favour any specific cost function used for translation averaging. In Sec. 6 of the main paper, we presented the results using BATA [57]. In the following subsection, we present the results using Revised LUD [41, 57] and observe a similar trend as seen for the BATA results.
11.1 Outlier-Free Data
In Sec. 6.2 of the main paper, we provided experimental results on outlier-free data. Here, in Table 5, we provide the details of the 1DSfM [55] datasets without outliers. It can be seen that only a small fraction of the nodes and edges are removed due to the removal of skewed triangles. Also, it can be seen that for many datasets, the condition number of the angle matrix (with matrix-2 norm) decreases. For remaining datasets, it either increases or is similar for both the unfiltered and filtered networks. As mentioned in Sec. 5 of the main paper, we perform non-aggressive pruning, due to which some
Table 5: Details of networks before and after removing skewed triangles from 1DSfM [55] datasets without outliers.  $\# {N}_{\text{rem }},\# {M}_{\text{rem }}$  : No. of nodes and edges removed,  ${\kappa }_{2}\left( {\mathbf{A}}_{\mathcal{G}}\right)$  : condition number of the angle matrix with matrix-2 norm,  ${t}_{filter}$  : time taken to remove skewed triangles.
Dataset	#Nodes	#Edges	#Nrem	#Mrem	κ2(AG)	tfilter (sec)
w/o filter	w/ filter	w/o filter	w/ filter	w/o filter	w/ filter
ALM	560	543	7351	7140	17	211	1.0e+12	1.0e+12	0.06
ELS	129	128	664	649	1	15	5.5e+05	2.4e+05	0.01
GMM	324	315	5231	5164	9	67	4.8e+07	2.5e+08	0.07
MDR	268	248	2545	2361	20	184	5.6e+07	2.6e+06	0.03
MND	445	439	10338	10175	6	163	7.8e+09	3.5e+09	0.25
NYC	337	317	3250	3065	20	185	9.9e+11	9.5e+11	0.02
ND	1199	1182	26414	25878	17	536	1.0e+12	1.0e+12	0.28
PDP	344	334	6196	5949	10	247	2.3e+08	3.3e+08	0.09
PIC	1844	1795	20880	20100	49	780	1.7e+11	1.2e+11	0.15
ROF	994	964	8696	8305	30	391	2.6e+10	2.6e+10	0.07
TOL	450	439	5017	4921	11	96	1.1e+09	5.6e+07	0.04
TFG	3962	3778	51450	49571	184	1879	1.1e+12	4.5e+09	0.52
USQ	403	382	3921	3682	21	239	3.8e+11	3.3e+11	0.04
VNC	724	705	12176	11901	19	275	2.3e+09	3.1e+09	0.17
YKM	385	372	4073	3892	13	181	3.1e+08	1.5e+09	0.04
Table 6: Absolute translation errors (in meters) on 1DSfM [55] datasets without outliers using Revised LUD [41, 57]. Removed Node Errors: Errors of removed nodes in the unfiltered network (meters),  $t_{RLUD}$ : time taken by Revised LUD.
Dataset	Mean-ATE	RMS-ATE	Removed Node Errors	tRLUD (sec)
w/o filter	w/ filter	w/o filter	w/ filter	Mean	RMS
ALM	2.5	2.4	4.0	3.5	9.3	10.9	6
ELS	1.4	1.5	1.8	2.0	1.5	1.5	1
GMM	5.8	5.3	9.2	8.7	20.6	24.0	4
MDR	8.8	7.9	18.1	16.2	26.9	35.9	2
MND	2.3	2.2	3.2	3.2	4.7	5.7	7
NYC	2.7	2.6	5.6	5.4	5.6	7.4	3
ND	3.5	3.5	5.7	5.5	6.7	8.8	28
PDP	2.6	2.4	3.9	3.5	6.3	8.1	5
PIC	2.9	2.8	6.2	6.1	6.6	8.2	26
ROF	16.5	17.3	29.6	30.5	43.4	59.4	7
TOL	11.1	10.9	23.3	23.2	42.5	60.1	4
TFG	7.9	7.2	17.1	16.3	21.8	27.3	63
USQ	5.2	5.0	7.0	6.8	10.7	13.0	3
VNC	7.6	7.5	11.8	10.8	13.8	18.9	11
YKM	8.2	8.3	21.4	21.1	25.9	44.6	3
skewed triangles are still present. We re-emphasize that non-aggressive pruning still ensures that the nodes are estimated reliably since the nodes are also a part of atleast one triplet, which belongs to the set of non-skewed triangles.
In Table 6, we provide the results obtained from Revised LUD. It can be seen that the absolute translation errors decrease for most of the datasets after the removal of skewed triangles. Moreover, errors of the removed nodes in the unfiltered network are high compared to the overall errors, indicating that the removed nodes are not estimated reliably in the unfiltered network. We also observe that the time taken to filter skewed triangles (from Table 5) is  $\sim 1\%$  of the time taken for translation averaging by Revised LUD (Table 6), which shows the efficiency of Algo. 1 of the main paper.
11.2 Real Data
In Sec. 6.3 of the main paper, we provided the details of the datasets and the translation averaging results using BATA [57]. In Table 7, we present the results using Revised LUD. It can be seen that similar to the trend in BATA results, the absolute translation estimates improve for most datasets. Also, the errors of the nodes removed due to filtering are high compared to the overall errors in the unfiltered network, indicating that the filtered nodes are not estimated reliably in the unfiltered network.
Table 7: Absolute translations errors (in meters) on 1DSfM [55] datasets using Revised LUD [41, 57]. Removed Node Errors: Errors of removed nodes in the unfiltered network (meters),  $t_{RLUD}$ : time taken by Revised LUD.
Dataset	Mean-ATE	RMS-ATE	Removed Node Errors	tRLUD (sec)
w/o filter	w/ filter	w/o filter	w/ filter	Mean	RMS
ALM	4.4	4.3	8.0	7.6	10.5	15.3	12
ELS	20.3	19.6	37.5	42.1	82.6	97.9	5
GMM	41.9	45.5	60.6	66.6	69.3	86.0	10
MDR	15.6	14.5	29.5	27.5	39.8	43.2	3
MND	3.9	3.8	5.8	5.8	12.7	13.4	12
NYC	4.5	4.2	8.2	7.3	18.9	24.9	5
ND	4.0	3.9	6.5	6.5	10.1	13.0	68
PDP	7.8	7.7	11.6	11.5	16.5	18.5	11
PIC	5.0	4.9	9.2	9.3	14.4	18.0	63
ROF	17.5	15.7	32.5	28.7	45.7	58.0	17
TOL	18.5	17.4	36.7	33.7	69.3	100.9	6
TFG	19.6	13.1	68.5	25.5	58.8	137.2	214
USQ	12.9	12.3	20.0	19.3	32.8	42.3	9
VNC	9.6	10.2	14.0	15.3	15.4	16.8	19
YKM	20.6	20.0	30.0	29.0	42.1	52.9	7
Table 8: Details of networks after applying 1DSfM outlier filter [55] without and with removal of skewed triangles (Algo. 1). #  $N_{rem}$ , #  $M_{rem}$ : No. of nodes and edges removed.
Dataset	#Nrem	#Mrem
Our filter →	w/o filter	w/ filter	w/o filter	w/ filter
ALM	19	36	269	416
ELS	2	13	368	427
GMM	4	76	1062	1470
MDR	1	32	136	326
MND	3	7	1242	1312
NYC	2	17	251	443
ND	0	6	3790	3832
PDP	3	32	504	827
PIC	13	101	5229	6443
ROF	17	75	1953	2429
TOL	1	21	339	449
USQ	130	166	1769	1991
VNC	2	2	2120	2120
YKM	313	389	3223	3872
Next, we apply 1DSfM outlier filter (Algo. 2, [55]) on the datasets to study the complementary benefits of removal of skewed triangles after applying an outlier filter. We reiterate that the removal of skewed triangles and outlier edges are two distinct aspects of the problem. In Table 8, we provide the network details after applying 1DSfM outlier filter without and with the removal of skewed triangles and compare their performance in Table 9. It can be seen that applying the 1DSfM outlier filter and then removing skewed triangles with our filter leads to better translation estimates. Moreover, our filter  $+1$  DSfM outlier filter removes the nodes which are poorly estimated than only 1DSfM outlier filter, as seen from removed node errors in the unfiltered network in Table 9. Since both outliers and skewed triangles are different issues, combining filters for both types improves accuracy.
In Sec. 5, we mentioned the improved speed using Algo. 1 compared to the brute force method. In Table 10, we compare the time taken by the brute force method to remove skewed triangles to that of our method in Algo. 1. In the brute force method, all three angles in every triangle are computed, and then the skewed triangles are identified and removed. We used 20 threads in parallel for the brute force method, while our method is based only on vectorized operations without any parallelization. It can be seen that Algo. 1 is significantly faster than the brute force method with  $\sim 100$  times faster for most datasets and  $\sim 1000$  times faster for large scale datasets. This shows the efficiency of Algo. 1.
Finally, we show the differences in reconstructions obtained without and with using our filter. In Fig. 4, the arch in the Notre Dame reconstruction is misplaced due to improper translation estimates, as seen in the lateral view. Using our filter removes such a misplaced arch. In Fig. 5, the wall in the
Table 9: Absolute translations errors (in meters) after applying 1DSfM [55] outlier filter without and with removal of skewed triangles (Algo. 1). Removed Node Errors: Errors of removed nodes in the unfiltered network.
Dataset	Mean ATE	RMS ATE	Removed Nodes Errors (Mean)	Removed Nodes Errors (RMS)
Our filter →	w/o filter	w/ filter	w/o filter	w/ filter	w/o filter	w/ filter	w/o filter	w/ filter
ALM	4.6	4.4	10.5	10.5	12.4	11.8	25.2	23.7
ELS	23.3	21.5	51.2	50.5	15.9	70.6	15.9	96.8
GMM	42.4	41.1	64.2	60.6	20.7	121.1	33.3	158.1
MDR	13.6	12.9	28.2	26.0	20.2	48.5	20.2	57.1
MND	4.5	4.3	10.0	9.8	9.5	20.9	15.3	26.5
NYC	6.6	5.2	15.4	12.1	9.4	36.8	9.4	46.4
ND	3.3	3.3	6.4	6.4	0	9.0	0	9.3
PDP	7.9	8.0	13.0	13.1	10.1	12.6	11.6	14.9
PIC	5.4	5.2	11.1	10.8	8.3	17.7	13.3	25.6
ROF	14.3	15.3	27.7	26.8	57.8	57.5	67.9	83.9
TOL	15.6	14.3	32.4	29.7	7.3	54.9	7.3	81.2
USQ	9.7	10.9	14.5	18.2	23.6	23.8	37.3	36.2
VNC	10.8	10.5	18.1	18.4	13.5	19.2	14.6	25.1
YKM	13.9	11.2	26.6	22.7	34.5	34.8	38.2	39.1
Table 10: Time taken (in sec) for removal of skewed triangles with brute force method and Algo. 1.
Dataset	Brute Force	Algo. 1
ALM	20.55	0.31
ELS	5.28	0.14
GMM	12.40	0.20
MDR	0.96	0.05
MND	55.18	0.67
NYC	2.16	0.08
ND	1047.90	2.66
PDP	16.07	0.32
PIC	118.46	0.55
ROF	17.87	0.25
TOL	4.27	0.14
TFG	1110.40	1.83
USQ	11.33	0.23
VNC	72.63	0.65
YKM	7.36	0.18
Piazza del Popolo reconstruction is misplaced, but such an effect is not seen after applying our filter. This reveals that removing skewed triangles helps get more accurate reconstructions.
(a) Without Algo. 1
(b) With Algo. 1
Figure 4: Lateral view of Notre Dame [55] reconstruction. Improper reconstruction of the arch (red ellipse) obtained due to incorrect estimation of camera translation, which is not present after filtering skewed triangles with our method (Algo. 1). Reference reconstruction shows the position of the arch.
(c) Reference reconstruction
(b) With Algo. 1
Figure 5: Plan view of Piazza del Popolo [55] reconstruction. A misplaced wall (red ellipse) obtained due to incorrect estimation of camera translations, which is not present after filtering skewed triangles with our method (Algo. 1).