SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- It should have a relation-agnostic encoder  $f$  producing mappings  $\mathbf{x} \mapsto \mathbf{z}$ .
- There must be a separate encoder  $\xi$  that takes relation-agnostic representations  $\mathbf{z}$  as input, and bridges the information gap  $I(\mathbf{x};\mathbf{r}|\mathbf{z})$ .
A.7 Semantic Consistency
A metric space  $(\mathcal{M},d)$  along with its embedding function  $\varphi :\mathbb{X}\to \mathcal{M}$  are called semantically consistent, if the following holds:
$$
I (\mathbf {x}; \mathbf {y} | \varphi (\mathbf {x})) \propto d (\varphi (\mathbf {x}), \mathbf {y})
$$
In other words,  $\mathcal{M}$  encodes the semantics of  $\mathbf{x}$  via its distance function  $d$ .
A.8 Semantic Emergence
Without loss of generality, let us assume that  $\mathbf{g}$  only contains semantic information. Semantic Emergence refers to the fact that if, for a particular view, its embedding  $\mathbf{z}_v$  encodes a large amount of semantic information, it must also have a large amount of information about the global-view (local-to-global correspondence in DiNo [9]). Intuitively, this is the case because all views  $\mathbf{v} \in \mathbb{V}$ , in some way, arise from  $\mathbf{g}$ , and local-to-global relationships encode semantic information. Quantitatively,
$$
d (\mathbf {z} _ {\mathbf {v}}; \mathbf {y}) \propto \frac {1}{I (\mathbf {v} ; \mathbf {g})}
$$
A.9 Proofs of Theorems
Theorem 1: For a relational representation  $\mathbf{r}$  that minimizes the information gap  $I(\mathbf{x};\mathbf{y}|\mathbb{Z}_{\mathbb{V}})$ , there exists a metric space  $(\mathcal{M},d)$  that defines a Semantic Relevance Graph such that:
$$
d (\mathbf {z} _ {i}, \mathbf {y}) > \delta \wedge d (\mathbf {z} _ {j}, \mathbf {y}) > \delta \Rightarrow
$$
$$
\exists \xi : \mathbb {R} ^ {n} \xrightarrow {\text {s e m}} \mathbb {R} ^ {n} \mid \mathbf {r} = \xi (\varphi (\mathbf {z} _ {i}), \varphi (\mathbf {z} _ {j})), d (\mathbf {r}, \mathbf {y}) \leq \delta
$$
where  $\mathcal{M} \in \mathbb{R}^n$ ,  $\xrightarrow{\mathrm{sem}}$  and  $d$  denote semantically consistent transformations and distance metrics respectively,  $\varphi: \mathbb{Z}_{\mathbb{V}} \xrightarrow{\mathrm{sem}} \mathcal{M}$ , and  $\delta$  is some finite, empirical bound on semantic distance.
Proof. By contradiction.
Let us assume that there exists no metric space  $(\mathcal{M},d)$  that induces a Semantic Relevance Graph on  $\mathbb{Z}_{\mathbb{V}}$  satisfying the given constraint. In other words, given  $\mathbb{Z}_{\mathbb{V}}$ , we get an upper-bound of  $\delta$  on the amount of label information  $\mathbf{y}$  that can be captured from  $\mathbf{x}$ , and there is no function  $\xi$  that can reduce the uncertainty about the label any further.
Since  $f$  and  $d$  are semantically consistent (Appendix A.7), in information theoretic terms,  $\mathbb{Z}_{\mathbb{V}}$  is a sufficient representation (Appendix A.3), i.e., both sides of the following are true:
$$
I (\mathbf {x}, \mathbf {y}) = I (\mathbb {Z} _ {\mathbb {V}}, \mathbf {y}) \propto \delta
$$
$$
\Longleftrightarrow I (\mathbf {x}; \mathbf {y} | \mathbb {Z} _ {\mathbb {V}}) \propto \sum_ {\mathbf {z} \in \mathbb {Z} _ {\mathbb {V}}} d (\mathbf {z}, \mathbf {y}) = \delta , \tag {2}
$$
where  $\delta \geq 0$  (Appendix A.1). However, since  $f$  is only a relation-agnostic encoder (as stated in the Preliminaries), it exhibits an information gap (Appendix A.2), i.e.,
$$
I (\mathbf {x}, \mathbf {y}) = I \left(\mathbb {Z} _ {\mathbb {V}}, \mathbf {y}\right) + I (\mathbf {x}; \mathbf {r} | \mathbb {Z} _ {\mathbb {V}}), \tag {3}
$$
where  $I(\mathbf{x}; \mathbf{r} | \mathbb{Z}_{\mathbb{V}}) > 0$ . This leads to a contradiction between Equation (2) and Equation (3), establishing the falsity of the initial assumption and proving Theorem 1.
Theorem 2: The graph  $\mathcal{G}^*$  underlying the output relational representation is a member of  $\mathbb{G}$  for which the mutual information with the label is maximal. Formally,
Let us assume the existence of a graph  $\mathcal{G}' \in \mathbb{G}$  such that  $I(\mathcal{G}', \mathbf{y}) > I(\mathcal{G}^*, \mathbf{y})$ . As shown in Theorem 1, since  $\mathcal{G}^*$  encodes both relation-agnostic (nodes) and relation-aware (edges) information, it is a sufficient representation (Appendix A.3) of  $\mathbf{x}$ . Hence, for  $\mathbf{z} \in \mathbb{Z}_{\mathbb{V}}$ ,
$$
I (\mathcal {G} ^ {*}; \mathbf {y}) = I (\mathbf {x}; \mathbf {r} | \mathbf {z}) + I (\mathbf {z}; \mathbf {y}) = I (\mathbf {x}; \mathbf {y})
$$
Thus, for a  $\mathcal{G}'$  to exist,
$$
I \left(\mathcal {G} ^ {\prime}; \mathbf {y}\right) > I (\mathbf {x}; \mathbf {y}) \tag {4}
$$
However, computing  $\mathcal{G}'$  involves the following Markov Chain:
$$
\mathbf {x} \rightarrow \mathbb {Z} _ {\mathbb {V}} \rightarrow \mathcal {G} ^ {\prime}
$$
Thus, from the Data Processing Inequality (Appendix A.1):
$$
I (\mathbf {x}; \mathbb {Z} _ {\mathbb {V}}) \geq I (\mathbb {Z} _ {\mathbb {V}}; \mathcal {G} ^ {\prime})
$$
Using the chain rule of mutual information (Appendix A.1):
$$
\begin{array}{l} \Rightarrow I (\mathbf {x}; \mathbf {y}) - I (\mathbf {x}; \mathbf {y} | \mathbb {Z} _ {\mathbb {V}}) \geq I (\mathbb {Z} _ {\mathbb {V}}; \mathcal {G} ^ {\prime}) \\ \Rightarrow I (\mathbf {x}; \mathbf {y}) - I (\mathbf {x}; \mathbf {y} | \mathbb {Z} _ {\mathbb {V}}) \geq I \left(\mathcal {G} ^ {\prime}; \mathbf {y}\right) - I \left(\mathcal {G} ^ {\prime}; \mathbf {y} | \mathbb {Z} _ {\mathbb {V}}\right) \\ \end{array}
$$
Now,  $I(\mathcal{G}'; \mathbf{y}|\mathbb{Z}_{\mathbb{V}}) = 0$ , since  $\mathbb{Z}_{\mathbb{V}} \to \mathcal{G}'$  ( $\mathcal{G}'$  is derived from  $\mathbb{Z}_{\mathbb{V}}$ ),
$$
\Rightarrow I (\mathbf {x}; \mathbf {y}) - I (\mathbf {x}; \mathbf {y} | \mathbb {Z} _ {\mathbb {V}}) \geq I (\mathcal {G} ^ {\prime}; \mathbf {y}) \Rightarrow I (\mathbf {x}; \mathbf {y}) \geq I (\mathcal {G} ^ {\prime}; \mathbf {y}) + I (\mathbf {x}; \mathbf {y} | \mathbb {Z} _ {\mathbb {V}})
$$
By the positivity property of mutual information [24],  $I(\mathbf{x};\mathbf{y}|\mathbb{Z}_{\mathbb{V}})\geq 0$  . Hence,
$$
I (\mathbf {x}; \mathbf {y}) \geq I (\mathcal {G} ^ {\prime}; \mathbf {y}),
$$
which is in contradiction to Equation (4). Hence the initial assumption must be false.
Theorem 3: A function  $\varphi (\mathbf{z})$  that reduces the uncertainty  $I(\mathbf{v}_i\mathbf{v}_j;\mathbf{g}|\varphi (\mathbf{z}_i)\varphi (\mathbf{z}_j))$  about the emergent relationships among the set of views can define a Semantic Relevance Graph by being transitivity recovering.
Proof. Without loss of generality, consider three transitively related views  $\mathbf{v}_1$ ,  $\mathbf{v}_2$ , and  $\mathbf{v}_3$  such that:
$$
d (\mathbf {z} _ {2}, \mathbf {y}) <   \delta \wedge d (\mathbf {z} _ {1}, \mathbf {y}) > \delta \wedge d (\mathbf {z} _ {3}, \mathbf {y}) > \delta
$$
In qualitative terms,  $\mathbf{v}_2$  plays a strong role in determining  $\mathbf{y}$  given  $\mathbb{Z}_{\mathbb{V}}$ , while the other views do not. From the property of Semantic Emergence (Appendix A.8),
$$
d (\mathbf {z} _ {2}, \mathbf {y}) <   \delta \Rightarrow I (\mathbf {v} _ {1} \mathbf {v} _ {2}; \mathbf {g}) > \gamma \wedge I (\mathbf {v} _ {2} \mathbf {v} _ {3}; \mathbf {g}) > \gamma
$$
However, since  $\mathbf{v}_1$ ,  $\mathbf{v}_2$ , and  $\mathbf{v}_3$  are transitively related,
$$
I (\mathbf {v} _ {1} \mathbf {v} _ {3}; \mathbf {g}) > \gamma
$$
Now,  $f$  being a relation agnostic encoder, it has no way to minimize the uncertainty in the emergence  $I(\mathbf{v}_1\mathbf{v}_3;\mathbf{g})$ . Hence, using the distributive property of  $f$  (Appendix A.5),
$$
I (\mathbf {v} _ {1} \mathbf {v} _ {3}; \mathbf {g}) > \gamma \Rightarrow I (\mathbf {z} _ {1} \mathbf {z} _ {3}; \mathbf {z} _ {\mathbf {g}}) > \gamma
$$
We know that  $I(\mathbf{x};\mathbf{r}) = I(\mathbf{v_1}\mathbf{v_3};\mathbf{g}) = I(\mathbf{z_1}\mathbf{z_3};\mathbf{g})$  (Appendix A.4).
Consider a function  $\varphi(\mathbf{z})$  that maps elements in  $\mathbb{Z}_{\mathbb{V}}$  to  $(\mathcal{M}, d)$  such that  $d(\varphi(\mathbf{z}_i), \varphi(\mathbf{z}_j)) \propto 1 / I(\mathbf{z}_i \mathbf{z}_j; \mathbf{g})$ .  $\varphi$  is thus transitivity recovering, as it allows for the identification of transitivity among  $\mathbf{v}_1$ ,  $\mathbf{v}_2$  and  $\mathbf{v}_3$  via the distance metric  $d$ . By allowing for such a distance computation,  $\varphi$  minimizes  $I(\mathbf{z}_1 \mathbf{z}_3; \mathbf{g} | \varphi(\mathbf{z}_1) \varphi(\mathbf{z}_3))$ , meaning that the knowledge obtained from  $\varphi$  reduces the uncertainty about the emergent relationships. This in turn also minimizes  $I(\mathbf{v}_1 \mathbf{v}_3; \mathbf{g} | \varphi(\mathbf{z}_1) \varphi(\mathbf{z}_3))$ , and consequently  $I(\mathbf{x}; \mathbf{r} | \varphi(\mathbf{z}_1) \varphi(\mathbf{z}_3))$ .
However,  $I(\mathbf{x};\mathbf{r}) = I(\mathbf{x};\mathbf{y}|\mathbf{z}_1\mathbf{z}_3)$  (Appendix A.4). Hence,  $\varphi$  also minimizes the information gap  $I(\mathbf{x};\mathbf{y}|\mathbf{z}_1\mathbf{z}_3\varphi (\mathbf{z}_1)\varphi (\mathbf{z}_3))$ , which implies that:
$$
\exists \xi | \mathbf {r} = \xi (\varphi (\mathbf {z} _ {1}), \varphi (\mathbf {z} _ {3})), d (\mathbf {r}, \mathbf {y}) <   \delta ,
$$
thus defining a Semantic Relevance Graph (Theorem 1).
A.9.1 Robustness
Definition 4 (Robustness). A classifier  $f_{\eta}$  is said to be robust (noise-tolerant) if it has the same classification accuracy as that of  $f$  under the distribution  $\mathcal{D}$  [26]. Formally,
$$
P _ {\mathcal {D}} [ f (x) ] = P _ {\mathcal {D}} [ f _ {\eta} (x _ {\eta}) ],
$$
where  $P_{\mathcal{D}}$  defines the probability distribution over  $\mathcal{D}$ , and  $x_{\eta}$  is the noisy version of the input  $x$ . In other words,  $f_{\eta}$  would base its prediction on the  $(1 - \eta)|\mathbb{V}|$  noise-free views only.
Lemma 1. Let  $\mathbb{V}^{\eta}$  be the set of noisy views in  $\mathbb{V}$ . Let  $\mathbb{V}^{*} = \mathbb{V} - \mathbb{V}^{\eta}$  be the set of noise free (and hence, transitively related - Theorem 3) views. Then, the cut-set  $\mathbb{C} = (\mathbb{V}^{*},\mathbb{V}^{\eta}) = \varnothing$  when mapped to the output space of  $\varphi :\mathbb{V}\to \mathbb{G}$ .
Proof.  $\varphi$  restricts  $\mathbb{G}$  to the Hausdorff space (Theorem 3, Inference and Learning objective). In the general case, since the Hausdorff distance defines a pseudometric [54], it satisfies the triangle inequality. In our specific case, it also defines a complete metric. This is because  $\varphi$  also satisfies the requirement of geometric relation agnositicity in the fine-grained scenario under the minimization of the cross-entropy loss with a categorical ground-truth [10]:
$$
\forall \mathbf {z} _ {l} \in \mathbb {Z} _ {\mathbb {L}}: n _ {\epsilon} (\mathbf {z} _ {l}) \cap n _ {\epsilon} (\mathbf {z} _ {g}) = \phi
$$
The above implies that the representation space of  $\varphi$  satisfies the separation axioms [57], and hence:
$$
h (x, y) > 0, \forall x \neq y,
$$
where  $h(\cdot, \cdot)$  is the Hausdorff edit distance [25, 64]. Therefore, for  $\varphi(\mathbf{z})$ , the Hausdorff distance defines a complete metric while satisfying the triangle inequality. Now, consider a set of views  $\{\mathbf{v}_i, \mathbf{v}_j, \mathbf{v}_k\}$ , where  $\mathbf{v}_i, \mathbf{v}_k \in \mathbb{V}^*$ , and  $\mathbf{v}_j \in \mathbb{V}^\eta$ . Suppose these nodes define a 3-clique (clique of size 3) in the instance graph. The completeness of the Hausdorff distance in  $\varphi(\mathbf{z})$ , as well as the satisfaction of the triangle inequality thus gives us:
$$
d (\varphi (\mathbf {v} _ {i}), \varphi (\mathbf {v} _ {k})) <   d (\varphi (\mathbf {v} _ {i}), \varphi (\mathbf {v} _ {j})) + d (\varphi (\mathbf {v} _ {j}), \varphi (\mathbf {v} _ {k}))
$$
Hence, for the aforementioned clique formed by  $\mathbf{v}_i,\mathbf{v}_j$  , and  $\mathbf{V}_k$  , the only edge that survives  $\varphi$  is  $(\mathbf{v}_i,\mathbf{v}_k)$  . In other words, upon convergence,  $\varphi$  would never produce a path between two transitively related views that involves a noisy view, rendering  $\mathbb{V}^*$  and  $\mathbb{V}^{\eta}$  disconnected. This completes the proof of the lemma.
Corollary 4.1. There is no path in  $\mathcal{G}^*$  connecting two views in  $\mathbb{V}^*$  that involves a noisy view from  $\mathbb{V}^\eta$ . Thus, the elements of  $\mathbb{V}^*$  and  $\mathbb{V}^\eta$  respectively form homogeneous subgraphs of transitive and noisy views in  $\mathcal{G}^*$  and  $\mathcal{G}_{\mathbf{p}}$ .
Definition 5 (Continents and Islands). A continent is a complete subgraph (clique) in  $\mathcal{G}^*$  exclusively composed of transitively related views. An island in  $\mathcal{G}^*$  is a subgraph that exclusively contains noisy views.
Lemma 2. Let  $\mathbb{E}_{12}$  be the set of edges connecting continents  $\mathbb{C}_1$  and  $\mathbb{C}_2$ . The elements of  $\mathbb{E}_{12}$  are mutually redundant, i.e.,
$$
H (e _ {j} | e _ {i}) = 0, \forall (e _ {i}, e _ {j}) \in \mathbb {E} _ {1 2}
$$
Proof. Consider a pair of vertices  $v_{i}, v_{j} \in \mathbb{C}_{1}$ , such that both are connected to  $\mathbb{C}_2$  via edges  $e_{i}$  and  $e_{j}$  respectively. However, since  $\mathbb{C}_1$  is a clique,  $v_{i}$  and  $v_{j}$  (and all other nodes in  $\mathbb{C}_1$ ) are also connected. Thus, the complete set of information that  $\mathbf{v}_{i}$  has access to (and can potentially encode via  $e_{i}$ ) is the transitivity of (local-to-global relationship emerging from) all the local views in  $\mathbb{C}_1$  given by:
$$
H (e _ {i}) = I (\mathbf {v _ {1}}, \mathbf {v _ {2}}, \dots , \mathbf {v} _ {| \mathrm {C} _ {1} |}; \mathbf {g})
$$
Let  $\gamma$  be a message passing function (like a GNN) that carries information across connected nodes. Thus, for  $\gamma$ , the following holds:
$$
e _ {j} \equiv v _ {j} \xrightarrow {\gamma} v _ {i} \xrightarrow {\gamma} e _ {i} \Longrightarrow I (e _ {j}; \mathbb {C} _ {2}) = I (e _ {i}; \mathbb {C} _ {2}) \Longrightarrow H (e _ {j} | e _ {i}) = 0,
$$
meaning that the knowledge of a single edge between  $\mathbb{C}_1$  and  $\mathbb{C}_2$  is sufficient to determine the connectivity of the two continents. No other edge in  $\mathbb{E}_{12}$  can thus provide any additional information, thereby concluding the proof.
Theorem 4: In the representation space of  $\varphi$ , the uncertainty in estimating the topology  $\tau^\eta$  of  $\mathbb{V}^\eta$  is exponentially greater than the uncertainty in estimating the topology  $\tau^*$  of  $\mathbb{V}^*$ , in terms of the error rate  $\epsilon$ .
Proof. It is known that the total number of possible topologies  $\tau$  for a graph with  $n$  nodes is given by [40]:
$$
\left| \tau \right| = 2 ^ {\left(n ^ {2} / 4\right)} \tag {5}
$$
In what follows, we show that it is possible to dramatically reduce this number to  $\mathcal{O}(n)$  for continents, but not for islands.
According to Lemma 1 and Corollary 4.1, the groups of transitive and noisy view nodes can be separated into disjoint sets with no edges between them. Let  $\mathbb{V}^* = \{\mathbf{v}_1,\mathbf{v}_2,\dots ,\mathbf{v}_k\}$  be a set of transitively related views (a continent) with topology  $\mathcal{T}$ . Let  $\mathbb{V}^\eta = \{\mathbf{v}_{k + 1},\mathbf{v}_{k + 2},\ldots ,\mathbf{v}_n\}$  be a set of noisy views (an island) with topology  $\mathcal{T}_{\eta}$ . Let  $\pi^{\tau}$  denote the probability mass of a topology  $\tau$  in  $\mathcal{D}$  (fraction of samples in  $\mathbb{X}$  with topology  $\tau$ ).
Let  $\tau^{*}$  be the ground truth topology of a class, determined exclusively by the continents (since noisy views should not determine the label). Let  $\hat{\tau}$  be the best approximation of  $\tau^{*}$  that can be achieved by observing  $M$  unique samples, such that:
$$
h \left[ \mathcal {G} \left(\tau^ {*}\right) - \mathcal {G} (\hat {\tau}) \right] \leq \epsilon ,
$$
where  $\mathcal{G}(\tau)$  is the graph with topology  $\tau$ . In other words,  $\epsilon$  is the upper-bound on the statistical error in the approximation of the ground truth topology that can be achieved with  $M$  unique samples.
Now, uniquely determining  $\mathcal{G}(\tau^{*})$  involves determining all the  $k$ -cliques involving the global-view (Transitivity Recovery from Theorem 3). However, the Turan's theorem [74] puts an upper-bound on the number of edges  $|\mathbb{E}|$  of a graph with clique number  $k$  and  $n$  nodes at:
$$
| \mathbb {E} | = (1 - \frac {1}{k}) \frac {n ^ {2}}{2},
$$
which gives a direct upper-bound on the sample complexity  $M$  for learning  $\hat{\tau}$ .
If there are  $|\mathbb{E}|$  edges representing transitive relationships among  $n$  views, and the maximum clique size for a class is  $k$ , the number of  $k$ -cliques to encode all possible clique memberships with this configuration would be given by:
$$
\log_ {k} e = \frac {\log_ {2} e}{\log_ {2} k} = \frac {\log_ {2} \left[ \left(1 - \frac {1}{k}\right) \frac {n ^ {2}}{2} \right]}{\log_ {2} k} = \frac {\log_ {2} \left[ \left(1 - \frac {1}{k}\right) n ^ {2} \right]}{\log_ {2} k} = \log_ {2} \left(1 - \frac {1}{k}\right) - \log_ {2} k + 2 \log_ {2} n = \mathcal {O} (\log_ {2} n) \tag {6}
$$
Since the maximum clique size  $k$  is a constant for a given class. The above number of samples would be required to gain the knowledge of all the cliques in the graph.
To analyze the connectivity across different continents in  $G^{*}$ , we can apply Lemma 2 to reduce all the continents in  $G^{*}$  into single nodes, connected by an edge if there is at least one node connecting the two continents in  $G^{*}$ . Using this observation, and Lemma 2, the number of topologies  $|\mathcal{R}|$  of such a graph can be given by:
$$
| \mathcal {R} | = 2 ^ {\left(\mathcal {O} \left(\log_ {2} n\right) / 4\right)} = \mathcal {O} (n) \tag {7}
$$
Thus with a clique number of  $k$ ,  $\mathcal{G}^*$  could have a maximum of  $\mathcal{O}(\log_2 n)$  continents (Equation (6)), connected with each other in  $\mathcal{O}(n)$  possible ways (Equation (7)). Since there is only 1 possibility for the topologies of each clique, the only variation would come from the cross-continent connectivity, which has  $\mathcal{O}(n)$  potential options. Using the Haussler's theorem [28], the number of samples  $M$  required to achieve a maximum error rate of  $\epsilon$  with probability  $(1 - \delta)$  for a continent is given by:
$$
M ^ {*} = \frac {\log_ {2} \mathcal {O} (n) + \log_ {2} \frac {1}{\delta}}{\epsilon \pi^ {\hat {\tau}}} = \frac {\log_ {2} \mathcal {O} (n) + \log_ {2} \frac {1}{\delta}}{\epsilon (1 / c)} \approx \frac {\log_ {2} \mathcal {O} (n) + \log_ {2} \frac {1}{\delta}}{\epsilon}, \tag {8}
$$
where  $c$  is the number of classes, and in practice  $1 / c \gg 0$ . Note that the above bound is tight since we restrict our hypothesis class to only having transitivity recovering decompositions (Theorem 3).
Thus, one would need a number of samples that is linear in the number of transitive views  $(n)$ , as well as the error rate  $(\epsilon)$ , and the certainty  $(\delta)$ , to determine  $\hat{\tau}$  for a class. It can also be seen that  $M^{*}$  is completely independent of the noise rate  $\eta$ .
However, for  $\mathbb{V}^{\eta}$ , since its constituent views are noisy, no prior assumptions can be made about its structure. Thus, the number of possible topologies for  $\mathbb{V}^{\eta}$  can be obtained by substituting  $n$  with  $\eta$  in Equation (5) as follows:
$$
| \tau^ {\eta} | = 2 ^ {(\eta^ {2} / 4)} = \mathcal {O} (2 ^ {\eta})
$$
Thus, based on the Haussler's theorem, the sample complexity for learning the topology of an island would be given by:
$$
M ^ {\eta} \geq \frac {\log_ {2} \mathcal {O} (2 ^ {\eta}) + \log_ {2} \frac {1}{\delta}}{\epsilon \pi^ {\tau^ {\eta}}} \approx \frac {\log_ {2} \mathcal {O} (2 ^ {\eta}) + \log_ {2} \frac {1}{\delta}}{\epsilon \cdot \epsilon} = \frac {\log_ {2} \mathcal {O} (2 ^ {\eta}) + \log_ {2} \frac {1}{\delta}}{\epsilon^ {2}}, \tag {9}
$$
since the probability of finding an instance with a specific noisy topology  $\pi^{\tau^\eta}$  is very small, and hence can be approximated with the error-rate  $(\epsilon)$ . Note that the small probability mass for a specific  $\tau^\eta$  comes from the worst-case assumption that the dataset would have all possible forms of noise, and hence, the probability for a specific kind would be very low. Also note that unlike Equation (8), the bound cannot be tightened as the premise of transitivity does not hold.
Thus, while the sample complexity for learning a transitive topology  $M^{*}$  is linear in the number of views and the error rate Equation (8), that for a noisy topology  $M^{\eta}$  is exponential in the number of views and quadratic in the error rate Equation (9). Hence, in the worst case, when  $n \approx \eta$  [26],  $M^{\eta} \gg M^{*}$ . This implies that it is significantly more difficult for  $\varphi$  to learn the topology  $\tau^{\eta}$  of an island than it is to learn the topology  $\hat{\tau}$  of a continent.
From Equation (9), the probability of  $\varphi$  making a correct prediction based on noisy views would be given by:
$$
\begin{array}{l} M _ {\eta} \geq \frac {\log_ {2} \mathcal {O} (2 ^ {\eta}) + \log_ {2} \frac {1}{\delta}}{\epsilon^ {2}} \Rightarrow \log_ {2} \frac {1}{\delta} \leq M _ {\eta} \epsilon^ {2} - \log_ {2} \mathcal {O} (2 ^ {\eta}) \\ \Longrightarrow \log_ {2} \frac {1}{\delta} \leq M _ {\eta} \epsilon^ {2} - \eta \\ \Longrightarrow \frac {1}{\eta} \log_ {2} \frac {1}{\delta} \leq \frac {M _ {\eta}}{\eta} \epsilon^ {2} - 1 \approx \epsilon^ {3} - 1 \\ \Rightarrow \log_ {2} \frac {1}{\delta} \leq \eta \epsilon^ {3} - \eta \\ \Rightarrow \frac {1}{\delta} \leq 2 ^ {\eta (\epsilon^ {3} - 1)} \\ \Rightarrow \delta_ {\eta} \geq \mathcal {O} \left(2 ^ {- \epsilon^ {3}}\right) \\ \end{array}
$$
where the noise rate  $\eta$  is a constant for a particular dataset, and  $M_{\eta} / \eta \approx \epsilon$ , since  $M_{\eta} \gg \eta$ , as  $M_{\eta} = \mathcal{O}(2^{\eta})$ . From Equation (8), the probability of  $\varphi$  making a correct prediction based on transitive views would be given by:
$$
\begin{array}{l} M _ {*} = \frac {\log_ {2} \mathcal {O} (n) + \log_ {2} \frac {1}{\delta}}{\epsilon} \Rightarrow \log_ {2} \frac {1}{\delta} = M _ {*} \epsilon - \log_ {2} \mathcal {O} (n) \approx M _ {*} \epsilon - \log_ {2} n \\ \Longrightarrow \log_ {2} \frac {1}{\delta} = (\theta \log_ {2} n) \epsilon - \log_ {2} n \\ \Longrightarrow \log_ {2} {\frac {1}{\delta}} = \log_ {2} n ^ {\theta \epsilon} - \log_ {2} n \\ \Longrightarrow \log_ {2} \frac {1}{\delta} = \log_ {2} \frac {n ^ {\theta \epsilon}}{n} = \log_ {2} n ^ {\theta \epsilon - 1} \\ \Longrightarrow \frac {1}{\delta} = n ^ {\theta \epsilon - 1} - 2 ^ {\Gamma \epsilon - 1} \\ \Longrightarrow \delta_ {*} = \mathcal {O} (2 ^ {- \epsilon}), \\ \end{array}
$$
where  $\theta = M_{*} / \log_2n$ , which is a constant for a particular class in a dataset, and so is the number of transitively related views  $n = |\mathbb{V}^{*}| = 2^{l}$  for a class, and  $\Gamma = \theta l$ . Thus the relative uncertainty
between  $\delta_{\eta}$  and  $\delta_{*}$  would be given by:
$$
\frac {\delta_ {\eta}}{\delta_ {*}} \geq \frac {\mathcal {O} (2 ^ {- \epsilon^ {3}})}{\mathcal {O} (2 ^ {- \epsilon})} \approx \frac {2 ^ {- \epsilon^ {3}}}{2 ^ {- \epsilon}} = \frac {(2 ^ {- \epsilon}) ^ {\epsilon^ {2}}}{2 ^ {- \epsilon}} = (2 ^ {- \epsilon}) ^ {\epsilon^ {2} - 1} = 2 ^ {- \epsilon^ {3} + \epsilon} = \mathcal {O} (2 ^ {- \epsilon^ {3}})
$$
This completes the proof of the theorem.
A.10 Discussion on Graph-Based Image Representations and Proxy-Based Graph Metric Learning