SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proxy / Concept Graph: We obtain the proxy / concept graph for each class via an online clustering of the Semantic Relevance node and edge embeddings  $(\varphi_{n}(\mathbf{z}),\varphi_{e}(\mathbf{z})$  respectively) of the train-set instances of that class, using the Sinkhorn-Knopp algorithm [16, 8]. We set the number of node clusters to be equal to  $|V|$ . The number of edge clusters (connecting the node clusters) is equal to  $|V|(|V| - 1)/2$ . Note that this is the case because all the graphs throughout our pipeline are complete. Based on our theory of transitivity recovery, sets of semantically relevant nodes should form cliques in the semantic relevance and proxy graphs, and remain disconnected from the irrelevant ones by learning an edge weight of  $\mathbb{O}^n$ . We denote the set of class proxy graphs by  $\mathbb{P} = \{\mathcal{G}_{\mathbf{P}_1},\mathcal{G}_{\mathbf{P}_2},\dots,\mathcal{G}_{\mathbf{P}_k}\}$ , where  $k$  is the number of classes.
Inference and Learning objective: To recover end-to-end explainability, we need to avoid converting the instance and the proxy graphs to abstract vector-valued embeddings in  $\mathbb{R}^n$  at all times. For this purpose, we perform matching between  $\mathcal{G}_s$  and  $\mathcal{G}_{\mathbf{p}} \in \mathbb{P}$  via a graph kernel [22]. This kernel trick helps us bypass the computation of the abstract relationship  $\xi : \mathbb{G} \to \mathbb{R}^{n^2}$ , and perform the distance computation directly in  $\mathbb{G}$ , using only the graph-based decompositions, and thereby keeping the full pipeline end-to-end explainable.
We choose to use the Graph Edit Distance (GED) as our kernel [58]. Although computing GED has been proven to be NP-Complete [79], quadratic time Hausdorff Edit Distance (HED) [25] and learning-based approximations [65] have been shown to be of practical use. However, such approximations are rather coarse-grained, as they are either based on linearity assumptions [65], or consider only the substitution operations [25]. Learnable (L)-HED [64] introduced two additional learnable costs for node insertions and deletions using a GNN, thereby making it a more accurate approximation. We observe that our  $\varphi$  already performs the job of the encoder prefix in L-HED (Appendix A.12). Thus, we simply apply a fully connected MLP head  $\psi : \mathcal{M} \to \mathbb{R}$  with shared weights on the  $\varphi(\mathbf{z})$  embeddings and the vertices of  $\mathcal{G}_p \in \mathbb{P}$  to compute the node insertion and deletion costs. Thus, in our case, L-HED takes the following form:
$$
h \left(\mathcal {G} _ {s}, \mathcal {G} _ {p}\right) = \alpha \left(\sum_ {\mathbf {u} \in \mathcal {G} _ {s}} \min  _ {\mathbf {v} \in \mathcal {G} _ {p}} c (\mathbf {u}, \mathbf {v}) + \sum_ {\mathbf {v} \in \mathcal {G} _ {p}} \min  _ {\mathbf {u} \in \mathcal {G} _ {s}} c (\mathbf {u}, \mathbf {v})\right) \tag {1}
$$
$$
c (\mathbf {u}, \mathbf {v}) = \left\{ \begin{array}{l} | | \psi (\mathbf {u}) | |, \text {f o r d e l t i o n s} (\mathbf {u} \to \varepsilon), \\ | | \psi (\mathbf {v}) | |, \text {f o r i n s e r t i o n s} (\varepsilon \to \mathbf {v}), \\ | | \mathbf {u} - \mathbf {v} | | / 2, \text {f o r s u b s t i t u t i o n s}, \end{array} \right.
$$
where  $\alpha = 1 / (2|\mathbb{V}|)$ , and  $\varepsilon$  is the empty node [64]. We use  $h(\cdot, \cdot)$  as a distance metric for the Proxy Anchor Loss [39], which forms our final learning objective, satisfying Theorem 2. Below we discuss how transitivity recovery additionally guarantees robustness to noisy views.
3.4 Robustness
Let  $\mathcal{D}$  be the data distribution over  $\mathbb{X} \times \mathbb{Y}$ . Consider a sample  $\mathbf{x} \in \mathbb{X}$  composed of views  $\mathbb{V} = \{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}$  with label  $\mathbf{y} \in \mathbb{Y}$ . A view  $\mathbf{v} \in \mathbb{V}$  is said to be noisy if  $\operatorname{label}(\mathbf{v}) \neq y$  [13]. The
2Graph kernels, in general, help bypass mappings  $\xi :\mathbb{G}\to \mathcal{H}$  , where  $\mathcal{H}$  is the Hilbert space [22].
Figure 3: Sample TRD explanations on CUB. Top: 8 nearest neighbors (columns) of the top-2 proxy nodes / concepts (rows) with highest emergence (Definition 2). Bottom: Instance (global view with green border) and proxy (global view with blue border) explanation graphs constructed using the top-6 nodes (nearest train set neighbor for the proxy nodes) with highest emergence. The edge-weights are  $L_{2}$ -norms of the edge embeddings (best visible on screen with zooming).
fraction of noisy views for a given sample is denoted by  $\eta$ . Let  $f$  be a classifier that minimizes the empirical risk of the mapping  $f(\mathbf{x}) \mapsto \mathbf{y}$  in the ideal noise-free setting. Let  $\mathbb{V}^{\eta}$  be the set of noisy views in  $\mathbb{V}$  with topology  $\tau^{\eta}$ . Let  $\mathbb{V}^{*} = \mathbb{V} - \mathbb{V}^{\eta}$  be the set of noise free (and hence, transitively related - Theorem 3) views with topology  $\tau^{*}$ . Recall that  $\mathcal{G}^{*}$  is the optimal graph of  $\mathbb{V}$  satisfying the sufficiency criterion in Theorem 2, and  $\mathcal{G}_{\mathbf{p}} \in \mathbb{P}$  is its proxy graph.
Theorem 4. In the representation space of  $\varphi$ , the uncertainty in estimating the topology  $\tau^\eta$  of  $\mathbb{V}^\eta$  is exponentially greater in the error rate  $\epsilon$ , than the uncertainty in estimating the topology  $\tau^*$  of  $\mathbb{V}^*$ .
Intuition: This theorem is based on the fact the number of topologies that a set of noisy views can assume, is exponentially more (in the desired error rate) to what can be assumed by a set of transitive views. Since  $\varphi$  is explicitly designed to be transitivity recovering, making predictions using  $\varphi$  based on noisy views would go directly against its optimization objective of reducing the output entropy (due to the exponentially larger family of topologies to choose from). So, to minimize the uncertainty in Equation (1),  $\varphi$  would always make predictions based on transitive views, disregarding all noisy views in the process, satisfying the formal requirement of robustness (Definition 4). In other words, the following property of TRD allowed us to arrive at this result - the structural priors that we have on the transitive subgraphs make them the most optimal candidates for reducing the prediction uncertainty, relative to the isolated subgraphs of noisy views which do not exhibit such regularities.
4.1 Experimental Settings & Datasets
Implementation Details: For obtaining the global view, we follow [84, 94] by selecting the smallest bounding box containing the largest connected component of the thresholded final layer feature map obtained from an ImageNet-1K [17] pre-trained ResNet50 [30], which we also use as the relation-agnostic encoder  $f$ . The global view is resized to  $224 \times 224$ . We then obtain 64 local views by randomly cropping  $28 \times 28$  regions within the global crop and resizing them to  $224 \times 224$ . We use a 8-layer Graph Attention Network (GAT) ([77], with 4 attention heads in each hidden layer, and normalized via GraphNorm [7] to obtain the Semantic Relevance Graph. We train TRD for 1000 epochs using the Adam optimizer, at an initial learning rate of 0.005 (decayed by a factor of 0.1 every 100 epochs), with a weight decay of  $5 \times 10^{-4}$ . We generally followed [77, 10] for choosing the above settings. We implement TRD with a single NVIDIA GeForce RTX 3090 GPU, an 8-core Intel Xeon processor, and 32 GBs of RAM.
Datasets: To verify the generalizability and scalability of our method, we evaluate it on small, medium and large-scale FGVC benchmarks. We perform small-scale evaluation on the Soy and Cotton Cultivar datasets [89], while we choose FGVC Aircraft [53], Stanford Cars [42], CUB [80],
and NA Birds [75] for medium scale evaluation. For large-scale evaluation, we choose the iNaturalist dataset [76], which has over 675K train set and 182K test set images.
4.2 Interpretability & Robustness
Qualitative Results: Figure 3 shows sample explanations obtained using TRD on the CUB dataset. The top rows represent proxy nodes (concepts), and the columns represent the 8 nearest neighbors of the corresponding proxy nodes, which is highly consistent across concepts and classes. In the bottom are instance and proxy explanation graphs (top-6 nodes with highest emergence) from very similar-looking but different classes. For the proxies, the nearest train set neighbors to the node embeddings are visualized. Even with similar appearance, the graphs have very different structures. This shows that TRD is able to recover the discriminative relational properties of an image, and encode them as graphs. We provide additional visualizations in the supplementary.
Figure 4: Fidelity vs Sparsity curves of TRD and SOTA GNN interpretability methods.
Quantitative Evaluation: We quantitatively evaluate the explanations obtained using TRD based on the Fidelity (relevance of the explanations to the downstream task) vs Sparsity (precision of the explanations) plots for the generated explanations, which is a standard way to measure the efficacy of GNN explainability algorithms [92, 62, 91]. We compare against SOTA, generic GNN explainers, namely SubgraphX [92], PGExplainer [51], and GNNExplainer [88], on candidate small (Cotton), medium (FGVC Aircraft), and large (iNaturalist) scale datasets, and report our findings in Figure 4. TRD consistently provides the highest levels of Fidelity not only in the dense, but also in the high sparsity regimes, outperforming generic GNN explainers by significant margins. Unlike the generic explainers, since TRD takes into account the edit distance with the proxy graph for computing the explanations, it is able to leverage the class-level topological information, thereby achieving higher precision. The supplementary contains results on the remaining datasets, details on the Fidelity and Sparsity metrics, and experiments on the functional equivalence of TRD with post-hoc explanations.
Figure 5: Classification accuracies of relational learning based methods with different noise models on FGVC Aircraft. Left: Increasing number of local views sampled randomly across the entire image (instead of just the global view). Right: Increasing the proportion of noisy views in a controlled manner by sampling from the region outside of the global view.
Robustness to Noisy-Views: Figure 5 shows the performance of TRD on FGVC Aircraft under the following two noise injection models - (1) The local views are sampled uniformly at random across
the entire image, instead of just the global view. This model thus randomizes the amount of label information present in  $|\mathbb{L}|$ . (2) A fraction  $\eta$  of the local views are sampled from outside the global view, and the remaining ones  $(1 - \eta)$  come from within the global view, where  $\eta$  is the variable across experiments. This puts a fixed upper bound to the amount of label information in  $|\mathbb{L}|$ . It can be seen that under model (1), TRD is significantly more stable to changing degrees of uncertainty in label information compared to other relational learning-based methods. Under model (2), TRD outperforms existing methods as the amount of label information is decreased by deterministically increasing the noise rate. As discussed in Section 3.4 and formally proved in Appendix A.9.1, transitivity acts as a semantic invariant between the instance and the proxy graph spaces, which is a condition that subgraphs induced by the noisy views do not satisfy, leading to the observed robustness.
Effect of Causal Interventions: Here, we aim to understand the behaviour of our model under corruptions that affect the underlying causal factors of the data generating process. To this end, we train TRD by replacing a subset of the local views for each instance with local views from other classes, both during training and inference. As the proxies are obtained via a clustering of the instance graphs, these local views consequently influence the proxy graphs. We report our quantitative and qualitative findings in Table 1 and Figure 6 (Appendix A.17) respectively. TRD significantly outperforms Relational Proxies at all noise rates (percentage of local views replaced), and the gap between their performances widens as the percentage of corruption increases (Table 1). Qualitatively (Figure 6), our model successfully disregards the views introduced from the negative class at both the instance and proxy level. Such views can be seen as being very weakly connected to the global view, as well as the correct set of local views that actually belong to that class. Under this causal intervention, the TRD objective is thus equivalent to performing classification while having access to only the subgraph of clean views from the correct class.
η	10	20	30	40	50
Relational Proxies	93.22	87.12	79.35	70.99	63.60
TRD (Ours)	94.90	91.54	82.80	76.35	70.55
4.3 FGVC Performance
Comparison with SOTA: Table 2 compares the performance of TRD with SOTA on benchmark FGVC datasets. Some of the most commonly used approaches involve some form of regularizer [21, 41], bilinear features [98], or transformers [29, 81]. However, methods that use multi-view information [94, 49], especially their relationships [4, 97, 10, 5, 71] for prediction have been the most promising, although their final image and class representations are abstract. TRD can be seen to provide the best performance across all benchmarks, while also being fully interpretable. We attribute this to the robustness of our method to noisy views, which is known to have a significant impact on metric learning algorithms [13].
Ablation Studies: Table 3 shows the contributions of the components of TRD, namely Complementarity Graph: CG (Section 3.3), Proxy Decomposition: PD (Corollary 1.1), and Transitivity Recovery: TR (Theorem 3)
to its downstream classification accuracy on the FGVC Aircraft, Cotton, and Soy Cultivar datasets. Row 0 is the baseline with none of our novel components. Row 1 corresponds to the setting where the proxy representation is still abstract, i.e., in  $\mathbb{R}^n$ . We concatenate the nodes in  $\mathcal{G}_s$  and propagate it through a 1-hidden layer MLP to summarize it in  $\mathbb{R}^n$ . As we decompose the proxy into a graph (PD, Row 2) based on the instance node and edge clustering-based approach as discussed in Section 3.3 for class-level interpretability, we can see that the performance
Table 1: Quantitative performance comparison between Relational Proxies and TRD with increasing rates of corruption ( $\eta$ : percentage local views from different classes).
ID	CG	PD	TR	Aircraft	Cotton	Soy
0.	94.60	67.70	46.00
1.	✓	95.10	69.60	47.70
2.	✓	94.94	68.31	46.70
3.	✓	✓	95.15	69.70	50.32
4.	✓	✓	95.40	70.10	50.99
5.	✓	✓	✓	95.60	70.90	52.15
Table 3: Ablations on the key components of TRD namely Complementarity Graph (CG), Proxy Decomposition (PD), and TR (Transitivity Recovery).
Method	Small	Medium	Large
Cotton	Soy	FGVC Aircraft	Stanford Cars	CUB	NA Birds	iNaturalist
MaxEnt, NeurIPS'18	-	-	89.76	93.85	86.54	-	-
DBTNet, NeurIPS'19	-	-	91.60	94.50	88.10	-	-
StochNorm, NeurIPS'20	45.41	38.50	81.79	87.57	79.71	74.94	60.75
MMAL, MMM'21	65.00	47.00	94.70	95.00	89.60	87.10	69.85
FFVT, BMVC'21	57.92	44.17	79.80	91.25	91.65	89.42	70.30
CAP, AAAI'21	-	-	94.90	95.70	91.80	91.00	-
GaRD, CVPR'21	64.80	47.35	94.30	95.10	89.60	88.00	69.90
WTFocus, ECCV'22	-	-	94.70	95.30	90.80	-	-
SR-GNN, TIP'22	-	-	95.40	96.10	91.90	-	-
TransFG, AAAI'22	45.84	38.67	80.59	94.80	91.70	90.80	71.70
Relational Proxies, NeurIPS'22	69.81	51.20	95.25	96.30	92.00	91.20	72.15
PMRC, CVPR'23	-	-	94.80	95.40	91.80	-	-
TRD (Ours)	70.90 ± 0.22	52.15 ± 0.12	95.60 ± 0.08	96.35 ± 0.03	92.10 ± 0.04	91.45 ± 0.12	72.27 ± 0.05
Table 2: Comparison with SOTA on standard small, medium, and large scale FGVC benchmarks.
drops. This happens because of a lack of prior knowledge about the proxy-graph structure, which increases room for noisy views to be more influential, harming classification robustness. To alleviate this, we introduce TR based on our findings from Theorem 3, which effectively wards off the influence of noisy views. With PD, Rows 3 and 4 respectively show the individual contributions of CG, and enforcing the Transitivity invariant between instance and proxy graphs. Row 5 shows that TRD is at its best with all components included.
We presented Transitivity Recovering Decompositions (TRD), an approach for learning interpretable relationships for FGVC. We theoretically showed the existence of semantically equivalent graphs for abstract relationships, and derived their key information theoretic and topological properties. TRD is a search algorithm in the space of graphs that looks for solutions that satisfy the above properties, providing a concrete, human understandable representation of the learned relationships. Our experiments revealed that TRD not only provides end-to-end transparency, all the way up to the class-proxy representation, but also achieves state-of-the-art results on small, medium, and large scale benchmark FGVC datasets, a rare phenomenon in the interpretability literature. We also showed that our method is robust to noisy input views, both theoretically and empirically, which we conjecture to be a crucial factor behind its effectiveness.
Limitations: Although robust to noisy views, our method is not fully immune to spurious correlations (Figure 3, rightmost graph – the wing and the sky are spuriously correlated; additional examples in the supplementary). Combined with recent advances in learning decorrelated representations [66], we believe that our method can overcome this limitation while remaining fully interpretable.
Societal Impacts: Our method makes a positive societal impact by adding interpretability to existing fine-grained relational representations, in a provably robust manner. Although we are not aware of any specific negative societal impacts that our method might have, like most deep learning algorithms, our method is susceptible to the intrinsic biases of the training set.
This work was supported by the MUR PNRR project FAIR - Future AI Research (PE00000013) funded by the NextGenerationEU.
[81] Jun Wang, Xiaohan Yu, and Yongsheng Gao. Feature Fusion Vision Transformer for Fine-Grained Visual Categorization. In BMVC, 2021.
[82] Xiang Wang, Yingxin Wu, An Zhang, Xiangnan He, and Tat-Seng Chua. Towards multi-grained explainability for graph neural networks. In NeurIPS, 2021.
[83] Zhuhui Wang, Shijie Wang, Haojie Li, Zhi Dou, and Jianjun Li. Graph-propagation based correlation learning for weakly supervised fine-grained image classification. In AAAI, 2020.
[84] Xiu Shen Wei, Jian Hao Luo, Jianxin Wu, and Zhi Hua Zhou. Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrieval. IEEE TIP, 2017.
[85] Xiu Shen Wei, Chen Wei Xie, Jianxin Wu, and Chunhua Shen. Mask-cnn: Localizing parts and selecting descriptors for fine-grained bird species categorization. PR, 2018.
[86] Guang Yang, Juan Cao, Zhineng Chen, Junbo Guo, and Jintao Li. Graph-based neural networks for explainable image privacy inference. PR, 2020.
[87] Xuhui Yang, Yaowei Wang, Ke Chen, Yong Xu, and Yonghong Tian. Fine-grained object classification via self-supervised pose alignment. In CVPR, 2022.
[88] Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In NeurIPS, 2019.
[89] Xiaohan Yu, Yang Zhao, Yongsheng Gao, Shengwu Xiong, and Xiaohui Yuan. Patchy image structure classification using multi-orientation region transform. In AAAI, 2020.
[90] Hao Yuan, Jiliang Tang, Xia Hu, and Shuiwang Ji. Xgnn: Towards model-level explanations of graph neural networks. In KDD, 2020.
[91] Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: A taxonomic survey. arXiv, 2020.
[92] Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In ICML, 2021.
[93] Vinicius Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David Reichert, Timothy Lillicrap, Edward Lockhart, Murray Shanahan, Victoria Langston, Razvan Pascanu, Matthew Botvinick, Oriol Vinyals, and Peter Battaglia. Deep reinforcement learning with relational inductive biases. In ICLR, 2019.
[94] Fan Zhang, Meng Li, Guisheng Zhai, and Yizhao Liu. Multi-branch and Multi-scale Attention Learning for Fine-Grained Visual Categorization. In MMM, 2021.
[95] Han Zhang, Tao Xu, Mohamed Elhoseiny, Xiaolei Huang, Shaoting Zhang, Ahmed Elgammal, and Dimitris Metaxas. Spda-cnn: Unifying semantic part detection and abstraction for fine-grained recognition. In CVPR, 2016.
[96] Ning Zhang, Jeff Donahue, Ross Girshick, and Trevor Darrell. Part-based r-cnns for fine-grained category detection. In ECCV, 2014.
[97] Yifan Zhao, Ke Yan, Feiyue Huang, and Jia Li. Graph-based high-order relation discovery for fine-grained recognition. In CVPR, 2021.
[98] Heliang Zheng, Jianlong Fu, Zheng-Jun Zha, and Jiebo Luo. Learning deep bilinear transformation for fine-grained image representation. In NeurIPS, 2019.
[99] Yuehua Zhu, Muli Yang, Cheng Deng, and Wei Liu. Fewer is more: A deep graph metric learning perspective using fewer proxies. In NeurIPS, 2020.
Below, we provide some theoretical results and quantification of empirical observations from the existing literature [24, 9, 10] that are used in the main text.
A.1 Identities of Mutual Information
Let three random variables  $\mathbf{x}$ ,  $\mathbf{y}$ , and  $\mathbf{z}$  form the Markov Chain  $\mathbf{x} \to \mathbf{z} \to \mathbf{y}$ . Then,
- Positivity:
$$
I (\mathbf {x}; \mathbf {y}) \geq 0, I (\mathbf {x}; \mathbf {y} | \mathbf {z}) \geq 0
$$
- Chain Rule:
$$
I (\mathbf {x}; \mathbf {y}) = I (\mathbf {x}; \mathbf {y} | \mathbf {z}) + I (\mathbf {x}; \mathbf {z})
$$
Data Processing Inequality:
$$
I (\mathbf {x}; \mathbf {z}) \geq I (\mathbf {x}; \mathbf {y})
$$
A.2 Information Gap
The Information Gap [10] is the uncertainty that remains about the label information  $\mathbf{y}$  given a relation-agnostic representation  $\mathbf{z}$ . Quantitatively,
$$
I (\mathbf {x}; \mathbf {r} | \mathbf {z}) = I (\mathbf {x}; \mathbf {y}) - I (\mathbf {z}; \mathbf {y})
$$
An implication of the above is that the label information in  $\mathbf{x}$  can be factorized as follows:
$$
I (\mathbf {x}; \mathbf {y}) = I (\mathbf {x}; \mathbf {r} | \mathbf {z}) + I (\mathbf {z}; \mathbf {y})
$$
A.3 Sufficiency
A representation  $\mathbf{z}$  of  $\mathbf{x}$  is sufficient [24] for predicting its label  $\mathbf{y}$  if and only if, given  $\mathbf{z}$ , there remains no uncertainty about the label of  $\mathbf{y}$  of  $\mathbf{x}$ . Formally,
$$
I (\mathbf {x}; \mathbf {y} | \mathbf {z}) = 0 \iff I (\mathbf {x}; \mathbf {y}) = I (\mathbf {z}; \mathbf {y})
$$
A.4 Relation-Agnostic Representations - Information Theoretic
An encoder  $f$  is said to produce relation-agnostic representations [10] if it independently encodes the global view  $\mathbf{g}$  and local views  $\mathbf{v} \in \mathbb{L}$  of  $\mathbf{x}$  without considering their relationship information  $\mathbf{r}$ . Quantitatively,
$$
I (\mathbf {x}; \mathbf {y} | \mathbf {z}) = I (\mathbf {x}; \mathbf {r}) = I (\mathbf {v} _ {1} \mathbf {v} _ {2} \dots \mathbf {v} _ {| \mathbb {L} |}; \mathbf {g})
$$
A.5 Distributive property of  $f$
Since  $I(\mathbf{x};\mathbf{r})$  refers to local-to-global relationships  $I(\mathbf{v}_1\mathbf{v}_2\dots \mathbf{v}_{|\mathbb{L}|};\mathbf{g})$ , the definition of relation ag-. nosticity in Appendix A.4 implies that  $\mathbf{z} = f(\mathbf{x})$  does not reduce any uncertainty about emergence, i.e.,
$$
I (\mathbf {x}; \mathbf {r}) = I (\mathbf {v} _ {1} \mathbf {v} _ {2} \dots \mathbf {v} _ {| \mathbb {L} |}; \mathbf {g}) = I (f (\mathbf {v} _ {1}) f (\mathbf {v} _ {2}) \dots f (\mathbf {v} _ {| \mathbb {L} |}); f (\mathbf {g})) = I (\mathbf {z} _ {1}, \mathbf {z} _ {2}, \dots \mathbf {z} _ {| \mathbb {L} |}; \mathbf {z} _ {\mathbf {g}})
$$
A.6 Sufficient Learner
Following are the necessary and sufficient conditions for a sufficient learner [10]: