SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 1: Dataset details and the results of our ICON compared with existing methods. Details on * in Section 5.1. WILDS 2.0 [47] datasets are highlighted in yellow, where models were evaluated on both the test data in  $S$  (first number) and in  $T$  (second number). The performance gain in  $T$  highlighted in red. "ERM (labeled  $T$  )" has full supervision in  $T$ . -/- means that the method is not applicable to the dataset. Other dataset details and the standard deviation of the results are in Appendix.
As shown in Table 1, we extensively evaluated ICON on 10 datasets with standard protocols [34, 47], including 2 classic ones: OFFICE-HOME [60], VISDA-2017 [43], and 8 others from the recent WILDS 2.0 benchmark [47]. WILDS 2.0 offers large-scale UDA tasks in 3 modalities (image, text, graph) under real-world settings, e.g., wildlife conservation, medical imaging and remote sensing. Notably, unlike the classic datasets where the same sample set in  $T$  is used in training (unlabelled) and testing, WILDS 2.0 provides unlabelled train set in  $T$  disjoint with the test set, which prevents explicit pattern mining of the test data. They also provide validation sets for model selection. For datasets in WILDS 2.0 benchmark, we drop their suffix -WILDS for simplicity (e.g., denoting AMAZON-WILDS as AMAZON).
Details of * in Table 1: OFFICE-HOME has 4 domains: artistic images (Ar), clip art (Cl), product images (Pr), and real-world images (Rw), which forms 12 UDA tasks by permutation, and we only show Cl→Rw as an example in the table. FMOW considers the worst-case performance on different geographic regions, where the land use in Africa is significantly different. POVERTYMAP considers the lower of the Pearson correlations (\%) on the urban or rural areas. CIVILCOMENTS correspond to a semi-supervised setting with no domain gap between  $S$  and  $T$ , and is evaluated on the worst-case performance among demographic identities.
On some WILDS 2.0 datasets, the test data may come from a separate domain different from the training target domain  $T$ , e.g., IWildCAM test data is captured by 48 camera traps disjoint from the 3215 camera traps in the training  $T$ . We highlight that ICON is still applicable on these datasets. This is because ICON works by learning the causal feature  $\mathbf{c}$  and discarding the environmental feature  $\mathbf{e}$ , and the theory holds as long as the definition of  $\mathbf{c}$  and  $\mathbf{e}$  is consistent across the training and test data, e.g., IWildCAM is about animal classification across the labeled, unlabeled, and test data.
Method	Ar→Cl	Ar→Pr	Ar→Rw	Cl→Ar	Cl→Pr	Cl→Rw	Pr→Ar	Pr→Cl	Pr→Rw	Rw→Ar	Rw→Cl	Rw→Pr	Avg
DANN [13] (2016)	45.6	59.3	70.1	47.0	58.5	60.9	46.1	43.7	68.5	63.2	51.8	76.8	57.6
CDAN [34] (2018)	50.7	70.6	76.0	57.6	70.0	70.0	57.4	50.9	77.3	70.9	56.7	81.6	65.8
SymNet [69] (2019)	47.7	72.9	78.5	64.2	71.3	74.2	64.2	48.8	79.5	74.5	52.6	82.7	67.6
MDD [70] (2019)	54.9	73.7	77.8	60.0	71.4	71.8	61.2	53.6	78.1	72.5	60.2	82.3	68.1
SHOT[28] (2020)	57.1	78.1	81.5	68.0	78.2	78.1	67.4	54.9	82.2	73.3	58.8	84.3	71.8
ALDA [7] (2020)	53.7	70.1	76.4	60.2	72.6	71.5	56.8	51.9	77.1	70.2	56.3	82.1	66.6
GVB [8] (2020)	57.0	74.7	79.8	64.6	74.1	74.6	65.2	55.1	81.0	74.6	59.7	84.3	70.4
TCM [67] (2021)	58.6	74.4	79.6	64.5	74.0	75.1	64.6	56.2	80.9	74.6	60.7	84.7	70.7
SENTRY [44] (2021)	61.8	77.4	80.1	66.3	71.6	74.7	66.8	63.0	80.9	74.0	66.3	84.1	72.2
CST [32] (2021)	59.0	79.6	83.4	68.4	77.1	76.7	68.9	56.4	83.0	75.3	62.2	85.1	73.0
ToAlign [64] (2021)	57.9	76.9	80.8	66.7	75.6	77.0	67.8	57.0	82.5	75.1	60.0	84.9	72.0
FixBi [40] (2021)	58.1	77.3	80.4	67.7	79.5	78.1	65.8	57.9	81.7	76.4	62.9	86.7	72.7
ATDOC [29] (2021)	60.2	77.8	82.2	68.5	78.6	77.9	68.4	58.4	83.1	74.8	61.5	87.2	73.2
SDAT [45] (2022)	58.2	77.1	82.2	66.3	77.6	76.8	63.3	57.0	82.2	74.9	64.7	86.0	72.2
MCC+NWD [6] (2022)	58.1	79.6	83.7	67.7	77.9	78.7	66.8	56.0	81.9	73.9	60.9	86.1	72.6
kSHOT* [55] (2022)	58.2	80.0	82.9	61.1	80.3	80.7	71.3	56.8	83.2	75.5	60.3	86.6	73.9
ICON (Ours)	63.3	81.3	84.5	70.3	82.1	81.0	70.3	61.8	83.7	75.6	68.6	87.3	75.8
Table 2: Break-down of the accuracies in each domain on OFFICE-HOME dataset [60]. *: kSHOT [55] additionally uses the prior knowledge on the percentage of samples in each class in the testing data. Published years are in the brackets after the method names.
Feature Backbone. We used the followings pre-trained on ImageNet [46]: ResNet-50 [19] on OFFICE-HOME, VISDA-2017 and IWILDCAM, DenseNet-121 [21] on FMOW and FasterRCNN [14] on GLOBALWHEAT. We used DistilBERT [48] with pre-trained weights from the Transformers library on CIVILCOMMENTS and AMAZON. On CAMELYON17, we used DenseNet-121 [21] pre-trained by the self-supervised SwAV [5] with the training data in  $S$  and  $T$ . On POVERTYMAP and OGB-MOLPCBA with no pre-training available, we used multi-spectral ResNet-18 [19] and graph isomorphism network [66] trained with the labelled samples in the source domain  $S$ , respectively.
Self-training. A general form of the self-training loss  $\mathcal{L}_{st}$  in Eq. (3) is given below:
$$
\mathcal {L} _ {s t} = \mathbb {E} _ {\mathbf {x} \sim T} [ \mathbb {1} (\max  f (\mathbf {x}) > \tau) \mathrm {C E} (f (\mathbf {x}), \operatorname {a r g m a x} f (\mathbf {x})) ], \tag {4}
$$
where  $\tau$  is a confident threshold that discards low-confidence pseudo-labels, and  $\mathrm{CE}(\cdot ,\cdot)$  is the cross-entropy loss. We detail the different implementations of Eq (4) in Appendix. ICON is agnostic to the choice of  $\mathcal{L}_{st}$ , and hence we choose the best performing self-training baseline for each dataset: FixMatch [53] on OFFICE-HOME and VISDA-2017, NoisyStudent [65] on IWILDCAM and FMOW, Pseudo-label [26] on CIVILCOMMENTS and AMAZON. ICON also does not rely on  $\mathcal{L}_{st}$ . We did not apply  $\mathcal{L}_{st}$  (i.e.,  $\beta = 0$ ) on CAMELYON17, POVERTYMAP, GLOBALWHEAT and OGB-MOLPCBA where self-training does not work well.
Regression and Detection. For the regression task in POVERTYMAP,  $f$  is a regressor that outputs a real number. As the number of clusters in  $T$  is not well-defined, we directly used rank statistics to determine if two samples in  $T$  have the same label. For the BCE loss, we computed the similarity by replacing  $f(\mathbf{x}_i)^{\top}f(\mathbf{x}_j)$  in Eq. (1) as  $-(f(\mathbf{x}_i) - f(\mathbf{x}_j))^2$ . For detection in GLOBALWHEAT, we used the ICON objective in Eq. (3) to train the classification head without other modifications.
Other Details. We aim to provide a simple yet effective UDA method that works across data modalities, hence we did not use tricks tailored for a specific modality (e.g., image), such as mixup [68] or extensive augmentations (16 are used in [11]). On classic UDA datasets, we used entropy minimization and SAM optimizer following CST [32]. On VISDA-2017, iWILDCAM, CAMELYON17 and FMOw, we pre-processed the features with UMAP [37] (output dimension as 50) and EqInv [61], which helps fulfill Assumption 1 (i.e., highlight causal feature to improve clustering). ICON is also resource efficient—all our experiments can run on a single NVIDIA 2080Ti GPU.
Classic UDA Benchmarks. In Table 1 (first 2 columns), ICON significantly outperforms the existing state-of-the-arts on OFFICE-HOME [60] and VISDA-2017 [43] by  $2.6\%$  and  $3.7\%$ , respectively. We include the breaks down of the performances of OFFICE-HOME in Table 2, where ICON wins in 10 out of 12 tasks, and significantly improves the hard tasks (e.g.,  $\mathrm{Ar} \rightarrow \mathrm{Cl}$ ,  $\mathrm{Rw} \rightarrow \mathrm{Cl}$ ). For VISDA-2017, we include more comparisons in Table 3. ICON beats the original competition winner (MT+16augs) [11] that averages predictions on 16 curated augmented views. ICON even outperforms all methods that use the much deeper ResNet-101 [19]. Notably, CST [32] and ATDOC [29] also aim to remove
Method	Backbone	Acc.
MT+16augs [11] (2018)	ResNet-50	82.8
MDD [70] (2019)	ResNet-50	77.8
GVB [8] (2020)	ResNet-50	75.3
TCM [67] (2021)	ResNet-50	75.8
SENTRY [44] (2021)	ResNet-50	76.7
CST [32] (2021)	ResNet-50	80.6
CAN [24] (2019)	ResNet-101	87.2
SHOT [28] (2020)	ResNet-101	82.9
FixBi [24] (2021)	ResNet-101	87.2
MCC+NWD [6] (2022)	ResNet-101	83.7
SDAC [45] (2022)	ResNet-101	84.3
kSHOT* [55] (2022)	ResNet-101	86.1
ICON (Ours)	ResNet-50	87.4
Table 3: Mean-class accuracy (Acc.) on VISDA-2017 Synthetic  $\rightarrow$  Real task with the choice of feature backbone. \*: details in Table 2 caption. Published years are in the brackets after the method names.
(a)
Figure 5: (a) Solid lines: test accuracy against training epochs for FixMatch, CST and ICON. Dotted black lines: average loss in each epoch with matching markers to denote the three methods. (b) Among the feature pairs where  $f$  fails, the percentage of pairs where  $g$  succeeds for the three methods.
the spurious correlation, and both attain competitive performance, validating the importance of this research objective. Yet their training scheme allows the correlation to persist (Section 4), hence ICON is more effective.
WILDS 2.0. In Table 1 (last 8 columns), ICON dominates the WILDS 2.0 leaderboard despite the challenging real-world data and stringent evaluation. For example, the animal images in IWILDCAM exhibit long-tail distribution that is domain-specific, because the rarity of species varies across locations. Hence under macro-F1 evaluation (sensitive to tail performance), the  $2.3\%$  improvement is noteworthy. On CIVIL COMMENTS with text data, ICON improves the worst performance across demographic groups by  $2.2\%$ , which promotes AI fairness and demonstrates the strength of ICON even under the SSL setting. On AMAZON, although ICON only improves  $0.5\%$  to reach  $54.7\%$ , the upper bound performance by training with labelled  $T$  ( $56.4\%$ ) is also low due to the strict evaluation metric. We also highlight that ICON increases the performance in  $T$  without sacrificing that in  $S$  (the first number). This empirically validates that the invariance objective of ICON resolves the tug-of-war scenario in Figure 2c. In particular, we tried implementing recent methods on this benchmark. Yet they are outperformed by ERM, or even not applicable (e.g., on text/graph data). This is in line with WILDS 2.0 organizers' observation that ERM as a naive baseline usually has the best performance, which is exactly the point of WILDS 2.0 as a diagnosis of the ever-overlooked issues in UDA. Hence we believe that dominating this challenging leaderboard is a strong justification for ICON, and encourage more researchers to try this benchmark.
Failure Cases. ICON performance is relatively weak on POVERTYMAP and OGB-MOLPCBA with two possible reasons: 1) no pre-training available, the backbone is trained only on  $S$  and hence biased in the beginning, and 2) the ground-truth number of classes in  $T$  is not well-defined in both datasets, hence Assumption 1 in Section 3 may fail.
Learning Curve. In Figure 5a, we compare our ICON with two self-training baselines: FixMatch [53] and CST [32]. The test accuracy and training loss w.r.t. training epochs on VISDA-2017 is drawn in colored solid lines and black dotted lines, respectively. We observe that the training losses for all methods converge quickly. However, while the two baselines achieve higher accuracy in the beginning, their performances soon start to drop. In contrast, ICON's accuracy grows almost steadily.
Effect of Supervision in  $T$ . To find the cause of the performance drop, we used the optimized  $g$  in Section 2.2 to probe the learned  $f$  in each epoch. Specifically, we sampled feature pairs  $\mathbf{x}_1, \mathbf{x}_2$  in  $T$  and tested if the predictions of  $f, g$  are consistent with their labels, e.g.,  $f$  succeeds if  $\operatorname{argmax} f(\mathbf{x}_1) = \operatorname{argmax} f(\mathbf{x}_2)$  when  $y_1 = y_2$  (vice versa for  $\neq$ ). Then among the pairs where  $f$  failed, we computed the percentage of them where  $g$  succeeded, which is shown in Figure 5b. A large percentage implies that  $f$  disrespects the inherent distribution in  $T$  correctly identified by  $g$ . We observe that the percentage has a rising trend for the two baselines, i.e., they become more biased to the spurious correlations in  $S$  that do not hold in  $T$ . This also explains their dropping performance in Figure 5a. In particular, CST has a lower percentage compared to FixMatch, which suggests that it has some effects in removing the bias. In contrast, the percentage steadily drops for ICON, as we train  $f$  to respect the inherent distribution identified by  $g$ .
Method	OFFICE-HOME	VISDA-2017
FixMatch	69.1	76.6
FixMatch+CON	74.1	82.0
FixMatch+CON+INV	75.8	87.4
Cluster with 2×#classes	69.7	78.6
Cluster with 0.5×#classes	67.5	76.2
Cluster with k-NN	72.1	85.6
Table 4: Ablations on each ICON component. CON denotes the consistency loss in  $S$  and  $T$  . INV denotes the invariance constraint.
α
β	OFFICE-HOME	α
β	VISDA-2017
0.25	0.5	1.0	2.0	0.25	0.5	1.0	2.0
0.05	73.8	75.0	75.5	73.9	0.05	85.2	86.2	86.3	81.5
0.1	73.9	75.4	75.8	74.4	0.1	85.6	86.4	86.7	82.6
0.25	70.2	72.8	75.2	74.5	0.25	76.8	81.2	87.4	86.3
Figure 6: Ablations of  $\alpha, \beta$  on the classic UDA datasets.
Components. Recall that the main components of ICON include: 1) Consistency loss (CON) that leverages the supervision from the labels in  $S$  and clusters in  $T$ ; 2) the use of invariance constraint (INV) in Eq. (3); 2) clustering in  $T$  with #clusters = #classes. Their ablations are included in Table 4. From the first three lines, we observe that adding each component leads to consistent performance increase on top of the self-training baseline (FixMatch), which validates their effectiveness. From the next two lines, when #clusters ≠ #classes, the model performs poorly, e.g., using one cluster in Figure 2b (i.e.,  $0.5 \times \#$  classes), the invariant classifier needs to predict the two red circles in  $S$  as dissimilar (two classes) and the two blue ones in  $T$  as similar (one cluster), which in fact corresponds to the erroneous red line (i.e., predictions in  $T$  are similarly ambiguous). In the last line, we tried k-means for clustering instead of rank-statistics, leading to decreased performance. We postulate that rank-statistics is better because its online clustering provides up-to-date cluster assignments. In the future, we will try other more recent online clustering methods, e.g., an optimal transport-based method [5]. Overall, the empirical findings validate the importance of the three components, which are consistent with the theoretical analysis in Section 3.
Hyperparameters. We used two hyperparameters in our experiments: 1) the weight of the self-training loss denoted as  $\alpha$ ; 2) the invariance penalty strength  $\beta$  in the REx [25] implementation of Eq. (3). Their ablations are in Figure 6 on the widely used classic UDA datasets. We observe that  $\alpha \in [0.5, 1.0]$  and  $\beta \in [0.1, 0.25]$  works well on both datasets. In particular, it is standard in the IRM community to choose a small  $\beta$  (e.g., 0.1). Then, tuning  $\alpha$  follows the same process as other self-training UDA methods. Hence the hyperparameter search is easy.
Confusion Matrix. We use the matrix to concretely show how existing methods are biased to the correlations in the source domain  $S$ . We compare the state-of-the-art self-training method CST with ICON in Figure 7. The matrix on the left reveals that CST predicted most "knife"s as "skateboard"s ("board" for short), and some "truck"s as "car"s. On the right, we see that ICON predicts "knife" accurately and also improves the accuracy on "truck".
Figure 7: CST and ICON confusion matrix on VIsDA-2017.
Grad-CAM Visualization. Why does CST predict "knife" in  $T$  as "board"? To answer this, in Figure 8, we visualized the GradCAM [50] of CST and ICON trained on VisDA-2017. In the first row, we notice that the self-training based CST predicts the two categories perfectly by leveraging their apparent differences: the curved blade versus the flat board ending with semi-circles. However, this leads to poor generalization on "knife" in  $T$ , where the background (image 1, 4), the knife component (2) or even the blade (3) can have board-like appearances. Moreover, biased to the standard "board" look in  $S$ , CST can leverage environmental feature for predicting "board" in  $T$ , e.g., under side-view (5) or slight obstruction (6), which is prone to errors (e.g., 8). As the board-like shape alone is non-discriminative for the two categories in  $T$ , in the second row, the cluster head  $g$  additionally leverages knife handles and wheels to distinguish them. In the third row, our ICON combines the
Figure 8: GradCAM on VisDA-2017 [43] using CST [32], the cluster head  $g$  and the proposed ICON. Ground-truth knives and boards are in solid and dotted boxes, respectively. Predictions from  $f, g$  are on top of each row. Red denotes wrong predictions. Pale blue and red background denotes training and testing, respectively. Note that  $g$  is trained without supervision in  $T$ .
supervision in  $S$  (first row) and  $T$  (second row) to learn an invariant classifier  $f$ . In  $S$ , ICON focuses on the overall knife shape instead of just blade, and incorporates wheels to predict "board". In  $T$ , by comparing images with the same number in the first and third row, we see that ICON focuses more on the discriminative parts of the objects (e.g., handles and wheels), hence generalizes better in  $T$ .
We presented a novel UDA method called Invariant CONsistency learning (ICON) that removes the spurious correlation specific to the source domain  $S$  to generalize in the target domain  $T$ . ICON pursues an invariant classifier that is simultaneously consistent with the labels in  $S$  and the clusters in  $T$ , hence removing the spurious correlation inconsistent in  $T$ . We show that ICON achieves generalization in  $T$  with a theoretical guarantee. Our theory is verified by extensive evaluations on both the classic UDA benchmarks and the challenging WILDS 2.0 benchmark, where ICON outperforms all the conventional methods. We will seek more principled pre-training paradigms that disentangle the causal features and relax the assumption of knowing the class numbers in  $T$  to improve regression tasks.
**Limitation.** Our approach is based on the assumptions in Section 4, i.e., the classes in  $T$  are separated by clusters, and there is enough diversity in  $T$  such that causal feature  $\mathbf{c}$  is the only invariance. Assumption 1 is essentially the clustering assumption, which is a necessary assumption for learning with unlabeled data [59]. In fact, UDA methods (and semi-supervised learning in general) commonly leverage feature pre-training and data augmentations to help fulfill the assumptions. One can also use additional pre-trained knowledge from foundation models, or deploy improved clustering algorithm (e.g., an optimal transport-based method [5]). Assumption 2 is necessary to rule out corner cases, e.g., in Figure 4b, without additional prior, there is no way for any UDA method to tell if each one of the blue cluster (unlabeled) should be pseudo-labeled as "0" or "1". In practice, these corner cases can be avoided by collecting diverse unlabeled data, which is generally abundant, e.g., collecting unlabeled street images from a camera-equipped vehicle cruise.
Broader Impact. Our work aims to capture the causal feature in the unsupervised domain adaptation setting. In particular, we significantly improve the state-of-the-art on WILDS 2.0 benchmark, which includes practical tasks that benefit wildlife preservation, tumour classification in healthcare, remote sensing, vision in agriculture, etc. The pursuit of causal feature can produce more robust, transparent and explainable models, broadening the applicability of ML models and promoting fairness in AI.
This research is supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award No: AISG2-RP-2021-022), MOE AcRF Tier 2 (MOE2019-T2-2-062), Alibaba-NTU Singapore Joint Research Institute, the A*STAR under its AME YIRG Grant (Project No.A20E6c0101), and the Lee Kong Chian (LKC) Fellowship fund awarded by Singapore Management University.
[1] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019.
[2] Yogesh Balaji, Rama Chellappa, and Soheil Feizi. Normalized wasserstein for mixture distributions with applications in adversarial learning and domain adaptation. In ICCV, 2019.
[3] David Berthelot, Nicholas Carlini, Ian Goodfellow, Avital Oliver, Nicolas Papernot, and Colin Raffel. Mixmatch: A holistic approach to semi-supervised learning. In NeurIPS, 2019.
[4] Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. Domain separation networks. In NeurIPS, 2016.
[5] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. In NeurIPS, 2020.
[6] Lin Chen, Huaian Chen, Zhixiang Wei, Xin Jin, Xiao Tan, Yi Jin, and Enhong Chen. Reusing the task-specific classifier as a discriminator: Discriminator-free adversarial domain adaptation. In CVPR, 2022.
[7] Minghao Chen, Shuai Zhao, Haifeng Liu, and Deng Cai. Adversarial-learned loss for domain adaptation. In AAAI, 2020.
[8] Shuhao Cui, Shuhui Wang, Junbao Zhuo, Chi Su, Qingming Huang, and Qi Tian. Gradually vanishing bridge for adversarial domain adaptation. In CVPR, 2020.
[9] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
[10] Zhekai Du, Jingjing Li, Hongzu Su, Lei Zhu, and Ke Lu. Cross-domain gradient discrepancy minimization for unsupervised domain adaptation. In CVPR, 2021.
[11] Geoffrey French, Michal Mackiewicz, and Mark Fisher. Self-ensembling for visual domain adaptation. In ICLR, 2018.
[12] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In ICML, 2015.
[13] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 2016.
[14] Ross Girshick. Fast r-cnn. In ICCV, 2015.
[15] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. Geodesic flow kernel for unsupervised domain adaptation. In CVPR, 2012.
[16] Chen Gong, Tongliang Liu, Dacheng Tao, Keren Fu, Enmei Tu, and Jie Yang. Deformed graph laplacian for semisupervised learning. IEEE transactions on neural networks and learning systems, 2015.
[17] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. In NeurIPS, 2004.
[18] Kai Han, Sylvestre-Alvise Rebuffi, Sebastien Ehrhardt, Andrea Vedaldi, and Andrew Zisserman. Automatically discovering and learning new visual categories with ranking statistics. In ICLR, 2020.
[19] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
[20] Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthew, Danilo Rezende, and Alexander Lerchner. Towards a definition of disentangled representations. arXiv preprint arXiv:1812.02230, 2018.