SlowGuess/ACL_2020_OCR huggingface dataset:
Table 3: Performance (in UUAS and DSpr.) on transfer between all language pairs in our dataset. All runs were repeated 3 times; runs for which the range in performance exceeded 2 points (for UUAS) or 0.02 (for DSpr.) are marked with an asterisk (*).
ar	cz	de	en	es	fa	fi	fr	id	lv	zh
ar	0.000	1.044	1.048	1.049	1.015	1.046	1.058	1.022	1.031	1.059	1.076
cz	1.044	0.000	0.982	1.017	0.970	1.064	1.021	1.007	1.053	1.011	1.083
de	1.048	0.982	0.000	1.005	0.973	1.044	1.017	0.971	1.022	1.029	1.065
en	1.049	1.017	1.005	0.000	0.983	1.051	1.033	0.994	1.035	1.040	1.060
es	1.015	0.970	0.973	0.983	0.000	1.038	1.023	0.936	1.010	1.024	1.065
fa	1.046	1.064	1.044	1.051	1.038	0.000	1.060	1.028	1.040	1.063	1.069
fi	1.058	1.021	1.017	1.033	1.023	1.060	0.000	1.020	1.042	1.011	1.058
fr	1.022	1.007	0.971	0.994	0.936	1.028	1.020	0.000	0.993	1.028	1.041
id	1.031	1.053	1.022	1.035	1.010	1.040	1.042	0.993	0.000	1.051	1.052
lv	1.059	1.011	1.029	1.040	1.024	1.063	1.011	1.028	1.051	0.000	1.068
zh	1.076	1.083	1.065	1.060	1.065	1.069	1.058	1.041	1.052	1.068	0.000
Table 4: Subspace angle overlap as evaluated by the pairwise mean principal angle between subspaces
UUAS greater than 2 points are labeled with an asterisk  $(^{*})$
C.2 Subspace Overlap
Table 4 presents the average principal angle between the subspaces parametrized by each language we evaluate. Table 5 contains the per-language Spearman correlation between the ordering given by (negative) subspace angle and structural probe transfer accuracy, reported both on UUAS and DSpr.
D Data Sources
We use the following UD corpora in our experiments: Arabic-PADT, Chinese-GSD, Czech-PDT, English-EWT, Finnish-TDT, French-GSD,
German-GSD, Indonesian-GSD, Latvian-LVTB, Persian-Seraji, and Spanish-Ancora.
E t-SNE reproducibility
Previous work (Wattenberg et al., 2016) has investigated issues in the interpretability of tSNE plots. Given the qualitative nature of our experiments, to avoid this confounder, we include multiple plots with various settings of the perplexity hyperparameter in Figure 11.
5576
Language	ar	cz	de	en	es	fa	fi	fr	id	lv	zh
Spearman Correl. (UUAS)	0.88	0.85	0.87	0.91	0.91	0.48	0.85	0.89	0.71	0.90	0.41
Spearman Correl. (DSpr.)	0.95	0.96	0.95	0.96	0.97	0.50	0.90	0.93	0.72	0.94	0.23
Table 5: The Spearman correlation between two orderings of all languages for each language  $i$ . The first ordering of languages is given by (negative) subspace angle between the  $B$  matrix of language  $i$  and that of all languages. The second ordering is given by the structural probe transfer accuracy from all languages (including  $i$ ) to  $i$ . This is repeated for each of the two structural probe evaluation metrics.
Figure 11: t-SNE visualization of head-dependent dependency pairs belonging to selected dependencies in English and French, projected into a syntactic subspace of Multilingual BERT, as learned on English syntax trees. Colors correspond to gold UD dependency type labels, as in Figure 1, varying the perplexity (PPL) t-SNE hyperparameter. From left to right, figures correspond to PPL 5, 10, 30, 50, spanning the range of PPL suggested by van der Maaten and Hinton (2008). Cross-lingual dependency label clusters are exhibited across all four figures.
5577