SlowGuess/ACL_2020_OCR huggingface dataset:
Table 15: Pairwise BiLSTM and Transformer configurations (with varying embedding dimension, hidden states and depth) vs. latency and  $F_{1}$  scores for distillation strategy  $D0 - S$ .
Embedding Dimension	BiLSTM States	F1-score	Std. Dev.	Params (MM)	Params (Compression)	Speedup (bsz=32)	Speedup (bsz=1)
300	600	88.64	3.8	31.8	5.6	14	8
200	600	88.5	3.8	22.5	8	15	9
300	400	88.21	4	29.4	6.1	23	11
200	400	88.16	3.9	20.2	8.9	25	12
100	600	87.93	4.1	13.1	13.7	16	9
100	400	87.7	4	11.1	16.1	24	13
50	600	87.67	4	8.5	21.1	16	10
300	200	87.54	4.1	27.7	6.5	40	15
200	200	87.47	4.2	18.7	9.6	46	16
50	400	87.19	4.3	6.5	27.5	27	13
100	200	86.89	4.2	9.6	18.6	49	15
50	200	86.46	4.3	5.1	35.1	51	16
300	100	86.19	4.3	27	6.6	62	16
200	100	85.88	4.4	18.1	9.9	68	17
100	100	85.64	4.5	9.2	19.5	74	15
50	100	84.6	4.7	4.7	38.1	77	16
Table 16: Parameter compression and inference speedup vs.  $F_{1}$ -score with varying embedding dimension and BiLSTM hidden states. Online inference is in Intel(R) Xeon(R) CPU (E5-2690 v4 @2.60GHz) and batch inference is in a single P100 GPU for distillation strategy  $D4$ .
Lang	#Train	Ours	BERT	MBERT	MMNER
af	5	87	89	91	84
hi	5	84	85	88	85
sq	5	91	93	93	88
bn	10	91	83	95	95
lt	10	87	89	90	86
lv	10	90	92	93	91
mk	10	92	93	94	91
tl	10	94	88	95	93
bs	15	91	93	93	92
et	15	89	92	91	90
sl	15	92	93	94	92
ta	15	77	82	84	84
ar	20	85	88	89	88
bg	20	90	93	93	90
ca	20	91	94	93	91
cs	20	91	92	93	90
da	20	91	93	93	90
de	20	84	89	89	86
el	20	86	90	90	89
en	20	78	83	84	81
es	20	90	92	93	90
fa	20	90	92	93	93
fi	20	89	91	92	89
fr	20	87	91	91	88
he	20	79	85	85	85
hr	20	90	92	93	89
hu	20	90	93	93	90
id	20	92	92	93	91
it	20	88	93	92	89
ms	20	90	92	93	91
nl	20	89	93	92	89
no	20	91	93	93	90
pl	20	88	91	92	89
pt	20	89	92	93	90
ro	20	93	94	94	92
ru	20	85	88	90	86
sk	20	92	93	94	91
sv	20	94	95	95	93
tr	20	90	92	93	90
uk	20	88	92	93	89
vi	20	89	91	92	88
Table 17:  $F_{1}$ -scores of different models per language. BERT represents MBERT fine-tuned separately for each language. Other models including XtremeDistil (ours) is jointly fine-tuned over all languages.