SlowGuess/ACL_2020_OCR huggingface dataset:
Table 9: F1 scores on 1-shot name entity recognition. CDT denotes collapsed dependency transfer. Scores below mid-line are from our models, which achieve the best performance. Ave. shows the averaged scores.
Model	5-shots Named Entity Recognition	Ave.
News	Wiki	Social	Mixed
Bi-LSMT	6.81 ±0.40	8.40 ±0.16	1.06 ±0.16	13.17 ±0.17	7.36 ±0.22
SimBERT	32.01 ±0.00	10.63 ±0.00	8.20 ±0.00	21.14 ±0.00	18.00 ±0.00
TransferBERT	15.36 ±2.81	3.62 ±0.57	11.08 ±0.57	35.49 ±7.60	16.39 ±2.89
MN	19.85 ±0.74	5.58 ±0.23	6.61 ±1.75	8.08 ±0.47	10.03 ±0.80
WPZ	4.09 ±0.16	3.19 ±0.13	0.86 ±0.23	0.93 ±0.14	2.27 ±0.17
WPZ+GloVe	16.94 ±0.10	5.33 ±0.07	5.53 ±0.12	3.54 ±0.03	7.83 ±0.08
WPZ+BERT	50.06 ±1.57	9.54 ±0.44	17.26 ±2.65	13.59 ±1.61	22.61 ±1.57
L-TapNet+CDT	45.35 ±2.67	11.65 ±2.34	23.30 ±2.80	20.95 ±2.81	25.31 ±2.65
Table 10: . F1 score results on 5-shots name entity recognition. Our methods achieve the best performance.
Model	1-shot	5-shots
Ours	22.19	24.12
- dependency transfer	-4.55	-4.83
- label semantic	-6.93	-1.46
Table 11: Ablation test over different components on NER task. Results are averaged F1-score of all domains.
1-shot and 5-shots Results for NER Table 9 and Table 10 respectively show the 1-shot and 5- shots name entity recognition results. Our best model outperforms all baseline in both settings.
The trend of results is consistent with slot-tagging results. But the overall score is much lower than slot-tagging results. This is because NER domains are from different datasets and the domain gap is much larger.
Our improvements on 5-shots is narrowed in margin. This is because NER domains have different genres and vocabulary. So compared to SNIPS, it is harder to transfer knowledge but benefits more to rely on domain-specific support examples. This trend is even more pronounced with more shots. In 5-shots setting, the strongest baseline WPZ benefits more from the increased shots because it only uses support set for prediction. But the benefit of more shots is weaker for our model because it uses more prior knowledge.
Ablation Analysis on NER We investigate effectiveness of collapsed dependency transfer and label semantic on the NER task. We perform ablations on two proposed components and observe
Figure 6: Impacts of projection space dimensionality.
performance drops on both 1-shot and 5-shots settings, which demonstrate the generalization ability of proposed two mechanism.
C Analysis of Projection Space Dimensionality
Fig 6 shows the performance on 1-shot Snips when using different projected-space dimensions in L-TapNet. As shown in the trend in the figure, the performance of the model becomes better as the dimension of the mapping space increases and gradually stabilizes. This shows the possibility of reducing the dimension without losing too much performance (Yoon et al., 2019).
D Slot Tagging Result with Standard Deviations
Table 12 and 13 show the complete results with standard deviations for slot tagging task.
Model	1-shot Slot Tagging	Ave.
We	Mu	Pl	Bo	Se	Re	Cr
Bi-LSMT	10.36±0.36	17.13±0.61	17.52±0.76	53.84±0.57	18.44±0.44	22.56±0.10	8.64±0.41	21.21±0.46
SimBERT	36.10±0.00	37.08±0.00	35.11±0.00	68.09±0.00	41.61±0.00	42.82±0.00	23.91±0.00	40.67±0.00
TransferBERT	55.82±2.75	38.01±1.74	45.65±2.02	31.63±5.32	21.96±3.98	41.79±3.81	38.53±7.42	39.06±3.86
MN	21.74±4.60	10.68±1.07	39.71±1.81	58.15±0.68	24.21±1.20	32.88±0.64	69.66±1.68	36.72±1.67
WPZ	4.53±0.18	7.43±0.31	14.43±0.73	39.15±1.10	11.69±0.16	7.78±0.38	10.09±0.74	13.59±0.51
WPZ+GloVe	17.92±0.05	22.37±0.11	19.90±0.08	42.61±0.08	22.30±0.03	22.79±0.05	16.75±0.08	23.52±0.07
WPZ+BERT	46.72±1.03	40.07±0.48	50.78±2.09	68.73±1.87	60.81±1.70	55.58±3.56	67.67±1.16	55.77±1.70
TapNet	51.12±5.36	40.65±2.83	48.41±2.27	77.50±1.09	49.77±1.36	54.79±2.32	61.39±2.41	54.80±2.52
TapNet+CDT	66.30±3.81	55.93±1.78	57.55±6.57	83.32±0.96	64.45±4.07	65.65±1.74	67.91±3.32	65.87±3.18
L-WPZ+CDT	71.23±6.00	47.38±4.18	59.57±5.55	81.98±2.08	69.83±1.94	66.52±2.72	62.84±0.58	65.62±3.29
L-TapNet+CDT	71.53±4.04	60.56±0.77	66.27±2.71	84.54±1.08	76.27±1.72	70.79±1.60	62.89±1.88	70.41±1.97
Table 12: 1-shot slot tagging results with standard deviations.
Model	5-shots Slot Tagging	Ave.
We	Mu	Pl	Bo	Se	Re	Cr
Bi-LSMT	25.17±0.42	39.80±0.52	46.13±0.42	74.60±0.21	53.47±0.45	40.35±0.52	25.10±0.94	43.52±0.50
SimBERT	53.46±0.00	54.13±0.00	42.81±0.00	75.54±0.00	57.10±0.00	55.30±0.00	32.38±0.00	52.96±0.00
TransferBERT	59.41±0.30	42.00±2.83	46.07±4.32	20.74±3.36	28.20±0.29	67.75±1.28	58.61±3.67	46.11±2.29
MN	36.67±3.64	33.67±6.12	52.60±2.84	69.09±2.36	38.42±4.06	33.28±2.99	72.10±1.48	47.98±3.36
WPZ	9.54±0.19	14.23±0.19	18.12±1.41	44.65±2.58	18.98±0.58	12.03±0.58	14.05±0.63	18.80±0.88
WPZ+GloVe	26.61±0.54	34.25±0.16	22.11±0.04	50.55±0.15	28.53±0.05	34.16±0.43	23.69±0.07	31.41±0.21
WPZ+BERT	67.82±4.11	55.99±2.24	46.02±3.19	72.17±1.75	73.59±1.60	60.18±6.96	66.89±2.88	63.24±3.25
TapNet	53.03±7.20	49.80±3.02	54.90±2.72	83.36±1.03	63.07±1.96	59.84±1.57	67.02±2.51	61.57±2.86
TapNet+CDT	66.48±4.09	66.36±1.77	68.23±3.99	85.76±1.65	73.60±1.09	64.20±4.99	68.47±1.93	70.44±2.79
L-WPZ+CDT	74.68±2.43	56.73±3.23	52.20±3.22	78.79±2.11	80.61±2.27	69.59±2.78	67.46±1.91	68.58±2.56
L-TapNet+CDT	71.64±3.62	67.16±2.97	75.88±1.51	84.38±2.81	82.58±2.12	70.05±1.61	73.41±2.61	75.01±2.46
Table 13: 5-shot slot tagging results with standard deviations.