SlowGuess/NAACL_2022_OCR huggingface dataset:
PENN TREEBANK(Marcus et al., 1999) The Penn Treebank (PTB) project selected 2,499 stories from a three year Wall Street Journal (WSJ) collection of 98,732 stories for syntactic annotation. We only utilize the dependency structure annotations to form our intermediate dependency parsing task. There are 39,832 (~40k) sentences.
C.4 Semantic Role Labeling
ONTONOTES(Weischedel et al., 2017) The OntoNotes project is built on two resources, following the PENN TREEBANK(Marcus et al., 1999) for syntax and the PENN PROPBANK for predicateargument structure. We select 40k sentences with SRL annotations to form intermediate task.
D Low-resource Datasets Description
We set three Low-resource Learning benchmark for AMR parsing:
1. BOLT Using only the BOLT split of AMR data of AMR2.0 dataset. The training, validation and test data each has 1061, 133 and 133 amrs respectively.
2. LORELEI Using only the LORELEI split of AMR data of AMR3.0 dataset. The training, validation and test data each has 4441, 354 and 527 amrs respectively.
3. DFA Using only the DFA split of AMR data of AMR2.0 dataset. The training, validation and test data each has 6455, 210 and 229 amrs respectively.
Compared with the AMR2.0 dataset which has 36521 training samples, the number of training samples in BOLT, LORELEI, DFA are  $2.9\%$ ,  $12.2\%$  and  $17.7\%$  of the number of AMR2.0.
We tune the hyper-parameters on the SPRING baseline, and then adding the auxiliary data using just those hyper-parameters without any changing.
We use RAdam (Liu et al., 2019) as our optimizer, and the learning rate is  $3e^{-5}$ . Batch-size is set to 2048 tokens with 10 steps accumulation. The dropout rate is set to 0.3.
Parameter	Searching Space
Learning rate	1e-5, 3e-5, 5e-5, 1e-4
Batch-size	256, 512, 1024, 2048, 4096
Grad. accu.	10
Dropout	0.1, 0.2, 0.3
Table 7: Hyper-parameters searching space
Model	Extra Data	SMATCH	NoWSD	Wiki	Concept-related	Topology-related
Conc.	NER	Neg.	Unll.	Reen.	SRL
AMR 2.0	SPRING (w/ silver) (Bevilacqua et al., 2021)	200k	84.3	84.8	83.1	90.8	90.5	73.6	86.7	72.4	80.5
Ours (w/ Semantic Role Labeling)	40k	84.5	84.9	84.0	90.2	91.8	74.6	87.7	74.2	82.8
- w/ Arg. Reduction(AR)	40k	84.8	85.2	83.9	90.4	92.2	74.2	88.1	74.5	83.0
- w/ Reen. Restoration(RR)	40k	85.0	85.4	83.5	90.6	92.1	75.6	88.2	75.5	83.7
- w/ AR+RR	40k	85.1	85.6	83.6	90.4	91.4	75.7	88.2	75.0	83.5
Ours (w/ Dependency Parsing)	40k	84.4	84.9	82.9	90.1	90.5	73.5	87.8	74.3	82.9
- w/ Redundant Relation Removal (RRR)	40k	84.5	85.0	83.5	90.2	91.2	74.3	88.0	74.5	82.9
- w/ Lemmatization (Lemma)	40k	84.7	85.2	83.8	90.2	91.2	75.0	88.0	74.1	83.0
- w/ RRR + Lemma	40k	85.0	85.4	84.1	90.4	92.5	74.7	88.2	74.7	83.1
Table 8: Full scores of ablation on AMRization methods.
Figure 7: Illustration of how to compute sentence representation distance of different tasks. The sentences used for evaluate are never seen in the training of AMR Parsing and other auxiliary tasks. Cosine Similarity is computed the same way. We collect all sentences' distance of one encoder to draw the Gaussian distribution curve.