SlowGuess/NAACL_2021_OCR huggingface dataset:
Table 12: Automatic evaluation results on TURK dataset (Xu et al., 2015) that focuses on lexical paraphrasing.
Models	SARI	add	keep	del	FK	SLen	OLen	CR	%split	s-BL	%new	%eq
Complex (input)	20.5	0.0	61.5	0.0	13.4	22.4	22.6	1.0	0.8	100.0	0.0	100.0
Simple (reference)	46.3	20.0	51.0	67.9	9.1	14.8	18.9	0.87	24.2	46.2	20.5	0.6
Hybrid-NG	29.8	0.1	37.0	52.2	9.0	13.3	13.4	0.6	0.8	38.2	1.4	3.1
LSTM	36.1	2.4	51.8	54.2	10.0	17.8	17.8	0.84	0.8	59.9	10.8	14.8
Transformerbert	38.7	5.0	53.5	57.7	10.4	16.7	18.8	0.85	10.9	62.1	11.2	11.1
EditNTS	37.8	2.7	56.0	54.9	11.7	20.9	22.4	1.02	6.4	63.6	13.4	0.0
Our Model	39.7	5.3	55.1	58.8	9.2	14.7	19.8	0.9	33.7	63.1	14.0	8.9
Table 13: Automatic evaluation results on ASSET (Alva-Manchego et al., 2020) dataset that contains all the three simplification operations.
We use the complex-simple sentence pairs from WIKI-AUTO (Jiang et al., 2020), which contains 138,095 article pairs and 604k non-identical aligned and partially-aligned sentence pairs. To capture sentence splitting, we join the sentences in the simple article mapped to the same sentence in the complex article. Similar to Newsela, we remove the sentence pairs with high (>0.9) and low (<0.1) BLEU (Papineni et al., 2002) scores. For validation and testing purposes, we use the following
two corpora: (i) TURK corpus (Xu et al., 2015) for lexical paraphrasing and (ii) ASSET corpus (Alva-Manchego et al., 2020) for multiple rewrite operations. While the former corpus has 8 human-written references for 2000 validation and 359 test sentences, the latter corpus provides 10 references for the same sentences. We remove the validation and test sentences from the training corpus. Tables 12 and 13 show the results on TURK and ASSET respectively.