SlowGuess/NAACL_2022_OCR huggingface dataset:
Approach	arXiv	PubMed
R-1	R-2	R-L	R-1	R-2	R-L
DANCER PEGASUS	45.01	17.6	40.56	46.34	19.97	42.42
BigBird-PEGASUS (large)	46.63	19.02	41.77	46.32	20.65	42.33
HAT-BART	46.68	19.07	42.17	48.36	21.43	37.00
LED (large)	46.63	19.62	41.83	-	-	-
PRIMER	47.6	20.8	42.6	-	-	-
T5.1.1 (large - 1k input)	39.79	14.02	36.23	42.18	16.60	38.96
T5.1.1 (large - 2k input)	42.84	16.62	39.01	45.51	19.55	42.10
T5.1.1 (large - 4k input)	44.51	18.20	40.62	47.90	22.08	44.36
T5.1.1 + PSG (large - 1k input)	38.53	13.61	35.08	43.34	17.55	40.10
T5.1.1 + PSG (large - 2k input)	42.85	16.50	38.99	46.51	20.37	43.00
T5.1.1 + PSG (large - 4k input)	45.86	18.40	41.62	48.94	22.92	45.4
LongT5 (base - 4k input)	44.87	18.54	40.97	47.77	22.58	44.38
LongT5 (large - 4k input)	45.64	18.6	41.51	48.38	23.32	44.93
LongT5 (large - 8k input)	46.61	19.67	42.44	49.81	24.3	46.26
LongT5 (large - 16k input)	48.28	21.63	44.11	49.98	24.69	46.46
LongT5 (xl - 4k input)	45.99	19.51	42.04	48.99	23.48	45.51
LongT5 (xl - 8k input)	47.44	20.84	43.34	50.04	24.45	46.42
LongT5 (xl - 16k input)	48.35	21.92	44.27	50.23	24.76	46.67
Approach	BigPatent	MultiNews
R-1	R-2	R-L	R-1	R-2	R-L
BigBird-PEGASUS (large)	60.64	42.46	50.01	-	-	-
TG-MultiSum	-	-	-	47.10	17.55	20.73
PRIMER	-	-	-	49.9	21.1	25.9
T5.1.1 (large - 1k input)	55.07	37.49	45.90	43.69	16.26	23.03
T5.1.1 (large - 2k input)	60.07	43.49	50.90	44.95	17.26	23.74
T5.1.1 (large - 4k input)	62.14	45.85	52.95	45.67	17.88	24.15
T5.1.1 + PSG (large - 1k input)	58.58	41.80	49.74	44.43	15.85	22.41
T5.1.1 + PSG (large - 2k input)	64.51	49.15	56.01	46.65	17.74	23.74
T5.1.1 + PSG (large - 4k input)	67.05	52.24	58.70	47.48	18.60	24.31
LongT5 (base - 4k input)	60.95	44.22	51.52	46.01	17.37	23.5
LongT5 (large - 4k input)	66.17	51.10	57.70	46.99	18.21	24.08
LongT5 (large - 8k input)	67.42	52.62	59.04	47.18	18.44	24.18
LongT5 (large - 16k input)	70.38	56.81	62.73	-	-	-
LongT5 (xl - 4k input)	75.82	64.64	69.54	48.15	19.30	24.76
LongT5 (xl - 8k input)	76.39	65.37	70.16	48.17	19.43	24.94
LongT5 (xl - 16k input)	76.87	66.06	70.76	-	-	-
Approach	MediaSum	CNN / Daily Mail
R-1	R-2	R-L	R-1	R-2	R-L
HAT-BART	-	-	-	44.48	21.31	41.52
BART (large)	35.09	18.05	31.44	-	-	-
T5.1.1 (large - 1k input)	30.68	14.88	27.88	42.60	20.41	40.03
T5.1.1 (large - 2k input)	32.83	16.75	29.79	42.55	20.25	39.99
T5.1.1 (large - 4k input)	34.37	18.09	31.12	42.27	19.93	39.72
T5.1.1 + PSG (large - 1k input)	32.02	16.15	28.89	42.62	20.46	40.02
T5.1.1 + PSG (large - 2k input)	34.04	17.87	30.77	42.69	20.40	40.06
T5.1.1 + PSG (large - 4k input)	36.11	19.48	32.67	43.41	20.99	40.77
LongT5 (base - 4k input)	35.09	18.35	31.87	42.15	20.11	39.6
LongT5 (large - 4k input)	35.54	19.04	32.20	42.49	20.51	40.18
LongT5 (xl - 4k input)	36.15	19.66	32.80	43.94	21.40	41.28
Table 8: Summarization results comparing T5, T5 with PEGASUS-style Principle Sentences Generation (PSG) pre-training, and LongT5 with best known approaches for the various datasets. All T5 scores are with standard T5.1.1 model. All LongT5 scores are with models using TGlobal attention. For each task, we scale up the input length depending on the statistics of the inputs, thus not all of the tasks were scaled to 16k. We do not include input length of other models because each model uses the input differently, and hence, direct comparison is not possible.