SlowGuess/NAACL_2021_OCR huggingface dataset:
Lu Wang and Wang Ling. 2016. Neural network-based abstract generation for opinions and arguments. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 47-57.
Genta Indra Winata, Samuel Cahyawijaya, Zihan Liu, Zhaojiang Lin, Andrea Madotto, Peng Xu, and Pascale Fung. 2020. Learning fast adaptation on cross-accented speech recognition. arXiv preprint arXiv:2003.01901.
Yu Yan, Weizhen Qi, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, and Ming Zhou. 2020. Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training. arXiv preprint arXiv:2001.04063.
Michihiro Yasunaga, Jungo Kasai, Rui Zhang, Alexander R Fabbri, Irene Li, Dan Friedman, and Dragomir R Radev. 2019. Scisummnet: A large annotated corpus and content-impact models for scientific paper summarization with citation networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7386-7393.
Wenpeng Yin. 2020. Meta-learning for few-shot natural language processing: A survey. arXiv preprint arXiv:2007.09604.
Tiezheng Yu, Dan Su, Wenliang Dai, and Pascale Fung. 2020. Dimsum@ laysumm 20: Bart-based approach for scientific document summarization. arXiv preprint arXiv:2010.09252.
Rui Zhang and Joel Tetreault. 2019. This email could save your life: Introducing the task of email subject line generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 446-456.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204-2213.
A Full Results of All Models
The full results of all models are shown in Table 8.
Our model contains  $\sim 139.4$  million parameters and we train all models on one GTX 1080 Ti. We train all the models for 50 epochs in around three hours. We manually tune the hyperparameter values.
Domains	ROUGE Scores	BART Fine-tuning	SDPT	SDPT w/ RecAdam	DAPT	DAPT w/ RecAdam	TAPT	TAPT w/ RecAdam
Dialog	ROUGE-1 F1	39.95	42.84	45.23	41.22	40.05	40.15	41.34
ROUGE-2 F1	17.50	17.51	19.43	17.88	17.62	16.99	17.88
ROUGE-L F1	31.64	33.79	35.37	32.40	31.36	31.21	32.31
Email	ROUGE-1 F1	24.71	25.16	26.97	26.50	25.66	25.30	25.73
ROUGE-2 F1	11.71	12.2	13.44	13.14	12.89	12.03	12.69
ROUGE-L F1	24.15	24.28	25.98	25.61	25.14	24.63	25.32
Movie R.	ROUGE-1 F1	25.13	25.45	26.06	24.25	25.78	25.27	25.65
ROUGE-2 F1	9.22	9.49	10.27	9.06	9.84	9.24	9.13
ROUGE-L F1	20.04	20.11	20.91	19.56	20.69	20.09	20.45
Debate	ROUGE-1 F1	24.48	25.61	25.17	26.71	25.01	24.59	24.70
ROUGE-2 F1	8.21	8.48	8.38	9.14	8.42	8.13	8.43
ROUGE-L F1	21.96	22.86	22.39	23.64	22.17	22.04	22.25
Social M.	ROUGE-1 F1	21.76	22.43	23.25	22.95	21.51	22.81	23.01
ROUGE-2 F1	8.11	9.06	9.01	9.66	8.25	8.96	8.49
ROUGE-L F1	21.03	21.03	22.18	21.93	20.69	22.06	21.95
Science	ROUGE-1 F1	72.76	73.09	72.60	71.88	72.23	73.08	72.80
ROUGE-2 F1	64.66	65.15	63.79	63.73	63.32	65.04	64.26
ROUGE-L F1	68.40	68.62	68.06	67.34	67.62	68.81	68.41
Table 8: Full results of all models.