SlowGuess/NAACL_2022_OCR huggingface dataset:
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. ArXiv, abs/1511.06709.
Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive learning rates with sublinear memory cost. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 4596-4604. PMLR.
Connor Shorten and Taghi M. Khoshgoftaar. 2019. A survey on Image Data Augmentation for Deep Learning. Journal of Big Data, 6(1):60.
Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426.
Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V Le. 2019. Unsupervised data augmentation for consistency training. arXiv preprint arXiv:1904.12848.
Ziang Xie, Sida I. Wang, Jiwei Li, Daniel Lévy, Allen Nie, Dan Jurafsky, and A. Ng. 2017. Data noising as smoothing in neural network language models. ArXiv, abs/1703.02573.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2019. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization.
Rongsheng Zhang, Yinhe Zheng, Jianzhi Shao, XiaoXi Mao, Yadong Xi, and Minlie Huang. 2020a. Dialogue distillation: Open-domain dialogue augmentation using unpaired data. ArXiv, abs/2009.09427.
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and William B. Dolan. 2020b. Dialogpt: Large-scale generative pre-training for conversational response generation. In ACL.
A Summary FillUp Model
Summary FillUp is finetuned from PEGASUSLarge public checkpoint. The model had  $\mathrm{L} = 16$ ,  $\mathrm{H} = 1024$ ,  $\mathrm{F} = 4096$ ,  $\mathrm{A} = 16$  (568M parameters), where L denotes the number of layers for encoder and decoder Transformer blocks, H for the hidden size, F for the feed-forward layer size and A for the number of self-attention heads. All finetuning experiments are done with a batch size of 8. For optimization, we used Adafactor (Shazeer and Stern, 2018) with square root learning rate decay with a learning rate of 0.0001 and a dropout rate of 0.01. The model was decoded with a beam size of 8 and a length penalty of 0.6.
B Back-translation
For back-translation, we adopted Xie et al. (2019)'s back-translation implementation to increase diversity. As reported by the authors, the models used were trained in WMT'14 English-French (in both directions). The authors used the hyperparameter sampling_temp to control the diversity and quality of the back-translation. We found that setting it to 0.5 yielded the best augmented examples.
C LDA model
Mallet LDA models were trained with all the 14,732 human-annotated summaries in the SAM-Sum train split. We varied the number of topics from 2 to 340, with a step of 2, and selected models with 100, 200, and 300 topics. The corresponding coherence scores are 0.524, 0.587, and 0.614. Given summaries generated by models trained with DADS and two baselines, the average topic distribution entropy values calculated from the three LDA models are shown in Table 5. DADS has the lowest average entropy in all three settings.
Model	t=100	t=200	t=300
Baseline	6.598	7.583	8.163
Back-trans.	6.604	7.592	8.172
DADS	6.597	7.583	8.162
Table 5: Average entropy values for Baseline, Backtranslation and DADS calculated from three LDA models with number of topics  $\mathrm{t} = {100},{200}$  ,and 300 .
D Entailment Classifier
Given summary and dialogue, the entailment classifier outputs the probability of the summary entailing the dialogue. We finetuned a transformer-based
model, initialized with a pretrained BERT-Large checkpoint (Devlin et al., 2018), on the Multi-NLI dataset (Williams et al., 2017).
E Faithfulness Assessment
We ran a small annotation task with three raters, all proficient in English and NLP researchers, who were asked to read the dialogue carefully and then grade the accompanying summary on a scale of 1-4 (fully unfaithful, somewhat unfaithful, somewhat faithful, and fully faithful). A summary is "fully faithful" if all of its content is fully supported or can be inferred from the document.
Gold	Emma was late and missed Andy's song, but she still had fun.
Dialogue	Emma: Hey it was fun right?
George: Yes, certainly.... but why you came so late. you missed andy's song.
Emma: I know : (but still i had a lot of fun.
George: yes.. will plan again
Emma: yes pleaseeeee
No Aug.
R1/R2/RL	George will plan again for Emma.
16.2 / 9.8 / 16.2
Back Trans.
R1/R2/RL	George will come to Emma's place again.
10.3 / 0.0 / 10.3
DADS
R1/R2/RL	Emma came late but still had a lot of fun. George will plan again.
52.2 / 24.0 / 47.8
Gold	Robert wants Fred to send him the address of the music shop as he needs to buy guitar cable.
Dialogue	Robert: Hey give me the address of this music shop you mentioned before
Robert: I have to buy guitar cable
Fred: < file_other >
Fred: Catch it on google maps
Robert: thx m8
Fred: ur welcome
No Aug.
R1/R2/RL	Robert has to buy guitar cable and Fred has to Catch it on google maps.
40.9 / 29.8 / 40.9
Back Trans.
R1/R2/RL	Robert and Fred will meet on google maps.
15.4 / 9.8 / 15.4
DADS
R1/R2/RL	Robert wants Fred to give him the address of this music shop.
37.2 / 22.2 / 32.6
Gold	Heidi wants Noah to take items away from the balcony and close all the windows.
Dialogue	Heidi: Could you take the things away from the balcony? I forgot about them and it's going to rain today.
Noah: I'll do it as soon as I am back home.
Heidi: And close all the windows in case of a storm.
Noah: of course
No Aug.
R1/R2/RL	Noah will take the things away from Heidi's balcony.
21.3 / 15.4 / 21.3
Back Trans.
R1/R2/RL	Noah will take the things away from Heidi.
21.7 / 15.7 / 21.7
DADS
R1/R2/RL	Noah will take the things away from the balcony as soon as he is back home.
34.6 / 27.1 / 34.6
Table 6: Dialogue summarization examples: the dialogue, its gold summary, and the model generated summaries. We also present the [ROUGE-1, ROUGE-2, ROUGE-L] F1 scores relative to the reference dialogue. The models are trained using 50 annotated examples in SAMSum, with No Augmentation (No Aug.), augmented by Back Translation (Back Trans.), and DADS.