SlowGuess/NAACL_2021_OCR huggingface dataset:
The BART-based models converge between 0-10 epochs, depending on the learning rate, and take around 5-10 hours. All learning rates are selected from the following set:  $(1e - 5,2e - 5,5e - 5,1e - 4,2e - 4,5e - 4)$ . Min-max token output lengths were tuned from the following set: (32-128, 32-256, 50-128, 50-256, 64-128, 64-200, 64-256). Batch sizes of (2,4) were explored and limited by GPU memory. Input token lengths explored were (800, 1024). All hyperparameter searching was done on the dev set.
The BERT-based models converge between 0-10 epochs as well and take from around 1-3 hours to converge. Weighted average parameter  $\alpha$  from  $(0,0.1,0.2,0.25,0.5,0.6,0.7,0.75,0.8,0.9,1)$ .
Combinations of learning rate and batch size  $(1e - 4,2e - 4)\times (256,512,1024)$  were exhaustively searched. Optimal learning rate, batch size pair  $= (2e - 4,512)$
A.2 Non-author Human Expert Generated Slides
Table 6 reports mean ROUGE F-scores (standard deviation in bracket) for non-author generated slides in comparison to our keyword model. Paper 960 was annotated by three human experts in order to measure human performance similarity. Although human experts outperform all systems by a large margin in terms of readability, informativeness, and consistency (see Figure 3), it seems that our model is comparable to and sometimes surpasses human performance regarding finding different pieces of relevant information.
A.3 Human Evaluation Survey System
We designed and implemented a web-based survey system to support the human evaluation study, as presented in the Human Evaluate section in the main text. Figure 4 shows a screenshot of the survey. The original slide deck was displayed at the top, along with a link to the original paper. This is to make sure that participants have everything they need to understand the slide content. In total, 14 participants said they referred to the original papers a few times.
In each round, the participant was given one or more original slides with the same title as reference and was asked to evaluate the corresponding slides generated by the three models, as well as
Paper(s)	Generator	ROUGE-1	ROUGE-2	ROUGE-L
960	Humans	23.91 (2.97)	6.55 (0.79)	24.23 (2.03)
960	Human-best	28.10	7.66	27.10
960	BARTKeyword (ours)	29.48	8.16	26.12
All	Humans	26.41 (4.80)	8.66 (2.24)	24.68 (2.03)
All	BARTKeyword (ours)	27.75 (1.62)	8.30 (0.36)	24.69 (1.18)
Table 6: ROUGE F-scores for non-author generated slides for four papers in comparison to our D2S system.
Figure 4: A screenshot of the web survey.
those from non-authors when available. The model names were hidden from the participants and the order of the methods were also randomized across rounds to ensure each round is evaluated independently without bias.
During the evaluation process, the participant could use the three buttons below the slide image to flip through the deck or go back to the slide that was under inspection for the current round. That slide was also shown at the beginning of each round. Occasionally, multiple original slides had the same title and they all need to be inspected. The participant was notified about this situation via a flashing highlighted message on top of the original slide image, as shown in the screenshot.
The bottom section contains the two tasks that the participants needed to complete. The first one contains the rating tasks and the second one the ranking task. The participant can only go to the next round after all tasks were completed. Participants were told that the model numbers can change from round to round, and that mentions of tables or figures should be ignored.
A.4 Slides Generated From Different Systems
Table 7 shows a few examples of slide content generated from different systems under the same title. Colored text snippets indicate that the generated contents are similar to the original slide contents.
Table 7: Slides generated from different systems.