SlowGuess/ACL_2020_OCR huggingface dataset:
Joseph Weizenbaum et al. 1966. Eliza—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1):36-45.
Chen Xing, Yu Wu, Wei Wu, Yalou Huang, and Ming Zhou. 2018. Hierarchical recurrent attention network for response generation. In Thirty-Second AAAI Conference on Artificial Intelligence.
Kaichun Yao, Libo Zhang, Tiejian Luo, Lili Tao, and Yanjun Wu. 2018. Teaching machines to ask questions. In *IJCAI*, pages 4546-4552.
Yao Zhao, Xiaochuan Ni, Yuanyuan Ding, and Qifa Ke. 2018. Paragraph-level neural question generation with maxout pointer and gated self-attention networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3901-3910.
Qingyu Zhou, Nan Yang, Furu Wei, Chuanqi Tan, Hangbo Bao, and Ming Zhou. 2017. Neural question generation from text: A preliminary study. In National CCF Conference on Natural Language Processing and Chinese Computing, pages 662-671. Springer.
A Examples of Data Labeling
In Table 7, we use a typical example to show how we relabeled CoQA. As introduced in our paper, we first deleted questions that cannot be answered by certain span from the passage. In Table 7, we deleted QA-pairs in turn 15, 18, 19 since they are yes/no questions, turn 3, 16 since the answer "female" is not a span from the input passage, and turn 13 since its answer is scattered in the sentence "Some of his cats have orange fur, some have black fur, some are spotted and one is white".
After deleting questions that are not suitable for SQG, we replaced the remaining answers into certain spans from the input passage. As shown in Table 7, in most cases the original answers were already a certain span. We slightly modified answers in turn 2, 7 from "Eight", "Three" into "8", "3" respectively. Finally, we rewrote all remaining questions to make them coherent. During this process, we mainly deal with information omission and coreference. In our example, we added a word "feline" into questions in turn 14 since the question 13 was deleted.
We used the 200-dimensional pre-trained GloVe word embeddings  ${}^{3}$  as initial value of word embeddings. During the training process, these embeddings were further fine-tuned. The NLTK  ${}^{4}$  package was used for sentence splitting and word tokenization. In our model, we set  ${d}_{s},{d}_{r},{d}_{g}$  to 200,256 and 128. For the passage-info encoder, we used 16 heads in the multil-attention layer. For the answer-info encoder, we used 8 vanilla self-attention heads and additional 6 answer-aware heads for each answer. To construct the two graphs, we set  $\delta$  into 3 . In our dual-graph interaction,we set  $T$  into 4 .
To train our model, we used an Adam optimizer with momentum  $\beta_{1} = 0.9$ ,  $\beta_{2} = 0.99$  and  $\epsilon = 10^{-8}$  to minimize the loss function. We varied the learning rate throughout training, including a warm-up step and a decreasing step similar to the original Transformer. Besides, we applied dropout between 0.4 and 0.5 to prevent over-fitting. Our model was trained on two Nvidia RTX 2080Ti graphics cards.
Since we noticed that the available baseline codes used different scripts to compute BLEU,
ROUGE and METEOR, we used new scripts<sup>5</sup> to compute the evaluation metrics in this paper.
<sup>3</sup>https://nlp.stanford.edu/projects/glove/
4https://www.nltk.org/
5https://github.com/tylin/ coco-caption/tree/master/pycocoevalcap
Brendan loves cats. He owns 8 cats. He has 7 girl cats and only 1 boy cat. Brendan brushes the cats' hair every day. He makes sure to feed them every morning and evening and always checks to see if the cats have water. Sometimes he feeds them special treats because he loves them. Each cat gets 3 treats. He doesn't give them food like chips and cake and candy, because those foods aren't good for cats. He likes to play with the cats. The cats like to chase balls of paper that Brendan makes for them. Some of his cats have orange fur, some have black fur, some are spotted and one is white. The white cat is Brendan's favorite. She is the first cat he owned. Her name is Snowball. When he first got Snowball she was a kitten. His other cats are named Fluffy, Salem, Jackie, Cola, Snickers, Pumpkin and Whiskers.
turn	Original QA-Pairs	New QA-Pairs
1	What does he care for? (cats)	What does he care for? (cats)
2	How many does he have? (Eight)	How many does he have? (8)
3	Are there more males or females? (females)	Deleted
4	How many? (7 girl cats and only 1 boy cat)	How many males and females?
(7 girl cats and only 1 boy cat)
5	What is groomed? (cat's hair)	What is groomed? (cat's hair)
6	What do they get fed? (treats)	What do they get fed? (treats)
7	How many? (Three)	How many? (3)
8	Why (because he loves them)	Why (because he loves them)
9	What foods are avoided?
(chips and cake and candy)	What foods are avoided?
(chips and cake and candy)
10	Why? (because those foods aren't good for cats)	Why? (because those foods aren't good for cats)
11	What toys do they like? (balls of paper)	What toys do they like? (balls of paper)
12	Who creates them? (Brendan)	Who creates them? (Brendan)
13	What colors are the felines?
(orange, black, spotted, and white)	Deleted
14	Which is the most liked?
(The white cat)	Which is the most liked?
(The white cat)
15	Is this his original one? (yes)	Deleted
16	What is its gender? (female)	Deleted
17	What does he call it? (Snowball)	What does he call it? (Snowball)
18	Is there one called Binky? (No)	Deleted
19	How about Scruff? (No)	Deleted
Table 7: Example for data labeling.