SlowGuess/NAACL_2022_OCR huggingface dataset:
Evaluation Metrics We compare the model performance along three automatic evaluation metrics: BLEU (Papineni et al., 2002), which is computed with the geometric average of the modified n-gram precision and the brevity penalty; Meteor (Denkowski and Lavie, 2014), which compares the generation with the gold question in terms of exact, stem, synonym, and paraphrase matches; and Rouge-L (Lin, 2004), which measures the shared longest common sub-sequence. We calculate these metrics with the package released by Du et al. (2017). We also conduct a human evaluation. As a sanity check and to evaluate the QG
$^{2}$ https://github.com.microsoft/unilm/tree/master/s2s-ft
Method	RACE	SciQ
B-1	B-4	MT	RG	B-1	B-4	MT	RG
None	21.99	4.11	13.68	21.31	25.94	8.67	15.53	26.59
DDS	random	21.91	4.02	13.74	21.26	26.15	8.97	15.56	26.62
LDA	21.97	4.29	13.72	21.47	26.57	8.88	15.67	27.07
BERT-DDS	22.06	3.99	13.61	21.30	26.43	9.08	15.70	26.70
KMeans	22.21	4.45	13.75	21.65	26.45	9.23*	15.72	27.15*
GMM	22.38	4.58	14.05	21.70	26.51	9.08	15.79*	27.05
AA-KMeans	22.28	4.40	13.92	21.71	26.26	8.85	15.66	26.82
AA-GMM	22.79*	4.79*	14.23*	22.15*	26.61*	9.09	15.73	26.90
ST	w/o-Norm	23.34	4.82	14.45	22.89	27.89	10.37	16.51	28.26
w/o-Filter	23.83	5.13	14.65	23.06	28.29	10.85	16.95	28.86
Fluency	24.20	5.11	14.74	23.66	28.22	10.76	16.92	28.92
PPL	24.38	5.22	14.85	23.43	28.30	11.04	17.12	29.03
Fluency&PPL	24.32	5.14	14.73	23.52	28.30	11.04	17.12	29.03
DDS+ST	w/o-Filter	23.43	4.93	14.43	22.78	28.21	11.00	16.90	28.93
Fluency	24.20	4.85	14.67	23.13	28.82	11.05	16.86	28.94
PPL	24.43	5.40*	15.08	23.49	29.12	11.04	16.92	29.38
Fluency&PPL	24.71	5.20	14.96	23.78	29.40*	11.23	17.13	29.52*
AA-Fluency	24.14	5.17	14.79	23.07	28.10	10.82	16.69	28.54
AA-PPL	24.50	5.14	15.09*	23.60	28.84	11.65	17.22*	29.30
AA-Fluency&PPL	24.71*	5.16	14.87	23.80*	28.68	11.70*	17.17	29.36
Table 4: Results of unsupervised domain adaptation for QG with answer-type aware (AA-) domain data selection(DDS) and self-training(ST) on RACE and SciQ test set. We compare three baseline methods: LDA (Druck et al., 2008), BERT-DDS (Ma et al., 2019), GMM (Aharoni and Goldberg, 2020).  $\star$  denotes the best results for DDS,  $\heartsuit$  denotes best results for ST, and  $\clubsuit$  denotes best results for DDS+ST.
Dataset	B-1	B-4	MT	RG
NQ	60.05	30.31	29.64	59.26
SciQ	46.99	33.22	29.47	42.73
RACE	37.86	17.90	23.91	37.56
Table 5: In-domain test results of the QG model (finetuned and tested on the same dataset).
model's ability to generate questions based on these datasets, we first conduct in-domain tests on these three datasets separately, i.e. we fine-tune and test the model on the training/test set from the same dataset. As shown in Table 5, we achieve results comparable with state-of-art for the NQ, RACE and SciQ datasets.
7.2 Experiments on Data Selection
In this experiment, we compare the proposed answer-type aware data selection with several baselines. We train the QG model with the selected data and evaluate the data selection method by comparing its performance. The first baseline is random data selection (random). With this baseline, we randomly sample 1,000 samples from NQ. The second baseline is LDA-based clustering (Druck et al., 2008). We use the gensim (Rehurek and Sojka, 2010) LDA implementation for this baseline.
The third method (BERT-DDS) is proposed by Ma et al. (2019), where a BERT-based domain discriminator is used for data selection. The discriminator is first trained with randomly sampled data from the datasets. The baseline model achieved  $99.85\%$  for RACE and  $92.35\%$  accuracy for the SciQ dataset. The last baseline method we compare is adopted from the unsupervised domain clustering method (GMM) proposed by Aharoni and Goldberg (2020), as described in Section 5. We use the BERT-base model implementation of huggingface transformers (Wolf et al., 2020) to get the context passage encoding. In addition to GMM, we also compare the K-Means method (Sculley, 2010). The results are presented in Table 4.
Impact of Domain Data Selection Re-training with randomly selected data does not improve our model's generalization performance. All other data selection methods outperform random data selection, except BERT-DDS. One reason is that BERT-DDS training needs sampling data from different domains, its performance relies on the sampled data, and also label examples that are similar to the target domain as source domain. Data selection with unsupervised domain clustering with BERT context encoding outperforms other methods,
which confirms its effectiveness.
On the RACE dataset, answer-type aware data selection with K-Means (AA-KMeans) and GMM (AA-GMM) outperform the same selection method without answer-type awareness. We note this result does not always hold for the SciQ dataset. One possible reason is due to the extremely unbalanced answer type distribution in SciQ: we have to select examples with generally low domain similarities wrt. the source domain to create the same answer-type distributions.
7.3 Experiments on Self-Training
We conduct self-training with the target-domain unlabeled data on the QG model fine-tuned on the NQ dataset. We first verify the effectiveness of the proposed normalized training objective. As the results show in Table 4, self-training with normalization (w/o-Filter) outperforms self-training without any confidence filtering and normalization (w/o-Norm), which indicates its effectiveness.
Impact of Generation Confidence Guided Self-training We explore two generation confidence metrics for self-training, the sentence perplexity, and the question fluency score. To train the BERT LM for generating fluency scores for question quality evaluation, we combine all questions from NQ and the Quora Question Pairs dataset $^3$ , creating a dataset consisting of 834,834 questions. The final model achieves a perplexity of 9.27 on the evaluation set. As the results in the ST part of Table 4 show, both proposed generation confidence metrics improve the performance considerably up to  $6\%$ . This can be explained by the removal of low-quality and noisy data, which hinders model training. As
(a)
Figure 4: Change of (a) average perplexity, and (b) standard deviation of generations along iterations.
Figure 4 shows, with perplexity filtering—although the changing curves of mean perplexity of the generated pseudo-labels in each iteration are similar—the standard deviation drops faster and more steady.
3https://www.kaggle.com/c/ quora-question-pairs
As Figure 5 shows, the average fluency score im
(a)
(b)
Figure 5: Change of (a) average fluency score, and (b) the percentage of generated questions whose fluency score is higher than  $\phi$  along iterations.
proves along iterations even without fluency filtering, but with fluency filtering, the average fluency score improves more steadily and increases towards the threshold value  $\phi$ . The proportion of questions with higher fluency score than  $\phi$  increase along iterations. As reflected in Figure 5b and Table 6, if the threshold value is too low, fewer noisy pseudo examples can be filtered out. If the threshold is too high, there would be less supervision for the QG model. Both of these settings would lead to performance degradation.
φ	B-1	B-4	MT	RG
RACE	8.5	23.83	5.12	14.65	23.06
9.5	24.20	5.11	14.74	23.66
10.5	24.23	5.06	14.68	23.29
11.5	23.93	5.10	14.46	23.09
12.5	23.78	4.55	14.41	23.05
Table 6: Influence of the fluency threshold  $(\phi)$
Impact of Joining Domain Data Selection and Self-Training We also conduct domain adaptation by joining domain data selection and self-training (DDS+ST). As shown in Table 4, joining DDS and self-training without filtering does not show performance improvement on both datasets, which implies with DDS, pseudo-labels during self-training may be noisier. With the proposed filtering with fluency score or question perplexity, the joint method outperforms DDS and self-training. On the RACE dataset, the answer-type aware joint methods generally achieves the best performance across all evaluation metrics.
In addition to the automatic evaluation results shown in Table 4, we also report on our human evaluation in Table 7. We randomly sampled 50 generated questions from the RACE and SciQ test
Method	RACE	SciQ
Syntax	Relevance	Answerability	Syntax	Relevance	Answerability
w/o-UDA	2.60 (0.66)	2.00 (0.78)	0.43 (0.49)	2.83 (0.40)	2.40 (0.64)	0.57 (0.50)
ST	2.78 (0.51)	2.12 (0.73)	0.46 (0.50)	2.94 (0.26)	2.49 (0.64)	0.67 (0.47)
DDS+ST	2.81 (0.47)	2.12 (0.75)	0.51 (0.50)	2.92 (0.27)	2.53 (0.63)	0.67 (0.47)
Table 7: Human evaluation (mean and standard deviation) on RACE and SciQ datasets. Syntax and Relevance evaluation adopt a 3-point scale. Higher is better; Answerability is boolean type (0-1).
set respectively and asked 3 domain experts (both male and female, ages ranging from 25 to 35) to rate the generated questions by the QG model without UDA (w/o-UDA), with self-training (ST), and self-training and domain data selection  $(\mathrm{DDS} + \mathrm{ST})$ . The experts are also presented with the context paragraphs, the answers, as shown in Figure 6 of the appendix. The generated questions are shown in Table 10 of the appendix. We rate questions along three dimensions: (i) syntax, i.e. the syntax correctness, in a 3-point scale, 1 for major syntax issues, 2 meaning minor issue and 3 is correct; (ii) relevance, i.e. whether the question is relevant to the context and the answer, also in a 3-point scale, 1 irrelevance, 2 for partial relevance and 3 meaning fully relevant; (iii) answerability, a boolean type value, indicating whether the question can be answered given the context and answer. As the results show, all QG with UDA methods outperform the QG model without domain adaptation. On the RACE dataset, the proposed unsupervised domain adaptation for QG with data selection and self-training  $(\mathrm{DDS} + \mathrm{ST})$  achieves the best performance along with all metrics; although the performance of UDA with self-training only outperforms  $\mathrm{DDS} + \mathrm{ST}$  slightly in terms of syntax and answerability,  $\mathrm{DDS} + \mathrm{ST}$  outperforms self-training.
We proposed an unsupervised domain adaptation approach for question generation. Our approach includes an answer-type aware unsupervised domain data selection method and a sequence generation confidence guided self-training algorithm. We conduct experiments on three domains. We use the Natural Questions dataset as labeled source domain, RACE as target education domain and SciQ as target science domain. Our results suggest our approach is effective for this application settings. We find that it significantly improves domain adaptation performance of our QG model. In future work, we plan to expand our work to more domains
and additional QG model types.
The first author has been supported by the China Scholarships Council (CSC). The second author has been supported by NWO project SearchX (639.022.722) and NWO Aspasia (015.013.027). We also would like to thank the anonymous ACL Rolling Review reviewers who kindly provided us with plenty of helpful comments for improving this paper and the experts who helped us with the human evaluation.
Steven Abney. 2007. Semisupervised learning for computational linguistics. CRC Press.
Roee Aharoni and Yoav Goldberg. 2020. Unsupervised domain clusters in pretrained language models. arXiv preprint arXiv:2004.02105.
Amittai Axelrod, Xiaodong He, and Jianfeng Gao. 2011. Domain adaptation via pseudo in-domain data selection. In Proceedings of the 2011 conference on empirical methods in natural language processing, pages 355-362.
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Jianfeng Gao, Songhao Piao, Ming Zhou, et al. 2020. Unilmv2: Pseudomasked language models for unified language model pre-training. In International Conference on Machine Learning, pages 642-652. PMLR.
Himanshu Sharad Bhatt, Deepali Semwal, and Shourya Roy. 2015. An iterative similarity based adaptation technique for cross-domain text classification. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning, pages 52-61.
Denny Britz, Quoc Le, and Reid Pryzant. 2017. Effective domain mixing for neural machine translation. In Proceedings of the Second Conference on Machine Translation, pages 118-126.
Yu Cao, Meng Fang, Baosheng Yu, and Joey Tianyi Zhou. 2020. Unsupervised domain adaptation on reading comprehension. In Proceedings of the AAAI
Conference on Artificial Intelligence, volume 34, pages 7480-7487.
Rita Chattopadhyay, Qian Sun, Wei Fan, Ian Davidson, Sethuraman Panchanathan, and Jieping Ye. 2012. Multisource domain adaptation and its application to early detection of fatigue. ACM Transactions on Knowledge Discovery from Data (TKDD), 6(4):1-26.
Xia Cui and Danushka Bollegala. 2019. Self-adaptation for unsupervised domain adaptation. Proceedings-Natural Language Processing in a Deep Learning World.
Anna Currey, Antonio Valerio Miceli-Barone, and Kenneth Heafield. 2017. Copied monolingual data improves low-resource neural machine translation. In Proceedings of the Second Conference on Machine Translation, pages 148-156.
Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the ninth workshop on statistical machine translation, pages 376-380.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Advances in Neural Information Processing Systems, pages 13063-13075.
Gregory Druck, Gideon Mann, and Andrew McCallum. 2008. Learning from labeled features using generalized expectation criteria. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pages 595-602.
Xinya Du, Junru Shao, and Claire Cardie. 2017. Learning to ask: Neural question generation for reading comprehension. arXiv preprint arXiv:1705.00106.
Alexander R Fabbri, Patrick Ng, Zhiguo Wang, Ramesh Nallapati, and Bing Xiang. 2020. Template-based question generation from retrieved sentences for improved unsupervised question answering. arXiv preprint arXiv:2004.11892.
Arthur Gretton, Karsten Borgwardt, Malte Rasch, Bernhard Scholkopf, and Alex Smola. 2006. A kernel method for the two-sample-problem. Advances in neural information processing systems, 19:513-520.
Arthur Gretton, Karsten Borgwardt, Malte J Rasch, Bernhard Scholkopf, and Alexander J Smola. 2008. A kernel method for the two-sample problem. arXiv preprint arXiv:0805.2368.
Jing Gu, Mostafa Mirshekari, Zhou Yu, and Aaron Sisto. 2021. Chaincqg: Flow-aware conversational question generation. arXiv preprint arXiv:2102.02864.
Han Guo, Ramakanth Pasunuru, and Mohit Bansal. 2020. Multi-source domain adaptation for text classification via distancenet-bandits. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7830-7838.
Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. 2020. Don't stop pretraining: adapt language models to domains and tasks. arXiv preprint arXiv:2004.10964.
Junjie Hu, Mengzhou Xia, Graham Neubig, and Jaime Carbonell. 2019. Domain adaptation of neural machine translation by lexicon induction. arXiv preprint arXiv:1906.00376.
Xin Jia, Wenjie Zhou, Xu Sun, and Yunfang Wu. 2020. Eqg-race: Examination-type question generation. arXiv preprint arXiv:2012.06106.
Devang Kulshreshtha, Robert Belfer, Iulian Vlad Serban, and Siva Reddy. 2021. Back-training excels self-training at unsupervised domain adaptation of question generation and passage retrieval. arXiv preprint arXiv:2104.08801.
Ghader Kurdi, Jared Leo, Biban Parsia, Uli Sattler, and Salam Al-Emari. 2020. A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education, 30(1):121-204.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453-466.
Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. Race: Large-scale reading comprehension dataset from examinations. arXiv preprint arXiv:1704.04683.
Dong-Hyun Lee et al. 2013. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, page 896.
Lillian Lee. 2001. On the effectiveness of the skew divergence for statistical language analysis. In International Workshop on Artificial Intelligence and Statistics, pages 176-183. PMLR.
Yin-Hsiang Liao and Jia-Ling Koh. 2020. Question generation through transfer learning. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pages 3-17. Springer.
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Miaofeng Liu, Yan Song, Hongbin Zou, and Tong Zhang. 2019a. Reinforced training data selection for domain adaptation. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1957-1968.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Lin Ma and Yuchun Ma. 2019. Automatic question generation based on mooc video subtitles and knowledge graph. In Proceedings of the 2019 7th International Conference on Information and Education Technology, pages 49-53.
Xiaofei Ma, Peng Xu, Zhiguo Wang, Ramesh Nallapati, and Bing Xiang. 2019. Domain adaptation with bert-based domain classification and data selection. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 76-83.
David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pages 152-159.
Rada Mihalcea. 2004. Co-training and self-training for word sense disambiguation. In Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004, pages 33-40.
Jan Niehues and Ngoc-Quan Pham. 2019. Modeling confidence in sequence-to-sequence models. arXiv preprint arXiv:1910.01859.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311-318.
Alan Ramponi and Barbara Plank. 2020. Neural unsupervised domain adaptation in nlp—a survey. arXiv preprint arXiv:2006.00632.
Sadaf Abdul Rauf, Kiran Kiani, Ammara Zafar, Raheel Nawaz, et al. 2019. Exploring transfer learning and domain data selection for the biomedical translation. In Proceedings of the Fourth Conference on Machine Translation (Volume 3: Shared Task Papers, Day 2), pages 156-163.
Radim Šehúrek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New
Challenges for NLP Frameworks, pages 45-50, Valletta, Malta. ELRA. http://is.muni.cz/publication/884893/en.
Steven Rennie, Etienne Marcheret, Neil Mallinar, David Nahamoo, and Vaibhava Goel. 2020. Unsupervised adaptation of question answering systems via generative self-training. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1148-1157.
Mrinmaya Sachan and Eric Xing. 2018. Self-training for jointly learning to ask and answer questions. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 629-640.
David Sculley. 2010. Web-scale k-means clustering. In Proceedings of the 19th international conference on World wide web, pages 1177-1178.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709.
Md Arafat Sultan, Shubham Chandel, Ramón Fernández Astudillo, and Vittorio Castelli. 2020. On the importance of diversity in question generation for qa. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5651-5656.
Chuanqi Tan, Fuchun Sun, Tao Kong, Wenchang Zhang, Chao Yang, and Chunfang Liu. 2018. A survey on deep transfer learning. In International conference on artificial neural networks, pages 270-279. Springer.
Marlies Van Der Wees, Arianna Bisazza, and Christof Monz. 2017. Dynamic data selection for neural machine translation. arXiv preprint arXiv:1708.00712.
Yansen Wang, Chenyi Liu, Minlie Huang, and Liqiang Nie. 2018. Learning to ask questions in open-domain conversational systems with typed decoders. arXiv preprint arXiv:1805.04843.
Johannes Welbl, Nelson F Liu, and Matt Gardner. 2017. Crowdsourcing multiple choice science questions. arXiv preprint arXiv:1707.06209.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing, and Taylor Berg-Kirkpatrick. 2018. Unsupervised text style transfer using language models as discriminators. arXiv preprint arXiv:1805.11749.
A.1 Examples Selected Data
Table 9 illustrates several data examples of selected from NQ dataset that are most similar to education domain, i.e. the RACE dataset, and to science domain, i.e. SciQ dataset using the GMMlast BERT based domain data selection method. The RACE dataset is a large dataset of English exams for middle-school and high-school Chinese students. Its vocabulary is middle-school and high-school level. Many passages in it are story-style. As NQ→RACE data examples show, the selected data from NQ are close to SciQ in terms of both the vocabulary and text style. Meanwhile, SciQ passages are chosen from science study textbooks of different topics including biology, chemistry, earth science and physics. The examples of selected data (NQ→RACE) can be categorized into the biology domain, which includes a lot of the biology terms, elucidating biological processes. These examples show the effectiveness of the data selection method.
Figure 7: Confusion matrix for unsupervised domain data clustering results on MLQuestions and NQ datasets. We use 3,000 data examples from NQ and MLQuestions each.
Figure 6: The interface for human annotation.
A.2 Experiments on MLQuestions
We conduct unsupervised domain adaptation experiments on MLQuestions (Kulshreshtha et al., 2021). We first conduct unsupervised domain data selection with  $GMM_{last}$  method and presents the confusion matrix in Figure 7 and select 1,000 data
Dataset	B-1	B-4	MT	RG
w/o-UDA	30.06	7.96	18.62	31.60
DDS	29.89	8.27	18.63	31.64
ST	32.58	9.41	19.41	34.20
DDS+ST	34.76	10.57	20.41	37.02
Net Gain	4.7↑	2.61↑	1.79↑	5.42↑
Table 8: Unsupervised domain adaptation results on MLQuestions dataset.