SlowGuess/ACL_2020_OCR huggingface dataset:
Masaki Murata, Masao Utiyama, Qing Ma, Hiromi Ozaku, and Hitoshi Isahara. 2001. Crl at ntcir2. In Proceedings of the Second NTCIR Workshop on Research in Chinese & Japanese Text Retrieval and Text Summarization.
Thuy Dung Nguyen and Min-Yen Kan. 2007. Keyphrase extraction in scientific publications. In Proceedings of the 10th International Conference on Asian Digital Libraries: Looking Back 10 Years and Forging New Frontiers, ICADL'07, pages 317-326, Berlin, Heidelberg. Springer-Verlag.
Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. 2019. Document expansion by query prediction. arXiv preprint arXiv:1904.08375.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. In NIPS 2017 Workshop Autodiff.
Shengwen Peng, Ronghui You, Hongning Wang, Chengxiang Zhai, Hiroshi Mamitsuka, and Shanfeng Zhu. 2016. Deepmesh: deep semantic representation for improving large-scale mesh indexing. Bioinformatics, 32(12):i70-i79.
Mark D. Smucker, James Allan, and Ben Carterette. 2007. A comparison of statistical significance tests for information retrieval evaluation. In Proceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management, CIKM '07, pages 623-632, New York, NY, USA. ACM.
C Rockelle Strader. 2011. Author-assigned keywords versus library of congress subject headings. Library resources & technical services, 53(4):243-250.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 3104-3112. Curran Associates, Inc.
Takafumi Suzuki, Kiyoko Uchiyama, Ryota Tomisaka, and Akiko Aizawa. 2011. Analyzing the characteristics of academic paper categories by using an index of representativeness. In Proceedings of the 25th Pacific Asia Conference on Language, Information and Computation, pages 587-596, Singapore. Institute of Digital Enhancement of Cognitive Processing, Waseda University.
Tao Tao, Xuanhui Wang, Qiaozhu Mei, and ChengXiang Zhai. 2006. Language model information retrieval with document expansion. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pages 407-414, New York City, USA. Association for Computational Linguistics.
Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling coverage for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 76-85, Berlin, Germany. Association for Computational Linguistics.
Peter D. Turney. 2003. Coherent keyphrase extraction via web mining. In Proceedings of the 18th International Joint Conference on Artificial Intelligence, IJCAI'03, pages 434-439, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
Xiaojun Wan and Jianguo Xiao. 2008. Collabrank: Towards a collaborative approach to single-document keyphrase extraction. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 969-976, Manchester, UK. Coling 2008 Organizing Committee.
I. H. Witten and O. Medelyan. 2006. Thesaurus based automatic keyphrase indexing. In Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '06), pages 296-297.
Ian H. Witten, Gordon W. Paynter, Eibe Frank, Carl Gutwin, and Craig G. Nevill-Manning. 1999. Kea: Practical automatic keyphrase extraction. In Proceedings of the Fourth ACM Conference on Digital Libraries, DL '99, pages 254-255, New York, NY, USA. ACM.
Peilin Yang, Hui Fang, and Jimmy Lin. 2017. Anserini: Enabling the use of lucene for information retrieval research. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '17, pages 1253-1256, New York, NY, USA. ACM.
Wei Yang, Kuang Lu, Peilin Yang, and Jimmy Lin. 2019. Critically examining the "neural hype": Weak baselines and the additivity of effectiveness gains from neural ranking models. In Proceedings of the 42Nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR'19, pages 1129-1132, New York, NY, USA. ACM.
Chengxiang Zhai. 1997. Fast statistical parsing of noun phrases for document indexing. In *Fifth Conference on Applied Natural Language Processing*, pages 312-319, Washington, DC, USA. Association for Computational Linguistics.
Jing Zhao and Yuxiang Zhang. 2019. Incorporating linguistic constraints into keyphrase generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5224-5233, Florence, Italy. Association for Computational Linguistics.
A.1 Related Work
Keyphrase extraction and generation
Identifying keyphrases for a given document is a long standing task in NLP. Earlier work typically involves two steps: 1) extracting keyphrase candidates; and 2) ranking those candidates by importance. Models mainly differ in how they do the latter, commonly used techniques being supervised learning (Witten et al., 1999; Turney, 2003; Nguyen and Kan, 2007; Jiang et al., 2009) and graph-based methods (Mihalcea and Tarau, 2004; Wan and Xiao, 2008; Bougouin et al., 2013; Florescu and Caragea, 2017). These models are, however, inherently limited in the sense that they can only output keyphrases that appear in the text. To allow the prediction of keyphrases describing implicit topics or using different wordings, previous work relied on external resources like controlled vocabularies (Witten and Medelyan, 2006; Bougouin et al., 2016), while recent attempts leveraged neural generative models (Meng et al., 2017; Chen et al., 2018; Zhao and Zhang, 2019).
Biomedical indexing
Also related to our work is the research done on biomedical semantic indexing using  $\mathrm{MeSH}^3$ , a hierarchically-organized controlled vocabulary. Automated methods for assigning MeSH terms make use of all sorts of techniques, such as pattern matching (Aronson et al., 2004) or learning to rank (Liu et al., 2015; Peng et al., 2016).
Document expansion
Our work is similar in nature to previous research on document expansion (Tao et al., 2006; Efron et al., 2012), and is closely related to recent work on document expansion using automatically generated queries (Nogueira et al., 2019).
A.2 Parameters
Table 6 displays the model parameters we use for seq2seq+copy and seq2seq+corr.
Table 7 presents the research fields used for dividing queries into two sets.
A.3 Example
An example of document along with automatically generated keyphrases is shown in Table 8.
Parameter	Value
Network	bi-GRU
Vocabulary size	50K
Word embedding size	150
Hidden layer	2
Hidden layer size	300
Optimizer	Adam
Initial learning rate	10-4
Gradient clipping	0.1
Dropout	0.5
Beam depth	6
Beam size	200
Table 6: Model Parameters.
Research field	In	Out
Electricity, information and control	✓	-
Chemistry	✓	-
Architecture, civil engineering	-	✓
Biology and agriculture	-	✓
Science	✓	-
Engineering	✓	-
Medicine and dentistry	-	✓
Cultural and social science	-	✓
# of queries	27	22
Table 7: Research fields for in- and out-domain queries.
<sup>3</sup>https://www.nlm.nih.gov/mesh/
title	Grammatical Inference for Concept Acquisition from Documents.
abstract	The purpose of this study is to acquire knowledge from large scale natural language documents. There are two types of knowledge in the documents. One is explicitly represented knowledge which is acquired using natural language processing. The other is implicit constrain. In this paper, how to acquire implicit constraint using grammatical inference from the documents is described. We propose a grammatical inference system which uses inference rules based on logic, and show that the system can learn easy pattern of character lists. We also discuss its application to knowledge acquisition from real documents.
gold	grammatical inference // knowledge acquisition // logic programming // concept learning
s2s+copy	grammatical inference // knowledge // grammatical // knowledge representation // natural language processing
s2s+corr	grammatical inference // knowledge acquisition // concept acquisition // inference rules // natural language processing
mp-rank	grammatical inference // documents // knowledge // large scale // concept acquisition
Table 8: Example document (id: gakkai-e-0001014453 from ntc2-e1g) with author keywords (gold) and automatically generated keyphrases. We note a typo in the gold annotation (progamming).