SlowGuess/ACL_2020_OCR huggingface dataset:
Asaf Amrami and Yoav Goldberg. 2018. Word sense induction with neural bilm and symmetric patterns. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 4860-4867. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 4171-4186.
Yoav Goldberg. 2019. Assessing bert's syntactic abilities. CoRR, abs/1901.05287.
Sonal Gupta and Christopher D. Manning. 2014. Improved pattern learning for bootstrapped entity extraction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, CoNLL 2014, Baltimore, Maryland, USA, June 26-27, 2014, pages 98-108.
Zellig S. Harris. 1968. Mathematical structures of language, volume 21 of Interscience tracts in pure and applied mathematics. Interscience Publ.
Marti A. Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In 14th International Conference on Computational Linguistics, COLING 1992, Nantes, France, August 23-28, 1992, pages 539-545.
John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word representations. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, pages 4129-4138.
Donald Hindle. 1990. Noun classification from predicate-argument structures. In 28th Annual Meeting of the Association for Computational Linguistics, 6-9 June 1990, University of Pittsburgh, Pittsburgh, Pennsylvania, USA, Proceedings, pages 268-275.
Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencies. TACL, 4:521-535.
Abhijit Mahabal, Dan Roth, and Sid Mittal. 2018. Robust handling of polysemy via sparse representations. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, *SEM@NAACL-HLT, New Orleans, Louisiana, USA, June 5-6, 2018, pages 265-275. Association for Computational Linguistics.
Jonathan Mamou, Oren Pereg, Moshe Wasserblat, Alon Eirew, Yael Green, Shira Guskin, Peter Izsak, and Daniel Korat. 2018. Term set expansion based NLP architect by intel AI lab. pages 19-24.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings.
Hiroaki Ohshima, Satoshi Oyama, and Katsumi Tanaka. 2006. Searching coordinate terms with their context from the web. In Web Information Systems - WISE 2006, 7th International Conference on Web Information Systems Engineering, Wuhan, China, October 23-26, 2006, Proceedings, pages 40-47.
Patrick Pantel, Eric Crestan, Arkady Borkovsky, Ana Maria Popescu, and Vishnu Vyas. 2009. Web-scale distributional similarity and entity set expansion. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, 6-7 August 2009, Singapore, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 938-947.
Patrick Pantel and Dekang Lin. 2002. Discovering word senses from text. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, July 23-26, 2002, Edmonton, Alberta, Canada, pages 613-619.
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019. Language models as knowledge bases? pages 2463-2473.
Shuming Shi, Huibin Zhang, Xiaojie Yuan, and JiRong Wen. 2010. Corpus-based semantic class mining: Distributional vs. pattern-based approaches. In COLING 2010, 23rd International Conference on Computational Linguistics, Proceedings of the Conference, 23-27 August 2010, Beijing, China, pages 993-1001.
Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT rediscovers the classical NLP pipeline. pages 4593-4601.
Andrew Trask, Phil Michalak, and John Liu. 2015. sense2vec - A fast and accurate method for word sense disambiguation in neural word embeddings. CoRR, abs/1511.06388.
Richard C. Wang and William W. Cohen. 2007. Language-independent set expansion of named entities using the web. In Proceedings of the 7th IEEE International Conference on Data Mining (ICDM 2007), October 28-31, 2007, Omaha, Nebraska, USA, pages 342-350.
6834
Huibin Zhang, Mingjie Zhu, Shuming Shi, and Ji-Rong Wen. 2009. Employing topic models for pattern-based semantic class discovery. In ACL 2009, Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2-7 August 2009, Singapore, pages 459-467.
A Appendix A: Finding candidate terms
For our first method, MPB1, the candidate terms we score are just the terms in the LM's vocabulary. For our second method, MPB2, we want to score candidates which are not in this vocabulary as well. Hence, we need a way to acquire these candidates. As running on all possible terms is prohibitive, we seek an efficient method to acquire a high-recall group of candidates for the desired semantic class. We get this using a simple distributional set-expander: we compute the mean vector for words in our seed set, and look for the top-k neighbours in a distributional space.
Specifically, we use the sense2vec pretrained vectors. Sense2vec (Trask et al., 2015) is a misleadingly-named algorithm from the w2v-family (Mikolov et al., 2013) that models each term as "term-part of speech". This allows it, for example, to learn different representations for "duck-verb" and "duck-noun".
More importantly, the pre-trained sense2vec vectors distributed by explosion.ai $^{14}$  are trained over a large and diverse English corpus (reddit posts and comments from 2015 and 2019), and its vocabulary includes not only single words but also multi-word units (NP-chunks and named entities).
$^{14}$ https://explosion.ai/demos/sense2vec