SlowGuess/NAACL_2021_OCR huggingface dataset:
Jennifer Tracey and Stephanie Strassel. 2020. Basic language resources for 31 languages (plus English): The LORELEI representative and incident language packs. In Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL), pages 277–284, Marseille, France. European Language Resources association.
Tatiana Tsygankova, Francesca Marini, Stephen Mayhew, and Dan Roth. 2020. Building low-resource ner models using non-speaker annotation. CoRR, abs/2006.09627.
Aminu Tukur, Kabir Umar, and Anas Saidu Muhammad. 2019. Tagging part of speech in hausa sentences. In 2019 15th International Conference on Electronics, Computer and Computation (ICECCO), pages 1-6. IEEE.
Clara Vania, Yova Kementchedjhieva, Anders Søgaard, and Adam Lopez. 2019. A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1105-1116, Hong Kong, China. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5998-6008. Curran Associates, Inc.
Hao Wang, Bing Liu, Chaozhuo Li, Yan Yang, and Tianrui Li. 2019. Learning with noisy labels for sentence-level sentiment classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6286-6292, Hong Kong, China. Association for Computational Linguistics.
Jason Wei and Kai Zou. 2019. EDA: Easy data augmentation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6382-6388, Hong Kong, China. Association for Computational Linguistics.
Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang. 2016. A survey of transfer learning. Journal of Big data, 3(1):9.
Garrett Wilson and Diane J Cook. 2020. A survey of unsupervised deep domain adaptation. ACM
Transactions on Intelligent Systems and Technology (TIST), 11(5):1-46.
Guillaume Wisniewski, Nicolas Pécheux, Souhir Gahbiche-Braham, and François Yvon. 2014. Cross-lingual part-of-speech tagging through ambiguous learning. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1779-1785, Doha, Qatar. Association for Computational Linguistics.
Shijie Wu and Mark Dredze. 2020. Are all languages created equal in multilingual BERT? In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 120-130, Online. Association for Computational Linguistics.
Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. 2015. Learning from massive noisy labeled data for image classification. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pages 2691-2699. IEEE Computer Society.
Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V. Le. 2020. Unsupervised data augmentation for consistency training. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
Hu Xu, Bing Liu, Lei Shu, and Philip Yu. 2020. DomBERT: Domain-oriented language model for aspect-based sentiment analysis. Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1725-1731.
Yaosheng Yang, Wenliang Chen, Zhenghua Li, Zhengqiu He, and Min Zhang. 2018. Distantly supervised NER with partial annotation learning and reinforcement learning. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2159-2169, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
Ze Yang, Wei Wu, Jian Yang, Can Xu, and Zhoujun Li. 2019. Low-resource response generation with template prior. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1886-1897, Hong Kong, China. Association for Computational Linguistics.
David Yarowsky, Grace Ngai, and Richard Wicentowski. 2001. Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of the First International Conference on Human Language Technology Research.
Michihiro Yasunaga, Jungo Kasai, and Dragomir Radev. 2018. Robust multilingual part-of-speech tagging via adversarial training. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:
Human Language Technologies, Volume 1 (Long Papers), pages 976-986, New Orleans, Louisiana. Association for Computational Linguistics.
Qinyuan Ye, Liyuan Liu, Maosen Zhang, and Xiang Ren. 2019. Looking beyond label noise: Shifted label distribution matters in distantly supervised relation extraction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3841-3850, Hong Kong, China. Association for Computational Linguistics.
Jihene Younes, Emna Souissi, Hadhemi Achour, and Ahmed Ferchichi. 2020. Language resources for maghrebi arabic dialects' nlp: a survey. LANGUAGE RESOURCES AND EVALUATION.
Mo Yu, Xiaoxiao Guo, Jinfeng Yi, Shiyu Chang, Saloni Potdar, Yu Cheng, Gerald Tesauro, Haoyu Wang, and Bowen Zhou. 2018. Diverse few-shot text classification with multiple metrics. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1206-1215, New Orleans, Louisiana. Association for Computational Linguistics.
BI Yude. 2011. A brief survey of korean natural language processing research. Journal of Chinese Information Processing, 6.
Meishan Zhang, Yue Zhang, and Guohong Fu. 2019a. Cross-lingual dependency parsing using code-mixed TreeBank. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 997–1006, Hong Kong, China. Association for Computational Linguistics.
Rui Zhang, Caitlin Westerfield, Sungrok Shim, Garrett Bingham, Alexander Fabbri, William Hu, Neha Verma, and Dragomir Radev. 2019b. Improving low-resource cross-lingual document retrieval by reranking with deep bilingual representations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3173-3179, Florence, Italy. Association for Computational Linguistics.
Shun Zheng, Xu Han, Yankai Lin, Peilin Yu, Lu Chen, Ling Huang, Zhiyuan Liu, and Wei Xu. 2019. DIAG-NRE: A neural pattern diagnosis framework for distantly supervised neural relation extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1419-1429, Florence, Italy. Association for Computational Linguistics.
Joey Tianyi Zhou, Hao Zhang, Di Jin, Hongyuan Zhu, Meng Fang, Rick Siow Mong Goh, and Kenneth Kwok. 2019. Dual adversarial neural transfer for low-resource named entity recognition. In Proceedings of the 57th Annual Meeting of the Association
for Computational Linguistics, pages 3461-3471, Florence, Italy. Association for Computational Linguistics.
Mengfei Zhou, Anette Frank, Annemarie Friedrich, and Alexis Palmer. 2015. Semantically enriched models for modal sense classification. In Proceedings of the First Workshop on Linking Computational Models of Lexical, Sentential and Discourse-level Semantics, pages 44-53, Lisbon, Portugal. Association for Computational Linguistics.
Yi Zhu, Benjamin Heinzerling, Ivan Vulic, Michael Strube, Roi Reichart, and Anna Korhonen. 2019. On the importance of subword information for morphological tasks in truly low-resource languages. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 216-226, Hong Kong, China. Association for Computational Linguistics.
A Existing Surveys on Low-Resource Topics and Languages
There is a growing body of task- and language-specific surveys concerning low-resource topics. We list these surveys in Table 2 as a starting point for a more in-depth reading regarding specific topics.
B Complexity of Tasks
While a large number of labeled resources for English are available for many popular NLP tasks, this is not the case for the majority of low-resource languages. To measure (and visualize as done in Figure 1 in the main paper) which applications are accessible to speakers of low-resource languages we examined resources for six different languages, ranging from high- to low-resource languages for a fixed set of tasks of varying complexity, ranging from basic tasks, such as tokenization, to higher-level tasks, such as question answering. For this short study, we have chosen the following languages. The number of speakers are the combined L1 and L2 speakers according to Eberhard et al. (2019).
(1) English: The most high-resource language according to the common view and literature in the NLP community.
(2) Yoruba: An African language, which is spoken by about 40 million speakers and contained in the EXTREME benchmark (Hu et al., 2020). Even with that many speakers, this language is often considered as a low-resource
language and it is still discussed whether this language is also endangered (Fabuni and Salawu, 2005).
(3) Hausa: An African language with over 60 million speakers. It is not covered in EXTREME or the universal dependencies project (Nivre et al., 2020).
(4) Quechua: A language family encompassing about 8 million speakers, mostly in Peru.
(5) Nahuatl and (6) Estonian: Both have between 1 and 2 million speakers, but are spoken in very different regions (North America & Europe).
All speaker numbers according to Eberhard et al. (2019) reflecting the total number of users  $(\mathrm{L}1 + \mathrm{L}2)$ . The tasks were chosen from a list of popular NLP tasks<sup>3</sup>. We selected two tasks for the lower-lever groups and three tasks for the higher-level groups, which reflects the application diversity with increasing complexity. Table 3 shows which tasks were addressed for each language.
Word segmentation, lemmatization, part-of-speech tagging, sentence breaking and (semantic) parsing are covered for Yoruba and Estonian by treebanks from the universal dependencies project (Nivre et al., 2020). Cusco Quechua is listed as an upcoming language in the UD project, but no treebank is accessible at this moment. The WikiAnn corpus for named entity recognition (Pan et al., 2017) has resources and tools for NER and sentence breaking for all six languages. Lemmatization resources for Nahuatl were developed by Martínez-Gil et al. (2012) and Lozano et al. (2013) developed resources for part-of-speech tagging, tokenization and parsing of Quechuan. The CoNLL conference and SIGMORPHON organized two shared tasks for morphological reinflation which provided lemmatization resources for many languages, including Quechuan (Cotterell et al., 2018).
Basic resources for simple semantic role labeling and entity linking were developed during the LORELEI program for many low-resource languages (Strassel and Tracey, 2016; Tracey and Strassel, 2020), including resources for Yoruba and Hausa (even though the latter "fell short" according to the authors). Estonian coreference resolution is targeted by Kübler and Zhekova (2016), but
3https://en.wikipedia.org/wiki/ Natural_language_PROCESSing#Common_NLP_ Tasks
Low-resource surveys	Cieri et al. (2016), Magueresse et al. (2020)
Method-specific	Active learning	Olsson (2009), Settles (2009), Aggarwal et al. (2014)
Distant supervision	Roth et al. (2013), Smirnova and Cudré-Mauroux (2018), Shi et al. (2019).
Unsupervised domain adaptation	Wilson and Cook (2020), Ramponi and Plank (2020)
Meta-Learning	Hospedales et al. (2020)
Multilingual transfer	Steinberger (2012), Ruder et al. (2019)
LM pre-training	Rogers et al. (2021), Qiu et al. (2020)
Machine translation	Liu et al. (2019)
Label noise handling	Frénay and Verleysen (2013), Algan and Ulusoy (2021)
Transfer learning	Pan and Yang (2009), Weiss et al. (2016), Tan et al. (2018)
Language-	African languages	Grover et al. (2010), De Pauw et al. (2011)
Arabic languages	Al-Ayyoub et al. (2018), Guellil et al. (2019), Younes et al. (2020)
American languages	Mager et al. (2018)
South-Asian languages	Daud et al. (2017), Banik et al. (2019), Harish and Rangan (2020)
East-Asian languages	Yude (2011)
Table 2: Overview of existing surveys on low-resource topics.
the available resources are very limited. Estonian sentiment is done by Pajupuu et al. (2016). All languages are covered by the multilingual fasttext embeddings (Bojanowski et al., 2017) and byte-pair-encoding embeddings (Heinzerling and Strube, 2018). Yoruba, Hausa and Estonian are covered by mBERT or XLM-RoBERTa as well.
Text summarization is done for Estonian by Mūürisep and Mutso (2005) and for Hausa by Bashir et al. (2017). The EXTREME benchmark (Hu et al., 2020) covers question answering and natural language inference tasks for Yoruba and Estonian (besides NER, POS tagging and more). Publicly available systems for optical character recognition support all six languages (Hakro et al., 2016). All these tasks are supported for the English language as well, and most often, the English datasets are many times larger and of much higher quality. Some of the previously mentioned datasets were automatically translated, as in the EXTREME benchmark for several languages. As outlined in the main paper, we do not claim that all tasks marked in the Table yield high-performance model, but we instead indicate if any resources or models can be found for a language.
Group	Task	Yoruba	Hausa	Quechuan	Nahuatl	Estonian
Num-Speakers	40 mil.	60 mil.	8 mil.	1.7 mil.	1.3 mil.
Text processing	Word segmentation	✓	✓	✓	✓	✓
Optical character recognition	Hakro et al. (2016)	Hakro et al. (2016)	Hakro et al. (2016)	Hakro et al. (2016)	Hakro et al. (2016)
Morphological analysis	Lemmatization / Stemming	Cotterell et al. (2018)	Cotterell et al. (2018)	Cotterell et al. (2018)	Martínez-Gil et al. (2012)	Cotterell et al. (2018)
Part-of-Speech tagging	Nivre et al. (2020)	Tukur et al. (2019)	Lozano et al. (2013)	X	Nivre et al. (2020)
Syntactic analysis	Sentence breaking	✓	✓	✓	✓	✓
Parsing	Nivre et al. (2020)	X	Nivre et al. (2020)	X	Nivre et al. (2020)
Distributional semantics	Word embeddings	FT, BPEmb	FT, BPEmb	FT, BPEmb	FT, BPEmb	FT, BPEmb
Transformer models	mBERT	XLM-R	X	X	mBERT, XLM-R
Lexical semantics	Named entity recognition	Adelani et al. (2020)	Adelani et al. (2020)	Pan et al. (2017)	Pan et al. (2017)	Tkachenko et al. (2013)
Sentiment analysis	X	X	X	X	Pajupuu et al. (2016)
Relational semantics	Relationship extraction	X	X	X	X	X
Semantic Role Labelling	Tracey and Strassel (2020)	Tracey and Strassel (2020)	X	X	X
Semantic Parsing	Nivre et al. (2020)	X	X	X	Nivre et al. (2020)
Discourse	Coreference resolution	X	X	X	X	Kübler and Zhekova (2016)
Discourse analysis	X	X	X	X
Textual entailment	Hu et al. (2020)	X	X	X	Hu et al. (2020)
Higher-level NLP	Text summarization	X	Bashir et al. (2017)	X	X	Müürisep and Mutso (2005)
Dialogue management	X	X	X	X	X
Question answering (QA)	Hu et al. (2020)	X	X	X	Hu et al. (2020)
SUM	13	10	8	6	15
Table 3: Overview of tasks covered by six different languages. Note that this list is non-exhaustive and due to space reasons we only give one reference per language and task.