Current developments in natural language processing offer challenges and opportunities for low-resource languages and domains. Deep neural networks are known for requiring large amounts of training data which might not be available in resource-lean scenarios. However , there is also a growing body of works to improve the performance in low-resource settings. Motivated by  fundamental changes towards neural models and the currently popular pre-train and fine-tune paradigm, we give an overview of promising approaches for low-resource natural language processing. After a discussion about the definition of low-resource scenarios and the different dimensions of data availability, we then examine methods that enable learning when training data is sparse. This includes mechanisms to create additional labeled data like data augmentation and distant supervision as well as transfer learning settings that reduce the need for target supervision. The survey closes with a brief look into methods suggested in non-NLP machine learning communities, which might be beneficial for NLP in low-resource scenarios
Deep neural networks and huge language models are becoming omnipresent in natural language applications. As they are known for requiring large amounts of training data , there is a growing body of work to improve the performance in low-resource settings. Motivated by the recent fundamental changes towards neural models and the popular pre-train and fine-tune paradigm, we survey promising approaches for low-resource natural language processing. After a discussion about the different dimensions of data availability, we give a structured overview of methods that enable learning when training data is sparse. This includes mechanisms to create additional labeled data like data augmentation and distant supervision as well as transfer learning settings that reduce the need for target supervision. A goal of our survey is to explain how these methods differ in their requirements as understanding them is essential for choosing a technique suited for a specific low-resource setting. Further key aspects of this work are to highlight open issues and to outline promising directions for future research.
R
Current developments
Deep neural networks and huge language models are becoming omnipresent
R
processing offer challenges and opportunities for low-resource languages and domains. Deep neural networks
applications. As they
D
which might not be available in resource-lean scenarios. However
D
also
R
works
work
A
the recent
D
currently
R
give an overview of
survey
D
definition of low-resource scenarios and the
R
then examine
give a structured overview of
R
The survey closes with a brief look into methods suggested in non-NLP machine learning communities, which might be beneficial for NLP in
A goal of our survey is to explain how these methods differ in their requirements as understanding them is essential for choosing a technique suited for a specific
R
scenarios
setting. Further key aspects of this work are to highlight open issues and to outline promising directions for future research.