Learning with Limited Text Data (2022)

Abstract: Natural Language Processing (NLP) has achieved great progress in the past decade on the basis of neural models, which often make use of large amounts of labeled data to achieve state-of-the-art performance. The dependence on labeled data prevents NLP models from being applied to low-resource settings and languages because of the time, money, and expertise that is often required to label massive amounts of textual data. Consequently, the ability to learn with limited labeled data is crucial for deploying neural systems to real-world NLP applications. Recently, numerous approaches have been explored to alleviate the need for labeled data in NLP such as data augmentation and semi-supervised learning. This tutorial aims to provide a systematic and up-to-date overview of these methods in order to help researchers and practitioners understand the landscape of approaches and the challenges associated with learning from limited labeled data, an emerging topic in the computational linguistics community. We will consider applications to a wide variety of NLP tasks (including text classification, generation, and structured prediction) and will highlight current challenges and future directions.

## introduction natural language processing (nlp) has achieved great progress in the past decade on the basis of neural models, which often make use of large amounts of labeled data to achieve state-of-the-art performance. the dependence on labeled data prevents nlp models from being applied to low-resource settings and languages because of the time, money, and expertise that is often required to label massive amounts of textual data. consequently, the ability to learn with limited labeled data is crucial for deploying neural systems to real-world nlp applications. recently, numerous approaches have been explored to alleviate the need for labeled data in nlp such as data augmentation and semisupervised learning. this tutorial aims to provide a systematic and upto-date overview of these methods in order to help researchers and practitioners understand the landscape of approaches and the challenges associated with learning from limited labeled data, an emerging topic in the computational linguistics community. we will consider applications to a wide variety of nlp tasks (including text classification, generation, and structured prediction) and will highlight current challenges and future directions. ## tutorial outline this will be a three-hour tutorial devoted to the cutting-edge topic of learning with limited text data, divided into three sessions. each session will be 40 minutes, followed by 10 minutes for q&a and 10 minutes for a break. each part includes an overview of the corresponding topic and widely used methods and a deep dive into a set of representative nlp work. ## diversity considerations this tutorial will cover techniques and topics beyond english as an application domain. we will also cover content around how learning with limited text data can be applicable to low-resourced language, dialects, and other related tasks. our presenter team has a diverse background from both academia (a junior female faculty from georgia institute of technology, and an assistant professor from university of north carolina, chapel hill) and industry (a research scientist from google). our presenter team will share our tutorial with a worldwide audience by promoting it on social media. we will work with acl/naacl d&i teams, and consult resources such as the big directory to diversify our audience participation. furthermore, we will engage with nlp initiatives like masakhane that our team has connections to. ## prerequisites the prerequisite includes familiarity with basic machine learning and deep learning models, especially those typically used in modern nlp, including attention mechanisms @xcite , the transformer architecture @xcite , sequence-to-sequence learning @xcite , etc. furthermore, this tutorial assumes background in basic probability, linear algebra, and calculus. we will also provide a more paced introduction to the material with additional readings. ## ethics statement we do not anticipate any ethical issues related to the topics of the tutorial.