P y TAIL : An Open Source Tool for Interactive and Incremental Learning of NLP Models with Human in the Loop for Online Data (2023)

Abstract: Online data streams make training machine learning models hard because of distribution shift and new patterns emerging over time. For natural language processing (NLP) tasks that utilize a collection of features based on lexicons and rules, it is important to adapt these features to the changing data. To address this challenge we introduce PyTAIL, a python library, which allows a human in the loop approach to actively train NLP models. PyTAIL enhances generic active learning, which only suggests new instances to label by also suggesting new features like rules and lexicons to label. Furthermore, PyTAIL is flexible enough for users to accept, reject, or update rules and lexicons as the model is being trained. Finally, we simulate the performance of PyTAIL on existing social media benchmark datasets for text classification. We compare various active learning strategies on these benchmarks. The model closes the gap with as few as 10% of the training data. Finally, we also highlight the importance of tracking evaluation metric on remaining data (which is not yet merged with active learning) alongside the test dataset. This highlights the effectiveness of the model in accurately annotating the remaining dataset, which is especially suitable for batch processing of large unlabelled corpora. PyTAIL will be open sourced and available at https://github.com/socialmediaie/pytail.

## introduction analysis of large scale natural language corpora often requires annotation of dataset in a given domain with pre-trained models. generally, these models are pre-trained on a fixed training dataset which is often different from the domain of the dataset under consideration. this often leads to poor performance of the model on this new domain. one way to address this gap is to utilize domain adaptation @xcite to improve the model accuracy. however, efficient domain adaptation requires labeled training data from the new domain, which is costly to acquire. the problem gets compounded for social media data @xcite @xcite , for which the vocabulary and language usage continuously evolve over time @xcite @xcite . take the example of sentiment classification, where the ways of expressing the same opinion also change with time. for example, the opinion label of the phrase "you are just like subject", will depend on the general opinion about "subject" when the phrase was expressed. similarly, many new words are coined on social media @xcite . this poses a challenge for maintaining these models retain their accuracy over time. in this work, we propose an approach to alleviate this issue by creating a system based on active human-in-the-loop learning which incrementally updates an existing classifier by requiring an user to provide few new examples from the new data. traditionally, this setup, called active learning @xcite only deals with suggesting new training examples to annotate. however, since many nlp models use feature based on existing rules or lexicons, with changing data characteristics it may be more desirable to also suggest rule and lexicon updates in the model. our system pytail (python text analysis and incremental learning) addresses the issues highlighted here by allowing human-in-the-loop active learning systems to integrate new data points, rules, and lexicons. our main contributions are as follows: (i) introduce pytail, an open source tool with an active learning workflow which uses new data, rules, and lexicons to continuously train nlp models. (ii) introduce a social media text classification benchmark for active learning research. (iii) introduce an evaluation setup on unconsumed data in active learning to quantify how quickly a corpus can be fully annotated with a reasonable accuracy. ## incremental learning of models with human in the loop in this section we describe pytail (python text analysis and incremental learning). pytail's goal is to enable efficient construction of training data using active learning, while supporting incremental learning of models using the most recent data. a description of pytail workflow is shown in figure 1 . pytail is built with the following features in mind: (i) low cost of continuous training data acquisition (ii) incorporation of domain knowledge using lexicon and rules (iii) efficient update of model using only the newly acquired training data. lexicons and rules traditional active learning systems usually rely on only using the data as an input. however, pytail's focus is on involving humans at multiple stages of the learning process. hence, pytail relies on data, a set of lexicons, and a set of rules. the lexicons are used for counting lexicon matches, e.g. positive or negative words from sentiment lexicons. rules are arbitrary rules for generating features from the data, e.g. presence of a regular expression pattern. the lexicons are often used via a rule to count lexicon matches in the text. these lexicons and rules are used to help human annotators make better decisions on annotating the data and also help in the training of the model. our rules are inspired from the labeling function approach of snorkel @xcite , however, they differ as they are used as feature generator. overview as shown in figure 1 , the user starts with a collection of artifacts in the bootstrap stage. this can include an pre-trained model, a small seed training dataset, existing rules, and lexicons. next, the user introduces their unlabeled data stream from their domain of interest, e.g. social media corpora. the bootstrap artifacts are used to predict this data stream. these predictions are then fed to the query strategy (described below) to identify artifacts for the suggestion stage. the user can then accept, reject, update these suggestions or even introduce new suggestions. next, the model is updated using updated artifacts such that the rules and lexicons are used for updating the model features and the annotated data is used for updating the model. finally, pytail shows continuous evaluation metrics which include metric on a test set, user accepted training set, and unobserved data stream. this process is repeated till a stopping criteria is met, e.g. the exhaustion of data stream or achieving reasonable evaluation score. pytail supports two modes for training, one is human in the loop (hitl) mode, and another is simulation mode. the simulation model uses pre-defined heuristics to simulate human actions based on model prediction scores. the default model when applied to benchmark datasets is the simulation mode. human in the loop (hitl) mode in the hitl mode, pytail uses the pre-trained model to suggest top k instances to the user. the user can sort the instances using the scoring criterion. in order to reduce the cognitive work of labeling an instance from scratch, the user is shown the model predictions (as well as the label probability). the user is only required to edit the labels if they disagree. model predictions for all the unlabeled instances from the top suggestions are now used as gold labels and fed to the model during the update process (this is similar to self-supervision with the possibility of human intervention). the user is also shown the prominent features for that instance, the user can select these features and mark them as useful or useless. lexicon matches with the annotations are also shown, along with prominent key phrases in the unlabeled data stream. the user can choose to update the lexicon with these new suggestions. once the model update has happened, the user is provided feedback on the change in model evaluation on a held out data. ## benchmark for social media active learning we introduce an active learning benchmark of 10 social media text classification datasets consisting of 200k posts. these datasets cover sentiment classification, abusive content identification, and uncertainty indication and is derived from the so-cialmediaie benchmark @xcite . the dataset is available at https://zenodo. org/doi/10.5281/zenodo.7236429 . ## pytail for social media text classification model we use a logistic regression model with l 2 regularization. the regularization parameter is selected for each model using cross validation. we track the model scores on the held out test as well as validation data. each text is represented using a set of features. each tweet is tokenized and preprocessed by normalizing all mentions of hashtags, urls, and mentions. we also use a large sentiment lexicon foot_0 . furthermore, we suggest including a domain specific negative filter, i.e., words which should not be used to identify classification signals. for sentiment classification this can be entities in the corpora which should not bias the model. ## conclusion we described experiments for evaluating active learning approaches for text classification tasks on tweet data. we introduced, pytail, an open source user interface for active learning of nlp models by only requiring the user to update the labels for the model prediction if required. we also release a benchmark dataset for social media active learning. one limitation of our work is that our experiments are only conducted using simple linear model as they are easier to experiment with for sparse text features which we used for feature importance. however, the api does not place any restriction on the type of model. in the future we plan to extend this strategy to non classification tasks for social media datasets e.g. structured prediction tasks like named entity recognition, pos tagging, and chunking @xcite @xcite @xcite @xcite . pytail is available as an open source tool at https://github.com/socialmediaie/ pytail/ and our dataset is available at https://zenodo.org/doi/10.5281/ zenodo.7236429 .