Like a Good Nearest Neighbor: Practical Content Moderation and Text Classification (2024)

Abstract: Few-shot text classification systems have impressive capabilities but are infeasible to deploy and use reliably due to their dependence on prompting and billion-parameter language models. SetFit (Tunstall, 2022) is a recent, practical approach that fine-tunes a Sentence Transformer under a contrastive learning paradigm and achieves similar results to more unwieldy systems. Inexpensive text classification is important for addressing the problem of domain drift in all classification tasks, and especially in detecting harmful content, which plagues social media platforms. Here, we propose Like a Good Nearest Neighbor (LaGoNN), a modification to SetFit that introduces no learnable parameters but alters input text with information from its nearest neighbor, for example, the label and text, in the training data, making novel data appear similar to an instance on which the model was optimized. LaGoNN is effective at flagging undesirable content and text classification, and improves SetFit’s performance. To demonstrate LaGoNN’s value, we conduct a thorough study of text classification systems in the context of content moderation under four label distributions, and in general and multilingual classification settings.

## introduction text classification is the most important tool for nlp practitioners, and there has been substantial progress in advancing the state-of-the-art, especially with the advent of large, pretrained language models (plm) @xcite . modern research focuses on in-context learning @xcite , pattern exploiting training @xcite , adapter-based finetuning with learned label embeddings @xcite , and parameter efficient finetuning @xcite . these methods have achieved impressive results on the superglue @xcite and raft @xcite few-shot benchmarks, but most are difficult to use because of their reliance on billion-parameter plms, pay-to-use apis, and/or prompting. constructing prompts is not trivial and may require domain expertise. one exception to these cumbersome systems is setfit. setfit does not rely on prompting or billion-parameter plms, and instead fine-tunes a pretrained sentence transformer (st) @xcite under a contrastive learning paradigm. setfit has comparable performance to more unwieldy systems while being one to two orders of magnitude faster to train and run inference. an important application of text classification is aiding or automating content moderation, which is the task of determining the appropriateness of user-generated content on the internet @xcite . from fake news to toxic comments to hate speech, it is difficult to browse social media without being exposed to potentially dangerous posts that may have an effect on our ability to reason @xcite . misinformation spreads at alarming rates @xcite , and an ml system should be able to quickly aid human moderators. while there is work in nlp with this goal @xcite @xcite , a general, practical, and open-sourced method that is effective across multiple domains remains an open challenge. novel fake news topics or racial slurs emerge and change constantly. retraining of ml-based systems is required to adapt this concept drift, but this is expensive, not only in terms of computation, but also in terms of the human effort needed to collect and label data. setfit's performance, speed, and low cost would make it ideal for effective content moderation, however, this type of text classification proves difficult for even state-of-the-art approaches. for example, detecting hate speech on twitter @xcite , a subtask on the raft few-shot benchmark, appears to be the most difficult dataset; at time of writing, it is the only task where the human baseline has not been surpassed, yet setfit is among the top ten most performant systems. 2 here, we propose a modification to setfit, called like a good nearest neighbor (lagonn). lagonn introduces no learnable parameters and instead modifies input text by retrieving information from its nearest neighbors (nn) seen during optimization. specifically, we append the label, distance, and text of the nns in the training data to a new instance and encode this modified version with an st (see figure 1 and ## related work there is little work on using sentence embeddings as features for classification despite the pioneering work being five years old @xcite . sts are pretrained with the objective of maximizing the distance between semantically distinct text and minimizing the distance between text that is semantically similar in feature space. they are composed of a siamese and triplet architecture that encodes text into dense vectors which can be used as features for ml. sts were first used to embed text for classification by @xcite , however, only pretrained representations were examined. setfit uses a contrastive learning paradigm @xcite to optimize the st embedding model. the st is fine-tuned with a distance-based loss function, like cosine similarity, such that examples with different labels are separated in feature space. input text is then encoded with the fine-tuned st and a classifier, such as logistic regression, is trained. this approach creates a strong, few-shot text classification system, transforming the st from a sentence encoder to a topic encoder. work done by @xcite showed that retrieving and concatenating text from training data and external sources, such as conceptnet @xcite and the wiktionary foot_2 definition, can be viewed as a type of external attention that does not alter the architecture of the transformer in question answering. @xcite used plms and k-nn lookup to prepend examples that are similar to a gpt-3 query, aiding in prompt engineering for in-context learning. @xcite demonstrated that prepending and appending training data helps plms in summarization, language modelling, machine translation, and question answering, using bm25 as their retrieval model @xcite . we alter the setfit training procedure by using fewer examples to adapt the embedding model for many-shot learning. lagonn decorates input text with its nn's gold label, euclidean distance, and text from the training data to exploit both the st's distance-based pretraining and setfit's distancebased fine-tuning objective. compared to retrievalbased methods, lagonn uses the same model for both retrieval and encoding, retrieving only information from the training data for classification. encoding and nearest neighbors lagonn first uses a pretrained sentence transformer to embed training text in feature space, f (x train ), and nn lookup with scikit-learn @xcite on the resulting embeddings. ## experiments we first study lagonn's performance on four binary and one ternary classification dataset related to the task of content moderation. each dataset is composed of a training, validation, and test split (see appendix a.1 for details). we study our system by simulating growing training data over ten discrete steps sampled under four different label distributions: extreme, imbalanced, moderate, and balanced (see this experimental setup is reflective of a practical setting, where we might construct a content flagging or text classification system with a relatively small number (100) of labeled examples for training. as time goes on, however, more samples are added and we must then determine whether or not it is worth the resources to retrain our system. we sampled over five seeds, reporting the mean and standard deviation. ## content moderation results in the cases of the extreme and imbalanced regimes, the performance of setfit exp steadily increases with the number of training examples. as the label distribution shifts to the balanced regime, however, the performance quickly saturates or even degrades as the number of training examples grows. lagonn, roberta f ull , and setfit, other finetuned plm classifiers, do not exhibit this behavior. lagonn exp , being based on setfit exp , exhibits a similar trend, but the performance degradation is mitigated; on the 10 th step of amazon counterfactual in lagonn cheap outperforms all other methods on the insincere questions dataset for all balance regimes, despite being the third fastest (see ## lagonn as a general classifier lagonn is effective for general text classification. thus far, we have focused on the important topic of content moderation, but here we turn our attention to general text classification, conducting experiments on 12 additional datasets (see appendix a.2 for details and appendix a.6 for multilingual experiments). our experimental setup remains largely the same, but here we restrict ourselves to the balanced sampling regime as it is nontrivial to design sampling strategies for datasets with more than three labels. we respectively compare lagonn lite against setfit lite and lagonn exp against setfit exp , showing results for one to five neighbors with lagonn. in figure 4 , we demonstrate that lagonn continues to stabilize and improve setfit, regardless of the number of neighbors we consider. this is especially clear for imdb, where in the case of lagonn lite vs setfit lite , all versions of our method saturate to an average precision of 98 with 300 fewer training samples. if we consider setfit exp vs lagonn exp , consistent with our analysis of other binary datasets, classifier performance begins to degrade if we continue to fine-tune the st, but lagonn mitigates this performance drop. continuing to fine-tune the embedding model is beneficial when we have many labels. for 20 newsgroups and emotion, which have 20 and 28 labels respectively, lagonn exp is the strongest model and shows no indication of plateauing or degrading, even with 1,000 samples. we attribute this to the relatively high number of labels present in both of these datasets. our findings related to sst-5 and our multilingual experiments (see appendix a.6) support this; in intermediate cases when we have five labels, all models saturate quickly and there are minimal gains with continued fine-tuning. ## discussion flagging potentially dangerous text presents a challenge even for state-of-the-art approaches. the content moderation datasets we consider proved more difficult than our general text classification datasets for all models, despite typically having fewer labels. it is imperative that we develop reliable and practical text classifiers for content moderation, such that we can inexpensively re-tune them for novel forms of hate speech, toxicity, and fake news. our results suggest that lagonn exp , a relatively expensive technique, can detect harmful content when dealing with imbalanced label distributions, as is common with realistic datasets. this is intuitive from the perspective that less common instances are more difficult to learn and require more effort. an exception would be our examination of insincere questions, where lagonn cheap excelled in the extreme and balanced settings. this demonstrates that if we choose our plm with care for related downstream tasks, lagonn can inexpensively extract pretrained knowledge and improve performance without the need for costly finetuning. indeed, considering the performance of set-fit suggests that, in this case, fine-tuning hurts performance and we actually overfit. however, even here, our proposed modifications with lagonn increase model robustness and lessen the effects of overfitting. fine-tuning with setfit hurts performance on more balanced datasets that are not few-shot. we have observed that setfit should not be applied "out of the box" to balanced, non-few-shot data. this can be detrimental to performance, directly affecting our own approach. however, lagonn can stabilize setfit's predictions and reduce its performance drop in many cases. figures 5 , 3 , and 4 show that when the label distribution is moderate or balanced (see for balanced data, it is sufficient to fine-tune the sentence transformer via setfit with 50 to 100 examples per label, while 150 to 200 instances appear to be sufficient when the training data are moderately balanced. the encoder can then be frozen and all available data embedded to train a classifier. this is more performant and efficient than full-model fine-tuning. lagonn is applicable to this case, inexpensively boosting and stabilizing setfit's performance. all models fine-tuned on hate speech offensive exhibited similar, upwardtrending learning curves, but we note the speed of lagonn relative to roberta f ull or setfit exp (see figure 3 and ## conclusion we have proposed lagonn, an inexpensive modification to setfit. lagonn improves setfit's performance by modifying text with the nearest neighbors in the training data. to demonstrate the merit of lagonn, we examined text classification systems for content moderation with different label distributions and for general and multilingual classification. we studied 17 datasets with growing training data. when the training labels are imbalanced, expensive systems, such as lagonn exp are performant. lagonn exp also excels on balanced datasets with many labels. however, when the labels are binary or ternary, typical for content moderation, and the distribution is balanced, fine-tuning with setfit yields minimal gains. we therefore proposed an alternative but strong training procedure. lagonn is a practical method for detecting harmful content and text classification.