F ast C lass: A Time-Efficient Approach to Weakly-Supervised Text Classification (2022)

Abstract: Weakly-supervised text classification aims to train a classifier using only class descriptions and unlabeled data. Recent research shows that keyword-driven methods can achieve state-of-the-art performance on various tasks. However, these methods not only rely on carefully-crafted class descriptions to obtain class-specific keywords but also require substantial amount of unlabeled data and takes a long time to train. This paper proposes FastClass, an efficient weakly-supervised classification approach. It uses dense text representation to retrieve class-relevant documents from external unlabeled corpus and selects an optimal subset to train a classifier. Compared to keyword-driven methods, our approach is less reliant on initial class descriptions as it no longer needs to expand each class description into a set of class-specific keywords.Experiments on a wide range of classification tasks show that the proposed approach frequently outperforms keyword-driven models in terms of classification accuracy and often enjoys orders-of-magnitude faster training speed.

## introduction text classification is one of the most used techniques in mining large-scale unstructured text. when sufficient labeled data are available, supervised classification techniques can achieve excellent performance. however, manually labeling example documents can be time-consuming and labor-intensive, a major burden when applying supervised text classification techniques in practice. recently, weakly-supervised text classification @xcite @xcite @xcite @xcite has been proposed to save labeling efforts. it refers to the ability for a machine learning model to start classifying documents by using only class descriptions and unlabeled data. since weakly-supervised text classification methods do not rely on any labeled data, it can greatly reduce the workload of data labeling. therefore, these methods are desirable in real-world text mining applications. the current mainstream weakly-supervised text classification methods are keyword-driven @xcite @xcite @xcite @xcite , where the users need to carefully choose initial class-relevant keywords for each class. such keywords are expanded into a richer set of keywords, which are then used to assign pseudolabels on unlabeled data. the critical step of these keyword-driven methods is the choice of initial class-relevant keywords and the subsequent keyword expansion strategy, which determines the quality of pseudo-labeled documents. however, if the user's initial choice of class descriptions is ambiguous or brief, then the results of keyword expansion will be negatively impacted. ideally, the user can provide initial class descriptions that are beneficial to keywords expansion. however, in reality, the quality of class descriptions is not guaranteed. early research often artificially modifies original class descriptions to improve the quality of the initial keywords @xcite . in sentiment classification tasks, "positive" and "negative" class descriptions have to be replaced by "good" and "bad", which are easier for keywords expansion @xcite . previous weakly-supervised methods also face the common problem of fully supervised methods. that is, model performance depends on the amount of (unlabeled) data available. widely used classification datasets in research literature often have high-quality class descriptions and sufficient unlabeled data. however, weakly-supervised classification methods are most needed in the early stage of practical text mining tasks, where practitioners are not ready to invest vast amounts of efforts in data labeling or computing resources in model training. in this stage, class descriptions can be crude and even unlabeled data can be scarce (e.g., classifying text related to an emerging event). as we will show in our experiments, the increasingly sophisticated keyword-driven weaklysupervised methods require increasing amount of computing resources and time in model training, and yet the resulting performance benefits can be minimal. this is concerning given the recent recommendation on green ai @xcite and efficient nlp @xcite . recent research found that transformer-based textual entailment models can provide more competitive performance on dataless classification tasks @xcite which is similar to weaklysurpervised text classification. in principle, such models only need to be trained once and can be applied to any classification tasks. however, it is not only difficult to adapt such models to a specific corpus, but also inefficient to run them at prediction time. that is, one has to run the entailment model k times to classify one document into k categories. the prediction speed slows down as more categories (larger k) are considered in a task. in this paper, we propose fastclass, a timeefficient weakly-supervised text classification model which constructs a classification model by selecting a optimal subset of retrieved documents to solve the above problems. fastclass has three steps: (1) extracting seed unlabeled documents with high semantic similarity to the original class descriptions; (2) using seed documents as queries to obtain pseudo-labeled documents from external corpus. (3) selecting an optimal subset of pseudolabels using the maximum entropy principle. our main contributions are as follows: ## related work we discuss related work from three perspectives: zero-shot text classification, dataless text classification, and weakly-supervised text classification. ## proposed methods in this section, we describe our proposed method fastclass for weakly-supervised text classification. we formulate the problem as follows. we are given a set of class descriptions at a high level, our proposed method uses class descriptions in d to obtain task-specific documents that are highly similar to class descriptions from x and uses these task-specific documents as queries to retrieve pseudo-labeled documents from external unlabeled data u . then we select the optimal subset of these pseudo-labeled documents to train a classifier. the general fastclass framework is presented in figure 1 . below we describe our method in detail. ## results in this section, we evaluate our proposed method and compare it with baseline models for dataless and weakly-supervised text classification. the comparison is not only in terms classification accuracy, but also model efficiency. ## conclusion we proposed a weakly-supervised text classification method fastclass which selects a document subset returned by dense retrieval models for classifier training. compared to keyword-driven methods, since our approach no longer needs to expand each class description into a set of class-specific keywords, it is less reliant on initial class descriptions. extensive experiments show that the proposed method is able to achieve competitive classification performance which does not require highquality initial class descriptions and often saves significant training time.