META : Metadata-Empowered Weak Supervision for Text Classification (2020)

Abstract: Recent advances in weakly supervised learning enable training high-quality text classifiers by only providing a few user-provided seed words. Existing methods mainly use text data alone to generate pseudo-labels despite the fact that metadata information (e.g., author and timestamp) is widely available across various domains. Strong label indicators exist in the metadata and it has been long overlooked mainly due to the following challenges: (1) metadata is multi-typed, requiring systematic modeling of different types and their combinations, (2) metadata is noisy, some metadata entities (e.g., authors, venues) are more compelling label indicators than others. In this paper, we propose a novel framework, META, which goes beyond the existing paradigm and leverages metadata as an additional source of weak supervision. Specifically, we organize the text data and metadata together into a text-rich network and adopt network motifs to capture appropriate combinations of metadata. Based on seed words, we rank and filter motif instances to distill highly label-indicative ones as “seed motifs”, which provide additional weak supervision. Following a bootstrapping manner, we train the classifier and expand the seed words and seed motifs iteratively. Extensive experiments and case studies on real-world datasets demonstrate superior performance and significant advantages of leveraging metadata as weak supervision.

## introduction weakly supervised text classification has recently gained much attention from the researchers because it reduces the burden of annotating the data. so far, the major source of weak supervision lies in text data itself @xcite @xcite @xcite . these methods typically require a few user-provided seed p 2 p 1 … … … (b) a text-rich network view of the papers. (c) a motif pattern and a motif instance. words for each class as weak supervision. they expand seed words with generated pseudo labels and improve their text classifier in an iterative fashion. metadata information (e.g., author, published year) in addition to textual information, is widely available across various domains (e.g., news articles, social media posts, and scientific papers) and it could serve as a strong, complementary weak supervision source. take a look at the research papers in figure 1 (a) as an example. it shall be learned in a data-driven manner that g. hinton is a highly-reputed machine learning researcher, thus his presence is a strong indicator of a paper belonging to the machine learning category. distilling effective metadata for weak supervision faces several major challenges. metadata is often multi-typed, each type and the type combinations could have very different semantics and may not be equally important. moreover, even entities within a single metadata type could be noisy. continuing our example in figure 1 (a), we shall notice that year is less helpful than an author to do classification. among the authors, j. dean might be an important figure but has research interests m 1,1 .95 .01 .04 m 1,2 .32 .30 .38 ## our meta framework as shown in figure 2 , meta is an iterative framework, generating pseudo labels and training the text classifier alternatively, similar to many other weakly supervised text classification methods @xcite @xcite . one iteration in meta consists of the following steps: the novelty of meta mainly lies in integrating two sources of weak supervisions, seed motif instances, and seed words. given each motif instance m or each word w, for each label l, we estimate a ranking score r m,l or r w,l ranging between 0 and 1, measuring how label-indicative it is to the particular label l. such ranking scores are utilized to select new seed motif instances and seed words. note that, while this selection is conducted at the instance level, it also selects motif patterns implicitly and therefore ensures robust performance when users provide some irrelevant motif patterns. ## experiments in this section, we evaluate meta and compare it with existing techniques on two real-world datasets in a weakly supervised classification setting. ## related work we review the literature about (1) weakly supervised text classification methods, (2) text classification with metadata, and (3) document classifiers. ## conclusion and future work in this paper, we propose meta, a novel framework that leverages metadata information as an additional source of weak supervision and incorporates it into the classification framework. our method organizes the text data and metadata together into a text-rich network and employs motif patterns to capture appropriate metadata combinations. using the initial user-provided seed words and motif patterns, our method generates pseudo labels, trains classifier, and ranks and filters highly label-indicative words, motifs in a unified manner and adds them to their respective seed set. experimental results and case studies demonstrate that our model outperforms previous methods significantly, thereby signifying the advantages of leveraging metadata as weak supervision. in the future, we are interested in effectively integrating different forms of supervision including annotated documents. also, we only consider positively label-indicative metadata combinations currently. there should be negatively label-indicative combinations as well which can eliminate some classes from potential labels. this is another potential direction for the extension of our method.