Instances and Labels: Hierarchy-aware Joint Supervised Contrastive Learning for Hierarchical Multi-Label Text Classification (2023)

Abstract: Hierarchical multi-label text classification (HMTC) aims at utilizing a label hierarchy in multi-label classification. Recent approaches to HMTC deal with the problem of imposing an overconstrained premise on the output space by using contrastive learning on generated samples in a semi-supervised manner to bring text and label embeddings closer. However, the generation of samples tends to introduce noise as it ignores the correlation between similar samples in the same batch. One solution to this issue is supervised contrastive learning, but it remains an underexplored topic in HMTC due to its complex structured labels. To overcome this challenge, we propose **HJCL**, a **H**ierarchy-aware **J**oint Supervised **C**ontrastive **L**earning method that bridges the gap between supervised contrastive learning and HMTC. Specifically, we employ both instance-wise and label-wise contrastive learning techniques and carefully construct batches to fulfill the contrastive learning objective. Extensive experiments on four multi-path HMTC datasets demonstrate that HJCLachieves promising results and the effectiveness of Contrastive Learning on HMTC. Code and data are available at https://github.com/simonucl/HJCL.

## introduction text classification is a fundamental problem in natural language processing (nlp), which aims to assign one or multiple categories to a given document based on its content. the task is essential in many nlp applications, e.g. in discourse relation recognition @xcite , scientific document classification @xcite , or e-commerce product categorization @xcite . in practice, documents might be tagged with multiple categories that can be organized in a concept hierarchy, such as a taxonomy of a knowledge graph @xcite , cf. figure 1 . the the british government raised the nation's base lending rate a quarter of a point yesterday, to 6 percent. the bank of england uses for loans to commercial banks, since february 1995 … his target for the underlying rate of inflation --excluding mortgage interest payments --is 2.5 percent by next spring, but the rate has been about 2.8 percent recently. new york times report root news classifieds world sport opinion job categories banking, finance and insurance countries united kingdom europe france marketing, pr ## experiments datasets and evaluation metrics we conduct experiments on four widely-used hmtc benchmark datasets, all of them consisting of multi-path labels: blurb genre collection (bgc) 2 , arxiv academic papers dataset (aapd) @xcite , ny-times (nyt) @xcite , and rcv1-v2 (lewis et al.). details for each dataset are shown in ## conclusion we introduce hjcl , a combination of two novel contrastive methods that better learn the representation for embedding in hierarchical multi-label text classification (hmtc). our method has the following features: (1) it demonstrates that contrastive learning can help retain the hierarchy information between samples. in the future, we plan to look into applying our approach in some special kinds of texts, such as arguments @xcite @xcite , news @xcite @xcite and events @xcite . furthermore, we will also further develop our approach in the setting of multi-modal @xcite @xcite classification, involving both texts and images.