Robust Representation Learning with Reliable Pseudo-labels Generation via Self-Adaptive Optimal Transport for Short Text Clustering (2023)

Abstract: Short text clustering is challenging since it takes imbalanced and noisy data as inputs. Existing approaches cannot solve this problem well, since (1) they are prone to obtain degenerate solutions especially on heavy imbalanced datasets, and (2) they are vulnerable to noises. To tackle the above issues, we propose a Robust Short Text Clustering (RSTC) model to improve robustness against imbalanced and noisy data. RSTC includes two modules, i.e., pseudo-label generation module and robust representation learning module. The former generates pseudo-labels to provide supervision for the later, which contributes to more robust representations and correctly separated clusters. To provide robustness against the imbalance in data, we propose self-adaptive optimal transport in the pseudo-label generation module. To improve robustness against the noise in data, we further introduce both class-wise and instance-wise contrastive learning in the robust representation learning module. Our empirical studies on eight short text clustering datasets demonstrate that RSTC significantly outperforms the state-of-the-art models.

## introduction text clustering, one of the most fundamental tasks in text mining, aims to group text instances into clusters in an unsupervised manner. it has been proven to be beneficial in many applications, such as, recommendation system @xcite , opinion mining @xcite , stance detection @xcite , etc. with the advent of digital era, more and more people enjoy sharing and discovering various of contents on the web, where short text is an import form of information carrier. therefore, it is helpful to utilize short text clustering for mining valuable insights on the web. ## experiment in this section, we conduct experiments on several real-world datasets to answer the following questions: (1) rq1: how does our approach perform compared with the state-of-the-art short text clustering methods? (2) rq2: how do the saot, and the two contrastive losses contribute to the performance improvement? (3) rq3: how does the performance of rstc vary with different values of the hyper-parameters? ## conclusion in this paper, we propose a robust short text clustering (rstc) model, which includes pseudo-label generation module and robust representation learning module. the former generates pseudo-labels as the supervision for the latter. we innovatively propose saot in the pseudo-label generation mod-ule to provide robustness against the imbalance in data. we further propose to combine classwise contrastive learning with instance-wise contrastive learning in the robust representation learning module to provide robustness against the noise in data. extensive experiments conducted on eight real-world datasets demonstrate the superior performance of our proposed rstc.