FNSCC : Fuzzy Neighborhood-Aware Self-Supervised Contrastive Clustering for Short Text (2025)

Abstract: Short texts pose significant challenges for clustering due to semantic sparsity, limited context, and fuzzy category boundaries. Although recent contrastive learning methods improve instance-level representation, they often overlook local semantic structure within the clustering head. Moreover, treating semantically similar neighbors as negatives impair cluster-level discrimination. To address these issues, we propose Fuzzy Neighborhood-Aware Self-Supervised Contrastive Clustering (FNSCC) framework. FNSCC incorporates neighborhood information at both the instance-level and cluster-level. At the instance-level, it excludes neighbors from the negative sample set to enhance inter-cluster separability. At the cluster-level, it introduces fuzzy neighborhood-aware weighting to refine soft assignment probabilities, encouraging alignment with semantically coherent clusters. Experiments on multiple benchmark short text datasets demonstrate that FNSCC consistently outperforms state-of-the-art models in accuracy and normalized mutual information. Our code is available at https://github.com/zjzone/FNSCC .

## introduction text clustering, a fundamental task in natural language processing, seeks to uncover the latent semantic structure of text without relying on prior annotations, and provides theoretical support for downstream applications @xcite . its significance is particularly evident in domains such as information retrieval @xcite , knowledge discovery @xcite , and natural language understanding @xcite . text clustering is inherently challenging due to high-dimensional sparsity, semantic ambiguity, and class imbalance. effective clustering requires capturing multi-level semantic features. traditional * corresponding author methods such as bag-of-words (bow) and tf-idf rely on word frequency and lack contextual awareness. word2vec improves semantic representation by learning context-aware embeddings through predictive objectives and efficient training strategies like negative sampling. bert @xcite and other pre-trained language models (plms) leverage global attention and positional encoding to capture long-range dependencies. however, prior work @xcite has shown that their sentence representations perform poorly in clustering tasks. this limitation stems from the fact that bert is pre-trained with objectives such as masked language modeling (mlm) and next sentence prediction (nsp), which are not designed to capture global semantic categories or learn clustering-friendly representations. to overcome this, sccl @xcite proposes a deep clustering framework that fine-tunes plms with joint contrastive and clustering objectives, improving semantic separability and explicitly modeling cluster structures for better clustering performance. since sccl aims to generate high-confidence sample partitions that align with the underlying clustering structure. however, it relies solely on the euclidean distance between text instance embeddings and cluster centers to compute soft cluster assignments, which can be susceptible to misjudgments caused by fuzzy noise. this limitation prevents high-confidence samples from receiving sufficient emphasis during training. additionally, in the contrastive learning module, neighboring samples often have higher semantic correlation. if only the corresponding augmented instances are taken as positive examples, these semantically similar samples are considered negative samples, and the model force the distance between them to be increased, thereby destroying the clustering structure. to address the aforementioned issues, we propose the fuzzy neighborhood-aware self-supervised contrastive clustering (fnscc) framework for short text. the primary goal is to enhance the clustering structure and instance contrastive module of sccl by incorporating fuzzy neighborhood information. specifically, we first construct a more balanced distribution of positive and negative samples in the instance contrastive module to prevent semantically similar samples from being classified as negative examples. additionally, we leverage the neighborhood semantic information of the current sample to improve the expressiveness of the local context in the clustering head through a fuzzy weighting mechanism, thereby mitigating the impact of single-point noise on soft clustering. in summary, the main contributions of this paper are as follows: ## the proposed fnscc method in this section, we provide a detailed description of the fnscc method. section 3.1 outlines the general framework and clustering objectives of fn-scc. sections 3.2-3.3 elaborate on the primary contributions of this work to contrastive clustering, specifically the fuzzy neighborhood-enhanced clustering mechanism, which includes both instancelevel and cluster-level modules. ## experiments in this section, we conduct experiments on several real-world short text datasets to demonstrate the contribution of our method. ## conclusion this paper proposes a fuzzy neighborhood-aware self-supervised contrastive clustering (fnscc) framework, comprising instance-level contrastive learning with neighborhood context and fuzzy neighborhood-aware clustering. it introduces two key innovations: leveraging neighborhood information to improve clustering confidence and reduce ambiguity, and removing false negatives within neighborhoods to enhance cluster-level discrimination, yielding a more suitable sample distribution for clustering. experimental results show that fnscc outperforms other state-of-the-art models in terms of acc and nmi on most short text datasets. ablation studies further confirm the effectiveness of fuzzy neighborhood information in enhancing contrastive representation learning and clustering performance.