Introduction
Text categorization, also known as document categorization, is a natural language processing (NLP) task that involves arranging texts into coherent groups based on their content. It has many applications such as spam detection Jindal and Liu ( 2007 ) , sentiment analysis Melville et al. ( 2009 ) , content recommendation Pazzani and Billsus ( 2007 ) , etc. There are two main approaches to text categorization: classification (supervised learning) and clustering (unsupervised learning). In text classification, the process involves training a model using a labeled dataset, where each document is associated with a specific category. The model learns patterns and relationships between the text features and the corresponding categories during the training phase. Text clustering, however, aims to group similar documents together without prior knowledge of their categories. Unlike text classification, clustering does not require labeled data. Instead, it focuses on finding inherent patterns and similarities in the text data to create clusters.

In the field of NLP, pretrained models have attained state-of-the-art performances in a variety of tasks Devlin et al. ( 2019 ); Liu et al. ( 2019 ); Reimers and Gurevych ( 2019 ) , one of which is text classification. In spite of that, text clustering using such models did not garner significant attention. To this day most text clustering techniques use the representations of texts generated by some pretrained model such as Sentence-BERT Reimers and Gurevych ( 2019 ) and often use classical clustering approaches such as k-means to obtain a partition of the texts. This is done without any fine-tuning due to the unsupervised nature of the clustering problem.

Recently, graph filtering has appeared as an efficient and effective technique for learning representations for attributed network nodes. The effectiveness of this technique has made it a backbone for popular deep learning architectures for graphs such as the graph convolutional network (GCN) Kipf and Welling ( 2016 ) . Simplified versions of this deep architecture have been proposed wherein the learning of large sets of weights has been deemed unnecessary. Their representation learning scheme works similar to Laplacian smoothing and, by extension, graph filtering. We can give as examples of these simplified techniques the simple graph convolution (SGC) Wu et al. ( 2019 ) , and the simple spectral graph convolution (S²GC) Zhu and Koniusz ( 2020 ) . Some researchers used GCNs for the task of text classification. Yao et al. ( 2019 ) proposed TextGCN which is GCN with a custom adjacency matrix built from word PMI and the TF-IDF of the documents with the attributes being word count vectors. Lin et al. ( 2021 ) proposed BertGCN which is similar to TextGCN with the difference that they use BERT representation for the GCN and combine their training losses. The issue is that these approaches are not suitable for learning unsupervised representations since labels are needed. This is a significant limitation towards their use in unsupervised tasks. Recently some graph-based unsupervised approaches were proposed to deal with text data represented using document-term matrices Fettal et al. ( 2022 , 2023 ) .

In this paper, we propose to use the concept of graph smoothing/filtering, which is the main component accredited with the success of GCNs Defferrard et al. ( 2016 ); Kipf and Welling ( 2016 ); Li et al. ( 2018 ) , to semantically "fine-tune" the representations obtained via sentence embedding models to help traditional clustering and classification algorithms better distinguish between semantically different texts and group together texts which have similar meanings, all in an unsupervised manner. To do this, we build a graph with respect to the text which describes the semantic similarity between the different documents based on the popular cosine similarity measure. Our approach yields almost systematic improvement when using filtering on the textual representations as opposed to using them without filtering in both facets of document categorization: classification and clustering. Experiments on eight popular benchmark datasets support these observations.

The code for the experiments is available at 1 1 1 https://github.com/chakib401/smoothing_sentence_embeddings .