A Generic Method for Fine-grained Category Discovery in Natural Language Texts (2024)

Abstract: Fine-grained category discovery using only coarse-grained supervision is a cost-effective yet challenging task. Previous training methods focus on aligning query samples with positive samples and distancing them from negatives. They often neglect intra-category and inter-category semantic similarities of fine-grained categories when navigating sample distributions in the embedding space. Furthermore, some evaluation techniques that rely on pre-collected test samples are inadequate for real-time applications. To address these shortcomings, we introduce a method that successfully detects fine-grained clusters of semantically similar texts guided by a novel objective function. The method uses semantic similarities in a logarithmic space to guide sample distributions in the Euclidean space and to form distinct clusters that represent fine-grained categories. We also propose a centroid inference mechanism to support real-time applications. The efficacy of the method is both theoretically justified and empirically confirmed on three benchmark tasks. The proposed objective function is integrated in multiple contrastive learning based neural models. Its results surpass existing state-of-the-art approaches in terms of Accuracy, Adjusted Rand Index and Normalized Mutual Information of the detected fine-grained categories. Code and data are publicly available at https://github.com/changtianluckyforever/F-grained-STAR.

## introduction fine-grained analysis has drawn much attention in many artificial intelligence fields, e.g., computer vision @xcite @xcite and natural language processing @xcite @xcite , because it can provide more detailed features than coarse-grained data. for instance, as illustrated in figure 1 , solely i want to buy a vehicle suited for weekend field adventures. intent: buy_vehicle i suggest purchasing a porsche roadster. intent: buy_off-road i recommend buying an off-road vehicle. based on coarse-grained analysis, the chatbot might incorrectly recommend a roadster, which is unsuitable for field adventures. detecting the finegrained intent would allow the chatbot to recommend an off-road vehicle that aligns with the user's requirements. however, annotating fine-grained categories can be labor-intensive, as it demands precise expert knowledge specific to each domain and involved dataset. addressing this challenge, @xcite recently introduced fine-grained category discovery under coarse-grained supervision (fcdc) for language classification tasks (details in section 3). solving fcdc tasks can significantly benefit numerous practical applications, for example, fine-grained classification of enterprise documents @xcite , fine-grained dialogue intent detection tasks @xcite , product labeling on online shopping websites based on text descriptions @xcite , and so on. fcdc aims to reduce annotation costs by leveraging the relative ease of obtaining coarse-grained annotations, without re-quiring fine-grained supervisory information. this approach has sparked significant research interest in the automatic discovery of fine-grained language categories @xcite @xcite . existing methods for addressing fcdc are typically grouped into three groups @xcite : language models, self-training methods, and contrastive learning methods. language models @xcite , including their fine-tuned versions with coarse labels, generally perform poorly on this task due to a lack of finegrained supervision. self-training methods @xcite and their variants often employ clustering assignments as fine-grained pseudo-labels, filtering out some noisy pseudolabels, and training with these labels. dominant contrastive learning methods @xcite @xcite typically identify positive and negative samples for a given query by measuring their semantic distances. the contrastive loss ensures that the query sample moves closer to positive samples and further away from negative samples. so these methods form clusters of samples in the embedding space, with each cluster representing a discovered fine-grained category, without requiring fine-grained category supervision. however, past methods did not utilize comprehensive semantic similarities (css) in the logarithmic space to guide sample distributions in the euclidean space. we define css as the finegrained semantic similarities measured by bidirectional kullback-leibler (kl) divergence in the logarithmic space between the query sample and each available positive or negative sample. although @xcite recently explored similarities measured by rank order between the query sample and positive samples, they ignore similarities with negative samples. we propose a method (star) for detecting fine-grained clusters of semantically similar texts through a novel objective function, with the core component considering css. this component guides sample distributions in the euclidean space based on the magnitude of css in the logarithmic space. large semantic differences (low similarity) in the logarithmic space between the query sample and an available sample push the query sample further away in the euclidean space, while small semantic differences bring the query sample closer to the available sample. thus, samples form dis-tinguishable fine-grained clusters in the euclidean space, with each cluster representing a discovered category. additionally, clustering inference used by previous works @xcite @xcite cannot support real-time scenarios, so we propose a variant inference mechanism utilizing approximated finegrained cluster centroids, delivering competitive results for the tasks considered. our main contributions in this work can be summarized as follows: ## problem formulation given a set of coarse-grained categories y @xmath0, c 2 , . . . , c m } and a coarsely labeled training set ## method star leverages comprehensive semantic similarities and integrates seamlessly with contrastive learning baselines by modifying the objective function. we have developed variants for three baselines: pseudoprototypicalnet (ppnet) @xcite , dna @xcite , and down @xcite . this section focuses on star-down because down outperforms other baselines, with additional method variants detailed in appendix a.3. down involves three steps: pre-training with coarse-grained labels (section 4.1), retrieving and weighting nearest neighbors (section 4.2), and training with a contrastive loss. star-down follows the same first two steps but replaces the third with a novel objective function (section 4.3). like down, star-down iterates the last two steps until the unsupervised metric, the silhouette score of the clustering into fine-grained clusters, does not improve for five consecutive epochs. the detailed algorithm is provided in appendix a.1.7. ## conclusion we propose the star method for fine-grained category discovery in natural language texts, which utilizes comprehensive semantic similarities in the logarithmic space to guide the distribution of textual samples, including conversational intents, scientific paper abstracts, and assistant queries, in the euclidean space. star pushes query samples further away from negative samples and brings them closer to positive samples based on the comprehensive semantic similarities magnitude. this process forms compact clusters, each representing a discovered category. we theoretically analyze the effectiveness of star method. additionally, we introduce a centroid inference mechanism that addresses previous gaps in real-time evaluations. experiments on three natural language benchmarks demonstrate that star achieves new state-of-theart performance in fine-grained category discovery tasks for text classification.