DNA : Denoised Neighborhood Aggregation for Fine-grained Category Discovery (2023)

Abstract: Discovering fine-grained categories from coarsely labeled data is a practical and challenging task, which can bridge the gap between the demand for fine-grained analysis and the high annotation cost. Previous works mainly focus on instance-level discrimination to learn low-level features, but ignore semantic similarities between data, which may prevent these models learning compact cluster representations. In this paper, we propose \textit{Denoised Neighborhood Aggregation} (DNA), a self-supervised framework that encodes semantic structures of data into the embedding space. Specifically, we retrieve \textit{k} -nearest neighbors of a query as its positive keys to capture semantic similarities between data and then aggregate information from the neighbors to learn compact cluster representations, which can make fine-grained categories more separatable. However, the retrieved neighbors can be noisy and contain many false-positive keys, which can degrade the quality of learned embeddings. To cope with this challenge, we propose three principles to filter out these false neighbors for better representation learning. Furthermore, we theoretically justify that the learning objective of our framework is equivalent to a clustering loss, which can capture semantic similarities between data to form compact fine-grained clusters. Extensive experiments on three benchmark datasets show that our method can retrieve more accurate neighbors (21.31% accuracy improvement) and outperform state-of-the-art models by a large margin (average 9.96% improvement on three metrics). Our code and data are available at https://github.com/Lackel/DNA.

## introduction many ai fields have progressed into fine-grained analysis, e.g., computer vision @xcite * corresponding author. right: label hierarchy with coarse-and fine-grained categories. @xcite and natural language processing @xcite @xcite , since it can provide much more information than coarsegrained analysis. for example, detecting more fine-grained user intents can help to provide more accurate recommendation and better services for customers (figure 1 left). however, labelling fine-grained categories can be time-consuming and labour-intensive since it requires more expert knowledge. to get out of this dilemma, a novel task called fine-grained category discovery under coarse-grained supervision (fcdc) was recently proposed by @xcite . taking figure 1 right as an example, fcdc aims at discovering fine-grained categories (e.g., desktop and tennis) using only coarse-grained (e.g., computer and sports) labeled data which are easier and cheaper to annotate. to solve the fcdc task, previous methods mainly focus on instance-level discrimination to learn low-level features through contrastive learning @xcite . despite the improved performance, these instancebased methods fail to encode cluster-level semantic structures of data. this is because these methods simply treat each instance as a single class and push away other instances, regardless of their se-mantic similarities @xcite , which can hinder the formation of compact fine-grained clusters. here we define 'compact' as samples with the same fine-grained categories are compactly clustered into the center of category and away from other samples from different fine-grained categories, which means smaller intra-class distance and larger interclass distance. since samples located around decision boundaries are easily misclassified into wrong categories, distributing samples near the category center compactly can avoid overlapping decision boundaries and make these categories more distinguishable. so learning compact cluster representations is important for the fcdc task to learn more separable fine-grained categories. to encode semantic structures of data to learn more compact cluster representations, we propose a novel model named denoised neighborhood aggregation (dna). dna can capture semantic similarities between data by retrieving k-nearest neighbors of a query and aggregating information from them. however, the retrieved neighbors can be noisy and contain many false-positive keys (i.e., keys with different fine-grained categories from the query), which can reduce the quality of representation learning. this situation is more severe in the fcdc setting since pretraining on coarsegrained labels can easily include wrong neighbors for those samples with the same coarse-grained labels but different fine-grained ones. to solve this problem, we propose three principles (named label constraint, reciprocal constraint, and rank statistic constraint) to filter out these false neighbors. these constraints consider bidirectional semantic structures and statistical features of data to help to retrieve more accurate neighbors. furthermore, we interpret our framework from a generalized expectation-maximization (em) perspective. at the e-step, we retrieve reliable neighbors from a dynamic queue under the proposed constraints, then at the m-step, we perform neighborhood aggregation to encode semantic structures of data to learn more compact representations. last but not least, we theoretically prove that the learning objective of our model is equivalent to a clustering loss, which can help to learn compact cluster representations to facilitate fine-grained category discovery. our main contributions can be summarized as follows: ## conclusion in this paper, we propose denoised neighborhood aggregation (dna), a self-supervised learning framework that iteratively performs representation learning and neighborhood retrieval for finegrained category discovery. we further propose three principles to filter out false-positive neighbors for better representation learning. then we interpret our model from a generalized em perspective and theoretically justify that the learning objective of our model is equivalent to a clustering loss, which can encode semantic structures of data to form compact clusters. extensive experiments on three benchmark datasets show that our model can retrieve more accurate neighbors and outperform state-of-the-art models by a large margin.