Title: Deep Taxonomic Networks for Unsupervised Hierarchical Prototype Discovery
Venue: NeurIPS 2025
Accepted: yes
Keywords: unsupervised learning

hierarchical clustering

representation learning

latent variable models

prototype discovery
Abstract: Inspired by the human ability to learn and organize knowledge into hierarchical taxonomies with prototypes, this paper addresses key limitations in current deep hierarchical clustering methods. 
Existing methods often tie the structure to the number of classes and underutilize the rich prototype information available at intermediate hierarchical levels. 
We introduce deep taxonomic networks, a novel deep latent variable approach designed to bridge these gaps.
Our method optimizes a large latent taxonomic hierarchy, specifically a complete binary tree structured mixture-of-Gaussian prior within a variational inference framework, to automatically discover taxonomic structures and associated prototype clusters directly from unlabeled data without assuming true label sizes.
We analytically show that optimizing the ELBO of our method encourages the discovery of hierarchical relationships among prototypes. 
Empirically, our learned models demonstrate strong hierarchical clustering performance, outperforming baselines across diverse image classification datasets using our novel evaluation mechanism that leverages prototype clusters discovered at all hierarchical levels.
Qualitative results further reveal that deep taxonomic networks discover rich and interpretable hierarchical taxonomies, capturing both coarse-grained semantic categories and fine-grained visual distinctions.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce a novel deep hierarchical clustering algorithm, where cluster hierarchies are defined through learned prototypes. This approach is conceptually interesting and demonstrates a degree of novelty. However, the empirical validation is currently limited. In particular, the paper lacks strong baselines and evaluations on complex image datasets, which hinders a thorough assessment of the method's effectiveness [1]. Incorporating additional deep flat clustering methods and testing on more challenging datasets would significantly strengthen the experimental section and better highlight the practical significance and robustness of the proposed approach.\n\n[1] Lu, Yiding, et al. \"A survey on deep clustering: from the prior perspective.\" Vicinagearth 1.1 (2024): 4.\n\nSTRENGTHS AND WEAKNESSES:\nThe technique is somewhat new in the context of deep clustering. It is theoretically/statistically well founded. I believe that the central promise and selling (\"deep taxonomic networks discover rich and interpretable hierarchical taxonomies, capturing both coarse-grained semantic categories and fine-grained visual distinctions\") would need more substantiation.\n\nAlthough the proposed prototype-based hierarchical clustering framework is conceptually novel, but its empirical validation is limited. The absence of strong baselines and the use of only simple datasets hinder a clear assessment of the method's effectiveness. Moreover, the lack of comparison to related approaches such as deep flat clustering further weakens the evaluation. Including more competitive methods and complex datasets would significantly improve the work’s credibility.\n\nQUESTIONS:\nYour experimental section includes comparisons with TreeVAE and DeepECT. However, it omits more recent and advanced deep clustering methods, such as SCAN. Could you clarify  the rationale behind excluding these methods from your comparison, and  how your approach performs when applied to more complex patterns, such as those found in  STIL-10 or Image-net?\n\nLIMITATIONS:\nThe central promise and selling (\"deep taxonomic networks discover rich and interpretable hierarchical taxonomies, capturing both coarse-grained semantic categories and fine-grained visual distinctions\") would need more substantiation.\nFurthermore, the evaluation lacks recent and stronger baselines such as SCAN, and does not address performance on complex datasets like  STIL-10 or Image-net, leaving the method’s  competitiveness unclear.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a deep hierarchical clustering method - deep taxonomic networks (DeepTaxonNet).  In contrast to existing methods such as DeepECT and TreeVAE, DeepTaxonNet puts more emphasis on the intermediate nodes, allowing clusters associated with every node in the tree to be a potential source for generating an observation.  The tree structure is defined as a complete binary tree with an associated hierarchical mixture of Gaussians prior.  Distributions at parent nodes are constrained to be a convex combination of its children's distributions.  Variational inference is performed by maximizing the Evidence Lower Bound, and a derivation is given that shows that maximizing this bound encourages maximization of Categorical Utility.  Comparison is given on several data sets with the existing methods of Cobweb, DeepECT, and TreeVAE, where the proposed method gives better performance using the proposed metrics in the paper that account for the explicit use of intermediate nodes.  Additionally, DeepTaxonNet allows for adaptation to a new data set without re-training, and is shown to give better results on this task compared with retraining TreeVAE.\n\nSTRENGTHS AND WEAKNESSES:\nstrengths:\n\n- evaluation with existing methods on multiple data sets, ablation study given in appendix, demonstration of adaptation to new task without retraining\n\nweaknesses:\n\n- it seems the main contribution/novelty of the proposed method is the more explicit use of non-leaf nodes as having their own generating distributions and serving as potential prototypes.  and this comes at the cost of having a rigid pre-defined binary tree structure.  \n- in that regard then, I would have liked to see more analysis of the benefits of this approach.  in part because the evaluation metrics have been adapted to reflect the use of intermediate nodes, so from a comparison standpoint are not completely fair to the existing methods that are only designed to use the leaf nodes.\n- for instance, new task classification without re-training seems like an advantage, but it would be nice to see more experiments along those lines\n- the qualitative analysis is 5.3 is interesting, but there is not an explicit comparison with existing methods, and in any case more quantitative analysis of the benefits of using the intermediate nodes would be more convincing.\n\nQUESTIONS:\nPlease see the weaknesses section above.  With regard to that section, would it be fair to say the main advantage of the proposed approach and leveraging the intermediate nodes is that the level of granularity does not need to be pre-defined, and so the same learned taxonomy can be used for varying tasks without re-training?  Are there additional benefits that you see beyond this?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents *Deep Taxonomic Networks* (DTN), a novel deep latent variable framework for unsupervised hierarchical prototype discovery. The method introduces a mixture-of-Gaussians prior over a complete binary tree to model hierarchical abstractions and leverages variational inference jointly with contrastive learning to learn taxonomic structures from unlabeled data.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. **Strong theoretical foundation**: The method section is rigorous, and the derivation of the ELBO objective is comprehensive. In particular, the reinterpretation of ELBO as a prototypicality maximization objective (Section 3.3) provides an insightful information-theoretic justification.\n    \n2. **Well-structured model design**: The use of a binary-tree-structured prior enables the discovery of hierarchical abstractions without assuming the number of ground-truth classes, which is a desirable property for unsupervised clustering.\n    \n3. **Interpretability**: The model discovers visually coherent and semantically meaningful sub-hierarchies, which enhances interpretability beyond flat clustering methods.\n    \n\n**Weaknesses and Suggestions:**\n\n1. **Outdated baselines**: The experimental comparison is mainly limited to older methods such as Cobweb, TreeVAE, and DeepECT. These baselines, while relevant historically, do not fully reflect the current state of unsupervised and self-supervised clustering, especially in the context of vision transformers or modern contrastive learning models. Recent approaches should be included for a more compelling comparison.\n    \n2. **Limited dataset diversity**: The evaluation is conducted on standard benchmarks like MNIST, Fashion-MNIST, CIFAR-10/20. These datasets, while widely used, are relatively outdated and limited in semantic complexity. They may not fully demonstrate the benefits of discovering deep taxonomic structures. I recommend evaluating on more recent and semantically structured datasets.\n    \n3. **Missing ablation analysis**: While the paper introduces several important components, I could not find any dedicated ablation study isolating the contributions of these individual components. As a result, it is difficult to assess which aspects of the method are most responsible for the observed performance gains. I strongly recommend including ablation experiments to disentangle the effects of the hierarchical prior, contrastive learning, and other design choices. Additionally, although the supplementary materials were available for download, I was unable to open them successfully; it is possible that relevant analysis was included there, but I could not verify it.\n\nQUESTIONS:\nPlease refer to Weakness.\n\nLIMITATIONS:\nPlease refer to Weakness.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new approach for hierarchical classification, i.e., instead of predicting \"collie\" for an image, a hierarchical classification would predict a chain animal --> mammal --> dog --> collie. The paper proposes to relax the condition of every node in the hierarchy tree to correspond to conventional semantic classes and associate them instead to clusters of samples (images). If I understood correctly, the paper claims to learn both the structure of the hierarchy as well as the classification technique. Each node in the tree is represented by a latent variable that is learned using a variational approach. The paper derives the expected lower bound to maximize for this problem and use encoder-decoder structure to learn the latents and predict classes.\n\nThe proposed method was tested on public datasets CIFAR, MNIST and Fashion datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Attempt to mathematically formulate the overall problem will be appreciated in the community. (Pls refer to other reviewers on method novelty).\n2. In several places, the writeup attempts to provide interpretation of mathematical quantities connecting them to the problem setting -- I think this is important for paper with heavy theoretic treatments. I also appreciate the effort on good amount of experimental details, results and analysis performed in the paper.\n\nWeaknesses:\n1. Goal/Contribution: I dont think I understood the merit of the proposed approach. If the tree leaves do not represent any semantic class, I dont understand their meaning, or more importantly, their benefit. Would these more freeform clusters at leaf advantageous for classification? The accuracies on Table 1 does not support that. Clarification on what is the benefit of this structure is needed for understanding the value of this contribution. \nSame concern can extended to internal nodes as well. From the writeup, if appears the clusters pertaining to internal nodes are not exclusive at the same level. What benefit does this form of internal nodes offers us?\n\n2. Method: Does the method learns the hierarchy structure or assumes it is provided? I assumed it does, but not completely sure on it. Lines 188~200 appear to suggest a hierarchy needs to be provided in the beginning but are not clear whether or not this is updated. If it does, this should be clarified in Section 3 how the structure is learned in the variational approach. \n\nThe method section also could elaborate a bit further how the $\\alpha_i$'s (introduced in Line 119) are learned by the encoder-decoder design. Line 169 briefly mentions it, could help to elaborate. In general, would help to include a paragraph how the quantities needed for the hierarchical prediction (e.g., as shown in Figure 2) are learned through encoder-decoder, i.e., connecting the mathematical formulation to implementation.\n\n3. Experimental validation:\n3a. The datasets (CIFAR, MNIST) selected for evaluation are a bit experimental and simpler datasets. In order to convince the reader the robustness of this method, results on more real life datasets, at least ImageNet or OpenImages, are necessary. \n3b. The proposed method seems to be achieving good performance on the simpler MNIST dataset. Given the DP and LP in 40s and 20s for CIFAR, and accuracy be 70%, I cannot help questioning the goal again as to how the mixed clusters at leaves and internal nodes are beneficial .\n\nQUESTIONS:\n..\n\nLIMITATIONS:\n..", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}