Bures-Isotropy Alignment: Manifold Learning of Generalized Category Discovery
Authors: Luyao Tang
Kunze Huang
Chaoqi Chen
Cheng Chen
Venue: ICLR 2026 Poster (2026)
Keywords: generalized category discovery
Bures metric
quantum informatics
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Generalized Category Discovery (GCD) seeks to discover categories by clustering unlabeled samples that mix known and novel classes. While the prevailing recipe enforces compact clustering, this pursuit is largely blind to representation geometry: it over-compresses token manifolds, distorts eigen-structure, and yields brittle feature distributions that undermine discovery. We argue that GCD requires not more compression, but geometric restoration of an over-flattened feature space. Drawing inspiration from quantum information science, which similarly pursues representational completeness, we introduce Bures-Isotropy Alignment (BIA), which optimizes the mini-batch class-token Gram toward an isotropic prior by minimizing the Bures distance. Under a mild trace constraint, BIA admits a practical surrogate equivalent to maximizing the nuclear norm of stacked class tokens, thereby promoting isotropic, non-collapsed subspaces without altering architectures. The induced isotropy homogenizes the eigen-spectrum and raises the von Neumann entropy, improving both cluster separability and class-number estimation. BIA is plug-and-play, implemented in a few lines on unlabeled batches, and generally boosts strong GCD baselines on coarse- and fine-grained benchmarks, improving overall accuracy and reducing errors in the estimation of class-number. By restoring the geometry of token manifolds rather than compressing them blindly, BIA supplies compactness for known classes and cohesive emergence for novel ones, advancing robust open-world discovery. Code is available at github.com/lytang63/BIA.

Published as a conference paper at ICLR 2026
BURES-ISOTROPY ALIGNMENT: MANIFOLD LEARNING OF GENERALIZED CATEGORY DISCOVERY
Luyao Tang",, Kunze Huang 2* Chaoqi Chen Cheng Chen
1 The University of Hong Kong 2 Xiamen University sytang1999egman gmaam cheng. chen@hku.
3 Shenzhen University
ABSTRACT
Generalized Category Discovery (GCD) seeks to discover categories by clustering unlabeled samples that mix known and novel classes. While the prevailing recipe enforces compact clustering, this pursuit is largely blind to representation geometry: it over-compresses token manifolds, distorts eigen-structure, and yields brittle fea- ture distributions that undermine discovery. We argue that GCD requires not more compression, but geometric restoration an over-flattenee feature space. Drawing inspiration from quantum information science, which similarly pursues represen- tational completeness, we introduce Bures-Isotropy Alignment (BIA), which optimizes the mini-batch class-token Gram toward isotropic prior by minimizing the Bures distance. Under mild trace constraint, BIA admits a practical surrogate equivalent to maximizing the nuclear norm of stacked class tokens, thereby promot- ing isotropic, non-collapsed subspaces without altering architectures. The induced isotropy homogenizes the eigen-spectrum and raises the von Neumann entropy, improving both cluster separability and class-number estimation. BIA is plug- and-play, implemented in a few lines on unlabeled batches, and generally boosts strong GCD baselines on coarse- and fine-grained benchmarks, improving overall accuracy and reducing errors in the estimation of class-number. By restoring the geometry of token manifolds rather than compressing them blindly, BIA supplies compactness for known classes and cohesive emergence for novel ones, advancing robust open-world discovery. Code is available github.com//ytang63//0
INTRODUCTION
Open-world learning (Zhou et al., 2022;
Wu et al., 2024) mandates models that can re-identify known classes while discover- ing novel ones from unlabeled data. Gen- eralized category discovery (GCD) (Vaze et al., 2022b) formulates this demand ex- plicitly, relaxing the assumptions of open- set recognition (OSR) (Geng et al., 2020) and novel-class discovery (NCD) (Han et al., 2019). Despite recent progress, the domi- nant paradigm in GCD still hinges on com- pact clustering as a universal target. This practice is geometry- intra-class variability indiscriminately, often collapsing token manifolds into few prin- cipal directions. The result is a degraded representation geometry with poor eigen- structure, skewed energy distribution, and fragile decision regions, that ultimately impedes category discovery and number estimation.
9e Conventional GCD
what Steretoed
Figure 1: (a) GCD is constrained by dimensional col- lapse due to strong clustering, leading to mixed class features and limited representational capacity. (b) BIA enhances the token geometry capacity, improving rep- resentational completeness and unlocking the model's potential in the open world.
it compresses
Equal Contribution; Corresponding Author.

---

Published as a conference paper at ICLR 2026
We contend that the bottleneck is not insufficient compactness, but the loss of geometric quality in learned representations. When the class-token subspace is over-compressed, the feature distribution becomes anisotropic and low-rank, masking fine-graine semantics and amplifying open- wonld emrors such as spurious merges. Thus, a central desideratum for GCD is to restore a well-posed geometry and preserves intra-class completeness while maintaining inter-class separability.
To this end, we propose Bures- Inores- Alignment (BIA), a geometry-aware principle inspired by quantum information. As shown in Figure I, green represents known classes and red represents novel' classes. The upper panel illustrates how conventional GCD methods, by enforcing compactness, cause dimensional collapse, leading uninformative feature dimensions (grey area). BIA restores geometry, enhancing separability by encouraging isotropy. BIA aligns the empirical class-token covariance with an isotropic prior by minimizing the Bures distance (Jozsa, 1994), canonical metric that instantiates the 2- 2-Wasserstein geometry over covariance operators. Conceptually, BIA redistributes spectral energy across eigen-directions, prevents dimensional collapse, and yields uniform, full-rank token manifolds that capture richer intra-class semantics without sacrificing discrimination.
BIA comes with a practical and rigorous surrogate. Under mild row-norm or trace control, minimizing the Bures distance to identity is equivalent to maximizing the nuclear norm of slacked class tokens. This equivalence connects isotropy alignment with a simple few-line implementation that is architecture-agnee aggostic and
compatible. It also bridges information geometry with GCD's s capacity-oriented diagnostics: by homogenizing the spectrum, BIA increases the von Neu- mann entropy (Nielsen & Chuang, 2010) and the effective rank of the class-token autocorrelation, which we observe to correlate with more reliable class-number estimation and more stable clustering.
We integrate BIA into representative GCD frameworks (contrastive and prototype- without modifying backbones and loss schedules. BIA consistently improves All/Old/New accuracy across standard coarse- and fine-gre aieeggranedd benchmarks, while reducing class-number estimation error. Ab- lations show that BIA stabilizes pseudo-label assignment in GCD, attenuates early collapse, and remains robust across backbones and batch sizes, negligible computational overhead thanks to a Gram-matrix implementation (Borgwardt et al., 2006).
We introduce BIA, casting GCD as isotropy alignment of class-token covariance via the Bures distance, thereby restoring representation geometry rather than compressing it blindly,
We establish an equivalence between BIA and nuclear-norm maximization under trace constraint, explaining BIA's effect on eigen-spectrum uniformity, von Neumann entropy, and effective rank, and linking information geometry to capacity-aware diagnostics GCD,
We provide a plug-and and---adppyy -lay implementation that yields consistent gains strong GCD baselines in both clustering accuracy and K Keestimation, with minimal code and overhead.
RELATED WORKS
2.1 GENERALIZED CATEGORY DISCOVERY
Generalized category discovery (Vaze et al., 2022b; Zhao et al., 2023; Wen et al., 2023; Choi et al., 2024) jointly recognizes known classes and discovers unseen ones. The seminal framework (Vaze et al., 2022b) integrates semi-sus sami-supervised k-means; SimGCD (Wen et al., 2023) introduces para- metric classifier with entropy regularization and self-distillation; CMS (Choi al., 2024) enhances representations via mean-shift clustering: and (Zhao et al., 2023) adapts prototype counts at infer- ence. ActiveGCD (Ma al., 2024) further queries labels for selected unlabeled samples to improve discovery. Most GCD methods still prioritize compact clustering and thus overlook central GCD desideratum: restoring well-posed geometry that preserves intra-class completeness while maintain- ing inter-class separability. We instead provide concise, method- mechanism that enforces Bures- Bures-isotropy representations, thereby sharpening decision boundaries across GCD pipelines.
2.2 REPRESENTATION COMPLETENESS
Representation completeness collapse (Grill al., 2020; Caron et al., 2020; Shi et al., 2023; Jing et al., 2021) occurs when embeddings concentrate in a din-enmnsioonl subspace, limiting diversity
In the task setting of GCD, do not differentiate between 'Old/New' and 'Known/Novel'.
2

---

Published as a conference paper at ICLR 2026
Any Framework
min di C2g
Learning
DINO WTT
Protetype Learning
collegued insan-class
(e) Ovenconpressed Taken Geometry
boorook
Learning
patch tollers
clos' tallen
GCD enaunes inter inter-class separahion
collopeed novel-dla
specta
DENO
VIT
min oe (Tg
Conventional GCD
non-collaped spane
BIA regularions the detribution
Figure Overview of Bures-Isotropy Alignment. (a) BIA targets the geometry of the [cls] token, which summarizes sample information. (b) The core mechanism restores compeessde low-rank representations high-rank, isotropic state, preserving richer intra-class semantics. (c) The resulting embeddings are plug-and-play compatible with any GCD framework.
and expressiveness. DirectCLR (Jing et al., 2021) optimizes the representation space without trainable projector to promote uniform dispersion. While Whitening (Tâo et al., 2024) equalizes covariance contributions, the non-contrastive objective for collaborative filtering (Chen al., 2024) emphasizes alignment and compactness without augmentation. Similarly, Bregman matrix diver- gence (Zhang al., 2024) pulls covariance toward the identity, and random orthogonal projection modeling (Haghighat et al., 2023) broadens the search for characteristics. Rather than directly targeting collapse, we reformulate GCD as isotropy alignment of class-token covariance measured by the Bures distance, prioritizing the restoration of representation geometry over blind compression.
2.3 BURES METRIC
The Bures metric (Safránek, 2017) is based on the Bures distance and is originally defined to quantify the difference between quantum states. widely applied in the study of quantum information geometry (Cho & Jae, 2025), quantum physics (Alsing et al., 2023), and related areas. By interpreting density matrices as generalized form of probability distributions, the Bures metric becomes an effective method for comparing probability distributions positive definite matrices. For example, in (Ji et al., 2019), it employed an alternative to the Wasserstein distance (Panaretos Zemel, 2019) to measure the discrepancy between generated and real distributions in more stable manner. The kernelized Bures metric (Gilo et al., 2024) leverages the Bures metric within a reproducing kernel Hilbert space to compare source and target domain distributions. Existing analyses of Bures distance have treated as metric. We have amplified its advantages and, for the first time, extrapolated it to serve optimization objective for category discovery in open-world world environments.
3 METHODOLOGY
3.1 NOTATION AND PRELIMINARIES OF GCD
For each dataset, consider a labeled subset D, = ((x), vt} C X' x Yi and an unlabeled subset D. {xx", } X' x )a. Only known classes can be found in Dr. while D., encompasses known and novel classes, translating to 3/- = Cenour and V., = Cknoun U Crovel- The task of models involves clustering on both the known and novel classes in D... The number of novel classes represented as Knovel can be determined beforehand (Vaze et al., 2022b; Zhao et al., 2023). The functions f(-) and 9(-) perform the feature extractor and projection head, respectively. Both the feature h; = fxx;) and the projected embedding z; = g(h,) are under fg-normalization A ViT-style encoder (Dosovitskiy, 2020) provides one class token [c]s] and H W visual tokens per image.

---

Published as a conference paper at ICLR 2026
3.2 TOKEN GEOMETRY
Scope. As shown in Figure 2, we first formalize the class-token space regulated by our method, keeping [cls] formation concise while retaining key mechanics for downstream spectral analysis. By token geometry, we mean the spectral/metric structure of class tokens (pairwise angles, covariance cigen-spectrum, effective rank, isotropy) computed over batches.
To simplity Icls]'s evidence aggregation, we summarize one attention step with one-head simplification (locally aggregating token evidence into global summary). After linear projec- tions Q, K, V XWQ, XWK, XWv R(L+HW)xx (or d for V; Wo, Wx, Wy learnable, da query/key dimension), let qc E Rixds be [cls] 's row in Q. The class attention weights are:
9cK dd
Oge = softmax
E E I
where is a probability vector over all (1 + HW) tokens. These normalized weights guide token evidence distribution and update [cls] residually, With the attention:
[cls]' [cls] + a.V EVERR E
(2)
where [cls]' the updated [cls] and the value matrix. Repeated across layers, Equations I and 2 defines [cls] for subsequent batch-scale geometric analysis. For GCD, token geometry governs two coupled goals: preserving intra-class completeness for known categories and enabling cohesive emergence of novel ones; in contrast, anisotropic, low-rank geometry under pseudo-label noise tends to induce spurious merges or over-fragmmenttioos (Figure 1).
Batch-scale token geometry. To study geometry at the mini-batch scale, consider an unlabeled mini-batch 8 of size and stack per-image class tokens row-wise as
2 = stack([cls)) I..., [cls] n) E TR Bxd
Σn 27 E BBB
(3)
where stack(-) concatenates row vectors, is the embedding dimension of each class token, and EB is the Gram (Gatys al., 2015) of class tokens. Under row fg-normalization, tr(Eg) || 2
B, and each entry (EB); (Z Zz:) reduces cosine similarity: thus the spectrum of En compactly captures how class tokens co-occupy directions in feature space.
IZ Σ 8
Since the feature Gram 7T and sample Gram 7ZT share identical nonzero singular spectra, we diagnose geometry via either the batch-scale ΣB or the global autocorrelation A (Sec. 3.4). A uniform eigen-spectrum signifies high effective rank and isotropic support, critical for GCD as the unlabeled pool mixes known and novel categories. Empirically, these isotropic manifolds provide richer discriminative capacity and tolerance to open-world uncertainty, stabilizing clustering updates against noise and class imbalance while improving naasbr estimation.
3.3 BURES-ISOTROP! ALIGNMENT
Motivation. Token geometry strongly affects discovery quality, yet most GCD pipelines do not explicilly correct in open-world settings. Building on our analysis above, we borrow the Bures met- ric (Bures, 1969; Uhlmann, 1976) from quantum information (a field that also values representation quality) and turn this metric into a simple optimization that matches GCD.
We first need way to quantify "how far" batch's class-token covariance is from an isotropic target; a natural choice is the Bures distance to identity
di (EB = tr(Errr(E))))))))))) + B- tr (rt(E11//) 12222
where Σg TZTE Bx is the batch Gram of stacked class tokens Z E IR Bxd B batch size, and I is the x B identity; with row fz-norms, the trace terms change little, so minimizing di mainly increases tr(2)//2) and thus spreads spectral energy.
4

---

Figure 3: Comparison between (og(rank(A)) and (A). The count of the largest eigenvalues necessary to account for 99% of the total eigenvalue Published as a conference paper at ICLR 2026
To implement this without changing the training loop, relate the square-root trace to familiar surrogate that depends directly on
tr (E = - = Es; 8j 11
(5)
where { are eigenvalues of E and { are singular values of Z and this identity converts the metric into the nuclear norm of stacked class tokens.
Metric-to-loss. With this link in place, aligning to isotropy becomes an extremely simple optimization on Z under standard normalization:
arg min 2 (E = arg max |Z
(6)
Intuitively, maximizing || lifts rank and homogenizes the spectrum, which raises entropy, reduces collapse, and stabilizes discovery in noisy unlabeled batches, We use either the metric itself or its nuclear-norm surrogate; both behave the same once row norms are stabilized.
LBAA di (E I)
-
RIA
This keeps the BIA architecture-agnosiic and adds only a single scalar term to the base GCD objective:
= L + XLRIA-
In summary, token geometry is central to GCD but seldom corrected explicitly. By importing the Bures metric and recasting it as a nuclear-norm objective, BIA offers a direct way to restore isotropy in the class-token manifold and improve open- open-world discovery. As shown below, the implementation of BIA is extremely concise, with core code consisting of only two lines (provided in Section A).
3.4
BIA INCREASES VON NEUMANN ENTROPY
The autocorrelation matrix (Schölkopf & Smola, 2002) of the sample's token geometry is A A CLS C CLS/N. We em- ploy von Neumann entropy (Petz, 2001; Boes et al.,
lostrank(.11
E N [cls];
N
2019) to measure token geometry. This gives the ad- vantage of focusing exclusively
the eigenvalues, al- lowing for graceful handling of eigenvalues that are extremely close O. The von Neumann entropy can be expressed A (A), representing the Shannon tropy (Shannon, 1948) of the eigenvalues of A, with values ranging between 0 and log d. A larger / (.A) indicates greater token geometry capacity of the fea- tures. Von Neumann entropy is an effective measure for assessing the uniformity of distributions and managing extreme values.
energy serves surrogate for the rank.
As illustrated in Figure 3, the incorporation of BIA results in a von Neumann entropy for the embeddings that is significantly higher than that of the original scheme. It is possible to relate von Neumann entropy to the rank of the [cls]. When A possesses uniformly distributed eigenvalues with full rank, the entropy is maximized. We clarify the connection between BIA and VNE. BIA serves as a local optimization objective for mini-batch Gram matrices. In contrast, VNE functions as global diagnostic metric derived from the complete dataset's 's autocorrelation matrix. BIA enhances the feature geometry directly, whereas VNE assesses the resulting improvements by evaluating spectral uniformity and effective rank.
5

---

Published as a conference paper at ICLR 2026
Table 1: Experimental results on fine-grained datasets, evaluated with the K for clustering.
CUB
Stanford Cars
FGVC Aircraft
Method
All
Old
New
All
Old
New
All
Old
New
Clastering with the ground trthh number of classes K given
Agglomerutive RankStats+
37.0
36.2
37.3
12.5
14.1
11.7
15.5
12.9
16.9
33.3
51.6
24.2
28.3
61.8
12.1
26.9
36.4
22.2
UNO
35.1
49.0
28.1
35.5
70.5
18.6
40.3
56.4
32.2
ORCA
35.3
45.6
30.2
23.5
50.1
10.7
22.0
31.8
17.1
GCD
51.3
56.6
48.7
39.0
57.6
29.9
45.0
46.9
ProtoGCD
63.2
68.5
60.5
53.8
73.7
44.2
56.8
62.5
53.9
PrCAL
62.9
64.4
62.1
50.2
70.1
40.6
$2.2
$2.2
52.3
ActiveGCD
66.6
66.5
66.7
48.4
57.7
39.3
53.7
51.5
56.0
PIM
62.7
75.7
56.2
43.1
66.9
31.6
SelEx
78.7
81.3
77.5
55.9
76.9
45.8
60.8
70.3
56.2
Ours
80.6
81.0
80.4
57.0
77.3
47.2
61.8
68.2
59.2
-0.3
+2,9
+1.1
+1.4
+3.0
SimGCD
60.7
65.6
57.7
51.2
69.4
42.4
54.0
58.8
51.5
Ours
62.1
65.8
60.3
52.3
70.0
43.7
55.1
58.9
53.1
CMS
67.1
74.9
63.2
56.7
76.8
37.5
53.6
60.3
47.0
Ours
71.1
74.1
66.9
57.4
79.4
36.2
55.7
63.7
47.9
+4.0
-0.8
+0.7
+2.6
-1.3
+2.1
+3.4
+0,9
SPTNet
62.0
69.2
56.0
56.2
70.3
46.6
51.6
60.7
45.9
+ Ours
63.3
70.7
59.6
58.8
75.4
50.8
54.7
65.3
48.5
+1.3
+1.5
+3.6
+2.6
+5.1
+4.2
+3.1
+4.5
+2.6
Avg. A
+2.2
+0,2
+3.2
+1.4
+2.2
+1.1
+1.8
+1,5
+2.0
4 EXPERIMENTS
4.1 SETUP
Benchmarks. BIA is evaluated on coarse- and fine-grained benchmarks. These include two conven- tional datasets, CIFAR 100 (Krizhevsky & Hinton, 2009) and ImageNet100 (Geirhos et al., 2019), and four fine- gre-gra datasets, CUB-200-2011 (Wah al., 2011), Stanford Cars (Krause et al., 2013), FGVC Aircraft (Maji et al., 2013), and Herbarium 19 (Tan et al., 2019). To segregate target classes into sets of known and unknown, we adhere to the splits defined by the SSB (Vaze al., 2022a) when working with CUB, Stanford Cars, and FGVC Aircraft. The splits from the previous study (Vaze et al., 2022b) are employed for the remaining datasets, we designate 80% of the classes as known under the CIFAR 100 benchmark. For the rest of the benchmarks, the proportion of known classes stands at 50%. Our labeled set D,, comprises 50% images from the known classes for all benchmarks.
Evaluation Protocols. We assess BIA's effectiveness via two-step process. First, cluster the complete collection of images defined as D. Then, we measure the accuracy on the set D., In line with previous research (Vaze et al., 2022b), accuracy is determined by comparing the assignments to the actual labels using the Hungarian optimal matching (Kuhn, 1955). This method bases the match on the number of instances that intersect between each pair of classes. Instances that do not belong to any pair, i.e., unpaired classes, are viewed as incorrect predictions. On the other hand, instances belonging the most abundant class within each ground-truth cluster are taken as correct for accuracy calculations. We present the accuracy for all unlabeled data, and the accuracy is classified as old/known and new/novel, respectively. The accuracy using the estimated number of classes and the ground-truth K are reported. This allows us to compare BIA with previous studies that have assumed the availability of the during the evaluation phase.
Implementation Details. The purpose of BIA is to empower existing GCD schemes to improve the completeness of representation. We closely adhere to their initial implementation details for an effective comparison. We use pre-trained DINO ViT-B/16 (Caron et al., 2021; Dosovitskiy, 2020),as our image encoder along with a projection head, an approach consistent with existing methods (Vaze et al., 2022b; Zhang et al., 2023; Pu et al., 2023). All experiments performed on four NVIDIA RTX 4090 GPUs. We follow the original training parameter details of each scheme to illustrate the generality and applicability of BIA.

---

Published as a conference paper at ICLR 2026
Table 2: Experimental results on coarse- and fine-grained datasets, evaluated with the K for clustering.
CIFARI00
ImageNet100
Herbarium 19
Method
All
Old
New
All
Old
New
All
Old
New
Clastering with the ground truth namber of classes K given
Agglomerutive RankStats+
56.9
$6.6
57.5
13.1
77.9
70.6
14.4
14,6
14.4
58.2
77.6
19.3
37.1
61.6
24.8
27.9
55.8
12.8
UNO-
69.5
80.6
47.2
70.3
95.0
57.9
28.3
53.7
14.7
ORCA
69.0
77.4
52.0
73.5
92.6
63.9
20.9
30.9
15.5
GCD
73.0
66.5
89.8
35.4
27.0
ProtoGCD
81.9
82.9
80.0
84.0
92.2
79.9
44.5
59.4
36.5
PrCAL
81.2
84.2
75.3
83.1
92.7
78.3
37.0
$2.0
28.9
ActiveGCD
71.3
75.7
66.8
90.2
PIM
78.3
84.2
66.5
83.1
95.3
77.0
42.3
56.1
34.8
SelEx
80.0
84.8
70.4
82.3
93.9
76.5
36.2
46.0
30.9
Ours
80.7
84.3
72.1
82.8
94.1
77.8
36.8
47.5
31.0
-0.5
+0.5
+1.3
+1.5
+0.1
SimGCD
80.1
81.5
77.2
83.3
92.1
78.9
44.7
57.4
37,9
Ours
80.2
81.5
77.5
86.7
93.1
83.6
45.6
57.8
39.0
+3,4
CMS
79.5
85.4
67.7
83.0
95.6
76.6
36.5
55.4
26.4
Ours
79.0
85.5
66.1
84.8
95.6
79.5
36.3
56.5
25.4
-0.5
+0.1
+1.8
+2.9
+1.1
-1.
SPTNet
81.1
84.3
75.0
85.3
93.1
81.3
43.5
58.9
35.6
+ Ours
82.1
84.8
76.2
85.4
93.4
81.3
44.2
58.9
36.3
+0.5
+0.1
+0.7
Avg. A
+0.3
+0.1
+0.4
+1.5
+0.4
+2.2
+0.5
+0.7
+0.2
Table 3: GCD Accuracy on coarse- and fine-grained datasets, evaluated without the K for clustering.
Mothod
CIFAR100
ImageNet100
CUB
Stanford Cars
FGVC Aireraft
Herbarium 19
All
Old
New
All
Old
New
All
Old
New
All
Old
New
All
Old
New
All
Old
New
Clustering without the grownd-rruch number of closser K given
Agglomentive
56.9
56.6
57.5
72.2
77.8
69.4
35.7
33.3
36.9
10,8
10.6
10.9
14.1
10.3
16.0
13.9
13.6
14.1
GCD
29.7
29.4
GPC
75.4
84.6
60.1
75.3
93.4
66.7
52.0
55.5
41.5
38.2
58.9
27.4
43,3
40.7
44.8
36.5
51.7
21.9
PIM
75.6
81.6
63.6
83.0
95.3
76.9
62.0
75.7
55.1
42.4
65.3
31,3
42,0
55.5
34.7
CMS
77.8
65.3
83.4
95.6
77.3
66.2
69.7
64.4
51.8
72.9
31,3
52.3
58.9
45.8
38.5
57.3
28.4
Ours
79.5
84.7
69.1
84.3
95.7
78.8
68.7
74,1
66.0
52.5
72.7
32,9
53.4
60.1
46.7
38.0
56.9
27.9
A
338
0..9
0.1
12.5
4.4
1.2
0.9
-0.5
4.2
MAIN RESULTS
Evaluation GCD. As shown in Tables 2 and 3, BIA brings consistent and notable gains across all evaluated GCD methods and datasets, under both known and unknown class number settings. Key findings CMS, SPTNet, and SelEx without any architectural changes. For example, on CUB with known class number, BIA enhances SimGCD by 2.6% on novel classes and CMS by 4.0% on all classes. On ImageNet100, it improves SimGCD by 3.4% in the all setting and boosts CMS by 2.9% on novel classes. These results highlight BIA's strong generalization across frameworks and confirm its plug-and- and-play compatibility. @ Generality. BIA yields stable gains on both coarse- coarse-grained datasets like 10F and ImageNet100 and fine-grained ones like CUB and Cars. Notably, on CUB, BIA improves CMS (known) by 4.4% under unknown class number settings. Average improvements on novel classes range from about 0.9% 3.8% across datasets, demonstrating the robustness to domain complexity and label granularity.
follows: O Compatibility. BIA improves all baselines including SimGCID,
Ablation study. The only hyperparameter of BIA is the coefficient of the loss. To gain a deeper understanding of the correlation between the degree of maximum token manifold capacity and the dimensionality in Figure 4. It can be observed that BIA is not sensitive to hyperparameters and can uniformly enhance clustering accuracy. A more thought- phoookig-rovoking finding is that directly reducing D to avoid dimensionality collapse is suboptimal. The reason is that each dimension of the manifold contributes
of the features, we conducted an ablation experiment on it, as shown
7

---

Published as a conference paper at ICLR 2026
185
808
183
(a) CUB
(b) Stanford Cars
(c) FGVC Aireraft
Figure Hyperparameter sensitivity of the degree BIA (A) and features dimensionality (D).
to the representation, and a reduction in D will directly lead loss of information. Even with BIA, it is impossible to make the representation complete. An appropriate number of dimensions enriches the representation while using BIA to prevent dimensionality collapse, which can maximize the model's performance enhancement.
HIERARCHICAL ANALYSIS OF WHY BIA IS EFFECTIVE
We conduct a comprehensive analysis from multiple dimensions: 1) eigenvalue distribution and Frobenius norm, 2) estimation of embedded space distribution, 3) dimensional collapse, and 4) comparison with similar schemes, to understand the necessity and effectiveness of BIA for GCD.
SimGCD
SimGCD Ours
CMS
Ours
600
600
25
120
122
150
375
202
25
300
325
364
100
Epoch
Spoch
Epach
(a) CUB
[b) Stanford Cars
(c) FGVC Aircraft
Figure 5: The Frobenius norm ||A- - c Lall? on three fine-grained benchmarks.
5.1 BIA HOMOGENIZES EIGENVALUE DISTRIBUTION AND REDUCES FROBENIUS NORM
The autocorrelation matrix of the test sample token geometry is denoted as Given || [c][[]]]]]]]]]] | =1 and A 2 0, it follows that Σ, > and Y₃ 2 0 (Parkhi et al., 2015; Liu et al., 2017; Mettes al., 2019), where {}₂} are the eigenvalues of A. Under ideal conditions, where A eigenvalue distribution AA becomes uniform, z uncorrelated (Cogswell et al., 2015), full-rank (Hua al, 2021), and isotropic (Vershynin, 2018). A linked various representation characteristics. The Frobenius norm (Ma et al., 1994; Peng et al., 2016), extensively studied in self-supervised learning methods (Cogswell et al.. 2015: Xiong et al., 2016; Choi & Rhee, 2019; Zbontar al., 2021), measures whether the representation depends on few dimensions. A smaller Frobenius norm indicates a larger manifold capacity. We applied singular value decomposition (SVD) (Golub & Reinsch, 1971) to the autocorelatiin matrix of the feature embeddings, plotting the first 200 singular values in Figure 6 and visualizing the Frobenius norm [A c Ialp in Figure 5. Compared to SimGCD and CMS, BIA achieves a more uniform and stable eigenvalue distribution.
Ia. the
5.2 BIA UNRAVELS DIMENSIONAL COLLAPSE.
We further explored the relationship between the accuracy and eigenvalues of GCD, respectively, and dimensional collapse, shown in Figure 6 and our findings follows: (1) Feature Completeness
8

---

Published as a conference paper at ICLR 2026
10
(5)
FGVC
Figure 6: BIA effectively mitigates dimensional collapse by providing a more uniform eigenvalue distribution and improves the clustering aceuracy.
Complete features improve intra-class representations, which enhances clustering accuracy by providing richer, higher manifold capacity. (2) BIA's Impact: BIA increases manifold capacity, leading higher singular values and more accurate clustering by better approxi- mating the true distribution.
and Clustering Accuracy:
(3) Dimension Collapse and Limitations of CMS/SimGCD: CMS and SimGCD operate in lower-dimensional spaces, limiting manifold capacity and causing incomplete representations (Caron et al., 2020; Shi et al., 2023). Dimension collapse results in oversimplified models, while BIA maximizes intra-class completeness for better decision boundaries. This break- down highlights how BIA addresses limitations in existing methods by optimizing the manifold capacity and the richness of intra-class representations, leading improved model performance.
5.3 BIA PROVIDES ACCURATE DISTRIBUTION ESTIMATION
We present the gap between BIA and SOTAs in estimating the number of clusters in Table 4. By leveraging CMS, which requires perparameters to estimate K, our optimization target becomes LcMs LRIA. Results demon- strate significant improvement with BIA incor- porated into the CMS framework. Notably, on the complex and diverse ImageNet 100 dataset, our method achieves 100% correct estimation rate. The improvement in estimating the number of clusters highlights the importance of representation completeness, enabling better capture of intra-class nuances and sharper inter-class separation.
Table 4: Estimated number and error rate of K.
specific hy-
CEARIIG
Scefrel FrVe Abooet
Batte
100
110
200
395
200
(m
331
133
DOCL
146
46
120
172
152
0.00
GFC
160
1m
315
301
010
Clay
96
100
183
10
150
11 189
99
5.4 COMPARISON WITH OTHER ISOTROPIC DISTRIBUTION SCHEMES
From the perspective of motivation and self- supervised learning based on Isotropic Distri- bution, which BIA is similar, sentatives CorInfoMax (Ozsoy et al., 2022) and VICReg (Bardes al., 2022) challengers. As shown in Table 5, in the context of GCD, VI- CReg, while promoting variance and reducing covariance, does not explicitly focus on maxi- mizing intra-class representation completeness, which is crucial for distinguishing fine-grained categories. CorInfoMax, on the other hand, pri-
Table 5: Comparison on accuracy in GCD with rep-
resentative isotropic feature distribution schemes.
chose repre-
CUB
Starfod Cup
PGVC Aliuafi
New
All
SinDCD
51.3
55.3
546
559
61A
63.1
65.8
68.3
52.
766
13.7
85.1
589
531
56.3
618
$3.4
VICCKe
78.1
550
57.8
55.2
68.1
65
604
21.3
71.1
74.1
663
574
784
162
55.7
63.7
479
61.
72.4
56.3
marily maximizes mutual information but does not explicitly prevent dimensional collapse ensure richer intra-class representations. As result, both methods struggle to capture the full complexity of the structure of the data, limiting their effectiveness in accurately discovering novel categories.
CONCLUSION
We introduce Bures-Isotrope Alignment, simple yet powerful method for enhancing generalized category discovery, which identifies known classes and discovers novel ones from unlabeled data containing both categories demand of open-world learning. BIA effectively addresses

---

Published as a conference paper at ICLR 2026
the limitation of traditional GCD methods, which often sacrifice representation quality for compact clustering (causing dimensional collapse, distorted eigen-structure), and fragile feature distributions that hinder category discovery. Inspired by quantum information science's 's pursuit of representational completeness, BIA optimizes class-token covariance toward an isotropic prior by minimizing the Bures distance. Under mild trace constraint, this equals maximizing the nuclear norm of class tokens (no model architecture changes), restoring the over-flattened feature space to ensure complete, rich intra-class representations while preserving inter-class separability. BIA significantly boosts accuracy (All/Old/New) and class-number estimation of GCD baselines with negligible computational cost. By restoring feature geometry, BIA unlocks the model's full potential, meets GCD's needs, and serves as an effective tool for more adaptable machine learning models in open-world scenarios.
ACKNOWLEDGMENTS
The work described in this paper is supported by grants from HKU Startup Fund and HKU Seed Fund for Basic Research
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Paul M Alsing, Carlo Cafaro, Orlando Luongo, Cosmo Lupo, Stefano Mancini. and Hernando Quevedo. Comparing metrics for mixed quantum states: Sjöqvist and bures. Physical Review A, 107(5):052411 2023.
Adrien Bardes, Jean Ponce, and Yann LeCun. VICReg: Varimne-invarianeecovvaiane regularization for self-supervised learning. International Conference on Learning Representations, 2022. URL https: htpp/pppeeeeiieeeeeeeeeeeeeeeeee net net forum? iidiiddmt Xm6YD62DUUb.
Paul Boes, Jens Eisert, Rodrigo Gallego, Markus P Müller, and Henrik Wilming. Von neumann entropy from unitarity. Physical review letters, 122(21):210402, 2019.
Karsten M Borgwardt, Arthur Gretton, Malte Rasch, Hans-Peter Kriegel, Bernhard Schölkopf, and Alex J Smola. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics, 22(14):e49-es) 2006.
Donald Bures. An extension of kakutani's theorem on infinite product measures the tensor product of semifinite w*-algebras. Transactions of the American Mathematical Society, 135:199-212, 1969.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9912-9924, 2020.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9650-9660, 2021.
Huiyuan Chen, Vivian Lai, Hongye Jin, Zhimeng Jiang, Mahashweta Das, and Xia Hu. Towards mitigating dimensional collapse of representations in collaborative filtering. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pp. 106-116 115, 2024.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Gunhee Cho and Jeongwoo Jae. Support-projected petz. monotone geometry of two-qubit families: Three-channel identity and non-reduction of curvatures. arXiv preprint MXXi:2550911477 2025.
Daeyoung Choi and Wonjong Rhee. Utilizing class information for deep network representation shaping. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 3396- 3403, 2019.
Sua Choi, Dahyun Kang, and Minsu Cho. Contrastive mean-shift learning for generalized cate- gory discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23094-23104 2024.
Michael Cogswell, Faruk Ahmed, Ross Girshick, Larry Zitnick, and Dhruv Batra. Reducing overfit- ting in deep networks by decorrelating representations. arXiv preprint arXiv:1511.06066 2015.
Alexey Dosovitskiy. An image is worth 16x 16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1199 19229, 2020.
John Duchi. Derivations for linear algebra and optimization. Berkeley, California, 3(1):2325-5870. 2007.
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:15088 056.66666 2015.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019. URL htw.t::/t//ottt /vR.:httss//oppmmrvieeee.. d

---

Published as a conference paper at ICLR 2026
Chuanxing Geng, Sheng-jun Huang, and Songcan Chen. Recent advances in open recognition: A survey. IEEE transactions on pattern analysis and machine intelligence, 43(10):3614 3631, 2020.
Obsa Gilo, Jimson Mathew, and Samrat Mondal. Kereliieed bures metric: A framework for effective domain adaptation sensor data analysis. Expert Systems with Applications, 255:12472: 2024.
Gene H Golub and Christian Reinsch. Singular value decomposition and least squares solutions. In Handdbook for Automatic Computation: Volume II: Linear Algebra, pp. 134-151. Springer, 1971.
Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-suue learning. Advances in neural information processing systems, 33:21271-21284, 2020.
Maryam Haghighat, Peyman Moghadam, Shaheer Mohamed, and Piotr Koniusz. Pre-training with random orthogonal projection image modeling, arXiv preprint urXiv:2310.18741 2023,
Kai Han, Andrea Vedaldi, and Andrew Zisserman. Learning discover novel visual categories via deep transfer clustering, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8401 8409, 2019.
Tianyu Hua, Wenxiao Wang, Zihui Xue, Sucheng Ren, Yue Wang, and Hang Zhao. On feature decor- relation in self-supervised learning. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 9598-9608, 2021.
Xu Ji, Joao F Henriques, and Andrea Vedaldi. Invariant information clustering for unsupervised image classification and segmentation. Proceedings of the IEEE/CVF international conference computer vision, pp. 9865 9874, 2019.
Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian. Understanding dimensional collapse in contrastive self-supervised learning. arXiv preprint arXiv:2110.09348, 2021.
Richard Jozsa. Fidelity for mixed quantum states. Journal of modern optics, 41(12):2315-2323, 1994.
Prannay Khosla, Piotr Teterwak, Chen Wang. Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 33:1 13.18661-18673, 2020.
Jonathan Krause, Michael Stark, Jia Deng, and Fei-Fei. object representations for fine- graiggri categorization. In Proceedings of the IEEE international conference computer vision workshops, pp. 554-561, 2013.
Alex Krizhevsky and Geoffrey Hinton. Cifar- 10 dataset. sttps://wwee http:///wwwwcc..ooooooooo eddu/ -kriz/cifar html, 2009. Accessed: 2025-05-20.
Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83-97, 1955.
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Song. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 212-220, 2017.
Changxue Ma, Yves Kamp, and Lei F Willems. A frobenius norm approach to glottal closure detection from the speech signal. IEEE Transactions on Speech and Audio Processing, 2(2): 258-265 1994.
Shijie Ma, Fei Zhu, Zhun Zhong, Xu-Yao Zhang, and Cheng-Lin Liu. Active generalized cate- gory discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16890-16900, 2024.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint 13X6v:1306.5155 2013.
12

---

Published as a conference paper at ICLR 2026
AW Marshall. Inequalities: Theory of majorization and its applications, 1979.
Pascal Mettes, Elise Van der Pol, and Cees Snoek. Hyperspherical prototype networks. Advances in neural information processing systems, 32, 2019.
Michael Nielsen and Isaac Chuang. Quantum computation and quanium information. Cambridge
university press, 2010.
Serdar Ozsoy, Shadi Hamdan, Sercan Arik, Deniz Yuret, and Alper Erdogan. Self- supervised learning with an information maximization criterion. In S. Koyejo, S. Mo- hamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural In- formation Processing Systems, volume 35, pp. 35240-35253. Curran Associates, Inc., 2022. URL https::/ppooeeddiiiieeee// hwwp:/rooooeiiinnneeeeeeeeeeeeee neurips cc/paper fiwes/ #aless/pape//222/ file/ /tte////ebbbbeecceaaaaaa/////aaa
Paper- Conference pdf.
Victor M Panaretos and Yoav Zemel. Statistical aspects of wasserstein distances. Annual review of statistics and its application, 6(1):405-4311 2019.
Omkar Parkhi, Andrea Vedaldi, and Andrew Zisserman. Deep face recognition. In BMVC 2015- Proceedings of the British Machine Vision Conference 2015. British Machine Vision Association, 2015.
Xi Peng, Canyi Lu, Zhang Yi, and Huajin Tang. Connections between nuclear- norm and frobenius- bose-based representations. IEEE transactions on neural networks and learning systems, 29(1): 218-224, 2016.
Dénes Petz. Entropy, von neumann and the von neumann entropy: Dedicated to the memory of alfred wchrl. In John von Neumann and the foundations of quantum physics, pp. 83-96. Springer, 2001.
Nan Pu, Zhun Zhong, and Nicu Sebe. Dynamic conceptional contrastive learning for generalized category discovery. In Proceedings of the IEEE/CVÉ conference computer vision and patlern recognition, pp. 7579-7588, 2023.
Dominik Šafránek. Discontinuities of the quantum fisher information and the bures metric. Physical Review A, 95(5):052320, 2017.
Bernhard Schölkopf and Alexander J Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002.
Claude E Shannon. A mathematical theory of communication. The Bell system technical journal, 27 (3):379-423, 1948.
Yujun Shi, Jian Liang, Wenqing Zhang, Chuhui Xue, Vincent YF Tan, and Song Bai. Understanding and mitigating dimensional collapse in federated learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Kiat Chuan Tan, Yulong Liu, Barbara Ambrose, Melissa Tulig, and Serge Belongie. The herbarium challenge 2019 dataset. arXiv preprint arXiv:1906.05337 2019.
Yang Tao, Kai Guo, Yizhen Zheng, Shirui Pan, Xiaofeng Cao, and Yi Chang. Breaking the curse of dimensional collapse in graph contrastive learning: A whitening perspective. Information Sciences, 657:119952, 2024.
MTCAJ Thomas and A Thomas Joy. Elements of information theory. Wiley-Interscience, 2006.
Armin Uhlmann. The "transition probability" in the state space of a-algebra. Reports on Mathematical Physies, 9(2):733-279 1976.
Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Open-set recognition: A good closed-set classifier is all you need. In International Conference on Learning Representations, 2022a. URL https: /pennevveww n forum? 1 d=5hLP5..99924
Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Generalized category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7492-7501, 2022b.
13

---

Published as a conference paper at ICLR 2026
Roman Vershynin. High-dimensionall probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018.
C. Wah, N. Rasiwasia, D. Hsu, J. Yao, L. Li, and G. Mori. Caltech-ucsd birds 200-2011 (cub- 200-2011). http: /www. vision. caltech. wwtp:eeeweeveieaa///eee 200 html,, 2011. Accessed: 2025-05-20.
Xin Wen, Bingchen Zhao, and Xiaojuan Qi. Parametric classification for generalized category discovery: A baseline study. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16590-16600, 2023.
Jianzong Wu, Xiangtai Li, Shilin Xu, Haobo Yuan, Henghui Ding, Yibo Yang, Xia Li, Jiangning Zhang, Yunhai Tong, Xudong Jiang, et al. Towards open vocabulary learning: survey. IEEE Transactions Pattern Analysis and Machine Intelligence, 2024.
Wei Xiong, Bo Du. Lefei Zhang, Ruimin Hu, and Dacheng Tao. Regularizing deep convolutional neural networks with a structured decorrelation constraint. In 2016 IEEE 16th international conference on data mining (ICDM), pp. 519-528. IEEE, 2016.
Jure Zhontar, Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International conference machine learning, pp. 123 2010- 12320. PMLR, 2021.
Sheng Zhang, Salman Khan, Zhiqiang Shen, Muzammal Naseer, Guangyi Chen, and Fahad Shahbaz Khan. Prompteal: Contrastive affinity learning via auxiliary prompts for generalized novel category discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3479-3488, 2023.
Yifei Zhang, Hao Zhu, Zixing Song, Yankai Chen, Xinyu Fu, Ziqiao Meng, Piotr Koniusz, and Irwin King. Geometric view of soft decorrelatio in self-supervised learning. In Proceedings of the 30th
ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4338-4349, 2024.
Bingchen Zhao, Xin Wen, and Kai Han. Learning semi-suse muni-supervised gaussian mixture models for generalized category discovery. In Proceedings of the IEEF/CVF International Conference Computer Vision, pp. 16623-16633, 2023.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. JEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4396 4415, 2022.
14

---

Published as a conference paper at ICLR 2026
USE OF LLMS
We use large language models (LLMs) solely for language polishing of the final manuscript (cor- recting grammatical errors and refining expression). The models play part in conceptualization, experimental design, theoretical analysis, or any substantive writing. All scientific viewpoints and results remain our sole responsibility.
ETHICS STATEMENT
This study strictly adheres the ethical guidelines and submission requirements of ICLR. The data and code used are legally sourced, with no unauthorized usage. The experimental code is either independently developed reasonably modified based on open-source projects, in compliance with intellectual property regulations, and is prepared for public release as required. All authors declare no relevant conflicts of interest, and the research conclusions are not unduly influenced. The entire work fully meets the academic ethics and compliance standards of ICLR
REPRODUCIBILITY STATEMENT
The code is provided in the supplementary materials to replicate the empirical results.
A PYTORCH-STYLE IMPLEMENTATION OF BIA
def forward (self, x_unlabel, loss):
f_cls_unlabel self. featurizer (x_unlabel) [;, 0] get class token "_cls_unlaber peofepro (f_cls_unlaber get feature embedding -, torch. sech.svd(e (2 (2_cls___nnlla
singular value decomposition loss - self, lambda * sorch.sum/e (s) # BIA
return loss
B DETAILS OF OPTIMIZATION OBJECTIVE OF GCD
The existing GCD proposals are all proposed for compact clustering. Summarizing the optimiza- tion objectives of mainstream schemes GCD (Vaze et al., 2022b), CMS (Choi al., 2024) and SimGCD (Wen et al., 2023), it can be observed that they are based on contrastive learning or proto- type learning significantly reduce the distance between potentially similar samples in the feature space.
B. GCD
The pioneering work (Vaze al., 2022b) divided the mini-batch into labeled B and unlabeled 8" using supervised (Khosla et al., 2020) contrastive learning CLeD
expl2, 21/r)
and self-supervised (Chen al., 2020) con- EiEB [+] Eje log Enei exp(2, 2'/77'
trastive learning LüCD = 18 Eier log Σns, exp(z, <//r) and balancing them using coefficients
exp(x, x//r)
X: LOCD (1 A) + AC where B' (i) represents the collection of samples with the same
label as i. The and z' are augmented from two different views, and the 7 is the temperature.
B.2 CMS
CMS (Choi et al., 2024) and GCD introduces similar supervised and self-supervised contrastive learning. The difference is that CMS introduced mean-shift into unsupervised learning. For the i-th sample, CMS collects the feature set V/ [ of training samples and calculates the k- nearest neighbours N (z,) = {z,} argmaax 2, Zj. where argmax returns a subset of the top-k items. By aggregating neighbor embeddings with weight kernel (p()), it obtains the
15

---

Published as a conference paper at ICLR 2026
embedded representation of samples after mean-shift: 2, EnyeN((s)) (a)a- Lcms and LOCD
ExyeN((z) pll, -2)2,
formally approximate.
B.3 SIMGCD
SimGCD (Wen et al., 2023) constructs a prototype classifier C = [(,, Keran Knnd for both
(k) exp(b (n cx o///
known and unknown classes. It obtains the posterior probability P Ev exp(1 (h, c in similar
way to FixMatch and uses cross-entropy loss L'smCIDD [89 EiES (Vi, P.) on labeled samples.
Self-distillation and entropy regularization SimGCD 1. (P/, Pb) AeH 2 [8]
Eies (Pr + p())
are performed using augmented samples with probability p/-
C PROOFS OF THEOREM
the entropy function H (pi.....Phi Pn)) Σi= Ps log Pt is strictly concave. Furthermore, it is upper-bounded by log n, as demonstrated by the inequality:
Lemma Given non-negative values ps such that Ei=1P: P:
log n H(//n......... 1/n) 2 (PP)....,P,))2
99)
Proof C.1. Refer Section D.I in (Marshall, 1979).
Lemma 2. The ballback-eeible (KL) divergence between two zero-mean, d-dimensional multivariate Gaussian distributions can be formulated as follows:
DKL(N E1)lIN((,, E2))
Σ2
(10)
tr 1 d + log
Σ
Proof C.2. Refer to Section in (Duchi, 2007).
Theorem 1. For given [cls] autocorrelation matrix = CLS CLS/N e Raxd of rank (S d).
log (rank (A)) 2 A (A),
(11)
where equality holds if the eigenvalues of A are uniformly distributed with V V== 1/k and
Qd 1= 0.
Proof C.3. We rely the property that the of eigenvalues equals I (see belows for the detailed proof).
log(rank(AI)
log(k) H(A,,..... Av.)
(12)
(by Lemma
(13)
Ag Aogog As
(14)
j=1
Σ
Ay log Aj
(15)
1=1
((A))
(16)
According to Lemma 1, the inequality equation 13 attains equality if and only 2 1 for all j = =1,2..... k. Equation equation 15 adheres to the convention that 0 Oloo 0 = 0, as per the definition in (Thomas & Joy, 2006).
Here we provide the detailed proof that the sum of cigenvalues of the autocorrelation matrix A is 1.
Suppose have a of n normalized vectors v V2...., Vn Rd where the f--norm of each vector is 1, i.c. ||val22 for all i. The autocorrelation matrix A is defined as the average of the outer products:
A
Σ v.v.
(17)
n
i=1
16

---

Published as a conference paper at ICLR 2026
We seek to show that Σ A 1, where {A,} are the eigenvalues of A. Recall that the trace of matrix is equal to the sum of its eigenvalues, i.e., tr(A) = Σك As.
By the linearity of the trace operator, we have:
tr(A) ir ViV
tr(v.v)).
(18)
72
Using the cyclic property of the trace, specifically tr(xy = x y. we obtain:
tr(v.v.) v. = ||ve 2-
(19)
Since the vectors are normalized (llvi/12 1), follows that vv Substituting this back into the trace equation:
n 1
tr(A) Σ
(20)
n
Thus, the sum of the eigenvalues of A is exactly 1.
C.I BIA'S SURROGATE UNDER TRACE CONSTRAINT
For completeness, we justify the equivalence between the Bures-based loss and the nuclear-norm surrogate used in our implementation. Here we provide a proof sketch and defer further discussion to this appendix.
Let Z E Bxd be the matrix of class tokens mini-batch after LayerNorm, and define the batch Gram DB = zz E BB Since LayerNorm makes each row approximately unit-norm, we have
| 2
tr(xB) tr(ZZT) = |12 ~ B.
(21)
Thus, throughout training the trace EB is well concentrated around the constant B. In other words, the eigenvalues { of ΣB lie on the simplex E, Hy x B.
The squared Bures distance between En and the identity In is
EE BB) tr(En) tr(Tn) 2 2r tr (x tr tr(En) B 21r
(22)
Combining equation 21 and equation 22, and treating tr (r(EB) approximately constant, obtain
(EB, JB) Rs 2B 2 2t (=
(23)
Consequently, minimizing the Bures loss is (up additive constant) equivalent maximizing tr(E1/))
We now relate tr(21//2 the nuclear norm of Z. Let { denote the eigenvalues of Σg and {s, (Z)} the singular values of Z. By construction Σg = ZZ" so its non-zero eigenvalues coincide with the squared singular values of Z:
Hj = sz(
for all non-zero modes j.
(24)
Therefore
1/2 Lr B
= Σ 15 Σ 22(2) = |
(25)
the nuclear norm of Z.
Putting equation 23 and equation 25 together, we obtain the following lemma.
Lemma 3 (Bures-nucleas norm equivalence). Let Z Bxd and ΣB .uppose that tr tr(BB) is (approximately) constant, as induced by LayerNorm or E normalization on rows of Z. Then any minimizer of the Bures distance di, (E,, IB) maximizer of the nuclear norm [|]| and conversely, up to an additive constant independent of Z.
Proof C.4 (Proof sketch). Under the trace constraint tr(EB) z B, the term tr(EB) B in equa- tion 22 is effectively constant, so minimizing d2g (En, IR) is equivalent to maximizing tr(x))/ ), see equation 23, Equation equation 25 shows that (r(E1//2) equals the nuclear norm of Z. Thus any Z that maximizes ||||| (under the same trace constraint) also minimizes d2 (EB, IB), and vice versa
17

---

Published as a conference paper at ICLR 2026
On the 'Mild' Trace Constraint and Approximation Quality. The "mild" nature of our trace constraint stems from its satisfaction by standard normalization practices in modern deep learning. We distinguish between two common scenarios.
Exact Equivalence with & Normalization. Many GCD and representation learning methods apply le normalization to the final embeddings 2;. If each 2; of the stacked matrix Z is fz-normalized such that | = 1, then the trace of the Gram matrix ΣB = ZZ becomes a strict constant:
tr(EB) tr tr(ZZ))
Σ
|
Σ 1=
In this scenario, the tr(EB) term in the Bures distance formula is constant, and minimizing the Bures distance becomes exactly equivalent maximizing the nuclear norm and equation 25.
| || as shown in equation 23
High-Fidelity Approximation with LayerNorm. Our method applied after a LayerNorm layer, standard component in Transformer architectures. While LayerNorm does not strictly enforce || 1, it normalizes the features of each sample to have zero mean and unit variance across the feature dimension, followed by learned affine transformation. This operation ensures that the row norms ||æ are tightly concentrated around a stable value during training, making ir(En) nearly constant. This renders the nuclear norm extremely high-fidelity and empirically effective surrogate for the Bures distance objective. Our experiments confirm this: using the exact Bures loss versus the nuclear- norm surrogate yields nearly identical training dynamics and final performance on all GCD benchmarks, validating that Lemma 3 captures the relevant regime for our method.
D MORE ANALYSIS
D.1 COMPARISON WITH SELE-SUPERVISED LEARNING SCHEMES
In Section 5.4 empirically compare BIA with two representative isotropy-related regularizers, VICReg (Bardes et al., 2022) and CorInfoMax (Ozsoy et al., 2022). Here we give a concise but more detailed analysis of how these objectives differ and why BIA is particularly effective in the GCD setting.
D.1.1 OBJECTIVES AND LEVEL OF OPERATION
VICReg. VICReg is self-super pre- pre-training method combining three terms: (i) an invariance loss between two augmented views, (ii) per-dimension variance term enforcing non-degenerate variance, and (iii) covariance penalizing off-diagonal entries of the feature covariance. Given a batch of features Z E TR. Bxd the covariance penalty acts on Cov(Z) via
Leoy Σ Cov(2)
while the variance term keeps each Var(Z); above threshold. Thus VICReg regularizes features at the instance level and per-coordinate statistics.
CorInfoMax. CorInfoMax also proposed in a self-supervised context and aims to maximize mutual information between representations and an auxiliary target distribution. It discourages trivial collapse by requiring representations remain informative, but does not explicitly control the eigen-spectrum of the covariance matrix.
BIA.
BIA instead operates on the batch class-token matrix used directly for GCD decisions. Let E Bxd be the stacked class tokens after LayerNorm, and let EB = ZZ T Bx denote the batch Gram. Since LayerNorm makes each row satisfy ||z/||2 zz have
13
tr tr(EB) =
Σ
|| B,
18

---

Published as a conference paper at ICLR 2026
which is nearly constant. BIA minimizes the Bures distance between Eg and I,
(Enn, /) tr(Entnnn))))))))))))(() + 2ur(E
Under the trace constraint this equivalent to maximizing tr(=1))) If {1z} are the eigenvalues of Eg, then
tr = Σ 8y =
the nuclear norm of Z. Hence BIA can be seen spectrum-shaping objective that maximizes concave function the eigenvalues of Σg under a mild trace constraint.
In summary, VICReg and CorInfoMax act primarily the instanccccoorddinaee level, whereas BIA di- rectly regularizes the batch class-token Gram in the space where GCD methods perform classification, clustering, and prototype updates.
D.1.2 INSTANCE- WISE VS. SPECTRUM- WISE REGULARIZATION
A key distinction is that VICReg and CorInfoMax essentially coordinate-we wisse instance-wise regularizers, while BIA explicitly spectrum-wise.
The covariance term of VICReg drives Cov(Z (Z) towards diagonal matrix with controlled diagonal entries. This decorrelates coordinates and constrains per-dimension variance, but does not directly reason about the global shape of the eigen-spectrum beyond these coordinate-level constraints. CorInfoMax promotes informative representations, but information can still concentrate a low- dimensional subspace; there no explicit mechanism prevent highly unbalanced eigenvalues.
By contrast, BIA treats the eigenvalues {uz} of En as a whole. Under Σj = tr(En) ~ B, maximizing Σ, vTr is Schur-concave objective: it favors eigenvalue configurations that are more uniform. Intuitively, BIA redistributes energy from overly dominant principal components to smaller ones, increasing the effective rank and von Neumann entropy of the class-token autocorrelation matrix. This directly targets the dimensional collapse and skewed energy distribution observed in GCD.
D.1.3 ALIGNMENT WITH THE GCD SETTING
The GCD setting introduces two challenges that are absent in standard self-supervised pre-training:
unlabeled batches contain a mixture known and novel classes; and
lseudo-labels for novel classes are noisy, often combined with class imbalance and domain
gap.
In this regime, the three regularizers behave differently:
VICReg. The invariance term encourages strong instance-level invariance. With noisy pseudo- labels and tinegraaieed novel categories, this can over-compress intra-class variability and merge distinct novel sub-classes into overly compact clusters. The coordinate- tiite covariance penalty may also suppress directions that remain informative for subtle novel distinctions, making performance sensitive to the loss weight and augmentation strength.
CorInfoMax. Maximizing mutual information can encode both signal and noise. Under noisy pseudo-labels, high mutual information with an imperfect target does not guarantee well-conditioned geometry and can reinforce spurious correlations. Without an explicit anti-collapse or spectrum- flattening term, CorInfoMax does not systematically correct the eigen-structure drift induced by GCD training.
BIA. BIA acts only on the geometry induced by the underlying GCD loss (e.g., SimGCD, CMS). The GCD objective shapes semantic directions by pulling samples of the same psseudo-cclass together and pushing different classes apart. BIA does not attempt to re-learn this structure; instead, it prevents it from collapsing into few dominant directions. By increasing the effective rank and entropy of
19

---

Published as a conference paper at ICLR 2026
the class-token Gram, BIA enlarges the manifold capacity available to encode intra-class variability, which is crucial for distinguishing multiple novel categories that differ only subtly. Since BIA is defined at the batch class-token level, where prototypes and the number of clusters estimated, a hetter-conditioned Gram leads to more stable prototype updates and sharper decision boundaries under noisy pseudo-labels.
Empirically, this alignment with the GCD decision space translates into consistent gains in overall and, in particular, novel-class accuracy when BIA is plugged into strong GCD baselines, whereas directly transplanting VICReg or CorInfoMax often yields mixed or fragile improvements.
D.1.4 ROBUSTNESS AND HYPERPARAMETER SENSITIVITY
Finally, we observe practical difference in robustness. VICReg and CorInfoMax introduce strong instance-level or information- -etioontttcrrticc constraints whose interaction with pseudo-label noise and class imbalance is highly sensitive to loss weights and augmentation policies; hyperparameters tuned on one backbone or dataset do not transfer easily.
BIA instead relies single scalar weight and the mild trace constraint from LayenNorm. Because it only reshapes the spectrum of already task-aligned class tokens, its effect is more uniform across backbones and datasets. Our ablations show that BIA is stable over wide range of À and feature dimensionality, and we use essentially the same À across all GCD methods and benchmarks.
Discussion. Overall, VICReg and CorInfoMax are strong self-supervised methods, but are not tailored to the mixed known/novel, pseudo-label-driven nature of GCD. BIA is explicitly designed as a batch class-token spectrum regularizer under trace constraint, complementary to existing GCD objectives. This design explains why BIA yields more consistent gains both overall and novel-categrry accuracy, as well as improved class-number estimation, in our experiments.
D.1.5 DETAILED COMPARISON AND HYPERPARAMETER ANALYSIS OF SSL METHODS
To provide more thorough assessment of BIA against other coorooy-promoting schemes, we offer a detailed analysis of the compared self-supervised learning (SSL) methods, CorInfoMax and VICReg. Our goal is to clarify their design motivations and conduct a new, in-depth hyperparameter analysis investigate their sensitivity within the GCD context.
Design Motivations and Implementation Details. First, we clarify the design and implementation of the compared methods.
CorInfoMax is an SSL method that maximizes the mutual information between represen- tations. Its loss function consists of similarity term align different views of the same sample and covariance term to regularize the feature covariance matrix. Following the original paper, we set the internal weights for the similarity loss to 500 and the covariance loss to 1. For the overall loss coefficient X, we searched around the value used for BIA (0.004) and found this be optimal for our main experiments.
VICReg an SSL method that learns representations by enforcing three principles: an invariance term (aligning augmented views), a variance term (preventing informational col- lapse along feature dimensions), and covariance term (decorrelating feature dimensions). The full loss is weighted sum of these three components.
Hyperparameter Sensitivity in the GCD Context. We hypothesize that for the GCD task, VI- CReg's invariance term is conceptually redundant with the contrastive objectives already present in GCD baselines like SimGCD. Therefore, the core of its prroopoig effect resides in the variance and covariance penalties, controlled by coefficients f and v. respectively.
To investigate this and ensure a fair comparison, we conducted a new, comprehensive hyperparameter sweep. We integrated only VICReg's variance and covariance terms into the SimGCD baseline and varied their respective coefficients on three grained datasets. The results are summarized in Table 11, Table A2, and Table A3.
20

---

Published as a conference paper at ICLR 2026
Table Al: Hyperparameter ablation for VICReg's uniformity loss components (variance coeff. μ and covariance coeff. v) the CUB dataset, integrated into SimGCD. We report All/Old/New accuracy (%).
j
10
25
40
0.2
58.3 / 62.7 56.0
60.2 64.5
56.5 62.2/112//55 53.99
61.1 / 66.0 58.1
59.8
62.1 58.5
61.3/ 62.7 60.0
60.1 / 66.5 57.2
59.5 64.3 57.1
Table A2: Hyperparameter ablation for VICReg's uniformity loss components on the Stanford Cars dataset.
10
25
40
0.2
67.6/67.8/ /42.3
51.1 / 66.5 43,4
50.6 65.9 43.7
51.1 /42.3
52.0 68.6
47.1
62.7 38.7
52.1 69.1 42.1
49.3 65.4
48.2 66.9 39.8
Table A3: Hyperparameter ablation for VICReg's 's uniformity loss components on the FGVC Aircraft dataset.
\
10
25
40
0.2
50.8
52.4 64.0/600/440 48.9
52.6 55.6 50.8
52.2
02.2/56.//55 50.1
54.6
56.2/
54.8
55.6 53.5
48.8 54.7
49.3 59.1
50.7 56.2 47.8
Conclusion from SSL Methods. Our new experiments reveal two important findings. First, the performance of VICReg is highly sensitive to the choice of hyperparameters, and the optimal setting varies significantly across datasets. For instance, on CUB (Table A1), the best All accuracy (61.3%) achieved with (14, v) (10, 5). while on Stanford Cars (Table A2), the best result (52. () requires after this extensive tuning, the best-performing VICReg configuration
(p, v) = (10,5) 5). Second, on each dataset still does not surpass the results of our proposed BIA method (BIA achieves 62. % All accuracy SimGCD+CUB, outperforming most VICReg settings and being more stable than its peak). BIA remains robust and uses a single hyperparameter setting across all datasets.
These findings reinforce conclusion: while general isotropy regularizers like VICReg can be beneficial, BIA provides a more direct, robust, and effective solution specifically tailored to the geometric challenges of GCD.
D.2 COMPARISON WITH ISOTROPY-ENCOURANEE REGULARIZERS
D.2.1
INTRODUCTION ISO-FROB AND IsO-ENT
In addition to VICReg and CorInfoMax, we also considered two simple, more "direct" isotropy objectives applied to the batch class-token Gram matrix ΣB 77
Liso-frob | tr(Eg) 1||2
(26)
Σg
Liso-aat - H(EB). ΣB tr tr(En)
(27)
where ((En) = -trEp log Ep) is the von Neumann entropy. We refer to these as Iso-Frob and Iso-Ent, respectively. All three objectives (BIA, Iso-Frob, Iso-Ent) promote more isotropic geometry, but they differ in how they act on the eigenvalues of ΣB and how strongly they constrain the covariance, which has practical consequences for GCD.
Spectral view under trace constraint. Let (1n),_1 be the eigenvalues of Σp. Because we apply BIA after LayerNorm class tokens, each 2, satisfies ||=/||2 and thus tr(En) E, 14 B
21

---

Published as a conference paper at ICLR 2026
is nearly constant. In this regime, all three losses can be interpreted as functions the simplex
SB f R E/ B
BIA. The Bures loss is
1) + trr(E
(28)
which, up to an additive constant, is equivalent to maximizing (r(E1)) E, Vij under Ej M4 B. Hence the effective spectral objective is
=
fLIA(H) = Σ ==1
VPy
(29)
The map μ Ej Vuy is symmetric and concave Sg. i.c. Schur-concave; it is maximized at the uniform point (eg 1) and prefers more balanced spectra, but does not force any eigenvalue to match fixed target. In particular, small eigenvalues receive relatively large positive gradients /D///00p 2 1/3), which gently lifts collapsed directions without overly penalizing moderate anisotropy.
Iso-Frob. For Iso- -so-Frob, the spectral form of equation 26 under E3 Ms =B is
fFrot (1)) = Σ (m²-1)²
(30)
f=1
This objective also has its minimum at the uniform point, but it penalizes quadratic deviations from the exact spherical target μ, along each eigen-direction. As result, Iso-Frob behaves like strict whitening penalty: any structured anisotropy (even if it is semantically meaningful for separating classes) is penalized proportionally to the squared deviation. In the GCD setting, where batches mix known and novel classes and pseudo-labels are noisy, we find this rigidity undesirable:
When pseudo-labels are imperfect, some anisotropy reflects meaningful semantic structure; aggressively driving Σg towards scaled identity can partially undo class separation learned by the GCD loss.
Quadratic penalties yield gradients that grow linearly with |0z - 1/, so large deviations (e.g. caused by outliers or class imbalance) dominate the update and lead to regularization of a few directions.
Empirically, Iso-Frob improves baselines modestly but is consistently weaker and less stable than BIA in our GCD experiments.
Iso-Ent. For Iso-Ent, using the normalized spectrum رy = 11// Σx M% = 14/ BB, the von Neumann entropy reduces to Shannon entropy:
H{En) = - Σ
na log Bg. [Fmi(1)) - HHSB) = Σ na log Pg.
(31)
This is again symmetric, Schur-concave function maximized the uniform spectrum. However, its gradient with respect to μı is proportional to log ū. + 1, which diverges as ğ. - 0. Thus Iso-Ent strongly amplifies very small eigenvalues, making numerically and statistically sensitive:
Small eigenvalues, which may correspond to noise spurious directions induced by incor- rect pseudo-labels, receive extremely large updates and can be over-emphasized.
Computing matrix logarithms and the corresponding gradients is less stable than the SVD- based square root used in BIA, especially when Σg is near- vear-sin early in training.
D.2.2 COMPARISON WITH ISOTROPY REGULARIZERS UNDER VARYING BATCH SIZES
To further situate BIA, we conducted additional experiments comparing it against two simple, more "direct" isotropy objectives applied to the class-token Gram matrix Σg. This analysis examines the
22

---

Published as a conference paper at ICLR 2026
Table A4: Comparison with more isotropy-enccorraiing regularizers on the CUB dataset.
Regularizer + Batch Size
CUB (AII)
CUB (Known)
CUB (Novel)
Iso-Frob
32
51.9
55.9
49.9
64
60.3
70.7
55.3
128
61.5
69.4
57.4
Iso-Ent
32
48.8
56.0
45.2
64
59.3
68.9
54.9
128
61.8
68.1
57.1
BIA (Ours)
32
53.1
58.5
50.4
64
59.5
69.3
54.6
128
62.1
65.8
60.3
robustness of each regularizer changes in batch size, critical factor for training stability and performance.
We integrated these regularizers and BIA into the SimGCD baseline and evaluated their performance on three fine-grained datasets with batch sizes of 32, 64, and 128. The results are presented in Table A4.
As shown in Table A4, our analysis yields several key insights. First, BIA consistently outperforms both Iso-Frob and Iso-Ent across all datasets and batch sizes, demonstrating its superior effectiveness. Second, BIA exhibits remarkable robustness to variations in batch size, with performance remaining stable from 32 128. contrast, both Iso-Frob and, particularly, Iso-Ent show greater sensitivity, with performance degrading more noticeably at smaller batch sizes.
These empirical results align with our theoretical understanding. (i) Iso-Frob enforces rigid whitening penalty, attempting to match an exactly spherical covariance. This can be overly restrictive the noisy GCD setting, where some degree of structured anisotropy learned from pseudo-labels may be beneficial for class separation. The aggressive regularization can thus suppress useful semantic information. (ii) Iso-Ent, which involves matrix logarithms, is known to numerically sensitive, especially to small cigenvalues that are common early in training with small batches. This sensitivity likely contributes to its more volatile performance.
In contrast, BIA's Bures nuclear- norm formulation avoids these pitfalls. It maximizes concave function of the eigenvalues under a mild trace constraint, which encourages a more uniform spectrum by reshaping it (lifting smaller eigenvalues and gently suppressing larger ones) without forcing all directions to a specific target. This more flexible and robust mechanism for promoting isotropy appears exceptionally well suited the noisy, mixed known/novel regime of GCD.
D.3 IMPACT OF EMBEDDING QUALITY
In Table 2, the accuracy gains the CIFAR100 and Herbarium19 datasets are marginal. We use this as starting point to analyze the conflict between enhancing feature completeness and low embedding quality in GCD. DINO, through self-supervision, already has a good feature representation capability, but due to the distribution of data, its embedding quality remains low. One source of low quality is the data size, and the other data semantics.
(1) Specifically, when the small-sized CIFAR100 images are interpolated and input into ViT, the high- frequency information is lost. For example, when identifying animal categories, the low-frequency features such as the outline of the animal may be captured relatively well, but the detailed features such as the texture and eyes of the animal (high-frequency features) are difficult to accurately extract. In this case, the model can only cluster through some shorteut information, rather than accurately clustering based on the complete intra-clas features. Since the manifold dimension the low- frequency features is relatively low, it is unable fully capture the diversity and complexity within
23

---

Published as a conference paper at ICLR 2026
(a) SimGCD
(b) with CorInfoMax
(c) with VICReg
(d) with Ours
Figure A1: Trends in the top 10 eigenvalues as the number of training steps grows.
the class. Therefore, enhancing the completeness of the intra-class representation on is challenging.
data
(2) Herbarium19 large-scale herbal plant recognition dataset, which is not in the model's training data and inherently cannot provide highly discriminative representations. Additionally, the large number categories makes the decision boundary more chaotic, and existing GCD schemes cannot cluster well. Therefore, enhancing the completeness of intra-class representation on overly low- quality embeddings is not feasible, as the overlap of feature spaces across categories is too large, and samples within cluster come from multiple categories.
D.4 ANALYSIS ON THE EVOLUTION OF EIGEN
Compared to the two optimization directions, VICReg and CorInfoMax, BIA offers smoother and more uniform convergence of feature values, addressing some key limitations in both methods (Figure AI). VICReg, as variance- regularization approach, promotes feature variance and decorrelation but lacks explicit emphasis on intra-class representation completeness. This results in less expressive class boundaries and less effective fine-grained category separation. CorInfoMax, on the other hand, focuses on maximizing mutual information between features and their target distribution but does not sufficiently prevent dimensional collapse or guarantee richer intra-class representations. Both methods, while effective some contexts, fail to fully capture the complex, high-dimensional structure of the data.
In contrast, BIA directly targets the manifold capacity of class tokens, ensuring that intra-class representations remain complete and informative. By maximizing the nuclear norm of the class token's ' singular values, BIA ensures that feature values converge uniformly, without the collapse seen in other methods. This leads more robust and accurate clustering, particularly when discovering novel categories. The smooth convergence of BIA reflects its ability to optimize representation quality while maintaining high inter-class separability, which is critical for open-world learning tasks.
D.4.1 COMPUTATIONAL OVERHEAD ANALYSIS
We quantitatively analyze the computational overhead of BIA to substantiate our claim that it is lightweight, plug-and-play module.
Empirical Validation. To verify this empirically, we conducted two sets of measurements. First, we measured the wall-clock time of the SVD step (the most complex part of BIA) and its relative overhead compared to the backbone's computation time. As shown in Table A5, the SVD step's contribution is minimal, accounting for less than 1.5% of the backbone's compute time even with batch size of 256. This confirms that the core operation of BIA is negligible in practice.
Table A5: Relative wall-clock time overhead of the SVD computation in BIA with respect to the VIT backbone's forward- forward-backward pass, measured across different batch sizes.
Batch Size
64
128
192
256
Time Overhead
0.37%
0.81
1.01 %
1.47%
Second, we evaluated the total wall-clock time per epoch and peak GPU memory for representative GCD frameworks with and without BIA. The results in Table A6 show that integrating BIA increases
24

---

Published as a conference paper at ICLR 2026
the total training time by less than 1% and the peak memory usage by only few tens of megabytes across diverse methods like SelEx, SimGCD, CMS, and SPTNet.
Table A6:
Total training time per epoch and peak GPU memory overhead of BIA when integrated into representative GCD frameworks on the CUB dataset.
Method
SelEx
SimGCD
CMS
SPTNet
Peak GPU Memory (MB)
Original
8640
6354 6390
6040
23322
BIA (Ours)
8674
6074
23670
Time per Epoch (s)
Original
24.99
24.50
28.81
22.09
+ BIA (Ours)
25.16
24.67
28.99
22.26
Collectively, these results provide strong quantitative evidence that BIA is computationally efficient, imposing negligible burden on standard training pipelines while delivering significant performance gains.
25


Reviews:
[{"review_id": "CW5HjV2X4t", "reviewer": "Reviewer_fKEF", "summary": "The paper tackled the setting of generalised category discovery (GCD). It proposes to add a generic loss on the CLS token to maximise isotropy across the batch in the embeddings. The paper shows this can be combined with several exists GCD works to yield fairly consisten improvements across standard GCD benchmarks.", "questions": "* Why do other isotropy regularisation objective not work as well for the most part? \n* What is the dependency of batch size?\n* Why does the method not yield gains on Herbarium?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* the method is extremely simple and only has one hyperparameter\n* the improvements are consistent\n* theoretical analysis is provided and insightful", "weaknesses": "* Figure 1 does not really provide any insight as it remains very high-level.\n* Same for Figure 2, it does not provide any new insights\n* The idea seems to be very related to VICReg and TIan's uniformity loss. Yet both of these relevant works from reprsentation learning literature are only briefly compared to in Sec 5.4. It's unclear what the setting of choosing (the extremely important!) hyperameters for these experiments were. Note that even if \"+VICReg\" performed better than the proposed \"+bures\", it would still be insightful for the ICLR community that an isotropy regulariser seems to be missing for GCD. \n* Similarly it would be good to ablate against other isotropy-encouraging regularisation losses and the dependency on the batchsize is also not analysed\n* the hypothesis of \"poor eigen-structure, skewed energy distribution, and fragile decision regions, that ultimately impedes category discovery and class-number estimation.\" is mentioned in the introduction as a key motivation but not quantitatively shown.\n* the explanation for improved performance seems to not be super clear. if full-rank manifolds lead to better performance, just using random matrices (which is full rank) should yield good performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "k0XAWc9u1O", "reviewer": "Reviewer_SYXh", "summary": "The paper introduces Bures–Isotropy Alignment (BIA) for generalized category discovery (GCD). It measures the Bures distance from the batch class-token Gram matrix to the identity. BIA improves accuracy and class-count estimation across multiple baselines and datasets.", "questions": "1. When does $d_B^2(\\Sigma_B,I)\\downarrow \\Longleftrightarrow |Z|_*\\uparrow$ fail?\n\n2. What happens if BIA uses feature-Gram $Z^\\top Z$ or includes patch tokens (e.g., pooling or grouped variants) instead of only $[\\mathrm{cls}]$? Any trade-offs in accuracy and cost on fine-grained datasets?  \n\n3. Analyze the regressions in Table 3. Are they linked to class imbalance, pseudo-label noise levels, or $K$-estimation errors in CMS?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper reframes GCD from “tighter clusters” to restoring isotropy of class-token geometry, and offers a plug-and-play, architecture-agnostic regularizer that targets dimensional collapse and stabilizes the feature spectrum.", "weaknesses": "1. You keep saying the equivalence holds “under a mild trace constraint” and “once row norms are stabilized with LayerNorm,” but the paper never pins down what that actually means. Please spell out the exact assumptions, edge cases where the surrogate can drift from the metric, and give at least a short proof sketch so readers know when it’s safe to use.\n\n2. Compute cost is called “negligible,” and the setup mentions 4×RTX4090, but there are no numbers. Report wall-clock time per epoch, peak GPU memory, and the stability of SVD or PD-root operations as $B$ and $d$ grow, plus failure rates if any. Without that, it’s hard to judge practicality.\n\n3. Section 3.2 builds geometry only on stacked [cls] tokens with $\\Sigma_B = ZZ^\\top$. That leaves patch-level interactions on the table. For fine-grained tasks, this likely matters. Either show ablations that use patch tokens or argue why a class-token-only view is sufficient. \n\n4. Gains are not uniform. With unknown $K$ in Table 3, some baselines drop when adding BIA, e.g., CMS on CIFAR100-New −1.6 and Herbarium19-New −1.0. Please analyze when BIA helps and when it hurts, and include stress tests for noise and class imbalance.\n\n5. Robustness ablations are thin. Figure 4 only varies $\\lambda$ and feature dimension $D$. Batch size $B$, within-batch class imbalance, and pseudo-label noise all directly change the batch Gram $\\Sigma_B$ but are not studied. These should be evaluated.\n\n6. Table 5 compares “representative isotropic feature distribution schemes,” but the theoretical relation to decorrelation/whitening objectives remains informal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lfScqy43DI", "reviewer": "Reviewer_aYVi", "summary": "This paper proposes a novel loss function, termed Bures-Isotropy Alignment (BIA) loss, which aims to enhance intra-class representation completeness, while maintaining inter-class separability in the context of Generalized Category Discovery (GCD)—a task involving clustering over both known and unknown categories. The BIA loss is inspired by the insight that when class-token covariance matrices, constructed from class-specific representations, approach an isotropic form, the learned representation space becomes more completed and less susceptible to collapse. Mathematically, the loss is formulated as the nuclear norm of stacked class representations based on the Bures-Isotropy alignment principle. Experimental results show that incorporating the BIA loss into baseline GCD models, *i.e.*, SelEX, SimGCD, CMS, SPTNet, consistently improves accuracy across various datasets and architectures, for both known and unknown categories. Furthermore, the method achieves superior performance in category number estimation, reinforcing its overall effectiveness.", "questions": "**Is the proposed BIA loss differentiable?** \n\nIn other words, is the process of computing eigenvalues from the covariance matrix differentiable? It appears that it should be, since (i) both matrix decomposition and squared eigenvalue operations, involved in loss calculation, have known derivative forms, and (ii) the paper reports successful training results using this loss. \n\nI would appreciate clarification on whether the authors have a direct explanation or confirmation regarding the differentiability of the BIA loss.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The motivation and insight that encouraging class-token covariance to become isotropic leads to more robust within-class representations is both intuitive and compelling. The theoretical support using Bures-Isotropy alignment (Section 3.3) is mathematically well-grounded and strengthens the conceptual foundation.\n\n- The process of deriving a simple and practical loss formulation from this motivation is clear and elegant, making the approach easily applicable to future work in the GCD research community.\n\n- The experimental design is thorough and effectively demonstrates both the simplicity and generality of the proposed method.\n\n- Incorporating the BIA loss consistently leads to improved performance in Generalized Category Discovery tasks and category number estimation across various datasets and baseline models, highlighting the robustness and versatility of the approach.", "weaknesses": "I do not have any major concerns with this submission; however, I have several minor suggestions for clarification and completeness.\n\n- In Section 3.4, the paper shows that incorporating the BIA loss during training improves Neumann Entropy, which measures the uniformity of class-specific representation distributions. However, since both BIA and Neumann Entropy are defined in a similar manner—being proportional to the eigenvalues—the observed positive correlation between the two is somewhat expected. It would be helpful to clarify what specific insight or takeaway this section intends to convey beyond confirming this relationship.\n\n- Section 5.4 presents interesting observations, but without sufficient details about the compared methods (e.g., CorInfoMax and VICReg), it is difficult to fully understand or assess the comparison. Please provide additional details or the design motivations of these methods to improve clarity.\n\n- Figure 1 is currently not referenced anywhere in the manuscript. It should be mentioned and discussed in an appropriate place to ensure proper integration into the paper’s narrative.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aYbFM673jV", "reviewer": "Reviewer_trb7", "summary": "This paper addresses a core challenge in the Generalized Category Discovery (GCD) task: characterizing geometric quality degradation. It proposes an innovative method called Bures-Isotropy Alignment. GCD aims to simultaneously identify known categories and discover novel categories from unlabeled data.", "questions": "See Weaknesses", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Instead of following mainstream approaches to further pursue feature compression, this paper astutely points out the fundamental bottleneck in current GCD research: the degradation of representational geometric quality. It offers a novel and insightful solution from the perspective of \"restoring geometry\" rather than \"compressing features.\"\n\nThe paper successfully introduces the Bures metric from quantum information science into the field of machine learning and cleverly derives that minimizing it is equivalent to maximizing the nuclear norm of the feature matrix.", "weaknesses": "1. The paper emphasizes its connection to the literature on representation collapse, but could delve deeper into the specific differences and connections between BIA and other methods in self-supervised learning aimed at solving similar problems, thus more clearly defining its unique contribution.\n\n2. The paper's core argument is \"geometry recovery,\" but BIA is essentially still an optimization objective that indirectly \"recovers\" geometry by changing the optimization direction. A more in-depth discussion could explore how this optimization dynamically corrects the geometric structure during training, rather than just post-hoc statistical analysis.\n\n3. The paper mentions that BIA has low computational overhead, but provides no quantitative data (such as the percentage increase in training time or changes in GPU memory usage) to support this claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)