Title: Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective
Venue: NeurIPS 2025
Accepted: yes
Keywords: Generalized Category Discovery

Domain Shift
Abstract: Generalized Category Discovery (GCD) aims to leverage labeled samples from known categories to cluster unlabeled data that may include both known and unknown categories. While existing methods have achieved impressive results under standard conditions, their performance often deteriorates in the presence of distribution shifts. In this paper, we explore a more realistic task: Domain-Shifted Generalized Category Discovery (DS\_GCD), where the unlabeled data includes not only unknown categories but also samples from unknown domains. To tackle this challenge, we propose a \textbf{\underline{F}}requency-guided Gene\textbf{\underline{r}}alized Cat\textbf{\underline{e}}gory Discov\textbf{\underline{e}}ry framework (FREE) that enhances the model's ability to discover categories under distributional shift by leveraging frequency-domain information. Specifically, we first propose a frequency-based domain separation strategy that partitions samples into known and unknown domains by measuring their amplitude differences. We then propose two types of frequency-domain perturbation strategies: a cross-domain strategy, which adapts to new distributions by exchanging amplitude components across domains, and an intra-domain strategy, which enhances robustness to intra-domain variations within the unknown domain. Furthermore, we extend the self-supervised contrastive objective and semantic clustering loss to better guide the training process. Finally, we introduce a clustering-difficulty-aware resampling technique to adaptively focus on harder-to-cluster categories, further enhancing model performance. Extensive experiments demonstrate that our method effectively mitigates the impact of distributional shifts across various benchmark datasets and achieves superior performance in discovering both known and unknown categories.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on generalized category discovery under domain shift, which is more practical compared to traditional UDA and GCD. To handle this scenario, this paper proposes a divide and conquer strategy based on the frequency domain, namely Free, which includes five objective functions.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\nThis paper tackles a more realistic scenario, and the reported results show noticeable improvements on the chosen benchmarks.\n\n\n## Weaknesses\n\n\n>1. Limited novelty in motivation and methodology.\n>> The proposed framework is a divide-and-conquer integration of UDA and GCD techniques. From the perspective of motivation, this paper only states that “GCD fails under domain shift” without offering deeper insight. From the perspective of methodology, frequency-domain manipulation has been widely used in UDA [1, 2], and the corresponding components in the proposed Free also handle domain shift merely. As a result, the method may appear more like an extension of UDA rather than a principled solution to the core challenges of GCD.\n>2. **Missing key related work.**\n>> CDAD-Net [3] tackles essentially the same setting but is neither cited nor compared. The manuscript must clarify conceptual differences and provide empirical comparisons.\n>3. Complex method without hyper-parameter analysis.\n>> The proposed pipeline contains five components, yet no parameter-sensitivity study is provided. Are all hyper-parameters fixed to the values listed in the paper?\n>4. Writing issues.\n>>* *Unknown domains* is confusing, since these unlabeled domains are visible during training. Target domains would be clearer.\n>>* The methodological description in the abstract and introduction is vague, where the rationale for each component is not evident.\n\n[1] Kumar, Vikash, et al. \"Improving domain adaptation through class-aware frequency transformation.\" International Journal of Computer Vision 131.11 (2023): 2888-2907.  \n[2] Kumar, Vikash, et al. \"Caft: Class aware frequency transform for reducing domain gap.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.  \n[3] Rongali, Sai Bhargav, et al. \"CDAD-Net: Bridging Domain Gaps in Generalized Category Discovery.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\nQUESTIONS:\n>1. Eq.(11) seems a probability, not an objective function, how to utilize it?\n>2. What is the setting in Fig. 2?\n\nLIMITATIONS:\nThe limitations can be found in the weaknesses and questions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a frequency-based approach to Generalized Category Discovery (GCD) that is designed to be robust under domain shift. The method consists of four main components. First, it separates known and unknown samples using a Gaussian Mixture Model (GMM). Next, it introduces Cross-Domain Frequency Perturbation (CDFP), which applies Fourier-based augmentations to samples of the same category across different domains, encouraging the model to generalize across domain boundaries. The third component, Intra-Domain Frequency Perturbation (IDFP), enhances robustness by applying similar augmentations within a single domain. Lastly, the method incorporates Difficulty-Aware Sampling, enabling the model to focus more on challenging samples during clustering.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1- The method leverages Fourier-based frequency perturbations for both inter-domain (same class across domains) and intra-domain (within-domain) settings, which enhances the model’s ability to generalize under domain shift.\n\n2- The introduction of Difficulty-Aware Sampling is a useful addition, helping the model focus on more challenging categories and potentially improving clustering performance across a range of scenarios.\n\n3- The approach demonstrates effectiveness across multiple state-of-the-art baselines and includes extensive ablation studies (detailed in the appendix), supporting the robustness of the proposed contributions.\n\n\n**Weaknesses:**\n\n1- Although the paper claims five key contributions, several of them do not appear to be particularly major novel contributions:\n\nFDS (Frequency-based Domain Separation): The use of a Gaussian Mixture Model (GMM) to separate known and unknown domains is introduced without clear justification or analysis of its necessity. It remains unclear how much this component contributes to the overall performance. While a semi-supervised K-means variant is briefly mentioned, it is not thoroughly explored or compared. Furthermore, it is not specified whether this clustering was performed in the frequency domain or on raw images, an important distinction, as K-means might perform better than GMM in image space.\n\nIDFP (Intra-Domain Frequency Perturbation): This is essentially a basic augmentation strategy that has been explored in prior work. It is not clearly tied to the specific goals of Generalized Category Discovery, limiting its significance as a standalone contribution.\n\n2- The related work section is lacking in important citations, both in the context of Generalized Category Discovery and frequency-based methods. For example, in line 112, the phrase \"recent research has explored frequency-based domain adaptation strategies\" is vague and does not cite any specific papers. Related work should explicitly reference prior studies to provide proper context, and prior methods such as ORCA are missing entirely from the related works section while being reported in experiments. \n\n3- For a method composed of multiple components, the ablation studies in the main paper are limited, and the experiments in the appendix are introduced without sufficient explanation or discussion. This makes it difficult to understand the individual contribution of each module or justify design choices.\n\n4- The writing could be improved for clarity. While the general structure is understandable, figures and tables are not always well explained, and several new terms and notations are introduced without clear definitions, which makes the paper more difficult to follow and interpret.\n\n*Minor:*\nConsider using an acronym (e.g., FREE) or consistently formatting the method name in bold or italics throughout the paper. Using “Free” in plain text makes it harder to distinguish from regular words.\n\nThe distinction between losses introduced by SimGCD and standard GCD losses is unclear. For clarity, it would help to explicitly label which components originate from prior GCD frameworks and which are specific to SimGCD.\n\nThe method mentions swapping low-frequency amplitudes, but it does not clearly define what constitutes \"low frequency.\" Specifying a threshold or frequency range would provide a better understanding. Although there is some mention of low frequency in the appendix, it would be helpful to explicitly state in the main text how low-frequency components are identified and how the swapping is implemented, or to clearly direct readers to the relevant section in the appendix.\n\nThere are several notations introduced that are used only once, which adds unnecessary complexity. Simplifying these would improve readability.\n\nLine 239: The notation $E(x_i)$ is used without definition; please clarify what this refers to.\n\nLines 261 and 264: It may be helpful to explicitly mention “HiLo” here so that readers understand the context without backtracking.\n\nFigure 3: The figure would benefit from a clearer explanation, both in the caption and in the main text. Specifically, mapping each part of the diagram to its corresponding loss component would improve interpretability.\n\nLine 333: \"hindering\" -> \"hinders.\"\n\nLine 447: The term \"Theoretical Analysis\" is misleading here; the content largely consists of background explanation on Fourier transforms. Consider renaming this section to Preliminaries for accuracy.\n\nLine 458: The phrase “relative strength between different directional templates” is ambiguous. Additionally, please ensure that all formulas and technical statements are carefully reviewed for accuracy. The supplementary material, particularly in lines 458, 460, 463, and 465, sometimes lacks clarity and displays inconsistent style. If these sections were drafted with the help of AI tools, please carefully review them.\n\n\nLine 472: The paper frequently omits proper citations. For example, DomainNet should be cited.\n\nA pseudocode explaining the method in the appendix can be helpful to understand the steps well.  \n\n**Quality:**\nWhile the motivation and core insights of the method are strong, the paper would benefit from further revision and refinement, especially in the appendix.\n\n**Clarity:**\nSome explanations lack clarity, and the use of notation is inconsistent, sometimes underexplained, other times overly detailed, which can make the paper challenging to follow.\n\n**Significance:**\nThe method enhances domain adaptability through augmentation. Given that HiLo has already addressed domain shift within generalized category discovery, this work primarily positions itself at the intersection of domain adaptation and generalized category discovery, and more as a solution to an already established problem. \n\n**Originality:**\nAlthough Fourier amplitude swapping has been explored in domain generalization and adaptation, its application to generalized category discovery is novel. Additionally, the introduction of difficulty-aware sampling is an interesting contribution.\n\nQUESTIONS:\n1- Regarding Table 6 in the appendix, GMM is set to two modes in the main paper, which aligns with separating known and unknown categories. However, in the context of the appendix, there appear to be more than just two relevant categories, and the distributions seem to differ significantly. Since there are no experiments directly demonstrating the impact of this GMM setup with different mode numbers, does this table suggest that using GMM in this way may not be effective?\n\n2- Since the proposed difficulty-aware clustering and Fourier augmentation are not inherently limited to scenarios involving domain shift, what would be their impact when applied to a standard GCD baseline? It would be interesting to see if these components also benefit generalized category discovery in the absence of domain adaptation.\n\nLIMITATIONS:\nWhile the authors acknowledge that their work is currently limited to image classification, it would also be valuable to discuss some method-specific limitations. For example, the use of FFT introduces additional computational overhead, which may impact scalability. Additionally, the effectiveness of the GMM component relies on its ability to accurately separate known and unknown categories, which may not always hold in practice. Addressing these potential limitations would provide a more balanced perspective on the approach.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces *Frequency-guided Generalized Category Discovery (Free)*, a novel framework for Domain-Shifted Generalized Category Discovery (DS\\_GCD), which aims to categorize both known and unknown classes from unlabeled data spanning both seen and novel domains. In contrast to existing approaches, *Free* takes a frequency-domain perspective by leveraging the amplitude and phase of the Fourier transform to model style and semantic content, respectively. Experiments on DomainNet and the newly proposed SSB-C benchmark demonstrate that *Free* achieves superior results compared to prior GCD and NCD methods.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. The proposed *Free* framework consistently outperforms a wide range of state-of-the-art methods on challenging domain-shifted benchmarks.\n2. The paper is clearly written and well-structured, making it accessible and easy to follow.\n\n**Weaknesses**\n\n1. While the use of Fourier-based techniques is innovative in the GCD context, such methods are well-studied in Domain Adaptation (DA) and Domain Generalization (DG). Given that the datasets are well-examined in DA/DG literature, it is not entirely surprising that *Free* achieves improved results.\n2. Although the paper explains how frequency manipulation may help mitigate domain shifts, it lacks a deeper analysis of how such manipulation specifically aids the discovery of unknown classes. For example, does the Intra-Domain Frequency Perturbation (IDFP) affect intra-class compactness or inter-class separability?\n3. The *Free* framework comprises four main components, resulting in a relatively complex pipeline. The reliance on Fourier transforms and extensive data augmentations may introduce considerable computational overhead. The paper does not provide an analysis of this computational cost (e.g., training time, FLOPs) relative to the baseline methods.\n4. Minor issue: Lines 458, 460, 463, and 465 contain Markdown syntax.\n\nQUESTIONS:\nPlease refer to the weakness part.\n\nLIMITATIONS:\nPlease refer to the weakness part.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel framework, \"Free,\" to tackle Domain-Shifted Generalized Category Discovery (DS-GCD), a challenging setting where unlabeled data contains both new categories and samples from unseen domains. The method is built on the core insight that an image's Fourier amplitude spectrum captures domain-specific style, while the phase spectrum retains semantic content. \"Free\" leverages this by first probabilistically separating samples into known and unknown domains based on amplitude similarity. It then employs class-aware frequency perturbation strategies to align distributions and enhance robustness, further guided by a difficulty-aware sampling mechanism. Extensive experiments on DomainNet and SSB-C benchmarks show that the proposed method significantly outperforms prior state-of-the-art approaches.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* S1: Novel and Effective Approach for an Important Problem. The paper tackles the highly relevant and challenging problem of Generalized Category Discovery under a domain shift (DS-GCD). The core technical contribution—using the Fourier amplitude and phase spectra to disentangle domain-specific style from semantic content—is an elegant, well-motivated, and novel approach.\n* S2: State-of-the-Art Empirical Performance. The empirical evaluation is extensive and demonstrates state-of-the-art performance. The proposed 'Free' method consistently and substantially outperforms all baselines, including the specialized DS-GCD method HiLo, across the challenging DomainNet and SSB-C benchmarks (Tables 1 & 2). The magnitude of the gains convincingly validates the method's effectiveness.\n* S3: High-Quality and Rigorous Presentation. The paper is well-written and easy to follow. The authors' claims are supported by detailed ablation studies (Table 3) that clearly justify the contribution of each component. The inclusion of a comprehensive appendix with additional analyses further strengthens the quality and completeness of the work.\n\nWeaknesses:\n* W1: Limited Comparison to Recent GCD Methods. A key premise of the paper is that standard GCD methods are not suitable for the DS-GCD setting. However, the baselines used to demonstrate this (e.g., GCD, SimGCD) are not the most recent. Including stronger, contemporary GCD methods (such as ProtoGCD [1], CMS [2], or SPTNet [3]) and showing that they also struggle under domain shift would provide a more powerful motivation for the necessity of a specialized approach like Free.\n* W2: Incremental Novelty of Some Components. While the overall framework is novel, the novelty of some individual components is limited. For instance, the frequency-based perturbation is a well-designed, class-aware extension of the technique from FDA[4], and the clustering-difficulty-aware sampling is a relatively common strategy for handling hard examples. The paper would be strengthened by a clearer discussion of these connections.\n* W3: High Methodological Complexity and Incomplete Analysis. The proposed framework is composed of four main modules, resulting in a system with high methodological complexity and lots of hyperparameters that could be a practical drawback for adoption. The hyperparameter sensitivity analysis in the appendix is appreciated but incomplete; it omits several key parameters introduced by the method, such as the confidence threshold $\\mu$. A more complete analysis would increase confidence in the method's robustness.\n\n[1] Protogcd: Unified and unbiased prototype learning for generalized category discovery\n\n[2] Contrastive mean-shift learning for generalized category discovery. \n\n[3] Sptnet: An efficient alternative framework for generalized category discovery with spatial prompt tuning. \n\n[4] Fda: Fourier domain adaptation for semantic segmentation.\n\nQUESTIONS:\n1. Regarding Baselines: The paper's motivation hinges on the premise that standard GCD methods struggle under domain shift. To make this point more compelling, could you add more recent and powerful GCD methods like ProtoGCD [1], CMS [2], and SPTNet [3] into your main table? Demonstrating that these stronger methods also fail in the DS-GCD setting would significantly strengthen the paper's central motivation.\n2. Regarding Hyperparameter Sensitivity: The proposed 'Free' framework is methodologically complex. While the sensitivity analysis in Section B.7 is helpful, it appears to omit a key hyperparameter. Could you please elaborate on the model's sensitivity to the confidence threshold $\\mu$ used in CDFP? A more complete analysis would increase confidence in the method's robustness.\n3. Regarding Statistical Significance: The main results in Tables 1 and 2 report the mean accuracy over three runs but omit standard deviations or other measures of variance. Could you provide these values? This information is crucial for assessing the statistical significance and stability of the reported performance gains, especially for a complex, multi-component method.\n\n[1] Protogcd: Unified and unbiased prototype learning for generalized category discovery\n\n[2] Contrastive mean-shift learning for generalized category discovery. \n\n[3] Sptnet: An efficient alternative framework for generalized category discovery with spatial prompt tuning.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}