Title: Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: edge filter

deep feature

frequency
Abstract: We introduce the Deep Edge Filter, a novel approach that applies high-pass filtering to deep neural network features to improve model generalizability. Our method is motivated by our hypothesis that neural networks encode task-relevant semantic information in high-frequency components while storing domain-specific biases in low-frequency components of deep features. By subtracting low-pass filtered outputs from original features, our approach isolates generalizable representations while preserving architectural integrity. Experimental results across diverse domains such as Vision, Text, 3D, and Audio demonstrate consistent performance improvements regardless of model architecture and data modality. Analysis reveals that our method induces feature sparsification and effectively isolates high-frequency components, providing empirical validation of our core hypothesis. The code is available at \url{https://github.com/dongkwani/DeepEdgeFilter}.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the Deep Edge Filter, a high-pass filter for deep features, to improve model generalizability. Its key innovations include: (i) a modality-agnostic Edge filter that can facilitate the extraction of generalizable features; (ii) a simple but effective hand-crafted edge filter operator for both CNN and ViT architectures, demonstrating good cross-task generalizability across various modalities including image, text, 3D and audio. Ablation studies demonstrate the effectiveness w.r.t. layer sparsity and frequency.\n\nSTRENGTHS AND WEAKNESSES:\nMajor Strengths\n1. \tThis worke finds an interesting pattern within deep features that a simple use of hand-crafted mean filter can decouple the semantic components from the domain-specific components within deep features. As far as I can tell, this finding, while being empirical, is novel and non-trivial.\n2. The deep edge operator is validated across a diverse range of multi-modal data, substantiating its universal applicability to models and data of differing modalities.\n3. The proposed fix is simple yet incredibly effective, exhibiting significant potential to extend to a wide range of tasks.\n\nMajor Weaknesses\n1.\tThe core hypothesis, h=h_dom+h_sem, relies predominantly on empirical evidence from prior literature without providing direct evidence; the paper only presents preliminary experiments indicating improved model performance upon removal of low-pass components.\n2.  The claim that “we generalize the concept of Edge Filters for deep features that can be applied directly to deep layers rather than the input layers” may be overclaimed, as learned convolutional filters are also a form of edge filters, potentially misleading readers regarding the contribution of the work.\n3.  The formulation may lack robustness; for instance, in “Sparse Coding through High-Pass Filtering”, the authors may fail to elucidate the nature of high-pass filtering under sparse coding, while merely presenting the high-pass filtering equation (3) and the sparse coding equation (4) alone.\n\nMinor Weaknesses\nAlthough the study encompasses multiple modalities, the experimental comparisons within individual modalities are limited by a few methods and the absence of SOTA approaches.\n\nQUESTIONS:\n1.\tDoes the position of the mean filter applied significantly influence the results? Have the authors tried to apply the filter to alternative layers? In addition, does multiple edge filters for multiple layers help?\n2.\tWhether the edge filter could work by training from scratch, or is its efficacy is limited to pre-trained models? Additional experimental results addressing this point may strengthen the application scenario of the proposition.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes introducing high-pass filters (edge filters) as a single filter in neural networks to improve their performance. The core hypothesis is that deep features can be decomposed into semantic- and domain-specific information, where the semantic information resides in the high-frequency components of the features and the domain information resides in the low-frequency components of the features. Edge filters operate by subtracting a mean-filtered features from the features, creating essentially a high-pass filter.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The method is simplistic and easy to integrate into existing architectures. The proposed method is architecture-agnostic and, therefore, is very compelling.\n2. The paper provides empirical evidence that shows an improvement when using their method.\n\nWeaknesses:\n1. The clean cut between high-frequency semantics and low-frequency domain bias is an oversimplification. Other works show that high-frequency components in the input are a domain bias (Geirhos et al., 2018) and that utilizing more low-frequency responses in neural networks can increase their performance and robustness (Finder et al., 2024).\n2. The linear probing observation in Section 3.1 is very similar (albeit still different in implementation) to an experiment by Wang et al. (2020), in which ResNet shows to be responding more to high frequencies in the input rather than low frequencies. Hence, it is not surprising that the results of the experiment here favor the high-frequency filtering. A more thorough analysis of previous literature is needed there.\n3. The support for the claims relies on previous research in domain adaptation. The authors interpret this as evidence for their claim, suggesting that aligning the first and second-order statistics supports their argument. While plausible, this is not proof, and the success of aligning BN statistics can have other explanations.\n4. The sparse coding analysis is flawed. Using a high-pass filter on the input does result in a sparse output $h$, but it does not guarantee that $\\alpha$ is also sparse.\n5. The point made at the end of Section 3, regarding the use of multiple edge-filters in a network causing substantial information loss, warrants further investigation.\n6. The experiments are conducted on dated architectures (ResNet and Wide-ResNet are from 2016) and relatively small-scale datasets (CIFAR, GLUE, etc.). While the authors point that out in the limitations and point at computational resources, there can still be better options than what's provided. E.g., it is common to use subsets of ImageNet (of 50,100,200 classes), and using more modern mobilenets and/or convnext will not require much more computational resources.\n\nReferences:  \nGeirhos et al. \"ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.\" ICLR 2018.  \nFinder et al. \"Wavelet Convolutions for Large Receptive Fields.\" ECCV 2024.  \nWang et al. \"High-frequency Component Helps Explain the Generalization of Convolutional Neural Networks.\" CVPR 2020.\n\nQUESTIONS:\nI would appreciate a response to the weaknesses listed above, and would consider raising the score if those are addressed appropriately.\n\nLIMITATIONS:\nThe authors provide a section listing the limitations. However, as mentioned in my listed weaknesses, I do not fully agree that those could not be (at least partially) resolved.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the deep edge filter, a technique that applies high-pass filtering to intermediate feature representations in deep neural networks to improve generalization. The authors hypothesize that neural networks encode task-relevant semantic information in high-frequency components while storing domain-specific biases in low-frequency components. Their edge filter, implemented as F_edge(h)=h-LPF(h), is applied to deep features across various modalities and architectures. The paper shows modest improvements on several benchmarks including test-time adaptation, sentiment analysis, NeRF rendering, and audio classification.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Universal applicability: The method works across different modalities and architectures\n- Simple implementation: Requires only adding a single filter layer\n- Improvements: Shows positive results\n\nWeaknesses:\n- Limited Novelty: The core idea of applying frequency filtering to deep features is not novel. As also cited by the paper, Lin et al. (2023) proposed \"Deep Frequency Filtering for Domain Generalization\" at CVPR 2023, which explicitly modulates frequency components in deep features using FFT during training.\n\n- Insufficient Experimental Validation: The vision experiments rely primarily on CIFAR-10/100, which are considered toy datasets by modern standards. ImageNet experiments are notably absent despite being the standard benchmark for vision tasks. No comparison with state-of-the-art domain generalization methods or the closely related DFF work.\n\n- Technical Concerns: The choice of mean filtering as the default LPF seems arbitrary and under-justified with limited ablation studies on this choice.\n\nQUESTIONS:\n- Dataset scale: Why were experiments not conducted on ImageNet or other large-scale datasets that are standard in the domain generalization literature?\n- Comparison with SOTA: Can you provide comparisons with state-of-the-art domain generalization methods beyond basic baselines?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 1, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the Deep Edge Filter which is a simple high-pass filtering technique applied directly to deep features of neural networks rather than inputs. The aim was to isolate high-frequency semantic information while suppressing low-frequency domain-specific bias. Their filter is constructed by subtracting a low-pass filtered version of the activations from the original feature map. The paper shows that this filter can enhance generalization performance across a wide range of data modalities like vision, language, 3d scene synthesis, and audio classification.\n\nSTRENGTHS AND WEAKNESSES:\n__Strengths__:\n\n1.⁠ ⁠The main insight of the paper that semantic features lie in high-frequency components while domain-specific noise is in low-frequency components is well-motivated and generally applicable to many domains. Applying these techniques to internal representations is an interesting adaptation.\n\n2.⁠ ⁠From the results (Table 2, 3, 4), it seems as though the Deep Edge Filter does well across diverse modalities like test-time adaptation (vision), sentiment classification (language), novel view synthesis (3D), and sound classification (audio). The paper also includes ablation studies to isolate and validate the effect of the edge filter.\n\n3.⁠ ⁠The method proposed in the paper does not require any learnable parameters and can be applied as a single module in any existing architecture.\n\n__Weaknesses__:\n\n1.⁠ ⁠The paper would benefit from feature visualizations like PCA or spectrograms to understand the effect of their Deep Edge Filter.\n\n2.⁠ ⁠While the NeRF results mention reduced floaters, this is unsubstantiated and would be better supported with patch-wise PSNR or SSIM metrics, especially around the regions of high-frequency/detail in the scene.\n\n3.⁠ ⁠While it can be costly computationally to validate the methods on a wider variety of tasks, it is challenging to accept generalization claims when there are no results on large models like CLIP or SAM.\n\n4.⁠ ⁠The paper lacks a theoretical framework, so a deeper mathematical analysis would elevate the work.\n\nQUESTIONS:\n1.⁠ ⁠Can the authors provide interpretability visualizations of the filtered vs. unfiltered deep features? Additionally, for the NeRF experiments, could the authors report view or region-specific metrics in high-frequency areas?\n\n2.⁠ ⁠How does the performance of the model change based on where, within the network, the filter is applied? How does the edge filter behave in deeper architectures or large models such as GPT-style Transformers?\n\n3.⁠ ⁠Does the Deep Edge Filter complement other robustness strategies like dropout or batch norm?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}