Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning
Authors: Dongkwan Lee
Junhoo Lee
Nojun Kwak
Venue: NeurIPS 2025 poster ()
Keywords: edge filter
deep feature
frequency
Primary area: deep_learning
TL;DR: We introduce the Deep Edge Filter, which improves model generalizability by applying high-pass filtering to neural network features based on our hypothesis that semantic information resides in high-frequency components.

Abstract: We introduce the Deep Edge Filter, a novel approach that applies high-pass filtering to deep neural network features to improve model generalizability. Our method is motivated by our hypothesis that neural networks encode task-relevant semantic information in high-frequency components while storing domain-specific biases in low-frequency components of deep features. By subtracting low-pass filtered outputs from original features, our approach isolates generalizable representations while preserving architectural integrity. Experimental results across diverse domains such as Vision, Text, 3D, and Audio demonstrate consistent performance improvements regardless of model architecture and data modality. Analysis reveals that our method induces feature sparsification and effectively isolates high-frequency components, providing empirical validation of our core hypothesis. The code is available at \url{https://github.com/dongkwani/DeepEdgeFilter}.

Deep Edge Filter: Return of the Human-Crafted Layer in Deep Learning
Dongkwan Lee"
Junhoo Lee"
Nojun Kwakt
Scoul National University
[biancco, mrr jo @snu ac
Abstract
We introduce the Deep Edge Filter, a novel approach that applies high-pass filter- ing to deep neural network features to improve model generalizability. Our method is motivated by our hypothesis that neural networks encode task semantic information high-frequency components while storing domain-specific biases in low-frequency components of deep features. By subtracting low-pass filtered outputs from original features, our approach isolates generalizable representations while preserving architectural integrity. Experimental results across diverse do- mains such as Vision, Text, 3D, and Audio demonstrate consistent performance improvements regardless of model architecture and data modality. Analysis re- veals that our method induces feature sparsification and effectively isolates high- frequency components, providing empirical validation of our core hypothesis. The ilter
code is available at /httts:/ //ttthbb. com/
1
Introduction
Edge filters have long been a widely used classical technique in image processing to effectively capture relevant information, providing strong priors that are robust to various types of noise while effectively extracting semantic information. However, modern deep learning remains vulnerable to perturbation and domain shift [25 23). This is because the superficial low-level texture dependen- cies acquired by deep learning models during training further exacerbate their vulnerability to per- turbations [8]. This vulnerability evident the fields of adversarial attack [10] 7 and domain
adaptation [18 42). For example, even small adversarial perturbations, difficult for the human eye
detect, can cause significant confusion models, or the performance of models can be greatly degraded simply because of a change from day to night.
Why have lost knowledge of edge detection? Indeed, numerous past attempts have sought to integrate edge detection techniques into the deep learning domain 332 48) The core idea was to ap- ply edge detection, or methodologies inspired by it, input images. These processed images would then be fed into deep learning models, aiming to leverage the inherent robustness edge detection against perturbations. However, such approaches failed to gain significant traction. This lack of suc- cess can be attributed to two primary factors. Firstly, while applying Edge Filters to images offers the benefit of robustness against perturbations, conversely leads the removal of fine-grained image details. This diminishes the informational content fed into deep learning models, limiting the ability fully leverage the strengths of deep learning models, which excel processing information rich inputs. Secondly, classical edge detection was limited to the image domain. This inherent limitation makes difficult employ universally within the current deep learning landscape, which increas- ingly focuses on handling diverse data modalities, not only visual data but also wide array of modalities including natural language, 3D scenes, and audio.
Equal Contribution Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this paper, we generalize the concept of Edge Filters for deep features that can be applied directly to deep layers rather than the imput layers, combining the benefits of traditional Edge Filters with deep learning to build models robust to perturbations and domain shifts. Our approach is motivated by our hypothesis that deep networks encode task-relevant semantic features at high frequencies and domain-specific biases at low frequencies. If this hypothesis is true, then generalizing Edge Filters (which essentially function as high-pass filters) should help isolate generalizable features. We contextualize this hypothesis based on domain adaptation research and validate it with simple observation based on ResNet.
We define the Edge Filter applied to deep layers as the residual obtained by subtracting the result of low-pass filter (LPF) from the original. Here, we can utilize low-pass filters such as mean, median, and Gaussian kernels. When applying these LPFs to the deep features, the dimensionality of the kernels is naturally tailored to the specific deep learning architecture. Specifically, we applied 2- dimensional kernels to CNN-based architectures and 1-dimensional kernels to transformer-basse architectures and MIPs.
We validate the universal effectiveness of Edge Filter across diverse modalities, tasks, and archi- tectures. Our experiments span four fundamentally different modalities: Vision, Text, 3D, and Au- dio. For each modality, we selected tasks demanding strong generalization [66 2 19:: test-time adaptation 42
sentiment analysis 41, few-shot neural radiance fields 24 27,, and audio classification 34 Testing on both CNN-based [52] and Transformer-
66 architectures con-
firmed architecture- with different domains to demonstrate the consistent effectiveness of our approach. Furthermore, we found that applying the Edge Filter induces feature sparsification, in line with our theoretical sparse-coding framework. This was clearly demonstrated by the significant decrease in activation density following filter application. Moreover, we validate the need of high frequency proof-by- contradiction by showing that applying low-pass filter impairs generalization ability.
applicability. We conducted ablation studies across filter configurations
We make the following contributions:
We introduce Edge Filter, filter built human intuition, that can be applied to the fea- ture deep neural networks in a modality-agnosti manner to facilitate the extraction of generalizable features.
We propose an Edge Filter for both CNN-based and ViT-based architectures, and empiri- cally demonstrate that the filter enhances the performance of generalizability-rre -ruuiia tasks across various modalities including image, text, 3D, and audio.
We analyze the experimental results through the lens of layer sparsity and frequency de- composition, and provide extensive ablation studies on Edge Filters for deep features.
2 Related Works
2.1 Frequency Perspective in Deep Learning
Research on frequency analysis within deep learning has largely centered on image data and Con- volutional Neural Networks (CNNs), offering valuable insights into how these models encode and process information. has been observed that CNNs trained on ImageNet exhibit a strong bias towards texture rather than shape, a tendency distinct from human perception, and preferentially recognize textural cues in style-transferred images g 45). Furthermore, Deep Neural Networks (DNNs) adhere to the 'Frequency Principle", learning low-frequency components earlier than high- frequency ones during training; this phenomenon is often interpreted as an implicit bias arising from the regularity of activation functions 49 50]
Building on these understandings, various techniques have leveraged frequency concepts to enhance In architectural innovations, the Fourier Convolutional Neural Network [II] has improved efficiency by performing convolutions in the frequency domain using Fast Fourier Transform, while similar efficiency benefits have been achieved through Discrete Cosine Transform operations D. From a training perspective, frequency domain data augmenta- tion techniques have proven effective in preventing networks from overfitting to specific frequency components, thereby enhancing generalization capabilities 47 40) The capacity of frequency anal- ysis to extract object location and boundary information has also led to successful applications in
model performance and efficiency 48 4
2

---

object detection [26]. Despite these advancements, existing approaches have primarily focused on image modalities and rarely explored the direct application of frequency decomposition or filtering techniques to the internal learned representations within deep neural networks.
2.2 Activation Filtering and Sparsity
Filter Response Normalization (FRN) 37 when paired with a thresholded linear unit as its activa- tion function, utilizes learnable thresholds effectively filter activation values. This combination has demonstrated superior performance and versatility compared to conventional batch normalization or group normalization, exhibiting robustness particularly with small batch sizes. Deep Frequency Fil- tering [22] enhances domain generalization by directly filtering the hidden feature representations of network within the frequency domain. transforms feature maps into the frequency domain using FFT and then dynamically adjusts each frequency component with learnable attention mask. Sim- ilarly, the Deep Edge-aware Filter [46] achieved model acceleration by implementing patch-wise processing and spatially varying filtering within an integrated deep learning framework.
Recent research neural network training dynamics has contributed to understanding activation filtering. Studies on the 'edge of stability' [1 show training under these conditions creates thresh- old neurons that activate selectively, promoting network sparsity and potentially improving gener- alization. ProSparse (39), designed for LLMs, combines ReLU functions with progressive sparsity regularization achieve high activation sparsity, enhancing inference efficiency while minimizing performance loss. While these approaches prove effective in specific domains, they haven't t explored general-pune purpose filtering layers applicable across diverse deep learning applications.
Method: Deep Edge Filters
3.1 Rethinking the Role of Edge Filters
We hypothesize that applying Edge Filters to deep features improves generalization by isolating semantic components and suppressing dopaif-specific features. This hypothesis is supported from sparse coding perspective, with justification drawn from domain adaptation literature [42].
Stem layer
Block I
Block 2
We define an Edge Filter as a high-pass operator constructed by subtracting a low-pass filtered version of h, the original deep feature:
Edge Filter
Block 3
Fedgo (h) h LPF(h).
(1)
where LPF denotes a low-pass filter such as a mean, median, or Gaussian kernel applied to h.
FC
Figure 1: Example of model ar- chitecture with Edge Filter attached. This figure illustrates the case when the Edge Filter is attached after Block of the base model.
The Edge Filter defined above is attached in series between the blocks of the base model, as shown in Fig. When im- plementing the Edge Filter, parameters such as kernel size and other filter-specific parameters may be additionally required, depending on the type of filter.
Linear Probing Observation. To validate our hypothesis, we conducted an observational study to examine the impact of low-pass filtering and high-pass edge filtering on linear probing performance. We loaded an SaaggeNt--peetrinne ResNet18 model and attached filters to the output of its final block. While performing linear probing on the CIFAR- dataset, only the parameters of the fully connected (FC) classifier are updated, with the feature extractor frozen. For low-pass filtering (LPF), we employed a mean filter with a kernel size of 3. The model was trained for 10 epochs with a learning rate of 0.01.
The experimental results are presented in Tab. We observed that applying LPF led to a decrease in performance compared to the unfiltered baseline, while application of an Edge Filter improved per- formance over the baseline. Given the nature of linear probing, where the feature extractor remains unchanged, the model must train the linear classifier using only the feature representations acquired

---

Table Linear probing accuracy(%) on train and validation set of an ImageNet pre-trained ResNet18 on the CIFAR-I00 dataset. Results compare three scenarios based on filtering applied to the deep features: Vanilla model (w/o Filter), Low-Pass Filter (LPF), and Edge Filter (Fedge).
w/o Filter
LPF
Fedge
train acc val acc
21.8
17.4
22.4
21.0
17.4
21.4
during its ImageNet pretraining, rather than adapting them to the CIFAR-100 dataset. The perfor- mance observed with the Edge Filter suggests that it effectively filters out superficial information from the feature extractor, thereby better conveying task-relevant semantic signals to the classifier. This, in turn, indicates that deep learning models tend encode domain-speeifie information in the low-frequency regions of deep features, while semantic information is predominantly encoded in the fighffrequency regions.
Decomposition of Deep Features. Let h E Id be a feature vector from a hidden layer of a deep network. We assume the feature can be additively decomposed as
h hsem + hdom:
where heem encodes generalizable, task-relevant semantic features, and haom represents domain- specific biases such as illumination, resolution, or background texture.
Empirical Justification from Domain Adaptation. Domain adaptation research offers strong empirical support for this decomposition. In particular, several studies have shown that simply align- ing batch normalization (BN) statistics, the mean and variance of feature activations, between source and target domains leads to significant improvements in target accuracy (21, 42 [1 44]. This sug- gests that domain-specific information is heavily embedded in the first- and second-order statistics of the features.
These statistics primarily capture the global. fow-frrquency structure of features, including texture, color tone, and overall feature scale. Hence, domain sseeifc components such as haom can be re- garded as low-frequency in nature.
On the other hand, semantic components h such object boundaries, key phrases, or spatial discontinuities, tend to be spatially localized and structurally distinct. These elements are naturally more variable and correspond to high-frequency signal components.
Sparse Coding through High-Pass Filtering. frequency assumptions, we have:
Under our proposed feature decomposition and
LPF(h) 2 haom
Fedge (h) 2 heem-
(3)
This approach of refining features by frequency filtering resonates strongly with the principles of sparse coding. sparse coding, the objective is to find a concise representation for features h by expressing them as a linear combination of small subset of basis vectors from overcomplete dictionary E idxk.
h Da, with |allo < k.
The connection becomes clear when we consider the effect of our frequency filtering: by remov- ing the low-frequency, domain-specific redundancies from h via edge filtering, essentially simplifying the signal that needs to be represented. This simplification naturally encourages greater sparsity in the corresponding coefficient vector The resulting feature representations, now fo- cused on semantic content, are therefore more compact and possess enhanced generalizability across different domains.
Through the lens of domain adaptation, we argue that domain-specinf features predominantly oc- cupy the low-frequency spectrum of hidden activations, while task-relevant semantic content resides in the high-frequency components. The proposed edge filtering strategy leverages this structure to improve feature generality and downstream performance across diverse domains.

---

3.2 Deep Edge Filter
We propose simple Deep Edge Filter applicable to deep features across all modalities. This filter functions as high-pass operator, defined as the subtraction of the original feature and its LPF value, as specified in Eq. (T). For simplicity, we employ a mean filter as the default LPF throughout this paper unless otherwise stated. The Edge Filter is implemented channel- on each deep feature, with reflect padding applied preserve input and output dimensions.
To match the dimensions of deep features, we use different types of Edge Filters for different archi- tectures. For CNN-based architectures, we apply 2D Edge Filters because CNNs naturally process spatial relationships between neighboring pixels both vertically and horizontally. The 2D Edge Filter works by taking the 1-th layer feature hy E R x WixC, and subtracting the LPF value across the spatial dimensions (Hi, Wi) for each channel. For MLP and Transformer based architectures, we apply ID Edge Filters since these architectures don't inherently process spatial relationships. For these models, the ID Edge Filter operates on the 1-th layer feature h; E IR X by subtracting the 1D LPF value across the sequence length dimension N₁ for each channel.
The LPF component of the Deep Edge Filter is detached in the model training process to inhibit gradient backpropagation. This design choice reflects the non-learnable nature of the LPF, objective is to train the model exclusively on the high-pass filtered input. Furthermore, we implement the Edge Filter in only a single layer rather than introducing multiple filters within model. This is because we have observed that using multiple filters in a model causes substantial information loss, making it difficult for the model to train properly.
4 Experiments
Modality Selection. Deep learning models extract features differently depending on input data modality. Images and videos, derived from natural world observations, typically exhibit sparse dis- tributions in their high-dimensional raw input space and contain inherent spatial patterns and visual attributes. In contrast, text data, generated according to human-define linguistic rules, primarily en- codes semantic relationships. Even when embedded into feature spaces of common dimensionality, features from distinet modalities retain unique, modality-speciff properties due to their fundamen- tally different underlying distributions.
To test our claim rigorously, we conducted experiments on four modalities that have different char- acteristics. These modalities are Vision, which follows a nature-oriented distribution, Text, which follows human-defined rules and is defined in a discrete domain, 3D, which estimates spatial den- sity, and Audio, which has temporal patterns and frequency characteristies. Our experiments demon- strate that our methodology, inspired by edge detection, can be generalized across deep features from diverse modalities and architectures.
Overview. To evaluate the efficacy of our approach, we selected tasks across various modalities where generalization capability plays crucial role. Through extensive experiments, we demonstrate consistent performance improvements, validating the hypothesis that Edge Filters effectively extract generalized features within deep neural networks. The following subsections correspond to exper- iments on selected tasks in each modality. Each subsection explains why the chosen task requires generalization capabilities and reports experimental results when applying Edge Filters. Implemen- tation details are provided in Sec. A All experiments were run using single A6000 GPU.
4.1 Vision: Test-Time Adaptation
Test-Time Adaptation (TTA) 42 is task that adapts models pre-trained source domain to a target domain without access to source data. model's generalization capability is evaluated using datasets with significant domain shifts between source and target domains. Model adaptability improves when it captures generalizable and semantically meaningful patterns instead of overfitting to superficial domain- -specifc cues. Therefore, better TTA performance indicates that the model has been effectively trained to extract general and semantic features during the learning process.
We compare the performance of vanilla models and models with attached Edge Filters, both pre- trained under identical conditions, when applying TTA algorithms. Model evaluation is conducted using the commonly used 10C/1000 and ImageNet200-C benchmarks [14] for TTA task.
5

---

Table 2: Test-time adaptation accuracy (%) results with and without filters measured by the SFAAR-10C/10C and ImageNet200-G benchmarks with each TTA method applied.
CIFAR-IOC
CITAR-
ImageNet200-C
Method
Source
Direct
NORM
TENT
Source
Direct
NORM
TENT
Source
Direct
NORM
TENT
WRN28-10
91.9
49.6
73.8
74.6
69.8
26.2
46.6
48.0
58.8
2.0
23.9
21.6
+Jay
90.8
(+8.0)
57.7
(+1.5)
75,3
(+1.2)
75.8
65.4
(+10.2)
36.4
(+3.1)
49.7
(+2.5)
50.5
55.5
(+0.1)
2.1
(+1.9)
25.8
(+1.6)
23.2
VIT-B/32
95.6
60.8
60.9
82.3
41.3
41.0
83.4
27.5
30.7
+Funky
95.3
(+7.3)
(+8.5)
(+0.4)
(+1.0)
93.1
(+1.9)
(+1.9)
These benchmarks validate models trained on each clean dataset CIFAR-10/100 [7 and Ima- geNet200 [33, against corrupted versions of each dataset. The results are presented in Tab. Source represents evaluation results on the original, uncorrupted validation sets, while Direct shows results when pretrained models applied directly to the corrupted sets without adaptation. Results after applying TTA methods NORM (35) and TENT [42) are also included. For Vit, since there is no BatchNorm layer, the NORM algorithm could not be applied. We modified the TENT implementa- tion to update parameters of LayerNorm layers instead of BatchNorm layers.
As shown in Tab. 2 applying the Edge Filter yields significant performance improvements in both backbones regardless of the TTA methodology used. Performance improvements ranged from 1.2%p to 8.5%p on CIFAR-IOC, from 0.4%p to 10.2%p on CIFAR- 100C, and from 0.1 0.p%p 1.9%p on ImageNet200-C across different methods. Particularly noteworthy is that despite the reduction in performance on the Source dataset when applying the Edge Filter, performance corrupted datasets improved. This suggests that while vanilla models tend to overfit to the training dataset during the training stage, the Edge Filter removes domain-specifer information, preventing overfitting and en- couraging the model to learn more general representations.
4.2 Language: Sentiment Analysis
General Language Understanding Evaluation (GLUE) benchmark is standard evaluation tool that includes various natural language understanding (NLU) tasks [4I]. We selected subtasks such as SST-2 (movie review sentiment analysis) (38), QQP (determining semantic equivalence of question pairs) 229]. QNLI (determining an answer is contained in a question) [31] for our experiments. A common characteristic of these language tasks is extracting specific discrete information (sentiment, semantic relationship, logical relationship) from vast bigh-dimensional text data (corpus). The cru- cial aspect in this process is effectively distinguishing necessary semantic information (signal) from unnecessary stylistic elements or additional information (noise). For example, in sentiment analy- sis, emotional expressions to develop generalization capability, the ability to filter out such noise is important, and Edge Fil- ter can assist by preserving semantic information in high-frequency components while removing domain-specific information in low-frequency components.
signals while movie content details noise. For language models
Experimental results presented in Tab, Performance improvements were observed in all GLUE subtasks when Edge Filter was applied. The most significant improvement (+1 ++.49%)) occurred in the sentiment analysis task SST-2, which can be attributed to movie review data containing relatively more noise, including numerous movie-related additional information (actors, plor, scene descrip- tions, etc.) beyond sentiment expressions. Edge Filter effectively removes this noise, allowing the model to focus on information crucial for sentiment judgment.
Table 3: Classification accuracy (%) for GLUE benchmark tasks with and without Fedge- Numbers in parentheses indicate performance improvement.
SST-2(Sentiment)
QQP(Paraphrase) ONLI(Inference)
Avg
Transformer
79.36
83.42
62.40
75.06
Fedge
80.85 (+1.49)
83.46 (+0.04)
63.30 (+0.90)
75.87 (+0.81)

---

4.3 3D: Few Shot Neural Radiance Field
Neural Radiance Field (NeRF) [24] relies on generalization capabilities that leverage deep learn- ing's s inherent interpolation abilities to accurately represent complex 3D spaces using a simple MLP architecture. Particularly, its ability to model 3D scenes from a small number of 2D images and ren- der from novel viewpoints demands generalization performance for unobserved viewpoints. These characteristics make an appropriate modality for validating the efficacy of Edge Filter.
While 3D objects possess complex geometric structures, NeRF essentially focuses on volume boundary prediction, which consists of discontinuities at object surfaces and boundaries that ex- hibit high-frequency characteristics. Conversely, low-frequency features such as uniform densities within objects or empty spaces are relatively less significant. Edge Filter can enhance generalization capability by preserving these high-freeg freefergeccc components (signal) while removing low- low-frequency components (noise). Since NeRF processes 3D data through 1D ray synthesis. we applied ID Edge Filter to the layer immediately preceding density calculation in the coarse network.
For experiments, we used the widely-adopted Blender dataset [4 with results presented in Tab. 4 Performance improvements were observed in most scenes after applying Edge Filter, with an average enhancement of 5.2% for overall metries. Most notably, the LPIPS metric decreased substantially to 11%. This supports our hypothesis because Edge Filter is designed to inject human prior. However, the scene ficus showed 4.5% performance drop-likely because its naturally rich high-frequency textures, as shown in Fig. rendered further enhancement by the Edge Filter redundant or even counterproductive.
Table 4: Few-shot NeRF rendering quality with 8-view inputs on various scenes. Numbers in paren- theses indicate improvements.
PSNRT
SSIM
LPIPS
MAE]
Scene
NeRF
+Jedge
NeRF
+_oodge
NeRF
+>eddee
NeRF
+>eegge
chair
25.20
26.08
(+0.88)
0.911
0.917 (+0.006)
0.090
0.076 (-0.014)
0.018
0.016
(-0.002)
drums
17.12
17.31
(+0.19)
0.761
0.770 (+0.009)
0.242
0.223 (-0.019)
0.055
0.053 (-0.002)
ficus
22.33
22.12 (-0.21)
0.889
0.881 (-0.008)
0.075
0.084 (+0.009)
0.024
0.025 (+0.001)
lego
24.58
24.53 (-0.05)
0.894
0.894 (+0.000)
0.074
0.071 (-0.003)
0.020
0.020 (+0.000)
mic
27.64
28.17 (+0,53)
0.958
0.961 (+0.003)
0.049
0.041 (-0.008)
0.010
0.010 (+0.000)
ship
20.80
22.19
(+1.39)
0.720
0.749 (+0.029)
0.224
0.175 (-0.049)
0.046
0.036 (-0.010)
Avg
22.95
23.39 (+0.44)
0.856
0.862 (+0.006)
0.126
0.112 (-0.014)
0.029
0.027 (-0.002)
NeRF
Edge Filter
Ground Truth
Figure 2: Qualitative analysis on NeRF Overall, Edge filtering reduces floating artifacts.
7

---

Overall, Edge Filter was confirmed to effectively remove view-dependent noise while preserving essential information such as boundary surfaces in 3D representation, thereby improving gener- alization performance. As shown in Fig. 2 NeRF with Edge Filter demonstrates visually cleaner rendering results. In partieular, 'floaters' (indistinct areas faintly floating around objects) commonly occurring in conventional NeRF are significantly reduced. This is because Edge Filter effectively removes superficial noise, such as incorrectly predicted volume densities, while preserving high- frequency signals like object boundaries. Overall, Edge Filter effectively eliminates view-dependent noise while preserving key information such as boundary surfaces in 3D representation, thus en- hancing generalization performance.
4.4
Audio: Audio Classification
Audio classification tasks identify the type or source of sounds by extracting meaningful patterns from acoustic signals. In real-world environments, audio signals contain substantial noise and un- necessary background sounds, making it essential for classification models to effectively distinguish characteristic sounds (signal) from noise. Due to these properties, audio classification another suitable modality for validating the effectiveness of Edge Filter. In our experiments, we used the UrbanSound8K dataset, which contains considerable noise acquired from urban environments. Ex- perimental results showed that models with Edge Filter applied demonstrated performance improve- ments from 77.42% to 81.72% compared to baseline models. This improvement is attributed to Edge Filter's ability to emphasize characteristic sound patterns while effectively removing background noise, thereby enhancing classification performance.
5 Analysis
To investigate the impact of the Deep Edge Filter on model training, we measured the density of layer outputs throughout the training process. Fig. 3a and Fig. 3b demonstrate the output density of each WRN-28-I1 10 layer by training epoch during the pretraining stage on the CIFAR-10 dataset as described Section 4.1 Density values represent averages across the entire training dataset for each epoch. For the conv layer and blockl, which are positioned before the Edge Filter, minimal density differences were observed. In contrast, block2 and block3, which are positioned after the Edge Filter, show significantly decreased output density when the Edge Filter is present. This effect most pronounced in block2, which located right after the filter. This dramatic reduction feature density following filter application experimentally validates the sparse coding of features resulting from high-pass filtering, as formulated in Eq.
Additionally, we performed FFT analysis to examine how the Deep Edge Filter influences the fre- quency domain characteristics of deep features. Fig. 3c demonstrates the average FFT results for the filter's 's input and output on the CIFAR-10 validation set after completing the training shown in Fig. (3b] The graph shows the x-axis cross-section the center of the 2D FFT, averaged across chan- nels. The FFT results demonstrate that the amplitude in the low-frequency region of deep features
0.5
0.5
beker lilrr
9.0
the llke
Outul Dandyy
0.4
Oapu Deaagy
0.4
Vadid Vadia
Amplnukk (lng
kwech
0.3
0.3
10
30
Epoch
30
40
50
10
20
Epoch
30
40
0.0
Normalized Frequesey
0.8
1.0
(a) Without Filter
Edge Filter attached
Deep Feature FFT
Figure 3: Statistical analysis of the impact of Edge Filter on deep features. Fig. Ba and Fig. 3b show the sparsity of output features for each block when training a vanilla model and a WRN-28 T0 model with Edge Filter attached to block1 on the CIFAR-10 10 dataset, respectively. Fig. 3c represents the FFT amplitude spectra of both inputs and outputs from the Edge Filter when evaluated on the validation set after the training in Fig. 3blis completed.
8

---

decreases after passing through the Edge Filter. This confirms that the Edge Filter, designed as high-pass operator through LPF subtraction, functions as intended.
5.1 Ablation studies
Fig. illustrates ablation studies conducted to examine the effects of Edge Filter's attachment po- sition and kemel size. Performance gains compared to the baselines are represented as heatmaps. Layer on the vertical axis denotes the stem network (conv layer) of CNN-based models. Figures 4a and 4b present results from applying various filter configurations to the WRN-28- 10 and ViT-B/32 models, respectively, on the CIFAR-IOC benchmark using the Direct method. The WRN model shows general trend of performance improvement with the Edge Filter application, achieving maximum improvement of 9.6%p. For the ViT model, performance improvements were pronounced when filters were applied to later layers, while applying filters earlier layers tended to decrease performance. Figure 4c shows results from applying filters with various configurations on SST-2, language domain task. Performance either remained unchanged improved regardless of the Edge Filter's ' position and kernel size, with maximum improvement of 3.6%p. In the audio domain, as shown in Figure 4d] performance fluctuated with variations ranging from -2.0%p to +3.6%p without exhibiting any clear trends. The statistical significance test results for the baseline of each task are shown in Sec. B
Table 5: Ablation study on vision domain (TTA) and language domain (SST-2) for filter types.
Modality
Vision
Language
Benchmark
CIFAR-IOC (Direct)
SST-2
Backbone
WRN
VIT
BERT
Baseline
49.64
60.84
79.36
+Mean Fedge
57.65 (+8.01)
68.09 (+7.25)
80.85 (+1.49)
+Median Fedge
59.69 (+10.05)
67.28 (+6.44)
81.60 11.60 (+2.24)
Gaussian Fodgo
57.66 (+8.02)
56.37
(-4.47)
80.58 (+1.12)
+Mean LPF
39.69 (-9.95)
47.96 (-26.68)
57.56 (-3.28)
Tab. presents an ablation study examining how model performance changes when different filters replace the mean filter the LPF component of the Edge Filter. We compared mean, median, and Gaussian filters as LPFs, and also tested applying the mean filter (LPF) directly to deep features instead of using the Edge Filter. Our experiments utilized WRN and ViT backbones for vision tasks and the BERT backbone for language tasks, with filter configurations matching those in 4 and the Gaussian filter's sigma set to 1.0. Results demonstrate that performance improvements remained consistent across different LPF types within the Edge Filter, indicating that various LPFs can be effectively incorporated into the Deep Edge Filter design. We observed a performance decline when using the Gaussian filter with VIT, likely due using default sigma value rather than optimizing it. When applying LPF directly to deep features, significant performance degradation occurred across all tasks, confirming that low-frequency components of deep features tend to encode domain-specific information, causing performance degradation in tasks requiring generalizability.
Vision WRN Direct
Visiu, VIT
Direct
NLP. NLP_SST2 SST2
Audio UrbunSoundSk
-0.4
0.4
2,6
0.9
2.2
3.8
0.5
4.9
-0,0
0.9
-2.0
1.4
9.6
Layer
Layer
3.2
-64
9.6
8.0
4.1
0.5
1.0
0.2
3.5
.1.8
5.2
-1.6
3.4
1.8
4.3
4.8
10.6
Kernel
Komel
(a) Vision, TTA on WRN (b) Vision, TTA VIT
(c) Language, SST2 (d) Audio, UrbanSound8k
Figure 4: Heatmap of performance gain (%p) compared to the cases without Edge Filter, depending on the position and the kernel size of the Edge Filter. CIFAR-10C TTA benchmark is used in the vision domain, SST-2 in the language domain, and UrbanSound8k classification in the audio domain.

---

6 Conclusion
In this paper, we show that Edge Filters can be applied directly deep features in deep learning models to extract more generalized features from input data, This is based on the observation that deep learning models encode task-relevant semantic information in the nigh-freuency region, while domain-specifee information in the low frequency region. Since domain -emain--ppccffc information often acts as a superficial feature semantic inference tasks, filtering out the low frequencies can improve the model's s generalization ability. Experiments on various domains such as vision, language, 3D, and audio, validate the effectiveness of the proposed Deep Edge Filter and show that broadly applicable regardless of the data modality.
Limitations and Future Works. Because our protocol requires retraining models from scratch after attaching the Edge Filter, we were unable to conduct more extensive experiments due to com- putational cost constraints. A limitation of this work is that we could not validate the methodology's effectiveness state-of-the-art models or across a wider variety of tasks. This limitation is particu- larly pronounced in the language domain, where experiments with LLMs were infeasible due to their high computational demands. Future work applying our approach to LLMs would provide valuable extensions to our claims from an application perspective. Additionally, given the domain -agnssic characteristics of the Deep Edge Filter, exploring its application to multimodal models represents another promising direction for future research.
Acknowledgments and Disclosure of Funding
This work was supported by NRF grant (2021R1A2C300669) and IITP grants (RS-2022-1222053, RS-2021-1221133, RS RS-2025-25442338), all funded by MSIT of the Korean Government.
References
[1] Kwangjun Ahn. Sébastien Bubeck, Sinho Chewi, Yin Tat Lee, Felipe Suarez, and Yi Zhang. Learning threshold neurons via edge of stability. Advances in Neural Information Processing Systems, 36:19540- 19569, 2023.
[2] Junbum Cha. Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sun- grae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Pro- cessing Systems, 34:22405-225 -24118, 2021.
[3] Wei- Yu Chen, Chenn Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look few-shot classification. arXiv preprint 1904:1194.042333 2019.
[4] Jeremy Cohen, Simran Kaur, Yuanzhi Li, Zico Kolter, and Ameet Talwalkar. Gradient descent on neural networks typically occurs at the edge stability. In International Conference on Learning Representa- tions, 2021.
[5] Jacoh Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidi- rectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chopter of the association for computational linguistics: human language technologies, volume (long and short papers), pages 4171-4186, 2019.
[6] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition scale. In International Conference on Learning Representations, 2021.
[7] Yonggan Fu, Shunyao Zhang, Shang Wu, Cheng Wan, and Yingyan Lin. Patch-fool: Are vision trans- formers always robust against adversarial perturbations? In International Conference on Learning Rep- resentations, 2021.
[8] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference Learning Representations, 2018.
[9] Arthita Ghosh and Rama Chellappa. Deep feature extraction in the det domain. In International Confer- ence Pattern Recognition, pages 3536-3541. IEFF, 2016.
10

---

[10] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial exam- ples. In International Conference on Learning Reprexentations, 2015.
[11] Yuna Han and Byung-Woo Woo Hong. Deep learning based on fourier convolutional neural network incorpo- rating random kernels. Electronics, 10(16):2004, 2021.
[12] Kaiming Hc, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 9729-9738, 2020.
[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.
[14] Dan Hendrycks and Thomas Dietterich, Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019.
[15] Gabriel Ilharco. Mitchell Wortsman, Nicholas Carlini, Rohan Taori, Achal Dave. Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannanch Hajishirzi, Ali Farhadi, al. Openclip. Zemodo, 2021.
[16] Kenji Kawaguchi, Leslie Pack Kactbling, and Yoshua Bengio. Generalization in deep learning. arXiv preprint arX1v:/710.05468,, 1(8), 2017.
[17] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
[18] Ananya Kumar, Tengyu Ma, and Percy Liang. Understanding self-training for gradual domain adaptation. In International Conference on Machine Learning, pages 5468 5479. PMLR, 2020.
[19] Dongkwan Lee, Kyomin Hwang, and Nojun Kwak, Unlocking the potential of unlabeled data in semi- supervised domain generalization. In IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pages 30599-30608, 2025.
[20] Da Li, Yongxin Yang. Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference computer vision, pages 5542 5550, 2017.
[21] Yanghao Li, Naiyan Wang, Jianping Shi, Xiaodi Hou, and Jiaying Liu. Adaptive batch normalization for practical domain adaptation. Pattern Recognition, 80:109-117, 2018.
[22] Shiqi Lin, Zhizheng Zhang, Zhipeng Huang. Yan Lu, Cuiling Lan, Peng Chu, Quanzeng You, Jiang Wang. Zicheng Liu, Amey Parulkar, et al. Deep frequency filtering for domain generalization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1797-11807 2023.
[23] Yawci Luo, Liang Zheng, Tao Guan, Junqing Yu, and Yi Yang. Taking a closer look at domain shift: Category-lever adversaries for semantics consistent domain adaptation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2507-2516, 2019.
[24] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
[25] Seyed- Mohsen Moosavi-Dezfodd Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal ad- versarial perturbations. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1765-1773, 2017.
[26] Varsha Nair, Moitrayee Chatterjee, Neda Tavakoli, Akbar Siami Namin, and Craig Snoeyink. Fast fourier transformation for optimizing convolutional neural networks in object recognition. arXiv preprint arXiv:2010.04255 042577 2020.
[27] Michael Niemeyer, Jonathan Barron, Ben Mildenhall, Mehdi SM Sajjadi, Andreas Geiger, and Noha Radwan, Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs. In IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 5480-5490, 2022.
[28] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 1406-1415 2019.
[29] Quora. Quora question pairs. Kaggle Competition, 2017.

---

[30] Alec Radford, Jong Wook Kim. Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In International Conference on Machine Learning, pages 8748-8763. PmLR, 2021.
[31] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Perey Liang. Squad: 100,000+ questions for machine comprehension of text. In Conference on Empirical Methods in Natural Language Processing, pages 2383-2392, 2016. 2016.
[32] Oren Rippel. Jasper Snoek, and Ryan P Adams. Spectral representations for convolutional neural net- works, Advances in Neural Information Processing Systems, 28, 2015.
[33] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstenn, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision (ICV), 115(3):211225 252, 2015.
[34] Justin Salamon, Christopher Jacoby, and Juan Pablo Bello. A dataset and taxonomy for urban sound research. In Proceedings of the 22nd ACM international conference on Multimedia, pages 1041-1044, 2014.
[35] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems, 33:1 33:11539-11533 11551, 2020.
[36] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis. Mitchell Wortsman, Patrick Schramowski, Sri- vatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion- 5b: An open large-scale dataset for training next generation image-text models. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
[37] Saurabh Singh and Shankar Krishnan. Filter response normalization layer: Eliminating batch depen- dence the training deep neural networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11237- 11246, 2020.
[38] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng. and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treehank. In Conference on Empirical Methods in Natural Language Processing, pages 1631 1631-1642 2013.
[39] Chenyang Song, Xu Han, Zhengyan Zhang, Shengding Hu, Xiyu Shi, Kuai Li, Chen Chen, Zhiyuan Liu, Guangli Li, Tao Yang, and Maosong Sun. Prosparse: Introducing and enhancing intrinsic activation spar- sity within large language models. In Proceedings of the 31 International Conference on Computational Linguistics, pages 2626-2644. Association for Computational Linguistics, 2025.
[40] Puru Vaish, Shunxin Wang, and Nicola Strisciuglio. Fourier-basir functions to bridge augmentation gap: Rethinking frequency augmentation in image classification. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17763-17772 2024.
[41] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding, In International Confer- ence on Learning Representations, 2019.
[42] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference Learning Representations, 2021.
[43] Zheng Wang. Zhenwei Gao, Guoqing Wang, Yang Yang, and Heng Tao Shen. Visual embedding augmen- tation in fourier domain for deep metric learning, IEEE Transactions on Circuits and Systems for Video Technology. 33(10)55538 5548, 2023.
[44] Yanan Wu, Zhixiang Chi, Yang Wang, Konstantinos N Plataniotis, and Songhe Feng. Test-time domain adaptation by learning domain-aware batch normalization. In AAAI Conference on Artificial Intelligence, volume 38, pages 11996, 2024.
[45] Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang, Yen-Kuang Chen, and Fengho Ren. Learning in the fre- quency domain. In IEEE/CVF Conference on Computer Vision and Pottern Recognition, pages 1740- 1749, 2020.
[46] Li Xu, Jimmy Ren, Qiong Yan, Renjie Liao, and Jiaya Jia. Deep edge-aware filters. In International Conference on Machine Learning, pages 1669-1678. PMI.R, 2015.
12

---

[47] Qinwei Xu, Ruipeng Zhang. Ya Zhang, Yanfeng Wang, and Qi Tian. A fourier-based framework for domain generalization. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14383-14392, 2021.
[48] Zhenlin Xu, Deyi Liu, Junlin Yang, Colin Raffel, and Mare Niethammer. Robust and generalizable visual representation learning random convolutions. In International Conference on Learning Representa- tions, 2021.
[49] Zhi Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, Zheng Ma. Frequency principle: Fourier analysis sheds light on deep neural networks. arXiv preprint arXiv:1901.06522. 2019.
[50] Zhiqin John and Hanxu Zhou, Deep frequency principle towards understanding why deeper learning is faster. In AAAI Conference on Artificial Intelligence, volume 35, pages 10541-10550, 2021.
[51] Jayeon Yoo, Dongkwan Lee, Inseop Chung, Donghyun Kim, Nojun Kwak. What and when should object detectors update in continually changing test domains? In IEEE/CVF Conference on Com- puder Vision and Pattern Recognition, pages 23354 23363, 2024.
[52] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Edwin R. Hancock Richard C. Wil- son and William A. P. Smith, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 87.1 17.1-87.112 BMVA Press, September 2016.
[53] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable ef- fectiveness of deep features perceptual metrie. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 586-595, 2018.
13

---

NeurIPS Paper Checklist
The checklist designed to encourage best practices for responsible machine learning research, ad- dressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes]. , [No] or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justiffcctin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evalu- ation. While "[Yes] is generally preferable "[No] ". it is perfectly acceptable answer "[No] provided a proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). gen- eral, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NearIPS Paper Check- list".
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract of the paper clearly reflects the contribution and scope of the work performed.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
fine to include aspirational goals motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
14

---

We provide the limitations of this paper in the Conclusion section with limitations and future work paragraph.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-texx system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers dis- cover limitations that aren't it acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community, Re- viewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: This paper does claim any theoretical contributions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The experiments this paper are based on open source code and we describe the hyperparameters we used to run the experiments in as much detail as possible.
15

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recogmize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: This thesis encompasses a wide range of experiments, which makes it difficult to submit code for all of them thoroughly. Nevertheless, code for the experiments that are important this thesis, such as the vision domain, have been submitted.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurlPS code and data submission guidelines (https://nips...
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
cttpssscccpppcccccccccceccccc wew.dddddee/oeeaaiiiiiiaa Sodeeeodeeeeeeeeeeeeeee sssssoooiiccysi for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
16

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We've detailed the datasets, hyperparameters, and other settings we used in our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Due to the large number of experiments performed and the computationally heavy nature of this thesis, which requires pretraining, we were unable run repeated experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK report I-sigma error bars, but should state The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors verified.
For asymmetric distributions, the authors should careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g, negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We briefly disclose the computational resources used in the experiment.
17

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
Our research conforms NeurIPS Code Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative cictal impacts of the work performed?
Answer: [NA]
Justification: Our work not expected to have societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
18

---

Answer:
[NA]
Justification:
Our paper is expected to pose no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use the model, for example by re- quiring that users adhere usage guidelines restrictions access the model or implementing safery filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have credited and cited the codes and assets we used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of the pack- age should be provided. For popular datasets, pwpprswi wwwerrewtthhhe....m com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that re- ce-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced the paper well documented and the documenta- tion provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, lim- itations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
19

---

Question: For crowdsoureing experiments and research with human subjects, does the pa- per include the full text of instructions given participants and screenshots, applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, cura- tion, or other labor should paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: No LLM was used in the core method development.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https://neuripp:: wcCCoonereecces////0000 for what should or should not be described.
20

---

A Implementation Details
A.1 Vision Domain
Following the convention of the benchmarks, we used two Convolution- models and one Transformer-based model: the CNN-structured WideResNer28-10 52 for CIFAR datasets, ResNet18 [13] for ImageNet200 dataset, and the Transformer-strucuureed ViT-B/32 [6] for all datasets. Learning rates of le-3 were applied to the convolution-base models, and 5e-5 was ap- plied to ViT. Convolution-baser models were trained from scratch, while ViT model was trained from the ImageNetlk pre-trained weights. We conducted training for 50 epochs for CIFARI0/100, while 5 epochs for ImageNet200, with batch size of 128. The severity level of dataset corruptions was set to 5. For WRN28-10. 2D Edge Filter with kernel size 11 was applied after block out of three blocks, while for ResNet18, kernel size was applied after block 2 out of four blocks. For ViT-B/32, ID Edge Filter with kernel size 5 for the CIFAR datasets and kernel size 3 for the ImageNet200 dataset after block index 11 out the block indices ranging from 11.
A.2 Language Domain
We used the vanilla 12-layer transformer, which is the standard BERT architecture [S] training from scratch, and applied ID Edge Filter with kernel size after block For training, we used Adam optimizer, with batch size of 512, learning rate of 5e-5, and weight decay of le-2 for 40 epochs.
A.3 3D Domain
Experiments conducted in the 8-view few-shot (2 setting using various 3D objects (ship, mic, lego, ficus, drums, chair) from the Blender dataset [24). We followed the same optimization settings the default NeRF implementation on the Blender dataset, and the kernel size was performed for 500 pixel epochs, and we used PSNR, SSIM, LPIPS [53], and MAE metrics for performance evaluation measure different aspects of rendering quality. We applied ID Edge Filter with kernel size 5 to the coarse network prior to computing the density a.
to Training
A.4 Audio Domain
dataset, which includes 10 urban sound classes (e.g., car horn, children playing, dog barking). The model architecture consists three convolutional blocks, and add 2D Edge Filter with a kernel size of 7 after the third layer. The model was trained for 20 epochs using the Adam optimizer with a learning rate of 0.001.
We used the UrbanSound8K 34
B Statistical Analysis
We conducted replication study using multiple experimental runs with different random seeds assess the statistical significance of the results presented in Fig. 4 Given the observed performance improvements over the baseline condition shown Fig. 4 we calculated the mean and standard етror of baseline performance metrics across replicate experiments to determine whether the effects of the Edge Filter method achieve statistical significance. Tab. 6 deviations computed from five independent experimental runs using different random seeds for each modality under investigation.
presents the means and standard
Table 6: Statistical analysis on baseline performances (%) across modalities
Domain
Vision
Language
Audio
Benchmark
CIFARIOC (Direct)
SST-2
UrbanSound8k
Backbone
WRN
ViT
BERT
CNN
Mean
49.37
61.84
79.89 0.52
76.99
SD
2.46
2.47
0.98
21

---

Sigma, Visloe
Direct
Sigma
Viskon
VIT _Dreett
Sigma NLP
Sigma UibanSoundSk
1.5g
6eee
Layer
1.0e
1.1e
0.0m
2.
0.2g
3.5e
-1.6g
-0.50
-0:40
1.0m
-1.90
+8.70
-190
4m
2,60
6.kg
2,00
41.40
0.5m
1.50
15
5
7
5
Kernel Size
Kernel Size
Kernel Size
Kemel Size
(a) Vision, TTA on WRN
(b) Vision, TTA on VIT
(c) Language, SST2
(d) Audio, UrbanSound8k
Figure 5: Heatmap of performance gains multiples of standard deviation (o), represents the
statistical significance of Fig.
To evaluate the statistical significance of the performance improvements, we demonstrate the nor- malized performance analysis by plotting the performance gains expressed in standard deviations (o) relative to the baseline standard deviation for each experimental condition, as presented Fig. 5
This analysis shows that the performance gains across multiple modalities and backhone architec- tures exceed two standard deviations (2x) from the baseline mean, thereby demonstrating statisti- cally significant improvements attributable the Edge Filter.
C T-SNE Visualizaiton
We performed t-SNE visualization vision domain data visually confirm the effect of the Edge Filter on deep learning models. Figures Fig. and Fig. / show the t-SNE visualization results for the CIFAR-10 10 validation set extracted from model trained
the CIFAR- 10 dataset and for the features of CIFARIO-G which is the CIFAR-10 10 dataset with added corruption, respectively. The WRN and VIT models, along with the attached mean filter, are identical to those used in Sec. 4.1
WRN vanilla
WRN mean filter
VIT vanilla
VIT mean filter
Figure 6: Comparison of t-SNE visualizations for vanilla WRN and VIT backbones with and without
Mean Filters applied to CIFARIO validation set.
vanilla
WRN mean
vanilla
VIT mean filter
Figure 7: Comparison t-SNE visualizations for vanilla WRN and VIT backbones with and without Mean Filters applied to corrupted images of CIFARIO validation set, CIFARIO-C.
Visualization results on the clean validation set, seen in Fig. showed no distinct difference in feature distributions between the vanilla model and the model with a mean filter attached, for both the WRN and ViT backbones. Mcanwhile, visualization results for corrupted images,
22

---

in Fig. 7 show that the model generally struggles to distinguish classes well compared to clean images, resulting blurred class clusters. Comparing the model with the mean filter attached the vanilla model here reveals that, regardless of the backbone, the model with the mean filter exhibits relatively clear boundaries between classes. Notably, the difference distribution between models with and without the mean filter is even more pronounced the ViT backbone. These visual results align with the numerical findings in Tab, 2 confirming the role of the Edge Filter enhancing model robustness, improving performance over the vanilla model on corrupted images.
D Full Experiment Result for TTA
Tab. and Tab. break down the results of Tab. 2bo 15 different corruption types that Edge Filter improves performance for the vast majority corruptions.
14
They show
Table 7: Classification accuracy (%) for all corruptions in CIFARIO-C
Digital
Method
Backbone
Gau
Shr
Imp
Def
Gls
Mm
7m
Snw
Frs
Fog
Bmt
Cnt
Fis
Px
Jpg
Avg
Direct
WRN28-10
+Fuat::
42.2
17.9
48.7
22.3
22.4
36.4
47.8
56.3
50.5
40.2
64.1
11
63.3
70.5
69.1
65.7
67.8
68.1
86.2
86.7
33.1
33.8
68.1
60.7
43.3
44.1
68.7
64.2
57.7
49.6
WRN28-10
68.4
70.5
59.4
82.7
60.8
80.5
83.2
76.5
77.7
80.7
87.4
83.2
72.6
74.8
71.1
75.3
TENT
WRN28-10
1.Fate
65.7
68.8
58.1
83.2
60.0
80.4
83.4
76.8
77.7
76.4
81.6
87.9
81.0
71.5
74.2
71.5
69.9
74.6
Direct
ViT-B/32
61.8
65.7
37.5
51.9
65.2
53.4
59.0
77.5
76.3
39.7
85.2
15.4
69.1
80.0
75.0
60,8
+.Fatt:
TENT
VIT-B/32
62.9
66.4
31.2
51.6
66.6
53.9
60.5
78.5
78.0
37.4
86.0
13.9
70.5
80.8
75.6
60.9
+.Fate:
69.2
73.3
46.2
73.4
70.3
69.2
73.9
80.5
56.1
85.5
23.8
76.7
81.8
79.4
69.4
Table 8: Classification accuracy (%) for all corruptions in CIFAR100-C
Weather
Digital
Method
Rackbone
Gau
Shr
Imp
Def
Gis
Min
7m
Snw
Frs
Fog
Bnt
Car
Fis
Px
Jpg
Avg
Direct
WRN28-10
+Fuate
25.3
10.2
27.4
11.7
14.2
6.5
24.0
33.6
39.7
18.8
35.7
29.8
39,2
29.3
36.4
42.6
42.9
26.5
32.0
34.4
52.1
57.3
14.5
16.7
41.4
46.4
50.9
16.8
48.9
33.3
26.2
36.4
NORM
WRN28-10
33.8
34.3
26.8
57.0
53.5
57.3
46.5
47.4
50.6
63.4
55.0
46.7
38.7
+.Fatge
42.3
43.4
33.2
58.1
44.5
54.7
58.3
49.3
50.9
47.2
60.7
45.1
50.5
56.3
50.8
49.7
TENT
WRN28-10
36.1
36.4
28.6
58.1
38.4
54.4
59.4
47.3
48.5
$1.6
63.7
56.8
48.1
52.0
40.1
48.0
odge
Direct
VST-B/32
37.5
41.1
21.0
31.7
35.4
37.4
43.9
$5.7
$3.7
27.6
67.4
8.0
46.6
57.3
48.6
41.3
TENT
V/T-B/32
+.Folee
34.8
40.0
42.5
39.1
21.5
17.4
43.1
35.5
34.6
35.9
41.9
35.3
43.5
49.7
54.8
$7.4
54.5
56.3
22.8
28.2
65.7
68.4
8.2
5.4
48.8
46.5
58.9
53.4
52.7
48.8
42,0
41.0
E
Expansion to Foundational Models
We validate whether our hypothesis remains effective in larger foundation models. To verify the effect of Edge Filter, adopt few-shot prototype matching evaluation protocol [ that mea- sures the quality of the encoder's feature representations. Specifically, we construct prototypes by averaging the features of samples belonging to each class, and measure classification performance based on these prototypes.
We select foundation model ViT-L/14 OpenCLIP 15 30 trained the LAION2B dataset [36). This model has learned robust representations across diverse visual domains through large-scale web-scale data training, making it well-suited for evaluating domain generalization capabilities.
We conduct cross-domain generalization experiments evaluate whether the Edge Filter is also effective for larger models. This task measures the performance when prototypes extracted from one domain are applied to samples from another domain. We conduct Photo-to-Sketch Sketch-to- Photo, Seal-to-Sketch, and Sketch-to-Red transfer scenarios the PACS 20) and DomainNet (28 datasets. As a result, we observed that applying Edge Filter also improves performance for larger foundation model, as shown in Tab.
23

---

Table 9: Cross-domain few-shot classification accuracy (%) on PACS and DomainNet datasets. Numbers in parentheses indicate improvements over the baseline.
PACS: Photo to Sketch
PACS: Sketch Photo
DomainNet: Real Sketch
DomainNet: Sketch to Real
Method
Baseline
+Futg
Baseline
+Folg
Bascline
+Soate
Baseline
+Frdy
Accuracy
59.07
63.22 (+4.15)
54.31
60.12 (+5.81)
24.79
25.00 (+0.21)
39.23
39.53 (+0.30)
Effect of increased computation
To demonstrate that the performance improvement from the Edge Filter is not merely an effect of increased computation, we additionally conducted experiments comparing its performance with a model where the Edge Filter replaced by Convolution layer with the same computational load. Since is widely known that performance tends to improve as the computational load of deep learning models increases, comparing these two
with identical computational loads allows us to isolate the pure effect of the Edge Filter, excluding the impact increased computation. The table below Tab. TO shows the results of replacing the Edge Filter with trainable 2D Conv layer of the same kernel size the CIFARIO-C WRN28-10 10 experiment from Tab. All other experimental conditions are identical those used with the original Edge Filter.
Table 10: Comparison of CIFARIO-C TTA performance when replacing the Edge Filter with a train- able 2D Cony layer with equivalent computational load
Source
Direct
NORM
TENT
No filter
91.9
49.6
73.8
74.6
Conv layer
90.6
46.9
71.3
72.3
Edge filter
90.8
57.7
75.3
75.8
As in Tab. TO replacing the Edge Filter with trainable Convolution layer of equivalent com- putation load actually resulted decreased performance on corrupted images. This demonstrates that the Edge Filter's effectiveness holds true when accounting for the increased computational load introduced by its addition.
24


Reviews:
[{"review_id": "W3w5y0LroY", "reviewer": "Reviewer_2BDh", "summary": "This paper introduces the Deep Edge Filter which is a simple high-pass filtering technique applied directly to deep features of neural networks rather than inputs. The aim was to isolate high-frequency semantic information while suppressing low-frequency domain-specific bias. Their filter is constructed by subtracting a low-pass filtered version of the activations from the original feature map. The paper shows that this filter can enhance generalization performance across a wide range of data modalities like vision, language, 3d scene synthesis, and audio classification.", "questions": "1.⁠ ⁠Can the authors provide interpretability visualizations of the filtered vs. unfiltered deep features? Additionally, for the NeRF experiments, could the authors report view or region-specific metrics in high-frequency areas?\n\n2.⁠ ⁠How does the performance of the model change based on where, within the network, the filter is applied? How does the edge filter behave in deeper architectures or large models such as GPT-style Transformers?\n\n3.⁠ ⁠Does the Deep Edge Filter complement other robustness strategies like dropout or batch norm?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "__Strengths__:\n\n1.⁠ ⁠The main insight of the paper that semantic features lie in high-frequency components while domain-specific noise is in low-frequency components is well-motivated and generally applicable to many domains. Applying these techniques to internal representations is an interesting adaptation.\n\n2.⁠ ⁠From the results (Table 2, 3, 4), it seems as though the Deep Edge Filter does well across diverse modalities like test-time adaptation (vision), sentiment classification (language), novel view synthesis (3D), and sound classification (audio). The paper also includes ablation studies to isolate and validate the effect of the edge filter.\n\n3.⁠ ⁠The method proposed in the paper does not require any learnable parameters and can be applied as a single module in any existing architecture.\n\n__Weaknesses__:\n\n1.⁠ ⁠The paper would benefit from feature visualizations like PCA or spectrograms to understand the effect of their Deep Edge Filter.\n\n2.⁠ ⁠While the NeRF results mention reduced floaters, this is unsubstantiated and would be better supported with patch-wise PSNR or SSIM metrics, especially around the regions of high-frequency/detail in the scene.\n\n3.⁠ ⁠While it can be costly computationally to validate the methods on a wider variety of tasks, it is challenging to accept generalization claims when there are no results on large models like CLIP or SAM.\n\n4.⁠ ⁠The paper lacks a theoretical framework, so a deeper mathematical analysis would elevate the work.", "was_revised": true, "final_justification": "The paper presents a straightforward but interesting approach worth investigating theoretically and experimentally (at large scales). Further, the authors clarified all questions and concerns with empirical evidence. Overall, with the new findings, I've raised my rating to an accept.  Authors are strongly encouraged to include all new findings in their final copy.", "rebuttal": "We appreciate the insightful feedback from the reviewer commenting on our method’s general applicability, validations across multiple modalities, and clear ablation studies.\n\n**W1&Q1 Visualization of Deep Edge Filter:** Visualizing how the distribution of features changes when applying the Edge Filter, as suggested by the reviewer, would be very helpful in understanding the characteristics of the Edge Filter. Thank you for your valuable comment. Following your suggestion, we performed tSNE visualization in the visual domain. As a result, we confirmed that when the deep edge filter was applied, the clusters for each class were more clearly distinguished. We will add this figure to the Supplementary Material. Please understand that we are unable to attach the image directly due to image attachment restrictions in this rebuttal.\n\n**W2&Q1 Region-specific NeRF metric:** To see the quantitative result of floaters, we calculated the patch-wise SSIM score for those scenes containing floaters. As for PSNR, the score explodes when the pixel scores are exactly the same, so we selected SSIM. We divided 400x400 generated scene images into 4*4 (16 patches), then calculated the SSIM score for each patch. Even though the patches are coarsely divided, there was a significant score margin between Ours and the Baseline, with the area floater occurring. The patches containing floaters show a severe gap compared to ours. We report the result as follows:\n\n| Scene | GT vs Base SSIM | GT vs Ours SSIM | SSIM Margin |\n|:-----:|:---------------:|:---------------:|:-----------:|\n| Drums |      0.919      |      0.996      |    +0.077   |\n| Drums |      0.936      |      0.990      |    +0.054   |\n| Drums |      0.775      |      0.824      |    +0.049   |\n| Chair |      0.849      |      0.999      |    +0.150   |\n| Chair |      0.618      |      0.739      |    +0.121   |\n| Chair |      0.714      |      0.822      |    +0.108   |\n\n**W3&Q2 Large models:** We verified whether our methodology could be applied to foundational architectures and evaluated its performance using the aforementioned CLIP. Based on the ViT-L-14 OpenCLIP model trained on the Laion2B dataset, we focused on the training-free zero-shot classification scenario. For the classification method, we designed the intuitive experiment using prototype matching methodology that measures the distance between the latent vector of the image to be predicted and a single data point, i.e., prototype, representing each class.\n\nTo verify the generalization ability, we used the standard benchmarks PACS and DomainNet datasets. Specifically, we randomly selected one image per class from the source domain as the class prototype and compared the zero-shot classification accuracy of the shifted target domain images using this prototype. We selected the Photo & Sketch domain for PACS and the Real & Sketch domain for DomainNet, which are considered to be severe domain shifts. The performance of the baseline case, which uses the embedding vector as is, and the case where the Edge Filter is applied are shown in the table below.\n\n|             |                |      PACS      |      | \\| |               |   DomainNet   |      |\n|:-----------:|:--------------:|:--------------:|:----:|----|:-------------:|:-------------:|:----:|\n|             | Photo > Sketch | Sketch > Photo |  avg | \\| | Real > Sketch | Sketch > Real |  avg |\n|   baseline  |      59.1      |      54.3      | 56.7 | \\| |      24.8     |      39.2     | 32.0 |\n| Edge Filter |  **63.2**    |  **60.1**   | **61.7** | \\| |   **25.0**  |  **39.5**  | **32.3** |\n\nWhen applying the Edge Filter, we observed an improvement in classification accuracy from an average of **56.6%->61.7%** in PACS and from an average of **32.0%->32.3%** in DomainNet. This suggests that even simple post-processing in latent space using the Deep Edge Filter can effectively improve the generalization performance of models in understanding data from domains different from the training data.\n\n**W4 Theoretical framework:** Our work focuses on discovering phenomena through experimentation. In the field of deep learning, research has tended to proceed with discoveries made through experimentation first, followed by the establishment of theories to explain those phenomena. Considering this trend, our discoveries may serve as the basis for future theoretical research. We are the first to discover the association between generalizability and the frequency domain of deep features. Although we have not yet established a theoretical framework, we believe that subsequent research may uncover such a framework.\n\n**Q3 Complement other robustness strategies:** Yes, Deep Edge Filter can be used simultaneously with dropout and BN. In fact, Deep Edge Filter has already been used in conjunction with dropout and BN in experiments using convolution-based networks (WRN in image modality, sound modality). Since each experiment confirmed that using Deep Edge Filter resulted in additional performance improvements, Deep Edge Filter can be viewed as a complementary methodology to existing robustness strategies."}, {"review_id": "Ud9kbFm1Zg", "reviewer": "Reviewer_Zz5q", "summary": "This paper introduces the deep edge filter, a technique that applies high-pass filtering to intermediate feature representations in deep neural networks to improve generalization. The authors hypothesize that neural networks encode task-relevant semantic information in high-frequency components while storing domain-specific biases in low-frequency components. Their edge filter, implemented as F_edge(h)=h-LPF(h), is applied to deep features across various modalities and architectures. The paper shows modest improvements on several benchmarks including test-time adaptation, sentiment analysis, NeRF rendering, and audio classification.", "questions": "- Dataset scale: Why were experiments not conducted on ImageNet or other large-scale datasets that are standard in the domain generalization literature?\n- Comparison with SOTA: Can you provide comparisons with state-of-the-art domain generalization methods beyond basic baselines?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 1, "originality": 1, "strengths_and_weaknesses": "Strengths:\n- Universal applicability: The method works across different modalities and architectures\n- Simple implementation: Requires only adding a single filter layer\n- Improvements: Shows positive results\n\nWeaknesses:\n- Limited Novelty: The core idea of applying frequency filtering to deep features is not novel. As also cited by the paper, Lin et al. (2023) proposed \"Deep Frequency Filtering for Domain Generalization\" at CVPR 2023, which explicitly modulates frequency components in deep features using FFT during training.\n\n- Insufficient Experimental Validation: The vision experiments rely primarily on CIFAR-10/100, which are considered toy datasets by modern standards. ImageNet experiments are notably absent despite being the standard benchmark for vision tasks. No comparison with state-of-the-art domain generalization methods or the closely related DFF work.\n\n- Technical Concerns: The choice of mean filtering as the default LPF seems arbitrary and under-justified with limited ablation studies on this choice.", "was_revised": true, "final_justification": "I change my mind. I believe this paper introduces well enough scientific information about generalization that is needed for the community.", "rebuttal": "We appreciate the insightful feedback from the reviewer commenting on our method’s universal applicability, simple methodology, and performance improvements.\n\n**W1 Novelty of paper:** Transforming data into the frequency domain for analysis or simply optimizing in the frequency domain is not a new idea. In the field of audio, it has long been common practice to apply convolution to spectrograms in the frequency domain. The DFF[2] cited in the review appears to be a work that tunes the optimization techniques in the frequency domain revealed in previous work[1]. However, we are not simply trying to solve the problem in the frequency domain. Our primary goal is to offer a new interpretation of how deep learning models store information, revealing a fundamental connection to human prior knowledge through a frequency-domain analysis. The core of this paper is the verification that deep learning stores a significant amount of generalizable information in the high-frequency region when viewed in the frequency domain.\n\nWhat is counterintuitive about our work is that, unlike previous work where “improved performance is a given” due to the nature of optimization, we achieved performance improvement through a non-trainable HPF where **“improved performance is not a given”**. While previous work has focused on proposing “techniques” that utilize frequency analysis, we propose a completely new \"intuition\" regarding how deep learning models form semantic features.\n\n**W2&Q1 Larger dataset:** We agree with the reviewer's opinion that our methodology would be more convincing if validated on larger datasets such as ImageNet. Taking into account the advice of reviewer oxzH, we conducted image modality experiments using the ImageNet200 dataset, which consists of 200 randomly selected classes from ImageNet. The results of the experiment using the ResNet18 model on ImageNet200 are as follows. For Deep Edge Filter, we applied a mean HPF with layer 2 and kernel size 11.\n\n|             |        Source | Direct | NORM | TENT |\n|-------------|:---------------:|:--------:|:------:|:------:|\n| baseline    | **58.8**          | 2.0    | 23.9 | 21.6 |\n| Edge Filter | 55.5          | **2.1**    | **25.8** | **23.2** |\n\nThe results of experiments on ImageNet are also consistent with those obtained using the existing CIFAR dataset. Models applying Edge Filter show lower performance on general validation datasets but higher performance on corrupted (domain-shifted) data, demonstrating that they learn more general features during training. This proves that our claim is not limited to low-resolution datasets.\n\n**W2&Q2 Comparison with SOTA:** We applied our proposed method to a semi-supervised DG task, one of the most challenging tasks in the DG field. As the baseline method, we used UPCSC[3], the SOTA method presented at CVPR 2025. We kept the model architecture and training procedure unchanged and conducted experiments using one labeled sample per class from the PACS dataset, which is the standard DG benchmark. We used ResNet18 as the model backbone and applied the same edge filter configuration as in the ImageNet experiment: a mean HPF with layer 2 and kernel size 11. The results of the SSDG experiment are as follows.\n\n|                       |   A  |   C  |   P  |   S  |  avg |\n|------------------|:----:|:----:|:----:|:----:|:----:|\n| UPCSC | 34.3 | 20.2 | 17.2 | 22.9 | 23.7 |\n| + Edge Filter | **41.6** | **43.9** | **21.1** | **47.9** | **38.6** |\n\nWhen the edge filter was applied, the classification accuracy improved significantly from an average of **23.7%->38.6%**. This shows that there is still plenty of room for performance improvement simply by applying the edge filter proposed in the SOTA model, and indicates that the edge filter effectively improves the generalization performance of the model.\n\n**W3 Choice of filter type:** The hypothesis presented in Section 3.1 applies generally to HPFs regardless of the type of filter. In fact, when comparing the results implemented using various filters such as mean, median, and Gaussian in the ablation study in Table 5, we observed a consistent trend in most cases. Among these, the mean filter was used as the main filter because it is simple to implement and computationally efficient, not because it has any particular advantages. Even if another type of LPF is used instead of the mean filter, the experimental trend will remain the same, and the claims made in the paper will not change. We appreciate your consideration of this point.\n\n**References:**\n[1] Li, Zongyi, et al. \"Fourier Neural Operator for Parametric Partial Differential Equations.\" International Conference on Learning Representations. 2021.\n[2] Lin, Shiqi, et al. \"Deep frequency filtering for domain generalization.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2023.\n[3] Lee, Dongkwan, et al. \"Unlocking the Potential of Unlabeled Data in Semi-Supervised Domain Generalization.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025."}, {"review_id": "MJ6UYLID9e", "reviewer": "Reviewer_oxzH", "summary": "The paper proposes introducing high-pass filters (edge filters) as a single filter in neural networks to improve their performance. The core hypothesis is that deep features can be decomposed into semantic- and domain-specific information, where the semantic information resides in the high-frequency components of the features and the domain information resides in the low-frequency components of the features. Edge filters operate by subtracting a mean-filtered features from the features, creating essentially a high-pass filter.", "questions": "I would appreciate a response to the weaknesses listed above, and would consider raising the score if those are addressed appropriately.", "limitations": "The authors provide a section listing the limitations. However, as mentioned in my listed weaknesses, I do not fully agree that those could not be (at least partially) resolved.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The method is simplistic and easy to integrate into existing architectures. The proposed method is architecture-agnostic and, therefore, is very compelling.\n2. The paper provides empirical evidence that shows an improvement when using their method.\n\nWeaknesses:\n1. The clean cut between high-frequency semantics and low-frequency domain bias is an oversimplification. Other works show that high-frequency components in the input are a domain bias (Geirhos et al., 2018) and that utilizing more low-frequency responses in neural networks can increase their performance and robustness (Finder et al., 2024).\n2. The linear probing observation in Section 3.1 is very similar (albeit still different in implementation) to an experiment by Wang et al. (2020), in which ResNet shows to be responding more to high frequencies in the input rather than low frequencies. Hence, it is not surprising that the results of the experiment here favor the high-frequency filtering. A more thorough analysis of previous literature is needed there.\n3. The support for the claims relies on previous research in domain adaptation. The authors interpret this as evidence for their claim, suggesting that aligning the first and second-order statistics supports their argument. While plausible, this is not proof, and the success of aligning BN statistics can have other explanations.\n4. The sparse coding analysis is flawed. Using a high-pass filter on the input does result in a sparse output $h$, but it does not guarantee that $\\alpha$ is also sparse.\n5. The point made at the end of Section 3, regarding the use of multiple edge-filters in a network causing substantial information loss, warrants further investigation.\n6. The experiments are conducted on dated architectures (ResNet and Wide-ResNet are from 2016) and relatively small-scale datasets (CIFAR, GLUE, etc.). While the authors point that out in the limitations and point at computational resources, there can still be better options than what's provided. E.g., it is common to use subsets of ImageNet (of 50,100,200 classes), and using more modern mobilenets and/or convnext will not require much more computational resources.\n\nReferences:  \nGeirhos et al. \"ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.\" ICLR 2018.  \nFinder et al. \"Wavelet Convolutions for Large Receptive Fields.\" ECCV 2024.  \nWang et al. \"High-frequency Component Helps Explain the Generalization of Convolutional Neural Networks.\" CVPR 2020.", "was_revised": true, "final_justification": "The authors provided additional experiments and addressed most of my concerns. \nHence I'm raising my score.", "rebuttal": "We appreciate the insightful feedback from the reviewer commenting on our method’s simplicity, architecture-agnostic applicability, and extensive experiments.\n\n**W1 Conflict with previous works:** Your point is that there are previous studies showing that low-frequency response is helpful in the input domain, but our claim focuses on high-frequency features, so it seems like we are making conflicting claims. However, our paper deals with high-frequency features in the **feature domain** of deep layers, not the input domain. This is also why our paper is titled “Deep” Edge Filter. Since we are addressing a different data level than previous works (deep features vs. input data), the two claims do not conflict. I hope this explanation helps clarify any misunderstandings you may have had.\n\n**W2 Similar paper:** Thank you for your suggestion for the paper by Wang et al. (2020). However, we respectfully disagree with your opinion that our paper is similar to that paper. This is because, recommended paper is about 1. Focusing on the image data - which is input level 2. Enforcing low frequency. Our paper deals with latent feature (i.e., deep feature) - which is feature level, and we also enforce high frequency feature in latents. So the scope is totally different.\n\n**W3 Empirical Justificationfrom from DA:** There seems to be a misunderstanding. We do not attempt to prove this using DA(domain adaptation), but rather take a conceptual approach to the problem based on empirical experience. We demonstrate a common phenomenon in deep learning models, namely that general features tend to exist mainly at high frequencies. Proving this phenomenon is considered impossible due to the complexity of deep learning, and this paper also stops at empirical observation. In other words, this paragraph is not a proof but an interpretation of the first and second order statistic matching techniques widely known in DA from a frequency perspective.\n\n**W4 Sparse coding:** Although the arithmetic density of deep features may not decrease immediately after HPF, when transformed into the frequency domain, the density is reduced. This is because the information filtering process of HPF occurs in the frequency domain. This density reduction phenomenon in the frequency domain can be directly confirmed in Fig. 3c. Another approach is that even though the output density decreases in the layers after HPF (Fig. 2a, 2b), the generalizable ability is maintained, which means that the model has acquired sparse coding ability. Since the only change in the model is the addition of HPF, it can be logically concluded that HPF has sparse coding ability.\n\n**W5 Multiple edge filters:** We understand your question concerns why multiple edge filters cause substantial information loss. This occurs due to cumulative frequency degradation: each edge filter (HPF) removes low-frequency components, and cascading multiple filters compounds this effect. When filters are applied sequentially, each filter is applied to already filtered data, resulting in exponential (quadratic) loss of information. This is why applying multiple edge filters causes severe information loss.\n\n**W6 Larger dataset:** The reviewer requested that we conduct experiments with larger datasets and modern architectures. The request for larger datasets was also a common question from reviewer Zz5q, so we felt the need for additional experiments, and we appreciate the advice on subset classes. Following the comments, we conducted experiments using the ResNet18 model on ImageNet200, which randomly selected 200 classes. The Deep Edge Filter applied a mean HPF with layer 2 and kernel size 11.\n\n|             |        Source | Direct | NORM | TENT |\n|-------------|:---------------:|:--------:|:------:|:------:|\n| baseline    | **58.8**          | 2.0    | 23.9 | 21.6 |\n| Edge Filter | 55.5          | **2.1**    | **25.8** | **23.2** |\n\nThe results of experiments on ImageNet are also consistent with those obtained using the existing CIFAR dataset. Models applying Edge Filter show lower performance on general validation datasets but higher performance on corrupted (domain-shifted) data, demonstrating that they learn more general features during training. This proves that our claim is not limited to low-resolution datasets.\n\n**W6 Modern architecture:** This paper already presents experimental results using ViT and BERT, which are modern architectures based on transformers, so it would be helpful to refer to those results.\n\nIn addition, we conducted additional experiments using the ViT-L-14 OpenCLIP model trained on the Laion2B dataset during the rebuttal process. We designed the training-free zero-shot classification scenario experiment using prototype matching methodology that measures the distance between the latent vector of the image to be predicted and a single data point, i.e., prototype, representing each class. Standard benchmarks PACS and DomainNet are used, with one randomly selected image per class from the source domain as the class prototype. The zero-shot classification accuracy of the shifted target domain images using the prototype is examined. The performance of the baseline case, which uses the embedding vector as is, and the case where the Edge Filter is applied are shown in the table below.\n\n|             |                |      PACS      |      | \\| |               |   DomainNet   |      |\n|:-----------:|:--------------:|:--------------:|:----:|----|:-------------:|:-------------:|:----:|\n|             | Photo > Sketch | Sketch > Photo |  avg | \\| | Real > Sketch | Sketch > Real |  avg |\n|   baseline  |      59.1      |      54.3      | 56.7 | \\| |      24.8     |      39.2     | 32.0 |\n| Edge Filter |  **63.2**    |  **60.1**   | **61.7** | \\| |   **25.0**  |  **39.5**  | **32.3** |"}, {"review_id": "UFOEssAdPK", "reviewer": "Reviewer_kGMU", "summary": "This paper introduces the Deep Edge Filter, a high-pass filter for deep features, to improve model generalizability. Its key innovations include: (i) a modality-agnostic Edge filter that can facilitate the extraction of generalizable features; (ii) a simple but effective hand-crafted edge filter operator for both CNN and ViT architectures, demonstrating good cross-task generalizability across various modalities including image, text, 3D and audio. Ablation studies demonstrate the effectiveness w.r.t. layer sparsity and frequency.", "questions": "1.\tDoes the position of the mean filter applied significantly influence the results? Have the authors tried to apply the filter to alternative layers? In addition, does multiple edge filters for multiple layers help?\n2.\tWhether the edge filter could work by training from scratch, or is its efficacy is limited to pre-trained models? Additional experimental results addressing this point may strengthen the application scenario of the proposition.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Major Strengths\n1. \tThis worke finds an interesting pattern within deep features that a simple use of hand-crafted mean filter can decouple the semantic components from the domain-specific components within deep features. As far as I can tell, this finding, while being empirical, is novel and non-trivial.\n2. The deep edge operator is validated across a diverse range of multi-modal data, substantiating its universal applicability to models and data of differing modalities.\n3. The proposed fix is simple yet incredibly effective, exhibiting significant potential to extend to a wide range of tasks.\n\nMajor Weaknesses\n1.\tThe core hypothesis, h=h_dom+h_sem, relies predominantly on empirical evidence from prior literature without providing direct evidence; the paper only presents preliminary experiments indicating improved model performance upon removal of low-pass components.\n2.  The claim that “we generalize the concept of Edge Filters for deep features that can be applied directly to deep layers rather than the input layers” may be overclaimed, as learned convolutional filters are also a form of edge filters, potentially misleading readers regarding the contribution of the work.\n3.  The formulation may lack robustness; for instance, in “Sparse Coding through High-Pass Filtering”, the authors may fail to elucidate the nature of high-pass filtering under sparse coding, while merely presenting the high-pass filtering equation (3) and the sparse coding equation (4) alone.\n\nMinor Weaknesses\nAlthough the study encompasses multiple modalities, the experimental comparisons within individual modalities are limited by a few methods and the absence of SOTA approaches.", "was_revised": true, "final_justification": "The authors have engaged actively to provide additional results and persuaded other reviewers. I believe this is an interesting work that deserves the attention from the community. My final recommendation is Accept.", "rebuttal": "We appreciate the insightful feedback from the reviewer commenting on our method’s novel and non-trivial finding, experiments on broad modalities, and potential for wide applicability.\n\n**W1 Core hypothesis:** The reviewer seems to be asking about equation 3, which states that LPF(h) contains domain-specific information, while HPF(h) contains mostly domain-general information. This equation represents our hypothesis, and as the reviewer pointed out, our paper does not provide direct evidence (or proof), nor do we believe it is possible to do so. This is because our hypothesis deals with the features of deep networks, which cannot be handled in the same way as shallow networks, where analytical solutions exist. Instead of a mathematical proof, we conducted experiments related to generalizability across a wide range of modalities to empirically validate our claims. Furthermore, in this rebuttal phase, we additionally demonstrated that our hypothesis holds scale-invariant on larger datasets (ImageNet, PACS, DomainNet) and larger model (ViT-L-14 OpenCLIP). We hope that these various experiments supporting Eq. 3 will help convince you.\n\n**W2 Conv layer as edge filter:** I agree with the reviewer's comment that learnable conv layers can sometimes act like edge filters. However, just because convolutional filters can play the role of edge filters does not mean that convolutional filters always become edge filters. Rather, learned conv layers generally do not have a clear role in filtering specific frequencies like LPF or HPF, as their weights change depending on the learning process. The filter proposed in this paper is not learnable and was designed with the explicit intention of being an HPF, contrasting with the conv layer. For a clearer explanation, I have included the experimental results from the CIFAR-10 WRN experiment in Table 2, where the edge filter was replaced with a 3x3 learnable conv layer.\n\n|             | Source | Direct | NORM |  TENT |\n|-------------|:------:|:------:|:----:|:-----:|\n| No filter   |  **91.9**  |  49.6  | 73.8 |  74.6 |\n| Conv layer  |  90.62 |  46.92 | 71.3 | 72.32 |\n| Edge filter |  90.8  |  **57.7**  | **75.3** |  **75.8** |\n\nThe results of applying the 3x3 conv layer showed lower performance than when applying the edge filter in all cases; Validation (source), corrupted dataset (Direct, NORM, TENT). Showing that the performance of extracting generalizable features was still insufficient due to overfitting on the train data.\n\n**W3 Formulation of Eq.3 and Eq.4:** This formula is not presented for robustness or theoretical rigor, but rather conceptually to explain our perspective to readers. Our perspective is that, when addressing issues related to generalization, certain features are superficial, and from this perspective, the problem naturally transitions to sparse coding. In the generalization problem, the truly important features are sparse, and our focus is on selecting these semantic features. From this conceptual algorithmic perspective, the transition from Equation 3 to Equation 4 is natural.\n\n**minorW Few methods and absence of SOTA approaches:** Since this paper covers a wide range of modalities, please understand that the types of tasks performed for each modality are limited. However, during this rebuttal process, we additionally performed SSDG tasks and comparisons with SOTA methods, as well as zero-shot classification (prototype matching) using the Open-CLIP model, and have attached the results of these experiments. We hope that these experiments will help address the reviewers' concerns.\n\nWe applied our proposed method to a semi-supervised DG task, one of the most challenging tasks in the DG field. We used UPCSC[1], the SOTA method presented at CVPR 2025 as the baseline method. We kept the model architecture and training procedure unchanged and conducted experiments using one labeled sample per class from the PACS dataset. We applied the mean HPF of kernel size 11 on layer 2 of ResNet18. The results of the SSDG experiment are as follows.\n\n|                       |   A  |   C  |   P  |   S  |  avg |\n|---------------------|:----:|:----:|:----:|:----:|:----:|\n| UPCSC | 34.3 | 20.2 | 17.2 | 22.9 | 23.7 |\n|+ Edge Filter     | **41.6** | **43.9** | **21.1** | **47.9** | **38.6** |\n\nWe also conducted prototype matching zero-shot classification experiments based on the ViT-L-14 OpenCLIP model trained with the Laion2B dataset. We used the PACS and DomainNet datasets as benchmarks. Specifically, we randomly selected one image per class from the source domain as the class prototype and compared the zero-shot classification accuracy of the shifted target domain images using this prototype. For PACS, we selected the Photo-Sketch domain, and for DomainNet, we selected the Real-Sketch domain. The performance of the baseline case, where the embedding vector used for prototype matching was used as-is, and the case where the Edge Filter was applied, is shown in the table below.\n\n|             |                |      PACS      |      | \\| |               |   DomainNet   |      |\n|:-----------:|:--------------:|:--------------:|:----:|----|:-------------:|:-------------:|:----:|\n|             | Photo > Sketch | Sketch > Photo |  avg | \\| | Real > Sketch | Sketch > Real |  avg |\n|   baseline  |      59.1      |      54.3      | 56.7 | \\| |      24.8     |      39.2     | 32.0 |\n| Edge Filter |  **63.2**    |  **60.1**   | **61.7** | \\| |   **25.0**  |  **39.5**  | **32.3** |\n\n**Q1 Filter position:** The position of the filter affects performance, and the degree of difference varies depending on the modality. As shown in Fig. 3, we conducted an extensive ablation study on filter position and kernel size in the vision, language, and audio modalities. In addition, when applying multiple edge filters, as described in lines 184–187, information loss becomes severe, resulting in poor performance.\n\n**Q2 Initial weight:** We didn't get to talk about this part in detail, so thank you for pointing it out. Except for the ViT backbone of the vision modality, all of our experiments were trained from scratch. We based our approach on training from scratch, but in the case of ViT, meaningful model training was impossible using only the given image dataset, so we had no choice but to use a pretrained model. In other words, existing experimental results have already demonstrated the applicability of our methodology in cases where both training from scratch and pretrained models were applied. We will clarify this point in the implementation details to avoid any confusion.\n\n**References:** [1] Lee, Dongkwan, et al. \"Unlocking the Potential of Unlabeled Data in Semi-Supervised Domain Generalization.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025."}]

Author rebuttal:

Meta-review: This work examines the effect of high-pass filtering / edge filtering on deep representations and its resulting impact on predictions. The experiments cover multiple modalities—images, audio, text, implicit representation of 3D—and multiple architectures—convolutional networks, transformers, and NeRFs.

Four expert reviewers with expertise on deep learning, computer vision, signal processing, and multi-scale architecture agree on acceptance (2BDh: 5, Zz5q: 5, kGMU: 5, oxzH: 4). The area chair sides with acceptance, finding no reason to overrule, and agrees with the positive evaluation of the reviewers that the empirical results are likely to inform and attract further interest as an instance of designed filtering having a role for learned representations. As a further point for acceptance, work at the intersection of deep learning and signal processing is valuable as a bridge between topics, and so both communities can learn from and extend this work.

While the initial reviews were not as positive and in agreement, the authors provided a rebuttal, and substantial discussion followed.
Although the trajectories of the reviews, rebuttals, and discussion are long and span multiple threads, ultimately every reviewer converged on acceptance. The area chair has followed each thread and likewise finds that issues that would argue against acceptance have been resolved. Nevertheless, the authors are strongly encouraged to incorporate the detailed feedback in the reviews especially w.r.t. the clarity of the contributions and its relationship to prior work focused on frequency analysis of deep features and filters plus domain adaptation and domain generalization.

Miscellaneous feedback:

Note that this comment has no bearing on the decision for the paper and is purely offered as information for the authors. Including a discussion of broader related work, for instance the on the frequency analysis of deep features and filters, alias, and multi-scale processing, may attract further interest and reach a broader audience. Examples could include Context contrasted feature (CVPR 2018), Octave Convolution (ICCV 2019), FrequencyLowCut Pooling (ECCV 2022), Improving native CNN robustness with filter frequency regularization (TMLR 2023), and other works at the intersection of signal processing and deep learning.
Decision: Accept (poster)