The Quest for Universal Master Key Filters in DS-CNNs
Authors: Zahra Babaiee
Peyman Kiasari
Daniela Rus
Radu Grosu
Venue: NeurIPS 2025 poster ()
Keywords: Convolutional Neural Networks
Depthwise Separable Convolutions
Gaussian Derivatives
Primary area: deep_learning

Abstract: A recent study has proposed the ``Master Key Filters Hypothesis" for convolutional neural network filters. This paper extends this hypothesis by radically constraining its scope to a single set of just 8 universal filters that depthwise separable convolutional networks inherently converge to. While conventional DS-CNNs employ thousands of distinct trained filters, our analysis reveals these filters are predominantly linear shifts (ax+b) of our discovered universal set. Through systematic unsupervised search, we extracted these fundamental patterns across different architectures and datasets. Remarkably, networks initialized with these 8 unique frozen filters achieve over 80\% ImageNet accuracy, and even outperform models with thousands of trainable parameters when applied to smaller datasets. The identified master key filters closely match Difference of Gaussians (DoGs), Gaussians, and their derivatives, structures that are not only fundamental to classical image processing but also strikingly similar to receptive fields in mammalian visual systems. Our findings provide compelling evidence that depthwise convolutional layers naturally gravitate toward this fundamental set of spatial operators regardless of task or architecture. This work offers new insights for understanding generalization and transfer learning through the universal language of these master key filters.

Ocean's 8: The Quest for Universal Master Key Filters in DS-CNNs
Zahra Babaiee" Technische Universität Wien
Peyman M. Kiasari* Technische Universität Wien
Vienna, Austria zahra. babaiee@tuwien. ac
Vienna, Austria
peyman wwwaaaiaiiirreeera serritttwwennn ac. at
Daniela Rus Massachusetts Institute of Technology Cambridge, MA rusêmit. edu
Radu Grosu
Technische Universität Wien
Vienna, Austria
radu. aadu.grosuttuiie.ccoo ac at
Abstract
A recent study has proposed the "Master Key Filters Hypothesis" for convolutional neural network filters. This paper extends this hypothesis by radically constraining its scope to a single set of just 8 universal filters that depthwise separable convo- lutional networks inherently converge to. While conventional DS-CNNs employ thousands of distinct trained filters, our analysis reveals these filters are predom- inantly linear shifts (ax+b) of our discovered universal set. Through systematic unsupervised search, we extracted these fundamental patterns across different archi- tectures and datasets. Remarkably, networks initialized with these unique frozen filters achieve over 80% ImageNet accuracy, and even outperform models with thousands of trainable parameters when applied to smaller datasets. The identified master key filters closely match Difference of Gaussians (DoGs). Gaussians, and their derivatives, structures that are not only fundamental to classical image pro- cessing but also strikingly similar to receptive fields in mammalian visual systems. Our findings provide compelling evidence that depthwise convolutional layers naturally gravitate toward this fundamental set of spatial operators regardless of or architecture. This work offers new insights for understanding generalization and transfer learning through the universal language of these master key filters. bbtaaaaaeeeeeeyfitteerrrrr Code available at: hhtps: //gt:/ub/tttbb http:///gg.o.oooo///aaaaaaaaaaaa
Introduction
Convolutional Neural Networks (CNNs) have significantly advanced computer vision through their hierarchical representations using trainable filters. As architectures evolved toward greater perfor- mance, models such as VGG (28). ResNet [10], and DenseNet [14] incorporated thousands of filters across their layers. This trend continued with the development of Depthwise Separable Convolutional Neural Networks (DS-CNNs) [3 12. which separate spatial and channel-wist computations for improved efficiency. Contemporary architectures like the ConvNeXt family 24 34
utilize DS-CNNs
with up to 50,000 trainable spatial filters.
Recent research identified a notable pattern in trained depthwise convolutional kernels across various DS-CNN architectures [2] (4. Through analysis of trained filters using unsupervised clustering, they demonstrated that these patterns converge into distinct clusters resembling Difference of Gaussian
Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

N input channels
Classical CNN
DS CN-
Uses N-M feature Sums
Weighted maps
Output Channels
Feature
Expect M output
Weights of MLP layer
channels
Figure 1: Comparison of Classical CNN and DS-CNN architectures: Left: Input with N channels. Center: In Classical CNNs, each output channel is produced by convolving a unique filter with each input channel, followed by summing the resulting feature maps. This results in NxM distinct filters and corresponding feature maps. Right: DS- DS-CNN uses only N filters (one per input channel) to create feature maps, then applies an NxM MLP layer to linearly combine these feature maps into M output channels. DS-CNNs represent parameter-efficien subser of classical CNNs. reducing the number of required convolutional operations.
(DoG) functions and their derivatives. Their study classified over 95% and 90% of filters from ConvNextV2 and ConvNeXt models, respectively, into Gaussian-related clusters, indicating consistent patterns in filter learning.
Subsequently, another work proposed the "Master Key Filters Hypothesis," " [5] proposing that there exist master key filter sets that are general for visual data, and that the depthwise filters in DS-CNNs tend to converge these master key filters, regardless of the specific dataset, task, or architecture. This hypothesis challenges the conventional understanding that convolutional filters become increasingly specialized in deeper layers and suggests instead that set of fundamental filters may underlie the performance of these networks. In this paper, we extend the "Master Key Filters Hypothesis" by radically constraining its scope through identification of minimal fundamental master key filter set. While the original hypothesis posited the existence of general-purpose filter sets for visual data potentially comprising numerous filters across multiple sets our systematic unsupervised analysis across architectures and datasets reveals remarkably compact representation. demonstrate that DS-CNNs predominantly converge toward a basis of just distinct filters, where substantial proportion of learned filters approximate linear shifts of these fundamental kernels. Notably, networks restricted to this compact basis maintain performance integrity, suggesting these filters capture essential visual processing primitives rather than task-specific features. This finding significantly refines the original hypothesis by establishing both the cardinality and specific form of universal filter basis for visual computing.
Linear Shift: Tensors and are linear shifts iff aY where a and b scalars.
Our work also substantially refines the observations made in [2]. which identified Gaussian- patterns in DS-CNN filters without constraining their potential variability. While that study demon- strated the prevalence of Guassian-like filters, it allowed for an effectively infinite continuum of these structures with arbitrary standard deviations and noise characteristics. In contrast, we' we're narrowing down as linear shift of a mere 8 fundamental filters. This significantly narrows the theoretical space.
These identified filters correspond to mathematical forms matching Difference of Gaussians (DoGs), Gaussians, and their derivatives, which are established components scale-space theory 20 and share structural similarities with receptive fields observed in mammalian visual systems [35 36)
Networks initialized with these 8 filters achieve over 80% ImageNet accuracy and demonstrate superior performance compared to models with thousands of trainable parameters when applied to smaller datasets.
Our findings provide empirical support for the "Master Key Filters Hypothesis" and suggest potential applications in efficient network design and transfer learning, while contributing to the understanding of generalizable patterns in visual processing systems.

---

2 Related Work
Depthwise Convolutional Filters. Depthwise Convolutions (DCs) have revolutionized the design of Convolutional Neural Networks (CNNs) by using only one feature map per input channel, leading to the development of lightweight and performant architectures such as MobileNet [3], Efficient- Net [30], and ConvNeXt (24). As illustrated in Figure | classical CNNs utilize Cin X Cout completely separate filters, creating independent feature maps for each input-output channel combination. Each output channel is computed as:
Cin
Y=
Y1
Σ Kiy Xy. for (i.5) E [Cous] x [cm]]
where X E RBHxxxx Xcm is the input tensor with Cin channels, Y; is the i-th output channel, K is the convolutional kernel for the i-th output channel and j-th input channel, and [v]] denotes the set (1.
In contrast, DS-CNNs force the model to use only Can feature maps (one per input channel) followed by linearly combining these feature maps using pointwise convolutions or MLP layers. is important to emphasize that DS-CNNs combine the feature maps resulting from the depthwise convolutions rather than linearly combining the kernels directly:
Y,
Wes K3 * X,, for (i,j) E [Cout] x [Cin] ==1
Although DS-CNNs are restricted CNNs, but this restriction significantly reduces the parameter count while maintaining competitive performance. They have demonstrated superior efficiency and accuracy within their class size on diverse casks-including ImageNet classification, scene understanding, and graph conceptualization-compp
to traditional CNNs and vision transformers 24
34 3
Recent studies have revealed striking properties of depthwise convolutional filters in these networks. Trockman et al. 333 observed that learned filters in their DS-CNN model ConvMixer exhibit highly structured covariance matrices. Furthermore, Babaice al. 2 discovered that trained depthwise convolutional kernels across all layers of DS-CNNs converge into few main clusters, each resembling the difference of Gaussian (DoG) functions and their first and second-order derivatives. The authors were able to classify the majority of the filters from state-of-the-arr DS-CNN models. Building on this work, recent work. (5] introduced the "Master Key Filters Hypothesis," " which proposes that depthwise filters in DS-CNNs exhibit generality across domains, architectures, and layer depths, challenging the conventional view that deeper layers become increasingly specialized. Our work builds upon these findings by investigating the potential of using limited number of unique filters in DS-CNNs, exploiting the observed clusterability depthwise convolutional kernels, and moving towards finding the master key filters.
Filter Diversity. Filter pruning and compression techniques have been widely explored reduce the computational complexity and memory footprint of CNNs M.. Structured pruning in CNNs typically achieved by removing redundant filters 19 These methods highlight the importance of "feature-map" diversity in CNNs, as removing redundant or less informative filters can lead to more efficient models without significant performance degradation. In contrast, our work is not attempting to reduce the number of parameters computational cost, but rather investigating the role of "filter" diversity DS-CNNs and challenging the assumption that a large number of unique filters is necessary for optimal performance. By discovering that a small set of carefully chosen filters can effectively replace a large number of learned filters in DS- ON-NNNs we are shedding light on the inherent limited diversity present in the learned depthwise filters.
Scale-Space Theory. Scale-space theory, which examines signals across different scales, was initially [1] and has since become a fundamental concept in signal processing,
developed in the particularly within the field of computer vision. Lindeberg 20 introduced a computational framework for visual receptive fields that exploits symmetry properties across space and time. This framework is [23]: First, offers a normative perspective on visual processing that closely aligns with the hierarchical stages observed in the visual systems of higher mammals i.. Second, provides provable approach to capture the natural transformations of images over space and time [22. Gaussian derivatives are the sole kernels satisfying isotropy (rotational invariance)
compelling for two reasons
3

---

fiter
Code [0.1]
50 Samples
Figure 2: Visualization of selecting candidate filters using autoencoder-basse dimensionality reduction. Left: An autoencoder compresses filters into a ID hidden code. Right: Heatmap of 50 uniformly sampled candidate filters from ID hidden code, generated by the decoder part of the autoencoder. These samples serve as the initial pool for our search for the master key filters.
and non-creativity (with respect to the causality principle) in scale-space theory [20] Remarkably, our work reveals that the master key filters required for efficient performance in DS-CNNs consists of only 8 distinct filters: Gaussians, Difference of Gaussians (DoG, which can be approximated by the Laplacian of Gaussians), and first derivatives of Gaussians. This finding establishes strong connection between the principles of scale-space theory and the design of CNN architectures.
Do We Need Thousands of Distinct Filters?
In this section, we investigate whether employing thousands unique filters is essential for maintain- ing the performance of DS-CNNs. In particular, explore what is the impact on the performance of the network, when we replace the trained filters with minimal set distinct filter variations.
3.1
The Quest for Master Key Filters
3.1.1 Autoencoder Design
To reduce the number of distinct filters in DS-CNNs distilled filters from publicly available trained models of various sizes. Specifically, we gathered all depthwise x 7 filters from every layer in our model bank, centered each filter and scaled its L2 norm to I, and used this normalized collection to train an autoencoder that encodes each filter into a single dimension (Fig. 2). following (2]. This procedure captures the essential filter characteristics while markedly reducing dimensionality.
The autoencoder architecture comprises two primary components: an encoder and a decoder. The encoder consists of four intermediate layers, each followed by leaky rectified linear unit (Leaky ReI.U) activation function. These layers progressively compress and abstract the input filter repre- sentations. The final layer of the encoder, known as the code layer, employs sigmoid activation function to map the compressed filter representations to values within the range [0, 1]. This mapping ensures that the encoded filters are dimensional, thus easy to sample.
The decoder on the other hand, is responsible for reconstructing the original normalized filters from the encoded representations. It mirrors the structure of the encoder, with four intermediate layers that gradually upsample and expand the encoded features. The final layer of the decoder utilizes hyperbolic tangent (tanh) activation function, which allows for accurate reconstruction of the normalized filters within [-1, 1j. This choice of activation function ensures that reconstructed filters maintain their centering and scale, aligning with the characteristics of the original normalized filters.
After training the autoencoder, we performed uniform sampling from the code layer. We took various numbers of samples, 50, 25, and 10, from the [0, interval to generate filter sets (Fig. (2). Using the decoder, we transformed these codes back into filters. We name them "candidate filters".
3.1.2 Linear shift (ax+b) approximation
In a layer, e-th channels depthwise filter can be denoted as F, e Bi where is the index of the channel. When flattened, F. can be represented as vector f, E IBA2 The matrix F composed of all flattened vectors of layer is F .ff....,fo) e Cx

---

For each depthwise filter F. learned by the ConvNeXtV2-tiin model, we identified the scalar coefficients a and which minimized the Euclidean distance between the corresponding flattened filter vector Je, and the linear combination a5% + b, where S% represents flattened candidate filter. The original filter was then substituted with the optimal linear combination alkm... + that exhibited the smallest distance to the original.
To solve for scalars and that minimize the distance between vectors fo and af% + b, we use linear regression. Here, the goal is to determine the coefficients a and 6 for two vectors æ and y such that by having " = az the length of the vector - y is minimized. This problem has well-known solution. If the length of our vectors is I we have:
1=
T.l4
im
Vi
b 1i
Liml T4y4
1E i=
x²)²
Calculating Equations ① can be computationally intensive, especially when dealing with hundreds of thousands of filters. To reduce computational complexity, we can use a normalization trick. Since any linear shift of x does not alter the optimal ÿ, normalize x using the transformation & [|-
With this normalization, E 24 and E'=1 d7 allowing us simplify Equation
(x, v) b 2im1 Di E Dith n
a
y
n
Consequentially, Given the vectors 11, ....... En as the rows of matrix X and the vectors y1 .2..... : Vm as the columns of matrix Y. we introduce the vector Ymean- which contains the means 71, .2,,..., Dm. Using these, we can calculate the coefficients dzj and biş for each pair of a;; and yj through matrix multiplication.
A= -=
B
ymean
For each layer, with the set of depthwise filter vectors matrix F and the sample filter vectors matrix F', we calculate the coefficients as above to find the closest linear-shift approximation. We chose linear shifts for approximating the original filters because they preserve the heatmap and essential characteristics of the filters. By applying a linear shift to a filter sample, we maintain the spatial structure and relative importance of different regions within the filter.
3.1.3 Evaluation of F"
In order to evaluate the impact of replacing the original filters with their lincar-shift approximations derived from the sampled filter set, we assessed the performance of the modified models on the test set. In Table T we present the accuracy of models with varying sizes from the ConvNeXtv2 and Hornet (27 families, along with their accuracy after their filter replacement. Quite remarkably, when replacing all the filters of the models with approximations based on only 50 sampled filters, the model performance remains robust, even without any fine-tuning. This resilience is particularly evident for larger model sizes. In the case of ConvNeXiv2 Huge, replacing nearly 50K filters with just 50 sampled filters results in less than two percent accuracy drop, without any fine-tuning.
As expected, reducing the number of sampled filters leads to larger accuracy gap, and the Con- vNeXtv2 models struggle to perform well when using small set of only 10 filters. However, it is important to note that the filter samples used in this experiment were obtained through uniform sampling from the code layer of the autoencoder. This immediately raises the following question: Is there a more strategically selected set of filter samples which can yield a better performance?
To elucidate this question, we focused on the ConvNeXt-v2-Tiin Tiny model and conducted a greedy search on a set of filter samples. We began with the 50 uniformly sampled filters and iteratively removed filters one by one. Figure illustrates the 50 filter samples used in this search. At each iteration, we evaluated the model aceuracy after removing each individual filter and eliminated the one whose removal resulted in the smallest accuracy drop. This process was repeated for all filters.
5

---

Table 1: Performance comparison when thousands of trained filters are replaced with linear shifts (ax+b) from candidate filters. Without any fine-tuning (For trained model see Table models with just selected filters from our greedy search on ConvNeXiv2 Tiny maintain remarkably high accuracy (e-g., only 3.5% drop for ConvNeXtv2 Huge despite reducing from 50k to 8 unique filter patterns) and even on different architecture, HorNet. Concidering models' high sensitivity to filter alterations, this is evidence that DS-CNN filters predominantly converge to these filters.
ConvNeXt
Hornet
ConyNeXtv2 Models
Pico
Tiny
Base
Large
Huge
Tiny
Small
Number of Filters
2944
624
18 048
27 072
49 632
11 488
17222
Original Ace
80.3%
83.0%
84.9%
85.8%
86.3%
82.3%
83.5%
Acc with 50 candidates
75.0%
75.4%
80.5%
83.2%
84.0%
79.4%
81.3%
Ace with 25 candidates
72.0%
66.9%
72.8%
79.6%
80.4%
78.3%
80.9%
Acc with 10 candidates
23.4%
1.0%
1.4%
3.0%
2.0%
66.3%
70.5%
Acc with 8 (greedy search)
73.1%
76.7%
79.3%
81.2%
82.8%
76.0%
78.1%
Acc with random filters
0. 0.11% %
0.10%
0.10%
0.12%
0.09%
0.96%
1.0%
The accuracy plot during the greedy search, shown in Figure 3 reveals interesting trend. The model's s accuracy remains relatively stable until the last 10 filters are removed, with the curve exhibiting distinct elbow around 8 samples. This observation suggests that a small subset of only 10 filters is playing a crucial role in maintaining the performance of the model. To further refine the search, selected the 10 best-performing filters from the previous step and expanded our search space by sampling additional filters around each of these 10 filters. This local exploration allows us to fine-tune the selection of filters and capture any potential variations that may enhance performance.
We then conducted a second round of the greedy search using this expanded set of filters. The search converged to a set of 8 filters located just after the curve elbow. These 8 filters, as
Accuracy Plot with Highlighted Elbow Point
Accuracy
50
Elbow Point
shown in Figure f represent highly informa- tive subset that can effectively replace the orig-
25
Non-essential Filters
Last 8 Filters
inal large set of filters, while minimally impact- ing the model's accuracy.
40
20
Number of Remaining Filters
In order to make sure that our search is not vi- olating the "Test Set Violation" principle, we repeated the experiment as follows. We sep- arated 100 random samples from each of the classes of the ImageNet training set. We then used this new set our evaluation set for the greedy search on the 50 filter samples. We followed the same steps as outlined before, and the search resulted the same filters.
Figure 3: Our systematic greedy search for the essential filters While the removal of most filters did not noticeably change accuracy, of them were essential, consistently in all models tested.
The last row of Table showcases the accuracy of the models, when their filters are replaced by the filter transformations, obtained from the greedy search. Remarkably, the results demonstrate that the No1v2eXtv2 models accuracy achieved with these 8 filters, surpasses even the performance of the 25-filters-sampl set. This finding underscores the effectiveness of the greedy search approach in identifying a highly discriminative subset of filters. Moreover, it highlights the potential for replacing large number of filters, up 50K in the case of ConvNeXt v2 models, with just 8 strategically selected filters, while maintaining an acceptable performance.
To validate the generalizability of these 8 filter samples, we extended our experiments to the ConvNeet--v2PPiie model, which represents a different model size. this case also, we arrived at a similar set of 8 filter samples, indicating the robustness and transferability of our findings across different model architectures. The consistency of the filter samples across different model sizes suggests that these filters capture fundamental and generalizable patterns in the data. It hints at the existence of set of universal filters that can effectively represent the essential information required for accurate classification.

---

Figure 4: Heatmap visualization of the eight universal filters discovered through systematic greedy search on the ConvNeXtv2 tiny model. Our empirical analysis demonstrates that DS-CNN filters predominantly converge to linear shifts (ax+b) of one of these eight filters, regardless of architecture or dataset. Filters 1-4 display central difference operator characteristics, and filters 5-8 correspond to established mathematical image processing fundamentals (See Figure
Filter Figure 5: Correspondence between the empirically discovered filters (5-8) and their theoretical mathematical counterparts. Left column shows the idealized mathematical forms: first derivatives of Gaussians in x and directions (filters 5-6). Difference of Gaussians (filter 7), and Gaussian function (filter 8). Right column shows our discovered filters that closely approximate these operators, demonstrating the network's natural convergence toward established visual processing primitives.
Gaussian
didy
The remarkable performance maintained when replacing thousands trained filters with just our 8 master key filters cannot be coincidental. Noting models' high sensitivity to filter alterations (evidenced by the catastrophic performance drop with 8 random filters), strongly indicates that DS- CNN filters predominantly converge to fundamental patterns during training. Even more compelling, these filters (discovered exclusively from ConvNextV2 Tiny) transfer seamlessly to architecturally distinct models like Hornet. This fascinating cross-architecture generalization suggests that DS- CNNs naturally gravitate toward a mathematically well-defined universal filter code that captures fundamental visual processing operations.
Understanding the Eight Filters
This subsection investigates into the functional characteristics of the eight filters identified through our systematic greedy search, and which have proven to be very effective across various datasets. By analyzing these filters, we aim understand their resemblance traditional image-processing operators and their potential roles in effective feature extraction within the network.
Filters 1-4: These 4 filters in Figure exhibit characteristics reminiscent of central difference operators, commonly used for approximating Gaussian derivatives discretely. The arrangement and weights of these filters mimic the theoretical models used in edge detection and texture analysis.
Filters 5-6: These 2 filters strongly resemble 1st order Gaussian derivatives along the x and y axis. These filter contribute more pronounced spatial smoothing than previous filters. This characteristic enables these filters to capture broader and more varied textural information from the input images, potentially allowing for better generalization across different visual contexts.
Filters 7: This filter resemble 2-D discrete analogue of the Difference of Gaussians (DoG) due to its positive center with slightly negative surround. The DoG filter, often approximated by the Laplacian of the Gaussian digital image processing, is crucial for blob detection and bar pattern recognition in images. These filter likely contribute to the model's ability to differentiate areas of rapid intensity change, enhancing edge and contour detection.
Filter This filter closely aligns with a very fine-scaled Gaussian kernel. image processing, Gaussian kernels smoothing filters used to reduce noise and detail. This results in blurred image that preserves edges better than uniform filters. Gaussian filters are mathematically proven to be the only function for scale-space representation.
Filters formal definition: completeness, we provide below the formal definition of the continuous functions corresponding to the 2D Gaussian, the 2D derivative of the Gaussian along the and the
7

---

Table ImageNet Top-1 accuracy comparison between conventional trainings and our 8-filter constraint. Models restricted to using only our 8 unique filters (plus learnable bias terms) achieve comparable accuracy their fully-trained counterparts. The consistent performance across different architectures (ConvNeXtv2 and Hornet) demonstrates the universality of these fundamental filters.
ConvNeXtv2
Hornet
Models
Pico
Tiny
Base
Large
Tiny
Number of Original Filters
2944
6224
18 048
27
11 488
Original model with FCMAE'
80.3%
82.9%
84.9%
85.8%
Original model
79.7%
82.5%
84.3%
84.5%
82.3%
Our 8 unique filters bias
80.2%
82.7%
84.6%
85.4%
81.8%
FCMAE (Fully Convolutional Masked Autoencnder Framework) is a heavy pretraining.
This accuracy is from the model released by the ConvNeX(V2's official paper repository. We trained the model using their exact same script and training parameters (with the default random seed 0) and achieved an accuracy of 82.7%, matching the result obtained with our 8-filter configuration.
axis, respectively, and the 2D difference of Gaussians (DoG, Laplacian, Mexican hat):
Gaussian: G(v, --
AGaussian: DoG(a, = Gi (a v) G2( v)
These formulations were used to construct the last four filters as depicted in Figure with the exception of the DoG, for which its approximation, the Ricker wavelet, was used for simplicity. The reconstructed filters bear strong resemblance to those discovered through our encoding and greedy search methods, validating our hypothesis. in function approximation and emphasizing the practical relevance of traditional image processing theories modern deep learning architectures.
Functional Approximation and Construction: We reconstructed the lower four filters using theoretical formulas typically associated with these image processing techniques, depicted Figure The reconstructed filters bear strong resemblance to those discovered through our encoding and greedy search methods, validating our approach filter selection and emphasizing the practical relevance of traditional image processing theories in modern deep learning architectures.
4 Experiments
So far, we' ve identified 8 unique filters that, when used as linear-shift approximations to replace the filters of trained models, maintain relatively stable accuracy despite this dramatic change in model parameters. These findings naturally lead to key question: Can models be successfully trained from initialization with just these filter types kept fruzen throughout training? this section, we present experimental evaluations on ImageNet and additional datasets to investigate this question.
4.1 ImageNet
The results in Table demonstratee that model accuracy remains stable despite significantly reduced filter diversity. In these experiments, model filters linear shifts of one of the 8 identified filters, mathematically expressed as a(x b). Given the architecture of DS-CNNs, the coefficient can be transferred the fully-connected layers following depthwise convolutions (the following pointwise layer), effectively simplifying the filters to x + b, where acts as a learnable bias. This insight motivated to train models from scratch using only these 8 fixed filters with learnable biases.
Training with Only Frozen Filters
To investigate the effectiveness of our 8 candidate filters, we trained ConvNeXtv2 models from
scratch, initializing each layer's filters with these 8 filters while allowing only the bias terms to be
trainable. We followed the same 300-epoch training pipeline described in the original paper [34]. with the critical difference that all convolutional filters remained frozen throughout training. Table presents our results. Remarkably, the ConvNeXtv2 Tiny model with only 8 types of filters achieved

---

an accuracy of 82.7%, merely 0.2% lower than the model trained with 6,624 trainable filters and FCMAE pretraining. Similarly, the smaller ConvNeXtv2 Pico model with 8 types of frozen filters reached 80.2% accuracy, just 0. 1 % below the model with 2,944 trainable filters.
To validate the generalizability of additional experiment with the Hornet model [27] a DS-CNN with substantially different structure than the ConvNext family. The Hornet Tiny model with only our 8 filters achieved 81.8% accuracy compared to 82.3% for the original model, representing only a 0.5% drop. Notably, these 8 filters were derived exclusively from ConvNext models through greedy search on the ConvNeXtv2 Tiny model, yet transferred effectively to Homet without modification.
findings across different architectures,
conducted an
It is worth emphasizing that despite our significant architectural modification, these experiments used the original training hyperparameters for each model. A dedicated hyperparameter search optimized for this fixed-filter approach could potentially enhance results further.
Table 3: Cross-dataset evaluation demonstrating the superiority of our universal filter approach on smaller datasets. The ConvNeXt Femto model restricted to using only 8 unique frozen filters outperforms both models trained from scratch and those using transferred ImageNet filters, on Oxford Pets and Oxford Flowers. We evaluate multiple model sizes (Atto, Femto, Pico, Tiny) for Flowers and Pets datasets verify that our observed advantage is consistent across architectures of varying capacity and not merely an artifact of dataset size limitations.
Dataset
CIFAR10
STL-10
Oxford Flowers
Oxford Pets 3680
Training Set Size
50000
5000
2040
Model
Femto
Femto
Atto Femto
Pico Tiny
Atto
Femto
Pico
Tiny
Filters
Original (normal traning)
96.9
80.4
63.3 66.0
60.2
75.7
38.4 36.3
40.1 65.4
ImageNet Transferring
97.1
83.2
72.2 73.2
74.8 81.8
58.5 56.0
66.3 80.1
Our 8 Unique Filters
96.3
83.1
77.8 77.7
77.2 85.1
66.5 66.4
72.8 81.8
4.2 Other Datasets
To investigate the generalizability of our findings, we extend our experiments to other datasets and compare the performance of the ConvNeXt Femto across various settings.
Datasets and Settings. We evaluate the low filter variety on four datasets: CIFAR-10 10 [6], Flow- ers [25). Pets (26). and STL-10 10 [8) These datasets have smaller scales compared to ImageNet, with the size of training sets ranging from 2040 to 50000 samples. We use the ConvNeXt Femto model as our base architecture for all datasets, and additionally use ConvNeXt Atto, Pico, and Tiny for the Flowers and Pets datasets. For a fair comparison, we train the model on all datasets for 300 epochs, following the training parameters from the ConvNeXt paper (24), and keep the training settings consistent across all runs and datasets. For each dataset, we first train the model to obtain baseline accuracy. We then evaluate two filter initialization strategies: (1) depthwise filters transferred from a ConyNeXt model pretrained on ImageNet, and (2) eight frozen filter types trained from scratch. Table shows the resulting accuracies for each setting.
Results. The results demonstrate the effectiveness of using the 8 frozen filters across different datasets. Notably, the performance improvement becomes more pronounced as the dataset size decreases. For the Flowers and Pets datasets, the frozen 8 filter types achieve remarkable improvements of up to 1 11% and 34.5%, respectively, compared to the baseline model. Interestingly, on these smaller datasets, the eight frozen filter types even outperform the transferred filters from the model trained on ImageNet. To further evaluate and verify the performance of our filters these datasets, used other sizes of the ConvNeXt model, the results of which showed consistent superior performance on all sizes.
This observation suggests that the carefully selected filter types capture fundamental patterns that are highly relevant to the task hand, even when the dataset size is limited. This finding has significant implications for scenarios where training data is scarce computational resources are limited.

---

5
Conclusions
This paper extends the "Master Key Filters Hypothesis" by identifying a set of just filters. While conventional DS-CNNs employ thousands of distinct trained filters, our analysis reveals these filters predominantly converge to linear shifts (ax+b) of one of the filters in our discovered set. This finding significantly narrows the theoretical space proposed in previous work. The discovered filters closely match established mathematical forms: Difference of Gaussians, Gaussians, and their derivatives, creating bridge between classical computer vision theory and modern deep learning practice. This correspondence to structures found in both scale-space theory and mammalian visual systems suggests DS-CNNs inherently rediscover optimal operators aligned with natural image statistics.
Our systematic experiments demonstrate that networks initialized with these 8 frozen filters achieve over 80% ImageNet accuracy. Particularly noteworthy is the superior performance of our filters on smaller datasets, where models initialized with our filters outperform even ImageNet transfer learning. This suggests that these filters encode fundamental visual processing primitives that transcend specific datasets and visual domains, offering a novel approach to transfer learning.
Future Work may explore direct optimization approaches to refine our master key filter set. While our filters demonstrate impressive performance, systematic optimization might further improve their effectiveness or reduce their number. The observed constraint on filter diversity invites to rethink the fundamental principles governing these architectures, potentially leading to insights about the complementary roles of depthwise spatial filters and pointwise channel mixing layers and opening opportunities for novel architecture designs.
References
[1] Zahra Babaiee, Ramin Hasani. Mathias Lechner. Daniela Rus. and Radu Grosu. On-off center-surround receptive fields for accurate and robust image classification. In Marina Meila and Tong Zhang, editors. Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 478-489, PMLR, 18-24 Jul 2021.
[2] Zahra Babaice, Peyman Kiasari, Daniela Rus, and Radu Grosu, Unveiling the unseen: Identifiable clusters trained depthwisc convolutional kemels.. In The Twelfth International Conference Learning Representations, 2023.
[3] Zahra Babaice, Peyman Kiasari, Daniela Rus, and Radu Grosu. Visual graph arena: Evaluating visual con- ceptualization of vision and multimodal large language models. In Forty-second International Conference on Machine Learning, 2025.
[4] Zahra Babaiee, Peyman M. Kiasari, Daniela Rus, and Radu Grosu. Neural echos: Depthwise convolutional filters replicate biological receptive fields. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 8216 8225, January 2024.
[5] Zahra Bahaiee, Peyman M. Kiasari, Daniela Rus, and Radu Grosu. The master key filters hypothesis: Deep filters are general. In Proceedings of the AAAI Conference on Arrificial Intelligence, 2025.
[6] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. In International Conference on Learning Representations, 2022.
[7] Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. Electra: Pre-training text encoders as discriminators rather than generators. In International Conference on Learning Representations. 2020.
[8] Adam Coates, Andrew Y. Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215-223, 2011.
[9] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoe Le. Randaugment: Practical automated data augmentation with reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 702-703, 2020.
[10] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR '16 pages 770-778. IEEE, June 2016.
10

---

[11] Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks, 2021.
[12] Andrew Howard, Mark Sandler, Grace Chu, Liang-Chied Chen, Bo Chen, Mingxing Tan, Weijun Wang. Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoe V. Le, and Hartwig Adam. Searching for mobilenetv3. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 1314 1324, JEEE,
2019.
[13] Andrew Howard, Menglong Zhu, Chen, Dmitry Kalemichenko, Weijun Wang, Tobias Weyand, Marco Andrectto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR, a 704,04861, 2017.
[14] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. Densely connected convo- lutional networks. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pages
IEEE, 2017.
[15] Jan J Koenderink. The structure of images. Biological cybernetics, 50(5):333-33) 1984.
[16] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical Report TR-2009, University of Toronto, Toronto, Canada, 2009.
[17] Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Fractalnet: Ultra-deep neural networks without residuals. In International Conference on Learning Representations. 2017.
[18] Siyuan Li, Zedong Wang, Zicheng Liu, Cheng Tan, Haitao Lin, Di Wu, Zhiyuan Chen, Jianghin Zheng, and Stan Z Li. Efficient multi-order gated aggregation network. arXiv preprint arXiv:2211.03225. 2022.
[19] Lucas Liebenwein, Cenk Baykal, Harry Lang, Dan Feldman, and Danicla Rus. Provable filter pruning for efficient neural networks, 2020.
[20] Tony Lindeherg. Scale-space theory in computer vision, volume 256. Springer Science & Business Media, 2013.
[21] Tony Lindeberg. Normative theory of visual receptive fields. Heliyon, 7(1)87 2021.
[22] Tony Lindeberg. Covariance properties under natural image transformations for the generalised gaussian derivative model for visual receptive fields. Frontiers in Computational Neuroscience, 17, June 2023.
[23] Tony Lindeberg. Approximation properties relative to continuous scale space for hybrid discretizations of gaussian derivative operators, 2024.
[24] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[25] Maria-Eless Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics and Image Processing, pages 722-729, 2008
[26] Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V. Jawahar. Cats and dogs. In IEEE Conference on Computer Vision and Pattern Recognition, pages 3498 3505, 2012.
[27] Yongming Rao, Wenliang Zhao, Yansong Tang. Jie Zhou, Ser-Lam Lim, and Jiwen Lu. Hornet: Effi- cient high-order spatial interactions with recursive gated convolutions, Advances in Neural Information Processing Systems (NeurlPS), 2022.
[28] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. CoRR, abs/ 11wp11009656661 2014.
[29] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1118-22262222 2016.
[30] Mingxing Tan and Quoc V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. CoRR, abs/1905. 1946, 2019.
[31] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou, Training data-efficient image transformers and distillation through attention. In International Conference on Machine Learning, pages 10347-10357, 2021.

---

[32] Asher Trockman and J. Zico Kolter. Patches are all you need? CoRR, 0es/2201.09792 2022.
[33] Asher Trockman, Devin Willmott, and J Zico Kolter. Understanding the covariance structure of convolu- tional filters. In The Eleventh International Conference on Learning Representations, 2023.
[34] Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. arXiv preprint arXiv:2301.00800 2023.
[35] R.A. Young, R.M. Lesperance, and W.W. Meyer. The gaussian derivative model for spatial-temporal vision: cortical model. Spatial vision. 1433-4)221-319,2200 2001.
[36] Richard A. Young. The gaussian derivative model for spatial vision: I. retinal mechanisms. Spatial Vision,
2(4):273 293, 1987.
[37] Sangdoo Yun, Dongyoon Han, Scong Joon Oh, Sanghyuk Chun, Junsuk Choc, and Youngjoon Yoo. Cutmix: Regularization strategy train strong classifiers with localizable features. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6023-6032, 2019.
[38] Hongyi Zhang. Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization, In International Conference Learning Representations. 2018.
12

---

A Technical Appendices and Supplementary Material
A.1 Greedy Algorithm pseudo code
Algorithm 1 Greedy Filter Selection
Require: E RC*k, original network filters
Require: F" E RRmx candidate filter pool, x 50
Ensure: S F" minimal filter set
1: Initialize S F"
2: repeat
3: for each S E S do
4: Approximate using S {ff} with linear shifts: F; arg min || Compute accuracy Acc( F:) end for
(aftys 6)ll
5:
6:
7: Take the least important filter: f 4- arg minges AAc((S) Acc(Fi)
8: Remove the least important filter: S + S S. {
9:
until Elbow point detected accuracy curve
10:
return
A.2 Autoencoder Model Bank Details
To construct comprehensive and diverse model bank for filter distillation, collected pretrained convolutional neural networks spanning multiple architecture families and training configurations. Our goal was capture wide range of depthwise convolutional filter characteristics representative of both lightweight and large-scale models.
The bank includes models from the ConvNeXtV2 (34), ConvNeXt 24], MogaNet [18], HorNet [27]
and ConvMixer (32] families, each trained on either ImageNet- or ImageNet-22K (and their 22k-+1k fine-tuned variants) at multiple input resolutions. All models were obtained from publicly available checkpoints released by their respective authors. From each model, we extracted all depthwise convolutional kernels of size 7 7.
Model Families and Variants:
The ConyNeXtV2 family includes the following variants: atto-224-Ik, femto-224-1k, 1ico-224-Kk nano-224-1k Ik, nano-224-22k, nano-384-22k, tiny-224-1k; tiny-224-22k, tiny-384-22k. base-2244 224-222, base-384-22k, large-224-1k, large-224-22k, large-384-22k, huge-224-1k, huge-384- 22k, and huge-512-222k
base-224-Ik.
The ConyNeXt family includes: tiny-224-Ik, tiny-224-22k Ik, tiny-384-22k: 1k, tiny-224-224 small- 224-1k, mmall-224-22k 1k,
small-224-22k, base-224-Ik, base-384-1k. base-224- 22se2222422,, large-224-1k, large-384-1k,
22k 1k, fase-384-22k_11 2aarge--224-14 224-2k-2
1k, large-384-
22k Ik, large-224-22k, xlarge-aaa---4
[2k_,,, xlarge-384-22k Ik, and Mlarge-224-22k.
The MogaNet family includes: tiny-224-1k, tiny-224-1k tiny-256 1k, small-224-1 base-224. 1k. large-224-11 Ik, and xlarge-224-14
The HorNet family includes: tiny-224-18 Ik, small-224-1k, base-224-1k and large-224-1k. Finally, the ConvMixer family includes: 512-20 22----layee-1k, 768-32 268-32_layer-1k Ik, and 768- "
In total, the model bank comprises more than one million filters from 49 pretrained models across five architecture families.
A.3 Filter Type Proportions
To illustrate the proportions of filters assigned to each our eight filter types, Figure/Bshove corresponding percentages for ConvNeXtV2 B and L,, as well as HorNet T and B. An interesting observation at first glance that each model family exhibits fairly consistent proportions of each filter type in its trained filters, regardless of model size. Depending the architecture, the most
13

---

Filter Type Proportions per Model
Filter Type
type 2
HorNet
Type 3
type
Type
Type 6
Type 7
HorNet T
Type
Model
ConvNeXtV2
ConvNextvz
20
40
60
80
100
Proportion (%6
Figure 6: Proportions of the eight filter types across ConvNeXtV2 and HorNet models. Each model family shows consistent internal distributions.
prevalent filter type differs. For instance, in ConvNeXtV2, the most frequent was the Gaussian filter (Type 8), whereas in HorNet, was Type 1. As shown, ConvNeXtV2 has similar proportions within its family but distinct distributions compared to the HorNet family. Despite the lower percentages of filter types 1 1-4 in ConvNeett22, they remain essential for model accuracy. Experiments in which these four types removed resulted in noticeable drop in accuracy.
A.4 Filter Type Approximation Quality
Figure shows the box plots of the distances between filters and their assigned filter types. For reference, a random filter and its corresponding distance are also included. As the plots indicate, the mean distance for all filter types is lower than that of the reference, confirming meaningful type-specific clustering. Notably, Filter exhibits the lowest mean distance while also accounting for more than 20% of all filters in the ConyNeXtV2 models (See Figure
Apprtsimetion Quality the filters comtoared with Random Daseline
20-
for
Figure 7: Box plots of distances between filters and their assigned filter types. All filter types show lower mean distances than the random reference, with Filter exhibiting the lowest mean and strongest clustering.
14

---

A.5 Experimental Settings
Table 4: Training (t) and fine-tuning (ft) hyperparameters used Section 4.2 experiments for
ConvNeXtv2 Tiny model, taken from (34..
config
value
optimizer
AdamW
base learning rate
8e-4
weight decay
0.05
optimizer momentum
31.82 0.9, 0.999
layer-wise Ir decay 7 6
0.9
batch size
1024
learning rate schedule
cosine decay
warmup epochs
(t) 40, (ft)
training epochs
(1) 300, (ft) 100
augmentation
RandAug (9, 0.5) 9
label smoothing 29
0.1
mixup [38
0.8
cutmix [37
1.0
drop path
0.2
head init 3
0.001
ema
0.9999
Table 5: Training (t) and fine-tuning (ft) hyperparameters used in Section 4.2 experiments for ConvNeXtv2 Pico model, taken from (34). config
value
optimizer
AdamW
base learning rate weight decay
2e-4
0.3
optimizer momentum
B1, B2 = 0.9, 0.999 0.9
layer-wise Ir decay 7 6
batch size
1024
learning rate schedule
cosine decay
warmup epochs
training epochs
(1) 600, (ft) 100
augmentation
RandAug (9, 0.5) 9
label smoothing 29
0.2
mixup [38
0.3
cutmix 37
0.3
drop path 17
0.0
head init 31
0.001
ema
0.9999
A.6 Training Curves
Figure shows the training and validation losses for CONVNEXTV2- trained with the original script and with the eight convolutional filters frozen ("8-filters"). The constrained model converges slightly more slowly and maintains higher training loss throughout (e.g.. final train loss 2.84 vs. 2.78 for the original). However, the validation losses converge to nearly the same level (final val loss 0.750 vs. 0.740), indicating comparable generalization despite reduced train-time flexibility.
A.7 Master Key Filters
In Figure we provide the full numerical values of the 8 discovered universal filters, each of size 7 7. These filters were derived from a greedy search over encoded depthwise filters, as described in Section 3.1, and used in all experimental evaluations (Sections 4.1 and 4.2).
15

---

Table Information of Datasets used in the study and sample sizes, training set size descending order.
Dataset
Classes
Train Samples
Test Samples
ImageNet
1000
1.2 million
50,000
CIFAR-10 16
10
50,000
10,000
STL-10 g
10
5,000
8,000
Oxford-IIT Pets 226
37
3,680
3,369
Oxford 102 Flowers 25
102
2,040
6,149
Table 7: Training hyperparameters used in Section 4.2 experiments. The setting taken from
ConvNeXt 24
config
value
optimizer
AdamW
base learning rate
4e-3
weight decay
0.05
optimizer momentum
B1 (/ 0.9, 0.999 4096
batch size
training epochs
300
learning rate schedule
cosine decay
warmup epochs
50
warmup schedule
linear
layer-wise wise Ir decay
None
randaugment
(9, 0.5)
mixup
0.8
cutmix
1.0
random erasing
0.25
label smoothing
0.1
layer scale
1e-6
head init scale
None
gradient clip
None
A.8 Experimental Compute Resources
We used NVIDIA TITAN RTX GPUs for experiments on other datasets. For ImageNet training and fine-tuning used NVIDIA TITAN RTX GPUs.
16

---

Trin
Ve ldation Loss vs. Epoch
03
0.0
300
Opech
Opech
(a) Train loss epoch (8-filters original).
(b) Validation loss epoch (8-filters vs. original).
Figure 8: Training curves for ConvNcXtV2-Ting with the original training setup and with eight frozen filters ("8-filters").
(a) Filter
(b) Filter
0.01
-0.02
-0.01
-0.00
-0.01
-0.02
10.01
0..0
-0.01
-0.02
-0.05
-0.04
-0.02
-0.00
0.01
-0.02
0.01
-0.11
-0.00
-0.01
-0.01
-0.02
-0.00
-0.01
-0.06
0.06
-0.01
-0.01
0003
-0.05
-0.09
-0.23
-0.06
-0.05
-0.03
0.00
-0.06
-0.46
0.85
0.13
0.07
-0.03
-0.06
0.02
0.94
0.04
-0.06
-0.03
0.00
0.01
-0.12
0.07
0.02
0.01
-0.02
-0.02
0.00
0.12
0.01
-0.02
-0.02
-0.01
-0.01
-0.01
-0.05
-0.03
-0.01
-0.01
(c) Filter
(d) Filter 4
0.03
-0.02
-0.02
0.07
-0.02
-0.03
-0.03
-0.04
-0.03
-0.02
-0.01
0.00
-0.00
-0.01
0.03
0.02
0.01
0.14
0.01
.0.02
0.03
0..4
.0.01
.0.04
-0.01
0.03
0.01
.0.01
-0.03
-0.04
0.83
-0.05
-0.04
-0.01
0.00
0.03
-0.05
0.00
0.02
0.01
-0.02
-0.02
-0.08
-0.36
-0.09
-0.03
-0.03
0.04
0.08
0.87
-0.35
-0.30
-0.00
-0.00
0.05
0.01
00.01
0.01
0.01
0.00
.0.11
0.01
0..33
0.01
.0.01
0.00
0.00
0.00
.0.02
-0.01
0.00
0.00
0.01
0.01
0.00
-0.00
-0.04
-0.02
-0.01
-0.01
-0.00
-0.00
-0.00
(e) Filter 5
(f) Filter 6
0.05
0.02
0.04
0.01
-0.04
-0.02
-0.07
-0.07
-0.05
-0.08
-0.16
-0.07
-0.04
0.04
0.03
0.05
0.02
.0.02
.0.01
.0.07
.0.03
.0.01
.0.06
-0.14
.0.04
0.00
.0.03
0.09
0.02
-0.47
0.20
0.20
0.54
-0.03
-0.53
-0.20
-0.22
-0.01
-0.01
0.01
0.02
0.01
-0.00
0.00
0.09
0.08
0.19
0.01
-0.22
-0.09
-0.11
0.02
0.03
0.20
0.68
0.20
0.02
0.03
0.04
0.03
0.07
0.01
.0044
0.02
-0.07
-..0
0.02
0.06
0.16
0.05
0.01
0.01
0.05
0.02
0.05
.0.00
-0.04
.0.03
-0.07
0.02
0.03
0.05
0.14
0.06
0.03
0.04
Filter 7
(h) Filter 8
-0.01
-0.04
-0.01
-0.00
-0.02
-0.05
-0.01
-0.00
-0.01
-0.04
-0.03
-0.04
-0.04
-0.04
-0.03
-0.04
-0.01
-0.01
-0.04
-0.06
-0.05
-0.01
-0.01
-0.04
-0.04
-0.02
0.16
-0.01
-0.04
-0.04
-0.01
-0.03
0.98
-0.02
-0.04
-0.02
-0.02
0.16
0.15
-0.04
-0.01
-0.01
-0.05
40.07
-0.06
-0.02
-0.02
-0.04
-0.05
-0.03
0.15
-0.03
-0.05
-0.04
.0.01
00.01
.0.02
.0.03
.0.02
.0.11
.0.01
.0.04
0.004
.0.04
-0.02
.0.04
.0.03
.0.4
Figure 9: The 8 filters.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in abstract and introduction are empirically supported in Sections and 4 through experiments and filter visualizations.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper discusses limitations such as the need for further optimization of the filters (Section 5).
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
18

---

Answer: [NA]
Justification: The paper primarily empirical and does not include formal theorems or proofs, though it provides equations for filter approximation (e.g.. Equation and 2 in Section 3.1).
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
The paper details the models used, datasets, training durations (300 epochs), and evaluation metries, It also outlines the autoencoder and greedy search method in detail (Sections 3.1 and 4.1).
Justification:
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
19

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
aublli//gulless/ooteSuummsssss httpeeSuubbmssiooooooocoy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips.co. cc/ppiccc guwdes/ddeeeee CooeeSumeeieeeeeeeeeee ssi onyylicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
The paper specifies training datasets, model sizes, training duration, and evaluation methods. all experiments with each model, we used the exact same parameters sourced from original paper. See appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Since most experiments are performed on ImageNet, it would be computation- ally unaffordable to do multiple rounds of training.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification: This work follows the NeurIPS Code of Ethies.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
21

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This work does not involve models data with high risk of misuse, such as generative models or private data.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets and models used are cited.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
22

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We present 8 master key filters and provide them in the appendix.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No research involving human participants was conducted.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
23

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: This research does not involve LLMs part of the core methodology.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy battps://neurre http://errip..cccoomeeeeeeee//// cc/CConfereences/2225L/LL for what should or should be described.
24


Reviews:
[{"review_id": "yV7EqDKOH4", "reviewer": "Reviewer_MKLF", "summary": "The paper investigates the shape of depth-wise convolution filters. Specifically, based on \"Master Key Filters Hypothesis\" from previous paper, the paper finds a few universal filters that can represent and replace depth-wise convolution filters in various networks. An auto-encoder in the filter domain is trained on a pre-trained filter set and utilized to find the initial 50 filters. The depth-wise convolution filters in ConvNeXt v2 models are quantized with 50 filters with two parameterizations, and reduced to 8 filters at last. The paper claims that it extends \"Master Key Filters Hypothesis\" to only 8 filters with meaningful performance.", "questions": "- \"Master Key\" topic is already addressed in previous papers. Considering these two papers, what is the unique novelty and contribution of the paper?\n    - The Master Key Filters Hypothesis: Deep Filters Are General in DS-CNNs, AAAI 2025\n    - Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels, ICLR 2024\n- Finding the minimum \"Master Key\" looks interesting. But, what are the practical benefits of this technique? Can this reduce computation or improve generalization performance? \n- I can't understand how \"shift\" works. Does \"shift\" use integers or floats? Is this done by learnable parameters like a and b?\n- I think the \"Master Key\" still has a few degree of freedom: a, b, and shifts. Does it perform better than 3x3 depth-wise convolution? Because it also reduces the # of params, I think it is worth comparing with \"Master Key\".\nThe proposed method should not be a weighted sum of \"Master Keys\". However, I think some points are confusing. Would you please add a formula for selecting \"Master Key\" $f'_c$ to clarify this?\n- What is the definition of $n$ in Eq. (1) and (2)?\n- ConvNeXt and Hornet are pre-LayerNorm based residual architectures. Does \"Master Keys\" work on architectures without pre-LayerNorm? How about non-residual architectures like MobileNetV1?", "limitations": "It would be better to explain the architecture limitation if it is not working on non pre-LayerNorm architecture.", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n- Writing is clear and easy to understand\n- It is interesting to replace ConvNeXt models's filters with only eight filters\n\nWeaknesses\n- The contribution overlaps with previous papers\n  - \"Master Key\" is interesting, but has already been discussed in previous papers. It significantly limits the novelty and contribution of the paper.\n  - I acknowledge that implementing the master keys as eight filters is novel and makes a contribution. But, it is a bit insufficient for me.\n    - The Master Key Filters Hypothesis: Deep Filters Are General in DS-CNNs, AAAI 2025\n    - Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels, ICLR 2024\n- Limited practical contribution\n  - I don't think \"Master Key\" has practical value. It doesn't reduce FLOPs or improve network throughput, and the reduction in the number of parameters is negligible. But it drops the performance.\n  - The experiments like Table 3 can make a practical contribution. But Table 3's settings are outdated, and improvements are not significant.\n- Additional parameters `A, B, shift` for \"master keys\"\n  - The \"Master Key\" introduces additional parameters to represent the filters. It is reasonable to use a few parameters to represent all filters, but I feel it conflicts with \"Master Key Filters Hypothesis\"\n  - Actually, I think `A, B` are necessary to make performance without re-training, even when \"Master Key Filters Hypothesis\" is true. But I wonder how the shift works. If the shift is a two-dimensional float vector that moves \"Master Keys\" with bilinear interpolation, it should be mentioned and clarified in the paper.\n- Limited applicable architectures\n  - \"Master Key\" is only validated on limited architectures that use depth-wise convolution with pre-Layer Norm blocks: ConvNeXt and HorNet. I doubt the finding can be generalized to CNN without pre-LayerNorm or depth-wise convolution.\n  - The paper's findings are only valid on depth-wise convolution-based architectures. Thus, it can't cover a wide range of architectures, such as ViT variants and CNN without depth-wise conv, which reduces the contribution.\n  - The paper's experiments only cover pre-LayerNorm architectures. It has to be verified in depth-wise conv without pre-LayerNorm: EfficientNet V2, and MobileNetV3.\n- Finding \"master keys\" requires evaluation on the test set\n  - The paper reduces the filters based on test accuracy, limiting insights into the filter shape required for performance. It would be better to provide an alternative way to reduce filters without an actual performance test.", "was_revised": true, "final_justification": "The rebuttal has partially addressed my concerns.\n\nThe paper still has the following weaknesses:\n- No practical benefits\n  - 8 filters are interesting, but computation costs and parameter reduction effects are negligible\n- Limited architectures\n  - The paper only covers pre-LayerNorm-based DS-CNNs: ConvNeXt, and HorNet. \n  - It can't be generalized to representative DS-CNN: EfficientNet series and MobileNet series\n- Finding “Master Keys” on the test set\n  - It is a significant issue for fair evaluation, which can't be resolved by small experiments in rebuttal comments.\n\nI adjust my rating to reject", "rebuttal": "We genuinely appreciate the time you have put to read and review our work.\n\n**Weaknesses:**\n\n---\n\n**1. Contribution overlaps / “limits the novelty”**\n\nThis is definitely not correct and actually surprised us. The paper starts by stating:  \n> “This paper extends this hypothesis by radically constraining its scope to a single set of just 8 universal filters.”\n\nConfining a hypothesis that could accept sets of hundreds of thousands of filters to an experimentally verified *8 filters* is undeniably novel. By the same logic, any paper building on the “lottery ticket hypothesis” or any other hypothesis would not be novel.\n\nWe appreciate that the reviewer acknowledged:  \n> “I acknowledge that implementing the master keys as eight filters is novel and makes a contribution.”  \n\nThank you for stating this.\n\nHowever, regarding:  \n> “But it is a bit insufficient for me.”  \n\nIf the implication is that after reading the previous two papers it was trivial to conclude that *all* filters could be replaced by just 8 without significant accuracy drop (Table 2), we must respectfully disagree. This was a surprising result for us and very exciting. Moreover, filters 1–4 were discovered *in this paper* and are not mentioned in prior work.\n\n---\n\n**2. Limited practical contribution**\n\nGreat practical values are a major potential of theoretical and analytical understanding, we shouldn't shut down analytical papers because they aren’t engineering papers. Despite this paper not being a practical paper, we showed our 8 filters have practical initialization values in Table 3 that they even beat ImageNet transfer learning on smaller datasets (isn’t that a surprise?). We hope the future “master Key” filters can show a much better practical value.\n\n---\n\n**3. Additional parameters A, B, shift**\n\nWe believe there is a misunderstanding about both our paper and the “Master Key Filters Hypothesis.”\n\n- First, we are not introducing additional parameters to represent our 8 filters. If you are referring to linear shifts (ax+b) in Table 1, we are showing that the filters of all those trained networks are closely converged to the linear shift of one of these 8 filters, out of the near infinite space they could converge to. And this is because DS-CNN filters are scale-free due to their architecture. And then, in Table 2, we are not using linear shifts. The master keys set we are introducing are these 8 filters.\n\n- Second, there should be a misunderstanding about the master keys hypothesis too, using parameters does not conflict with the master keys hypothesis. This hypothesis says that there exists sets (not necessarily one set) of master key filters (can be thousands of different filters) that are universal for any input data, in contrast to previous beliefs. To put it simply, it says the previous hypothesis of “specialized(data-specific) filters in deep layers” is not true.\n\n\nRegarding:  \n> “I think A, B are necessary”  \n\nIf by A, B you mean linear shifts, they are *not* necessary. Table 2 shows that a model with only 8 unique filters (no shifts) can achieve comparable accuracy.\n\nOn:  \n> “bilinear interpolation”  \n\nWe do *not* use interpolation. In Table 1, we simply apply linear shifts (aX + b). For example, (7, 7) is a shift of (3, 3) because (7, 7) = 2 × (3, 3) + 1. In Table 1 we are showing with an unsupervised experiment that filters of all models are closely converging to a linear shift of one of those 8 filters.\n\n---\n\n**4. Limited applicable architectures**\n\nThis is a DS-CNN analysis paper, but DS-CNN models are far from “limited.” Almost all top CNN-based models today are DS-CNNs. The last state-of-the-art classical CNN we recall is ResNeXt-101 from 2017, seven years ago.  \nIt is not a paper about ViTs, and this is *not* a limitation, it is focus.\n\n---\n\n**5. Finding “Master Keys” requires evaluation on the test set**\n\nTo address your concern, we’ll use a validation set from the training set to perform the search for the samples. The results are the same. Actually you can repeat the same experiment on any large test set you want, the results would be similar.\n\n---\n\n**Questions:**\n\n1. We hope the difference between our work and prior work is now clear. If not, we ask the reviewer to show:  \n   - (a) where in those papers it is demonstrated that trained filters converge close to a linear shift of one of our 8 filters;  \n   - (b) where it is shown that you can train a model with only 8 unique filters and achieve comparable accuracy;  \n   - (c) where 50% of our filters (filters 1–4) appear in those papers.\n\n2. We hope future works can find practical benefits, but right now finding the minimum set is more than just interesting to us, it’s scientifically important.\n\n3. Linear shifts are discussed in our previous discussion as (aX+b) where “a” and “b” both float scalars. We don’t learn them, we only calculate them in Table 1 with linear regression discussed in line 147.\n\n4. As we discussed, we don’t use linear shifts for training these 8 filters, the linear shift is only for Table 1.\n\n5. In both cases It’s the length of the vectors, here 7×7 = 49. But I totally get where the confusion comes from! In the next lines “n” is different and it’s the number of different vectors (here number of filters we have). It’s a mistake from our part and we have to fix the notation.  A genuine thank you for noticing this.\n\n6. **On “without pre-LayerNorm”**: Our intuitional answer would be yes. If you visualize the filter of all these models you’d see clear gaussian patterns no matter the task. There is no reason for us to believe this wouldn't work on non pre-LayerNorm when we can see the filters look the same. But can we show experimental results right now? No, is it because of the LayerNorm? No! It’s because of the filter size. The models you are introducing preliminary use 3×3 filters and our autoencoder method (Figure 2) just does not converge well on 3×3 filters and we couldn’t fix it although the gaussian filters are there. That’s why we have to go with 7×7 filters. If you know any non pre-LayerNorm 7×7 DS-CNN model we would be happy to get the test right now with the same 8 filters we extracted from ConvNext-base."}, {"review_id": "J5cXe4JKLf", "reviewer": "Reviewer_MAm3", "summary": "The paper presents an investigation on the diversity of learned layers in convolutional NNs with depthwise separable convolutions (DS-CNNs). The results of this investigation show that all the learned spatial convolutional kernels are the linear transformations of 8 basic layers. Two main experimental verifications are as follows: 1) All spatial kernels in a trained DS-CNN can be exchanged for the linear transformation of a closest kernel from this set with small accuracy drop; 2) All spatial kernels can be initialized with kernels from this set, leaving only the bias trainable, with small accuracy drop.", "questions": "Please see **action items** in “Weaknesses”. I will be willing to accept the paper if the first three are done. Moreover, I have some smaller questions:\n1. Did you think about using the discovered fact for purposes of quantization? How effective would such quantization be?\n2. Are you planning to release the code? It would be nice for everyone to be able to initialize/exchange existing layers in their models with the discovered 8 classes.", "limitations": "The authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n1. The paper presents and experimentally verifies a novel and fascinating idea. The experimental part is convincing.\n2. The process of searching for these filters is explained in detail.\n\n**Weaknesses**:\n\nThe paper is very insightful and captivating. My comments are mostly about the lack of detailed experimental description and suggestions for new experiments:\n1. The paper glosses over two important experimental details: 1) How large is the model bank which the pretrained kernels were taken from? 2) How to choose the correct kernel from these 8 for a specific layer in the model during initialization? **Action item**: please answer these questions in the paper.\n2. Also, after establishing the 8 classes of filters, a question about the importance and role of each filter naturally arises. It would be nice to see some statistics: which class is the most popular in the pretrained networks? How are they situated with respect to the layer depth? **Action item**: please compute these statistics.\n3. Moreover, it would be very interesting to compare the training loss dynamic with and without initialization, which was done in [1]. **Action item**: please report the graphs with training/validation loss dynamic.\n4. The authors consider only kernels of size 7 by 7. Arguably, the 3 by 3 kernels are also widely used – do they have the same properties? **Action item**: please confirm experimentally what would happen if 3 x 3 convolutions would be replaced by a closest from a similar set of 8 3 x 3 convolutions.\n\nAlso, some small problems with the manuscript:\n1. In my opinion, the “Related work” section is unnecessarily long. What is the connection between the current paper and scale-space theory? Moreover, I would assume that there is no need in writing formulae for depthwise convolutions, as it is a widely known concept. **Action item**: please reduce this section by removing irrelevant discussion.\n2. In lines 149-150 the authors propose a well-known solution to a linear regression problem, but there is no link or citation. **Action item**: please cite a book or another article with this formula.\n3. In line 197 the authors refer to the last row of Table 1, while they probably meant the row before last.\n\n[1] Xu, Chunyu, and Hong Wang. \"Research on a convolution kernel initialization method for speeding up the convergence of CNN.\" Applied Sciences 12.2 (2022): 633.", "was_revised": true, "final_justification": "During the rebuttal, the authors addressed W1, W2 and W4 from my initial review, and promised to add missing graphs from W3. Thus, I raise the “clarity” score to 4 and the rating to 4. Overall, I think that the achieved result is interesting, but I am not sure if it is significant enough for rating 5.", "rebuttal": "We sincerely thank you for the thorough review, especially for providing clear action points. This form of review is something we truly appreciate, and we were even inspired to adopt a similar style in our own future reviews. Below we provide our responses:\n\n---\n\n**Weaknesses:**\n\n**1.1** We gathered around 1 million filters in the bank from 54 models of different architectures, all with 7×7 depthwise filters, and with varying model sizes, dataset sizes (ImageNet-1K or 21K), and input image resolutions. We will add a summary to the main text and a comprehensive list to the appendix of the paper to cover this information.\n\n**1.2** Our observations of the proportions of filters in each model revealed that each model family has rather consistent proportions of each filter type in its trained filters, regardless of model size (more details are presented in the next action item). The proportions vary slightly between layers, but our experiments suggest that a uniform distribution per layer works similarly. We used the proportions derived from the trained models for initializing the networks.\n\n---\n\n**2.** Depending on the architecture, the most popular filter type differs. For instance, in ConvNeXtV2, the most frequent was the Gaussian filter (Type 8), whereas in HorNet, it was Type 1. We have prepared plots showing statistics in total and per layer for each model. Unfortunately, we cannot include these here due to rebuttal rules, but we will add them to the paper and appendix. Below is an illustrative table:\n\n| Model   | Type 1   | Type 2   | Type 3   | Type 4   | Type 5   | Type 6   | Type 7   | Type 8   |\n|---------|----------|----------|----------|----------|----------|----------|----------|----------|\n| ConvNeXtV2 B | 2.77%    | 3.71%    | 3.33%    | 3.55%    | 3.08%    | 6.34%    | 24.17%   | 53.05%   |\n| ConvNeXtV2 L | 2.99%    | 3.96%    | 3.89%    | 3.50%    | 3.55%    | 6.99%    | 22.47%   | 52.64%   |\n| HorNet T | 33.89%   | 13.97%   | 3.35%    | 4.64%    | 6.35%    | 5.54%    | 15.62%   | 16.64%   |\n| HorNet B| 28.57%   | 14.25%   | 8.34%    | 4.80%    | 6.72%    | 5.56%    | 14.66%   | 17.10%   |\n\nAs shown, ConvNeXtV2 has proportions similar within its family but distinct from the HorNet family. Despite the lower percentages for filter types 1–4 in ConvNeXtV2, they are essential for model accuracy. We performed experiments removing those four types, and the accuracy dropped noticeably.\n\n---\n\n**3.** Thank you for the suggestion. For the original models, we took trained weights and accuracies directly from the official repositories, so we did not initially have the training logs. We have now started training the original models on our own GPUs and have plotted the loss dynamics for training and validation for one model, both with and without frozen initialized filters. While we cannot provide figures and plots here, we will include them in the final paper.\n\n---\n\n**4.** DS-CNNs with 3×3 filters (such as MobileNet) also show similar patterns in their trained filters. Our autoencoder method (Figure 2) does not converge well on 3×3 filters, even though Gaussian patterns are visible. In this paper, we performed our search on 7×7 filters since we could gather a large enough bank of filters and our autoencoder converged to a reach code layer.\n\n---\n\n**Small:**\n\n**1.** Based on our experience, not all readers are familiar with DS-CNN architectures. Regarding scale-space theory, there is a fascinating link to our paper: the theory mathematically proves that convolution with Gaussian or derivative filters is the only operation that can scale an image. This classical computer vision theory proposed the Gaussian filters we found in an unsupervised manner, years before modern deep learning. We agree this related work section can be shortened, and we will do so to make room for the new suggested details and plots.\n\n**2.** We have added the missing citation. Thank you for pointing this out.\n\n**3.** We have fixed the noted issue. Thanks again.\n\n---\n\n**Questions:**\n\n**1.** We do not have prior experience with quantization, but the idea is interesting and our results may indeed be useful in this direction.\n\n**2.** Yes, all code will be released in a public repository. Also, the 8 filters are already provided in full in the appendix, and can be directly used for initialization or other experiments."}, {"review_id": "1fVES3lKUN", "reviewer": "Reviewer_tTfe", "summary": "The paper describe a method to identify master key filters from separable CNNs, and claims that these filters mimics the receptive field of in mammalian visual systems. Furthermore, the authors show that the separable CNNs with frozen master key filters (with learnable bias) are able to achieve comparable performance as the fully trainable counterpart.", "questions": "**Writing**: I would strongly recommend the authors to enhance the technical precision of the writing.\n  - The paper only studies depth-wise CNN, but keep writing depth-wise separable CNNs (DS-CNNs). For a 7x7 filter, a depth-wise filter has 49 parameters, but a depth-wise separate one has only 14 parameters.\n  - From Line 114-117, I don't understand why centering is an issue as the filter is flattened into a vector anyway. Also, the authors write 'scaling their length to 1', but the readers would naturally think it is the filter length (7).\n - The network has 4 encoder/decoders layers, but Figure 2 only has one.\n - In Line 140, the matrix F is never defined before and never used later.\n - In Line 142, the authors write 'a linear approximation with respect to the decoded filters' --- the readers would think it is the linear combination of decoded filters.\n - Equation (1)-(3) are very standard linear regression --- there is no need to distract the audience with details.\n - The greedy algorithm is only mentioned by words, and not in maths (or pseudo code).\n - The whole Section 3.1 is three page long without subsection or delimiters.\n\n**Method**:\n  - The auto encoder-decoder model is not the most up-to-date method for unsupervised clustering.\n  - I wonder if there is any specific reason why a standard VAE (or a more advanced categorical VAE) is not used in this setting.\n  - Also, the proposed method naturally applies to standard CNN instead of depth-wise CNN --- does it mean the phenomenon can only be observed in depth-wise but not a standard one?\n\n**Experiments**:\n  - From my understanding, the majority of the parameters remain learnable in the experiment setup. For example, if the number of channels if 1024, the number of parameters in the depth-wise filter is only 49*1024~=50K, while the number of parameters in the fully-connected part is 1024x1024=1M --- so the parameters in the depth-wise filter is negligible.\n  - I hypothesize that if you initialize the depth-wise filters randomly and remain unchanged during learning, the model is able to achieve similar results as the fully-learnable one.\n  - Also, I would recommend adding experiments compressing a standard CNN in a similar manner - if the compressed CNN achieve similar result, that will make the hypothesis in the paper more convincing.", "limitations": "See above.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The idea that the filters in separable CNNs mimics receptive fields in mammalian visual systems is very intriguing, which also challenges the assumptions that the filters for later layers extract are more complicated than earlier ones.\n2. The flow of the paper is smooth --- from hypothesis to investigation; from results to analyzing; from observations to experiments.\n\nWeakness:\n1. The writing overall is not very clear, especially when technical precision is required (see Questions section for details). But I would like to point out an example that annoys me most here --- the paper heavily uses the term 'linear shift' without a precise description. When I first read it, I thought it means geometric shift instead of amplitude shift. Also, such mapping should be called 'affine' instead of 'linear'.\n2. The methods used in the paper (both for analysis and experiments) are not the most solid ones in my opinion. For example, a variational encoder is more suitable for the scenario, but the authors choose to use a vanilla encoder-decoder.", "was_revised": true, "final_justification": "The rebuttal addresses part but not all of the concerns --- so I am willing to increase the score to 3.", "rebuttal": "We thank you for your time and comments. We have carefully considered each point raised and have done our best to address them in detail below. Thank you for helping us improve the paper.\n\n---\n\n**Weaknesses:**\n\n1.**“'linear shift' without a precise description”**: We tried to define linear shifts as soon as line 6 of the abstract as a simple (ax + b) operation where “a” and “b” are scalars. For example (7, 7) = 2 × (3, 3) + 1. We mentioned this once again in line 264 “mathematically expressed as a(x + b).”. We would clarify this more by writing an example in the paper. This is a very simple operation and linear shift was the best term we could come up with. If you have any suggestion about using a better term we would appreciate your input. Affine transformation requires matrix multiplication to move a geometric shape, but here “a” is only a scalar. \n\n2.VAEs are not suitable for this task. VAEs are used as a probabilistic generative model to produce new samples and generate interpolations between samples, but here we strongly do not want any new sample. And we strongly need to keep the code dimension to be 1, because then we can select limited samples (in this paper 50). But 1D latent space is much more restrictive for VAE and doesn’t make sense. We’ve spent months and tried many different models to finally come to auto encoders, I think you are underestimating the power of AE in this task to just call them “vanilla”. By the way, we would be happy for sure if we can find a better model/method to do this task. If there is any other method that is not the most solid we are happy to hear.\n\n\n---\n\n**Writing:**\n\n- We believe the reviewer is not familiar with the literature of DS-CNNs. We are indeed studying DS-CNNs models [1], We call the convolution part of DS-CNNs “Depthwise Conv” and when it is combined with the next fully connected layer we call the whole model Depthwise Separable Convolution model. And No! 7×7 filters indeed have 49 parameters, not 14. Working on this topic for long, we really don’t know which reference you are pointing to here (49 parameter depth-wise CNN) vs (14 parameter DS-CNN). Could you please cite a model that is DS-CNNs and has 14 parameter for its 7×7 filter?\n\n- Thanks for helping us improve the paper. We would change “length” to “norm” to not make this confusion. In our experiments center-norming helped the autoencoder to encode better.\n\n- Figure 2 is just for visualization purposes not the actual model. Our model also inputs 7×7=49 parameters but we showed a 3×3=9 input, because the other one was just too big to show.\n\n- Matrix F is actually  defined in line 140, and it is used later, in line 159. Matrix F serves as a compact representation of all the depthwise filters in a layer.\n\n- “In Line 142” You are totally right in this matter this is a valid confusion that might accrue. We have to rewrite that section carefully to clarify this. Thank you for pointing that out.\n\n- We wrote in 147 that “we use linear regression” and in 149 “ This problem has a well-known solution.” But we needed to put the formula to make the other steps clear to the reader, as we say in 151 “Calculating Equations (1) can be computationally intensive” then we follow by our own method to reduce the computation by normalising the candidate filters first. We don’t think this is trivial for all readers.\n\n- “greedy algorithm” At first we considered this as trivial, but now we would add pseudo code to the appendix to improve the clarity.\n\n- “ three pages long without subsection”: thank you for pointing this out. We would try our best to make that long section more clear.\n---\n\n**Method:**  \n- We have addressed this above.  \n\n- “Does it mean the phenomenon can only be observed in depth-wise but not a standard one?” This is an excellent question that we can write a lot about. There is evidence that classical CNNs are converging towards some “Master Key filters” too, but the evidence for DS-CNN filters is much stronger. On the other hand, unlike classical CNNs, DS-CNN filters have been shown to follow gaussian patterns, and that was our motivation for this paper to see how much can we shrink this pool? Although maybe “Master Key Hypothesis” is true for classical CNNs, we don’t think we ever can find a set as small as 8 filters for them.\n\n---\n\n**Experiments:**  \n- This is completely true and valid, and we did not claim we do parameter reduction in this paper. This is an analytical paper for scientific understanding.\n\n- This hypothesis is wrong. First, we already put a random experiment in Table 1, in which we showed that the trained filters are converging to a linear shift of one of our 8 filters out of nearly infinite possible filters in the space. Second, for you, we conducted the experiment of Table 2 with 8 random filters, and after 80 epochs we stopped the experiment due to low performance. The accuracy was 23.7 at epoch 80. For your reference, our accuracy was 78 at this epoch.\n\n\n- We doubt similar results can be achieved for classical CNNs, but it is worth trying, and we will explore this.\n\n---\n[1] [Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1251–1258. https://doi.org/10.1109/CVPR.2017.195]"}, {"review_id": "0vjX0Bib9b", "reviewer": "Reviewer_kyfb", "summary": "This work focuses on the Master Key Filters Hypothesis for Depthwise Separable Convolutional Neural Networks (DS-CNNs). It improves on previous works by showing that the trained filters can be replaced by affine transformations of a small set of base filters, with little harm to the model accuracies. These filters are found by training an autoencoder on a collection of existing trained models' filters and then drawing samples with it. This work also draws theoretical insights by comparing these base filters with Gaussian filters and shows they coincide. Extensive numerical experiments are done on ImageNet, CIFAR-10, and a few others using different sizes of ConvNeXt models. Through these experiments, this work freezes the filters to be the few base filters it has chosen and arrives at a comparable accuracy after training for certain epochs.", "questions": "1. What is $n$ in Eq (1)?\n2. The authors find the affine transform of a single base filter for each trained filter. What is different if one finds a linear combination of all base filters for each trained filter? Since the authors use only a small set of base filters, I think the number of coefficients is still acceptable.\n3. From a perspective different from saving memory, can one facilitate training by first freezing the 8 base filters for a few epochs and then allowing them to be trained for a few epochs? It seems from Table 3 that frozen filters give faster convergence in 300 epochs, compared to fully trainable filters.\n4. Is there a connection to overparametrizations? See Strengths.\n5. Is there also a connection to the recent research on the implicit low-rank bias of deep CNNs [1,2]? This line of research essentially shows, when trained with regularization, deep CNNs tend to have most of their filters aligned with the pooling (up to linear transformation) so the information kept by the trainable filters is also kept by pooling. In this case, most filters are linear transformations of each other. In particular, it raises the question if the base filters found in this work happen to allow most information to pass under any (or the popular) poolings.\n\n*I would appreciate any answers the authors have to these questions.\n\nReferences:\n\n[1] Súkeník, Peter, Marco Mondelli, and Christoph Lampert. \"Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal?.\" arXiv preprint arXiv:2405.14468 (2024).\n\n[2] Wen, Yuxiao, and Arthur Jacot. \"Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature Learning.\" International Conference on Machine Learning. PMLR, 2024.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The results presented are inspiring and practically beneficial.\n- The experiments are done extensively on a number of datasets.\n- The paper is well written.\n- The accuracy loss incurred by filter pruning (in Table 1) is less when the model is larger, which is good since large models are the motivation for pruning. In particular, this trend seems to have a connection with overparametrization and the NTK results e.g. [1,2], in which case large MLPs find good parameters close to their initializations. The trend in Table 1 may imply the larger the model is, the less it needs to diverge from the base filters. This may be an interesting connection to make, if I am not mistaken here.\n\nWeaknesses:\n- No statistical confidence is given. The authors argue that running experiments on ImageNets is costly. But is it possible to run and give a confidence on the smaller datasets used in Section 4.2?\n- Apart from ImageNet, the other datasets are relatively easy tasks. It remains a question whether such a nice concentration on the filters still holds under more complicated visual tasks.\n\nTypos:\n- Line 173, \"a small set\".\n- Line 226, \"Figure 4\".\n- Line 236, \"This filter likely contributes\".\n- Line 244-245, missing $\\sigma$ as subscript in defining gaussians (for $G_1$, $G_2$ to make sense in DoG).\n- Table 3, \"normal training\"\n\nReferences:\n\n[1] Jacot, Arthur, Franck Gabriel, and Clément Hongler. \"Neural tangent kernel: Convergence and generalization in neural networks.\" Advances in neural information processing systems 31 (2018).\n\n[2] Allen-Zhu, Zeyuan, Yuanzhi Li, and Zhao Song. \"A convergence theory for deep learning via over-parameterization.\" International conference on machine learning. PMLR, 2019.", "was_revised": true, "final_justification": "The authors have addressed my questions to a satisfactory extent. I believe there is valuable insight in the proposed \"basis\" filters from a theoretical perspective, despite that its practical benefits are not yet clear. I increase my score to Accept conditioned on that the authors include/fix what has been discussed during the rebuttal. \n\nHowever, I will not be able to argue against other reviewers' more negative reviews, because there are grounds for rejection: this work indeed does not have a well-rounded set of experiments to support its more theoretical findings. The boxplot I mentioned during rebuttal is one example, but may not be exhaustive.", "rebuttal": "We want to thank the reviewer for their review and for accepting our paper. We have to mention that although our paper can be viewed as a kind of pruning, ours is more of an analytical paper and we are more focused on understanding the DS-CNN filters in this paper regardless of what the application can be.\n\n---\n1. Weaknesses:\n- **“confidence on the smaller datasets”**: For sure, For Flowers and Pets datasets we repeated the experiments 4 times to serve as a confidence, but we can complete them even more with confidence intervals. We repeated the CFAR10 for you and our results seem to be consistent. Each experiment was repeated 4 times, and we now report the results with error bars (mean ± standard deviation). The experiments are on-going that we will complete and update that table. Please find the confidence table below:\n| Dataset   | Setting                      | Accuracy            |\n|----------|------------------------------|--------------------:|\n| CIFAR-10 | Original                      | 96.95 ±0.10         |\n| CIFAR-10 | ImageNet Filters              | 97.07 ±0.18         |\n| CIFAR-10 | Acc with 8 unique Filters     | 96.36 ±0.13         |\n\n- **“filters still hold under more complicated visual tasks”:** This is a valid argument! We definitely should try our filter on other tasks. We would try the COCO object detection task but it takes time. Is there any task or dataset that you are interested in?\n\n---\n2. Typos:\n- Thank you very much. We would correct them.\n---\n3. Questions:\n- In both Eq (1) and Eq (2) It’s the length of the vectors, here 7×7 = 49. But I totally get where your confusion comes from! In the next lines “n” is different and it’s the number of different vectors (here number of filters we have). It’s a mistake on our part and we have to fix the notation. A genuine thank you for noticing this.\n- **“What is different if one finds a linear combination of all base filters for each trained filter?”:** First, we are not sure if we should use “affine transform” here because most of the time affine transform means matrix multiplication transform (Are we wrong?) but here we are just using a linear shift (ax+b) where “a” and “b” are just scalars. Second, about linear combinations, we should remember how powerful linear combinations are. To give you a simple example, linear combination of just 9 random 3×3 filters would span the space of all possible 3×3 filters because 3×3 filters have 9 dimensions. But that is not the case for linear shifts. So mathematically, if we show the filters are converging to linear shifts, we are finding a far greater constraint to the space of filters compared to if we’ve used linear combination. \\\n**TLDR:** Using linear combinations would definitely increase the accuracy, but the scientific importance of it is very low compared to linear shifts.\n- You are completely right. Actually we are adding a loss comparison chart that would show that. For fast convergence, we recommend initializing your DS-CNN models with these filters; they do make a real difference for the starting epochs.\n- Sorry, we thought about it a lot, our guess would say “Yes”; however, we couldn’t come up with a scientific experiment or explanation to answer this question the professional way.\n- Oh! Those papers look super interesting! Especially “Which Frequencies Do CNNs Need?” Indeed there could be a connection because each DS-CNN is mathematically a classical CNN. We have to dive into that to figure out the connection. About the “base filters” you mentioned, I wish I could send pictures in this rebuttal, but we suspect they are bases for natural images, because we could construct them by finding the eigenvectors of the covariance matrix of any natural image we tested. It would be hard to explain without figures unfortunately. But anyways, thanks for the papers."}]

Author rebuttal:

Meta-review: This paper makes the compelling claim that the vast diversity of trained filters in modern Depthwise-Separable CNNs (DS-CNNs) are, in fact, predominantly simple affine transformations (ax+b) of a foundational set of just eight universal "master key" filters. The authors substantiate this by demonstrating that networks initialized with only these eight unique, frozen spatial filters can achieve competitive performance (e.g., >80% on ImageNet). This represents a significant sharpening of the recent Master Key Hypothesis to a set of only eight filters, and its experimental validation across multiple architectures and datasets. The discovery that these filters intrinsically match fundamental image processing operators (DoGs, Gaussians) provides an intriguing challenge to popular narratives around the importance of feature learning (of course, only as far as the _spatial_ filters are concerned!)

The reviewers expressed some concern regarding the relative novelty of the present work compared to previous work on the master key filter hypothesis, and the restriction to pre-LayerNorm architectures with 7x7 filters. However, given previous universality results on spatial filters in convolutional neural networks (see e.g. Guth & Ménard, 2024) I do not think that these limitations take away from the fundamental insight that this work provides. Reviewer MKLF raised a valid concern about finding the filters on the test set; during the rebuttal period, the authors reported additional experiments that assuaged this concern. During rebuttals, reviewers ​​MAm3 and kyfb, and to a lesser extent, tTfe, were convinced by the rebuttals and increased their score. 

I therefore recommend acceptance, while strongly encouraging the authors to take the feedback from the reviewers into account in preparing the final version.
Decision: Accept (poster)