Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels
Authors: Zahra Babaiee
Peyman Kiasari
Daniela Rus
Radu Grosu
Venue: ICLR 2024 poster (2024)
Keywords: Depthwise Convolutions
Explainability
Neuroscience
Computer Vision
ConvNext
Primary area: visualization or interpretation of learned representations
TL;DR: This paper reveals that discernible patterns resembling biological vision models consistently emerge in the trained kernels of depthwise convolutional neural networks.

Abstract: Recent advances in depthwise-separable convolutional neural networks (DS-CNNs) have led to novel architectures, that surpass the performance of classical CNNs, by a considerable scalability and accuracy margin. This paper reveals another striking property of DS-CNN architectures: discernible and explainable patterns emerge in their trained depthwise convolutional kernels in all layers. Through an extensive analysis of millions of trained filters, with different sizes and from various models, we employed unsupervised clustering with autoencoders, to categorize these filters. Astonishingly, the patterns converged into a few main clusters, each resembling the difference of Gaussian (DoG) functions, and their first and second-order derivatives. Notably, we classify over 95\% and 90\% of the filters from state-of-the-art ConvNeXtV2 and ConvNeXt models, respectively. This finding is not merely a technological curiosity; it echoes the foundational models neuroscientists have long proposed for the vision systems of mammals. Our results thus deepen our understanding of the emergent properties of trained DS-CNNs and provide a bridge between artificial and biological visual processing systems. More broadly, they pave the way for more interpretable and biologically-inspired neural network designs in the future.

Published as a conference paper at ICLR 2024
UNVEILING THE UNSEEN: IDENTIHIABLE CLUSTERS IN TRAINED DEPTHWISE CONVOLUTIONAL KERNELS
Zahra Babaiee TU Vienna & MIT
Peyman M. Kiasari TU Vienna pe peyman. k asart Bhow an....s
Daniela Rus MIT
Radu Grosu
TU Vienna radu aadu-grosu@uud an ac-att
zbabo teggitt.edu
ru5gmi eddo
ABSTRACT
Recent advances in depthwise-separable convolutional neural networks (DS- CNNs) have led to novel architectures, that surpass the performance of classi- cal CNNs, by considerable scalability and accuracy margin. This paper re- veals another striking property of DS-CNN architectures: discernible and explain- able patterns emerge in their trained depthwise convolutional kernels in all layers. Through extensive analysis of millions of trained filters, with different sizes and from various models, we employed unsupervised clustering with autoencoders, to categorize these filters. Astonishingly, the patterns converged into few main clusters, each resembling the difference of Gaussian (DoG) functions, and their first and second-order derivatives. Notably, were able classify over 95% and 90% of the filters from state-of-the-art ConvNextV2 and ConvNeXt models. respectively. This finding is not merely technological curiosity: it echoes the foundational models neuroscientists have long proposed for the vision systems of mammals. Our results thus deepen our understanding of the emergent properties of trained DS-CNNs and provide a bridge between artificial and biological visual processing systems. More broadly, they pave the way for more interpretable and biologically-inspired neural network designs in the future.
INTRODUCTION
Convolutional neural networks (CNNs) have been extensively studied in order to understand how they learn ever since their inception. As early as the seminal papers on CNNs, researchers discovered that CNNs learn filters in their initial layers that detect edges or specific edge colors when applied to natural images (Krizhevsky et al. 2012). These learned features bear a strong resemblance to Gabor filters, which are related to responses in the primary visual cortex, and thus present some of the most striking links between neuroscience and machine learning (Goodfellow et al., 2016). However, examines deeper layers. Consequently. subsequent
interpretability significantly decreases work has mainly focused on inspecting the features learned by convolutional layers, rather than the weights themselves, elucidate how CNNs operate (Zeiler & Fergus| 2014; Yosinski et al., 2015) Olah et al., 20177] Bau et al. 2017). While studying the learned features by convolutional layers is intuitive, comprehending the filter weights of deep layers of CNNs remains an open question.
In recent years, depthwise-separable convolutional neural networks (DS-CNNs) have become widely adopted in computer vision. The significantly lower computational costs of DS-CNNs en- abled MobileNets (Howard et al., 2017) to achieve state-of-the-a accuracy, with substantially fewer parameters and multiplication- addition operations, than traditional CNNs. Following this break- through, DS-CNNs have become the de facto standard in most modern CNN architectures, they offer a favorable option for scaling up networks (Liu et al., (2022). However, most studies analyzing and interpreting the learned kernels and feature maps of CNNs, remained confined to the traditional architectures, using regular convolutions (Zhou et al. 2018 Bau et al. 2017). In contrast, the emer- gent properties and associated interpretability of DS-CNN CNNS nowadays largely under-explored.
iullippiiatiionzdddiionn
Through an extensive investigation of several model types and sizes, including regular CNNs and DS-CNNs, trained on ImageNet-1k 1k and ImageNet-2Ik, discovered striking property of DS- CNN kernels. Unlike regular convolutions, depthwise convolutions exhibit repeating patterns in their trained kernel weights. Moreover, these patterns persist throughout all layers, in deeper

---

Published as a conference paper at ICLR 2024
(a) (oG(z.
(b)-DoG(z,))
dDoG(s.v)
d (DDGGry
Figure 1: 3D and 2D plots of the DoG function and its derivatives. utilized in Neuroscience for modeling visual receptive fields and in Image Processing for edge and brightness change detection.
stages of the DS-CNNs. This reveals a new level of structure and interpretability in the emergent representations learned by DS-CNNs that has not been identified and characterized before.
Furthermore, we discovered that these identifiable patterns are highly clusterable. To this end, we employed unsupervised autoencoder-basee based clustering methodology, mapping each kernel sin- gle hidden dimension. In the autoencoder outputs, clear clusters emerged, enabling us to categorize nearly all kernels across all layers into eight main classes. Through this classification, ered an intriguing property: the prominent clusters have spatial patterns resembling the difference of Gaussian (DoG) functions and their first and second-order derivatives, shown in Figure !
uncov-
Fascinatingly, DoG derivatives have been extensively proposed in neuroscience to model biological visual receptive fields (Young) 1987 Young al. 2001). Incorporating fixed-weight DoG kernels alongside traditional convolutional kernels has shown enhancements in network performance, par- ticularly under conditions of changing lighting and the presence of noise Babaice al. (202I). This suggests a profound link between the representations learned by DS DS-CNNs and those employed in mammalian visual systems. Moreover, the identifiable patterns emerging in DS-CNN kernels have not been previously characterized for modern DS-CNN architectures. Thus, our findings may inform the development of novel bio-inspired network designs and training methodologies. More broadly, this research helps narrow the gap between contemporary machine learning and neuroscience mod- els of sensory processing.
In summary, the key contributions of our work
We conduct the first large-scale analysis of structures emerging in trained DS-CNN kernels.
We develop unsupervised clustering of millions of DS-CNN filters into core underlying patterns.
We demonstrate that these discemible patterns are present all layers of the DS-CNNs.
We show that the patterns strikingly similar to the neuroscientific DoG-derivatives models.
Our findings thus reveal new interpretability and biological parallels of DS-CNNs, respectively.
2 RELATED WORK
Our survey covers depthwise separable convolutions (DSC), kernel analysis, and biological vision.
Depthwise Separable Convolutions. DSCs decouple the spatial and channel-wise computation in two steps: depthwise convolutions (DC) and pointwise convolutions. This decoupling significantly reduces the computational demands without compromising model efficacy. MobileNet popularized the use of DS-CNNs for efficient, high-performance architectures in resource-limited environments (Howard et al., 2017). In its wake, DS-CNNs have emerged as the cornerstone for a plethora of CNN architectures, including EfficientNet (Tan & Le 2019a). MobileNetv2 (Howard et al. 2017), MobileNetv3 (Howard et al. 2019), MixNet (Tan & Le 2019b), and MNasNet (Tan et al. 2019).
Post-Vit CNNs: A Resurgence. Leveraging vision transformers' approach (Dosovitskiy et al. 2020). recent studies enhance CNNs with transformer-style and large kernel convolutions. DSCs add efficiency, similar to transformers, for better scalability. ConvNeXt (Liu et al. 2022 exten- sively analyzes recent vision transformers and presents highly performant pure convolutional model using 7x7 DCs. Further modernized CNNs have since emerged, including ConvMixers
2

---

Published as a conference paper at ICLR 2024
(a) VGG
(b) ResNet
(c) DenseNet
(a) MobileNetV3 (b) EfficientNet
(c) ConvNextV2
(i) Samples of Regular CNNs
(ii) Samples Depthwise CNNs
Figure 2: Random samples of regular (i) and depthwise (ii) convolutional kereels across all layers. Depthwise convolutions show structured patterns while regular convolutions appear uninterpretable.
which utilize patching and isotropic depthwise blocks (Trockman & Kolter 2022). Hornets with recursive gated convolutions for high-order spatial interactions (Rao al.] 2022), and MogaNets using dilated DCs (Li et al. 2022). Most recently, ConvNextV2 introduced a fully convolutional masked autoencoder with global-response normalization to enhance inter-channel competition (Woo et al. 2023). Together, these innovations demonstrate that DSCs are a key enabler for scalable, transformer-isppired CNN architectures, which are achieving sate-of-the-ars results.
Studies Trained Convolutional Kernels. Studies trained convolutional kernels have sought to elucidate the learned representations of deep-vision systems. However, most prior work was focused visualizing kernels in initial layers, where Gabor-like and edge-detecting filters emerge (Krizhevsky et al. 2012). Analyzing deeper layers proved far more challenging, as dis- cernible patterns in kernel weights became increasingly opaque. Thus, many studies centered on visualizing and analyzing learned feature maps rather than the kernels themselves (Olah et al.. 2020) Voss al. 2021). (Gavrikov Keuper 2022) assessed distribution shifts between filters across axes like dataset, task, and architecture. Our previous work investigates the presence of on/off- center DoG filters as the two most prominent clusters in depthwise convolutional filters [abaiee et al. (2024). However, the exploration does not extend to other potential clusters in these filters. In this work, we are extending our study covering up to 90-95% of filters clustered, and discovering first and second order derivatives of DoGs in these clusters. Our work thereby spotlights DCs as an avenue to unpacking the black box of trained convolutional networks.
Biological Models of Vision. The neuroscientific investigations have led to mathematical mod- els capturing the response properties biological vision systems. (Young) 1987) proposed the Gaussian derivative model, where retinal ganglion and cortical simple cells act linear filters, well- approximated by Gaussian derivatives. This aligns with SnreerenceeofGGausiin
(DoG) models of retinal ganglion receptive fields (Rodieck] 1965). Furthermore, Gabor filters effectively characterize simple cell tuning in the primary visual cortex (V1), but their mathematical formulation requires complex functions (Jones & Palmer 1987). The same tuning, however, can be achieved with even greater accuracy, by biologically plausible DoG derivatives (Lindeberg) 2021 Tomen et al.) 2021). These seminal neuroscience works thus established DoG filters and their directional derivatives, Our findings align with established models of low-level visual processing in mammals, showing parallels between patterns in trained DC kernels and biological vision's receptive field.
ANALYSIS OF TRAINED KERNELS
We conducted an extensive empirical analysis compare the patterns emerging trained convo- lutional kernels of both regular and DCs, respectively, across diverse state-of-the-art CNN archi- tectures. Our goal was to discern any interpretable patterns these learned representations. We gathered pre-trained models on ImageNet for the following architectures: AlexNet. VGG, ResNet, DenseNet, MobileNetV2, MobileNetV3, EfficientNet, EfficientNetV2. MixNet, MNasNet, Con- vNeXtV1, ConvNextV2, ConvMixer, HorNet, and MogaNet. These architectures different ker- nel sizes, and for each model, we used several model sizes and configurations available.
The DS-CNN- analyzed all begin with eegular-convolutional layer, followed DSC layers. This first layer (also called patching layer), uses kernel size equal to the stride, to divide the input into patches in the new generation of DS-CNNs, similar VIT (Dosovitskiy et al. 2020). As observed
3

---

Published as a conference paper at ICLR 2024
Off-Center dx=
Off Off-Centre
Off-Centre Cross
Off-Centre
Off-Centre dy
On- On-Center dr²
On-Centre
On-Centre Cross
On-Centre dx
On-Centre dy
Figure 3: Reconstructed spectrum of 7x7 kernel filters from ID hidden code of autoencoder model, trained on more than million filters. Each segment of the spectrum is marked with the corresoonding cluster label. (Sec Figure 16 for the architecture of the autoencoder.)
in (Krizhevsky al. 2012), the patching layer learns Gabor-like like filters. We therefore omitted it from our analysis and focused on the subsequent DC layers, which are far less studied, up to the deepest stages. This allows us to rigorously characterize the patterns emerging in trained DC kernels across a wide range of layers, from early features in the first layers to late abstractions in the last layers.
To investigate the patterns trained kernels, we first visually inspected the filters across layers for each model. As shown in Figure ii). the regular-convolutin structure across models and layers. This aligns with prior analysis that found minimal interpretabil- ity beyond early layers in regular CNNs. In contrast, the DC kernels shown in Figure 2ii), exhibit clear visual patterns that are consistent across diverse model sizes and layers. These patterns are con- sistent with DoG derivatives shown in Figure I The patterns persist even in the deepest layers, indicating that interpretable representations emerge throughout the full network. Moreover, filters of different kernel sizes converge similar patterns, suggesting common underlying structure.
filters appear devoid of any discernible
These visualizations reveal a stark difference in the emergent patterns of regular versus DC rep- resentations. The recurring interpretable pat- terns in DCs explored next through quan- titative analysis. We focus on the 7x7 ker- nels from ConvNeXt. which were vectorized and centered before visualization, apply prin- cipal component analysis (PCA) on the kernels across layers. We derive the first PCs explain- ing the most variance, and we project each ker- nel onto these 3 components. We then visualize the embeddings in a 3D scatter plot. As shown in Figure 4 the PCA projection reveals distinct clusters forming in the ConvNeXt kernel em- bedding space. This indicates recurrent iden- tifiable patterns emerging in the learned repre- sentations. The recurring filter patterns motivate categorization and further investigation to decode their meaning, as done in the next section.
Figure 4: Scatter plot PCA applied on 7x7 Con- vNeXt filters. Sample filters from the 4 most clear clus- ters are visualized on the side of the plot.
4 CLUSTER IDENTIFICATION OF TRAINED KERNELS
4.1 MODEL AND METHOO
To earry out comprehensive classification of the trained kernels, we developed an unsupervised clustering method using autoencoders. The primary objective was to project the kernels onto compact hidden dimensional space and subsequently execute clustering within this dimensionally reduced space. Distinct models were trained for each kernel size of 5 5x5 and 7x7. For every distinct kernel size, kernels learned from diverse models, and seales exhibiting the corresponding size were assembled. The compilation comprised over million kernels for each size category. For extensive enumeration the models used, please refer to the Appendix.
The autoencoder consists of two main components: encoder and a decoder. The encoder has four intermediate layers, each followed by a leaky rectified linear unit (Leaky ReLU) activation. The code layer employs sigmoid activation, to map filters values within [0.1]. The final decoder
4

---

Published as a conference paper at ICLR 2024
(a) On-Centre
(c) On-Centre Cross
(e) On- Centre de
(g) On-Centre dy
(b) Off-Centre
(d) Off-Centre Cross
(f) Off-Conniie Centre dx
(h) Off-Centre dy
Figure 5: Random samples from each of prominent classes 7x7 kernels ConvNextV2-tiny trained on ImageNet. Our method classifies the samples with notable accuracy.
layer uses tanh activation, to accurately reconstruct the original normalized filters in [-1,1]. We utilize a mean-centered cosine similarity loss to accommodate the invariance of filter patterns to linear transformations. Although higher code dimensions yield lower reconstruction loss, we opt for a ID code to enable convenient visualization and labeling of distinet class intervals for employing the clustering classification. While the reconstruction quality is slightly reduced, the interpretability of the clusters is substantially enhanced with this compromise.
initiated with the centering of the filters and the normalization of their lengths to unity, ensuring alignment all filters on a central hyperplane, represented as 1"r O. Given the uniform alignment of the normalized filters reduction was possible by transforming the basis of the space to the central hyperplane
Data preprocessing
singular hyperplane, dimensionality
4.2 INFERENCE STAGE
After training the autoencoder, the next step is to identify and label the clusters that emerge, corre- sponding different code intervals. To this end, uniformly sample 500 codes from the 1D space [0,1], with equal spacing, and pass each code through the trained decoder. This generates a spec- trum of reconstructed kernels, representing the space of clusters. By visualizing this reconstruction spectrum, we can discern distinct intervals that correspond to different structural patterns.
We manually assign labels the most prominent clusters, based their visual patterns.
we illustrate the reconstruction spectrum for 7x7 ConvNeXt-V2 DC kernels, with labeled intervals corresponding to the 10 most prevalent clusters. The emerging patterns span diverse range of patterns, as found in the DC kernels. Strikingly, the patterns of DoG functions and their derivatives are clearly discernible, as shown Figure
In Figure 3
Based on the visual motifs detected, we assign semantic labels to each cluster interval. The DoG function DoG(x.y) appears as an On-Centre pattern. Similarly, we label its inverted version -DoG(x, y) as Off-Centre. Following this nomenclature, we identify the Off-Centre and On-Centre first and second-order derivatives of these DoG functions, respectively, based on their spatial pat- terns. Interestingly, another common pattern discovered is the shape-of-a-crope pattern, occurring with both on- and off-centers. We refer to them as Off-Centre Cross and On-Centre Cross patterns.
In total, we consistently converge just 10 core structural patterns spanning DoG functions, their first and second-order derivatives, respectively, and cross-like 2DDaabsolut--sinc-lle centers and offsets. Next, study the prevalence and properties of these kernels. Remarkably, through this unsupervised clustering, we are able to categorize millions of heterogeneous depthwise convolution kernels into just small set of identifiable recurring patterns that arise during training.

---

Published as a conference paper at ICLR 2024
(a) On-Centre
(c) On-Centre Cross
( On-CCetre de
(g) On-Centre dy
(b) Off-Centre
(d) Off-Centre Cross
(f) Off- Centre dz
(h) Off-Centre dy
Figure 6: Random samples from each of the prominent classes 5x5 kernels of EfficientNet-be trained on ImageNet.
To analyze the learned convolutional filters, perform inference by sampling 10,000 scalar codes uniformly from [0, 1] and passing them through the decoder. This reconstruction from the latent space helps mitigate potential inconsistencies from the encoder. We then compare each recon- structed filter to its original via cosine dissimilarity, taking the minimum value. If this minimum dissimilarity is below chosen threshold, we assign the filter that scalar code's cluster. This clustering based on minimal reconstruction loss allows precise classification of similar filters.
The threshold value a key parameter controlling cluster assignment and enables reliable clustering. We select threshold of 0.3 for 7x7 kernels and 0)2 for 5x5 kernels. Using stricter threshold for 5x5 kernels improves robustness because lower dimensional spaces tend to have closer vectors angularly. Table 1: Percentages of filters in each model that we could classify with high accuracy by using our nutoencoder-basee clustering method, alongside the model size and model accuracy ImageNet.
Model
Parameters
Model Accuracy
Filters Clustered
ConvNexiV2_tiny 22k
28 M
83.9%
98.16%
VonvNextV2 tiny
28 M
83.0%
97.33%
ConvNeXt tiny
28 M
82.1 %
95.21 %
HorNet_tiny
22 M
82.8%
80.71%
MogaNet_smmll
25 M
83.4%
78,8 78.87%
ConvMixer_768332
21 M
80.1%
56.64%
ConvNextV2 www.ee/ehhhheel_e1 224
660 M
86.3%
82.08%
ConvNexxV22hhnge_222. 22g
660
88.7%
92.41%
ConnNextV2.arree_Ikk 1222
198 M
84.3%
90.82%
CoavNexxV22largee222...
198 M
86.6%
96.63%
In Table I we show the proportion filters that we found, from each model, all having 7x7 ker- nels, and that we were able to successfully classify, with high accuracy. The classification pertains to filters exhibiting reconstruction loss lower than 0.3. This table, for reasons of succinctness, represents single model from each unique architecture. A comprehensive table is available in the appendix.
Significantly, the classification was especially successful for the ConvNeXt series. Over 97% of the filters from ConvNextV2 and more than 95% from ConvNeXtvl were effectively classified, highlighting the efficacy of our methodology in discerning structural patterns within these models.
Moreover, in the MogaNet filters, despite their use dilated convolutions, distinct structural pattern also exhibited over 80% classification success. This observation is crucial it illustrates the ubiquity the discovered patterns: they emerge all the DSC-CNN models considered, even they employ varied convolutional structures, such dilated DCs.
Furthermore, models with higher accuracy generally had more clusterable filters. The ConvMixer model is interestingly the weakest performer, it had the most unclustered kernels and somewhat

---

Published as a conference paper at ICLR 2024
Propartion of Clustered Filters in Each Layer of ConvNextV2 Tiny
0.8
off-center
Proportion
0.6
on-center cross
officenter cross
derivative
0.4
2nd derivative
others
0.2
10
11
12
13
14
15
16
17
Barplot of relative cluster proportions across all ConvNeXrV2 layers, showing the dominance of DoG cluster patterss across layers.
Figure 7:
noisier weights. Models trained more data (ImageNet-22kk also exhibited higher clusterabil- ity. Overall, these results demonstrate that the recurring patterns we uncovered, arise consistently, especially in high-performing architectures. The depthwise kernel structure becomes increasingly pronounced as models improve, suggesting the patterns are linked to generalization ability.
UNDERSTANDING THE IDENTIFIED CLUSTERS
In this section, we conduct further in-depth analysis to better understand the properties the dis- covered clusters. We study the ConvNeXt-V2 V2 model with 7x7 kernels which has total of 6624 DC kernels. We selected this model for several reasons. First, larger 7x7 kernels exhibit more diverse and clearer patterns compared smaller sizes. Second, among 7x7 models, ConvNeXt-VV demonstrated the cleanest filters with minimal noise and achieved the highest ImageNet accuracy. Third, over 97% of ConvNeXt-V2 kernels were accurately categorized by our clustering approach. By leveraging this state-of-the architecture with high clusterability, we can gain key insights into the properties of the recurring patterns uncovered DC kernels across models. Through both quantitative characterization and qualitative visualization, we unravel the structure of the learned representations in the following. Among the models which have 5x5 kernels, we have selected EfficientNet-b4, This model has total of 49632 DC kernels.
Models with 3 Kernels. We observed that the autoencoder does not very well on the ker- nels with size 3. Our hypothesis is that in low dimensional space, the angles between kernels smaller and harder to disentangle them. Instead, we applied k-means clustering on min-max encoded 3x3 kernels, categorizing them into 10 classes. Visualizations for MobileNetV3 (Appendix Section[N] [ reveal the same recurring On/Off-Center and 1t derivative-like patterns despite using an alternate unsupervised methodology.
To characterize the relative prevalence of each cluster, we compute the proportion of the kernel patterns assigned to each class, across layers, for each model. In Figure 7 we display the barplot visualizing the percentage of ConvNeXt-V2 kernels categorized into each distinct class (pattern). The most frequent patterns are the On-Centre and Off-Centre clusters, followed by their cross vari- ants. For visual clarity, the four first derivative subtypes are merged into one class, which appears next most common. The second derivatives comprise the least frequent group. We label remaining unrecognized patterns as "Others", they constitute small fraction. This barplot only includes accurately classified kernels, excluding indiscernible patterns, hence proportions do not total 1.
We have observed consistent prevalence hierarchy, maintained across layers, with DoG structures dominating the initial layers. Very interestingly, the proportions of cross-shaped clusters increase in the later layers, while the proportion of DoGs and their first derivatives decreases. In the final convolutional layer, the cross motifs comprise almost all of the most common patterns.
In Figure II we display the barplot of the cluster proportions for EfficientNet-B4 kernlls, across layers. Similarly to ConvNeXt, the most frequent patterns are the On-Centre and Off-Centre pat- terns. The first derivative clusters comprise the next most common group. Unlike in ConvNeXt-V2 however, the cross-shaped clusters are far less prevalent in EfficientNet. As shown in Figure 9 we additionally identified two clusters, named "Square-On" and "Square-Off" that exhibit larger

---

Published as a conference paper at ICLR 2024
Proportion of Clustered Filters in Each Layer of EfficientNet B4
on-center
off-center
Proportion
0.6
on-center cross square
cross
square
first derivative
0.2
2nd derivative
others
10
11
12
13
144
15
16
17
Layer
Figure 8: Barplot of relative cluster proportions across all EfficientNet-b4 layers, highlighting the sudden
emergence of square kernels one layer (see appendix Section D)
solid centers, resembling the On-Centre and Off-Centre patterns. As shown Figure 11 these square clusters uniquely emerge in layer 10. Notably, the first layer in EfficientNet has lower per- centage correctly classified kernels, with unrecognized "Other" patterns being the most frequent.
The On-Square and Off-Square patterns emerge in almost all models with 5x5 kernels. Notably, the square shape does not appear centered but rather manifests in the upper left bottom right corners. We hy- pothesize this offset localization is due to the small odd size of 5x5 kernels, where larger central block would not fit properly. Intriguingly, each model learns these squared clusters fixed to the same corner posi- tion for both the On and Off variants. This indicates certain architectural hyperparameters, like kernel size. induce consistent localized deforma- tions in the recurring patterns. We have included more investigations into this in the appendix Section D. Nonetheless, the core identity of the discernible patterns remains intact. Quantifying such subtleties between models provides further insights into the underlying structural represen- tations learned by the DSC-CNNs across a diverse set of network archi- tectures. We have included a more comprehensive set of cluster proportions barplots for additional models in the Appendix.
Figure On-Square and Off-Square kernels, emerg- ing only in 5x5 Kernels.
To further illustrate the consistency of uncovered, learned patterns within each pattern-cluster, we il- lustrate in Figure random ConvNeXt-V2 kernel samples, drawn from the On-Centre, Off-Centre, Cross, and the first-order derivatives classes. The samples in each category exhibit strong visual similarity to their assigned label, with clean and coherent structures. The fact that thousands of heterogeneous kernels, converge such (almost) identical motifs is remarkable, and reveals be- havioral simplicity, underlying the emergent representations in depthwise convolutions.
Rather than memorizing wide range of random patterns, the network distills the filters into small set of basic building blocks like DoG derivatives and crosses. This provides intuitive insight into how DCs operate: by learning a compact basis set of structural features, that are replicated densely.
To assess total cluster proportions across models, we compared barplots of all models and found notable consistency among versions of each architecture. Figure IT illustrates this uniformity in models Con ConvNeXtV2, and Hornet, irrespective of model size or training dataset. The full plot containing all models is available the appendix Section J.I] However, Moganet and the dual sets in ConveNextV2 are exceptions, with Moganet varying first derivative filters and ConveNextV2 showing shift between on/off-center and cross filters, as detailed in the appendix Section T.2 Training on ImageNet IK and 21K datasets did not significantly impact these proportions. These observations indicate the architectural design's primary influence over filter distribution, suggesting inherent stability and scalability across these filters.
In order to quantify the emergent patterns, we compute the distribution of total activation (sum of kernel weights) for each cluster. In Figure I0 show box plots summarizing these distributions. The first-order derivative clusters are centered at zero, indicating balanced positive and negative weights. This distribution is in accordance with the symmetric nature of the derivatives of the
8

---

Published as a conference paper at ICLR 2024
Proportion Main Filters Acrosa Different Models
0.4
on-center cross croas
Proportio
firtt dorteaive
0.2
0.1
HorNet base
ConvNexTV2 large
ConuNexri2 large 22k
base ConvNext large 2244
MogaNet
small
large
baso
huge
huge 22k 384
huge 22%
Mlare
smail
base
targe
Figure 11: Comparative Barplot of Cluster Proportions Across Models, Showcasing Consistency across ar- chitectures, regardless of model size or training data.
Gaussian functions and provides further evidence that these kernels might indeed compute these basis functions.
The cross-shaped clusters exhibit much higher total activations, cor- responding to their visual patterns. The On-Centre and Off-Centre clus- ters show mirrored activation distri- butions, with the On-Centre DoG ker- nels having a greater overall weight. These quantitative results reinforce our qualitative observations. The ac- tivation statistics capture unique sig- natures of each pattern that match their assigned visual labels.
aff-center croos off-center off-center
Calogory
on-oenter
off-center
on-center dy others on-center
Ot-ceeter cross
Sum
Figure 10: Box plots of total activation for each cluster of Con- vNextV2, revealing the symmetry and zero-mean most clusters.
CONCLUSION
In large-scale study, analyzed patterns in trained DC kernels from various CNN architec- tures. By visualizing and clustering millions Notably, the predominant patterns resemble DoGs and their derivatives, akin models of visual receptive fields in neuroscience.
of filters, we identified recurring, interpretable motifs.
Our discoveries provide fundamental new insights into the representations learned by modern DS- CNNs. We showed these networks distill dense convolutional filters into a simple vocabulary of basic building blocks, reminiscent of those identified in biological vision. The structural motifs become increasingly pronounced in higher- perforriir models, suggesting link between pattern recurrence and generalization capability.
This work bridges the gap between deep learning, neuroscience, and classical image processing. Our approach sets the stage for more interpretable, biologically-inspire convolutional architectures.
Future Work. Our study focused on image models. The next steps include analyzing video ar- chitectures with 3D convolutions to understand pattern evolution over time, which might align with spatiotemporal visual cortex receptive fields. The identified motifs also lay the groundwork for initialization and regularization methods, aiming to enhance model generalization and efficiency.
The cause of cross-shaped filters is uncertain; potential link to orthogonal Gaussian function sum- mation is explored in Appendix Section Further investigation into these patterns is needed.
Finally, the clusters could inform the development of novel differentiable image filters mimicking the DoG-like learned representations. Integrating these bio-inspired learned kernels into existing CNN operators could lead to enhanced performance and interpretability.
There remain many open questions about the root causes and downstream impacts of the identifiable recurring structures uncovered in depthwise convolutional neural networks. Our discoveries open up numerous avenues for future work, to elucidate the implications this surprising simplicity, underlying complex emergent deep learning representations.

---

Published as a conference paper at ICLR 2024
7 ACKNOWLEDGEMENTS
Z.B. is partially supported by the Doctoral College Resilient Embedded Systems, which is run jointly by the TU Wien's Faculty of Informaties and the UAS Technikum Wien. P.K. is supported by the Doctoral Colledge on Trustworthy Autonomous CPS. This research was funded in part, by the Austrian Science Fund (FWF) I 6605.
REFERENCES
Zahra Babaiee, Ramin Hasani, Mathias Lechner, Daniela Rus, and Radu Grosu, On-off center- surround receptive fields for accurate and robust image classification. In Marina Meila and Tong Zhang (eds.). Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 479-489. PMIR, 18-24 Jul 2021. URI. htttps://procet
ml wrw.ssppes///cc.......... /vi v139/ 39/babaabbaaa.aee ee21a. htm
Zahra Babaiee, Peyman M. Kiasari, Daniela Rus, and Radu Grosu. Neural echos: Depthwise con- volutional filters replicate biological receptive fields. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 8216-8225, January 2024.
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6541-6549, 2017.
Xiaohan Ding, Xiangyu Zhang, Yizhuang Zhou, Jungong Han, Guiguang Ding, and Jian Sun. Scaling up your kernels to 31x31: Revisiting large kernel design in cnns. arXiv preprint arXiv:2203.06717, 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words; Transformers for image recognition at scale, 2020.
Paul Gavrikov and Janis Keuper. Cnn filter db: An empirical investigation of trained convolutional filters. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 19066-19076, June 2022.
Ian Goodfellow, Yoshua Bengio. and Aaron Courville. Deep Learning. MIT Press, 2016. http: w/ww.deepleamiingooo..
Andrew Howard. Mark Sandler, Grace Chu, Liang-Chich Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V. Le, and Hartwig Adam. Searching for mobilenetv3. Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), October 2019.
Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang. Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR, abs/1704.04861, 2017. URL http::tttt/ii..ottot//
17C4.04861 04861
J. P. Jones and L. A. Palmer. The two-dimensional spatial structure of simple receptive fields in cat striate cortex. Journal of Neurophysiology. 58(6):1187 1987. doi: 10.1152/jm. 10.1152/j..1987.58.. 1987.58.6. 1187. URL https: / doi. http:/..111//////////....
1tt.i/152//ppp in. 1987 58. 6. 1187 PMID: 3437330.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger (eds.), Advances in Neural Information Processing Systems, volume 25, Curran Associates, Inc., 2012. URL. httpp::/pprccee////
neuri cc/ wwwpa..r.pppp.......... Paper....... pdf
fflles/paper/2012/
File wtt.://3966222 (e1.99622bbbbbbbb.b http://9952cbbbbcccc333e3eeeeaa2
Siyuan Li, Zedong Wang, Zicheng Liu, Cheng Tan, Haitao Lin, Di Wu, Zhiyuan Chen, Jiang- bin Zheng, and Stan Li. Efficient multi-order gated aggregation network. arXiv preprint 2..0292 2022.
10

---

Published as a conference paper at ICLR 2024
Tony Lindeberg. Normative theory of visual receptive fields. Heliyon, 7(1):e05897, 2021.
ISSN 8405- doi: http:///daarr////.//o//o/oooooo0
URL https::/www.
sciencedirect. com/ htt.://cww......eeooo/..e.aaaaaa
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11):e7, 2017.
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 2020. doi: 1.3391//iitiill0002224.. https://disilllpp///2200ccriiiis
Yongming Rao, Wenliang Zhao, Yansong Tang, Jie Zhou, Ser-Lam Lim, and Jiwen Lu. Hornet: Efficient high-order spatial interactions with recursive gated convolutions. Advances in Neural Information Processing Systems (NeurIPS), 2022.
R. Rodieck. Quantitative Analysis of Cat Retinal Ganglion Cell Response to Visual Stimuli. Vision Research, 5(12):583-66 101, 1965.
Mingxing Tan and Quoc Le. EfficientNet: Rethinking model scaling for convolutional neural net- works. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th In- lernational Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 6105-6114, PMLR, 09-15 Jun 2019a. URL pestps://proceedie mml. press/v97/tan 99.html
Mingxing Tan and Quoc V Le. Mixconv: Mixed depthwise convolutional kernels. arXiv preprint arXiv:1907.09595 2019b.
Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoe V Le. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2820-2828, 2019.
Nergis Tomen, Silvia-Laaur Pintea, and Jan Van Gemert. Deep continuous networks. Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference Machine Learning, volume 139 Proceedings of Machine Learning Research, pp. 10324-10335, PMLR, 18 24 Jul 2021. URL
mlr wttpa//mww.......peeee////////// pttpsppps/ss/////een//e/l html
Asher Trockman and J. Zico Kolter. Patches all you need? CoRR, https://arxiv:
2022.
2022. URL
httpg//b//200100 09792
Chelsea Voss, Nick Cammarata, Gabriel Goh, Michael Petrov, Ludwig Schubert, Ben Egan, Swee Kiat Lim, and Chris Olah. Visualizing weights. Distill, 2021. doi: 007.
Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders, arXiv preprint arXiv:2301.008800 2023.
Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. Understanding neural networks through deep visualization. In International Conference on Machine Learning, pp- 2067-2075, 2015.
R.A. Young, R.M. Lesperance, and W.W. Meyer. The gaussian derivative model for spatial-temporal vision: cortical model. Spatial vision, 14(3-4):211311, 2001.
Richard A. Young. The gaussian derivative model for spatial vision: I. retinal mechanisms. Spatial
Vision, 2(4):273 293, 1987. doi: http://ko..rg///1111////66666220 URL https:
//brill. www/coiiiiieeeee vtom/vieww/// Journa wwwpunaasssv///o/ htt.mm///////////aartclee-----p/
xm1]
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In
European conference computer vision, pp. 818-833. Springer, 2014.

---

Published as a conference paper at ICLR 2024
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Revisiting fine- grained visual categorization. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 394-410 2018.
12

---

Published as a conference paper at ICLR 2024
Appendix
A COMPLETE DoG PLOTS
Employing two different first-degree DoGs results in diverse set of derivatives: four first-order and six second-order derivatives are derived. However, for simplicity and due to the relative rarity of filters in our filter set, Figure 12 seleetively illustrates only two of these second- order derivatives.
(a) DoG(x, v)
dDoC(a.y) da
d DoG(a.y)
(e) -Doo(G(() v)
-dDoG(a..)
dd DoG(x.u)
Figure 12: Complete diagram of the On and Off DoG filters and their derivatives.
For more comprehensive understanding, DoG and its associated derivatives have been visualized on a 7x7 grid, as depicted in Figure [3
(a) DoG y)
(b)
dDoG(r.)) de
(c)
dDaG(sn.)
(d)
d DoG(a.y) do
dy
OP
4:1
(c) - -DoG(z, y)
(z.v) dla
dDDo((a.g)
(h)
dd (oG(s.)) d22
dy
Figure 13: Complete diagram of the On and Off DoG filters and their derivatives on 7x7 grid.
13

---

Published as a conference paper at ICLR 2024
B HYPOTHESIS ON MATHEMATICALL FORMULA OF CROSS-PATTERI FILTERS
The DoGs and their higher derivatives are a well-established concept in image processing. As figure 14 shows, cross-pattern filters share visual affinity with DoGs, yet their precise mathematical Formulation remains unknown to us.
Figure 14: Samples of Cross-Pattern Filters, magnified by 100 to show the first two decimals.
Our investigations suggest that the sum of orthogonal Gaussian functions provides near semblance, as demonstrated in Figure 15 where the functions are visualized on a 7x7 grid across spectrum standard deviations:
exp
exp
2n
This formulation is mostly consistent with the spatial characteristics of cross- pross-patter filters and offers potential mathematical model for their behavior.
0.4
(b) 0.5
0.6
(d) 0.7
(e) 0.8
Figure 15: Sum of Gaussians with standard deviations ranging from 0.4 to 0.8, on a 7x7 grid.
While this model mirrors the cross-pattern scheme, dissimilarities exist, particularly the more intense central density of the cross-pattern filters, which diverges from the Gaussian summation.
Further research needed. Future work should refine the model to account for observed central density and spatial details of the cross-pattern filters, possibly through the use of Gaussian functions.
C AUTOENCODER ARCHIRTECTURE
Figure shows the architecture of the autoencoder for kernel size 7, with ID hidden code in red.
Reconstructed Kemel
Input Kemel
Le
L-
ReLU
Sigmold
L-ReLU
ReeUU
ReLL
Tan+
Figure 16: Autoencoder architecture and its layers.
14

---

Published as a conference paper at ICLR 2024
D THE CURIOUS CASE OF SQUARE KERNELS
Analysis of the filter clustering proportions in EfficientNets and Mo- bileNets reveals consistent pattern where one or two layers predom- inantly learn square kernels in almost all models. For instance, in the EfficientNet-BB model, this is notably observed in the 22nd layer(layer 10 of 5x5 kernels). To delve deeper into this phenomenon and assess the influence of training procedures and random initialization, we con- ducted an experiment where the EfficientNet-BA. model was trained us- ing two distinet random seeds over a period of 50 epochs. The resulting barplots, depicted in Figure [8| demonstrate that both training instances exhibit a similar trend to that observed in the relucheeeeeaedd, fully- trained model. Remarkably, in both cases, the 16th layer (layer of 5x5 kernels), shows a higher concentration clustered filters, mirroring the pattern in the fully trained model (referenced Figure in the main text). This trend is particularly intriguing as these layers are positioned at the beginning of block, suggesting that the model may prioritize training these initial layers, potentially impacting the types of filters learned.
Figure 17: On-Square and Off-Square kernels
Proportion of Clustered Filters in Each Layer of EfficientNet-BB random-seed v1
0.6
0.5
off-center
0.4
on-center cross
Proporton
off-center cross square square
0.3
0.2
first derivative
2nd derivative
0.1
others
0.0
21
22
23
25
26
27
28
Layer
Proportion of Clustered Filters Each Layer of EfficientNet-B4 random-seed v2
0.6
0.5
off-center
Proportion
0.4
on-center cross
off-center
0.3
square
square
0.2
first derivative
2nd derivative
0.1
others
16
17
18
21
22
23
24
28
26
27
28
29
Layer
Figure 18: barplot of relative cluster proportions of EfffiientNet-B.. after 50 epochs of training with two different random seeds
15

---

Published as a conference paper at ICLR 2024
E THE CURIOUS CASE OF HORNET TINY
In contrast to other HorNet models (small, base, and large), HorNet Tiny illustrated by an abnor- mal behavior observed in Layer 14. Layer presents an anomaly, with significant reduction or absence of clustered filters. This deviation is evident in Figure 19
Proportion of Clustered Filters in Each Layer of hornet_tiny weights
0.8
off-center on-center cross
Proportion
0.6
off-center cross
first derivative
2nd derivative
0.2
others
0.0
5
7
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Layer
Figure 19: Barplot depicting clustered filter proportions in HorNet Tiny layers, highlighting the unexpected emergence of unidentified filters Layer 14.
To further clarify this anomaly, we have visualized random samples from Layers 13, 14, and 15 in Figure 20) This comparison distinctly showcases the unusual patterns observed in Layer 14, in contrast its adjacent layers.
(a) Layer 13
(b) Layer 14
(c) Layer 15
Figure 20: Random Samples from Layers 13, 14, and 15 of HorNet Tiny, highlighting the abrupt appearance of seemingly uninterpretable patterns in Layer 14.
Despite investigations, the underlying cause of this phenomenon remains unclear to us. This aspect of our research has yet to be fully understood, presenting an avenue for future exploration.
16

---

Published as a conference paper at ICLR 2024
F EXPLORING APPLICATIONS: AN EXPERIMENT ON CONVMIXER
In models utilizing 7x7 filters, the ConvMixer model displayed a notably low score (56%) in filter clustering proportions. Analysis of the model's kernels, as illustrated in Figure 21 revealed the presence of noisy kernels. Despite apparent underlying similarities, many were classified as unclustered by our algorithm, due to the strict loss threshold. We hypothesize these filters are inclined to align with DoG cluster patterns.
To investigate our hypothesis, performed an experiment on network ini- tialization. We chose the ConwMixer768-32 model, utilizing a 14-patch size, and trained it over 50 epochs. A second model was initialized with various DoG functions, using random variances similar to trained kernels. The ini- tialization was uniformly distributed: 45% on-center, 10% off-center, 15% cross, 20% first derivative, and the rest second derivatives. This model was then trained under the same conditions.
Figure 21: Samples of convMixer filters showing noisy fea- tures.
The table below summarizes our experiment's findings, showing notable improvements in accu- racy and an increased proportion of clustered filters in the DoG-initialized model compared to the baseline. Additionally, consistent layer proportions suggest minimal filter changes during training. Figure 25 illustraas these cluster proportions post-training in the barplot.
Disclaimer.
This experiment was conducted a single time without optimizing proportions and vari- ances; further refinement could improve results, but that
main objective here.
Model
Initilization
Accuracy (%)
Clustered Filters (%)
CixeMixer 768-32
Kaiming DoGs
66.35
56.65
ConyMixer 768-32
69.01
96.03
Table Comparison of Normal and Initialized ConvMixer 768-32
Figure 22 compares random filters from the Con-Mixer768-92 at epoch 50, both with and without initialization. is evident that the initialized filters exhibit much clearer patterns.
(a) Trained filters with Kaiming initilization
(b) Trained filters with DoGs initilization
Figure 22: Random Samples from ConvMixer768-32 trained without and with DoG initialized weights. Filters with initialization are cleaner and less noisy.
Proportion Clustered Filters Layer of ConvMixer 768 32_layer Epoch 1.0
on-center
off-center
Proportion
0.6
on-center
off-center cross
0.4
first derivative
2nd derivative
others
15
18
19
22
23
24
25
26
7
28
36
31
Figure 23: barplot of cluster proportions of ConvMixer initialized by DoGis after training
17

---

Published as a conference paper at ICLR 2024
G FILTER PATTERNS TIMELINE
We monitored the Convmixer768-33 --33 model's filters from the initial stage to epoch 50, less than its full 300-epoch training, observe DoG pattern development. Figure 24 presents barplots of filter proportions at 5, 10, 25, and 50 epochs, showing a gradual emergence of DoG patterns, with the on-center DoG appearing first.
Proportion
Layer CanvMlsor Jayer epoch
0.14
0.12
00-00maa
0.10
cencernar cross
6.08
of cerrrrr cross
6.08
2nd derivative offers
Ind dentrsttve
0.04
0.02
0.00
Layer
Propartion
Layer
32_layer epoch
0.4
Pepertoon
6.2
fistt derieetive
of cerror cross
2nd derivative othere
0.0
Proportion Clusterad Fillers
ConwMixer
[a_aayer apoch
0.8
6.5
Propette
2nd dorivative
others
6.1
Proportion Clustorad Filters Each
ComMitter
0.4
0.5
detialive
others
Figure 24: Barplot of relative cluster proportions of Mox-Mixer768-32 in various epochs.
Proportions of Clustered filters by Epochs
Proportions of Clustered filters
0.2
10
20
30
40
50
Epochs
Figure 25: Total proportion of filters clustered across time.
18

---

Published as a conference paper at ICLR 2024
H GENERALIZATION OR MEMORIZATION??
This section delves into how overfitting impacts the patterns filters learn. We experimented by train- ing the ConwMixer768-32 model on limited portion of the ImageNet dataset, specifically subset of just 10 classes. The training was conducted for 200 epochs without any form of regularization, de- liberately steering the model towards overfitting. The results of this experiment were quite revealing. Figure 26.b displays random selection of kernels from the overfitted model. A striking difference observed when compared to the model trained the complete ImageNet dataset (Figure 26)a): a): the kernels lack clear patterns. This finding hints at potential link between the development of DoG patterns and the model's ' comprehensive understanding of images, rather than mere memorization.
To further probe this hypothesis, we subjected the same model to training on the small-sized Cifar10 dataset. The kernels from this training, as shown in Figure 2 similarly lack discernible patterns, reinforcing the notion that dataset with larger variety is necessary for the model to develop recognizable patterns in its filters.
(a) Trained on full ImageNet
(b) Trained on 10-class ImageNet
(c) Trained on Cifar10
Figure 26: Comparative Analysis of Filter Patterns in Con-Mixer768-32 Models trained on different data.
I LARGE 27x27 KERNELS
In order to investigate the presence of the DoG family filters in DS-CNN models with larger kernel sizes, we examined trained filters of model RepLKNet-XL (Ding et al. 2022). Our investigation revealed the presence of DoG-shaped filters in this model, especially in early layers, akin to those observed smaller models. Figure 27 shows selected samples from the trained kernels of this model. As one see, despite the large kernels, the patterns emerge in small central area.
Figure 27: Selected Samples of RepIKNET-X- XL Model 27 x7 Kernels.
19

---

Published as a conference paper at ICLR 2024
J RELATIVE CLUSTER PROPORTIONS
J.1 COMPLETE PROPORTIONS PLOTS
A comprehensive of bar plots, referenced in Figure 11 is provided here for detailed analysis.
Proportin Mann fooe Diferent Models
0.35
off-center
0.30
center cross
derivative
0.20
0.10
0.05
alo 1k 224 .ma
famis N 234 Jen
nan 32%, 224 oma
nato 22%, 384 oom
basa
bate 32k, 201 Lema
base 22k. 384 ana
Large
huge, 1k Lemr
naro 1t, 204 .eni
wiw 30% you win
wim 220 304 wITH
Hoo 22h 201 en
204
304
It. 234
512
Proporton otMain Fllrrs
ot-ccnter
on-onntecross
0.20
first dertistive
0.15
0.05
Sny 22 W. 224
Try 22, 200
omal
bosse fR. 384
bave 22k, Pr 2006
bare 22k, iik 2244
laro
lagg 20N We 2944
large 23k 224
Karge 22k 1kk 204 omaa
swige 20N 304 ema
andl 29k 1e.22
23k
20k
Proporton Filters Offerent Models
01060
off contor CIO60
Firs: chr VNIV
0.25
Progoria
0.05
0.00
Figure 28: Proportion plots for ConveNextV2, ConveNext, HorNet, and Moganet Models. magenne strrpe, eocoA noott n0300
home small
homee Jagg
moopene stay v/224 Edde 128, wo3t0
mappenn sry vutta Entu122, s000
mrggant suite 8ibu128 00000
mepanet small Bibs22e c300
meganet bote, pozkk 286112 0030)
rrggnet Ltrpe 20000
homel
20

---

Published as a conference paper at ICLR 2024
J.2 CONVENEXTV2 DUAL CLUSTER PROPORTIONS
In ConveNextV2 models, a unique duality in cluster proportions is observed, distinct from other models, Figure 28 reveals separate proportion in ConveNextV2: 11 versions in the first and in the second.
Figure 29 comparing ConveNextV2 huge Ik and 22k suggests the difference may stem from shift to on/off-cross from on/off-center filters.
Proportion of Clustered Filters in Each Layer of convnext22 huge_ 1k 224 _emae_ma___ weights
0.8
on-center
off-center
Proportion
on-center
off-center cross first derivative
0.4
2nd derivative others
0.2
10
11.
12
13
14
15
17
18
19
20
21
23
24
25
26
27
28
29
30
31
33
34
Proportion of Clustered Filters in Each Layer of Swwneext22hhue__22 _222222k____________ _ak_384_ema 1144_mma_eeat ama__eightss
1.0
0.8
on-center
off-center
Proportion
0.6
on-center
aff-center
derivative
derivative
others
0.2
10
12
13
16
17
13
19
20
21
22
23
24
25
26
27
28
29
30
51
32
33
34
35
Figure 29: Barplot of relative cluster proportions across all Layers
To validate our hypothesis, we merged labels of on-center and on-cross filters (and off-center with off-cross) in ConveNextV2's proportion barplot. Figure 33 confirms our hypothesis, displaying homogeneous patterns across all proportions. notable that seemingly the merged labels are nearly 50/50 across all models.
Propartion Man Filees Across filientt Models
0.4
offcenterr cross
first derivatie
derivative
0.3
Progoron
400 14, 224 unT
fanto Te, 224 umaa
m9mo 32% 234, Lema
mano 224, 384 ona
Wegg 22s 224420 arm
Weg 72t, 394, Lrne
hage 16. 224 uma
poo, 14 234 ema
neno 14 224, arme
1ny
Jage 22s 224_2mm4 L076
384,
384
512
large
Figure 30: Homogenized Proportions in ConveNextV2 Filters, Merging On--Center/OO-Cross and
Off-CenterrOft-Croos Labels.
21

---

Published as a conference paper at ICLR 2024
K
THE COMPLETE HIDDEN DIMENSION SPACE RECONSTRUCTION PLOTS
Figure 31: Hidden space spectrum of Autoencoder trained on 5 x 5 kernels used for labeling classes.
Figure 32: Hidden space spectrum Autoencoder trained on 7 7 kernels used for labeling classes.
22

---

Published as a conference paper at ICLR 2024
L
MODELS USED AND THE PROPORTIONS OF THEIR FILTERS CLUSTERED
Model
Filters Clustered (%)
MobileNetV3 large 100 miil
87.09
MobileNerV3 large 100 in21 miil
85.71
MobileNetV3 large 100 ra
78.03
MobileNetV3 large 21k
85.71
MobileNetV3 large
87.09
MobileNetV3 100
78.44
EfficientNet b0 ra
73.12
EfficientNet b0
73.12
EfficientNet bl
69.37
EfficientNet 62 ra
70.48
EfficientNet b3 ra2
74.28
EfficientNet ra2 320
73.78
EfficientNet el pruned70
50.91
EfficientNet el
52.74
EfficientNet ra2
58.46
EfficientNet es pruned75
49.78
EfficientNet
50.43
EfficientNet liteO ra EfficientNet b0 aa tf EfficientNet b0 ap EfficientNet b0 ns tf EfficientNet bl aa EfficientNet bl ap if EfficientNet bl ns EfficientNet b2 aa tf EfficientNet b2 ap EfficientNet b2 ns tf EfficientNet b3 EfficientNet b3 ap EfficientNet b3 ns EfficientNet b4 aa EfficientNet b4 ap tf EfficientNet b4 ns EfficientNet b5 ap tf Efficienntet b5 ns EfficientNet b5 ra tf EfficientNet b6 aa EfficientNet b6 ap tf EfficientNet b6 ns
61.08
76.61
74.97
77.49
73.36
73.83
78.31
71.62
71.81
77.50
70.32
71.48
76.81
67.03
69.67
75.34
69.76
73.32
67.12
63,80
68.15
71.96
MixNet s
73.25
MixNet
66.34
MixNet
69.43
MixNet ra
58.91
MnasNet al
71.59
MnasNet
62.61
SpnasNet 100
58.66
FBNNet 100
59.94
Table 3: List of Models with 5 x 5 kernels used and their Filter Clustering Results
23

---

Published as a conference paper at ICLR 2024
Model
Filters Clustered (%)
ConvNextV2 atto 224 lk
96.47
ConvNextV2 femto 224 lk
97.74
ConvNextV2 V2 pico 224 lk
95,96
ConvNext V2 nano 224
96.94
ConvNexrV2 nano 224 22k
97.94
ConvNcxtV2 nano 384 22k
97.45
ConvNextV2 tiny 224 Ik
97.33
ConvNextV2 tiny 224 22k
98.16
ConvNextV2 tiny 384 22k
98.20
ConvNextV2 base 224 lk
93.38
ConvNextV2 base 224 22k
97.56
ConvNextV2 base 384 22k
97.56
ConvNextV2 large 224 lk
90.82
ConvNextV2 large 224 22k
96.63
ConvNextV2 large 384 22k
96.67
ConvNextV2 huge 224 1k
82.08
ConvNextV2 huge 384 22k
92.41
ConvNextV2 huge 512 22k
92.43
ConvNext tiny 224 1k
95.21
ConvNext tiny 224 22k_ _
87.55
ConvNext tiny 384 22k_1k
86.91
ConvNext tiny 224 22k
88.09
ConvNext small 224 Ik
94.68
ConvNext small 224 22k 1k
94.48
ConvNext small 384 22k_16 1k
93.84
ConvNext small 224 22k
94.57
ConvNext base 224 lk
93.57
ConvNext base 384 lk
93.13
ConvNext base 224 22k_1k
93,82
ConvNext base 384 22k lk
92.91
CenvNext base 224 22k
93.87
ConvNext large 224 1k
91.26
ConvNext large 384 lk
90.77
ConvNex large 224 22k_1k
88.59
ConvNext large 384 22k.1k
88.06
ConvNext large 224 22k
88.93
ConvNext xlarge 224
87.67
ConvNext xlarge 224 22k_1k
87.95
ConvNext xlarge 384 22k_1k
87.67
ConvNext xlarge 224 22k
88.11
MogaNet xtiny 224 lk
78.19
MogaNNe tiny 224 lk
80.82
MogaNet tiny 256 lk
84.16
MogaNet small 224 1k
78.87
MogaNet base 224 lk
70.22
MogaNet large 224 1k
58.49
MogaNet x xlarge 224 1k
55.30
HorNet tiny 224 1k
80.71
HorNet small 224 1k
79.09
HorNet base 224 1k
72.86
HorNet large 224 1k
68.58
ConvMixer 512 20 layer 1k ConvMixer 768 32 layer
58.02
56.65
ConvMixer 768 initialized
96.03
Table 4: List of Models with 7 x kernels used and their Filter Clustering Results "Note: Please see Section
24

---

Published as a conference paper at ICLR 2024
M PROPORTION BAR CHARTS
Proportion of Clustered Filters in Each Layer of moganet tiny $2256 8xbs128 Bxbs128_ep300 ep300 weights
1.0
on-center
off-center
Proportion
0.6
on-center
off-center first derivative 2nd derivative others
0.4
0.0
3
4
5
8
9
12
1.3
14
15
16
Layer
Proportion of Clustered Filters in Each Layer moganet base s2224 8xbs128 ep300 weights
1.0
on-center off-center
Proportion
0.6
on-center cross
off-center cross first derivative
0.4
2nd derivative
others
0.0
10
11
12
13 14
15
16
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
Propartion af Clustered Filters Each Layer of moganet large_sz24
__s2222_8xbb64
_e10004_p300_0__11 _p300_weights
off-center
Proportion
on-center cross
off-center cross first derivative 2nd derivative others
0.4
0.2
0.0
Layer
Proportion Clustered Filters in Each Layer moganet wwarrelalaaea2a s2224_8xbs64 1a00664_e300__e11 weighh_
0.8
on-center
0.6
off-center
on-center cross off-center cross first derivative 2nd derivative others
0.4
0.2
0.0
Layer
Figure 33: Barplot of relative cluster proportions across all Layers
25

---

Published as a conference paper at ICLR 2024
Proportion of Clustered Filters in Each Layer of convmixer_ 768_ sr__3__ss__p__eee relu weights
0.7
on-center
off-center
Proportion
0.5
on-center
off-center cross first derivative 2nd derivative
0.3
0.2
others
0.1
10
11
13
14
16
17
18
20
21
22
23
24
25
26
27
28
29
31
Layer
Proportion of Clustered Filters Each Layer of convnext Linexxt_tnyy__ttt tiny_ingy 22k 224_weighne
1.0
0.8
on-center
off-center
Propertion
0.6
on-center cross
off-center cross first derivative
0.4
2nd derivative
others
0.2
0.0
10
15
16
17
Layer
Proportion Clustered Filters in Each Layer of convnext_bast ba____aaeee___e___ weights
1.0
on-center
off-center
Proportion
0.6
on on-center cross
off-center first derivative
0.4
2nd derivative others
0.2
0.0
10
11
12
13
14.
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
50
31"
32
33
Layer
Proportion of Clustered Filters in Each Layer of convnext canvnext_large_ large_rree_22 22k 224 weights
1.0
0.8
on-center
off-center
Proportion
on-center
off-center cross first derivative
0.4
2nd derivative
others
0.2
0.0
10
11
12
13
14
15-
16
17
18
19
20
21
22
23
24
25
26
27-1
28
29
30
32
53
34
35
Layer
Figure 34: Barplot of relative cluster proportions
all Layers
26

---

Published as a conference paper at ICLR 2024
Proportion Clustered Filters in Each Layer of convnextv2 tinntttn_i_______tt Siny_weihhts
1.0
0.8
on-center
off-center
Proportion
0.6
on-center cross
off-center
first derivative
0.4
2nd derivative
others
0.2
0.0
10
11
12
13
14.
15
16
17
Layer
Proportion of Clustered Filters in Each Layer of wtwpnnexxtt_lrrg_eeeea weights
1.0
0.8
on on-center
off-center
Proportion
0.6
on-center cross
off-center cross first derivative
0.4
2nd derivative
others
0.0
9
10
11
12
13
14
15
1.6
18
19
20
21
22
23
24
25
26
27
28
29
31
32
33
34
35
Layer
Proportion of Clustered Filters in Each Layer of convnextv2 h__hhee___ 1r_11224444 21__224_eaa sana_weights
1.0
0.8
on-center
off-center
Proportion
0.6
on-center
off-center cross first derivative
0.4
2nd derivative
others
0.0
10
11
13
14
15
16
17
18
20
21
22
23
24
25
26
27
28
29
30
31
32
34
Proportion of Clustered Filters in Each Layer of convnextv2_huge S_tite_hhhhe__2 1_.22k_388 338__ema_welleee ema__eiggts
1.0
on-center
off-center
Proportion
0.6
on-center cross
off-center cross first derivative
0.4
2nd derivative
others
0.2
10
11
12-
14
15-
16
17-
19
20
21-
22-1
23
24
26
27
28
29
31
32
33
34
Figure 35: Barplot of relative cluster proportions across all Layers
27

---

Published as a conference paper at ICLR 2024
Proportion of Clustered Filters in Each Layer hornet hornet_tiny. tiny_weight
on-center
off-center
Proportion
0.6
an-center cross
off-center cross first derivative
2nd derivative
0.2
others
0.0
6
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Layer
Proportion of Clustered Filters in Each Layer of hornet_small _eights
0.8
on-center
off-center
Proportion
0.6
on-center cross
off-center cross first derivative
0.4
2nd derivative
others
11
12
14
16
17
20
21
22
23
Layer
Proportion of Clustered Filters in Each Layer of hornet base_weight
0.8
on-center
off-center
Proportion
0.6
on-center cross
off-center cross first derivative
0.4
2nd derivative
0.2
others
0.0
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Layer
Proportion of Clustered Filters in Each Layer of hornet hornet_large, weeehht
0.8
on-center
aff-center
Proportion
0.6
an-center
off-center cross
first derivative
2nd derivative
0.2
others
11
12
14
15
16
17
18
19
20
21
22
23
24
Layer
Figure 36: Barplot of relative cluster proportions across all Layers
28

---

Published as a conference paper at ICLR 2024
Proportion Clustered Filters in Each Layer of efficientnet_bo__ee
weights botiighhhhtb_wwhhhhhh
0.8
on-center
off-center
on-center
Proportion
0.6
off-center cross
0.4
square off first derivative
0.2
2nd derivative others
0.0
11
12
14
Layer
Proportion of Clustered Filters in Each Layer of efficientnet Baatiinte_b_rraaaeee 22__rabbddf3bb__wegge
on-center
off-center
on-center cross
Proportion
0.6
off-center cross
0.4
square off first derivative
2nd derivative others
0.2
0.0
Layer
Proportion of Clustered Filters in Each Layer of efficientnet b4 14_ra2___ 320-7eb33cd5_weige
0.8
on-center off-center off-center cross square on square off first derivative 2nd derivative others
Proportion
0.6
0.4
0.2
10
11
12
13
14
15
16
Layer
Proportion of Clustered Filters in Each Layer of tf wee.effieeeeeettbo b6 ns-515488356 1
0.8
on-center
off-center on-center off-center cross square on square off first derivative 2nd derivative others
Proportion
0.6
0.4
0.2
0.0
10
12
14
16
18
20
21
22
23
24
Layer
Figure 37: Barplot of relative cluster proportions across all Layers
29

---

Published as a conference paper at ICLR 2024
Proportion of Clustered Filters in Each Layer of tf mobilennttv sampeetttsssmslllll_llll 075-da427f52_weltt weights
1.0
on-center
0.8
off-center
on-center cross
Proportion
0.6
off-center cross square
0.4
square
first derivative
2nd derivative
0.2
others
0.0
Layer
Proportion of Clustered Filters in Each Layer of mobilenet v3 la_artge___rgeeeeeeeeeaee weights
on on-center off-center on-center off-center cross square
0.8
Proportion
square off first derivative
0.2
2nd derivative others
0.0
12
13
14
Layer
Proportion of Clustered Filters in Each Layer of mobilenetv3 large_ 100 ______n2k_________ mill_weighhss
0.8
on-center off-center on-center cross off-center cross square on square off first derivative derivative others
Proportion
0.4
0.2
Layer
Proportion of Clustered Filters Each Layer of mnasnet a1-d9418771_weite
0.8
on-center off-center on-center off-center cross square
Proportion
0.6
square off first derivative 2nd derivative others
0.2
Layer
Figure 38: Barplot of relative cluster proportions across all Layers
30

---

Published as a conference paper at ICLR 2024
N K-MEANS CLUSTERING OF MOBILENETV3 LARGE 3 X 3 KERNELS
31

---

Published as a conference paper at ICLR 2024
32


Reviews:
[{"review_id": "jD8NWS5Mri", "reviewer": "Reviewer_Jbbg", "summary": "This paper proposes a study on the nature of the learned convolutional filters of various CNN architectures. They show that the kernels learned by depthwise-separable convolutions (DSC) are similar to DoG-derivatives filters, which draws connections with biologically inspired models.", "questions": "1) In Figure 7, layer 10 has a strong proportion of ‘square off’ kernels, do you have an intuition on why this is the case ?\n\n2) Why do you need to use an autoencoder to learn kernel representations, could you not just use the kernels themself and use k-means ?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The paper offers a very insightful and novel view on the fact that DSC kernels learn DoG filters, which is to the best of my knowledge a new observation for CNN architectures. The visualizations, especially Figure 2, are very informative and clearly show this property of DSC.\n\n2) The clustering of kernels approach shows that the learned kernels which are DoG filters actually form a basis of DoG filters with specific shapes that the authors identify and show in Figure 5. This observation shows that DSC are more similar to biological models than regular convolutions which could inspire future CNN architectures.\n\n3) The claims of the paper are validated on a wide variety of models and many visualizations are presented, including in-depth study of each layer of each model and a lot of examples of kernels.", "weaknesses": "1) The authors suggest that better understanding the nature of what CNNs learn, and in this particular case the fact that DSC learn DoG-filters, would lead to architectural improvements or new architectures. However no actual concrete improvements or suggestions are proposed in the paper.\n\n2) The observations are interesting but a more interpretable model is not necessarily correlated to a better performance of the model, for example, vision transformers are the most popular architecture nowadays, they don’t learn filters and are not inspired by biology. A more in-depth study of the performance on various tasks of models with DS-conv against models without would have been a good addition. For example, a study on the differences between ResNet and ResNeXt/ConvNeXt families. Or with a specific architecture by only replacing regular convolution with DSC.\n\n3) The paper claims that DSC learns a basis of DoG filters, which is highlighted by the visualizations and the clustering approach, but it might be the case that ResNet models and other models using standard convolutions also learn a basis of other functions/filters that are less interpretable, but still efficient for the final performance of the model. It is not clear that interpretability is a key advantage.\n\n4) The interpretation of what is the role of each ‘basis vectors’ DoG filter (such as “off-center” or “off-center cross”) is doing would help clarify what is the advantage for a CNN model to learn such filters.\n\n5) DS, DC, and DSC notations are used interchangeably throughout the paper and it is not clear if they mean the exact same thing or not. Could you clarify or use the same term for consistency ?\n\n6) Most captions of figures are not self-contained and very unclear, especially on describing how the visualizations are obtained and what conclusion should be drawn from the figure.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "M7b3FlpkZA", "reviewer": "Reviewer_hU9E", "summary": "This paper performs an extensive study on how the depthwise convolution (dwconv) kernels in the depthwise-convolution-employed convolutional neural networks (this paper calls them DS-CNNs)  look after training on an image classification task. The authors employ the ImageNet pre-trained DS-CNNs such as ConvNeXts, EfficientNets, and MobileNets for the analysis. The main analysis tools are unsupervised clusterings via autoencoder and k-means clustering, and the clustered kernels are seemingly similar in shape to those of DOG filters (including the first and second derivatives). The authors report several qualitative results of clustered kernel shapes from different architectures and the histogram reflecting the portions of the representative kernel shapes.", "questions": "- I am still not convinced that the patterns of the learned kernels follow DoG shapes, because 7x7 kernels are still small. There are the DS-CNNs that employed much larger kernel sizes, if the authors would like to claim the filter shape is restricted to DOG or its derivatives, the authors should try to verify the claim on the models consisting of larger kernels [1, 2].\n  - [1] Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs, CVPR 2022\n  - [2] More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity, ICLR 2023\n\n-  What is the primary goal of computing the filter classification score in Table 1? and what does it imply when the filter classification accuracy is high?\n\n- Table 1 repots wrong ConvNeXtV2_tiny's top-1 accuracy on ImageNet-1k, which is not 84.9%. Please refer to the original manuscript.\n\nPre-rebuttal comments)\nThis reviewer finds analyzing the depthwise convolution and revealing common DoG-like patterns from DS-CNNs, which gives some insights. However, the impact of the analysis itself and further use cases remain unclear; the potential impact is also problematic and uncertain. Overall, the reviewer believes the paper does not reach the publication standards of ICLR in its present state, I would like to see the other reviewers' comments and the authors' responses.", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- To my knowledge, this paper first analyzes the depthwise convolution.\n- The idea of analyzing DS-CNNs by visualizing the weights that are clustered to representative shapes is interesting. \n- This paper is easy to follow.", "weaknesses": "- The analysis is thought-provoking, but its significance is not fully realized. The ultimate goal of the analysis remains unclear to me. The paper effectively renders the clustered kernel in shapes akin to DOG or its derivatives, but it does not progress towards a more meaningful conclusion. This reviewer feels like there is a missed opportunity to extract and impart lessons, such as a connection to new network design principles for CNNs, which the analysis could potentially offer, yet such insights are not provided in the end.\n\n- This reviewer is questioning the necessity for the advanced and training-needed autoencoder-like denoising/clustering technique in the analysis when K-means clustering appears to suffice for simply examining the trends of the learned kernels, as shown in the authors’ treatment of MobileNetV3. It is speculated that the denoising effect may smooth the actual filter's learned patterns, so the results may not be convincing. Furthermore, while the autoencoder is presumed to be more suitable for this analysis, a clear justification for this expectation is needed to validate its use as an analytical tool.\n\n- The introduced autoencoder-based analysis presumably gives more sophisticated results over K-means-like methods. However, there is speculation that the authors are employing a tool that aligns more closely with the desired DoG patterns. Furthermore, such learning-based methods would give underfitted results when training is not successfully performed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4zDFStK7E9", "reviewer": "Reviewer_GKik", "summary": "The work posits that convolutional architectures that have depth-wise separable convolution layers learn a few characteristically Gabor-like filters, which is a widely intuited concept for CNNs in general but never really proven in any sense. They demonstrate analytically that this observation spans a zoo of models of various origins with various characteristics and is not a happenstance. They also show that this is not only a phenomenon that shows up in the initial layers of the network as was already demonstrated by several works prior, but in deeper layers of the network as well. However, the authors stop short of drawing strong conclusions out of their novel and interesting observations.", "questions": "1. The obvious unanswered question from the paper is the mystery layer 10 of EfficientNet-b4 and b6, and layer 11 of b2. This would ideally be the case where multiple checkpoints be needed to verify that this is not just a quirk of the random seed of training this model.\n\n2.  What is happening in the layer 14 of hornet tiny?\n\n3. Is it possible to create a graph of the number of identifiable filters by your clustering algorithm v. the generalization of the network on a standard test dataset? This would further bolster the authors' argument that better networks have more simpler features.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The work is not completely, original in its thesis - scientists have long intuited the nature of convolutional features from the perspective of biological vision and compositionality. There are several well-known images in well-read pieces of work from the original AlexNet paper to the VGG paper and so on, that present the filters in their first layer and their seeming relationships to Fourier or Gabor basis functions, lest one forget the image and its mystery in the AlexNet paper that shows that one GPU learnt color while the other learnt grayscale. What is original in this work is that the authors have noticed that for depth-wise convolutional networks, these features remain Gabor-like much deeper into the network. Not only do they demonstrate that, they also show that there are only a handful of these features and they are all lower-order filters as well. Stronger yet is the correspondence the authors have shown that the more performant the network, the more stronger these observations hold. This observation raises several questions regarding filter complexity, model complexity and regularization for convolutional networks and its relationship to the the order of these filters. This work is strong is so far as it is set to drive other future works in this area in the direction of a potential canonical and foundational vision model.", "weaknesses": "As strong as these observations are, the reviewer feels that this work is incomplete. The following are the reasons why:\n\n1. The regularization argument: The authors note in several times including in their conclusions that the more lower-order and identifiable the features in a network are, the higher the generalization performance. One leads to wonder if these lower-oder DoG-like or Gabor-like filters have a regularization effect on the networks leading to higher generalization performance. This tantalizing hint should be explored further, even unto discovering a regularization process.\n\n2. Multiple pre-trained models for same architecture: The authors used pre-trained models on Imagenet for their analysis. There is a need here to make these conclusions stronger, that this effect is not only present on these particular instance of these models but are an inherent property of the architecture. To do so, further analysis need to be performed to demonstrate these observations over multiple training runs on imagenet and also over other datasets of both natural images and also images from other domains. This would make the observations stronger.\n\n3. Initialization/fine-tuning argument: In many a work on fine-tuning strategies such as in distillation, knowledge transfer, continual/incremental learning, observations has been made that so-called higher-frequency features often change and lower-frequency features remain unchanged. It also been noted in several works that fine-tuning mobilenets are harder than fine-tuning \"general purpose\" networks such as ResNets. This is largely the reason why most commercial use-cases involving fine-tuning convolutional networks use non depth-wise convolutional networks as their backbones. This here is a good stage to test this hypothesis and provide analytical evidence for or against the idea that networks that learn lower-order features do not change much during fine-tuning.\n\n4. Well-fit, over-fit, under-fit models: These observations are made at the end checkpoint of these models. There arises a temporal question of when in the process of training do these filters form. It was noted by Fergus et al., that the first layer filters on convolutional networks take the shapes of these DoG-like or Gabor-like filters very early on in training and the latter stages of training only change the high-frequency filters. The author's workbench is the ideal setup to test this hypothesis. Accompanying this question is also that of over-fit and under-fit networks. High-frequency filters are often an indication of overfitting in non-neural network settings, such as in dictionary learning for instance. The authors could deliberately overfit some of these networks and demonstrate the emergence of more of their \"other\" category of filters. \n\nThe reviewer understands that ICLR has a page limit on its work, but also would like to recognize that while this work is strong, there are a lot of actionable directions that could make stronger and more conclusive statements on top of the already well-done analysis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YS42lhDsJ0", "reviewer": "Reviewer_9rPz", "summary": "The authors examine trained filters in depthwise-separable CNNs in order to shed light into learn representation in this type of CNNs.\nPreliminary visual inspection of those filters shows surprising regularity in their patterns and consistency across layers.\nAfter an initial PCA-based exploration of those patterns, the authors devised an auto-encoder based clustering to identify and visualize those patterns, which correspond to DoG functions and their 1st- and 2nd-order derivatives.\nThe authors analyzed those patterns in various depth-wise CNNs trained on ImageNet-1k and ImageNet-22k to study the prevalence of corresponding filters at different layers.", "questions": "See above", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ A novel analysis of CNN filters that focuses specifically on depthwise-separable CNNs.\n+ The analysis results seem novel, robust, and insightful. They demonstrate an important property of depthwise convolution that might make it better suited for future CNN-based architectures than standard convolution.\n+ The insights can help improve the design of depthwise-separable CNNs e.g. w.r.t. kernel size, number of filters per layer.\n+ The insights offer useful parallels to neuroscientific DoG-derivatives models.", "weaknesses": "These are not major weaknesses, but suggestions to generalize and strengthen the results:\n- ImageNet is object centric. It might be useful to analyze the patterns in a model trained on a scene classification (e.g. Places-365) or other scene-centric tasks (e.g. semantic segmentation of CityScapes) in order to find if new patterns emerge.\n  - To appreciate the impact of the dataset, check the examples of the 1st-layer filters reported in https://arxiv.org/abs/2204.08601\n- Have you considered the impact of other elements of convolution? For example, padding in intermediate layers can skew the filters learned if applied in a one-sided way as reported in https://openreview.net/forum?id=m1CD7tPubNy\n  - I think padding-induced skewness might offer explanation for the observed On-Square and Off-Square patterns. They might simply be skewed versions of the On-Center and Off-Center patterns.\n- Have you considered the impact of global pooling? It was shown to have a profound impact of the weights learned and the patterns that emerge within them https://distill.pub/2020/circuits/weight-banding/\n- It would be insightful to compare the reported findings with Deep Continuous Networks (Tomen et al., ICML 2021), where Gaussian derivative functions also emerge in the learned filters and the authors also draw similar biological parallels.\n\n\nLanguage issues:\n- Use \\citet instead of \\cite to avoid interference with the text (e.g. \"natural images Krizhevsky et al. (2012)\")\n- of both regular and DSCs => regular CNNs and DSCs\n- . in => In\n- We focus on [...[, apply [...] => and apply \n- The samples in each category, exhibit => no comma\n- The On-Centre and Off-Centre clusters, show => no comma\n- the recurring patterns we uncovered, arise => no comma\n- we demonstrated the predominant => that predominant\n- KMEANS (in appendix A.2) => K-Means", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)