P. Gavrikov - Analysis of Trained Convolution Filters - Deep Learn Summer School, Gran Canaria 2022
Channel: Paul Gavrikov
Date: 02/08/2022

Convolutional Neural Networks (CNNs) can be successfully applied to solve many computer vision problems including but not limited to classification, segmentation, and image generation. Additionally, CNNs can process many images from various visual domains like natural photos, but also depth maps, thermal images, medical x-ray images, and others. Surprisingly, the success of the commonly practiced pretraining of models is not limited to adjacent image datasets. Which in turn must mean that there are some fundamental similarities. To investigate these potential similarities, and rather than analyzing saliency or feature maps of CNNs, we explore their learned convolution filters on a large, heterogeneous model zoo of CNNs trained for different tasks, on different datasets originating from various image domains, and based on different architectures. Our results show that the distributions of learned filters are surprisingly largely independent of the learned task, source domain, or even the filter position in the model layer hierarchy, yet, filter distributions are inherently dependent on the architecture [1]. Additionally, our findings show that common off-the-shelf models tend to form “degenerated” filters when the architecture is overparameterized for the task, which wastes capacity and potentially hinders models from transferring well. Lastly, we investigated the impact of adversarial training which is commonly applied to increase the robustness of CNNs, and we find that these models form highly distinct early filters regardless of their generally higher filter diversity [2].
___________________________________________________________
[1] Paul Gavrikov, Janis Keuper, “CNN Filter DB: An Empirical Investigation of Trained Convolutional Filters”, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[2] Paul Gavrikov, Janis Keuper, “Adversarial Robustness through the Lens of Convolutional Filters”, IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2022.
___________________________________________________________
Connect with me:
Web: www.pgavrikov.com
Email: paul.gavrikov@hs-offenburg.de
Twitter: @paulgavrikov