Finding Discriminative Filters for Specific Degradations in Blind Super-Resolution
Authors: Liangbin Xie
Xintao Wang
Chao Dong
Zhongang Qi
Ying Shan
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: blind super-resolution
network interpretation
degradation prediction

Abstract: Recent blind super-resolution (SR) methods typically consist of two branches, one for degradation prediction and the other for conditional restoration. However, our experiments show that a one-branch network can achieve comparable performance to the two-branch scheme. Then we wonder: how can one-branch networks automatically learn to distinguish degradations? To find the answer, we propose a new diagnostic tool -- Filter Attribution method based on Integral Gradient (FAIG). Unlike previous integral gradient methods, our FAIG aims at finding the most discriminative filters instead of input pixels/features for degradation removal in blind SR networks. With the discovered filters, we further develop a simple yet effective method to predict the degradation of an input image. Based on FAIG, we show that, in one-branch blind SR networks, 1) we could find a very small number of (1%) discriminative filters for each specific degradation; 2) The weights, locations and connections of the discovered filters are all important to determine the specific network function. 3) The task of degradation prediction can be implicitly realized by these discriminative filters without explicit supervised learning. Our findings can not only help us better understand network behaviors inside one-branch blind SR networks, but also provide guidance on designing more efficient architectures and diagnosing networks for blind SR.

Finding Discriminative Filters for Specific Degradations in Blind Super-Resolution
Lianghin Xie 1,2.3
Xintao Wang*3 Chao Dong 4.4
Zhongang Qi
Ying Shan 3
Shenzhen Key Lab of Computer Vision and Pattern Recognition,
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
University of Chinese Academy of Sciences
ARC Lab, Tencent PCG
'Shanghai Al Laboratory, Shanghai, China (lb. xie, chao. dong) (slat ac en
(xintaowang, zhonganggi, yingsshan @tencent
Abstract
Recent blind super-resolution (SR) methods typically consist of two branches, one for degradation prediction and the other for conditional restoration. However, our experiments show that a one-branch network can achieve comparable performance to the two-branch scheme. Then we wonder: how can one-branch networks automatically learn to distinguish degradations? To find the answer, we propose a new diagnostic tool Filter Attribution method based on Integral Gradient (FAIG). Unlike previous integral gradient methods, our FAIG aims finding the most discriminative filters instead of input pixels/features for degradation removal in blind SR networks. With the discovered filters, we further develop simple yet effective method to predict the degradation of an input image. Based on FAIG, we show that, in one-branch blind SR networks, 1) we are able to find very small number of (I%) discriminative filters for each specific degradation; 2) The weights, locations and connections the discovered filters are all important to determine the specific network function. 3) The task of degradation prediction be implicitly realized by these discriminative filters without explicit supervised learning. Our findings can only help us better understand network behaviors inside one-brance brancr blind SR networks, but also provide guidance on designing more efficient architectures and diagnosing networks for blind SR. The codes are available at ht https: / CRt//eeneeeeett///AA.AA...
Introduction
As special category of image Super-Resolution (SR) [11, 6, 22]. blind super-resolution [28, 3, 48] an active research topic towards real-world restoration applications, and has attracted increasing attention. Blind SR aims at reconstructing high-resolution image from its low-resolution counterpart which contains unknown and complex degradations (e.g., blur, noise). Recent deep- lear-len bassed blind SR methods typically consist of two branches: one for degradation prediction and the other for conditional restorations [14, 25, 39], as illustrated Fig. This design incorporates human knowledge about the degradation process [8, 23] and is consistent in classical blind SR methods [28, 11], thus implying certain degree of interpretability.
With the powerful representation ability of deep learning, it is natural to wonder whether a unified one-branch network can effectively address the blind SR problem without the delicate two-branch
Equal contributions. Lianghin Xie is an intern in ARC Lab, Tencent PCG.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

PSNRABB
Degradation
28
Two-branch
One-branch
Prediction
27
Restoration
26
25
Conditional Restoration
24
23
blur
blur-inois
blur
blurtneise
Two-branch Network
One-branch Network
DASR
Figure 1: Leftt: Illustration for two-branch and one-branch blind SR networks. Right: A unified
one-branch network could achieve comparable performance under similar computation budgets for state-of-the-ar blind SR methods DAN [25] and DASR [39].
design from human prior knowledge. Thus, we conduct preliminary experiments on several state-of- the-art methods (25,39) (more details in Sec. 3). and draw the observation that unified one-branch network can achieve comparable results with similar computation budgets under proper training strategies, shown in Fig. 1.
Despite its comparable performance, the one-brance network is more like 'black-box', and curious about its connection with delicate two-branch designs with higher interpretability. Specifi- cally, the phenomenon mentioned above motivates us raise key questions: 1) Could one-branch networks automatically learn to distinguish degradations as what we specially design in two-branch methods? 2) Are there any small sub-network (i.e., set of filters) existing for a specific degrada- tion? For these open questions, we lack systematic investigations and analysis tools to gain further understandings.
In this paper, we make the first attempt to explore the underlying mechanism of blind SR networks. Our key findings number of (at least to I 1%) discriminative filters for each specific degradation (e.g., blur, noise). When mask these discovered filters for specific degradation, the corresponding function is eliminated while functions for other degradations are maintained (as shown in Fig. 2, more details in Sec. 5.2.1). 1). 2) Except for the filter weights, the locations and connections of the discovered filters also play an essential role in the network function for a specific degradation (see Sec. 5.2.2). 3) Based on these discovered filters, we could casily predict the degradation of input images without training in the supervision of degradation labels (see Sec. 5.3). In a word, a unified one- branchrnchhhhhh network is similar to well-designed two-branch network in the working mechanism. The degradation prediction branch and several small sub-networks for various degradations could be automatically learned within the bre-branch network, even is trained without any supervision about degradation distinctions.
follows: 1) In one-branch blind SR networks, are able find very small
To analyze the mysterious blind SR networks and draw the above key findings, we propose a novel Filter Attribution method based on Integral Gradient (FAIG), aiming finding core filters in network that make the greatest contribution to the function of specific degradation removal. Different from previous integral gradient methods [37, 36] that cumulate gradients along paths in the pixel/feature space, our FAIG utilizes paths in the parameter space, which has a clearer meaning in attributing network functional alterations to filter changes (i.e., the changes of filter parameters rather than inputs result in the network function alteration). Built on FAIG, we further develop simple yet effective method to predict the degradation of an input image (see Sec. 4.3).
To summarize, the contributions are fhree-fold. 1) Our findings provide a better understanding of the mechanism under blind SR networks, especially, bringing insightful connections between popular two-branch methods and unified one-branch networks. 2) The discovered discriminative filters for specific degradations allow not only perform degradation prediction, but also achieve a controllable adjustment of restoration strength without introducing extra parameters. 3) Exploiting the interpretability of blind SR would bring great significance for future works i) designing more efficient architectures; ii) diagnosing an SR network, such as determining the boundary of network restoration capacity and improving algorithm robustness.
2 Related Work
The super-resolution field has witnessed variety of developments since the pioneer work of
SRCNN [6, 7]. Different architectures are proposed, such as deeper networks [17, 19, 15], residual blocks [20, 22, 51, 43, 41], recurrent architectures [18, 38], and attention mechanism [50, 4, 24, 27].

---

Input image
Network output
Mask debluruun filters Mask denoising filters
Bluryy input
Noisy input
Figure 2: For a blurry input (①) and noisy input (@), the one-branch SRResNet [20] for blind SR could remove blur (2) and noise (@), respectively. When mask the 1% deblurring filters (discovered by the proposed FAIG), the corresponding network function of deblurring is eliminated (3) while the function of denoising is maintained (.). Similar phenomenon happens (@ and @) when we mask the 1 % denoising filters in the same network. Zoom for best view.
Most of these methods unified one-branch networks and achieve excellent performance SR with a fixed downsampling (e.g., bicubic) kernel.
Recently, blind SR has attracted increasing attention to solve real-world restoration problems with complicated degradations. Classical blind SR methods typically incorporate human understanding of degradation process [8, 23]. and are divided into two stages: degradation estimation (especially blur kernels) [28] and super-resoluticn based on the estimated degradation [11, 45]. Deep-learning-bee -based methods inherit this design and develop two-branch blind SR networks. KernelGAN [3] leverages the internal distribution of image patches to estimate blur kernel. DASR [39] learns degradation
representation and prediction with contrastive learning. SRMD [48] and [49, 46] incorporate these estimated degradations into CNN networks for restoration. IKC [14] and DAN [25] further propose to jointly perform degradation prediction and restoration in iterative manner. Recent works also attempt to address blind SR within a one-branch networks [40, 47J. There are few related works investigating the relationship between two-branch and one-branch networks for blind SR. Moreover, the underlying mechanism behind blind SR is still mysterious.
Network interpretation methods for high-level visions be divided into two categories: gradient- based methods [44, 33, 35, 34, 1, 32] 32] and perturbation-base approaches [9, 52, 30, 31j. The latter usually localizes the discriminative image regions by performing perturbation to the input neurons [10]. They heavily depend on lots of samplings and resources to guarantee performance. The most relevant gradient- rradient-based method to ours is Integrated Gradients (IG) [37, 36]. which accumulates gradients at all points along a path from the baseline input to target input. IntInf [21] and neuron conductance [5] further extend feature-importunn IG to neurons. Unlike these methods that cumulate gradients along paths in the pixel/feature space, our FAIG utilizes paths in the parameter space for attributing network functional alterations to filter changes. Recently, Gu and Dong [13] introduce interpretation method with local attribution maps to SR networks, while we further try to uncover the underlying mechanism in blind SR networks.
Interpolation in the network parameter space has been used to explore the generalization and flat property of loss landscapes [12, 16]. It also employed image manipulation for continuous imagery effect transition [42]. In this work, we utilize the paths in parameter space (with interpolation) find discriminative filters for specific degradations.
3 Preliminary
Blind SR aims to restore high-resolution images from low-resolution ones with unknown and complex degradations, e.g., blur, noise and downsampling. The classical degradation model [8] is usually adopted to synthetic the low-resolution input. Generally, the ground-truth image r is first convolved with Gaussian blur kernel k. Then, downsampling operation with scale factor is performed. Finally, the low-resolution x is obtained by adding white Gaussian noise n::
gt + k)) 4r +n.
(I)

---

Table 1: PSNR (on RGB channels) comparisons between two-branch and one- one-branch networks on blind SR.
DAN [25]
DASR [39]
PSNR (dB)
blur
blur+noise
blur
blur+noise
Official two-branch
26.168=00.00 26.182-+0.011
27.341+0.072
27.518+0.034
25.1166:0.012
SRResNet one-branch
27.288+0.02
27.573++.010
25.143-+0.013
Recent blind SR methods including IKC [14], DAN [25] and DASR [39] typically adopt a two-branch design, shown in Fig. 1. We then compare the two-branch design with commonly-used one-brande networks on two state-of-the-ar methods: DAN and DASR. For a fair comparison, all experiments conducted with their corresponding officially released codes, and the settings remain unchanged except the network structures. Specifically, SRResNet without batch normalization [20] is employed as the one-branch network. We adjust the number of residual blocks to match similar computation budget (FLOPs and parameters) to two-branch methods.
We compare them on the blur and blur+noise settings (more specifications are in the supp. material). The PSNR comparisons are shown in Tab. and Fig. is observed that a unified one-branch network can also achieve comparable results under similar computation budgets on both settings. This interesting observation motivates us to investigate the underlying mechanism of blind SR networks, especially one-branch networks. It is worth noting that do not claim that one-branch networks always better than two-branch designs. The two-branch methods that incorporate domain-specinf knowledge may have advantage in training stability and robustness. In this paper, this observation is our primal motivation for interpreting blind SR networks.
Integrated Gradient (IG) method [37, 36] is usually used in high-level tasks (e.g., classification network). It attributes the most important input components (e.g., pixels in input images) that affect the network predictions. Suppose we have a deep classification network F IRd + [0, 1]. Let E I be the input image and I E I be the baseline input. 5 is usually black image or blurry images, The IG method accumulates the gradients all points along straight-line path in TRd from the baseline to the input x. Formally, the integrated gradient along the 8th dimension for an input and baseline B defined as:
aF ax da.
(z) = (x. - Iz))
x
(2)
o==
Ōz;
where xy is its 8th pixel value and #F(x) De, is the gradient of F along the 8th dimension x. Each
change in the input component adds up to the final prediction difference Σ, IG,(2) F(x) F()).
Intuitively, the function varies from a less informative value (produced by the baseline 7) to its final informative value. The gradients of F with respect to the image pixels explain each step of the variation the value of F. The integration over the path gradients cumulates these small attributions and accounts for the difference of network outputs. In other words, the IG could interpret deep networks by finding out the most important input components (or features) that explain the network predictions.
4 Method
4.1
Filter Attribution Integrated Gradients
Our goal is reveal the underlying mechanism of one-branch blind SR networks and answer the question: how can one-branch networks automatically learn to distinguish degradation. Specifically, we aim find small set of core filters" that could explain the network functions of specific degradation removal.
Different from the IG method that attributes the network output changes to the inputs or feature, we propose Filter Attribution Integrated Gradients (FAIG) to attribute network functional alterations to filter changes. Inspired by IG, our key idea is that: the baseline network is a pure SR network that cannot remove any degradations, while the target network is re- re-trained network that can deal with complex degradations. Given the same input, the changes of the network output can be attributed to the changes of network parameters (i.e., filters).
Let F(6.z)) .r) be the blind SR network with parameters 0 and input v. We quantify the network function of degradation removal by:
C(0,z) || x) - x
PI this paper, we define each K x K weight in convolutional layer filter, where K is the kernel size.

---

where n is the ground-truti of I. L(0,2) measures the distance between the network output and the bround-truthh Lower distances indicate a stronger degradation removal function.
Let (e)) a E [0 be a continuous path between the baseline model and the target model, satisfying ((1) 8, 7() = 0. For input image I, then attribute the changes of network functions the changes of filters with path integrated gradient as follows:
[(0,x) x) - L(0, [(7(1),)) - (()(0),
(4)
= =Σ al(n(a),
7(a); da, da
Ōr(a);
Then, the 1th dimension (i.e., different network parameters) of the FAIG could be defined as:
(L(7(a),3)
A(a): da
FOI
(5)
Jam 0イ(a);
da
As described in [29, 13), the choice of baselines and path functions affect the final attribution results. We now present the choice for the baseline model and integrated path.
Baseline model. A good baseline model should have the following properties: 1) The baseline should represent the 'absence' the desired function, Le., not having the corresponding degradation removal function in our case. 2) The output of baseline models should also be an image with the same content as the input, otherwise, the discovered filters may capture degradation-irreleve
changes. 3) The baseline model is better to locate in a smaller neighborhood around the target model, that no drastic changes will happen and fewer core filters will be found.
To satisfy the above properties, we propose fine-tuning strategy to construct such a pair of models. Specifically, we first train common SR model for the bilinear downsampling kernel as the bascline model F(0), which could not resolve ve degradations such as blur and noise. Then, we fine-tune it to obtain F(0) for downsampling, blur and noise together, as the target model.
Integrated path. Instead of cumulating gradients along paths in the pixel/feature space, our FAIG utilizes paths in the parameter space. A direct yet effective path is straight-line path between F()) and F(0). Formally, the path could be represented by r(a)) = al + (1 a)00, where 7(1) = 0.
Thus, the Eq. can be re-written as follows. In practice the integral can be approximated via discrete points uniformly sampled along the path.
aL(7(a).
F0,
10--0,
0),
da
(6)
07((a)
(()==0ax
al(7(a),
N 0 0
07(a)
((())=++axx (0000)00000000) k/NN
where N is the total steps in the approximation of the integral, and empirically set it to 100.
Discussions. Comparing with directly finding filters with the largest changes of absolute values. Given the baseline model and the target model, one may propose to directly calculate the absolute value changes of parameters to determine the most important filters for the network functional alteration, i.e., 10 - 0 However, such method does not consider the influence of network outputs, thus the filter changes may result in other dimensions transition, instead of the degradation removal function. Moreover, it cannot discover different filter sets for various degradations inside one model.
Difference from Intinf [21] and neuron conductance [5]. IntInf and neuron conductance are integral gradient methods that measure feature importance with respect to inputs and filters together. Yet, they still alter the input from a baseline to the input hand x, and then calculate gradients with respect to filters along this path in the input space. Therefore, their gradient calculation cannot directly attribute function alteration to filters. We show the superior performance of our method in Sec. 1.2.
5

---

4.2 Finding Discriminative Filters for Specific Degradation
The above FAIG capable of finding important filters for one degradation, but it still has two limitations: 1) the discovered filters do not guarantee to be only responsible for this degradation. Specifically, the filters found for the deblurring function may also have functions for other degrada- tions. 2) Eq. 6 is calculated for single input image, while we want to find discriminative filters that are only correlated with degradations but not image contents.
Therefore, we further propose two improvements. 1) The intuitive idea of finding filters only responsible for one degradation is suppress gradients to other degradations while retaining the interested gradients. Thus, for the tth parameter, we calculate the gradient difference between specific degradation of interest D and other degradations D. 2) We average all the gradient difference in whole dataset to eliminate the impact of image contents. Formally, we have:
FAIG, (0)
IFAIG,(O, ZEX
E [A TEA
X
attribution for degradtion
attribation other degradtions
where DD denotes specific degradation while 'D denotes the other degradations: X' is the dataset that we are used for calculating FAIG. We then sort the obtained FAIG? (0) descending order, and get the top filters by certain percentage. Those discovered filters are the discriminative filters for specific degradation D.
4.3 Degradation Prediction
Based the discovered filters, further develop simple yet effective method to predict the degradation an input image. Specifically, let (filter") be the set of discovered filters for degradation 'D (found by Eq.7), and let (filter") be the set of discovered filters for input image (found by Eq.6) Then, we calculate the overlap score (OS) to measure the intersection the two sets of filters:
OS(2, D) = [[filter P) (filterr)
(8)
[filttr")]
Higher overlap score OS, D) represents that the degradation of input image is more similar to the degradation D. Thus, we are able to predict the degradation of input images by comparing the (x, D) with a threshold TD for different degradations. The threshold could be determined by validation dataset.
Experiments
5.1 Implementation Details
Network architecture. Our analyses are conducted on two representative network architectures: 1) SRCNN-style style [7] with nine convolutional layers except that the upsampling is the end of the network; 2) SRResNet [20] without batch normalization layers. Our FAIG method could perform well on both of these networks. Networks with more complicated components such attention modules [50,4 4, 24] are left as future work.
Degradations. As the first work that interprets blind SR networks, mainly consider the bilinear downsampling with a scale factor of 2, Gaussian blur with a sigma of 2, and Gaussian noise with a level of 0.1. We use Eq. and they are applied with probability of 0.5.
generate degraded images. Blur and noise are applied independently,
Datasets. We train and fine-tune models the DIV2K training dataset [2], which consists of 800 high-resolution images. The analyses are conducted on Set14 [45], BSD100 [26], and DIV2K validation set (100 images) [2]. All these datasets commonly used in SR and licensed for research purposes.
Training details. We first train network with common bilinear downsampling to obtain the baseline model. Then, we fine-tune it to obtain the target model that could solve bilinear downsampling, blur and noise together. The initial learning rate for fine-tuning is set to 5 10-5 The optimization is

---

Table 2: We compare the performance drop with other methods. For blurry (noisy) inputs, we mask the
corresponding debluring (denoising) filters. Larger values indicates a large performance drop. Test on Set14.
(10 (10")
mask 1% discovered filters
mask 5% discovered filters
Input
FAIG (ours)
IG
10 8-
Random
FAIG (ours)
IG
10 9
Random
Blurry imuge
6.68:10.33
1.31
0.18:0.13 0.49=0.10
0.0710.01 0.0410.01
7.53=0.24
6,41
2.16=0.61
0.55++0.32
Noisy image
6.62:10.54
4.22=00.4
16.2843.88
8.01
1.25
0.19
performed with Adam optimizer for 100K iterations. Training batch size is set to 16 and training patch size of ground-trunt -ruth images is set to 64 x 64. All the training and analyses are performed with PyTorch on NVIDIA V100 GPUs in an internal cluster.
We also run each experiment and analysis for three different random seeds and report error bars in our results.
5.2 Evaluations and Analyses
As the filters in SR networks closely relate to each other, simply deleting or inserting filters will introduce severe artifacts outputs. Considering that, in FAIG, there are two strongly connected models: a target model F(0) and a baseline model F(O), that locate closely in the parameter space. The former could effectively remove degradations while the latter cannot. Such nice properties could be utilized to inspect the function alteration of networks. Based on this, we propose two ways to verify the effectiveness of finding discriminative filters for specific degradations by FAIG (see Sec. 5.2. and Sec. 5.2.2). We further analyze the distribution of discovered filters in Sec. 5.2.3.
5.2.1 Mask Discovered Filters
We measure the importance of discovered filters by replacing them with the filters in the baseline model (at the same locations). After substitution, we could obtain a new model. Then, given different degraded images (e.g., blurry noisy images) inputs, we inspect the model outputs. If those discovered filters play an important role in removing a specific degradation, the new substituted model will lose such function and could not remove the corresponding degradation. We name this method as "masking' the functionality of discovered filters.
Thus, could quantify the discovered filters' contribution to the network function by measuring output differences (e.g., MSE) of target model F(0) and the substituted model. order to reduce the effects of brightness and contrast changes, calculate output differences on image gradients of their gray counterparts. A large difference represents a large performance drop caused by the replacement of discovered filters, implying that those filters have strong effects a specific network function.
The results of masking discovered filters with different proportions are shown in Fig. 3. We can draw the following observations. 1) When we mask FAIG-discovered filters for deblurring (even very small portion), the performance for deblurring drops drastically (1) while the function of denoising is maintained (2) at small portions and then has a slow performance drop for most proportions, and finally has a quick decline for very large proportions. We are more interested in the small proportion with discriminative filters, they are more informative for interpretation. It is also similar to mask filters for denoising (3 and +). 2) If we randomly select a portion of filters to mask, the network shows similar performance drops for all cases, implying that the randomly selected filters are non- discriminative. 3) For the small proportion (e.g., 1%), the randomly selected filters could not have a large impact network function alteration, obviously shown in the illustrative images. Note that Fig. 3 shows the results of SRResNet, and SRCNN-style networks have similar phenomenons (see supplementary materials).
We also compare our FAIG with other methods: modified IG (i.e., IntInf [21]), absolute values of filter changes (00 0),, and random selection. We measure the MSE error of image gradients as above. As shown Tab. 2, our FAIG can discover filters that result in larger performance drop, i.e., discover more important filters for corresponding degradations.
5.2.2 Re-train only Discovered Filters
With the method of masking filters, we could conclude that the weights of discovered filters are essential for network functions. We are also curious whether the locations and connections of the discovered filters are important. Thus, we further examine the FAIG-discovered filters in another direction, i.e. from the baseline model, we re-train only the discovered filters for blind SR. Specifically,
7

---

Blurry input
Noisy input
xig
100
Mask cleb deblurring filters
How to Do Everything
How to Do Everything with
prrformanc drop
Performance drop
1488 mrape
with
(AAU)
BAX!
How to Do Everything with
How to Do Everything with
Propertion
Proportinnll
*:10"
Noisy input
Blurry input
Mask denoising filters
How to Do Everything with
How to Do Everything with
performance drop
gal ofiireg gra
performennt mormmmenee drop
How to Do Everything with
How to Do Everything with
1459
Prooortion masked filters
Proportion of masked fikees
Figure 3: Results of masking discovered filters (by FAIG and random selection) with different proportions (on SRResNet). The shown images are produced by 1% mask. Zoom in for best view.
Table 3: Results of re- re-training baseline models with 1% filters for deblurring and denoising. Test on Sct14.
PSNR(dB) Input
Re-truin 1% filters for deblurring
Re-train 1% filters for denoising
Upper bound
FAIG
IG
[0 A
Random
FAIG
IG
|0 0 6
Random
29 293203
27.889
26.389
26.444
26.691
27.642
26.534
26.444
26.668
Hlurry
(+0.021)
(+:0,207)
(+00274)
(+0.097)
(#0.092)
(+:0.007)
(=:0.125)
(=0.096)
(#:0.126
26.712
25.268
25.211
25,288
25.239
25,743
25,141
25.275
25.204
Noisy (1:0.008)
(4:0.035)
(1:0.005)
(+0.044)
(#0.004)
(#:0.033)
(10.116)
(=0.035)
(400016)
1) we first find the discriminative filters for a degradation D on the target model, and record the locations of those filters. 2) Re-train the baseline model on the desired degradation. Note that only re-train the corresponding filters with the same locations in the baseline model, while keeping other filters unchanged.
We compare the performance (measured by PSNR) of the re-trained models. These models are different in the ways of finding such discriminative filters - 1) FAIG, 2) modified IG (i.e. IntInf [2 [2, 3) absolute values of filter changes, ie.,, 0 0). 4) random selection. We also include the upper bound that re-train all parameters of the baseline model.
As shown in Tab. 3, when we only re-train deblurring filters for the deblurring task, our FAIG has large performance gain (than random selection). Yet, the performance of FAIG is close to that of random selection when we only re-train denoising filters for the deblurring task. This implies that the locations and connections of discovered filters also have discriminative characteristics for specific degradations. It is also interesting to observe that re-traiming filters found by other methods leads to inferior (or similar) performance than random selection. We conjecture that the locations and connections discovered by those methods may hinder the re-training process, which requires further investigation in future work.
The above observations has shown that the weights, locations and connections of the discovered filters are all important to determine the network function for specific degradation.
5.2.3 Distribution of Discovered Filters in a Network
We then delve into the discovered filters for deeper observations. We visualize the discovered I 1% filters for different network functions Fig. 4. observed that the filters for deblurring and denoising have very different distributions inside the network. The deblurring filters are more located in the back part of the network while denoising filters locate more uniformly. We conjecture that the deblurring operation requires larger receptive field for equivalent 'deconvolution' kernel. Moreover, their complementary distribution also implics the division of labor in a network.
8

---

I ayer Index
Layer Index
Filter in Layer
Deblurring
Discovered Denoising
Figure 4: Visualization of the discovered I % filters (yellow lines) in a blind SRResNet network for deblurring and denoising. (Yellow lines are thickened for better visualization while not affecting the distribution). It is observed that the networss for deblurring and denoising have very different distribution inside the network, implying their division of labor. Zoom in for best view.
Rlurry input
Noisy input
Ourput target
FATO gradient elimination
FAIG gradient elimination
without
gradient climination
gradicnt elimination
Figure 5: Importance of gradient climination for other degradations. We mask 1% deblurring filters in this comparison. Zoom in for best view.
5.3 Degradation Classification
We adopt Set14 to discover filters for a specific degradation D. The thresholds are then calculated on the BSDI00 dataset. Finally, the degradation prediction is performed the DIV2K dataset. We Thoise and hhue to 0.6 and 0.5, and the prediction accuracy can reach 98% and 96%, respectively (more results are in the supp. material). Therefore, the task of degradation prediction can be implicitly realized by these discriminative filters without explicit supervised learning.
5.4 Importance of Gradient Elimination for Other Degradations
In Sec. 4.2, we introduce an improvement of gradient elimination for other degradations. Such design is important to find discriminative filters for a specific degradation. The comparison is shown Fig. 5. mask 1 % deblurring filters in this comparison. For the blurry input, both models can produce expected results (i.e., lose the deblurring function). While for the noisy input, FAIG without gradient elimination for other degradations could also lose denoising function, indicating that discovered filters not discriminative. In contrast, our FAIG with gradient elimination for other degradations could also maintain its ability to denoising.
Limitations
Our work has several limitations. 1) The setting of considered degradations in our analyses is simple one in blind SR. It only considers one discrete level for each degradation. Future work should take more complex and practical degradations into consideration. 2) Our analyses are conducted in SRCNN-styl and SRResNet. However, recent progress network architectures such as attention modules is necessary to analyze in blind SR. 3) There is still gap to apply our findings to design more efficient networks and diagnose networks for blind SR. We believe that our investigation is inspiring and the above limitations should be addressed in future works.
Conclusion
In this paper, we make the first attempt to investigate the mechanism underlying the unified one- branch blind SR network. propose new diagnostic tool Filter Attribution method based on Integral Gradient (FAIG) that utilizes paths in the parameter space in attributing network functional alterations to filter changes. Based on the proposed FAIG, we are able to find very small number of (at least to 1 1%) discriminative filters for specific degradations in blind SR. We also show that

---

the weights, locations and connections of the discovered filters are all important to determine the specific network function. Moreover, the task of degradation prediction can be implicitly realized by these discriminative filters without explicit supervised learning. Our findings can help us better understand network behaviors inside one-branch blind SR networks. We believe that exploiting the interpretabilityy of blind SR would bring great significance for future works in designing more efficient architectures and diagnosing an SR network, such as determining the boundary of network restoration capacity and improving algorithm robustness.
7
Acknowledgements
This work is partially supported by National Natural Science Foundation of China (61906184) the Science and Technology Service Network Initiative of Chinese Academy of Sciences (KFJ-STS- QYZX-092), the Shanghai Committee of Science and Technology, China (Grant No. 21DZ1 21D22100110)) 100100).
References
[1] Julius Adchayo, Justin Gilmer, Ian Goodfellow, and Been Kim. Local explanation methods for deep neural networks lack sensitivity to parameter values. arXiv preprint (eXXiv/810003300) 2018.
[2] Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge single image super-resolutien Dataset and study. In CVPRW. 2017.
[3] Sefi Bell-Kligler, Assaf Shocher, and Michal Trani. Blind super-resolution kernel estimation using an internal-gan. In NeurlPS, 2019.
[4] Tao Dai, Jianrui Cai, Yongbing Zhang, Shu- Tao Xia, and Lei Zhang, Second-order attention network for single image super-resolution. In CVPR, 2019.
[5] Kedar Dhamdhere, Mukund Sundararajan, and Qiqi Yan. How important is neuron? In NeurlPS, 2018.
[6] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Learning deep convolutional network for image super-resolution. In ECCV, 2014.
Chao Dong, Chen Change Loy. Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. 38(2):295-307 2016.
[8] Michael Elad and Arie Feuer. Restoration of single superresolution image from several blurred, noisy, and undersampled measured images. IEEE transactions on image processing, 6(12):1646-166 1997.
[9] Ruth C Fong and Andrea Vedaldi. Interpretable explanations of black boxes by mcaningful perturbation. In ICCV, 2017.
[10] Amirata Ghorbani and James Zou. Neuron shapley: Discovering the responsible neurons. In Neur!PS, 2020.
[11] Daniel Glasner, Shai Bagon, and Michal Trani. Super-resolution from single image. ICCV, 2009.
[12] Ian J Goodfellow, Oriol Vinyals. and Andrew M Saxe. Qualitatively characterizing neural network optimization problems. In JCLR, 2015.
[13] Jinjin Gu and Chao Dong. Interpreting super-resolution networks with local attribution maps. In CVPR, 2020.
[14] Jinjin Gu, Hannan Lu, Wangmeng Zuo, and Chao Dong. Blind super-resolution with iterative kernel correction. In CVPR, 2019.
[15] Muhammad Haris, Greg Shakhnarovich, and Norimichi Ukita. Deep backprojection networks for super-
resolution. In CVPR, 2018
[16] Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep lcarning: Generalization gap and sharp minima. In ICLR, 2017.
[17] Jiwon Kim. Jung Kwon Lee, and Kyoung Mu Lee. Accurate image super-resolution using very deep convolutional networks. In CVPR, 2016.
[18] Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. Deeply-recursive convolutional network for image super-resolution. CVPR, 2016.
[19] Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming- Ming-Hsuan Yang. Deep laplacian pyramid networks for fast and accurate super-resolution. In CVPR, 2017.
[20] Christian Ledig, Lucas Theis, Ferene Huszár. Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz. Zehan Wang. et al. Photo-realistic single image super- resolution using a generative adversarial network. CVPR, 2017.
[21] Klas Leino, Shayak Sen, Anupam Datta, Matt Fredrikson, and Linyi Li. Influence-directed explanations for deep convolutional networks. In 2018 IEEE International Test Conference, pages 1-8. TERE, 2018.
[22] Bee Lim, Sanghyun Son, Heewon Kim. Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. In CVPRW, 2017.
10

---

[23] Ce Liu and Deqing Sun. On bayesian adaptive video super resolution. IEEE transuctions puttern analysis and machine intelligence, 36(2):346-360, 2013.
[24] Ding Liu, Bihan Wen, Yuchen Fan, Chen Change Loy, and Thomas S Huang. Non-local recurrent network for image restoration. In NeurlPS, 2018.
[25] Zhengxiong Luo,, Yan Huang. Shang Li. Liang Wang, and Tieniu Tan. Unfolding the alternating optimiza- tion for blind super resolution. NeurlPS, 2020.
[26] David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In
ICCV, 2001.
[27] Yiqun Mei, Yuchen Fan, Yuqian Zhou, Lichao Huang, Thomas S Huang, and Honghui Shi. Image super-resolution with cross-scale non-local attention and exhaustive celf-exemplars mining. In CVPR, 2020.
[28] Tomer Michaeli and Michal Irani. Nonparametric blind super-resolution. In CVPR, 2013.
[29] Scott Lundberg Pascal Sturmfels and Su-In Lee. Visualizing the impact of feature attribution baselines. In Distill, 2020.
[30] Vitali Petsiuk, Abir Das, and Kate Sacnko, Rise: Randomized input sampling for explanation of black-bor box models. arXiv preprint 1oXXiv:806.007553 2018.
[31] Zhongang Qi, Saeed Khorram, and Fuxin Li. Visualizing deep networks by optimizing with integrated
gradients. Proceedings of the AAAI Conference on Artificial Intelligence, 34:1 14:1118001118 1189,, 04 2020. [32] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV, 2017.
[33] Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. Not just a black box: Learning important features through propagating activation differences. arXiv preprint arXi::605.001133
2016.
[34] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad:
removing noise by adding noise. arXiv preprint (tt:::7700033880 2017.
[35] Jost Tobias Springenberg. Alexey Dosovitskiy. Thomas Brox. and Martin Riedmiller. Striving for simplicity:
The all convolutional net. arXiv preprint 14 2014.
[36] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International
Conference on Machine Learning, pages 13288 PMLR, 2017.
[37]
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Gradients of counterfactuals. ICLR, 2017.
[38]
Ying Tai, Jian Yang, and Xiaoming Liu. Image super-resolution via deep recursive residual network. In
CVPR, 2017.
[39] Longguang Wang, Yingqian Wang, Xiaoyu Dong, Qingyu Xu. Jungang Yang. Wei An, and Yulan Guo. Unsupervised degradation representation learning for blind super-resolution. In CVPR, 2021.
[40] Xintao Wang. Lianhhin Xie, Chao Dong, and Ying Shan. Real-esrgam: Training real-world blind super- resolution with pure synthetic data. In International Conference on Computer Vision Workshops (ICCWW). [41] Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Recovering realistic texture in image super- resolution by deep spatial feature transform. In CVPR, 2018.
[42] Xintao Wang, Ke Yu, Chao Dong, Xiaoou Tang, and Chen Change Loy. Deep network interpolation for continuous imagery effect transition. In CVPR, 2019.
[43] Xintao Wang. Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu. Chao Dong, Yu Qiao, and Chen Change Loy.
Esrgan: Enhanced super-resolution generative adversarial networks. In ECCVW, 2018.
[44] Matthew Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In ECCV, 2014.
[45] Roman Zeyde. Michael Elad, and Matan Protter. On single image scale-up using sparse-repreeentation..
In International Conference Curves and Surfaces. Springer, 2010.
[46] Kai Zhang, Lue Van Gool, and Radu Timofte. Deep unfolding network for image super-resolution. In CVPR, 2020.
[47] Kai Zhang. Jingyun Liang, Luc Van Gool, and Radu Timofte. Designing practical degradation model for deep blind image super-resolution. arXiv preprint arXiv:2103. /4006, 2021.
[48] Kai Zhang, Wangmeng Zuo, and Lci Zhang. Learning single convolutional super-resolution network for multiple degradations. In CVPR, 2018
[49] Kai Zhang, Wangmeng Zuo, and Lei Zhang. Deep plug-and-play super-resolution for arbitrary blur kernels.
In CVPR. 2019.
[50] Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. Image super-resolution using very deep residual channel attention networks. In ECCV, 2018.
[51] Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, and Yun Fu. Residual dense network for image
super-resolution. In CVPR, 2018
[52] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Object detectors emerge in deep scene cnns. arXiv preprint arXiv: 141..6866..6 2014.
[53] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In CVPR, 2016.
11


Reviews:
[{"review_id": "tTU3_hQBy5d", "reviewer": "Reviewer_dqh1", "summary": "This paper proposes a Filter Attribution method based on Integral Gradient (FAIG) to analyze the behaviors of blind super-resolution networks on different degradations. It experimentally proves that a one-branch blind SR network is similar to a well-designed two-branch one in the working mechanism. The degradation prediction branch could be automatically learned in a unified one-branch network. These findings are interesting and can help us better understand the behaviors of blind SR networks.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well-written and easy to read. \n\nThe finding that a one-branch blind SR network can achieve comparable performance with a well-designed two-branch one is exciting. The proposed FAIG is also interesting, and the adequate experiments also demonstrate the effectiveness of FAIG in discovering discriminative filers.\n\nHowever, the reason why the one-branch network can be comparable with the two-branch one is still not well explained.  With the help of FAIG, we can know which filters are more important to the specific degradation, but still do not know why they do. It is better to give a further explanation of how the discovered filters work in removing the corresponding degradations.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. The overlap score OS(x, D)  can help in measuring the similarity of the degradation of x and D. However, the degradation space seems to be continuous and there should be an infinite number of different degradations. How do you predict the degradation by calculating OS(x, D)? I mean that there is an infinite number of D, and you can not calculate all of them to choose the largest one. \n\n2. Is it possible to use your findings to help design a better blind SR network? It will be great if you can provide some ideas.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}, {"review_id": "oobZHJBv-pr", "reviewer": "Reviewer_fvKT", "summary": "Most existing blind super-resolution methods use a two-branch network for degradation prediction and conditional restoration. In this paper, authors show that a one-branch network can also automatically learn to distinguish degradations for blind SR with the proposed novel filter attribution method based on integral gradient (FAIG). By using FAIG, they could find 1% filters as the most discriminative filters for each specific degradation in the one-branch blind SR networks. Experimental results show that these selected discriminative filters do have big influence on the corresponding degradation, while having a minor impact on the other degradation. This finding is promising since it may help to design more efficient architectures for blind SR for future work.\n\n  \n\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "However, there are still several issues to be addressed.\n1. Authors claim that the proposed FAIG method is able to find a small number of filters as the most discriminative filters for degradation removal in blind SR networks. However, only one setting is validated, i.e., the bicubic downsampling with a scale factor of 2, Gaussian blur with a sigma of 2, and Gaussian noise with a level of 0.1. What about the experiments on other settings, like changing the scale factor, the sigma, or the noise level? By only using this same setting for all experiments may not be able to prove that the FAIG would work for various degradations. \n\n2. One important advantage about using the two-branch network for SR is that they provide an opportunity for training a network which can generalize well on images with multiple degradations, even though these degradations are unseen during training. For example, the two listed two-branch methods DAN[27] and DASR[41] directly apply their pretrained networks on some low-resolution images from real world to show their ability for generalization.\nHow would this finding be helpful for improving the ability for generalization of the one-branch network, especially on those LR images from real world which may be difficult to separate the blurry part and noisy part from the images?   \n \n3. In Figure 3 (4), when masking a smaller number of the denoising filters (<20%) for blurry input, it seems that the random is better than the proposed FAIG, is there any particular reason for that?  \n\n4. It is not clear how to determine how much proportion of filters should be selected in advance and obtain close performance as the upper bound.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, they have addressed the limitation and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5 hour"}, "extra_text": {}}, {"review_id": "hkBRre8uzfF", "reviewer": "Reviewer_vM65", "summary": "Existing methods for blind super resolution have recently adopted a design consisting of two branches where one branch is responsible for predicting (or representing) the degradation and the other branch is responsible for reconstruction while conditioning on the degradation information of the first branch. This paper highlights that a single network with one branch can achieve a similar performance as the two-branch approaches and analyses this in detail. More specifically, the authors design a filter attribution method to find most discriminative filters as opposed to most discriminative input pixels or features. This analysis also allows the development of a simple predictor for degradation.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While early works in upscaling have focussed on fixed degradations, more recent upscaling work that considers the blind setting has helped making super resolution more practical and flexible. However, to my knowledge it has not yet been explored to what extent the solutions deployed in the blind methods are strictly required and in which way a single network can achieve similar performance. As such I find this paper to provide an interesting and fairly unique contribution and I certainly enjoyed reading the paper.\n\nThrough the analysis of filter attribution, the method highlights and dissects how certain filters of an otherwise 'black box' network exhibit a given functionality. It is an interesting insight, that even without any form of supervision about degradation distinctions, it can be observed that small sub-networks emerge that are responsible for different degradations.\n\nThe paper clearly lists limitations, which is first and foremost that it considers a toy example in blind SR. It only allows a single discrete level for each degradation and the degradations (blur and noise) do not exist at the same time in a given image. \n\nNevertheless, I think this is an interesting direction to gain more understanding on and in follow up work, it would be interesting to analyse the behaviour for a wider range of continuous degradation settings. In terms of related work, in the space of blind super resolution [CORNILLÈRE et al., Blind Image Super-Resolution with Spatially Variant Degradations, Siggraph Asia 2019] could be cited.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are clearly described and I don't expect negative societal impact of this work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2-3 hours"}, "extra_text": {}}, {"review_id": "bcjL5NCCHWv", "reviewer": "Reviewer_Ub7A", "summary": "This paper aims to analyze blind SR models and gain a deeper understanding on how a one-branch CNN functions with different degradations. After empirically showing that a one-branch CNN performs on par with two-branch CNNs that are commonly employed in blind SR, the authors propose a filter attribution method based on integral gradients and make important key observations that a small number of discriminative filters ($\\sim$1%) can be found for different degradations in one-branch models. ", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n* This paper presents a simple idea for interpreting filters in deep networks that may otherwise be black-box modules. \n* Papers focusing on network interpretability are often geared towards high-level vision tasks such as image classification. However, this paper handles blind SR, and with strong empirical evidence that few discriminative filters exist for different degradations (blur and noise), the readers can gain a deeper understanding into how the internal filters are working for image restoration CNNs. I believe the presented analyses will be a great contribution to the low-level vision community.\n* The visual results are very impressive and it can be directly observed that discriminative filters exist for different degradations.\n* Quantitative comparisons to other analytical methods by masking and re-training 1% filters show that the proposed method based on filter attribution can find more discriminative filters.\n* The authors present abundant results (visual results, comparisons, ablation studies, analyses) in the main paper and the supplementary material.\n* The paper is clearly written and provides many insights on blind SR models.\n* The authors discuss reasonable limitations of their work.\n\nSuggestions/questions:\n* For one-branch backbones, network depth (number of residual blocks) is adjusted to match the computational complexity of two-branch networks. Therefore, in Table 1, one-branch models may perform better due to the increased depth. How about increasing the network width / the number of output channels (e.g. 64 channels to 256 channels) to match complexity? Do the authors think results would be similar in this case? It would be more convincing to show results of one-branch models with increased network width as well.\n* Urban100 and Manga109 are also widely used testsets in SR. Adding results on these two datasets would make the analysis more complete.\n* In Table 3, more discriminative filters can be found for deblurring than denoising with the proposed method compared to the next best method. Any intuitions why?\n* The original DAN model is trained iteratively. Is the one-branch DAN also trained iteratively?\n* Minor: Fig. 2 could be mentioned in Section 5.2.1 to support its claims.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations of their work in Section 5.5. I agree with the authors that societal impact is N/A as their work focuses on interpreting blind SR models.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers are enthusiastic about the paper's findings and gave it high scores. The use of one-branch instead of two for blind super resolution is interesting, and one reviewer was particularly interested in the analytical aspect of the paper. 
Decision: Accept (Spotlight)