Sauron U-Net: Simple automated redundancy elimination in medical image segmentation via filter pruning
Authors: Juan Miguel Valverde
Artem Shatillo
Jussi Tohka
Venue: NeurIPS 2022 Submitted (2022)
Keywords: medical image segmentation
model compression
filter pruning
convolutional neural networks

Abstract: We present Sauron, a filter pruning method that eliminates redundant feature maps by discarding the corresponding filters with automatically-adjusted layer-specific thresholds. Furthermore, Sauron minimizes a regularization term that, as we show with various metrics, promotes the formation of feature maps clusters. In contrast to most filter pruning methods, Sauron is single-phase, similarly to typical neural network optimization, requiring fewer hyperparameters and design decisions. Additionally, unlike other cluster-based approaches, our method does not require pre-selecting the number of clusters, which is non-trivial to determine and varies across layers. We evaluated Sauron and three state-of-the-art filter pruning methods on three medical image segmentation tasks. This is an area where filter pruning has received little attention and where it can help building efficient models for medical grade computers that cannot use cloud services due to privacy considerations. Sauron achieved models with higher performance and pruning rate than the competing pruning methods. Additionally, since Sauron removes filters during training, its optimization accelerated over time. Finally, we show that the feature maps of a Sauron-pruned model were highly interpretable. The Sauron code is publicly available at https://github.com/blindedrepository.

Sauron U-Net:
Simple automated redundancy elimination in medical image segmentation via filter pruning
Anonymous Author(s)
Affiliation
Address
email
Abstract
We present Sauron, filter pruning method that eliminates redundant feature maps by discarding the corresponding filters with automaticalll--ajjstteo layer- specific thresholds. Furthermore, Sauron minimizes a regularization term that, as we show with various metrics, promotes the formation of feature maps clusters. In contrast most filter pruning methods, Sauron is single-phase similarly typical neural network optimization, requiring fewer hyperparameters and design decisions. Additionally, unlike other cluster-based approaches, our method does not require pre-selecting the number of clusters, which is non-trivial to determine and varies across layers. We evaluated Sauron and three state-of-the- art filter pruning methods on three medical image segmentation tasks. This an area where filter pruning has received little attention and where it can help building efficient models for medical grade computers that cannot use cloud services due privacy considerations. Sauron achieved models with higher performance and pruning rate than the competing pruning methods. Additionally, since Sauron removes filters during training, its optimization accelerated over time. Finally, we show that the feature maps of Sauron- pruned model were highly interpretable. The Sauron code is publicly available at ahttps: :estps://giiiee //github. wtw.:/hhw..commiinnneeeeeeee http//ddddddeesitooo...
2
3
10
11
12
14
15
16
17
Introduction
19
Pruning is the process of eliminating unnecessary parameters to obtain compact models and accelerate their inference. There are two main strategies for pruning convolutional neural networks (CNNs): weight pruning and filter pruning. In weight pruning, weights for unimportant connections are zeroed without consideration of the network structure, leading, in practice, sparse weight matrices 12 1O II 40). On the other hand, filter pruning methods eliminate CNNs filters directly. Thus, unlike weight-pruned pruned models, utilizing filter- networks efficiently requires no specialized hardware or software 77 35]. Most pruning methods have been developed or evaluated exclusively for natural image classification. Other tasks, such as medical image segmentation, have received significantly less attention [32]. In medical imaging, small models can enable computationallly-Imin
20
21
22
24
25
26
27
28 medical grade computers to segment medical images that cannot be uploaded cloud server due privacy reasons. Moreover, models with a few filters can be easier interpret than large models, which is erucial not only in clinical applications but also in research. Motivated by these possibilities, propose a filter pruning method called Sauron that generates small CNNs. We demonstrate its
29
Submitted to 36th Conference Neural Information Processing Systems (NeurIPS 2022). Do distribute.

---

32
application to prune U-Net-like networks (36], bringing together filter pruning and medical image
38
segmentation.
34
Sauron applies filter pruning during optimization single phase, while most filter pruning frame- works consist of three distinct phases: Pre-training the model, pruning its filters, and fine-tuning to compensate for the loss of accuracy (or re-training from scratch [27 ()). Other approaches combine resulting in two-phase frameworks, and 29 Single-phase filter pruning methods 48). such as Sauron, are advantageous since they require fewer hyperparameters and design decisions, including the number of epochs for training and fine-tuning, pruning iterations, or whether to combine pruning with training or fine-tumin tine-tuning. In particular, Sauron does not insert additional parameters into the optimized architecture identify filter candidates for pruning, such as channel importance masks This avoids potential optimization hindrance and requires less extra training time and GPU memory.
35
36
37
pruning with training 46 48
13
38 or fine-tuning [29 26
38
other methods repeat these phases multiple times 46
39
40
41
6 29
15
26 16].
44
45
Sauron facilitates and promotes the formation of feature map clusters by optimizing a regularization
46
term, and, unlike previous cluster-based approaches [| [3] Sauron does not enforce the number of these clusters. Since these clusters vary depending on the training data and across layers, the optimal number of feature maps per cluster is likely differ. Thus, determining the number of clusters not trivial and may limit the accuracy and the pruning rate.
47
50
Our specific contributions are the following:
51
We introduce Sauron, single-phase filter pruning method that resembles the typical CNN optimization, making it easier to use, and that does not add any additional parameters to the optimized architecture.
52
58
We show that Sauron promotes the formation of feature map clusters by optimizing regularization term.
58
We compare Sauron other methods on three medical image segmentation tasks, where Sauron resulted in more accurate and compressed models.
57
58
We show that the feature maps generated by a model pruned with Sauron were highly interpretable.
59
80
We publish Sauron and the code to run all our experiments at https://github. com/ blindedrepository
61
62
2
Previous work
63
Filter importance Most filter pruning approaches rely on ranking filters to eliminate the unimpor- lant filters. The number of climinated filters can be determined by either fixed [3] or an adaptive threshold [38]. Filter importance can be found via particle filtering 3 or it can be computed via
64
65
66
28 or post-pruning accuracy 1. Pruning methods can include extra terms in the loss function, such as group sparsity constraints, although these extra terms guarantee no sparsity in CNNs [45). Other methods aim learn filter importance by incorporating channel importance masks into CNNs' architectures (6 29 [5 26 [6].
heuristic relying on measures such as Lp norms 23
44
38 entropy
67
88
70
However, these adjustments modify the architectures to be optimized, increasing the required GPU memory during training, optimization time, and potentially hindering the optimization. Alterna- tively, other methods consider the scaling factor of batch normalization layers as channel importance but in e.g. medical image segmentation, batch normalization is occasionally replaced by other normalization layers due to the small mini-batch size [8]
71
72
73
45 48),
74
75
Difference minimization Methods that remove filters while trying to preserve characteristics loss [46), and the feature maps 30 of the original unprumed models. A disadvantage of these methods is that they require
78
such classification accuracy (27). Taylooeexaasioo-ppprroimia
47
42
44
large GPU memory to avoid loading and unloading the models in memory constantly, which would

---

Algorithm Sauron
Input: training data: D.
1: Given: A, maximum threshold Tmmax epochs, percentage of pruned filters μ4, patience p, number
of steps K.
2: Initialize: model's weights W + ( - }), layer-specific thresholds 7 + { =
3: for e = l= epochs do
4:
for b= 1; S N do # Mini batches
5:
Compute predictions D, loss L. Sopt (Eq. 2
prune (Bq.
3
# Forward pass # Backward pass
6: Update 0
7:
end for
8:
for 1
1; S do
# Pruning step
9:
## Procedure I: Increasing Ti ##
C1: Training loss is converging; C2: Validation loss improving
11:
C3: Less than μ% of filters pruned in (e 1); C4: Ti has not increased in last p epochs (CI C2 A C3 C4) A (TI Tmas) then Tmaz/K
13:
71 end if
15:
## Procedure 2: Pruning ##
16:
W (W' d' Ti)
17:
end for
18:
end for
Output: Pruned CNN.
79 slow down the training. Furthermore, since finding the appropriate filters for their elimination is Na-hard, certain methods resorted to selecting filters based on their importance 46 or
80
81
genetic (27 or greedy [30) algorithms.
Redundancy elimination Approaches, including Sauron, that identify redundant filters by com- puting similarity metric among all 43 39 or within clusters of filters/feature maps [3 9
84
Previously, cluster-based approaches have considered redundant those within-cluster filters near the Euclidean center [9] and median [13,, or filters with similar L, norm over several training epochs [4]. disadvantage these approaches an extra "number of clusters" hyperparameter, which is data dependent and the same hyperparameter value might be optimal across layers. Other methods have used Pearson's 's correlation between the weights [3] between the feature maps (39) within the same layer, and feature maps' rank (25) to indicate redundancy. although, their computations more expensive than utilizing distances as in cluster-based methods.
85
86
87
88
3 Sauron
82 In this section, we present our approach to filter pruning, which we call Simple AUtomated Redundancy eliminatiON (Sauron). Sauron optimizes, jointly with the loss function, a regular- ization term that leads to clusters of feature maps at each convolutional layer, accentuating the redundancy of CNNs. It then eliminates the filters corresponding to the redundant feature maps by using automatically-adjusti layer-specific thresholds. Sauron requires minimal changes from the typical neural network optimization since prunes and optimizes CNNs jointly, i.e., training involves the usual forward- wearrd-bckwwar backward passes and pruning step after each epoch. Moreover, Sauron does not integrate optimizable parameters, such as channel importance masks 6
93
94
95
36
98
89
15 26 16], into the CNN
100
architecture. This avoids complicating the optimization task and increasing the training time and the required GPU memory. Algorithm summarizes our method.
101
102
3.1
Preliminaries
Let D = (x, ) represent the training where x; denotes image i, y; its corresponding segmentation, and N is the number of images. Let w x be the weights, composed

---

105 by 91418; filters of size k x k at layer I, where 8141 denotes the number of output channels, St the number of input channels, and is the kernel size. Given feature maps O E RaxAxx of h x w image dimensions, the feature maps 0 E xxxw at the next layer are computed as
106
107
o a (Noorm(W O')),
108
where # is the convolution operation, Norm is normalization layer, and a an activation function. For simplicity, we omit the bias term in Eq. Q and we include all CNN's parameters in (W) W where L is the number of layers. We denote the predicted segmentation of the image by ÿs-
109
112
3.2
Forward pass
113
Sauron minimizes a loss L consisting of Cross Entropy LCE, Dice loss D
31 and novel
114
channel distance regularization term óopt: L LCB + DDice + Adopts where
1 L
Sopt
(2)
1=1 S1+1
À is a hyperparameter that balances the contribution of óopt. and denotes average pooling with window size and strides w. Before computing dept- feature maps O and O',, (all channels ex- cept the first) are normalized to the range [0, 1] via min-max normalization, as we experimentally found this normalization strategy to be the best (see Appendix A). For pruning, Sauron com-
116
117
118
119 putes distances between a randomly-chosen feature map π E {l,..., ,1411 and all the others: I + l,..,,,.,............ S1+1}, where prune /dt// max, d, l I..... L, I......TT .T 1,
120
d, = [000O, (O,):
121
Importantly, æ is different in every layer and epoch, enabling Sauron to prune different feature map clusters. Moreover, since finding an appropriate pruning threshold requires the distances to lie within known range, Sauron normalizes d" such that their maximum 1, i.e. d² + d' mxxx()))).
122
123
124
3.3
Backward pass: Sapt regularization
Optimized CNNs have been shown to have redundant weights and to produce redundant feature maps (Appendix E). By minimizing the extra regularization term copt. CNNs further promote the formation of clusters, facilitating their subsequent pruning. Sopt regularization makes those feature maps near the feature map in the first channel Of (i.e., within the same cluster) even closer. At the same time, those feature maps that are dissimilar to O (i.c., in other clusters) become more similar to other feature maps from the same cluster, as holds that [|o(O)::) ((O): w)]l2 S - - via cpp regularization- an upper bound of the left hand side. We demonstrate this clustering effect in Section 4.2 Furthermore, for pruning, we focus on the feature maps rather than on the weights since different non-redundant weights can lead similar feature maps. Thus, eliminating redundant weights guarantees reduction in feature maps redundancy.
13
43
127
128
129
130
+ [lx(O]: (OO); w)l22 for i + j. i.e. the right hand side miie-minniized
131
132
133
134
135
3.4
Pruning step
Sauron employs layer-specific thresholds = (f....,,77 TI.). where all are initialized to zero and increase independently (usually at different pace) until reaching Tmaz- This versatility is important as the ideal pruning rate differs across layers due to their different purpose (i.e., extraction of low- and high-level features) and their varied number of filters. Additionally, this setup permits utilizing high thresholds without removing too many filters at the beginning the optimization, as feature maps may initially lie close to each other due to the random initialization. In consequence, pruning is embedded into the training and remains always active, portraying Sauron single-phis phase filter pruning method.
138
139
140
141
142

---

145
Procedure 1: Increasing Pruning with adaptively increasing layer-specific thresholds raises Two important questions: how and when to increase the thresholds? Sauron increases the thresholds linearly in R steps until reaching Tmaar Then, thresholds are updated once the model has stopped improving (C1 and C2 in Algorithm [T and has pruned only a few filters (C3). An additional "patience" hyperparameter ensures that the thresholds are not updated consecutively (C4). Conditions
146
147
148
149
150
C1, ,.... C4 are easy to implement and interpret, and they rely on heuristics commonly employed for
detecting convergence.
152
Procedure 2: Pruning Sauron considers nearby feature maps to be redundant since they likely belong to the same cluster. In consequence, Sauron removes all input filters W ..t whose corre- sponding feature map distances 5prune are lower than threshold 77- In contrast to other filter pruning methods, Sauron needs to store no additional information, such as channel indices, and the pruned models become more efficient and smaller. Additionally, since pruning occurs during training. Sauron accelerates the optimization of CNNs. After training, pruned models can be easily loaded by specifyyng the new post- pest--puning number of input and output filters in the convolutional layers.
153
154
155
156
157
3.5
Implementation
160
Sauron's is simple design permits its incorporation into existing CNN optimization frameworks easily. As an example, in our implementation, convolutional blocks are wrapped into a class that computes Sopt and prune effortlessly in the forward pass, and the pruning step is a callback function triggered after each epoch. This implementation, together with the code for running our experiments and processing the datasets, was written in Pytorch [33] and is publicly available at /ttttps://gittub www.bnddddeessittttooo
181
162
163
164
165
In our experiments, we utilized an Nvidia GeForce GTX 1080 Ti (IIGB), and a server with eight Nvidia A100 (40GB).
186
187 4 Experiments
168
In this section, we compare Sauron with other state-of-the-art filter pruning methods and conduct an ablation study to show the impact on pruning and performance of dopt regularization. We empirically demonstrate that the proposed ōapt regularization increases feature map clusterability, and we visualize the feature maps of a Sauron-pruned model.
170
171
172
We employed three 3D medical image segmentation datasets: Rats, ACDC, and KiTS. Rats comprised 160 3D T2-weighted magnetic resonance images of rat brains with lesions [41] and the segmentation task was separating lesion from non-lesion voxels. We divided Rats dataset into 0.8:0.2 train-test splits, and the training set was further divided into a 0.9:0.1 train-validation split, resulting in 115, 13, and 32 images for training, validation, and test, respectively. ACDC included the Automated Cardiac Diagnosis Challenge 2017 training set (5) (CC BY-NC-SA 4.0), comprised by 200 3D magnetic resonance images of 100 individuals. The segmentation classes were background, right ventricle (RV), myocardium (M), and left ventricle (LV). We divided ACDC dataset similarly to Rats dataset, resulting in 144, 16, and 40 images for training, validation, and test, respectively. We only utilized ACDC's s competition training set due to the limitation to only four submissions to the online platform of ACDC challenge. Finally, KiTS was composed by 210 3D images from Kidney Tumor Challenge 2019 training set, segmented into background, kidney and kidney tumor [4] (MIT). split, resulting in 183 and 21 images for
Datasets
173
174
175
177
178
179
180
181
182
KITS training set was divided into a 0.9:0.1 training and validation. We report the results on the KiTS's competition test set (90 3D images). All 3D images were standardized to zero mean and unit variance. The vain-validation-tes divisions and computation of the evaluation criteria was at the subject level, ensuring that the data from a single subject was completely in the train set or the test set, never dividing subject's data between train and test sets. See Appendix C for preprocessing details.
185
186
187
188
189
Model and optimization Sauron and the compared filter pruning methods optimized nnUNet [18 via deep supervision [22] with Adam 20) starting with learning rate of 10-3, polynomial learning
5

---

Table 1: Performance on Rats
Table 2: Performance on ACDC dataset.
dataset.
Bold: best performance among pruning methods.
Method
Lesion
LV
Dice
HD95
Dice
HD95
Dice
HD95
Dice
HL245
nnUNet
0.94 = 0.03
1.1 0.3
0.91
+.0.5
4.4
3.0
0.90
+0002
3,4
+ 5.8
0.95
00003
2.5
+ 1.8
Sauron
0.94
0.03
0.90
0.06
4.7
1.3.
0.90
1 0.02
3.6
1.8
0.95
0.03
2.7 1 2.0
Sauron (A = 0)
0.93
= 0.03
1.2 0.5
0.89
0.08
5.3
4.4
0.90
0.02
2.4
1.7
0.95
0.00
3.1
: 3.0
cSGD 0.5)
0.86
0.13
9.6
+ 16.8
0.10
0.15
72.6
1.74.
0.54
0.19
19.5
1. 35.6
0.64
0.20
13.9
1. 8.2
FPGM (r 0.5)
0.93
0.04
0.5 0.5
0.57
0.13
37.8
+ 7.3
0.89
: 0.03
2.2
=
1.6
0.00
= 0.00
194.1
+ 23.5
Autopruner
0.91 0.04
0.8
0.88
+ 0.07
5.9
4.66
0.88
+ 0.03
2.5
1.7
0.95 0.03
3.1
- 3.0
Table 3: Performance on KiTS
Table 4: Decrease in FLOPs with respect to the baseline nn UNet.. Bold: highest decrease.
datasets.
Kidacy
Tomor
Method
Rais
ACDC
KITS
Method
Dice
Dice
nnUNet
0.00%
0.00%
0.00%
nnUNee T7
Sauron
0,9595
0.9564
0.7657
0.7482
Sauron (A
Sauron
cSGD (r 0.5)
0)
96.62%
96.45%
50.03%
92.41%
89.04%
49.80%
49.81%
93.02 %
85.82%
eSGD
Sauron (A 0)
0.5)
0.9047
0.9556
0.5207
0.7352
FPGM L3
0.5)
50.00%
50.0%
49.98%
HPGM 13 (r 0.5)
0,9509
0.6830
Autopruner
83.61%
88.52%
82.00%
Autopruner
0.9167
0.5854
192
rate decay, and weight decay of 10-5. During training, images were augmented with TorchIO
193
34 (see Appendix C). nnUNet is a celf-connig U-Net and the dataset optimized nnUNet architectures slightly differed on the number of filters, encoder-decoder levels, normalization layer, batch size, and number of epochs (see Appendix C).
194
195
196
Sauron decreased feature maps dimensionality via average pooling with window size and stride of w and utilized À 0.5 in the loss function, maximum pruning threshold Tmoa 0.3, pruning steps x 15, and patience & = 5 (C4 in Algorithm []. Additionally, we employed simple conditions to detect convergence for increasing the layer-specific thresholds
Pruning
197
198
Convergence in the training loss (C1) was detected once the most recent training loss lay between the maximum and minimum values obtained during the training. We considered that the validation loss stopped improving (C2) once its most recent value increased with respect to all previous values. Finally, the remaining condition (C3) held true if the layer-specific threshold pruned less than 2% of the filters
200
201
202
203
204
pruned in the previous epoch, i.e., μ = 2.
205
4.1
Benchmark on three segmentation tasks
We optimized and pruned nnUNet [8] with Sauron, and we compared its performance with eSGL
208
207
29 using a pruning rate similar to the one achieved by Sauron. Since cSGD and FPGM severely underperformed in this setting, re-run them with their pruning rate set to only 50% (r 0.5). Additionally, to understand the influence of the proposed regularization term Sppt on the performance and pruning rate, we conducted ablation experiments with À = 0. We computed the Dice coefficient [8] and 95% Hausdorff distance (HD95) (37] on Rats and ACDC | and 2}. In KiTS dataset, only the average Dice coefficient was provided by
91. FPGM2 [13], and Autopruner
210
211
212
test sets (see Tables the online platform that evaluated the test set (see Table 3 addition to Dice and HD95, we computed the relative decrease in the number of floating point operations (FLOPs) in all convolutions: FLOPs = HW (WC,,, Cout) where H, W is the height and width of the feature maps, Cins Cout is the number of input and output channels, and K is the kernel size. For the 3D CNNs (KITS dataset), an extra D (depth) and K are multiplied to compute the FLOPs.
213
214
217
216 Sauron obtained the highest Dice coefficients and competitive HD95s across all datasets and segmen- tation classes (Tables to Sauron also achieved the highest reduction in FLOPs, although, every
219
https:/github.ccmm/// http://hhth...ooo///n///nn//n//a
P hhups://githubbbb bttps///gituub.om//////////

---

Epoch
Epoch 200 (withzut Sipe?
Epoch 200 (with 6go 0.5)
Dip-test vakce during training
Wirroct dur
0.20
0:35
Other duannels Firt crannel
0.00
0.00
1.50
150
200
Egrchs
Dip test trend
Number of reigh bers trend
9)
Oge value trend
h) increase trend different metrics
Widlod m0.))
Aumbir of cone
5
trusese
Decrrase
Sindar
Desreeds
5mm
Decroase
Simikrr
Figure 1: a-c) ISNE plot dec block__k feature maps at initialization (epoch 0), and after optimizing with and without dopt- Corresponding dip-test values during the optimization. e-g) Summary of the trends across the three clusterability measures in all convolutional layers. h) Number of layers with an increasing trend the three clusterability measures with higher values of À (dashed line: Sauron's is default configuration).
220
method, including Sauron, can further reduce the FLOPs at the risk of worsening the performance (Table 4. cSGD and FPGM could not yield models with high pruning rates possibly because they aim at reducing only 8 and not 8; from W E 0g*t+1X04 xxxx-xxxkxxxx xk Thus, very high pruning rates cause great imbalance between the number of input and output filters in every layer that may hinder the training. Note also that eSGD and FPGM were not tested with pruning rates higher than 60% p K In contrast, Sauron and Autopruner that achieved working models with higher pruning rate reduced both input filters 8; and output filters $141-
221
222
223
226
227
Sauron without the proposed regularization term dept (Sauron (A = 0)) achieved similar or less compressed models and worse Dice coefficients than when minimizing 5opt- Overall, the results from these ablation experiments indicate that typical CNN optimization (without dopt regularization) yields redundant feature maps that be pruned with Sauron, 2) pruning rate is generally higher with dopt regularization, and 3) pruning with no dopt regularization can affect performance, possibly due to the accidental elimination of non-redundant filters. In summary, the pruning rate and performance achieved our ablation experiments demonstrate that promoting clusterability via 8ept regularization advantagcous for eliminating redundant feature maps.
228
229
232
233
234
4.2
Minimizing 8apt promotes the formation of feature maps clusters
236
We investigated feature map clustering tendency during nnUNet's optimization. For this, we deacti- vated Sauron's pruning step and optimized on Rats dataset with and without dopt while storing feature maps each epoch (including epoch O, before the optimization) of every convolutional layer. Since quantifying clusterability a hard task, we utilized three differrntt measures: 1) We em- ployed dip-test [19] Adolfsson et al. demonstrated its robustness compared to other methods for quantifying clusterability. High dip-test values signal higher clusterability. 2) We computed the average number of neighbors of each feature map layer-wise. Specifically, we counted the feature maps within r, where r corresponded to the 20% of the distance between the first channel and the farthest channel. Distance r is computed every time since the initial distance between feature maps typically reduced while training. An increase in the average number of neighbors indicates that
237
238
239
242
243
244
245
246
feature maps have become more clustered. 3) We caleulated the average distance to the first feature
map channel (i.e., dept) for each layer, which illustrates the total reduction of those distances achieved during and after the optimization.
7

---

249
In agreement with the literature [13 43 Figure ! shows that optimizing nnUNet (without Sopt regularization) yields clusters of feature maps. Feature maps in layer "dec_ @iee_block_"" (see Appendix B) show no apparent structure suitable for clustering at initialization (Fig. D a), and, at the end of the optimization, feature maps appear more clustered (Fig. | b). Figure | (d. blue line) also illustrates this phenomenon: dip-test value low in the beginning and higher at the end of the training. However, this increasing trend did not occur in all layers. To illustrate this, we compared, for each layer, the average dip-test value, number of neighbors, and distance Supt in the first and last third of the training. Then, we considered the trend similar if the difference between these values was smaller than 0.001 (for the dip-test values) or smaller than 5% of the average value in the first third (for the number of neighbors and distance Sapt). Figure | (e) shows that the number of layers in which the dip-test value increased and decreased were similar when not minimizing the Sopt regularization term. In contrast, the number of layers with an increasing trend was proportionally larger with Sapt regularization. Figure ( shows a similar outcome regarding the average number of neighbors, i.e. 5 optt regularization led to proportionally more neighbors near each feature map. In the same line, the average distance between the first feature map and the rest decreased more with Sapt regularization
250
251
252
253
254
256
257
258
259
260
261
262
263
(Fig. (f)). Additionally, Figure | (c) also illustrates that incorporating the Sapt regularization term
264
265
enhances the clustering of feature maps, as there are more clusters and the feature maps are more
286
clustered than when not minimizing Sopt (Fig. (b)).
We observed higher clusterability in the convolutional layers with more feature maps (see Appendix D) This is likely because such convolutional layers contribute more to the value of dopt (Eq. 2 On the other hand, convolutional layers with fewer feature maps have larger feature vectors (e.g.. enc enc_blocee bloc_lloc__ feature vectors are (256 x 256) x 32 in Rats dataset) whose distances tend to be larger due to the curse of dimensionality. Sauron accounts, to some extent, for these differences in the convolutional layers with the adaptively-incrrasinng layer-specific thresholds т. Another possible
269
270
271
272
273
way to tackle these differences is by using different layer-specific X's to increase the contribution of the distances of certain layers. We investigated the impact on feature map clusterability with higher
274
À values and, illustrated in Figure I (h), higher À tended to increase the average number of neighbors, decrease cept, and somewhat increase the dip-test values, which, overall, signals higher
276
277
clusterability.
4.3
Feature maps interpretation
279
Sauron produces small and efficient models that can be easier to interpret. This is due to öept regularization that, as we showed Section 4.2, increases feature maps clusterability. Each feature maps cluster can be thought of as a semantic operation and the cluster's feature maps as noisy outputs of such operation. To test this view, we inspected the feature maps from the second-to-last convolutional block fiec_block__ 8, see Appendix B) of bauron-pruned unet. For comparison, we included the feature maps from the same convolutional layer of the baseline (unpruned) nnUNet in Appendix E.
280
281
282
285
286
The first feature map depicted Figure 2 (top) captured the background and part of the rat head that does not contain brain tissue. The second feature map contained the rest of the rat head without brain lesion, and the third feature map mostly extracted the brain lesion. Although the third feature map seems suffice for segmenting the brain lesion, the first feature map might have helped the model by discarding the region with no brain tissue at all. Similarly, the first and second feature maps in Figure 2 (middle) detected the background, whereas feature maps 3, 4, and 5 extracted, with different intensities, the right cavity (red), myocardium (green), and left cavity (blue) of the heart. In Figure 2 (bottom), we can also see that each feature map captured the background, kidney (red), and tumor (blue) with different intensities. This high-level interpretation facilitates understanding the role of the last convolutional block which, in the illustrated cases, could be replaced by simple binary operations. This shows the interpretability potential of feature map redundancy elimination methods such as Sauron.
287
288
291
282
293
294
295
8

---

nap
Image Ground truth
Feature map 1
Feature map
Feature map 3
Feature map
Feature map 5
Figure 2: Image slice from Rats (top), ACDC (middle), and KiTS (bottom) datasets, its ground-truth segmentation, and all feature maps at the second-to-last convolutional block after pruning with Sauron.
5
Conclusion
298
299
We presented our single-phase phase filter pruning method named Sauron, and we evaluated it on three medical image segmentation tasks which Sauron yielded pruned models that were superior to the compared methods in terms of performance and pruning rate. In agreement with the literature, our experiments indicated that CNN optimization leads redundant feature maps that be clustered. Additionally, introduced Sauron's 's dopt regularization that,
300
301
302
showed with three different clusterability metrics, increased feature maps clusterability without pre-selecting the number of clusters, unlike previous approaches. In other words, we enhanced CNN's 's innate capability to yield feature maps clusters via oopt regularization, and we exploited it for filter pruning. Finally, we showed that the few feature maps after pruning nnUNet with Sauron were highly interpretable.
305
306
307
Limitations and potential negative impact Sauron relies feature maps for identifying which filters to prune. Thus, although Sauron is suitable for training models from scratch and fine-tuning pre-trained networks, Sauron is unable to prune CNNs without access to training data, unlike Furthermore, Sauron cannot enforce a specific compression rate due to its simple distance thresholding. Although we have evaluated Sauron with respect to the segmentation quality, we are not able to evaluate the potential clinical impact. could be that even small difference in segmentation would have large clinical impact, vice versa, large difference in segmentation could be clinically meaningless. Depending the application these impacts could be either positive or negative.
310
311
312
313
References
318
[1] Reza Abbasi-Asl and Bin Yu. Structural compression convolutional neural networks. arXiv preprint arXiv:1705.073:: 2017.
319
[2] Andreas Adolfsson, Margareta Ackerman, and Naomi C Brownstein. To cluster, not to cluster: An analysis of clusterability methods. Pattern Recognition, 88:13-26. 2019.

---

322
[3] Sajid Anwar, Kyuyeon Hwang, and Wonyong Sung. Structured pruning of deep convolutional neural networks. ACM Journal on Emerging Technologies in Computing Systems (JETC), 13 (3)11-88, 2017.
323
324
325
[4] SH Basha, Mohammad Farazuddin, Viswanath Pulabaigari, Shiv Ram Dubey, and Sneha- sis Mukherjee. Deep model compression based the training history. arXiv preprini arXiv:2102.00160, 2021.
326
328
[5]
Olivier Bernard, Alain Lalande, Clement Zotti, Frederick Cervenansky, Xin Yang, Pheng-Ann Heng, Irem Cetin, Karim Lekadir, Oscar Camara, Miguel Angel Gonzalez Ballester, et al. Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: Is the problem solved? IEEE transactions on medical imaging, 37(11):2514-522) 2018.
329
330
331
[6] Jingfei Chang, Yang Lu, Ping Xue, Xing Wei, and Zhen Wei. Ucp: Uniform channel prun- ing for deep convolutional neural networks compression and acceleration. arXiv preprint arXiv:2010.01251, 2020.
333
334
335
[7]
Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El- laniv,, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or- arXiv preprint arXiv: 02830, 2016.
336
337
338
[8]
Lee R Dice. Measures of the amount of ecologic association between species. Ecology, 26(3):
329
297-302, 1945.
340
[9] Xiaohan Ding, Guiguang Ding, Yuchen Guo, and Jungong Han. Centripetal sgd for pruning very deep convolutional networks with complicated structure. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 4943-4953, 2019.
341
342
[10] Song Han, Huizi Mao, and William Dally. Deep compression: Compressing deep neural net- works with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00145 2015.
344
345
346
[11] Song Han, Jeff Pool, John Tran, and William J Dally. Learning both weights and connections for efficient neural networks. arXiv preprint arXiv:1506.02628 2015.
347
[12] Babak Hassibi, David G Stork, and Gregory J Wolff. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks, pages 293-299. IEEE, 1993.
349
350
[13] Yang He, Ping Liu, Ziwei Wang, Zhilan Hu, and Yi Yang. Filter pruning via geometric median for deep convolutional neural networks acceleration. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 4340-4349, 2019.
351
352
[14] Nicholas Heller, Niranjan Sathianathen, Arveen Kalapara, Edward Walezak, Keenan Moore, Heather Kaluzniak, Joel Rosenberg, Paul Blake, Zachary Rengel, Makinna Oestreich, et al. The kits19 challenge data: 300 kidney tumor cases with clinical context, ct semantic segmentations, and surgical outcomes. arXiv preprint 1904.0990 2019.
354
355
356
357
[15] Saihui Hou and Zilei Wang. Weighted channel dropout for regularization of deep convolutional neural network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 8425-8432, 2019.
358
380
[16] Zehao Huang and Naiyan Wang. Data-driven sparse structure selection for deep neural networks. In Proceedings of the European conference on computer vision (ECCV), pages 304-320, 201 88
361
362
[17] Fabian Isensee, Paul F Jaeger, Peter M Full, Ivo Wolf, Sandy Engelhardt, and Klaus H Maier- Hein. Automatic cardiac disease assessment on cine-mri via time-series segmentation and domain specific features. In International workshop on statistical atlases and computational models of the heart, pages 120-129. Springer, 2017.
383
10

---

386
[18]
Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Petersen, and Klaus H Maier-Hein nnu-net: a self-configuring method for deep learning-haa based biomedical image segmentation. Nature methods, 18(2)2033221, 2021.
367
368
369
[19] Argyris Kalogeratos and Aristidis Likas. Dip-means: an incremental clustering method for estimating the number of clusters. Advances in neural information processing systems, 25: 2012.
370
371
372
[20] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69N8 2014.
373
[21] Yann LeCun, John S Denker, and Sara A Solla. Optimal brain damage. In Advances in neural information processing systems, pages 598-605, 1990.
375
[22] Chen- Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu. Deeply- supervised nets. In Artificial intelligence and statistics, pages 562-570. PMIR, 2015.
378
[23]
Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. arXiv preprini arXiv: 000.007000 2016.
379
380
[24] Yuchao Li, Shaohui Lin, Baochang Zhang, Jianzhuang Liu, David Doermann, Yongjian Wu, Feiyue Huang, and Rongrong Ji. Exploiting kernel sparsity and entropy for interpretable enn compression. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2800-2809, 2019.
381
384
[25] Mingbao Lin, Rongrong Ji, Yan Wang, Yichen Zhang, Baochang Zhang, Yonghong Tian, and Ling Shao. Hrank: Filter pruning using high-rank feature map. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1529-1538, 2020.
387
[26] Shaohui Lin, Rongrong Ji, Yuchao Li, Yongjian Wu, Feiyue Huang, and Baochang Zhang. Accelerating convolutional networks via global & dynamic filter pruning. In IJCAI, volume 2, page 8, 2018.
390
[27] Zechun Liu, Haoyuan Mu, Xiangyu Zhang, Zichao Guo, Xin Yang, Kwang-Ting Cheng, and Jian Sun. Metapruning: Meta learning for automatic neural network channel pruning. In Proceedings of the IEEF/CVF International Conference Computer Vision, pages 3296-3305, 2019.
393
[28] Han-HHo Luo and Jianxin Wu. An entropy-based pruning method for enn compression. arXiv preprint 170Xiv1700...m 05791, 2017.
396
[29]
Jian-Hao Luo and Jianxin Wu. Autopruner: An end-to-end trainable filter pruning method for efficient deep model inference. Pattern Recognition, 107:107461, 2020.
398
30 Jian-Hao Luo, Hao Zhang, Hong- Yu Zhou, Chen-Wei Xie, Jianxin Wu, and Weiyao Lin. Thinet: pruning enn filters for a thinner net. IEEE transactions on pattern analysis and machine intelligence, 41(10):2525-25 2538, 2018.
399
401
[31] Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pages 565-571. IEFE, 2016.
402
403
404
[32] Suraj Mishra, Peixian Liang, Adam Czajka, Danny Z Chen, and X Sharon Hu, Cc-net: Image complexity guided network compression for biomedical image segmentation. In 2019 IEEE 16th International Symposium Biomedical Imaging (ISBI 2019), pages 57-60. IEEE, 2019.
405
407
[33] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high beg-pperformanc deep learning library. Advances in neural information processing systems, 32:8026-8037 2019.
408

---

[34] Fernando Sérez-Garcia Rachel Sparks, and Sébastien Ourselin. Torchio: a python library for efficient loading, preprocessing. augmentation and patch-based sampling of medical images in deep learning. Computer Methods and Programs in Biomedicine, page 106236, 2021.
412
413
414
ISSN 0169-2607. doi: http:///o...ggp//00000ooo....ooa
URL https: Sttps://oi www
415
aciencedirect.com wttp:///w....aaaaaaaaaaaaa////aa
[35] Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. In European conference on computer vision, pages 525-542. Springer, 2016.
419
[36] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234-241. Springer, 2015.
420
421
422
[37] Günter Rote. Computing the minimum hausdorff distance between two point sets on a line under translation. Information Processing Letlers, 38(3):123-121 1991.
[38] Pravendra Singh, Vinay Kumar Verma, Piyush Rai, and Vinay P Namboodiri. Play and prune: 2019.
Adaptive filter pruning for deep model compression. arXiv preprint
426
[39]
Xavier Suau, Nicholas Apostoloff, et al. Filter distillation for network compression. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 3129-3138. IEEE, 2020.
427
428
429
[40] Frederick Tung and Greg Mori. Clip-q: Deep network compression learning by in-parallel
430
pruning-quanization. In Proceedings of the IEEE conference on computer vision and pattern recognition,, pages 7873-7882, 2018
[41] Juan Miguel Valverde, Artem Shatillo, Riccardo De Feo, Olli Gröhn, Alejandra Sierra, and Jussi Tohka. Ratlesnet22: fully convolutional network for rodent brain lesion segmentation. Frontiers in neuroscience, 14:1333, 2020.
434
435
[42] Dong Wang, Lei Zhou, Xueni Zhang, Xiao Bai, and Jun Zhou. Exploring linear relationship in feature map subspace for convnets compression. arXiv preprint 10030:290030579,, 2018.
436
437
[43] Wenxiao Wang, Cong Fu, Jishun Guo, Deng Cai, and Xiaofei He. Cop: Customized deep model compression via regularized correlation-based filter-level pruning. arXiv preprint arXiv:1906.10333 10337, 2019.
438
[44] Zihao Xie, Li Zhu, Lin Zhao, Bo Tao, Liman Liu, and Wenbing Tao. Localization-aware channel pruning for object detection. Neurocomputing, 403:400-408, 2020.
442
[45] Jianbo Ye, Xin Lu, Zhe Lin, and James Z Wang. Rethinking the smalle-noom-les--nfoomaiie assumption in channel pruning of convolution layers. arXiv preprint arXh:: 2018.
443
444
[46] Zhonghui You, Kun Yan, Jinmian Ye, Meng Ma, and Ping Wang. Gate decorator: Global filter pruming method for accelerating deep convolutional neural networks. arXiv preprint 190900 174 2019.
445
[47] Ruichi Yu, Ang Li, Chun-Fu Chen, Jui-Hsin Lai, Vlad I Morariu, Xintong Han, Mingfei Gao, Ching Yung Lin, and Larry S Davis. Nisp: Pruning networks using neuron importance score propagation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9194-9203 2018
448
449
450
451
[48] Chenglong Zhao, Bingbing Ni, Jian Zhang, Qiwei Zhao, Wenjun Zhang, and Qi Tian. Varia- tional convolutional neural network pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattrn Recognition, pages 2780-2789, 2019.
452
12

---

Checklist
454
455
1. For all authors...
456
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] Particularly 1) Sauron being a single-phase method (Section 3.4). 2) its outperformance over other methods (Section[41]) 3) the proposed
457
458
459
Sopt increasing feature maps clusterability (Section 4.2). and 4) Feature maps easier to
480
interpret (Section (.3)
481
(b) Did you describe the limitations of your work? [Yes] See Section [
462
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Section S
464
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
485
486
2. If you are including theoretical results...
467
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
468
(b) Did you include complete proofs of all theoretical results? [N/A]
469
3. If you ran experiments...
470
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] The supplemen- tary material and https://githhii run all our experiments. Additionally, it includes README file specifying how to run each experiment.
471
472
www/bliindddeeppeeeoocccoeeeeeee httpb//nnnddeepsttorrr contain the code to
474
475
(b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [Yes] Most important details were described in Section 4 and we specified all data augmentation, architectural and optimization settings in Appendix C. These settings can also be seen in our code.
478
479
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No]
480
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Section 3.5
482
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]] Data: see Section 4
485
"Datasets" paragraph. Code: see Section 4. for the compared methods
486
(b) Did you mention the license of the assets? [Yes] We mentioned the license for ACDC and KiTS datasets (Section 4 "Datasets").
487
488
(c) Did you include any new assets either in the supplemental material or as URL? [No]
409
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
480
491
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
492
483
5. If you used crowdsourcing or conducted research with human subjects...
494
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
495
486
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
498
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
499
13


Reviews:
[{"review_id": "q9guf_9EAjG", "reviewer": "Reviewer_sPLR", "summary": "The authors proposed Sauron, a filter pruning methodology designed for U-Net-like medical image segmentation networks. In comparison to most filter pruning frameworks that consist of more than one distinct phase, Sauron applies filter pruning during optimization in a single phase, making it require fewer hyperparameters and design decisions. To achieve this goal, Sauron facilitates and promotes the formation of feature map clusters by optimizing a regularization term, and does not enforce the number of these clusters. In experiments, Sauron achieves comparable segmentation performance and largely reduced FLOPs on nnUNet.", "questions": "1. Lines 29-30: The statement \"models with a few filters can be easier to interpret than large models, which is crucial not only in clinical applications but also in research.\" needs citations.\n\n2. Table 4 is never referred in the main text.\n\n3. The provided github link does not exist.", "limitations": "The authors adequately addressed the limitations and potential negative societal impact of their work.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The proposed clustering-based filter pruning method sounds interesting and reasonable. The regularization term $\\delta_{opt}$ in Sauron makes those feature maps near the feature map in the first channel even closer. Meanwhile, based on the upper bound in lines 129-130, $\\delta_{opt}$ forces those feature maps that are dissimilar to the feature map in the first channel to be more similar to other feature maps from the same cluster. \n\n2. Sauron performs well on 2D nnUNet, providing comparable performance on Rats and ACDC datasets with largely reduced FLOPs.\n\nWeaknesses:\n\n1. In Tables 1-3, it seems that the regularization term $\\delta_{opt}$ does not have a significant impact on the segmentation performance. For example, Sauron $\\delta_{opt}=0$ performs comparably with Sauron on Rats, ACDC, and KiTS (Kidney). In my opinion, I think these results cannot fully reflect the significance of the introduced regularization term (the major contribution of Sauron).\n\n2. The results of 3D nnUNet on KiTS are not satisfactory, especially on the Tumor class. Actually, I recommend the authors to lay more emphasize on 3D UNets, which often require many more computational resources than 2D UNets. However, most of the experiments were made on 2D UNets, which are not deep and maintain high inference speed (not really in need of filter pruning).\n\n3. The authors stated that Sauron is a single phase pruning method, which is more advantageous since it requires fewer hyperparameters and design decisions, including the number of epochs for training and fine-tuning, pruning iterations, etc. The authors should verify their statements with either experimental results or appropriate citations.\n\n4. The pruning baselines, i.e., cSGD, FPGM, and Autopruner, are somewhat out-of-dated, which were published about three years ago. The authors should consider including more recent (published within two years) filter pruning approaches for fair comparisons.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "mxgTz5X4rOO", "reviewer": "Reviewer_Vhi7", "summary": "The authors proposed a regularization term in the loss for the filter pruning in medical image segmentation tasks. The regularization term minimizes the difference of feature maps from each convolutional layer, which enforces the similarity of all features against the feature map of the first channel. The filter pruning is conducted by removing the filter producing similar features. Three medical segmentation datasets (with 3D volumes) are employed for the experiments, and Superior results of the proposed method are reported in comparison to previous feature pruning methods. I have several concerns about the presented work, mainly regarding the experimental results. ", "questions": "It will be helpful to see the authors' response to my abovementioned concerns.", "limitations": "Good for the presented application.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ The proposed pruning method can achieve better results than prior arts, proving the add-on regularization term effectively promotes feature clustering.\n+ The manuscript is overall easy to follow.\n\nWeakness:\n- The motivation for including the regularization term is not clear to me. For instance, I am not sure why the similarity of features is always enforced between the feature of the first channel and the rest. I did not see how the statement \"... makes those feature maps near the feature map in the first channel O_1 even closer. At the same time, those features that are dissimilar to O_1 become more similar to other feature maps...\" is achieved, especially how are the other clusters formed as in the second part?  \n- The proposed regularization term is rather straightforward, which is not technically innovative.\n- Are the computed FLOPs computed in training or testing? It will be helpful to show the number of filters removed in the model and the exact number of parameters for the adjusted models.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Xpj9J-RO5fJ", "reviewer": "Reviewer_qmVv", "summary": "This paper tackles filter pruning on deep convolutional networks for medical image segmentation. Training loss is augmented with a regularization term that penalizes distances between feature maps and forms clusters, and thus, pruning does not introduce extra training parameters or phases such as pre-training. Feature maps whose distances are below a threshold are then dropped, where the thresholds are adaptive based on layer and epoch. This strategy is used for training the state-of-the-art nn-UNet on 3 benchmark medical image segmentation datasets. The resulting pruned models maintain the segmentation performance of unpruned models and outperform other pruning methods in terms of segmentation performance and inference time.   \n", "questions": "N/A", "limitations": "Limitations and potential impacts are discussed.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: Pruning deep neural networks for medical image segmentation is an important problem due to high dimensionality and long training times. The paper is well-written, method and experiment setup are clearly explained. \n\nWeaknesses: Following recent and similar works are not mentioned in related works/experiments, hindering the clarity of novelty. Particularly, Zhou et al. (2020), Chen et al. (2020) and Dinsdalea et al. (2022) focus on filter pruning for medical image segmentation; it is suggested that the authors compare/contrast their approach with these works. \n\nZhou et al. “Evolutionary Compression of Deep Neural Networks for Biomedical Image Segmentation” (2020)\n\nChen et al. “α-UNet++: A Data-Driven Neural Network Architecture for Medical Image Segmentation” (2020)\n\nDinsdale et al. “STAMP: Simultaneous Training and Model Pruning for Low Data Regimes in Medical Image Segmentation” (2022)\n\nHe et al. “CAP: Context-Aware Pruning for Semantic Segmentation” (2021)\n\nDitschuneit et al. “AUTO-COMPRESSING SUBSET PRUNING FOR SEMANTIC IMAGE SEGMENTATION” (2022)\n\nChen et al. “MTP: MULTI-TASK PRUNING FOR EFFICIENT SEMANTIC SEGMENTATION NETWORKS” (2022)\n\nSabih et al. “DyFiP: Explainable AI-based Dynamic Filter Pruning of Convolutional Neural Networks” (2022)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "PhQCgu1Xj6I", "reviewer": "Reviewer_xHRW", "summary": "The paper proposed a method for pruning filters in image segmentation networks by removing filters during training that are closely clustered. Unlike prior works, the approach is described as single-phase, meaning it prunes during normal training. To promote smaller networks, a term which promotes feature map clustering is added to the loss. The experiments uses nnUNet as a baseline network and 3 other recent network pruning methods as comparison. The results show better performance (Dice/HD95) with more pruned networks.", "questions": "- It seems like the proposed method does have a number of additional hyperparameters. See for instance lambda (l114), the layer-specific thresholds (l137), the patience hyperparameter (l149), the number of pruning steps (l198), etc. Considering that more hyperparameters are mentioned as a weakness of the prior works in contrast to the proposed, the authors should maybe clarify why and if they consider their approach superior in this respect?\n", "limitations": "- Aside from the above limitations, I think the work addresses limitations as aspected.\n", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ Well motivated.\n\n+ clearly written and easy to follow.\n\n- I find significance is hard to asses, as the results and the novel contributions of the present work is no\nt put and discussed in the context of prior work to a high enough degree.\n\n- Having a list of contributions at the end of introduction, I feel is a good way to summarize reasons the a particular manuscript is relevant to read. In this case, however, it repeats too much for my taste. I would\n have preferred a stronger focus on novelty and what separates this work from prior works specifically.\n\n- The section on Previous work hard to follow. I would have preferred more detail and more relevance given the context of the submitted work. As it is, it just reads as a listing of prior approaches, with too little detail to be informative.\n\n- The section on Feature maps interpretation could benefit greatly from having the comparison to the non-pruned nnUnet in the paper and not in the appendix. As it is, it is not possible to assess the claims that the pruning makes the models easier to interpret.\n\n- It is unclear to me if the clustering experiments (Figure 1) were done with multiple training runs. A single experiment where clustering is improved is perhaps not so interesting.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject