Unveiling The Mask of Position-Information Pattern Through the Mist of Image Features
Authors: Chieh Hubert Lin
Hsin-Ying Lee
Hung-Yu Tseng
Maneesh Kumar Singh
Ming-Hsuan Yang
Venue: NeurIPS 2022 Submitted (2022)
Keywords: positional information
position encoding
padding
CNN

Abstract: Recent studies show that paddings in convolutional neural networks encode absolute position information which can negatively affect the model performance for certain tasks. However, existing metrics for quantifying the strength of positional information remain unreliable and frequently lead to erroneous results. To address this issue, we propose novel metrics for measuring (and visualizing) the encoded positional information. We formally define the encoded information as PPP (Position-information Pattern from Padding) and conduct a series of experiments to study its properties as well as its formation. The proposed metrics measure the presence of positional information more reliably than the existing metrics based on PosENet and a test in F-Conv. We also demonstrate that for any extant (and proposed) padding schemes, PPP is primarily a learning artifact and is less dependent on the characteristics of the underlying padding schemes.

Unveiling The Mask of Position-Information Pattern Through the Mist of Image Features
Anonymous Author(s)
Affiliation
Address
email
Abstract
Recent studies show that paddings in convolutional neural networks encode ab- solute position information which can negatively affect the model performance for certain tasks. However, existing metrics for quantifying the strength of po- sitional information remain unreliable and frequently lead to erroneous results. To address this issue, we propose novel metrics for measuring (and visualizing) the encoded positional information. We formally define the encoded information PPP (Position-informatii information Pattern from Padding) and conduct series of ex- periments to study its properties as well as its formation. The proposed metrics measure the presence of positional information more reliably than the existing metrics based on PosENet and a test in F-Conv. We also demonstrate that for any extant (and proposed) padding schemes, PPP is primarily a learning artifact and is less dependent on the characteristics of the underlying padding schemes.
3
10
11
12
Introduction
13.
14 Padding, one of the most fundamental components in neural network architectures, has received much less attention than other modules. Zero padding is frequently used in CNNs, perhaps due its simplicity and low computational costs. This design preference remains almost unchanged in the past decade. Recent studies [1, 2, 3, 4] show that padding can implicitly provide a network model with positional information. Such positional information can cause unwanted side-effects by interfering and affecting other sources of position-sensitiee cues (e.g., explicit coordinate inputs [5, 6, 7, 8, 9], embeddings [10], or boundary conditions of the model [4, 11, 12]). Furthermore, padding may lead 21 to several unintended behaviors [5, 7, 8, 9], degrade model performance [10, 11. 12], or sometimes create blind spots [6]. Meanwhile, simply ignoring the padding pixels (known no-padding valid- padding) leads the foveal effect [13, 14] that causes model to become less attentive to the features on the image border. These observations motivate to thoroughly investigate the phenomenon of positional encoding including the impact of commonly used padding schemes.
17
18
19
20
24
25
26 Conducting such a study requires reliable metric to detect the presence of positional information
27 introduced by padding, and more importantly, quantify its strength consistently. We observe that the existing methods for detecting and quantifying the strength of positional information yield inconsistent results. In Section 3, we revisit two closely related evaluation methods, PosENet [1] and F-Conv [3]. Our extensive experiments demonstrate that (a) metrics based on PosENet are unreliable 31 with an unacceptably high variance, and (b) the 'Border Handling Variants' (BHV) test in F-Cony
30
32 suffers from unaware confounding variables in its design, leading to unreliable test results.
The sounce codes and data collection scripis will be made publicly available.
Submitted to 36th Conference Neural Information Processing Systems (NeurIPS 2022). Do distribute.

---

56 In this section, we first define symbols for expressing the functionality of paddings and define the Expectation of Channel-Averaged Image Features
PPP
1 Sample
10 Samples
100 Samples
480 Samples
480 Samples
Imperceptible
Recognizable
Pattern from Padding (PPP). We propose method that can consistently and effectively extract PPPs through the distributional difference between optimally- padded (gray-scale surfaces) and serpihhmmicall-pppddede features (colored surfaces). The results show that the two distributions become distinguishable as the number of sample increases. Following the procedure in Section 2.2, extract clear view of PPP with the expectation of the pair- wise differences between optimally-padded and algorithmically-padeee visualization in tilted view (first row) and top view (second row). The colors represent the magnitude (blu//cold/weak to preen//wrmmsstoong. at each pixel. The features are extracted at the 3rd layer of interest (Appendix A) from aundn-padded (Section 2.4) ResNet50 pretrained on ImageNet.
Figure 1:
features. We render each
33 In addition, we observe all commonly-used padding schemes actually encode consistent patterns underneath the highly dynamic model features. However, such pattern is rather obscure, noisy, and visually imperceptible' in most cases. Fortunately, tently revealed with sufficient number of samples by defining optimal padding scheme (see Section 2.1 and Figure 1). We accordingly propose new evaluation paradigm and develop method Pattern from Padding (PPP), which 39 a persistent pattern embedded in the model features to retain positional information. We present two metrics to measure the response of PPP from the signal-to-noise perspective and demonstrate its robustness and low deviation among different settings, each with multiple trials of training.
34
show that such patterns be consis-
37
38 to consistently detect the presence of the
40
41
To weaken the effect of PPP, we design padding scheme with built-in stochasticity to halt the model from constructing consistent patterns in Section 2.4. However, our experiments show that the models can still circumvent the stochasticity and end up consistently constructing certain PPPs. This observation suggests that a model likely constructs PPPs purposely to facilitate training, rather than falsely or accidentally learning some filters that respond to padding features.
43
44
45
46
With reliable PPP metrics, conduct series of experiments analyze the characteristics of PPP in Section 4.1. Specifically, monitor the formation of PPP throughout each model training process in Section 4.3. The results show PPPs are formed expeditiously at the early stage of model training,
49
50
slowly but steadily strengthened through time, and eventually shaped in clear and complete patterns. These results show that a model intentionally develops and reinforces PPPs to facilitate its learning
51
52 process. Moreover, we observe the PPPs of all pretrained networks are significantly stronger than those in their initial states. This indicates an unbiased training procedure of great importance in resolving the critical failures caused by PPP in numerous vision tasks [6, 7, 10, 11].
53
Observations and Methodology
55
57 optimal-padding scheme. We then give a formal definition of Position-information Pattern from
"xceptt the zeros prros-padding is already well-known with its clear shaped pattern [6. 1].

---

11
Optimally Input
Algorijamically- Padded
Optimalt-ppddeed feature
w/o Margin Correction
Iaput
405
209
224
180
Correction
Replicate-paddel
PPP Correction
Principal Caused by Strided Conv2d
(c) Visuatization Principal Point Shift
(b) Margin Correction
(d) Impact on PPP
Principal point shift. (a) The stride-2 Conv2d only pads on side, causing the principal point shift (red squares) in earlier layers. (b) Such shift requires careful margin correction while aligning algorithmically-addee
Figure 2:
and aptimally-padded features (we describe the details of point shift in Appendix A). (c) The shift is visible in the feature space (spade-shaped and ceesstioonmar--hheee patterns the marked box). (d) It is crucial correct the principal point shift while measuring PPP. The PPP calculation involves pixel-wise distance functions, which are not robust to spatial shifts [15].
58
Padding (PPP) and utilize the optimal-padding scheme to develop propose method to capture PPP and measure its response with two metrics.
59
80
2.1
Optimal Padding
81
The process of capturing an image from the real world can be simplified as the 3D information of the environment is first projected onto an infinitely large 2D plane, and then the camera determines resolution as well as field-of-views to form an image from such infinitely large and continuous 2D 64 signals [16, 17]. Let S {s n= be collection of such infinitely large and continuous 2D signals, (
62
63
and the collection of 2D images captured by cameras spatial size (h,, w.) be S' {6,, L x=1-
85
I. by padding A padding scheme produces set of dlgorithmiccallypaddeee images S
function p:
s [ = s' (p(s i,5)
if < i hn and 0 < Wny
8n [.
otherwise,
68 where and j are index a pixel in the spatial dimension. We define a theoretical optimally-padderr 89 collection St { IN with an padddig function p by:
= [(,] if <i<<i hn and < Wn. (pt(s,, = s otherwise.
[
(2)
70
In practice, such an aptimal-ppddiing scheme difficult achieve. However, it can be simulated have access to images beyond the sizes (h,, wn) and artificially create S'.
71
72 2.2 Positional-informatio Pattern from Padding
73 As PPP has not been well defined in the literature, there no effective metric to detect or quantify
74 Ideally, PPP should have two properties. First, it is spatial pattern as the padding pixels different locations contribute differently the formation of PPP. Its shape enables the network to develop and exploit the absolute positional information of each pixel, eventually leading the unattended and undesirable effects in certain tasks [5, 6, 7, 8, 9, 10, 11].
77
78 Second, as it represents the positional information purely contributed by the padding, it a constant
79 term irrelevant to image contents. Unfortunately, PPP shares space with image features, and
80 these two spaces interfere with each other, causing the appearance of PPP extremely obscure in most cases (except zeros padding). Figure I shows if we visualize features sample-b--sampe there no obvious differences between optimally-padeer features (gray-scale surface) and algorithmically-

---

83
padded features (colored surface). Fortunately, if we assume the interferences between PPP and 84 image features to be random, then its expectation over a large set of images will saturate to a constant bias and no longer hinder us from capturing PPP.
85
86
Based on these observations, we define PPP the constant component independent of model inputs, and its presence is completely contributed by the existence of padding scheme p. Given S and model ((;; 0, p). which 9 is the model parameters and p is a padding scheme applied to F. Let the model feature extracted at k-th layer be fn, = ( f, p). where F6. is the model from the first layer to the k-th layer. The PPP at k-th layer (PPP)) can be formulated by: PPPk E d (F p'). Fa(8ni
88
89
90
]
where .): can be any distance function, and we use & distance in this work.
Pitfalls: feature misalignment. It is important to note that, some CNN components can cause serious 92
93
feature misalignment while computing PPP and leads to erroncous results. A typical example is principal point shift, where the uneven padding in stride-2 convolution causes the centers of features slightly drifted, as shown in Figure 2. Since the measurement of PPP requires perfect alignment,
94
95
96 such drift should be carefully considered while integrating PPP into new architectures. We further discuss the issue along with other pitfalls in Appendix A and provide three detailed examples of correcting the principal point shifting.
98
2.3
Metrics
100
In order to measure the strength of PPP, proper baseline signal is needed. As discussed above, strong PPP should be distinguishable from the interferences of the model features, so that the model can successfully extract the positional information from PPP. Thus, if we consider the model features as a background noise signal and PPP as the signal of interest, we can measure the significance of
102
103
104
PPP using the signal-to-noise ratio (SNR). We define the SNR for PPP at k-th layer as:
IE Es p') Fa P) lla 1) / / (fni p))
SNR-PPP;
(4)
105
where y and a are the mean and standard deviation the spatial dimensions.
106
However, SNR only measures the significance of the signal versus the noise but ignores the location
107
of the signal. Given PPP is a spatially varying pattern, we further include Mean Absolute Error (MAE) to measure PPP versus the average of the noise map with:
108
MAE-PPP& E MAE ( ( p"p). Fa((n::. ].
(5)
n
109
2.4
Randn Padding
110
Most of the existing padding schemes (e.g., zeros, reflect, replicate, circular) exhibit certain consistent patterns that can be easily detected by some designed convolutional kernels. One may argue that the nature of easy detectability can be root cause of encouraging the models to learn rely on these obvious patterns. This motivates to design additional sampling- based padding scheme without any consistent patterns, namely randn (i.e., random normal) padding, which produces dynamical values from a normal distribution while following the local statistics. We first determine the maximal and minimal values of a sliding window (which can be easily achieved with max-pooling), use the average of them as a proxy mean (l, and use the difference between the mean and the maximal value as a proxy standard deviation dp- For each padding location, we sample the padding value according to normal distribution N N(Hpp of) from the nearest sliding window. We include more implementation details in Appendix A.
111
112
115
116
117
118
119
120
121
Aside from creating a pattern-less padding scheme with sampling, the design of randn padding is based on several factors. The sampled padding pixels are allowed to occasionally exceed the min/max bound of the sliding window. Without breaking the min/max bound can introduce detectable patterns certain extreme cases, such as a gradient-like feature that has its maximal intensity at the top-left corner and minimal intensity at the bottom-right corner. We also design the padding scheme to follow the local distribution. The padding exhibits high entropy when the local variation is high, while degenerates value repetition with imperceptible perturbations while padding flat area. As
122
123
124
125

---

128
such, not only do the padding pixels exhibit less pattern, but it also prevents the padding pixels from breaking the features in the border region. We later show that a model still deliherately and incredibly built up PPP over time even with such a sophisticated padding scheme.
129
130
3 Revisiting Prior Work
131
132
In this section, we first reproduce two experiments from the prior art, which aim to assess positional information from paddings. We show several critical design issues in these experiments and discuss how these problems affect the drawn conclusions. Finally, we propose two additional experiments to quantify the amount of positional information embedded in the paddings.
133
134
135
136
3.1
PosENet
137
Islam et al. show reros-padding provides CNN models positional information cues, and propose
138
PosENet [I] to quantify the amount of positional information encoded within CNN features. A
139
PosENet experiment involves several components: a pretrained CNN model F, a shallow CNN
Epem (i.e.. position encoding module), an image dataset X = {2, } to examine, and a constant target pattern y (e.g., 21) Gaussian pattern.. PosENet first extracts intermediate features at k- th layer with f(a.)) (x;) using the pretrained CNN, and then optimizes Epem to minimize B. p (f(.,) - y (2] Finally, the amount of positional information is quantified by the average Spearman's 's correlation (SPC) and Mean Absolute Error (MAE) overall Epem (f(,)) toward y.
140
143
144
145
critical issue with PosENet is the use of an optimization based metric. sensitive to hyper- parameters with large variation. As shown in Table 2, for all the PosENet results, the standard deviation over five trials significantly dominates the differences between different types of paddings, and thus no definitive conclusions can be drawn. We also observed that PosENet can report NaN results in certain setups. Furthermore, PosENet quantifies the amount of positional information by the faithfulness of the final reconstruction. However, a better reconstruction does not have a clear relationship to measuring the strength and significance of positional information. For instance, the VGG architecture with zeros-padding in Table 2, PosENet cannot recognize the positional information has been strengthened after training, which can be seen in Figure PosENet falsely assigns a much lower SPC to the fully pretrained model. Moreover, for the no- no-padding entries in Table 2, PosENet can still sometimes show responses to no-padding models, demonstrating it is a metric with an indefinite bias pending on the memorization ability of Epem-
146
149
150
151
152
153
154
158
157
Another issue is that the no-padding scheme used in Epem is known to have the foveal effect [13, 14], where a model pays less attention to the information on the edge of inputs. Using such a padding scheme for detecting positional information from paddings, which is mostly concentrated on the edge of the feature maps, is less effective. This is inevitable dilemma PosENet aims to identify positional information from the padding of the pretrained F, while applying any padding scheme to E ppm introduces intractable effects between the paddings of the two models.
158
159
182
163
3.2 F-Cony
184
Kayhan et al. propose a full- balldpadiing scheme (F-Conv) [3] and demonstrate it is more translational invariant than the alternatives. One of the critical results is on "border handling variants" (Exp 2 of [3), which we call it BHV test. The BHV test creates a toy dataset, where each image has a black background with a green square and a red square in the foreground. The task to predict if the red square is on the left of the green square (class 1), or vice versa (class 2). In addition, Kayhan et al. intentionally adds location bias such that both squares are located in the upper half of the image for class 1, and located in the lower half of the image for class 2. During testing, a "similar test" inherits the same bias, while a "dissimilar test"" exchanges the bias (i.c., both squares are in the lower half of the image for class 1). As a truly mannslaatoninnvriann CNN model should not be affected by the
165
186
187
168
170
171
172
173
location bias, it should focus on the relation between the red and green squares and perform similarly
174
on both tests. Since the experimental results show that F-Conv performs best on the dissimilar test,
175
concluded that F-Conv is less sensitive to the location bias. The authors also conclude the circular padding performs worse due to the behavior of wrapping the pixels to the other side of the image,
which leads to confusion between two classes.
5

---

constructing absolute coordinates for each input pixel. Note that we only show the design of the BHV Table I: Background color critical confounding variable in BHV test. We show that using grey background similar to Figure leads to discrepant results. The standard deviations are reported among 10 individual trials. We mark the best performance in green, and the worst two in red.
Black Background
Ciry Rackground
Pudding
I-Cone?
Similar (5)
Dissimllar (%)
(9)
Incunsistency (%)
Similar (%)
Dissimilar (%)
Dift (96)
Inconsistency (%)
N
99. 20..83...000
3.21 6:35
-87.68
95.81 2.07
100 1 0.00
4.96 5.9g
-95.04
97.85 4.55
Zeros
89. 29..44.0.00 ae
89.244
0.96
18.021 5.08
100. 1.0.001 0.00
4.77g 6.52
-95.23
96.791 7.13
Circalar
4:3 23
80.314
9:25
34.254
6.32
72.764 0.96
72.75-
0.98
0.00
26.30. 5.55
99. 20.20
93. 14+
2.38
6.06
18.48+ 3.68
88 26+ 0.50
92.40- 4.29
5.87
28. 68.677 0.18
Reflect
Y
100. +0.000
100. 00000 00
11.70
15.67=12.72
88.30
-4.33
91.18=13.10
97.33
100. 100.00
100.00 +0 0.00
19.06+18.64
17.16412.19
-8.88
-8004
90.33+11.00
98.13
11.005
Replicute
N
Y
100. 100000
98. 98.3240.30
43.39
93.655
43
1,80
-66661
-4.67
T5.324 8.30
32 32.604 4.97
100 100 0.00
97.17 0.48
38.16 6.42
94,99 1/20
-08.83
-2.18
32.15
84.09 6.47
Rands
N
Y
100.00.0.00
100.00
100. 10000
20.80111.15
10.31 16
-79.20
-9. 70
92.511 8:37
94.884
77.28 13
99 99.97 0.13
35.47210.88
35.
66.70111.08
-64.50
-10.59
83. 83.59
45.70120.62
No-pad
100. 100000
3.21g
6.35
-96.70 79
95.81
100 100.00g 0.00
30.07 4.06
-69 .93
81. 81.30
178
However, as shown in Figure 3, we find the experimental design does not consider a crucial confounding variable: the black back- ground has a zero intensity, making zeros padding the optimal padding that perfectly follows the background distribution. In Ta-
Zieo Parking
Zono Puoking
179
180
181
182
ble 1, show that the dissimilar test is no longer in favor of
183
F-Conv zeros after changing the background color to grey. We also
184
show that F-Conv replicate and F-Conv circular perform best on the dissimilar test, which is different from the original observation.
The BHV test trains binary classifier to predict the relative position of the two col- ored squares. It hypothesizes if the padding provides no po- sitional information, the classi- fier will only focus on the rela- tive position of the two squares. (Left) The black background is confounding variable. (Right) Zeros padding no-longer pads optimum values after changing the background color.
Figure 3:
185
186
Finally, we report an additional inconsistency rate to show that the CNN architecture used in the BHV test actually has access to the absolute position of the squares. Given random sample in class
187
188
1, we create trajectory of samples by simultaneously moving the two squares to the bottom of the canvas and recording the CNN- model prediction in all intermediate states. We label a trajectory to be inconsistent if the prediction of the CNN-model switches classes at any step of the trajectory. A CNN model with no access
190
181
192
193
194
to the absolute-pon pootttionn information should have all trajectories maintaining consistent predictions, with 0% inconsistency. Table 1 shows the inconsistent ratio over 228 uniformly sampled trajectories, where all models maintain high inconsistency rates, even with a no-padding architecture. These results show that the CNN model used in the BHV test is not translation invariant. This can be attributed to that a CNN model has a large receptive field covering the whole experiment canvas, therefore capable of gradually 199
195
196
197
198
200
201
test is not suitable for quantifying the amount of positional information exhibited in a CNN model.
202
Such conclusion does not imply that F-Conv cannot potentially improve the translation-invariann
203
property of CNNs.
Experiments and Analysis
204
205
Datasets Since most vision models are trained on tasks for recognizing objects, an image collection containing a diverse object appearance is more suitable for the task. We collect a set of 480 satellite images at 2,048 2,048 pixels from Google Map for experiments. All the PPP metries are measured with this image collection. We crop such images depending on the requested input image sizes and principal point shifts from each model (see Appendix A for details). We will release the script for collecting and composing these large images.
206
207
208
4.1
Visualizing iiaition-iffommatioon Pattern from Padding (PPP)
212
We start with visualizing PPP in Figure 4. All the visualizations are conducted at the 4th layer of interest as detailed in Appendix A. We compute PPP using Eq. 3 and e norm as the distance metric,
6

---

Table 2: Comparing PosENet and our proposed PPP metrics. The standard deviation is computed by five different pretrained models for each test. The performance shows the accuracy for the classification task or weighted F-measure score [18] for the saliency object detection task. Note that we use 2D Gaussian as PosENet reconstruction pattern, and the PPP metries are measured the 4th layer of interest. Here, (*) indicates a NaN is reported in any of the trials, and (1) indicates a higher value corresponds to stronger positional information or better performance on the task (vice versa for (4)). For each group of pretrained models, we label the strongest and weakest positional information
response with red and blue.
PosENet
PPP (ours)
Model
Padding
Pretrained
Performance (t)
SPC (t)
MAE (L)
SNR-PPP (1)
MAE-PPP (1)
Zeros
ImageNet
0.518 121
0.1422+0.18
0.184 o 004
0.19440.000
0024 0.0665
1.2280+0.000
0.01322 0006
0.0176+00 00000000
74.0972+0.0870
Circular
x
0.001 +0. 092
0.197 .00
0,0000
0000
0. 0.0000++.0000
ImageNet
0.102 136
0.197 007
1.1488 go g000se
0.015810.00 0006
74.471610.0003
0.001 091
0.197 002
0.0000
0.00001..00 0000
VGG-19
Reflect
ImageNet
0.116 184
0.196 006
0.02222..0228
0.015844..oo 0002
74.051620.0621
Replicate
x
0.001 +0.001
00.99
0.0000=0.o000
0.0000+0.0000
ImageNet
0.116
0,195 .oo
1.219120.0204
0.01444o.000
73.9961 73.996140.1079
Rando
x
0.001 +0.0os
0.197 1197+0.002
0.0000
0.0000
ImageNet
0.115 146
0.195 0.006
1.2366
0.018240.no0z
73.7716=0.0765
No- padding
ImageNet
0.000 091
0.001 +0.220
0.197 002
0.203 0.012
0.0000.0.0000
0.0000=0.0000
0.0000+0.0000 +o,000
0.000010.00 0000
62.0396+0.0830
Zeros
x DUTS
0. 0.6822x..om 20005
0.171=0.000
0. 0.030620.0020
0. 0.0068+p..0007
0.343
0.180go
0.2429 0035
0.004491..0m
0.6209
Circular
x DUTS
0.001 081
0.197
0.0000
0.0000
0.158 20.1
0.196 0.013
0.2677=00.00d 2000002
0.0062+po.000
0.6200+0..000
VGGI6-SOD
Reflect
DUTS
0.160 20.00.0.5 223
0.0002 pioo
0.197 .002
0.195 0.014
0. 0.1972
0. 0.0000=+0.000
0.005340.0001
0.0000+0..000
0.6243g0.0022
Replicate
x DUTS
0.002 +0.067
0.197 2 000
0,0000 2 0000
0.000010.0000
0,075
0,201
0.1908
0.004334.0000
0.6255+0.0018
0.000
0.197
0.0000=0.0000
0.0000+0.0000
Randn
DUTS
0.004490100 106
0.196 om1
0.000520.0001
0.000110.0000
0.257020.0022
No-padding
0.000 087
0.197 0000
0..000020.000
0.0000:g0.0000
DUTS
0.003 +0.23
0.200 0.000
0. 0.0000
0. 0.0000++.0000
0.4759+0.0015
Zeros
ImageNet
X
0.329 20.901
0.09 0.096
0.196 .000
0.185
0.8171
0.0918=0.0119
0.016240.007
0. 0.002+000000
73.6856_0.0022
Circular
ImageNet
*0.027 +0.098
0.1844 4..0000.01
*0.197
0.19220.010
0. 0.701880.o0g 2000050
0.045420.004
0. 1.01888o..0o1
0.0032+o.n000
76. 76.1432
ReaNer50
Reflect
ImageNet
x
0.0004 i.. 09s
0.293 10.11.1.... 181
"0.198
0.187
0.6960
0.0291.0.00g
0.0150
0.0018110.00m 000000011
75.5068
00.002+o.mo
*0.198 1.98+0.000
0.0226+0.0013
0.0015++..m
Replicate
ImageNet
0.347 200205
0.184_0.012
0.7461.200225 22002244
0.0138 0000
75.61
22.1022200011
Randa
0.0006 peo
10.198 ooa
1.0326_0.om
0.0020 0002
ImageNet
0.358
0.181g0.omt
0.66118
0.0147
75.9076g0.100z
Zeros
ImageNet
X
0.360
0.667 +0.111
0.1664
0.180
0.7590+0.0228
0.507420.0a0
0.047 1.0471+0.0022
0 0.039844..00
61.8652+0..555
Circular
0.004 192
0.205 0.10
0.3008_20.0ee
0.0222+o.0045
ImageNet
0.020 +0.4....3
0.203
0.4326-0.022
1.0256-+0.0017
61.2208
EficientNet
Reflect
x
0.003
0.205
0.2245-2.00m 06529
0.0183
ImageNet
0.062 20.21.1.10 116
0.201
0.4667 1 0232
0.026844..0 0014
60.4164
Replicate
ImageNet
0.131 1:3
0.004 183
0,205
0.197 oas
0.5257 155257.00.3334
0.263410.oo
0.0206 0035
0.0279.0.0007
60.950120.2114
0.001 2o. 150
0.202- 0111
0.3606=p.ooos
0. 1.0248++.n000
Randa
ImageNet
0.324 20.210
0.18910.012
0.5686g0.0112
0.020911.000
58.639220.227
then average the resulting PPP in the channel dimension to generate gray-scale image. Since the quantities are small and difficult to perceive, we normalize the gray-scale image to [0. 1] range, and thus the colors between images are not directly comparable.
215
216
217
In all scenarios, a noticeable difference is that PPP spreads out after pretraining on ImageNet. In Table 2, the PPP-SNR of the VGG19 and ResNet50 also reflects that the response of PPP is significantly strengthened after model training. That is, the model training has substantial effects on the construction of PPP. Although the formation of padding pattern is suggested to mainly caused by
218
7

---

Pretrained
VGG19
ResNet50
Zoros
Circular
Reflect
Replicate
Randn
Zeros
Circular
Reflect
Replicate
Randa
x
ImageNet
Figure 4: Visualization of calculated based on Eq. 3 over 480 GMap samples extracted the 3rd layer-of-interest (Appendix A). The results show that the pretrained model significantly reinforces PPP compared to randomly initialized networks. Note that each image is normalized to [0, separately, therefore the colors between images are not comparable. More visualizations are presented in Appendix B.
Pattern from Padding (PPP). The visualizations
Leve-eleewet 1-
Lecestintett (558)
Lavena Prccoeetooraa.ooo (153)
$237
944-
VGG19
1300
6.0
333
3500
1yp onderd (959)
3351
113
ReaNet50
640
300
600
20
Figure 5: Chronological PPP. We quantify PPP every 10 epochs and plot its development in four different layer of depth (the rightmost layer the one closest model output). All curves consistently show sudden surge the early stage, and all the later layers are slowly but steadily gaining stronger PPP until the end of training. The shadow region represents standard deviations among 5 individual training episodes. The colors represent zeros, circular, reflect, replicate, and randn paddings.
the distributional difference between features and paddings [6], results show that it only increases the response slightly, compared to the considerable PPP-SNR gain through training.
223
Another intriguing observation is that, despite some variations in the detailed patterns, the overall structure PPP remains similar. Regardless padding minimum values with zero zero- parddig (consider the features are processed with ReLU activation), sandn-padding that can sometimes produce large quantities by chance, or the unbalanced initial state ResNet50 caused by strided convolution (the first of ResNet50 in Figure 4), all models tend to have the maximal PPP response in the corner of the features after fully trained. While the underlying mechanism causing such consistent preferences remains unknown, such preferences may be an important factor to consider in future model design.
224
225
226
229
4.2
Quantifying PPP and Comparing with PosENet
Table 2 shows the measurements of PPP and PosENet on various architectures and padding schemes. We train five models for each setup and measure the standard deviation of these models. Our PPP metrics have significantly lower standard deviations compared to PosENet, where the standard deviation dominates the differences between padding variants, and thus the quantities from PosENet cannot provide sufficient information for any analysis. The main reason that PosENet has such a large formulation. and thus the final quantities highly depend on the convergence of the PosENet training. In fact, we also observe a similar level of standard deviation even when the PosENet is measured on the same model for multiple trials. On the other hand, PPP metrics are based closed-fon formulation, and thus the variations are only introduced by the differences among the parameters of the pretrained models. Furthermore, PosENet frequently reports
235
236
variation due to its optimization-
237
238

---

241
positive SPC responses from no- no-padding models, as shown in its large standard deviation. In contrast, PPP has zero response to no-padding models by definition, and therefore is less biased for measuring the positional information from padding.
242
243
244
SNR-PPP and MAE-PPP assess the response of PPP from two different perspectives, the ratio of the overall PPP magnitude to the image feature variation, and the position-ave average gain of PPP. Despite both measuring the PPP gain and mostly following similar trends, the two metrics can sometimes have discrepancies, such as the randn padding case in EfficientNet pretrained on ImageNet in Table 2. We note that the two metrics should be both measured and considered altogether.
246
247
248
249
Although certain paddings seem to have lower SNR-PPP or MAE-PPP on trained networks, we find the differences are not significant when comparing the extremely low SNR-PPP and MAE-PPP from the randomly initialized networks. In most cases, the network can effectively construct its PPP, even with the highly stochastic randn padding. The only exception seems to be the case of randn padding in the salient object detection (SOD) task, where the network fails to achieve a compatible performance
252
253
254
to other paddings'. The results show that the model training plays an important role in the formation of PPP, and perhaps its contribution is much larger than which underlying padding scheme is being used. This motivates us to further analyze the PPP formulation during model training.
255
256
257
4.3
Chronological PPP
258
To understand the formulation of PPP through time, we snapshot checkpoints every 10 epochs for all training episodes. By measuring the PPP metrics at all the checkpoints, we plot a chronological curve and monitor the progress of PPP. We train 5 individual models for each pair of model- maddd-g setup and report the standard deviations, which demonstrates the significance of the trend.
259
261
262
Figure 5 shows all models achieve a significant gain of PPP within the first 10 epochs in all inter- mediate layers. Most models continuously increase their PPP as training proceeds, especially in the fourth layer of interest, which is the last output from the convolutional layers before the final linear projection. Another interesting observation is that randn padding, which is designed be less easily detectable with built-in stochasticity, indeed shows less PPP built-up at the intermediate stages in certain layers. However, the network still adjusts the behavior and ends up forming complete PPPs at the fourth layer of interest in all scenarios. All these evidences show that the network builds PPP purposely as a favorable representation to assist its learning.
263
264
265
267
268
269
5
Conclusion and Limitations
In this paper, we develop reliable method for measuring PPP and conduct series of analyses toward understanding the formation and properties of PPP. Through large-scale study, we demonstrate that PPP is representation that the network favorably develops as part of its learning process, and its formation has weak connections to the underlying padding algorithm. We show that reliable PPP metrics are important steps for understanding the effects of PPPs in different tasks, and useful for measuring the effectiveness of future methods in debiasing PPP.
273
274
275
276
However, an unfortunate and inevitable limitation of the PPP metrics is that their measure is biased by the model architecture and parameters. Since the PPP metrics are based on the distributional differences between the paired model outputs (i.c., optimal padding to algorithmic padding), different architecture and layers of depth exhibit different and intractable biases due to different interactions between PPP and model parameters. Such a bias makes PPP metrics less useful for evaluating models, and therefore cannot be used to study the effect of architectural changes. This limitation is inevitable for any (and all existing) metric that attempts to measure PPP using the outputs of model. We note future studies in measuring PPP without model inferences* will be an important step toward tackling and understanding the property of PPP under different architectural choices.
279
280
281
282
283
We follow PosENet that evaluates PiCANet [19] the SOD task. PiCANet is initialized by model pretrained on ImageNet (with zero padding). The discrepancy in the padding scheme can be the major cause of failure while training the network on SOD task with randn padding.
A related analogy of the contradictory problem can be found in neural architecture search literature [20].

---

References
286
[1] Md Amirul Islam". Sen Jia", and Neil D. B. Bruce. How much position information do convolutional neural networks encode? In International Conference on Learning Representations, 2020. 1, 2,5
288
289
[2] Md Amirul Islam, Matthew Kowal, Sen Jia, Konstantinos G Derpanis, and Neil DB Bruce Position, padding and predictions: A deeper look at position information in enns. arXiv preprint arXiv:2101.12.22 2021.
290
291
[3] Osman Semih Kayhan and Jan C van Gemert. On translation invariance in cnns: Convolutional layers exploit absolute spatial location. IEEE Conference on Computer Vision and Pattern Recognition, 2020.
293
294
1,5
285
[4] Carlo Innamorati, Tobias Ritschel, Tim Weyrich, and Niloy Mitra. Learning on the edge: Investigating
296
boundary filters in cnns. International Journal of Computer Vision, 2020.
287
[5] Chich Hubert Lin, Yen-Ch Cheng, Hsin- Hsin-Ying Lee, Sergey Tulyakov, and Ming- Hsuan Yang. InfinityGAN:
298
Towards infinite-pixel image synthesis. In International Conference on Learning Representations, 2022. 1,
299
300
[6] Bilal Alsallakh, Narine Kokhlikyan, Vivek Miglani, Jun Yuan, and Orion Reblitz-Richardson. Mind the pad [cnn]s can develop blind spots. In International Conference on Learning Representations, 2021. 1,
301
2, 3, 8
303
7 Rui Xu, Xintao Wang. Kai Chen, Bolei Zhou, and Chen Change Loy. Positional encoding as spatial inductive bias in gans. In IEEE Conference on Computer Vision and Puttern Recognition, 2021. 1, 1,
304
305
[8] Evangelos Ntavelis, Mohamad Shahbazi, Iason Kastanis, Radu Timofte, Martin Danelljan, and Luc Van Gool. Arbitrary-scale image synthesis. In IELE Conference on Computer Vision and Pattern Recognition, 2022. 1,3
306
308
[9] Jooyoung Choi. Jungbeom Lee. Yonghyun Jeong, and Sungroh Yoon. Toward spatially unbiased generative models. In JEEE International Conference Computer Vision, 2021. 1,3
310
[10] Songwei Ge, Thomas Hayes, Harry Yang. Xi Yin. Guan Pang. David Jacobs, Jia-Bin Huang, and Devi Parikh. Long video generation with time-agnostic vqgan and time-sensitive transformer. arXiv preprint orXiv:2204.03633 2022. 2,
312
[11] Antonio Alguacil, Wagner Gonçalves Pinto, Michael Bauerheim, Mare C Jacob, and Stéphane Moreau. Effects of boundary conditions in fully convolutional networks for leamning spatio-temporal dynamics. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2021. 1, 2, 3
314
315
[12]
Md Amirul Islam, Matthew Kowal, Sen Jia, Konstantinos G. Derpanis, and Neil Bruce. Boundary effects in [enn]s: Feature or bug? https: / //opnrrveerrrreieer. hett/foorum/ddd/Mgggggkkk3x. 2021.
317
318
[13] Bilal Alsallakh, Vivek Miglani, Narine Kokhlikyan, David Adkins, and Orion Achlizz-Richardson. Are convolutional networks inherently foveated? In SVRHM 2021 Workshop at NeurlPS, 2021. 1.5
319
[14] Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel. Understanding the effective receptive field in deep convolutional neural networks. In Neural Information Processing Systems, 2016. 1,
321
322
[15] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang- The unreasonable effectiveness deep features as a perceptual metric. In IEEE Conference on Computer Vision and Pattern Recognition, 2018.
323
324
325
[16] Shichen Liu, Tianye Li, Weikai Chen, and Hao Li. Soft rasterizer: differentiable renderer for image-based 3d reasoning. In JEEE International Conference on Computer Vision 2019.
326
[17]
Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d. arXiv preprint axXiv220770880// 2020.
328
329
330
[18] Ran Margolin, Lihi Selnik-Manor, and Ayellet Tal. How to evaluate foreground maps? In IEEE Conference on Computer Vision and Puttern Recognition, 2014. 7
331
332
[19] Nian Liu, Junwei Han, and Ming-Hsuan Yang. Picanet: Learning pixel-wise contextual attention for
333
saliency detection. In IEEE Conference on Computer Vision and Pattern Recognition, 2018. 9
334
[20] Joe Mellor, Jack Turner, Amos Storkey, and Elliot J Crowley. Neural architecture search without training.
335
In International Conference on Machine Learning, 2021. 9
Checklist
1. For all authors...
10

---

338
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
339
contributions and scope? [Yes]
340
(b) Did you describe the limitations of your work? [Yes]
341
(c) Did you discuss any potential negative societal impacts of your work? [No]
342
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
344
2. If you are including theoretical results...
345
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
346
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
348
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material as URL)? [No] All the codes for reproducing all results shown in the paper will be made publicly available.
351
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
352
353
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
354
355
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [No] It is not critical computational constraint to the experiments. In order to properly report the standard deviation, we use a total of 24 GPUs over 3 clusters to train I 150 CNN models on ImageNet and DUTS datasets. These computations are completely for analyses. Running our PPP metrics only need IGB of memory on any type of GPU, or even CPU.
357
358
359
360
361
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
382
(a) If your work uses existing assets, did you cite the creators? [Yes]
383
(b) Did you mention the license of the assets? [No] The assets used in our codes are released under MIT or BSD-3, which have restricted usage.
385
(c) Did you include any new assets either in the supplemental material or as a URL? [No]
(d) Did you discuss whether and how consent obtained from people whose data you're using/curating? [N/A] We did not obtain personal data.
368
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A] We did not use personal data.
369
370
5. If you used crowdsourcing or conducted research with human subjects...
371
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
372
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
375
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
378


Reviews:
[{"review_id": "vTN8dcV8Nm", "reviewer": "Reviewer_rick", "summary": "Authors present a novel metric for detecting the presence and quantifying the strength of positional information encoding due to padding. They define PPP as a spatial statistic, i.e., expected absolute difference between algorithmically padded and optimally padded images’ activations, and summary statistics are defined from the spatial ones, i.e., SNR and MAE for PPP. In addition, they propose a new padding strategy, which aims to preserve local variability around a boundary when padding. The proposed metric is applied on pre-trained networks as well as those that are trained from scratch at different epochs. Results suggest that the position information encoding gets stronger with training, and very strong for pre-trained models. Compared to two existing alternatives, the proposed metric has lower variance, and therefore seems to be a more reliable metric. ", "questions": "1.\tMy main suggestion is to better motivate the investigation and the developed metrics for applications of CNNs. How will this technology actually be used? A roadmap towards improving the current state of things using the PPP is unclear. \n2.\tWhat is the value of quantifying the strength of position encoding?  Better explanation towards this end would substantially improve the article. \n3.\tIt is shown in figure 5, that proposed metrics are different at different layers. In this setting, it may be important for authors to provide some guidelines telling which layer one should look at. More specifically, difference (quantified with PPP) at which layer would cause the biggest problem for prediction accuracy? \n", "limitations": "Yes, they have. ", "rating": 3, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1.\tThe investigated concept is intriguing and potentially affect all application of CNNs. \n2.\tThe proposed measure is very intuitive and easy to compute. \n3.\tResults suggest that the proposed metric shows lower variability compared to alternatives, and show that almost all networks encode position to some extent. \n4.\tResults showing emphasized position encoding for pre-trained models is interesting. \n\nWeaknesses: \n1.\tA solution towards removing the position encoding is not discussed. \n2.\tImportance of quantifying the strength of PPP is not clear to me. \n3.\tAuthors state that reliable PPP metrics are important for understanding PPP effects in different tasks. While this point is surely intriguing, such an explanation or understanding is not explicitly given in the article. Can the authors explicitly explain what type of understanding one reaches by looking at the PPP maps? \n4.\tThe conclusion of the article remains a bit vague. While the proposed metrics have some more desirable attributes, value of these attributes for applications is unclear to me.  How will this actually improve the practice or our understanding? \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "GIj57bviwux", "reviewer": "Reviewer_qWQu", "summary": "This paper proposes a novel metric for measuring and visualizing the encoded positional information arising due to padding in Convolutional Neural Networks. They formally define the encoded information as PPP (Position-information Pattern from Padding) which measures the distance between feature maps arising from an algorithmic padding (such as zero padding) and an optimal notion of padding (i.e. the ground truth pixels that would be around the image). They demonstrate that PPP is a more reliable metric (low variance) than alternate metrics and hence it is useful for measuring the effectiveness of methods debiasing PPP due to padding. They show that PPP is a representation that the network develops as a part of its learning process.", "questions": "What can be more quantitative metrics to justify a good PPP metric? Currently, the majority of the focus is on the large standard deviation of PosENet. However, there must be other ways to compare which metric is a better representative of measuring the result of padding.", "limitations": "The authors already mention an important limitation of PPP being linked to the architecture of the network as it is evaluated on the outputs of a model.", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n1. The paper is well-written, and contains sufficient details.\n2. The proposed problem of determining the amount of position information encoded due to padding is important as it can be interfering to actual cues and degrade model performance.\n3. The proposed metric is motivated well, and the analysis of PPP and prior works is thorough.\n4. The idea of using an optimal padding scheme to quantify PPP is sound and reveals consistent patterns in experiments.\n​\n## Weaknesses\n1. The evaluation is conducted on only one dataset of just 480 images. A more thorough investigation can be done by incorporating diverse datasets and observing the consistency of PPP as a metric over PosENet.\n2. Fig. 5 and Section 4.3 demonstrate that PPP is a representation that the network favorably develops as a part of its learning process. Can the authors add some discussion on the implications of this? What new insights can be drawn from this experiment?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "DJ7YBXUfiqR", "reviewer": "Reviewer_MMJm", "summary": "This study investigates position information patterns learned due to the use of padding. A new evaluation method has been proposed by taking the difference between an optimal padding strategy and the “algorithmically-padded features”. The difference pattern becomes more recognizable when more samples are applied. Then signal-to-noise ratio and mean absolute error are proposed to measure the layer-wise difference between the two paddings. This whole system is claimed to be a better measure for the position information.\n", "questions": "The two metrics SNR-PPP and MAE-PPP, proposed in this paper are questionable. The use of signal-to-noise ratio and mean absolute error is not new, the new metric is essentially the proposed “optimal” padding, which can be used to compare against other paddings and compute differences, including SNR and MAE or any other metrics. Then let’s focus on the basic question, what the “optimal” padding is.\n\nThis proposed position measure is conducted at feature level, which is more difficult to draw conclusions than those studies at output level. Accuracy on the BHV test from [1] can be considered a measure against human perception, which is a golden standard in CNN understanding, e.g., why the classifier behaves differently than human and what could be the factor behind (absolute position). Similarly, the artificial ground-truth used in [2] can validate CNNs in a human readable way, if the model can output the wanted pattern based on position. This type of validation has also been applied in other CNN understanding papers[3,4].\n\nThe measure proposed in this paper is computed at the feature level, which is not human readable, or is still under exploration. Then the defined “optimal” padding becomes a golden standard to compare, which is very questionable. The paper also agrees with the “optimal” padding shown in Eqs. (1) and (2) “In practice, such an optimal-padding scheme is difficult to achieve.” I personally doubted if such padding exists in practice. Without further explanation, Sec2.2(Fig 1., line 81) directly shows the visual difference between the “optimal” and the “algorithmical” padding, this is really important but looks confusing. I can only guess the “optimal” refers to the randn padding from sec 2.4(please correct me if I am wrong), and the large difference to zero-padding is not surprising. The padded value by zero-padding is very different from other content-based, e.g., circular, reflect, replicate, which has been already studied in previous studies[1,5]. (In addition, the finding in Fig 3 and Table 1 that background color matters can already be found in previous study[5].) More importantly, I am still seeking an explanation why this randn padding could be the “optimal” to compute the two metrics, it looks more like another well-designed “algorithmical” padding. The “optimal” golden standard needs to be set and studied thoroughly, then the resulting findings (Table 2 and 3) can be analyzed reliably. Otherwise, it makes less sense no matter how high or low values a metric can achieve.\n\nTherefore, I suggest a more thorough study on how the “optimal” padding can be achieved before computing metrics on top. And based on Table 1, the proposed randn padding behaves similarly to the replicate padding, which can be considered a special case drawn from the same distribution as randn padding.\n\nBased on the questions above, I would suggest this study can be further improved for a better presentation and less confusion, the use of SNR and MAE is not main focus. More importantly, the design of \"optimal\" padding should be better addressed.\n\n[1] Kayhan, Osman Semih, and Jan C. van Gemert. \"On translation invariance in cnns: Convolutional layers can exploit absolute spatial location.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.\n[2] Islam, Md Amirul, Sen Jia, and Neil DB Bruce. \"How much position information do convolutional neural networks encode?.\" arXiv preprint arXiv:2001.08248 (2020).\n[3]Geirhos, Robert, et al. \"ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.\" arXiv preprint arXiv:1811.12231 (2018).\n[4]Geirhos, Robert, Kristof Meding, and Felix A. Wichmann. \"Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency.\" Advances in Neural Information Processing Systems 33 (2020): 13890-13902.\n[5]Islam, Md Amirul, et al. \"Position, padding and predictions: A deeper look at position information in cnns.\" arXiv preprint arXiv:2101.12322 (2021).", "limitations": "The paper can be further polished and re-organized to reduce misleading. The applicability of randn can be further discussed, the previously studied findings can be omitted. ", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The proposed idea of randn padding is plausible, a window is used to compute the mean and deviation values within. Then we can sample padding values based on the learned normal distribution. Addressing the effect of strided convolution also makes the investigation more comprehensive. The proposed randn padding can deliver less position information than other padding strategies.\nWeakness: This study also has some essential problems that need to be addressed, please see the questions below.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Reject