Achieving Rotational Invariance with Bessel-Convolutional Neural Networks
Authors: Valentin Delchevalerie
Adrien Bibal
Benoit Frenay
Alexandre Mayer
Venue: NeurIPS 2021 Poster (2021)
Keywords: Rotational invariance
Convolutional neural networks
Bessel functions
Bessel-Convolutional neural networks

Abstract: For many applications in image analysis, learning models that are invariant to translations and rotations is paramount. This is the case, for example, in medical imaging where the objects of interest can appear at arbitrary positions, with arbitrary orientations. As of today, Convolutional Neural Networks (CNN) are one of the most powerful tools for image analysis. They achieve, thanks to convolutions, an invariance with respect to translations. In this work, we present a new type of convolutional layer that takes advantage of Bessel functions, well known in physics, to build Bessel-CNNs (B-CNNs) that are invariant to all the continuous set of possible rotation angles by design.

Achieving Rotational Invariance with Bessel-Convolutiona Neural Networks
Valentin Delchevalerie, 1 Adrien Bibal,² 2 Benoît Frénay, 2- Alexandre Mayer PReCISE, NADI & naXys institutes, University of Namur, Belgium 2 PReCISE, NADI institute, University of Namur, Belgium Department of Physies, naXys institute, University of Namur, Belgium [valentin, deeeeaaede,, adrien.bibal, benoit.frenay. alexandre.maye mayer )
@ unaauurbb
Abstract
For many applications in image analysis, learning models that invariant translations and rotations is paramount. This is the case, for example, in medical imaging where the objects of interest can appear at arbitrary positions, with arbitrary orientations. As of today, Convolutional Neural Networks (CNN) are one of the most powerful tools for image analysis. They achieve, thanks convolutions, an invariance with respect to translations. In this work, we present a new type of convolutional layer that takes advantage of Bessel functions, well known in physics, to build Messel-CNNG (B-CNNs) that are invariant to all the continuous set of possible rotation angles by design.
Introduction
Deep learning models, and more particularly Convolutional Neural Networks (CNNs), are known as being among the most powerful tools for image analysis. For this reason, they still constantly upgraded in order to achieve better performance M. One of the main reasons why CNNs are so much used in computer vision lies in the fact that they achieve translation invariance thanks to convolutions. Filters sweep the image locally and patterns can be recognized regardless of their absolute position in the image. However, some other important types of invariance are more difficult to obtain. It is for example the case for the rotational invariance, which is relevant for many applications. One could for example consider medical imaging where tissues, cells, tumors or other objects of interest have local, arbitrary, orientation in the images (2]. Another example is satellite imaging of ships, where both the global orientation of the satellite and the local orientation of ship are arbitrary [.
Multiple works proposed solutions in order to bring rotational invariance in CNNs. However, many of them (i) only make the model more robust to rotations without providing guarantees regarding the rotational invariance 4 5]
7 8. (ii) only provide guarantees for a finite set of rotation 15 or (iii) only handle global rotational invariance, while local one is 14
angles 10 12
13
sometimes more relevant (4, 5 6. In addition to that, and more similarly to the method proposed in
this paper, other works present solutions to build CNNs that are rotational invariant for all rotation
angles, while providing mathematical guarantees. One can for example cite [6 17 18 Our work in the continuity of those. We integrate new kind of convolutional layer in CNNs to build Neural Networks (B-CNNs). In B-CNNs, Bessel functions from physics are used to build a representation of the images that is more adapted to deal with rotations. propose new way to use this representation in order to compute feature maps in rotational equivariant way (see Figure [ for an example). This rotational equivariance can then lead to rotational invariance with proper choice of architecture.
"B. Frénay and A. Mayer are co-last authors.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Figure 1: This figure illustrates the lack of rotational equivariance in standard CNNs (left), as opposed to the rotational equivariance of B-CNNs (right). For each triplet of images, the first image is the input image, the second is the feature map obtained, and the last one is the feature map reoriented to make the comparison easier. It can be seen that the last images in the two left triplets for the standard CNN are different. Hence, different orientation of the input image produces a different feature map. However, feature maps are identical except for a rotation for the B-CNN on the right.
2 formally defines the problem of rotational invariance in CNNs and provides the necessary notations. Next, Section presents the related works bringing rotational invariance into CNNs. Our method, B-CNN, then introduced in Section 4 along with some background on Bessel functions. Experiments are described and discussed in Section [ Finally, more general discussion about some aspects of B-CNNs, as well as some future works, are presented in Section before concluding the paper in Section 7
Section
Background and problem definition
For several applications in computer vision, patterns of interest may appear in arbitrary orientations. This typically happens in medical imaging (2), satellite imaging (3) astronomical imaging [I9], texture recognition (20), etc. In such cases, the fact that the orientation is meaningless be used as a prior during learning through feature extraction that is orientation-agnostic. However, building deep learning models that present guarantees with respect to the rotational invariance Indeed, in the particular case of Convolutional Neural Networks (CNNs), the computation of a feature map is clearly not rotation invariant since, general, for a rotation of an angle a, we have
trivial.
l 4-1ks
l 1 1-1kg-
Σ 0
Σ m=
m,y m K (m, n) # Σ TV
(x -
1 n.) K (m,
(1)
==
where z' = cos a y sin a, 1 = x sin a + y cos T is the input of the convolutional layer and K represents a particular filter of size k x l22 Figure iillstrrate this lack of invariance.
Two important notions are central in this work: rotational invariance and rotational equivariance, On the one hand, if f (I) represents the computation of feature maps in a particular layer of CNN, f (T) is rotational invariant if f (R (a) 1) = (I), Va [0, 2T), where R (a: is an operator that rotates by an angle a. However,
(R (a) I) R (a) f (1). Va E [0, 2x) ). then f (I) rotational cquivariant since feature maps are identical except for rotation (see Figure D]. For particular architectures where the feature maps the final layer have a size of 1 1, this rotational equivariance leads to rotational invariance. Indeed, if is one single value, R (a) I = I. Such a strong rotational invariance something usually not guaranteed, or sometimes only for a finite subset of [0, 2x).
3 Related work on rotational invariance for CNNs
The use of Bessel functions in the Machine Learning community has already been proposed for image denoising, classification or recognition purposes. One can for example cite [2] 22 23 24), which already use a decomposition using Bessel functions similar to the one used in this paper. However, these works differ from ours since their objective is to build rotational invariant covariance matrix, and to obtain rotational invariant models on top of this, while the objective of B-CNNs is to perform rotational invariant convolutional operation between filters and images.

---

Methods for introducing rotational invariance in CNNs can be categorized in different groups: those (i) that just make CNNs more robust to rotations, without providing any mathematical guarantees. those (ii) that provide mathematical guarantees but only for finite set of rotations (typically obtained by discrete group of symmetries), and finally those (iii) that introduce continuous rotational invariance while providing mathematical guarantees.
Among the first category of methods, one can cite data augmentation 4 spatial transformer net- works (STN) [5). rotation-invariant and fisher discriminative CNNs (RFID-CNNs) [6], deformable CNNs 77, and SIFT-CNNS [8] For the second category, one of the best-known method is Group CNNs (G-CNNs) 191, which define the transformations for which an invariance is desired using the theory of groups and symmetries. However, one can also cite maassformaiion-invaiian pooling (TI-POOLING) [O]] deep symmetry networks (DSN) [[T] stecrable CNNs [12]. steerable filter
CNNs (SFCNNs) 13,, dense steerable filter CNNs 14, and spherical CNNs [5] Finally, there are already few methods in the third category. i.e. methods that are invariant to the continuous set of rotations. General eq))-vauivariant steerable CNNs 17 present a general solution to obtain E(2)- equivariant CNNs by constraining the kernels using a group representation. LieConv 3 generalizes equivariance to arbitrary transformations from Lie groups, and Harmonic networks [6 use circular harmonics to achieve a rotational invariance by maintaining disentanglement of rotation orders in the network. Our work is in the continuity of these techniques. It proposes a new framework to achieve the invariance. For example, compared to harmonic networks, B B-CNNs present different advantages: (i) it involves real-valued feature maps, which make B-CNNs casier to use with already existing techniques (it can use classic non-linear activation functions, classic batch- norichoommaization. etc.), (ii) it uses an arbitrary number of rotation orders, and (iii) it can also be made invariant to other types of invariance in future works.
In the next section, we introduce Bessel-CNNs, based on the use of the Bessel functions well known in physics, to make CNNs invariant by design to all rotation angles in the continuous set [0, 2x).
4 Exploiting Bessel functions for CNNs
The mathematical background on Bessel functions and the motivation of their use for rotational invariant image recognition tasks are presented in this section. Some of these developments are inspired from the use of Bessel functions in physics (25). These mathematical developments lead to new type of Convolutional Neural Networks (CNN) that we call ConsollCannllltionnal Neural Network (B-CNN). Mathematical evidence of a rigorous rotational invariance is pointed out.
4.1 Bessel functions
Bessel functions are partieularly well known in physics because they arise when solving some important problems in polar or cylindrical coordinates. For example, they appear when dealing with wave or heat propagation. Bessel functions may be of different order v and are defined as a particular solution of the Bessel's differential equation
d'y dx
dy dx
+ (~2- 0.
x²²
This equation involves two particular kinds of Bessel functions Ju (x) and Y, (z), called the Bessel function of the first and of the second kind, respectively. In this work, v must be an integer and we only consider Ju (x) since Y, (z) diverges for x O. For E N. J/ (x) and J_ (x) are not independent and
(x) (-))= (x).
(3)
Bessel functions also satisfy
J. (-1
(4)
which means that J, is an even function if v/ is even, and an odd function otherwise. Bessel functions of the first kind can be used to build particular basis for the representation in polar coordinates of images defined in a circular domain of radius R, as
R PJB (kusp) dp,
Ju (Kusp) ii0 Vv,j E J where Nos = 1/v 2T
(5)
3

---

forms an orthonormal basis for all squared- -interbbbee functions f such that f D2 C R - R (where the domain Da is a disk in R²². Indeed, we can show that
J
R 2m I."
P [NugJo (Kens)) e eivoy N J. (kvi ewoo) d0dp = ke
(6)
the k are solutions of either Ju (kv = 0 or J (Mv = 0 126]. We use J (kv. = 0 as condition for the ky because it makes it more convenient to represent arbitrary functions (by the fact that for = 0 there is a ku.j = 0 with this condition, we can use (0.0p))e = to describe an arbitrary constant intensity) [25]. The key advantage of using this basis is that can deal easily with rotations. Later, this basis will be used to build new representation for images.
4.2 Bessel coefficients
An arbitrary function V/ (p, 0) E E. where p and are polar coordinates, can be expressed with the basis in Equation (5) as
00
V (p.
Σ
Vns NogJy (kuu e00 e
==0
where the Bessel coefficients L Mnj are obtained by projecting vi (p, 0) on this particular basis
2nt /,²² R
V Ves -
p [NNusJu (krss) e'voj" (p. 0) d0dp.
In order be mathematically exact when describing (p, 0) using its Bessel coefficients, one should consider all v E {-x..... ) and all j E {0.... x) to represent V (p, faithfully. Nevertheless, as only a finite to be chosen. By taking into account Properties (3) and (4) when looking at Equation (8) and as (p,0) I. we end up with the relations
of coefficients can be considered in practice, values of Vmaa and Jmax need
(V
v-) = (-1) (W
(3 = (-1) 10/t1310 (V2,)),
where R and 3 stands for the real and imaginary part, respectively. One can then only compute the Bessel coefficients for v (resp j) in {0...., Vmar (resp. ] since & and Yes are not independent. Also, note that, when or j increases, kv.j also increases. The values of Vmax and Jmaz can then be determined through a mathematical insight about the maximal value of kes to use in given problem (for example, using the Nyquist frequency, in (2)).
To understand now why working with Bessel coefficients is particularly useful, one can see how an arbitrary rotation of & (p. 0) by angle a modifies its Bessel coefficients V Vuj. To do so, let us
consider sprot (p, V (p, a) for an angle a E [0, 2m). Its Bessel coefficients are given by
rot / 2n de f. R
dpp [NujJu (ku e eiro grot (p,
(10)
By defining 0' = = a, this leads to
2m d0'
[. R
dpp Ju (kvsp) " v (p. (p, eiva Vuyee
(11)
This motivates our work, as a rotation of an arbitrary function by an angle a only modifies its Bessel coefficients by multiplying factor e iia which makes rotations conveniently expressed in the Fourier-Bessee transform domain (analogously to how the Fourier transform maps translations to multiplications by complex exponentials). Therefore, if V (p, 8) represents an image, its Bessel coefficients constitute a more adapted representation to build rotational invariant operations.
4.3 Defining a rotational invariant operation
In order to build rotational invariant operation between the Bessel coefficients Vny and filter made of the same number of complex numbers Ky.j. one can consider
1
2n
K V
Ky Vuise Vense-ivape -ira ( da.
(12)
2T
Jn

---

This operation is, by design, necessarily invariant to rotations as Equation [II shows that multiplying each Ve. by eiva equivalent to applying rotation the image. By performing the integration for a going from 0 to 2m, the image achieves a complete rotation around itself. Therefore, K y does not depend anymore on the particular initial orientation of the image. Note that, without the squared modulus, we can show that only the coefficients Vo. contribute to the final results. The operation would therefore rely on an incomplete representation of the image.
Using Equation as such may look like a computationally expensive brute force strategy, since requires to evaluate the numerical integration over a. However, given that
0:2
2
R (am) (ax)) |mm 25] cos (0m - -0,) Σ Σ
S (am) ) (az) |mm cos (0m - 0z) m.j
m.j
Σ
R (ay) 3 (am) I2m sin (fm 0g)
(13)
m.j
where 0g and 2;
|2/l are complex numbers, the squared modulus in Equation
becomes
Σ R (K2r) R (K |Vn Werg cos (Ous Ouis a (v v'))
+
Σ
3 (KWr) 3 (K)rg) WV Vey cos (fv.) -a (v-vv vv))
v'
-2 Σ
3 (Kts) R (K)r) Wes Varg sin (Ous - Our - -
(14)
v'
if wr 1 - The three terms should be integrated over thanks to Equation the trigonometrie functions are a-dependent. A simple integration leads to
12 but only
I. sc (0p.) - o - a (v v') da = 2T se (Ou.j Ou) if v'
(15)
otherwise,
where se can represent the cosine or the sine function. Therefore,
K
Σ R
R (Kt Mous Mese cos (Ou.) Ow (Kt,)
Σ
(K))) 3 (Kig.) Mi Mndl cos (0nj 0v(
"
- 3 (Kz)) R (Kzjy) | Vay sin (0u.) 02 Ow'r)),
Σ K.J,S'
(16)
and by using once again the result presented in Equation 13 we have
K.W V
Σ
Σ K KK Mpsl?
(17)
which is therefore equivalent to Equation 12 but much more convenient. By construction, the results obtained by Equation (17) do not depend on the particular orientation the image. Furthermore, K & E E (with K E C and L Pr C). which is something convenient in order to use, for example, classic activation functions. Using Equation|] [2 (or Equation [7p to achieve the rotational invariance one of the main contribution of this paper.
4.4 Bessel-Convolutional Neural Networks (B-CNNs)
In standard CNNs, a feature map is the result of the convolution between the image and a filter of 1 x li weights, as described in Equation (I). The weights of this filter are real numbers that are tuned during the learning process of the network. In Bessel-CCnvvlutional Neural Networks (B-CNNs), the process to build feature maps is slightly different as the representation of the image based on Bessel
5

---

Input
Feature map
kn.
1
Vug (m.
(m.))
Figure 2: Illustration of how the feature maps are computed in B-CNNs.
coefficients are used. A filter K made of (Umaz + 1) x (Jmas + 1) complex weights K2.j is used to sweep the image sub-regions of size k x k. Note that Pmaz and Jmaz are generally larger than 11 and k2, which results in an increase of the number of trainable parameters in cach layer. However, as it easier for B-CNNs deal with rotational invariance, we show in Section [5 architectures with less layers than for other methods still achieve similar or even better performances. For each sub-region of the image centered position n1), its Bessel coefficients V (//m..))
computed, and
a (m.n)
- K V (m. Mg
(18)
computed, where a (m,.) is real value that invariant rotation, i.e., does depend on the particular orientation of the sub-region. All the values obtained for the different sub-regions
constitute the feature map, which is then rotation equivariant. This is schematized in Figure
4.5 An efficient implementation of B-CNNs
Implementing B-CNNs as described in Section can for each sub-region of the image. However, developing Equation (88 with Equation
can be slow, as it requires to evaluate Equation (8)
gives
I, NugJJ (ku,))) wew.wrggmmmm (m..)) (p. 0) pdpd0)
2x
R
a (m.
Σ
K
2m (
R wimm) (0,0)
Σ1
E KigNNojd (kuzp) e ivo pdpd0
(19)
where (mm.n) (p,0) represents a particular sub-region of the image centered at (m, n) in polar coordinates. When going back to a Cartesian coordinates system, one can get
a (m.n) Σ 1
1mm I,n v) Σ Kin EKig (HegV x²) (a drdyy drdyl 2
(20)
~u Ju (ku,p)) is defined as where 7 (Ix, y) is the input of the layer, 0 (x, = x aretan and
Jn (kvsp) Jn (ku.jp) if S R otherwise.
(21)
Finally, by defining Trg (I, y) = NusJu (Birs v x²
y2 e (1.()))) Equation 20 leads to
= (z, Σ Kig (I, 12
Σ 11 (x, (z. 12
(22)
where denotes convolutional product and F, (x. y) corresponds to a filter modified by the Bessel functions. Compared to a standard CNN, the only difference is that, actually, a layer in B-CNN performs Umex + convolutions instead of one only, and it also needs to update the filters with the Bessel functions. Nevertheless, Tug (ar, y) part of the model and only needs to be computed once, as it does not depend on the input of the layer, but only the choice of Poaw and Jmaz.

---

5 Experiments
This section presents the details of the experiments used to test the rotational invariance of B-CNNs, including the datasets (Section[5.1) the baseline architectures (Section 5.2) and the experimental setup (Section 5.3). Experimental results are presented in Section and a discussion on the rotational equivariance of feature maps proposed in Section 55,
5.1 Datasets
In this comparative study, three datasets are used in an increasing complexity. In all of them, orientations present in the test set are not present in the training set. The first dataset is MNIST (27). a classic baseline of 28 x 28 images of handwritten digits, where images in the test set are randomly rotated by an angle in [0, 2T). For all runs, the training set contains 60, 000 images and the test set 10, 000 images. The second dataset is Outex-TC-00010
dataset (28), a classic dataset for a more complex classification involving both global and local rotations. contains 128 x 128 grayscale images of 24 particular textures. The training set contains 480 images with 20 orientations, and the test set is composed of 3840 images with 160 orientations. The third dataset made of 128 x 128 x brain MRI images [29] that are either cancerous or non-cancerous. This dataset is used as real-world, more complex dataset, which contains local entities (the brain tumors) that can occur at arbitrary orientations. Each image is cropped to make the brain fill the entire image. The training and test sets contain 190 and 63 images, respectively.
5.2 Baseline architectures
Experiments aim to evaluate how good B-CNNs are for problems where rotational invariance is desired. We decided to use G-CNNs, as well as standard CNNs in our comparative study. For G-CNNs, the symmetry group C4 is used (rotations of angle 3. n. 2 and 2T). For each dataset, one architecture based on CNNs, G-CNNs and B-CNNs has been chosen by picking the one that performs the best on a validation set. Importance is given to the fact that the three models contain roughly the same number of trainable parameters, in order to perform fair comparisons. Attention is also paid to the fact that the feature map in the last convolution layer should be of size 1 1, in order to achieve rotational invariance for G-CNNs and B-CNNs.
3r
The standard CNN architecture for MNIST is made of 7 conv. layers with 32 filters for the first two layers, and 64 filters of size 3 3 for the other layers; one max- max-pooling layer that operates over x regions after the second and last layers; and one dense layer with 10 units. The G-CNN architecture similar, except that the number of filters is divided by 2 in order to keep the same number of parameters. The B-CNN for MNIST is made of 5 Bessel-conyolutional (B-Conv) layers (one of 16 filters with Umar = 11, Jmas = and k = 3: three of 16 filters with Umer = 9, Jmaz = 7 and k = 5; and one of 32 filters with Vmaz = 7, Jmox = 5 and k- = 7), a max-pooling layer over 2 2 regions after the fourth layer and a dense layer with 10 units. Activation functions are relu, except for the B-CNN where tanh leads to better performances on the validation set. Adam optimization with learning rate of 0.0001 and an exponential decay rate of 0.9 is used for the three models.
The standard CNN for Outex is made of 5 conv. layers (one with 48 filters of size 5 x 5, one with 64 filters of size 5 x 5, two with 64 filters of size 3 3 and one with 128 filters of size 3), one max-pooling layer over 3 x 3 regions after the first layer, one over 2 x 2 regions after the second and third layers, and one dense layer with 24 units. The G-CNN architecture for Outex similar, but the number of filters is divided by 2 again to have the same number of parameters. The model based on B-CNN is made of 3 B-Conv layers (one with 16 filters with Pmaoz = 15, Jmaz = 11 and k = 15; one with 32 filters with Mmaz = 10, Jmaz 9 and k = 9; and one with 32 filters with Vmaa = 7. Imes and k = 7), a max-pooling layer over x 3 regions after the first layer, one over 2 x 2 regions after the two others, and dense layer with 24 units. Again, activation functions are relu, except for B-CNN where tanh are used. Adam optimization with a learning rate of 0.0005 and an exponential decay rate of 0.9 is used for the three models.
The standard CNN for brain MRI is made of 5 conv. layers (one with 32 filters of size x 7, one with 32 filters of size 5 5. two with 64 filters of size 5 x 5 and one with 64 filters of size 3 x 3), one max- pax-pooling layer over 3 x regions after the first layer, one over x 2 regions after the second and third layers, and one dense layer with 2 units. The G-CNN architecture is similar but the number
7

---

160
50-
G-CXN B-CNN
Acurcey (8)
Accurcty
G-Chh
G-5NN BKNN
21-
CNN
50-
130
Looch
Acsency 150
G-CNN
C-CMN
G-CNN
21-
CVN
CMN
CNN
:00
300
130
151
301
Epoch
(a) Performance MNIST
(b) Performance Outex
(c) Performance brain MRI
Figure 3: Mean test accuracy over 40 runs with 99% confidence intervals for B-CNN, G-CNN and standard CNN on MNIST, Outex and brain MRL the first row, the training sets contain input images as they are originally in the dataset, whereas data augmentation is used in the second row.
of filters is once again divided by 2. The model based on B-CNN is made B-Conv layers (one with 32 filters with Vinar 11, Jmax = 11 and & = 9; one with 16 filters with Vmaz 9, Jmaa 9 and & 7: one with 16 filters with Vnne 7, Jmas and k 5). max-pooling layer over 3 x 3 regions after the first layer, one over x regions after the three other layers, and a dense layer with 2 units. Again, activation functions are relu, except for B B-CNN where tanh are used. Adam optimization with learning rate of 0.001 and a exponential decay rate of 0.9 is used for the three models.
and & 5; and one with 32 filters with
Umaz Jmaz
5.3 Experimental setup
Each experiment consists of 40 independent runs and mean accuracy scores with 99% confidence intervals are reported. In order to isolate intrinsic rotational invariance, each model is trained for each dataset in two different settings. In the first setting, images are rotated all during training. In the second setting, input images are rotated by random angle during training (a form of data augmentation). In order to avoid border effects due to the appearance black corners when images are rotated for the Outex dataset, images are cropped from an initial size of 128 x 128 to 88 x 88. This is not performed for MNIST and brain MRI, since image corners are already black.
In order to highlight the ability of B-CNNs to use the invariance for the continuous set angles as prior knowledge, another experiment in low data setting is also performed. This experiment is exactly the same as the previous one on the Outex dataset (with rotation of the input images), except that 75% of the training is removed, and placed in the test set.
5.4 Experimental results
Figure 3 shows the mean test accuracy of B B-CNNs, G-CNNs and standard CNNs on MNIST, Outex and brain MRI. The first observation is that standard CNNs have poor performance in almost all cases, which justifies the need for better methods. A second observation is that using data augmentation (second row), as opposed to using the input images as they originally are in the dataset (first row), increases the performance of all methods in all cases. Furthermore, in the setup without data augmentation, can be observed that B-C B-CNNs largely outperform G-CNNs and standard CNNs on MNIST, This indicates that G-CNN need data augmentation in addition to the groups of symmetry to learn rotational invariance, while Bessel convolutions intrinsically impose the invariance.
The results are interestingly reversed for Outex, where B-CNNs perform better than G-CNNs with data augmentation, while being roughly equivalent without data augmentation. The good performance of G- G-CNNs without data augmentation on Outex may be explained by the fact that the texture images

---

contain an intrinsic data augmentation (a rotation of an image can be present in the original dataset). This is not the case of MNIST, where all images the original dataset have the same orientation.
It can also be observed that the variance of G-CNNs is generally higher than B-CNNs, due to the fact that discrete of rotations is encoded into the groups of the G-CNNs. Because of that, mismatch can happen between the rotations learned with the randomly rotated input images and the groups on the one hand, and the rotation of the images in the test on the other hand. This mismatch is minimized for B-CNNs, as the continuous set of rotation angles is imposed by design. Thanks to that, the predictions of B- B-CNNs are more stable than the one of G-CNNs.
The generalized instability in performance for all methods on brain MRI (see Figure 3c) can be explained by the small number of instances used at each epoch (190 for training and 63 for testing). This characteristic of the dataset may also explain that B-CNN achieves better performance more quickly, as the rotational invariance is present by design (i.e., invariance is by opposition to G-CNN that more heavily relies on the training instances to learn this invariance.
prior knowledge),
Figure 4 presents the results obtained on the Outex dataset in low data setting, where 75% of the training data is removed. When, in the initial experiment presented in Figure 3b) we slightly outperformed G-CNNs by less than 5% accuracy, in this experiment where less data is provided during training. we outperform G-CNNs by a little bit less than 15%. Furthermore, the learning curves again much more stable with B-CNNs.
5.5 On the equivariance of feature maps
In addition their performance stability, B-CNNs also offer certain stability from the equivariance of their feature maps. Unlike standard CNNs in Figure the feature maps of B-CNNs extract the exact same latent features, matter the orientation of the input image. This characteristic of B- B-CNNs can help in providing trust to its users. For instance, medical experts would see that the model extract the same latent features for tumorous brain images, and this no matter the orientation of the tumor.
Figure 5 shows the equivariance B-CNNs and G-CNNs. The feature maps in the figure are randomly taken from the second convolution layer of the B-CNN and G-CNN. It can be observed that the feature maps in the first row (for a non-rotated image from Outex) for the B-CNN are generally the same as those of the second row (corresponding to the same image, but rotated of 60%, see Figure 5a). However, when training a G-CNN with the same number of layers and the same filter sizes, its feature maps change significantly after rotating the input image (see Figure (5)).
In addition to Figure 5 we also performed a quantitative study order to assess the rotational equivariance of B- B-CNNs, G-CNNs and CNNs. To do so, we performed the same experiment as the previous one, but for the whole test set and for random rotational angles. For each sample, the 4 feature maps that are the most active are min-max scaled between () and 1 (this step is needed since feature maps of different architectures may have absolute values at a different scale). Then, the difference between the non-rotated and the rotated feature maps is computed (the difference between the first and second row in Figure 5). The bigger this value, the less equivariant the method is. We obtained mean value of 1.77% 2.78% for the B-CNN, 10.64% + 5.64% for the G-CNN and 5.73% + 5.64% for the CNN. B- B-CNNs therefore achieve a better rotational equivariance.
100
G-CNN
80
B-CNN
Accuracy (%)
CNN
60-
40-
0-
50
100
150
200
Epoch
Figure 4: Performance on the Outex dataset in low data setting (120 images in the training set).

---

(a) Some feature maps of a B-CNN
(b) Some feature maps of a G-CNN
The first row corresponds to random feature maps for a non-rotated image from the Outex dataset. The second row corresponds to the same feature maps, but for 60 rotated version of the image. To ease the comparison with the first row, these feature maps have been reoriented. The corner pixels of the images of the second row are hidden they correspond to pixels that are out of the bound. For the sake of comparison, the same pixels have been hidden in the images of the first row.
Figure 5:
6 Discussion and future work
In practice, we are not always able to obtain a perfect rotational invariance. The main reason is that the Bessel functions are discretely sampled filters. It directly follows that, if too small filters are used, perfect rotational invariance may not be achieved due to discretization errors. This is the reason why data augmentation helps, even when using B-CNNs. Therefore, using larger filters helps to achieve a better rotational invariance. It would be interesting to consider other interpolation methods improve the sampling of input images and reduce this effect (similarly to what is already done by [30] to ensure translation invariance in CNNs).
grid, with resolution fixed by the size of the
In future works, one could consider bringing several updates to B-CNNs. Firstly, the actual V'aax + 1 convolutions can be converted to only one convolution. Secondly, performances of B-CNNs are deeply dependent in this work on the choice of Vmar and Jmaz- However, it is possible consider the Nyquist frequency in order to rather impose an upper limit to the ku.j as done in 21. This will both minimize the aliasing effect and maximize the amount of information preserved by the Bessel coefficients. Thirdly, it is also possible to bring other types of invariance using the same framework, like for example invariance under reflections (in order to make B-CNNs B(2)-invariant).
Before concluding, we would like to point out again that B-CNNs achieve a global rotational invariance by a succession of local invariance, provided by the multiplication between the image and the filters (it is the case for all similar techniques). This property can probably make B-CNNs and similar models more easily fooled by adversarial techniques. In the future, it could be interesting to evaluate the robustness of such rotation invariant CNNs, as adversarial techniques is an important concern for many applications.
7 Conclusion
This paper proposes new kind of CNNs, called Bessel CNNs (or B-CNNs), that are intrinsically invariant to the rotation of input images. In order to achieve this, new convolutional layer has been designed, based on Bessel functions from physics. Experiments show that B-CNNs do not need any preprocessing of the input images to be rotational invariant, and that the full rotational invariance brought by the Bessel convolutions lead to more stable results. We conclude that Bessel convolutions can be used in any application involving global and/or local rotations of images.
Acknowledgments and Disclosure of Funding
The authors want to thank José Oramas M. from the University of Antwerp for his insights on the paper. A.M. is funded by the Fund for Scientific Research (F.R.S.-FNRS)) of Belgium. V.D. benefits from the support of the Walloon region with Ph.D. grant from FRIA (F.RSS.-FRRS. This research used resources of PTCI at UNamur, supported by the F.R.S.-FNRS under the convention 2.5020. 11.
10

---

References
[1] Waseem Rawat and Zenghui Wang. Deep convolutional neural networks for image classification:
A comprehensive review. Neural Computation, 29:1-98, 1-1 2017.
[2] Benjamin Chidester, Tianming Zhou, Minh N Do, and Jian Ma. Rotation equivariant and invariant neural networks for microscopy image analysis. Bioinformatics, 35(14)11500-1537, 2019.
[3] Linhao Li, Zhiqiang Zhou, Bo Wang. Lingjuan Miao, and Hua Zong. A novel CNN-based method for accurate ship detection in HR optical remote sensing images via rotated bounding box. IEEE Transactions on Geoscience and Remote Sensing, 59 (9(6)6666669, 2020.
[4] Facundo Quiroga, Franco Ronchetti, Laura Lanzarini, and Aurelio F Bariviera. Revisiting data augmentation for rotational invariance in convolutional neural networks. In International Conference on Modelling and Simulation Management Sciences, pages 127-141, 2018.
[5] Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial trans- former networks. In Advances Neural Information Processing Systems (NIPS), pages 2017- 2025, 2015.
[6] Gong Cheng, Peicheng Zhou, and Junwei Han. RIFD-CNN: dotation-invariant and fisher discriminative convolutional neural networks for object detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2884-2893, 2016.
[7] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. In IEEE International Conference Computer Vision (ICCV), pages 164-773,22 2017.
[8] Abhay Kumar, Nishant Jain, Chirag Singh, and Suraj Tripathi. Exploiting sift descriptor for rotation invariant convolutional neural network. In IEEE India Council International Conference (INDICON), pages 1-5, 2018.
[9] Taco Cohen and Max Welling. Group equivariant convolutional networks. In International Conference on Machine Learning (ICML), pages 2990-299000 2999, 2016
[10] Dmitry Laptev, Nikolay Savinov, Joachim M Buhmann, and Mare Pollefeys. TI-POOLING: mannsformaioonnnvaaiaan pooling for feature learning in convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 289-297, 2016.
[11] Robert Gens and Pedro M Domingos. Deep symmetry networks. In Advances in Neural Information Processing Systems (NIPS), pages 2537-2545, 2014.
[12] Taco Cohen and Max Welling. Steerable CNNs. In International Conference on Learning Representations (ICLR), 2017.
[13] Maurice Weiler, Fred A Hamprecht, and Martin Storath. Learning steerable filters for rotation equivariant CNNs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 849-858, 2018.
[14] Simon Graham, David Epstein, and Nasir Rajpoot. Dense steerable filter CNNs for exploiting rotational symmetry in histology images. IEEE Transactions on Medical Imaging, 39(1))4112- 4136, 2020.
[15] Taco S Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical CNNs. In International Conference on Learning Representations (ICLR), 2018.
[16] Daniel E Worrall, Stephan J Garbin, Daniyar Turmukhambetov, and Gabriel J Brostow. Har- monic networks: Deep translation and rotation equivariance. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 5028-500 0038-503,0000 2017.
[17] Maurice Weiler and Gabriele Cesa. General E())-equivariant steerable CNNs. Advances in Neural Information Processing Systems (NeurIPS), pages 14334-14345, 2019.

---

[18] Mare Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Wilson. Generalizing convolutional neural networks for equivariance lie groups on arbitrary continuous data. In Internationaa Conference Machine Learning (ICML), pages 3165-3176, 2020.
[19] Sander Dieleman, Kyle W. Willett, and Joni Dambre. Imation-invvriann convolutional neural networks for galaxy morphology prediction. Monthly Notices of the Royal Astronomical Society, 450(2):1441-1459 2015.
[20] Diego Marcos, Michele Volpi, and Devis Tuia. Learning rotation invariant convolutional filters for texture classification. In International Conference on Pattern Recognition (ICPR), pages 2012-2017, 2016.
[21] Zhizhen Zhao and Amit Singer. Fourier-Bessel rotational invariant eigenimages. Journal of the Optical Society of America A, 30(5):871, 2013.
[22] Zhizhen Zhao, Yoel Shkolnisky, and Amit Singer. Fast Steerable Principal Component Analysis. IEEE Transactions on Computational Imaging, 2(1):1-12, 2016.
[23] Zhizhen Zhao and Amit Singer. Rotationally invariant image representation for viewing direction classification in cryo-EM. Journal of Structural Biology, 186(1):153-16) 2014.
[24] Cédric Vonesch, Frédéric Stauber, and Michael Unser. Steerable PCA for Rotation-Invarian Image Recognition. SIAM Journal on Imaging Sciences, 837:11557 2015.
[25] Alexandre Mayer and Jean-Paul Vigneron. Transfer matrices combined with Green's functions for the multiple-scattent simulation of electronic projection imaging. Physical Review B, 60(4):2875-288 1999.
[26] G. N. Watson. A Treatise on the Theory of Bessel Functions. Cambridge University Press, 2nd edition, 1995.
[27] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning 1998.
applied to document recognition. Proceedings of the IEEE,
(28) Timo Ojala, Topi Maenpaa, Matti Pietikainen, Jaakko Viertola, Juha Kyllonen, and Sami Huovinen. Outex-new framework for empirical evaluation of texture analysis algorithms. In Object Recognition Supported by User Interaction for Service Robots, pages 701-706, 2002.
[29] Navoneel Chakrabarty. sion 1. Retrieved May 9, 2021 from https::/www.kett
Brain MRI images for brain tumor detection, ver- www.oooooneelllllleeeeoeeooeeeee
brain mri- -ea---maees-ae for brain tumor- detection
[30] Anadi Chaman and Ivan Dokmanic. Truly shift-invariant convolutional neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 3773-3783, 2021.
12


Reviews:
[{"review_id": "rEqfozEVOP5", "reviewer": "Reviewer_nxbX", "summary": "The paper introduces Bessel convolutions, an approach to computing rotation equivariant representations using the well-known Bessel functions. The operation can be described mathematically as an integral over rotations of  an inner product between a rotated/phase shifted filter and the input. It is shown that this can be computed efficiently without an integral, as a modulus of a complex inner product between filter and input. The idea is validated by training networks on rotated MNIST, Outex (textures), and brain MRI scans. Results show that in among the specific models that were compared, the B-CNN outperforms on several datasets / configurations.", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well written, and technically sound. The approach is clearly motivated and explained, and enough detail is provided to enable others to reproduce the work. However, despite many strong points of the paper, I am not convinced that this method truly advances the state of the art in rotation-equivariant convolutional networks. There are a number of reasons for this:\n\n- The method is compared to CNNs and G-CNNs that are equivariant to 90 degree rotations. It is natural to expect that a method dealing with continuous rotations can do better than 90 degree rotations, and indeed the results confirm that in several configurations (but not all) the B-CNN outperforms the G-CNNs. However, there are several methods that deal with continuous / finely discretized rotations, and these are not compared to. The main methods are harmonic networks by Worrall and, what I believe to be the current best approach, the \"General E(2)-equivariant steerable CNNs\" of Weiler & Cesa. Open source code is available for both, so it should not be too hard to benchmark against these.\n\n- The networks trained are relatively small (5-7 layers) without modern design elements such as residual connections and batchnorm. Also, the datasets used for benchmarking either don't have an established state of the art result or these are not reported/compared to. For these reasons it is not clear if the comparisons reported in the paper reflect the very best that the G-CNN and B-CNN can achieve. It would be more convincing to see that B-CNNs can outperform the results reported in other papers on a competitive benchmark. There are a number such benchmarks that can be found in other papers on rotation equivariant CNNs.\n\n- For some of the plots in figure 3, it seems like the baseline methods are not yet converged. Also, slower convergence seen for the CNN/G-CNN could indicate that the weights are not initialized at the right scale or optimizer hyperparameters are not properly tuned. Again, such concerns could be taken away by reporting results on a competitive benchmark. \n\n- It is claimed that B-CNNs achieve equivariance to continuous rotations, but this is only guaranteed in the continuous setting, not when the Bessel functions are discretely sampled on a grid. To achieve rotation equivariance in practice, one would need to use low frequencies or relatively large filters (which have not been favored in modern CNN designs). Weiler & Cesa have an extensive discussion of this. The same is true for any steerable CNN, including ones based on circular harmonics such as harmonic networks and Weiler & Cesa's networks, so it's no shame but at least should be explained clearly. \n\n- The property of Bessel functions that makes the B-CNN equivariant is the fact that they undergo a phase shift under rotation. That is, they are steerable filters, and so this methods falls in the category of steerable CNNs. It is not clear why the Bessel functions are a better choice than say circular harmonics, which have the same property and have been used in previous works (Worrall and Cesa & Weiler). The circular harmonics already provide a complete basis for the space of equivariant filters, which implies one does not need anything else. Perhaps there is a numerical reason why Bessel functions are more stable or otherwise preferable, but this should then be explained clearly and demonstrated empirically in a head-to-head comparison with circular harmonics based approaches.\n\n- I'm not 100% clear on the implementation details, but it seems that the current implementation requires nu_max+1 calls to conv2d, which would make it considerably slower than methods that use a single call. Perhaps the implementation can be improved though?\n\nAlthough I have focussed on points that need to be improved, I want to reiterate that this is not a bad paper at all. In fact it's based on a neat idea, with solid and interesting underlying mathematics. However to really advance the field it is necessary to deeply study the current best methods, understand and address their weaknesses. Studying steerable filter bases for equivariant CNNs is a topic worthy of study, so I encourage the authors to dig deeper into this topic to better understand what makes for a good basis of filters, and perform direct comparisons to the most closely related and best performing existing methods.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "As noted in the main review some limitations could be better discussed. Societal impact for this paper is the same as any other paper in (equivariant) CNN design, in that CNNs can be used for positive and negative uses. In practice, equivariant CNNs have mostly been applied in medical imaging and problems in the natural sciences.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "h61f_BGqgZq", "reviewer": "Reviewer_gUNY", "summary": "The paper aims at achieving rotation invariance of images by introducing Bessel transforms to deep convolutional neural networks. \nLocal operations are carried out on submatrices over sub-regions by shifting kernels and the inner products are taken for Bessel coefficients over radial directions. The experiments show that the proposed Bessel-convolutional neural networks perform extremely well. ", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main idea is to use Bessel transforms for representing images and to take inner products on coefficients over the radial directions. Since Bessel transforms turn rotations into multiplications with exponentials, the induced inner products becomes rotation invariant. This is a very nice observation and the authors' idea is novel. The experiments with MNIST, Outex and brain MRI provide very convincing results. The paper is also well written. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It would be great somewhere else if the authors could provide some theoretical justification for the proposed method. Though some computations are given on page 6 about the implementation of the Bessel transform, more details and computational complexity should be provided. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "W7Mej_8_bVZ", "reviewer": "Reviewer_dNyU", "summary": "This work proposes a rotationally invariant CNN based on local Fourier-Bessel expansion of input images. To define the new type of convolution, for each local region of the image, the proposed B-CNN takes the averaged (over angles) inner product between kernel and “rotated” Fourier-Bessel coefficients. This convolution is invariant to the rotation of the local patches. Unlike G-CNN that deals with discrete sets of rotations, the B-CNN can handle any rotation in the continuous space $[0, 2\\pi]$. The numerical experiments demonstrate the advantage of B-CNN in various settings.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is in general well-written. In theory, it is a nice property that B-CNN is able to achieve rotation invariance for any continuous rotations. It seems to perform better than G-CNN especially when there is no data argumentation. However, I have the following concerns on the novelty and experiments of the paper:\n\n1). I feel that the contributions of this paper are over claimed. For example, the paper claims itself to be the first work in machine learning that introduces the Bessel functions. However, the following papers already introduced Fourier-Bessel, and in general, steerable basis functions, to achieve rotation invariance in dimension reduction, image classification/recognition/denoising: \n\n1. “Fast Steerable Principal Component Analysis”.\n2. “Steerable PCA for Rotation-Invariant Image Recognition”.\n3. “Fourier–Bessel rotational invariant eigenimages”.\n4. “Rotationally invariant image representation for viewing direction classification in cryo-EM”.\n\nThe motivation of the above works is almost the same as this paper. For example, see (15)-(20) in https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3711886/ that uses similar integral over angle $\\alpha$ compared to (12) of this B-CNN paper. The only difference is that (15)-(20) in the above link computes the rotationally invariant (angular averaged) mean (or the inner product with a constant kernel) and covariance, and in this B-CNN paper (12) computes the rotationally invariant (angular averaged) inner product with a kernel (not a constant kernel). I hope the authors can clarify the strong relevance of these papers.\n\n2). The experimental setup looks a bit unfair for G-CNN which uses $\\mathbb Z_4$ to represent the discrete rotations. I understand that the original G-CNN paper also uses $\\mathbb Z_4$ group, but it seems to be generalizable to e.g. $\\mathbb Z_8$, $\\mathbb Z_{16}$ (although careful interpolation may be required), at the cost of increasing the number of parameters. However, since in the experiment B-CNN has fewer layers than G-CNN, one can certainly reduce the number of layers in G-CNN and increase its number of discrete rotations to keep the similar number of model parameters. Another simpler way is to consider implementing CNN on the polar-transformed image with cyclic paddings, like in paper “Polar Transformer Networks”. I am curious about the experimental results in this new setting.\n\n3). The paper claims its advantage on handling continuous rotations, but I doubt its importance for discrete images. For example, for images with low resolution, due to the discretization error it is sufficient to use the set of discrete rotations. The number of rotations $n_R$ should be proportional to the image resolution. One may argue that for high resolution more discrete rotations are needed and this increases the model complexity of G-CNN. However, the same issue applies to B-CNN. Indeed, for low resolution images, one should use smaller $\\nu_{\\max}$ and $j_{\\max}$ to avoid the aliasing problem. Namely, one should keep them below the Nyquist frequency of the image, which is proportional to the image resolution. For higher resolution, one should increase the number of angular and radial frequencies accordingly and this increases the model complexity. From this perspective, the parameter $\\nu_{\\max}$ in B-CNN and $n_R$ in G-CNN play similar roles (although the two are respectively in frequency and real domain). Thus, it is a bit suspicious that you take a large $\\nu_{\\max}$ in B-CNN but a much smaller number of discrete rotations ($n_R=4$) in G-CNN (see also my point 2) ). The question is that when $\\nu_{\\max} \\approx n_R$ and with the same number of layers, can B-CNN still beat G-CNN? A more convincing experiment set up is to compare the performance between dfifferent neural networks with the same number of layers for varying $\\nu_{\\max}$ and $n_R$ .\n\nMinor comments:\n1. In (5), $\\nu', j’$ should be $\\nu, j$.\n2. In line 105, $J’$ should be $J$.\n3. In the first line below line 143, the subscript of the last $\\theta$ should be $\\nu', j’$. \n\nMissing references:\n1. Dense Steerable Filter CNNs for Exploiting Rotational Symmetry in Histology Images.\n2. General E(2) - Equivariant Steerable CNNs.\n\n\nIn summary, this paper applies ideas from Fourier-Bessel steerable basis to CNNs, and is well-written in general. However, its novelty is limited in my opinion (which is overclaimed in the paper), and more experiments and clarifications are expected. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "12 hrs"}, "extra_text": {}}, {"review_id": "3lEpdhS1OC", "reviewer": "Reviewer_oaoL", "summary": "The submission proposes Bessel-convolutional neural networks (B-CNNs), which are by design guaranteed to be equivariant w.r.t. translations and continuous rotations in two dimensions, i.e. the group SE(2). Convolution kernels are hereby expanded in an orthonormal function basis which consists of Bessel functions in the radial part and circular harmonics in the angular part (as commonly done in previous work). The central building block of B-CNNs is a non-linear operation, which (in its final form, eq. 22) computes the convolution response of the image with each angular harmonic frequency component individually, takes their square norm to produce locally rotation invariant responses, and finally aggregates those. This operation is motivated by an initial operation in eq. 12, which performs a group averaging over all filter or image rotations. Sections 4.3 and 4.5 are analytically manipulating this initial formulation in eq. 12 to its final formulation in eq. 22.\n\nB-CNNs are evaluated on three small scale classification datasets, namely MNIST digits, Outex-TC-00010-r textures and brain MRI images. Each experiment compares B-CNNs with a non-equivariant CNN baseline and a G-CNN architecture, which is equivariant w.r.t. translations and rotations by multiples of 90 degrees. Classification accuracies (figure 3) are reported in two settings: First, each model is trained on non-rotated inputs and then tested on rotated samples. This setting evaluates the OOD generalization over rotations which is theoretically ensured by rotation equivariant networks. The second setting trains and tests on rotated samples. While the equivariant models perform significantly better than the standard CNN baseline, neither B-CNNs nor G-CNNs are clearly outperforming each other in all settings.\nSection 5.5 and figure 4 present a small experiment which compares the equivariance of B-CNN, G-CNN and standard CNN feature maps. B-CNNs perform hereby slightly better than G-CNNs, which in turn perform better than CNNs. This is not surprising since the G-CNNs which are used are in contrast to the B-CNNs not continuously rotation equivariant.", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:\nA main argument of the authors (e.g. lines 6-8 and 31-32) is that their model is the first which is equivariant w.r.t. continuous rotations in SO(2). This claim is incorrect as there exist many continuously rotation equivariant models, for instance:\n- Harmonic Networks (Worral et al., CVPR 2017)\n- E(2)-CNNs (Weiler and Cesa, NeurIPS 2019)\n- LieConv (Finzi et al., ICML 2020)\n- Zernet (Sun et al., 2018)\n- Harmonic surface networks (Wiersma et al., Transactions on Graphics, 2020)\n\nThe method relies essentially on the use of a decomposition of the kernels' angular part into circular harmonics. It has been shown by (Lang and Weiler, ICLR 2021) that any SO(2)-equivariant model necessarily relies on such a decomposition (sometimes implicitly). However, the kernels' radial part is shown to be entirely unconstrained in rotation equivariant models - the use of Bessel functions to expand the radial parts is therefore arbitrary for the rotation equivariance of B-CNNs. The authors should clarify this point since the current submission gives the impression that this choice is essential (e.g. lines 8-11, 28-29). The use of a polynomial radial part is not new (see e.g. Zernet, who use Zernicke polynomials) but is in principle an interesting choice which is worth to be investigated.\n\n\nQuality:\nThe presented theoretical results seem in general to be technically correct. The mathematical notation could at some points be made mode formal, however, this is not a big issue in an applied field. The authors may for instance choose to adapt the following points:\n(line 51, 55): [0,2pi] should probably be changed to [0,2pi)\n(line 51): the group action R on the input and output of an equivariant function f may in general differ\n(line 104 / eq. 6): the basis is not only orthogonal but also orthonormal\n(lines 101-105): this sentence is a bit confusing. Eq. 5 does not use proper set builder notation (the \", \\nu, j in N\" should be inside the curly brackets).\n\nThe empirical evaluation is formally correct and evaluates the key theoretical claims. However, it would be desirable to evaluate the models beyond toy datasets. The claim in line 184 that MNIST-rot consists of digits which are rotated by multiples of 2pi/8 is incorrect. A comparison against previous work on this dataset would be interesting since it is the main toy benchmark for rotation equivariant models. Such an experiment would be particularly interesting since the current submission evaluates the model only against their own weak baselines. Since a particular focus is put on continuous rotation equivariance, it would furthermore be interesting to add a harmonic network baseline and a G-CNN baseline for G=C_N with N>4.\n\n\nClarity:\nThe submission is mostly clearly written, however, the irrelevant role of the Bessel radial parts for rotation equivariance remains unclear (as stated above, only the circular harmonic angular parts are relevant). The necessity of the derivation in Sections 4.3 - 4.5 is in my view unclear. One could immediately propose eq. (22) as an rotationally equivariant operation without deriving it from eq. (12), which is itself not further motivated.\n\n\nSignificance:\nGiven that continuously SE(2)-equivariant models which are based on kernels with circular harmonic angular parts, the significant contributions of the submission are 1) the use a Bessel expansion of the radial parts and 2) the nonlinear mapping of non-zero frequency harmonics responses to invariant scalars via the square norm. While the current experiments evaluate the continuous equivariance of B-CNNs, they are not investigating the significance of these two choices. The authors should therefore add experiments with baselines that use 1) different radial parts and 2) different methods to map the higher order responses to scalars.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors address some limitations of the model.\nNo immediate negative societal impact is to be expected.", "review_text": "", "extra_scores": {"ethical_concerns": "There are no ethical concerns with this paper.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: While reviewers enjoyed aspects of the paper, the key remaining concern after the discussion period was a distinct lack of comparisons (and theoretical advantages) over the state-of-the-art in rotation invariant neural networks. Several reviewers have left their final comments as replies to the replies. Please consider these carefully in preparing your revisions.
Decision: Accept (Poster)