SCALING-TRANSLATION-EQUIVARIANT NETWORKS WITH DECOMPOSED CONVOLUTIONAL FILTERS
Year: 2019

Abstract:
Encoding the scale information explicitly into the representation learned by a convolutional neural network (CNN) is beneficial for many vision tasks especially when dealing with multiscale inputs. We study, in this paper, a scaling-translationequivariant (ST -equivariant) CNN with joint convolutions across the space and the scaling group, which is shown to be both sufficient and necessary to achieve ST equivariant representations. To reduce the model complexity and computational burden, we decompose the convolutional filters under two pre-fixed separable bases and truncate the expansion to low-frequency components. A further benefit of the truncated filter expansion is the improved deformation robustness of the equivariant representation. Numerical experiments demonstrate that the proposed scaling-translation-equivariant networks with decomposed convolutional filters (ScDCFNet) achieves significantly improved performance in multiscale image classification and better interpretability than regular CNNs at a reduced model size.

heading: 1 INTRODUCTION
text: Convolutional neural networks (CNNs) have achieved great success in machine learning problems such as image classification (Krizhevsky et al., 2012), object detection (Ren et al., 2015), and semantic segmentation (Long et al., 2015; Ronneberger et al., 2015). Compared to fully-connected networks, CNNs through spatial weight sharing have the benefit of being translation-equivariant, i.e., translating the input leads to a translated version of the output. This property is crucial for many vision tasks such as image recognition and segmentation. However, regular CNNs are not equivariant to other important group transformations such as rescaling and rotation, and it is beneficial in some applications to also encode such group information explicitly into the network representation.
Several network architectures have been designed to achieve (2D) roto-translation-equivariance (SE(2)-equivariance) (Cheng et al., 2019; Marcos et al., 2017; Weiler et al., 2018b; Worrall et al., 2017; Zhou et al., 2017), i.e., roughly speaking, if the input is spatially rotated and translated, the output is transformed accordingly. The feature maps of such networks typically include an extra index for the rotation group SO(2). Building on the idea of group convolutions proposed by Cohen & Welling (2016) for discrete symmetry groups, Cheng et al. (2019) and Weiler et al. (2018b) constructed SE(2)-equivariant CNNs by conducting group convolutions jointly across the space and SO(2) using steerable filters (Freeman & Adelson, 1991). Scaling-translation-equivariant (ST -equivariant) CNNs, on the other hand, have typically been studied in a less general setting in the existing literature (Kanazawa et al., 2014; Marcos et al., 2018; Xu et al., 2014; Ghosh & Gupta, 2019). In particular, to the best of our knowledge, a joint convolution across the space and the scaling group S has yet been proposed to achieve equivariance in the most general form. This is possibly because of two difficulties one encounters when dealing with the scaling group: First, unlike SO(2), it is an acyclic and unbounded group; second, an extra index in S incurs a significant increase in model parameters and computational burden. Moreover, since the scaling transformation is rarely perfect in practice (due to changing view angle or numerical discretization), one needs to quantify and promote the deformation robustness of the equivariant representation (i.e., is the model still “approximately” equivariant if the scaling transformation is “contaminated” by a nuisance input deformation), which, to the best of our knowledge, has yet been studied in prior works.
The purpose of this paper is to address the aforementioned theoretical and practical issues in the construction of ST -equivariant CNN models. Specifically, our contribution is three-fold: 1. We propose a general ST -equivariant CNN architecture with a joint convolution over R2and S ,
which is proved in Section 4 to be both sufficient and necessary to achieve ST -equivariance. 2. A truncated decomposition of the convolutional filters under a pre-fixed separable basis on the
two geometric domains (R2 and S) is used to reduce the model size and computational cost.
3. We prove the representation stability of the proposed architecture up to equivariant scaling action of the input signal.
Our contribution to the family of group-equivariant CNNs is non-trivial; in particular, the scaling group unlike SO(2) is acyclic and non-compact. This poses challenges both in theory and in practice, so that many previous works on group-equivariant CNNs cannot be directly extended. We introduce new algorithm design and mathematical techniques to obtain the first general ST -equivariant CNN in literature with both computational efficiency and proved representation stability.

heading: 2 RELATED WORK
text: Mixed-scale and ST -equivariant CNNs. Incorporating multiscale information into a CNN representation has been studied in many existing works. The Inception net (Szegedy et al., 2015) and its generalizations (Szegedy et al., 2017; 2016; Li et al., 2019) stack filters of different sizes in a single layer to address the multiscale salient features. Dilated convolutions (Pelt & Sethian, 2018; Wang et al., 2018; Yu & Koltun, 2016; Yu et al., 2017), pyramid architectures (Ke et al., 2017; Lin et al., 2017), and multiscale dense networks (Huang et al., 2017) have also been proposed to take into account the multiscale feature information. Although the effectiveness of such models have been empirically demonstrated in various vision tasks, there is still a lack of interpretability of their ability to encode the input scale information. Group-equivariant CNNs, on the other hand, explicitly encode the group information into the network representation. Cohen & Welling (2016) proposed CNNs with group convolutions that are equivariant to several finite discrete symmetry groups. This idea is later generalized in Cohen et al. (2018) and applied mainly to the rotation groups SO(2) and SO(3) (Cheng et al., 2019; Weiler et al., 2018a;b). Although ST -equivariant CNNs have also been proposed in the literature (Kanazawa et al., 2014; Marcos et al., 2018; Xu et al., 2014; Ghosh & Gupta, 2019), they are typically studied in a less general setting. In particular, none of these previous works proposed to conduct joint convolutions over R2 × S as a necessary and sufficient condition to impose equivariance, for which reason they are thus variants of a special case of our proposed architecture where the convolutional filters in S are Dirac delta functions (c.f. Remark 1.) The scale-space semi-group correlation proposed in the concurrent work (Worrall & Welling, 2019) bears the most resemblance to our proposed model, however, their approach is only limited to discrete semigroups, whereas our model does not have such restriction.
Representation stability to input deformations. Input deformations typically induce noticeable variabilities within object classes, some of which are uninformative for the vision tasks. Models that are stable to input deformations are thus favorable in many applications. The scattering transform (Bruna & Mallat, 2013; Mallat, 2010; 2012) computes translation-invariant representations that are Lipschitz continuous to deformations by cascading predefined wavelet transforms and modulus poolings. A joint convolution over R2 × SO(2) is later adopted in Sifre & Mallat (2013) to build roto-translation scattering with stable rotation/translation-invariant representations. These models, however, use pre-fixed wavelet transforms in the networks, and are thus nonadaptive to the data. DCFNet (Qiu et al., 2018) combines a pre-fixed filter basis and learnable expansion coefficients in a CNN architecture, achieving both data adaptivity and representation stability inherited from the filter regularity. This idea is later extended by Cheng et al. (2019) to produce SE(2)-equivariant representations that are Lipschitz continuous in L2 norm to input deformations modulo a global rotation, i.e., the model stays approximately equivariant even if the input rotation is imperfect. To the best of our knowledge, a theoretical analysis of the deformation robustness of a ST -equivariant CNN has yet been studied, and a direct generalization of the result in Cheng et al. (2019) is futile because the feature maps of a ST -equivariant CNN is typically not in L2 (c.f. Remark 2.)

heading: 3 ST -EQUIVARIANT CNN AND FILTER DECOMPOSITION
text: Group-equivariance is the property of a mapping f : X → Y to commute with the group actions on the domain X and codomain Y . More specifically, let G be a group, and Dg, Tg, respectively, be group actions on X and Y . A function f : X → Y is said to be G-equivariant if
f(Dgx) = Tg(f(x)), ∀ g ∈ G, x ∈ X. (1)
G-invariance is thus a special case of G-equivariance where Tg = IdY . For learning tasks where the feature y ∈ Y is known a priori to change equivariantly to a group action g ∈ G on the input x ∈ X , e.g. image segmentation should be equivariant to translation, it would be beneficial to reduce the hypothesis space to include only G-equivaraint models. In this paper, we consider mainly the scaling-translation group ST ∼= S × R2 ∼= R × R2. Given g = (β, v) ∈ ST and an input image
x(0)(u, λ) (u ∈ R2 is the spatial position, and λ is the unstructured channel index, e.g. RGB channels of a color image), the scaling-translation group action Dg = Dβ,v on x(0) is defined as
Dβ,vx (0)(u, λ) := x(0) ( 2−β(u− v), λ ) . (2)
Constructing ST -equivariant CNNs thus amounts to finding an architecture A such that each trained network f ∈ A commutes with the group action Dβ,v on the input and a similarly defined group action Tβ,v (to be explained in Section 3.1) on the output.

heading: 3.1 ST -EQUIVARIANT CNNS
text: Inspired by Cheng et al. (2019) and Weiler et al. (2018b), we consider ST -equivariant CNNs with an extra index α ∈ S for the the scaling group S ∼= R: for each l ≥ 1, the l-th layer output is denoted as x(l)(u, α, λ), where u ∈ R2 is the spatial position, α ∈ S is the scale index, and λ ∈ [Ml] := {1, . . . ,Ml} corresponds to the unstructured channels. We use the continuous model for formal derivation, i.e., the images and feature maps have continuous spatial and scale indices. In practice, the images are discretized on a Cartesian grid, and the scales are computed only on a discretized finite interval. Similar to Cheng et al. (2019), the group action Tβ,v on the l-th layer output is defined as a scaling-translation in space as well as a shift in the scale channel:
Tβ,vx (l)(u, α, λ) := x(l) ( 2−β(u− v), α− β, λ ) , ∀ l ≥ 1. (3)
A feedforward neural network is said to be scaling-translation-equivariant, i.e., equivariant to ST , if
x(l)[Dβ,vx (0)] = Tβ,vx (l)[x(0)], ∀ l ≥ 1, (4)
where we slightly abuse the notation x(l)[x(0)] to denote the l-th layer output given the input x(0). The following Theorem shows that ST -equivariance is achieved if and only if joint convolutions are conducted over S × R2 as in (5) and (6). Theorem 1. A feedforward neural network with an extra index α ∈ S for layerwise output is ST -equivariant if and only if the layerwise operations are defined as (5) and (6):
x(1)[x(0)](u, α, λ) = σ (∑ λ′ ∫ R2 2−2αx(0)(u+ u′, λ′)W (1) λ′,λ ( 2−αu′ ) du′ + b(1)(λ) ) , (5)
x(l)[x(l−1)](u, α, λ) = σ (∑ λ′ ∫ R2 ∫ R 2−2αx(l−1)(u+ u′, α+ α′, λ′) ·
W (l) λ′,λ
( 2−αu′, α′ ) dα′du′ + b(l)(λ) ) , ∀l > 1, (6)
where σ : R→ R is a pointwise nonlinear function.
We defer the proof of Theorem 1, as well as those of other theorems, to the appendix. We note that the joint-convolution in Theorem 1 is a generalization of the group convolution proposed by Cohen & Welling (2016) to a non-compact group ST in the continuous setting.
Remark 1. When the (joint) convolutional filter W (l)λ′,λ(u, α) takes the special form W (l) λ′,λ(u, α) = V (l) λ′,λ(u)δ(α), the joint convolution (6) over R2×S reduces to only a (multiscale) spatial convolution
x(l)[x(l−1)](u, α, λ) = σ (∑ λ′ ∫ R2 x(l−1)(u+ u′, α, λ′)V (l) λ′,λ ( 2−αu′ ) 2−2αdu′ + b(l)(λ) ) , (7)
i.e., the feature maps at different scales do not transfer information among each other (see Figure 1a). The previous works (Kanazawa et al., 2014; Marcos et al., 2018; Xu et al., 2014; Ghosh & Gupta, 2019) on ST -equivariant CNNs are all based on this special case of Theorem 1.
Although the joint convolutions (6) on R2 × S provide the most general way of imposing ST - equivariance, they unfortunately also incur a significant increase in the model size and computational burden. Following the idea of Cheng et al. (2019) and Qiu et al. (2018), we address this issue by taking a truncated decomposition of the convolutional filters under a pre-fixed separable basis, which will be discussed in detail in the next section.

heading: 3.2 SEPARABLE BASIS DECOMPOSITION
text: We consider decomposing the convolutional filters W (l)λ′,λ(u, α) under the product of two function bases, {ψk(u)}k and {ϕm(α)}m, which are the eigenfunctions of the Dirichlet Laplacian on, respectively, the unit disk D ⊂ R2 and [−1, 1], i.e.,{
∆ψk = −µkψk in D, ψk = 0 on ∂D, and { ϕ′′m = −νmϕm in [−1, 1] ϕm(−1) = ϕm(1) = 0.
(8)
In particular, the spatial basis {ψk}k satisfying (8) is the Fourier-Bessel (FB) basis (Abramowitz & Stegun, 1965). In the continuous formulation, the spatial “pooling” operation is equivalent to rescaling the convolutional filters in space. We thus assume, without loss of generality, that the convolutional filters are compactly supported as follows
W (1) λ′,λ ∈ Cc(2 j1D), and W (l)λ′,λ ∈ Cc(2 jlD × [−1, 1]), ∀ l > 1. (9)
Let ψj,k(u) := 2−2jψk(2−ju), then W (l) λ′,λ can be decomposed under {ψjl,k}k and {ϕm}m as
W (1) λ′,λ(u) = ∑ k a (1) λ′,λ(k)ψj1,k(u), W (l) λ′,λ(u, α) = ∑ m ∑ k a (l) λ′,λ(k,m)ψjl,k(u)ϕm(α), l > 1 (10)
where a(1)λ′,λ(k) and a (l) λ′,λ(k,m) are the expansion coefficients of the filters. During training, the basis functions are fixed, and only the expansion coefficients are updated. In practice, we truncate the expansion to only low-frequency components (i.e., a(l)λ′,λ(k,m) are non-zero only for k ∈ [K], m ∈ [Kα]), which are kept as the trainable parameters. Similar idea has also been considered in the prior works (Qiu et al., 2018; Cheng et al., 2019; Jacobsen et al., 2016). This directly leads to a reduction of network parameters and computational burden. More specifically, let us compare the l-th convolutional layer (6) of a ST -equivariant CNN with and without truncated basis decomposition:
Number of trainable parameters: Suppose the filters W (l)λ′,λ(u, α) are discretized on a Cartesian grid of size L×L×Lα. The number of trainable parameters at the l-th layer of a ST -equivariant CNN without basis decomposition is L2LαMl−1Ml. On the other hand, in an ScDCFNet with truncated basis expansion up toK leading coefficients for u andKα coefficients for α, the number of parameters is instead KKαMl−1Ml. Hence a reduction to a factor of KKα/L2Lα in trainable parameters is achieved for ScDCFNet via truncated basis decomposition. In particular, if L = 5, Lα = 5,K = 8, and Kα = 3, then the number of parameters is reduced to (8× 3)/(52 × 5) = 19.2%.
Computational cost: Suppose the size of the input x(l−1)(u, α, λ) and output x(l)(u, α, λ) at the l-th layer are, respectively, W ×W × Nα ×Ml−1 and W ×W × Nα ×Ml, where W ×W is the spatial dimension, Nα is the number of scale channels, and Ml−1(Ml) is the number of the unstructured input (output) channels. Let the filters W (l)λ′,λ(u, α) be discretized on a Cartesian grid of size L× L× Lα. The following theorem shows that, compared to a regular ST -equivariant CNN, the computational cost in a forward pass of ScDCFNet is reduced again to a factor of KKα/L2Lα. Theorem 2. Assume Ml L2, Lα, i.e., the number of the output channels is much larger than the size of the convolutional filters in u and α, then the computational cost of an ScDCFNet is reduced to a factor of KKα/L2Lα when compared to a ST -equivariant CNN without basis decomposition.

heading: 4 REPRESENTATION STABILITY OF SCDCFNET TO INPUT DEFORMATION
text: Apart from reducing the model size and computational burden, we demonstrate in this section that truncating the filter decomposition has the further benefit of improving the deformation robustness of the equivariant representation, i.e., the equivaraince relation (4) still approximately holds true even if the spatial scaling of the input Dβ,vx(0) is contaminated by a local deformation. The analysis is motivated by the fact that scaling transformations are rarely perfect in practice — they are typically subject to local distortions such as changing view angle or numerical discretization. To quantify the distance between different feature maps at each layer, we define the norm of x(l) as
‖x(0)‖2 = 1 M0 M0∑ λ=1 ∫ ∣∣∣x(0)(u, λ)∣∣∣2 du, ‖x(l)‖2 = sup α 1 Ml Ml∑ λ=1 ∫ ∣∣∣x(l)(u, α, λ)∣∣∣2 du, l ≥ 1. (11) Remark 2. The definition of ‖x(l)‖ is different from that of RotDCFNet (Cheng et al., 2019), where an L2 norm is taken for the α index as well. The reason why we adopt the L∞ norm for α in (11) is that x(l) is typically not L2 in α, since the scaling group S , unlike SO(2), has infinite Haar measure.
We next quantify the representation stability of ScDCFNet under three mild assumptions on the convolutional layers and input deformations. First,
(A1) The pointwise nonlinear activation σ : R→ R is non-expansive. Next, we need a bound on the convolutional filters under certain norms. For each l ≥ 1, define Al as A1 := πmax { sup λ M0∑ λ′=1 ‖a(1)λ′,λ‖FB, M0 M1 sup λ′ M1∑ λ=1 ‖a(1)λ′,λ‖FB } , Al := πmax supλ Ml−1∑ λ′=1 ∑ m ‖a(l)λ′,λ(·,m)‖FB, 2Ml−1 Ml ∑ m sup λ′ Ml∑ λ=1 ‖a(l)λ′,λ(·,m)‖FB  , (12)
where the Fourier-Bessel (FB) norm ‖a‖FB of a sequence {a(k)}k≥0 is a weighted l2 norm defined as ‖a‖2FB := ∑ k µka(k)
2, where µk is the k-th eigenvalue of the Dirichlet Laplacian on the unit disk defined in (8). We next assume that each Al is bounded:
(A2) For all l ≥ 1, Al ≤ 1. The boundedness of Al is facilitated by truncating the basis decomposition to only low-frequency components (small µk), which is one of the key idea of ScDCFNet explained in Section 3.2. After a proper initialization of the trainable coefficients, (A2) can generally be satisfied. The assumption (A2) implies several bounds on the convolutional filters at each layer (c.f. Lemma 2 in the appendix), which, combined with (A1), guarantees that an ScDCFNet is layerwise non-expansive: Proposition 1. Under the assumption (A1) and (A2), an ScDCFNet satisfies the following. (a) For any l ≥ 1, the mapping of the l-th layer, x(l)[·] defined in (5) and (6), is non-expansive, i.e.,
‖x(l)[x1]− x(l)[x2]‖ ≤ ‖x1 − x2‖, ∀x1, x2. (13)
(b) Let x(l)0 be the l-th layer output given a zero bottom-layer input, then x (l) 0 (λ) depends only on λ.
(c) Let x(l)c be the centered version of x(l) after removing x (l) 0 , i.e., x (0) c (u, λ) := x(0)(u, λ) −
x (0) 0 (λ) = x (0)(u, λ), and x(l)c (u, α, λ) := x(l)(u, α, λ) − x(l)0 (λ), ∀l ≥ 1, then ‖x (l) c ‖ ≤ ‖x(l−1)c ‖, ∀l ≥ 1. As a result, ‖x(l)c ‖ ≤ ‖x(0)c ‖ = ‖x(0)‖.
Finally, we make an assumption on the input deformation modulo a global scale change. Given a C2
function τ : R2 → R2, the spatial deformation Dτ on the feature maps x(l) is defined as Dτx
(0)(u, λ) = x(0)(ρ(u), λ), and Dτx(l)(u, α, λ) = x(l)(ρ(u), α, λ), l ≥ 1, (14) where ρ(u) = u− τ(u). We assume a small local deformation on the input:
(A3) |∇τ |∞ := supu ‖∇τ(u)‖ < 1/5, where ‖ · ‖ is the operator norm. The following theorem demonstrates the representation stability of an ScDCFNet to input deformation modulo a global scale change.
Dβ,vx
(0), the four figures in each dashed rectangle are: x(l)[x(0)] (l-th layer feature of the original input), x(l)[Dβ,vx
(0)] (l-th layer feature of the rescaled input), Tβ,vx(l)[x(0)] (rescaled l-th layer feature of the original input), and the difference (x(l)[Dβ,vx(0)] − Tβ,vx(l)[x(0)]) displayed in a (signal intensity) scale relative to the maximum value of x(l)[Dβ,vx(0)]. It is clear that even after numerical discretization, ST -equivariance still approximately holds for ScDCFNet, i.e., x(l)[Dβ,vx(0)]− Tβ,vx(l)[x(0)] ≈ 0, but not for a regular CNN.
Theorem 3. Let Dτ be a small spatial deformation defined in (14), and let Dβ,v, Tβ,v be the group actions corresponding to an arbitrary scaling 2−β ∈ R+ centered at v ∈ R2 defined in (2) and (3). In an ScDCFNet satisfying (A1), (A2), and (A3), we have, for any L,∥∥∥x(L)[Dβ,v ◦Dτx(0)]− Tβ,vx(L)[x(0)]∥∥∥ ≤ 2β+1 (4L|∇τ |∞ + 2−jL |τ |∞) ‖x(0)‖. (15) Theorem 3 gauges how approximately equivariant is ScDCFNet if the input undergoes not only a scale change Dβ,v but also a nonlinear spatial deformation Dτ , which is important both in theory and in practice because the scaling of an object is rarely perfect in reality.

heading: 5 NUMERICAL EXPERIMENTS
text: In this section, we conduct several numerical experiments for the following three purposes. 1. To verify that ScDCFNet indeed achieves ST -equivariance (4). 2. To illustrate that ScDCFNet significantly outperforms regular CNNs at a much reduced model
size in multiscale image classification. 3. To show that a trained ScDCFNet auto-encoder is able to reconstruct rescaled versions of the
input by simply applying group actions on the image codes, demonstrating that ScDCFNet indeed explicitly encodes the input scale information into the representation.
The experiments are tested on the Scaled MNIST (SMNIST) and Scaled Fashion-MNIST (SFashion) datasets, which are built by rescaling the original MNIST and Fashion-MNIST (Xiao et al., 2017) images by a factor randomly sampled from a uniform distribution on [0.3, 1]. A zero-padding to a size of 28× 28 is conducted after the rescaling. If mentioned explicitly, for some experiments, the images are resized to 64× 64 for better visualization. The implementation details of ScDCFNet are explained in Appendix B.1. In particular, we discuss how to discretize the integral (6) and truncate the scale channel to a finite interval for practical implementation. We also explain how to mitigate the boundary “leakage” effect incurred by the truncation. Moreover, modifications to the spatial pooling and batch-normalization modules of ScDCFNet to maintain ST -equivariance are also explained.

heading: 5.1 VERIFICATION OF ST -EQUIVARIANCE
text: We first verify that ScDCFNet indeed achieves ST -equivariance (4). Specifically, we compare the feature maps of a two-layer ScDCFNet with randomly generated truncated filter expansion coefficients and those of a regular CNN. The exact architectures are detailed in Appendix B.2. Figure 2 displays the first- and second-layer feature maps of an original image x(0) and its rescaled version Dβ,vx(0) using the two comparing architectures. Feature maps at different layers are rescaled to the same spatial dimension for visualization. The four images enclosed in each of the dashed rectangle correspond to: x(l)[x(0)] (l-th layer feature of the original input), x(l)[Dβ,vx(0)] (l-th layer feature of the rescaled input), Tβ,vx(l)[x(0)] (rescaled l-th layer feature of the original input, where Tβ,v is understood as Dβ,v for a regular CNN due to the lack of a scale index α), and the difference
x(l)[Dβ,vx (0)] − Tβ,vx(l)[x(0)]. It is clear that even with numerical discretization, which can be modeled as a form of input deformation, ScDCFNet is still approximately ST -equivariant, i.e., x(l)[Dβ,vx
(0)] ≈ Tβ,vx(l)[x(0)], whereas a regular CNN does not have such a property. We also examin how the numerical error in equivariance (incurred by the the boundary “leakage” effect after the scale channel truncation) evolves as the network gets deeper. The error in equivariance is measured in a relative L2 sense at a particular scale α, i.e.,
Error = ‖x(l)[Dβ,vx(0)](·, α)− Tβ,vx(l)[x(0)](·, α)‖L2/‖Tβ,vx(l)[x(0)](·, α)‖L2 . (16)
It is clear from Figure 3 that the boundary “leakage” effect is unavoidable as the network becomes deeper. However, the error can be alleviated by either choosing joint convolutional filters with a smaller support in scale (i.e., the filter size, or the number of “taps”, after discretization in scale is much smaller compared to the number of scale channels in the feature map), or using a replicatepadding in the scale channel for the joint convolution. See Appendix B.1 for detailed explanation.

heading: 5.2 MULTISCALE IMAGE CLASSIFICATION
text: We next demonstrate the improved performance of ScDCFNet in multiscale image classification. The experiments are conducted on SMNIST and SFashion, and a regular CNN is used as a performance benchmark. Both networks are comprised of three convolutional layers with the exact architectures (Table 2) detailed in Appendix B.3. Since the scaling group S ∼= R is unbounded, we compute only the feature maps x(l)(u, α, λ) with the α index restricted to the truncated scale interval [−1.6, 0] (2−1.6 ≈ 0.3), which is discretized uniformly into Nα = 9 channels (again, see Appendix B.1 for implementation details.) The performance of the comparing architectures with and without batch-normalization is shown in Table 1. It is clear that, by limiting the hypothesis space to ST - equivaraint models and taking truncated basis decomposition to reduce the model size, ScDCFNet achieves a significant improvement in classification accuracy with a reduced number of trainable parameters. The advantage of ScDCFNet is more pronounced when the number of training samples is small (Ntr = 2000), suggesting that, by hardwiring the input scale information directly into its representation, ScDCFNet is less susceptible to overfitting the limited multiscale training data.
We also observe that even when a regular CNN is trained with data augmentation (random cropping and rescaling), its performance is still inferior to that of an ScDCFNet without manipulation of the training data. In particular, although the accuracies of the regular CNNs trained on 2000 SMNIST and SFashion images after data augmentation are improved to, respectively, 93.85% and 79.41%, they still underperform the ScDCFNets without data augmentation (93.91% and 79.94%) using only a fraction of trainable parameters. Moreover, if ScDCFNet is trained with data augmentation, the accuracies can be further improved to 94.30% and 80.62% respectively. This suggests that ScDCFNet can be combined with data augmentation for optimal performance in multiscale image classification.

heading: 5.3 IMAGE RECONSTRUCTION
text: In the last experiment, we illustrate the ability of ScDCFNet to explicitly encode the input scale information into the representation. To achieve this, we train an ScDCFNet auto-encoder on the SMNIST dataset with images resized to 64× 64 for better visualization. The encoder stacks two ST - equivaraint convolutional blocks with 2× 2 average-pooling, and the decoder contains a succession of two transposed convolutional blocks with 2× 2 upsampling. A regular CNN auto-encoder is also trained for comparison (see Table 3 in Appendix B.4 for the detailed architecture.)
Our goal is to demonstrate that the image code produced by the ScDCFNet auto-encoder contains the scale information of the input, i.e., by applying the group action Tβ,v (3) to the code C of a test
image before feeding it to the decoder, we can reconstruct rescaled versions of original input. This property can be visually verified in Figure 4. In contrast, a regular CNN auto-encoder fails to do so.

heading: 6 CONCLUSION
text: We propose, in this paper, a ST -equivaraint CNN with joint convolutions across the space R2 and the scaling group S, which we show to be both sufficient and necessary to impose ST -equivariant network representation. To reduce the computational cost and model complexity incurred by the joint convolutions, the convolutional filters supported on R2 × S are decomposed under a separable basis across the two domains and truncated to only low-frequency components. Moreover, the truncated filter expansion leads also to improved deformation robustness of the equivaraint representation, i.e., the model is still approximately equivariant even if the scaling transformation is imperfect. Experimental results suggest that ScDCFNet achieves improved performance in multiscale image classification with greater interpretability and reduced model size compared to regular CNN models.
For future work, we will study the application of ScDCFNet in other more complicated vision tasks, such as object detection/localization and pose estimation, where it is beneficial to directly encode the input scale information into the deep representation. Moreover, the memory usage of our current implementation of ScDCFNet scales linearly to the number of the truncated basis functions in order to realize the reduced computational burden explained in Theorem 2. We will explore other efficient implementation of the model, e.g., using filter-bank type of techniques to compute convolutions with multiscale spatial filters, to significantly reduce both the computational cost and memory usage.

heading: A PROOFS
text: A.1 PROOF OF THEOREM 1
Proof of Theorem 1. We note first that (4) holds true if and only if the following being valid for all l ≥ 1,
Tβ,vx (l)[x(l−1)] = x(l)[Tβ,vx (l−1)], (17)
where Tβ,vx(0) is understood as Dβ,vx(0). We also note that the layer-wise operations of a general feedforward neural network with an extra index α ∈ S can be written as
x(1)[x(0)](u, α, λ) = σ (∑ λ′ ∫ R2 x(0)(u+ u′, λ′)W (1)(u′, λ′, u, α, λ)du′ + b(1)(λ) ) , (18)
and, for l > 1,
x(l)[x(l−1)](u, α, λ) =σ (∑ λ′ ∫ R2 ∫ R x(l−1)(u+ u′, α+ α′, λ′)
W (l)(u′, α′, λ′, u, α, λ)dα′du′ + b(l)(λ) ) . (19)
To prove the sufficient part: when l = 1, (2), (3), and (5) lead to
Tβ,vx (1)[x(0)](u, α, λ) = x(1)[x(0)] ( 2−β(u− v), α− β, λ ) =σ
(∑ λ′ ∫ x(0) ( 2−β(u− v) + u′, λ′ ) W (1) λ′,λ ( 2−(α−β)u′ ) 2−2(α−β)du′ + b(1)(λ) )
=σ (∑ λ′ ∫ x(0) ( 2−β(u− v + ũ), λ′ ) W (1) λ′,λ ( 2−αũ ) 2−2αdũ+ b(1)(λ) ) , (20)
and
x(1)[Dβ,vx (0)](u, α, λ)
=σ (∑ λ′ ∫ R2 Dβ,vx (0)(u+ u′, λ′)W (1) λ′,λ ( 2−αu′ ) 2−2αdu′ + b(1)(λ) )
=σ (∑ λ′ ∫ R2 x(0)(2−β (u+ u′ − v), λ′)W (1)λ′,λ ( 2−αu′ ) 2−2αdu′ + b(1)(λ) ) . (21)
Hence Tβ,vx(1)[x(0)] = x(1)[Dβ,vx(0)].
When l > 1, we have
Tβ,vx (l)[x(l−1)](u, α, λ) = x(l)[x(l−1)] ( 2−β(u− v), α− β, λ ) =σ
(∑ λ′ ∫ ∫ x(l−1) ( 2−β(u− v) + u′, α− β + α′, λ′ ) ·
W (l) λ′,λ
( 2−(α−β)u′, α′ ) 2−2(α−β)du′dα′ + b(l)(λ) ) =σ
(∑ λ′ ∫ ∫ x(l−1) ( 2−β(u− v + ũ), α− β + α′, λ′ ) W (l) λ′,λ ( 2−αũ, α′ ) 2−2αdũdα′ + b(l)(λ) ) (22)
and x(l)[Tβ,vx (l−1)](u, α, λ)
=σ (∑ λ′ ∫ ∫ Tβ,vx (l−1)(u+ u′, α+ α′, λ′)W (l) λ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ + b(l)(λ) )
=σ (∑ λ′ ∫ ∫ x(l−1)(2−β(u+ u′ − v), α+ α′ − β, λ′)W (l)λ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ + b(l)(λ) ) (23)
Therefore Tβ,vx(l)[x(l−1)] = x(l)[Tβ,vx(l−1)], ∀l > 1. To prove the necessary part: when l = 1, we have
Tβ,vx (1)[x(0)](u, α, λ) = x(1)[x(0)] ( 2−β(u− v), α− β, λ ) =σ
(∑ λ′ ∫ x(0) ( 2−β(u− v) + u′, λ′ ) W (1) ( u′, λ′, 2−β(u− v), α− β, λ ) du′ + b(1)(λ) )
=σ (∑ λ′ ∫ x(0) ( 2−β(u+ u′ − v), λ′ ) ·
W (1) ( 2−βu′, λ′, 2−β(u− v), α− β, λ ) 2−2βdu′ + b(1)(λ) ) , (24)
and
x(1)[Dβ,vx (0)](u, α, λ) = σ (∑ λ′ ∫ Dβ,vx (0) (u+ u′, λ′)W (1) (u′, λ′, u, α, λ) du′ + b(1)(λ) )
=σ (∑ λ′ ∫ x(0) ( 2−β(u+ u′ − v), λ′ ) W (1) (u′, λ′, u, α, λ) du′ + b(1)(λ) ) (25)
Hence for (17) to hold when l = 1, we need W (1) (u′, λ′, u, α, λ) = W (1) ( 2−βu′, λ′, 2−β(u− v), α− β, λ ) 2−2β ,∀ u, α, λ, u′, λ′, v, β. (26)
Keeping u, α, λ, u′, λ′, β fixed while changing v in (26), we obtain that W (1)(u′, λ′, u, α, λ) does not depend on the third variable u. Thus W (1) (u′, λ′, u, α, λ) = W (1) (u′, λ′, 0, α, λ) , ∀u. Define W
(1) λ,λ(u ′) as
W (1) λ′,λ(u ′) := W (1) (u′, λ′, 0, 0, λ) . (27)
Then, for any given u′, λ′, u, α, λ, setting β = α in (26) leads to W (1) (u′, λ′, u, α, λ) = W (1) ( 2−αu′, λ′, 2−α(u− v), 0, λ ) 2−2α
=W (1) ( 2−αu′, λ′, 0, 0, λ ) 2−2α = W
(1) λ′,λ
( 2−αu′ ) 2−2α. (28)
Hence (18) can be written as (5).
For l > 1, a similar argument leads to W (l) (u′, α′, λ′, u, α, λ) = W (l) ( 2−βu′, α′, λ′, 2−β(u− v), α− β, λ ) 2−2β ,∀u, α, λ, u′, α′, λ′, v, β.
(29) Again, keeping u, α, λ, u′, α′, λ′, β fixed while changing v in (29) leads us to the conclusion that W (l)(u′, α′, λ′, u, α, λ) does not depend on the fourth variable u. Define
W (l) λ′,λ(u ′, α′) := W (l) (u′, α′, λ′, 0, 0, λ) . (30) After setting β = α in (29), for any given u′, α′, λ′, u, α, λ, we have
W (l) (u′, α′, λ′, u, α, λ) = W (l) ( 2−αu′, α′, λ′, 2−α(u− v), 0, λ ) 2−2α
=W (l) ( 2−αu′, α′, λ′, 0, 0, λ ) 2−2α = W
(l) λ′,λ
( 2−αu′ ) 2−2α. (31)
This concludes the proof of the Theorem.
A.2 PROOF OF THEOREM 2
Proof of Theorem 2. In a regular ST -equivariant CNN, the l-th convolutional layer (6) is computed as follows:
y(u, α, α′, λ, λ′) = ∫ R2 x(l−1)(u+ u′, α+ α′, λ′)W (l) λ′,λ ( 2−αu′, α′ ) 2−2αdu′, (32)
z(u, α, λ, λ′) = ∫ R y(u, α, α′, λ, λ′)dα′, (33)
x(l)(u, α, λ) = σ Ml−1∑ λ′=1 z(u, α, λ, λ′) + b(l)(λ)  . (34) The spatial convolutions in (32) take 2W 2L2NαLαMlMl−1 flops (there are NαLαMlMl−1 convolutions in u, each taking 2W 2L2 flops.) The summation over α′ in (33) requires LαNαW 2MlMl−1 flops. The summation over λ′, adding the bias, and applying the nonlinear activation in (34) requires an additional W 2NαMl(2 +Ml−1) flops. Thus the total number of floating point computations in a forward pass through the l-th layer of a regular ST -equivariant CNN is
W 2NαMl(2L 2LαMl−1 + LαMl−1 +Ml−1 + 2). (35)
On the other hand, in an ScDCFNet with separable basis truncation up to KKα leading coefficients, (6) can be computed via the following steps:
y(u, α, λ′,m) = ∫ R x(l−1)(u, α+ α′, λ′)ϕm(α ′)dα′ (36)
z(u, α, λ′, k,m) = ∫ R2 y(u+ u′, α, λ′,m)ψjl,k(2 −αu′)2−2αdu′ (37)
x(l)(u, α, λ) = σ  Kα∑ m=1 K∑ k=1 Ml−1∑ λ′=1 z(u, α, λ′, k,m)a (l) λ′,λ(k,m) + b (l)(λ)  . (38) The convolutions in α (36) require 2LαNαW 2KαMl−1 flops (there are W 2KαMl−1 convolutions in α, each taking 2LαNα flops.) The spatial convolutions in (37) take 2W 2L2NαMl−1KαK flops (NαMl−1KαK convolutions in u, each taking 2W 2L2 flops.) The last step (38) requires an additional 2W 2NαMl(1 + KKαMl−1) flops. Hence the total number of floating point computation for an ScDCFNet is
2W 2Nα(KKαMl−1Ml +Ml + L 2Ml−1KαK + LαKαMl−1). (39)
In particular, when Ml L2, Lα, the dominating terms in (35) and (39) are, respectively, 2W 2NαMlMl−1L
2Lα and 2W 2NαMl−1MlKKα. Thus the computational cost in an ScDCFNet has been reduced to a factor of KKαL2Lα .
A.3 PROOF OF PROPOSITION 1
Before proving Proposition 1, we need the following two lemmas. Lemma 1. Suppose that {ψk}k are the FB bases, and F (u) = ∑ k a(k)ψj,k(u) =∑
k a(k)2 −2jψk(2 −ju) is a smooth function on 2jB(0, 1), then∫ |F (u)| du, ∫ |u| |∇F (u)| du, 2j ∫ |∇F (u)| du ≤ π‖a‖FB = π (∑ k µk · a(k)2 )1/2 . (40)
This is Lemma 3.5 and Proposition 3.6 in Qiu et al. (2018) after rescaling u. Lemma 1 easily leads to the following lemma.
Lemma 2. Let a(l)λ′,λ(k,m) be the coefficients of the filter W (l) λ′,λ(u, α) under the joint bases {ψk}k and {ϕm}m defined in (10), and define W (l) λ′,λ,m(u) as
Wλ′,λ,m(u) := ∑ k a (l) λ′,λ(k,m)ψjl,k(u). (41)
We have
B (1) λ′,λ, C (1) λ′,λ, 2 j1D (1) λ′,λ ≤ π‖a (1) λ′,λ‖FB, B (l) λ′,λ,m, C (l) λ′,λ,m, 2 jlD (l) λ′,λ,m ≤ π‖a (l) λ′,λ(·,m)‖FB, ∀l > 1,
(42)
where B (1) λ′,λ := ∫ ∣∣∣W (1)λ′,λ(u)∣∣∣ du, B(l)λ′,λ,m := ∫ ∣∣∣W (l)λ′,λ,m(u)∣∣∣ du, l > 1, C (1) λ′,λ := ∫ |u| ∣∣∣∇uW (1)λ′,λ(u)∣∣∣ du, C(l)λ′,λ,m := ∫ |u| ∣∣∣∇uW (l)λ′,λ,m(u)∣∣∣ du, l > 1,
D (1) λ′,λ := ∫ ∣∣∣∇uW (1)λ′,λ(u)∣∣∣ du, D(l)λ′,λ,m := ∫ ∣∣∣∇uW (l)λ′,λ,m(u)∣∣∣ du, l > 1. (43)
We thus have
Bl, Cl, 2 jlDl ≤ Al, (44)
where
B1 := max { sup λ M0∑ λ′=1 B (1) λ′,λ, M0 M1 sup λ′ M1∑ λ=1 B (1) λ′,λ } ,
C1 := max { sup λ M0∑ λ′=1 C (1) λ′,λ, M0 M1 sup λ′ M1∑ λ=1 C (1) λ′,λ } ,
D1 := max { sup λ M0∑ λ′=1 D (1) λ′,λ, M0 M1 sup λ′ M1∑ λ=1 D (1) λ′,λ } ,
(45)
and, for l > 1,
Bl := max supλ Ml−1∑ λ′=1 ∑ m B (l) λ′,λ,m, 2Ml−1 Ml ∑ m Bl,m  , Bl,m := supλ′ Ml∑ λ=1 B (l) λ′,λ,m,
Cl := max supλ Ml−1∑ λ′=1 ∑ m C (l) λ′,λ,m, 2Ml−1 Ml ∑ m Cl,m  , Cl,m := supλ′ Ml∑ λ=1 C (l) λ′,λ,m,
Dl := max supλ Ml−1∑ λ′=1 ∑ m D (l) λ′,λ,m, 2Ml−1 Ml ∑ m Dl,m  , Dl,m := supλ′ Ml∑ λ=1 D (l) λ′,λ,m.
(46)
In particular, (A2) implies that Bl, Cl, 2jlDl ≤ 1, ∀l.
Proof of Proposition 1. To simplify the notation, we omit (l) in W (l)λ′,λ, W (l) λ′,λ,m, and b (l), and let M = Ml, M ′ = Ml−1. The proof of (a) for the case l = 1 is similar to Proposition 3.1(a) of Qiu et al. (2018) after noticing the fact that∫
R2 ∣∣W (2−αu)∣∣ 2−2αdu = ∫ R2 |W (u)| du, (47)
and we include it here for completeness. From the definition of B1 in (45), we have
sup λ ∑ λ′ B (1) λ′,λ ≤ B1, and sup λ′ ∑ λ B (1) λ′,λ ≤ B1 M M ′ . (48)
Thus, given two arbitrary functions x1 and x2, we have∣∣∣(x(1)[x1]− x(1)[x2]) (u, α, λ)∣∣∣2 = ∣∣∣∣∣σ (∑
λ′
∫ x1(u+ u ′, λ′)Wλ′,λ ( 2−αu′ ) 2−2αdu′ + b(λ) )
−σ (∑ λ′ ∫ x2(u+ u ′, λ′)Wλ′,λ ( 2−αu′ ) 2−2αdu′ + b(λ) )∣∣∣∣∣ 2
≤ ∣∣∣∣∣∑ λ′ ∫ x1(u+ u ′, λ′)Wλ′,λ ( 2−αu′ ) 2−2αdu′ − ∑ λ′ ∫ x2(u+ u ′, λ′)Wλ′,λ ( 2−αu′ ) 2−2αdu′ ∣∣∣∣∣ 2
= ∣∣∣∣∣∑ λ′ ∫ (x1 − x2)(u+ u′, λ′)Wλ′,λ ( 2−αu′ ) 2−2αdu′ ∣∣∣∣∣ 2
≤ (∑ λ′ ∫ |(x1 − x2)(u+ u′, λ′)| 2 ∣∣Wλ′,λ(2−αu′)∣∣ 2−2αdu′)(∑ λ′ ∫ ∣∣Wλ′,λ(2−αu′)∣∣ 2−2αdu′)
= (∑ λ′ ∫ |(x1 − x2)(u+ u′, λ′)| 2 ∣∣Wλ′,λ(2−αu′)∣∣ 2−2αdu′)(∑ λ′ B (1) λ′,λ )
≤B1 ∑ λ′ ∫ |(x1 − x2)(ũ, λ′)| 2 ∣∣Wλ′,λ(2−α (ũ− u))∣∣ 2−2αdũ (49) Therefore, for any α,∑
λ ∫ ∣∣∣(x(1)[x1]− x(1)[x2]) (u, α, λ)∣∣∣2 du ≤ ∑ λ ∫ B1 ∑ λ′ ∫ |(x1 − x2)(ũ, λ′)|
2 ∣∣Wλ′,λ(2−α (ũ− u))∣∣ 2−2αdũdu =B1
∑ λ′ ∫ |(x1 − x2)(ũ, λ′)| 2 (∑ λ ∫ ∣∣Wλ′,λ(2−α (ũ− u))∣∣ 2−2αdu) dũ =B1
∑ λ′ ∫ |(x1 − x2)(ũ, λ′)| 2 (∑ λ B (1) λ′,λ ) dũ
≤B21 M
M ′ ∑ λ′ ∫ |(x1 − x2)(ũ, λ′)| 2 dũ
=B21M‖x1 − x2‖2
≤M‖x1 − x2‖2, (50) where the last inequality makes use of the fact that B1 ≤ A1 ≤ 1 under (A2) (Lemma 2.) Therefore
‖x(1)[x1]− x(1)[x2]‖2 = sup α
1
M ∑ λ ∫ ∣∣∣(x(1)[x1]− x(1)[x2]) (u, α, λ)∣∣∣2 du ≤ ‖x1 − x2‖2. (51)
This concludes the proof of (a) for the case l = 1. To prove the case for any l > 1, we first recall from (46) that
sup λ ∑ λ′ ∑ m B (l) λ′,λ,m ≤ Bl, and ∑ m Bl,m ≤ Bl M 2M ′ , where Bl,m = sup λ′ ∑ λ B (l) λ′,λ,m. (52)
Thus, for two arbitrary functions x1 and x2, we have∣∣∣(x(l)[x1]− x(l)[x2]) (u, α, λ)∣∣∣2 = ∣∣∣∣∣σ (∑
λ′
∫ R2 ∫ R x1(u+ u ′, α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ + b(λ) )
−σ (∑ λ′ ∫ R2 ∫ R x2(u+ u ′, α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ + b(λ) )∣∣∣∣∣ 2
≤ ∣∣∣∣∣∑ λ′ ∫ R2 ∫ R (x1 − x2)(u+ u′, α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ ∣∣∣∣∣ 2
= ∣∣∣∣∣∑ λ′ ∫ R2 ∫ R (x1 − x2)(u+ u′, α+ α′, λ′)2−2α ∑ m Wλ′,λ,m ( 2−αu′ ) ϕm(α ′)dα′du′ ∣∣∣∣∣ 2
= ∣∣∣∣∣∑ λ′ ∑ m ∫ R2 Gm(u+ u ′, α, λ′)2−2αWλ′,λ,m(2 −αu′)du′ ∣∣∣∣∣ 2
≤ (∑ λ′ ∑ m ∫ R2 |Gm(u+ u′, α, λ′)| 2 ∣∣Wλ′,λ,m(2−αu′)∣∣ 2−2αdu′) ·(∑ λ′ ∑ m ∫ R2 ∣∣Wλ′,λ,m(2−αu′)∣∣ 2−2αdu′)
= (∑ λ′ ∑ m ∫ R2 |Gm(ũ, α, λ′)| 2 ∣∣Wλ′,λ,m(2−α(ũ− u))∣∣ 2−2αdũ)(∑ λ′ ∑ m B (l) λ′,λ,m )
≤Bl ∑ λ′ ∑ m ∫ R2 |Gm(ũ, α, λ′)| 2 ∣∣Wλ′,λ,m(2−α(ũ− u))∣∣ 2−2αdũ, (53) where
Gm(u, α, λ ′) := ∫ R (x1 − x2)(u, α+ α′, λ′)ϕm(α′)dα′. (54)
We claim (to be proved later in Lemma 3) that
M ′‖Gm‖2 = sup α ∑ λ′ ∫ R2 |Gm(u, α, λ′)| 2 du ≤ 2M ′‖x1 − x2‖2, ∀m. (55)
Thus, for any α,∑ λ ∫ R2 ∣∣∣(x(l)[x1]− x(l)[x2]) (u, α, λ)∣∣∣2 du ≤ ∑ λ ∫ R2 Bl ∑ λ′ ∑ m ∫ R2 |Gm(ũ, α, λ′)|
2 ∣∣Wλ′,λ,m(2−α(ũ− u))∣∣ 2−2αdũdu =Bl
∑ λ′ ∑ m ∫ R2 |Gm(ũ, α, λ′)| 2 (∑ λ ∫ R2 ∣∣Wλ′,λ,m(2−α(ũ− u))∣∣ 2−2αdu) dũ =Bl
∑ λ′ ∑ m ∫ R2 |Gm(ũ, α, λ′)| 2 (∑ λ B (l) λ′,λ,m ) dũ
≤Bl ∑ λ′ ∑ m ∫ R2 |Gm(ũ, α, λ′)| 2 Bl,mdũ
=Bl ∑ m (∑ λ′ ∫ R2 |Gm(ũ, α, λ′)| 2 dũ ) Bl,m
≤Bl · 2M ′‖x1 − x2‖2 ∑ m Bl,m
≤B2l · 2M ′‖x1 − x2‖2 M
2M ′ ≤M‖x1 − x2‖2. (56)
Therefore
‖x(l)[x1]− x(l)[x2]‖2 = sup α
1
M ∑ λ ∫ R2 ∣∣∣(x(l)[x1]− x(l)[x2]) (u, α, λ)∣∣∣2 du ≤ ‖x1 − x2‖2. (57)
To prove (b), we use the method of induction. When l = 0, x(0)0 (u, λ) = 0 by definition. When l = 1, x (1) 0 (u, α, λ) = σ(b (1)(λ)). Suppose x(l−1)0 (u, α, λ) = x (l−1) 0 (λ) for some l > 1, then
x (l) 0 (u, α, λ) = σ (∑ λ′ ∫ R2 ∫ R x (l−1) 0 (u+ u ′, α+ α′, λ′)W (l) λ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ + b(l)(λ) )
= σ (∑ λ′ x (l−1) 0 (λ ′) ∫ R2 ∫ R W (l) λ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ + b(l)(λ) )
= σ (∑ λ′ x (l−1) 0 (λ ′) ∫ R2 ∫ R W (l) λ′,λ (u ′, α′) dα′du′ + b(l)(λ) ) = x
(l) 0 (λ). (58)
Part (c) is an easy corollary of part (a). More specifically, for any l > 1,
‖x(l)c ‖ = ‖x(l) − x (l) 0 ‖ = ‖x(l)[x(l−1)]− x (l) 0 [x (l−1) 0 ]‖ ≤ ‖x(l−1) − x (l−1) 0 ‖ = ‖x(l−1)c ‖. (59)
Lemma 3. Suppose ϕ ∈ L2(R) with supp(ϕm) ⊂ [−1, 1] and ‖ϕ‖L2 = 1, and x is a function of three variables
x : R2 × R× [M ]→ R (60) (u, α, λ) 7→ x(u, α, λ) (61)
with ‖x‖2 := supα 1M ∑ λ ∫ R2 |x(u, α, λ)| 2du. Define G(u, α, λ) as
G(u, α, λ) := ∫ R x(u, α+ α′, λ)ϕ(α′)dα. (62)
Then we have
M‖G‖2 = sup α ∑ λ ∫ R2 |G(u, α, λ)|2 du ≤ 2M‖x‖2. (63)
Proof of Lemma 3. Notice that, for any α, we have∑ λ ∫ R2 |G(u, α, λ)|2 du = ∑ λ ∫ R2 ∣∣∣∣∫ 1 −1 x(u, α+ α′, λ)ϕ(α′)dα′ ∣∣∣∣2 du ≤ ∑ λ ∫ R2 (∫ 1 −1 |x(u, α+ α′, λ)|2 dα′ ) ‖ϕ‖2L2du
= ∫ 1 −1 (∑ λ ∫ R2 |x(u, α+ α′, λ)|2 du ) dα′
≤ ∫ 1 −1 M‖x‖2dα′ = 2M‖x‖2. (64)
Thus
sup α ∑ λ ∫ R2 |G(u, α, λ)|2 du ≤ 2M‖x‖2. (65)
A.4 PROOF OF THEOREM 3
To prove Theorem 3, we need the following two Propositions. Proposition 2. In an ScDCFNet satisfying (A1) and (A3), we have 1. For any l ≥ 1,∥∥∥x(l)[Dτx(l−1)]−Dτx(l)[x(l−1)]∥∥∥ ≤ 4(Bl + Cl)|∇τ |∞‖x(l−1)c ‖. (66) 2. For any l ≥ 1, we have
‖Tβ,vx(l)‖ = 2β‖x(l)‖, (67) and ∥∥∥x(l)[Tβ,v ◦Dτx(l−1)]− Tβ,vDτx(l)[x(l−1)]∥∥∥ ≤ 2β+2(Bl + Cl)|∇τ |∞‖x(l−1)c ‖, (68) where the first Tβ,v in (68) is replaced by Dβ,v when l = 1.
3. If (A2) also holds true, then∥∥∥x(l)[Dβ,v ◦Dτx(0)]− Tβ,vDτx(l)[x(0)]∥∥∥ ≤ 2β+3l|∇τ |∞‖x(0)‖, ∀l ≥ 1. (69) Proposition 3. In an ScDCFNet satisfying (A1) and (A3), we have, for any l ≥ 1,∥∥∥Tβ,vDτx(l) − Tβ,vx(l)∥∥∥ ≤ 2β+1|τ |∞Dl‖x(l−1)c ‖ ≤ 2β+1|τ |∞Dl‖x(0)‖. (70) If (A2) also holds true, then∥∥∥Tβ,vDτx(l) − Tβ,vx(l)∥∥∥ ≤ 2β+1−jl |τ |∞‖x(0)‖. (71) Proof of Theorem 3. Putting together (69) and (71), we have∥∥∥x(L)[Dβ,v ◦Dτx(0)]− Tβ,vx(L)[x(0)]∥∥∥
≤ ∥∥∥x(L)[Dβ,v ◦Dτx(0)]− Tβ,vDτx(L)[x(0)]∥∥∥+ ∥∥∥Tβ,vDτx(L)[x(0)]− Tβ,vx(L)[x(0)]∥∥∥ ≤2β+3L|∇τ |∞‖x(0)‖+ 2β+1−jL |τ |∞‖x(0)‖ =2β+1 ( 4L|∇τ |∞ + 2−jL |τ |∞ ) ‖x(0)‖ (72)
This concludes the proof of Theorem 3.
Finally, we need to prove Proposition 2 and Proposition 3, where the following lemma from Qiu et al. (2018) is useful. Lemma 4 (Lemma A.1 of Qiu et al. (2018)). Suppose that |∇τ |∞ < 1/5, ρ(u) = u− τ(u), then at every point u ∈ R2,
||Jρ| − 1| ≤ |∇τ |∞(2 + |∇τ |∞), (73) where Jρ is the Jacobian of ρ, and |Jρ| is the Jacobian determinant. As a result,
||Jρ| − 1| , ∣∣|Jρ−1| − 1∣∣ ≤ 4|∇τ |∞, (74)
and, |Jρ| , ∣∣Jρ−1∣∣ ≤ 2. (75)
Proof of Proposition 2. Just like Proposition 1(a), the proof of Proposition 2(a) for the case l = 1 is similar to Lemma 3.2 of Qiu et al. (2018) after the change of variable (47). We thus focus only on the proof for the case l > 1. To simplify the notation, we denote x(l)[x(l−1)] as y[x], and replace x (l−1) c , W (l), b(l), Ml−1, and Ml, respectively, by xc, W , b, M ′, and M . By the definition of the deformation Dτ (14), we have
Dτy[x](u, α, λ) = σ (∑ λ′ ∫ R2 ∫ R x(ρ(u) + u′, α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ + b(λ) ) ,
(76)
y[Dτx](u, α, λ) = σ (∑ λ′ ∫ R2 ∫ R x(ρ(u+ u′), α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ + b(λ) ) .
(77)
Thus
|(Dτy[x]− y[Dτx])(u, α, λ)|2
= ∣∣∣∣∣σ (∑
λ′
∫ R2 ∫ R x(ρ(u) + u′, α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ + b(λ) )
− σ (∑ λ′ ∫ R2 ∫ R x(ρ(u+ u′), α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ + b(λ) )∣∣∣∣∣ 2
≤ ∣∣∣∣∣∑ λ′ ∫ R2 ∫ R (x(ρ(u) + u′, α+ α′, λ′)− x(ρ(u) + u′, α+ α′, λ′))Wλ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ ∣∣∣∣∣ 2
= ∣∣∣∣∣∑ λ′ ∫ R2 ∫ R (xc(ρ(u) + u ′, α+ α′, λ′)− xc(ρ(u) + u′, α+ α′, λ′))Wλ′,λ ( 2−αu′, α′ ) 2−2αdα′du′ ∣∣∣∣∣ 2
= ∣∣∣∣∣∑ λ′ ∑ m ∫ R2 ∫ R (xc(ρ(u) + u ′, α+ α′, λ′)− xc(ρ(u) + u′, α+ α′, λ′)) ·
Wλ′,λ,m ( 2−αu′ ) ϕm (α ′) 2−2αdα′du′ ∣∣2 , (78)
where the second equality results from the fact that x(u, α, λ)− xc(u, α, λ) = x0(λ) depends only on λ (Proposition 1(b).) Just like the proof of Proposition 1(a), we take the integral of α′ first, and define
Gm(u, α, λ ′) := ∫ R xc(u, α+ α ′, λ′)ϕm(α ′)dα′. (79)
Thus
|(Dτy[x]− y[Dτx])(u, α, λ)|2
≤ ∣∣∣∣∣∑ λ′ ∑ m ∫ R2 (Gm(ρ(u) + u ′, α, λ′)−Gm(ρ(u+ u′), α, λ′))Wλ′,λ,m ( 2−αu′ ) 2−2αdu′ ∣∣∣∣∣ 2
= ∣∣∣∣∣∑ λ′ ∑ m ∫ R2 Gm(v, α, λ ′)Wλ′,λ,m ( 2−α(v − ρ(u)) ) 2−2αdv
− ∑ λ′ ∑ m ∫ R2 Gm(v, α, λ ′)Wλ′,λ,m ( 2−α(ρ−1(v)− u) ) 2−2α|Jρ−1(v)|dv ∣∣∣∣∣ 2
= |E1(u, α, λ) + E2(u, α, λ)|2 , (80) where
E1(u, α, λ) = ∑ λ′ ∑ m ∫ R2 [ Wλ′,λ,m ( 2−α(v − ρ(u)) ) −Wλ′,λ,m ( 2−α(ρ−1(v)− u) )] · (81)
2−2αGm(v, α, λ ′)dv, (82) E2(u, α, λ) = ∑ λ′ ∑ m ∫ R2 Wλ′,λ,m ( 2−α(ρ−1(v)− u) ) [ 1− ∣∣Jρ−1(v)∣∣] · (83) 2−2αGm(v, α, λ ′)dv. (84)
Therefore
M ‖Dτy[x]− y[Dτx]‖2 = sup α ∑ λ ∫ R2 |(Dτy[x]− y[Dτx])(u, α, λ)|2 du
≤ sup α ∑ λ ∫ R2 |E1(u, α, λ) + E2(u, α, λ)|2 du
=M‖E1 + E2‖2 (85) Hence
‖Dτy[x]− y[Dτx]‖ ≤ ‖E1 + E2‖. (86) We thus seek to estimate ‖E1‖ and ‖E2‖ individually. To bound ‖E2‖, we let
k (2) λ′,λ,m(v, u, α) := Wλ′,λ,m
( 2−α(ρ−1(v)− u) ) [ 1− |Jρ−1(v)| ] 2−2α. (87)
Then
E2(u, α, λ) = ∑ λ′ ∑ m ∫ R2 Gm(v, α, λ ′)k (2) λ′,λ,m(v, u, α)dv, (88)
and, for any given v and α∫ R2 ∣∣∣k(2)λ′,λ,m(v, u, α)∣∣∣ du =∫ R2 ∣∣Wλ′,λ,m (2−α(ρ−1(v)− u))∣∣ ∣∣1− |Jρ−1(v)|∣∣ 2−2αdu = ∣∣1− |Jρ−1(v)|∣∣ ∫
R2 |Wλ′,λ,m(ũ)| dũ
≤4|∇τ |∞B(l)λ′,λ,m, (89)
where the last inequality comes from (74). Moreover, for any given u and α,∫ R2 ∣∣∣k(2)λ′,λ,m(v, u, α)∣∣∣ dv =∫ R2 ∣∣Wλ′,λ,m (2−α(ρ−1(v)− u))∣∣ ∣∣1− |Jρ−1(v)|∣∣ 2−2αdv =
∫ R2 ∣∣Wλ′,λ,m(ṽ − 2−αu)∣∣ · ||Jρ(2αṽ)| − 1| dṽ ≤4|∇τ |∞B(l)λ′,λ,m, (90)
where the last inequality is again because of (74). Thus, for any given α,∑ λ ∫ R2 |E2(u, α, λ)|2 du = ∑ λ ∫ R2 ∣∣∣∣∣∑ λ′ ∑ m ∫ R2 Gm(v, α, λ ′)k (2) λ′,λ,m(v, u, α)dv ∣∣∣∣∣ 2 du
≤ ∑ λ ∫ R2 (∑ λ′ ∑ m ∫ R2 |Gm(v, α, λ)|2 ∣∣∣k(2)λ′,λ,m(v, u, α)∣∣∣ dv ) ·(∑
λ′ ∑ m ∫ R2 ∣∣∣k(2)λ′,λ,m(v, u, α)∣∣∣ dv ) du
≤ ∑ λ ∫ R2 (∑ λ′ ∑ m ∫ R2 |Gm(v, α, λ)|2 ∣∣∣k(2)λ′,λ,m(v, u, α)∣∣∣ dv )(∑ λ′ ∑ m 4|∇τ |∞B(l)λ′,λ,m ) du
≤4|∇τ |∞Bl ∑ m ∑ λ′ ∫ R2 |Gm(v, α, λ)|2 (∑ λ ∫ R2 ∣∣∣k(2)λ′,λ,m(v, u, α)∣∣∣ du ) dv
≤4|∇τ |∞Bl ∑ m ∑ λ′ ∫ R2 |Gm(v, α, λ)|2 (∑ λ 4|∇τ |∞B(l)λ′,λ,m ) dv
≤16|∇τ |2∞Bl ∑ m (∑ λ′ ∫ R2 |Gm(v, α, λ)|2 dv ) Bl,m
≤16|∇τ |2∞Bl ∑ m M ′‖Gm‖2Bl,m (91)
Since ‖Gm‖2 ≤ 2‖xc‖2 (by Lemma 3), and ∑ mBl,m ≤
M 2M ′Bl by definition (46), we thus have∑
λ
∫ R2 |E2(u, α, λ)|2 du ≤ 16|∇τ |2∞Bl M 2M ′ Bl · 2M ′‖xc‖2 = M(4|∇τ |∞Bl‖xc‖)2, ∀α. (92)
Taking supα on both sides gives us
‖E2‖ ≤ 4|∇τ |∞Bl‖xc‖. (93)
Similarly, to bound ‖E1‖, we introduce
k (1) λ′,λ,m(v, u, α) := [ Wλ′,λ,m ( 2−α(v − ρ(u)) ) −Wλ′,λ,m ( 2−α(ρ−1(v)− u) )] 2−2α. (94)
Then
E1(u, α, λ) = ∑ λ′ ∑ m ∫ R2 Gm(v, α, λ ′)k (1) λ′,λ,m(v, u, α)dv, (95)
and, for any given v and α, we have∫ R2 ∣∣∣k(1)λ′,λ,m(v, u, α)∣∣∣ du, ∫ R2 ∣∣∣k(1)λ′,λ,m(v, u, α)∣∣∣ dv ≤ 4|∇τ |∞C(l)λ′,λ,m. (96) The proof of (96) is exactly the same as that of Lemma 3.2 in Qiu et al. (2018) after a change of variable, and we thus omit the detail. Similar to the procedure we take to bound ‖E2‖, (96) leads to
‖E1‖ ≤ 4|∇τ |∞Cl‖xc‖. (97) Putting together (86), (93), and (97), we thus have
‖Dτy[x]− y[Dτx]‖ ≤ ‖E1 + E2‖ ≤ ‖E1‖+ ‖E2‖ ≤ 4(Bl + Cl)|∇τ |∞‖xc‖. (98) This concludes the proof of (a).
To prove (b), given any β ∈ R, and v ∈ R2, we have
‖Tβ,vx(l)‖2 = sup α
1
Ml ∑ λ ∫ R2 ∣∣∣Tβ,vx(l)(u, α, λ)∣∣∣2 du = sup
α
1
Ml ∑ λ ∫ R2 ∣∣∣x(l)(2−β(u− v), α− β, λ)∣∣∣2 du = sup
α
1
Ml ∑ λ ∫ R2 ∣∣∣x(l)(ũ, α− β, λ)∣∣∣2 22βdũ = 22β‖x(l)‖2 (99)
Thus (67) holds true. As for (68), we have∥∥∥x(l)[Tβ,v ◦Dτx(l−1)]− Tβ,vDτx(l)[x(l−1)]∥∥∥ = ∥∥∥Tβ,vx(l)[Dτx(l−1)]− Tβ,vDτx(l)[x(l−1)]∥∥∥
=2β ∥∥∥x(l)[Dτx(l−1)]−Dτx(l)[x(l−1)]∥∥∥
≤2β+2(Bl + Cl)|∇τ |∞‖x(l−1)c ‖, (100)
where the first equality holds valid because of Theorem 1, and the second equality comes from (67).
To prove (c), for any 0 ≤ j ≤ l, define yj as
yj = x (l) ◦ x(l−1) ◦ · · · ◦ Tβ,v ◦Dτx(j) ◦ · · · ◦ x(0). (101)
We thus have∥∥∥x(l)[Dβ,v ◦Dτx(0)]− Tβ,vDτx(l)[x(0)]∥∥∥ = ‖yl − y0‖ ≤ l∑ j=1 ‖yj − yj−1‖
= l∑ j=1 ∥∥∥x(l) ◦ · · · ◦ Tβ,v ◦Dτx(j) ◦ · · · ◦ x(0) − x(l) ◦ · · · ◦ x(j) ◦ Tβ,v ◦Dτx(j−1) ◦ · · · ◦ x(0)∥∥∥ ≤
l∑ j=1 ∥∥∥Tβ,v ◦Dτx(j)[x(j−1)]− x(j)[Tβ,v ◦Dτx(j−1)]∥∥∥ ≤
l∑ j=1 2β+2(Bj + Cj)|∇τ |∞‖x(j−1)c ‖
≤ l∑
k=1
2β+2 · 2|∇τ |∞‖x(0)‖ = 2β+3l|∇τ |∞‖x(0)‖, (102)
where the second inequality is because of Proposition 1(a), the third inequality is due to (68), and the last inequality holds true because Bl, Cl ≤ Al ≤ 1 under (A2) (Lemma 2.) This concludes the proof of Proposition 2.
Proof of Proposition 3. The second inequality in (70) is due to Proposition 1(c). Because of (67), the first inequality in (70) is equivalent to∥∥∥Dτx(l) − x(l)∥∥∥ ≤ 2|τ |∞Dl‖x(l−1)c ‖ (103) Just like Proposition 2(a), the proof of (103) for the case l = 1 is similar to Proposition 3.4 of Qiu et al. (2018) after the change of variable (47). A similar strategy as that of Proposition 2(a) can be used to extend the proof to the case l > 1. More specifically, denote x(l−1), x(l−1)c ,W (l), b(l), respectively, as x, xc,W, and b to simplify the notation. We have∣∣∣(Dτx(l)[x]− x(l)[x]) (u, α, λ)∣∣∣2
= ∣∣∣∣∣σ (∑
λ′
∫ R2 ∫ R x (ρ(u) + u′, α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdu′dα′ + b(λ) )
− σ (∑ λ′ ∫ R2 ∫ R x (u+ u′, α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdu′dα′ + b(λ) )∣∣∣∣∣ ≤
∣∣∣∣∣∑ λ′ ∫ R2 ∫ R x (ρ(u) + u′, α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdu′dα′
− ∑ λ′ ∫ R2 ∫ R x (u+ u′, α+ α′, λ′)Wλ′,λ ( 2−αu′, α′ ) 2−2αdu′dα′ ∣∣∣∣∣ =
∣∣∣∣∣∑ λ′ ∫ R2 ∫ R xc (ρ(u) + u ′, α+ α′, λ′) ∑ m Wλ′,λ,m ( 2−αu′ ) ϕm (α ′) 2−2αdu′dα′
− ∑ λ′ ∫ R2 ∫ R xc (u+ u ′, α+ α′, λ′) ∑ m Wλ′,λ,m ( 2−αu′ ) ϕm (α ′) 2−2αdu′dα′ ∣∣∣∣∣ =
∣∣∣∣∣∑ λ′ ∑ m ∫ R2 Gm(v, α, λ ′)kλ′,λ,m(v, u, α)du ′ ∣∣∣∣∣ , (104)
where
Gm(u, α, λ ′) := ∫ R xc(u, α+ α ′, λ′)ϕm(α ′)dα′, (105)
kλ′,λ,m(v, u, α) := 2 −2α [Wλ′,λ,m (2−α(v − ρ(u)))−Wλ′,λ,m (2−α(v − u))] . (106)
Similar to (96), we have the following bound∫ R2 |kλ′,λ,m(v, u, α)| du, ∫ R2 |kλ′,λ,m(v, u, α)| dv ≤ 2|∇τ |∞D(l)λ′,λ,m. (107)
Again, the proof of (107) is the same as that of Proposition 3.4 in Qiu et al. (2018) after a change of variable. The rest of the proof follows from a similar argument as in (92) and (93).

heading: B EXPERIMENTAL DETAILS IN SECTION 5
text: B.1 IMPLEMENTATION DETAILS
Discretization and scale channel truncation. In practice, the scale channel S ∼= R needs to be truncated to a finite interval I ⊂ S such that the layerwise feature map x(l)(u, α, λ) is only computed for α ∈ I . This scale channel truncation unavoidably destroys the global scaling symmetry (similar to the fact that truncating an image to a finite spatial support destroys translation symmetry), which leads to the boundary “leakage” effect after a (joint) convolution in scale. This boundary “leakage” effect can be alleviated through the following two ways: (1) we can choose a joint convolutional filter that is compactly supported on a much smaller scale interval Iα 3 0 compared to the truncated scale interval I , i.e., |Iα| |I| (a similar idea has been explored in (Worrall & Welling, 2019)); (2) instead of using a zero-padding (which is typically chosen by default as a spatial padding method before spatial convolution), a “replicate”-padding in the scale channel (i.e., extending the truncated scale channel according to Neumann boundary condition) should be implemented before the convolution in scale. After a scale channel truncation, the layerwise feature maps x(l)(u, α, λ) and the separable basis functions {ψk(u)}k, {ϕm(α)}m are discretized both in space and scale on a uniform grid. To avoid the spatial aliasing effect, the number K of the spatial basis functions {ψk(u)}k∈[K] is not allowed to be too large (typically we set K ≤ 10.)
ST -equivariant average pooling. Given a feature x(l)(u, α, λ), the traditional average-pooling in u with the same spatial kernel size across α destroys ST -equivariance (4). To remedy this issue, similar to (Zhang, 2019), we first convolve x(l) with a scale-specific low-pass filter before downsampling the convolved signal on a coarser spatial grid. Specifically, we have x̃(l)(u, α, λ) =∫ R2 x
(l)(Nu + u′, α, λ)η(2−αu′)2−2αdu′, where x̃(l) is the feature after pooling, η is a low-pass filter, e.g., a Gaussian kernel, and N ∈ Z+ is the pooling factor. We refer to this as ST -equivariant average-pooling.
ST -equivariant batch-normalization. Batch-normalization Ioffe & Szegedy (2015) accelerates network training by reducing layerwise covariate shift, and it has become an integral part in various CNN architectures. With an extra scale index α in the feature map x(l)(u, α, λ) of an SDCFNet, we need to include α in the normalization in order not to destroy ST -equivariance, i.e., a batch of features {x(l)n (u, α, λ)}Nn=1 should be normalized as if it were a collection of 3D data (two dimensions for u, and one dimension for α.)
B.2 VERIFICATION OF ST -EQUIVARIANCE
The ScDCFNet used in this experiment has two convolutional layers, each of which is composed of a ST -equivariant convolution (5) or (6), a batch-normalization, and a 2 × 2 ST -equivariant average-pooling. The expansion coefficients a(1)λ′,λ(k) and a (2) λ′,λ(k,m) are sampled from a Gaussian distribution and truncated to K = 8 and Kα = 3 leading coefficients for u and α respectively. Similarly, a regular CNN with two convolutional layers and randomly generated 5× 5 convolutional kernels is used as a baseline for comparison.
B.3 MULTISCALE IMAGE CLASSIFICATION
In the experiments on multiscale image classification on the SMNIST and SFashion dataaset, the network architectures for the ScDCFNet and the regular CNN are shown in Table 2. Stochastic
gradient descent (SGD) with decreasing learning rate from 10−2(10−1) to 10−4(10−3) is used to train all networks without (with) batch-normalization for 160 epochs.
B.4 IMAGE RECONSTRUCTION
The network architectures for the SDCFNet and regular CNN auto-encoders are shown in Table 3. The filter expansion in the SDCFNet auto-encoder is truncated to K = 8 and Kα = 3. SGD with decreasing learning rate from 10−2 to 10−4 is used to train both networks for 20 epochs.

Review summaries:
The authors propose a CNN architecture, that is theoretically equivariant to isotropic scalings and translations. For this they add an extra scale-dimension to activation tensors, along with the existing two spatial dimensions. In practice they implement this with scale-steerable filters, which are discretised and truncated in both the spatial and scale-dimensions. They also provide a deformation robustness analysis.