SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Md Ashiqur Rahman Raymond A. Yeh  
Department of Computer Science, Purdue University  
{rahman79, rayyeh}@purdue.edu
In computer vision, models must be able to adapt to changes in image resolution to effectively carry out tasks such as image segmentation; This is known as scale-equivariance. Recent works have made progress in developing scale-equivariant convolutional neural networks, e.g., through weight-sharing and kernel resizing. However, these networks are not truly scale-equivariant in practice. Specifically, they do not consider anti-aliasing as they formulate the down-scaling operation in the continuous domain. To address this shortcoming, we directly formulate down-scaling in the discrete domain with consideration of anti-aliasing. We then propose a novel architecture based on Fourier layers to achieve truly scale-equivariant deep nets, i.e., absolute zero equivariance-error. Following prior works, we test this model on MNIST-scale and STL-10 datasets. Our proposed model achieves competitive classification performance while maintaining zero equivariance-error.
Consider the task of image classification; if an object in the image is scaled (resized), then its corresponding object label should remain the same, i.e., scale-invariant. Similarly, for semantic segmentation, if an object is scaled, then its corresponding mask should also be scaled accordingly, i.e., scale-equivariant. Similarly, one would expect the features extracted to be scale-equivariant; see Fig. 1 for illustration. These invariant and equivariant properties are important to many computer vision tasks due to the nature of images. A photo of the same scenery can be taken from different distances, and objects in the scenes may come in different sizes. Developing representations that effectively capture this multi-resolution aspect of images has been a long-standing quest [1, 9, 11, 17, 53].
Recently, there has been a line of work on developing scale-equivariant convolutions networks [8, 13, 41, 42, 46] to more effectively learn multi-resolution features. At a high level, these works achieve scale-equivariant convolution layers through weight-sharing and kernel resizing, i.e., use the "same" but resized kernel across all scales [5]. The innovation of these works is how to properly resize the kernel. For example, Bekkers [2] and Sosnovik et al. [41] formulate kernel resizing as a continuous operation and then discretize the kernel when implemented in practice. However, this discretization leads to non-negligible equivariance error. On the other hand, Worrall and Welling [46] and Sosnovik et al. [42] directly formulate kernel resizing in the discrete domain, e.g., using dilation or solving for the best kernel given a fixed scale set, and achieve low equivariance-error.
Despite these successes, we point out that the aforementioned works are not truly scale-equivariant in practice. Specifically, these works are derived using a continuous domain down-scaling operation, i.e., there is no need to consider anti-aliasing. However, when performing a down-scaling on discrete space, the Nyquist theorem [23, 30] tells us that an anti-alias filter is necessary to avoid high-frequency content to alias into lower frequencies. The canonical example of aliasing is the "wagon-wheel effect", where a wheel in a video appears to be rotating slower or even in reverse from its true rotation. To
Figure 1. Comparison of scale-equivariance on CNN vs. our model. For a regular CNN, the features extracted from the corresponding high/low image resolution look very different. On the other hand, downsampling the high-res feature is guaranteed to achieve the same feature obtained from the low-resolution image.
address this gap from prior work, we consider the down-scaling operation directly in the discrete domain, taking he anti-aliasing into account.
In this work, we formulate down-scaling as the ideal downsampling from signal processing [23]. We then propose a family of deep nets that are truly scale-equivariant based on this ideal downsampling. This involves rethinking all the components in the deep net, including convolution layers, nonlinearities, and pooling layers.
With the developed deep net, we focus on the task of image classification. We further point out that truly scale-invariant classifiers are not desirable. A truly scale-invariant model's performance is limited by the lowest-resolution image. Instead, the more desirable property is that a high-resolution image should achieve a better performance than its corresponding low-resolution image. This motivated us to design a classifier architecture suitable for this property.
Following prior works, we conduct our experiments on the MNIST-scale [40] and STL [4] dataset. By design, our method achieves zeros scale equivariance-error both in theory and in practice. In terms of accuracy, we compare to recent scale-equivariant CNNs. We found our approach to be competitive in classification accuracy and exhibit better data efficiency in low-resource settings.
- We formulate down-scaling in the discrete domain with considerations of anti-aliasing.
- We propose a family of deep nets that is truly scale-equivariant by designing novel scale-equivariant modules based on Fourier layers.
- We conduct extensive experiments validating the proposed approach. On MNIST and STL datasets, the proposed model achieves an absolute zero end-to-end scale-equivariance error while maintaining competitive classification accuracy.
Scale-equivariance and invariance. The notation of scale-equivariance is deeply rooted in image processing and computer vision. For example, classic hand-designed scale-invariant features such as SIFT [21, 22] have made tremendous contributions to the field of computer vision. Earlier works propose to use an image or spatial pyramid to capture the multi-resolution aspect of an image [1, 9, 17] by extracting features at several scales in an efficient manner.
More recently, there have been interests in developing scale-equivariant CNN [2, 8, 13, 41, 42, 46, 54]. Based on Group-Conv [5], these works achieve scale-equivariance convolution layers through weight-sharing and kernel resizing. Different from these works, we consider the down-scaling in the discrete domain formulated as ideal downsampling from signal processing. We then develop modules that are truly scale-equivariant to enable a deep net that achieves zero equivariance-error measured from end to end. Finally, we note that there is a rich literature of equivariant deep nets [3, 5, 33, 36–38, 44, 45, 50]
with numerous applications applied to various domains, e.g., sets [10, 26, 32, 34, 48, 51], graphs [6, 7, 15, 19, 20, 25, 28, 39, 49], etc. Moreover, several recent studies have also identified and tackled the issue of aliasing generated from the pooling layer to attain finer translation equivariance [35, 47, 52] and image generation [14].
Fourier transform in neural networks. Fourier transforms have been previously used in deep learning. For example, Mathieu et al. [27] proposes to use Fast Fourier Transform (FFT) to speed up CNN training. Fourier transform has also been used to develop network architectures, including various convolutional neural networks operating in the Fourier space [16, 31]. Recently, Fourier layers are capable of handling inputs of varying resolution, which have been employed in neural operators, facilitating applications in partial differential equations and in state space models [18, 29]. Fourier convolutions have also found success in low-level image processing tasks, e.g., inpainting [43], deblurring [24]. Different from these works, we focus on developing truly scale-equivariant deep nets and leverage Fourier layers to achieve this goal.
We briefly introduce and review the definition of Fourier transform, ideal downsampling, and scale-equivalence. For readability, we use 1D data to define these concepts. These ideas are extended to 2D data with multiple channels when implemented in practice.
Discrete Fourier Transform (DFT). Given an input vector  $\mathbf{x} \in \mathbb{R}^N$ , we consider  $\mathcal{F}: \mathbb{R}^N \to \mathbb{C}^N$  be the discrete Fourier Transform (DFT) which has the form
$$
\mathbf {X} = \mathcal {F} (\mathbf {x}) \text {s u c h t h a t} \mathbf {X} [ k ] \triangleq \frac {1}{N} \sum_ {n = 0} ^ {N - 1} \mathbf {x} [ n ] e ^ {- j \frac {2 \pi}{N} k n}, \tag {1}
$$
where  $j$  denotes the unit imaginary number, i.e.,  $j^2 = -1$ . The index  $k$  in Eq. (1) is commonly within the domain of  $[0,N]$ . Note that as Eq. (1) is  $N$ -periodic, for readability, we will use  $k$  from  $\left[-\frac{N-1}{2}, \frac{N-1}{2}\right]$  where  $k = 0$  corresponds to the lowest frequency.
The corresponding inverse DFT (IDFT)  $\mathcal{F}^{-1}:\mathbb{C}^N\to \mathbb{R}^N$  is defined as
$$
\mathbf {x} = \mathcal {F} ^ {- 1} (\mathbf {X}) \text {s u c h t h a t} \mathbf {x} [ n ] = \sum_ {k = 0} ^ {N - 1} \mathbf {X} [ k ] e ^ {j \frac {2 \pi}{N} k n}. \tag {2}
$$
By the convolution property of DFT, the circular convolution between  $\mathbf{x}$  and a kernel  $\mathbf{k} \in \mathbb{R}^N$  can be represented as the element-wise multiplication in the Fourier domain, i.e.,
$$
\mathcal {F} (\mathbf {x} \circledast \mathbf {k}) = \mathcal {F} (\mathbf {x}) \odot \mathcal {F} (\mathbf {k}) = \mathbf {X} \odot \mathbf {K}, \tag {3}
$$
where  $\circledast$  denotes the circular convolution and  $\odot$  denotes element-wise multiplication. Unless explicitly mentioned, we will represent the input vector with lowercase letters (e.g.,  $\mathbf{x}$ ) and its corresponding DFT with uppercase letters (e.g.,  $\mathbf{X}$ ).
Down-scaling operation. To reduce the scale (or resolution) of a signal  $\mathbf{x} \in \mathbb{R}^N$ , one could perform a subsampling  $\mathrm{Sub}_R$  by a factor of  $R$
$$
\operatorname {S u b} _ {R} (\mathbf {x}) [ n ] = \mathbf {x} [ R n ]. \tag {4}
$$
However, naively subsampling leads to aliasing. Hence, anti-aliasing is performed in a multi-rate system. In signal processing, the analysis commonly uses the ideal anti-aliasing filter  $\mathbf{h}$ , which zeros out all the high-frequency content, i.e., its DFT  $\mathbf{H} \triangleq \mathcal{F}(\mathbf{h})$  is defined as:
$$
\mathbf {H} [ k ] = 1 \text {i f} | k | \leq \frac {N}{2 R} \text {a n d} 0 \text {o t h e r w i s e .} \tag {5}
$$
See Fig. 2a for an illustration of the ideal anti-aliasing filter.
In this work, we define the overall down-scaling operation to be the ideal downsampling  $\mathcal{D}_R$  by a factor of  $R$ , which performs anti-aliasing followed by a subsampling operation:
$$
\mathcal {D} _ {R} (\mathbf {x}) \triangleq \operatorname {S u b} _ {R} (\mathbf {h} \circledast \mathbf {x}) \forall R <   N, \tag {6}
$$
(a) Illustration of ideal anti-aliasing (low-pass) filter.
(b) Illustration of Claim 1.
Figure 2. In (a), we illustrate an ideal low-pass filter showing that it zeros out the high frequencies. In (b), we illustrate the structure described in Claim 1 for a linear  $\tilde{G}$ . The gray regions correspond to the value being zero.
where their DFT are related by
$$
\mathcal {F} \left(\mathcal {D} _ {R} (\mathbf {x})\right) = \mathcal {F} (\mathbf {x}) [ - N / 2 R: N / 2 R ]. \tag {7}
$$
Scale-equivariance. With the down-scaling operation defined, a deep net  $\mathbf{g}:\{\mathbb{R}^1,\mathbb{R}^2,\dots ,\mathbb{R}^N\} \mapsto$ $\{\mathbb{R}^1,\mathbb{R}^2,\ldots ,\mathbb{R}^N\}$  is scale-equivariant if:
$$
\mathbf {g} \left(\mathcal {D} _ {R} (\mathbf {x})\right) = \mathcal {D} _ {R} (\mathbf {g} (\mathbf {x})) \quad \forall \mathbf {x} \in \left\{\mathbb {R} ^ {1}, \mathbb {R} ^ {2}, \dots , \mathbb {R} ^ {N} \right\} \text {a n d} R <   \dim (\mathbf {x}), \tag {8}
$$
where  $\{\mathbb{R}^1,\mathbb{R}^2,\dots ,\mathbb{R}^N\}$  represents the space of input/output signals at different scales. In this paper, we are interested in designing a family of deep nets that satisfies the equality in Eq. (8). Scale-invariance can be defined in a similar manner as
$$
\mathbf {g} \left(\mathcal {D} _ {R} (\mathbf {x})\right) = \mathbf {g} (\mathbf {x}) \quad \forall \mathbf {x} \in \left\{\mathbb {R} ^ {1}, \mathbb {R} ^ {2}, \dots , \mathbb {R} ^ {N} \right\} \text {a n d} R <   \dim (\mathbf {x}). \tag {9}
$$
Fourier layer. Given a multi-channel input vector,  $\mathbf{x} \in \mathbb{R}^{C_{\mathrm{in}} \times N}$  and kernel  $\mathbf{k} \in \mathbb{R}^{C_{\mathrm{out}} \times C_{\mathrm{in}} \times N}$ , where  $C_{\mathrm{in/out}}$  is the number of input/output channels, the circular convolution layer is defined as
$$
\mathcal {F} ((\mathbf {x} \circledast \mathbf {k})) [ c ^ {\prime} ] = \sum_ {c = 1} ^ {C _ {\mathrm {i n}}} \mathbf {X} [ c ] \odot \mathbf {K} [ c ^ {\prime}, c ], \tag {10}
$$
where  $\mathbf{X}$  and  $\mathbf{K}$  denotes the DFT of  $\mathbf{x}$  and  $\mathbf{k}$  applied independently for each channel.
Our goal is to design truly scale-equivariant deep nets. To accomplish this goal, we propose scale-equivariant versions of CNN modules, including, the convolution layer, non-linearities, and pooling layers. In Sec. 4.1, we detail the operation for each of the proposed modules. In Sec. 4.2, we demonstrate how to build a classifier that is suitable for image classification with scale-equivariant features. We now explain our overarching design principle for the scale-equivariant modules.
From a frequency perspective, as reviewed in Eq. (6), the ideal downsampling operation results in the loss of higher frequency terms of the signal. In other words, if a feature's frequency terms depend on any higher frequency terms of the input, then it is not scale-equivariant, as the information will be lost after downsampling. We now formally state this observation in Claim 1.
Claim 1. Let  $g$  denote a deep net such that  $\mathbf{y} = g(\mathbf{x})$ . If this deep net  $g$  can be equivalently represented as a set of functions  $\tilde{G}_k: \mathbb{C}^{2k+1} \to \mathbb{C}$  such that
$$
\mathbf {Y} [ k ] = \tilde {G} _ {k} (\mathbf {X} [ - k: k ]) \forall k \tag {11}
$$
then  $g$  is scale-equivariant as defined in Eq. (8). In other words, an output's frequency terms can only have dependencies on the terms in  $\mathbf{X}$  that are even lower in frequencies. We illustrate this structure with a linear function in Fig. 2b.
Proof. We denote the deep net's input and output as  $\mathbf{x}$  and  $\mathbf{y}$  with corresponding DFT  $\mathbf{X}$  and  $\mathbf{Y}$ . We denote the deep net's down-scaled input and output as  $\mathbf{x}' = \mathcal{D}_R(\mathbf{x})$  and  $\mathbf{y}' = g(\mathbf{x}')$  with corresponding DFT  $\mathbf{X}'$  and  $\mathbf{Y}'$ . Now assume that  $g: \mathbb{R}^n \to \mathbb{R}^n \forall n \in \{1, 2, \dots, N\}$  is a deep net that satisfies Claim 1 then
$$
\begin{array}{l} \mathbf {Y} [ k ] = \tilde {G} _ {k} (\mathbf {X} [ - k: k ]) \forall k \leq \frac {N}{R} (12) \\ = \tilde {G} _ {k} \left(\mathbf {X} ^ {\prime} [ - k: k ]\right) = \mathbf {Y} ^ {\prime} [ k ] \text {F o l l o w i n g t h e p r o p e r t y} \mathcal {D} _ {R} (13) \\ \end{array}
$$
Therefore,  $\forall k\leq \frac{N}{R}\mathbf{Y}[k] = \mathbf{Y}'[k]$ . By the definition of ideal downsampling  $\mathbf{Y}' = \mathcal{D}_R(\mathbf{Y})$  concluding that  $g(\mathcal{D}_R(\mathbf{x})) = \mathcal{D}_R(g(\mathbf{x}))$ , i.e.,  $g$  is scale-equivariant.
For ease of understanding, here we assume that the deep net's input and output are of the same size. A version with a more relaxed assumption is provided in Appendix Sec. A1.
4.1 Scale Equivariant Fourier Networks.
We now describe the proposed modules and show that they are truly scale-equivariant.
Spatially local Fourier layer. For computer vision, learning local features is crucial. The Fourier layer in Eq. (10) is global in nature. To efficiently learn local features, we propose a localized Fourier layer where we constrain the degree of freedom in the kernel  $\mathbf{K}$  such that the respective spatial kernel  $\mathbf{k}$  is spatially localized.
Let  $\mathbf{k} \in \mathbb{R}^d$  and  $\mathbf{k}^l \in \mathbb{R}^l$  to be  $d$  and  $l$  dimensional kernel such that  $\mathbf{k}[i] = \mathbf{k}^l[i]$  if  $i < l$  otherwise 0, i.e.,  $\mathbf{k}$  is spatially local to have a receptive field of size  $l$ . We denote  $\mathbf{K}$  and  $\mathbf{K}^l$  be the DFT of the kernel  $\mathbf{k}$  and  $\mathbf{k}^l$  respectively. We claim that  $\mathbf{K}$  can be written as
$$
\mathbf {K} [ p ] = \frac {1}{d} \sum_ {m = - \frac {l}{2}} ^ {\frac {l}{2}} \left(\mathbf {K} ^ {l} [ m ] \sum_ {n = 0} ^ {l - 1} e ^ {- 2 \pi j n \left(\frac {p}{d} - \frac {m}{l}\right)}\right). \tag {14}
$$
From Eq. (14), instead of modeling all the degrees of freedom in  $\mathbf{K}$ , we will directly parameterize  $\mathbf{K}^l$  to enforce the learned kernel to be localized spatially. We defer the proof to the Appendix Sec. A2.
Claim 2. The spatially local Fourier layer is scale-equivariant.
Proof. The kernel  $\mathbf{k}$  has a corresponding DFT  $\mathbf{K}$ . As reviewed, a circular convolution between  $\mathbf{k}$  and input  $\mathbf{x}$  can be expressed as
$$
\mathbf {Y} [ k ] = \mathbf {K} [ k ] \odot \mathbf {X} [ k ] \forall k. \tag {15}
$$
Observe that  $\mathbf{X}[k]$  is a subset of  $\mathbf{X}[-k:k]$ , i.e., Claim 1 is satisfied.
Scale-equivariant non-linearity  $(\sigma_{\mathrm{s}})$ . Element-wise non-linearities, e.g., ReLU, in the spatial domain are generally not scale-equivariant under the ideal downsampling operation  $\mathcal{D}_R$ . While applying element-wise non-linearity in the frequency domain is scale-equivariant, this strategy empirically leads to degraded performance on classification tasks. To address this, we propose a scale-equivariant non-linearity  $\sigma_{\mathrm{s}}$  in the spatial domain.
Given a non-linearity  $\sigma$ , e.g., ReLU, we construct a corresponding scale-equivariant version  $\sigma_{\mathbf{s}}$  that satisfies Claim 1. Let  $\mathbf{x} \in \mathbb{R}^N$  and  $\mathbf{y} \in \mathbb{R}^N$  to denote the input and output of  $\sigma_{\mathbf{s}}$ . We define scale-equivariant non-linearity  $\sigma_{\mathbf{s}}(\mathbf{x}) = \mathcal{F}^{-1}(\mathbf{Y})$  where  $\mathbf{Y}$  takes the following form:
$$
\mathbf {Y} [ k ] = \left\{ \begin{array}{l l} \mathcal {F} \left(\sigma \circ \mathcal {F} ^ {- 1} (\mathbf {X} [ 0 ])\right) [ 0 ], & k = 0 \\ \mathcal {F} \left(\sigma \circ \mathcal {F} ^ {- 1} (\mathbf {X} [ - 1: 1 ])\right) [ 1 ], & k = 1 \\ \vdots & \vdots \\ \mathcal {F} \left(\sigma \circ \mathcal {F} ^ {- 1} (\mathbf {X} [ - | k |: | k | ])\right) [ k ], & k = k \\ \vdots & \vdots \end{array} \right. \tag {16}
$$
$\forall |k|\leq \frac{N}{2}$  and  $\mathbf{X}$  denotes the DFT of the input, i.e.,  $\mathcal{F}(\mathbf{x})$  . In practice, we choose  $\sigma$  to be ReLU in our implementation.
Next, it is generally computationally expensive to achieve equivariance over all scales. In practice, we only enforce a set of scales for which we want to achieve equivariance, which can be denoted in terms of corresponding resolutions as  $\mathcal{R} = (m,\dots N)$  with  $\mathcal{R}[i] < \mathcal{R}[i + 1]$ . To achieve scale-equivariant non-linearity over the scales of  $\mathcal{R}$ ,  $\sigma_{\mathrm{s}} = \mathcal{F}^{-1}(\mathbf{Y})$  can be efficiently computed as
$$
\mathbf {Y} [ k ] = \mathcal {F} \left(\sigma \circ \mathcal {F} ^ {- 1} \left(\mathbf {X} \left[ - \frac {\mathcal {R} ^ {\prime} [ i ]}{2}: \frac {\mathcal {R} ^ {\prime} [ i ]}{2} \right]\right)\right) [ k ] \quad \text {f o r t h e} i \text {s . t .} \frac {\mathcal {R} ^ {\prime} [ i - 1 ]}{2} <   | k | \leq \frac {\mathcal {R} ^ {\prime} [ i ]}{2}. \tag {17}
$$
Here, the ordered set  $\mathcal{R}' = \mathcal{R} \cup \{0\}$ . By Eq. (17), all the Fourier coefficients  $k$  between any two consecutive resolutions in  $\mathcal{R}$ , i.e.,  $\mathcal{R}[i - 1] / 2 < |k| \leq \mathcal{R}[i] / 2$  can be computed by a single Fourier transform pair.
Scale-equivariant pooling. Pooling operation is crucial for deep nets' scalability to larger images and datasets as they make the network more memory and computationally efficient. Commonly used pooling operations are max/average pooling, which reduces the input size by the factor of its window size  $w$  and is not scale-equivariant. To address this, we propose scale-equivariant pooling  $\mathsf{Pool}1_{\mathbf{s}}^{w}$ .
Let  $\mathbf{Pool}^w$  denote a max/average pooling operation with a window size  $w$  and  $\mathbf{Pool}^w: \mathbb{R}^d \to \mathbb{R}^{\frac{d}{w}}$ . We define scale-equivariant pooling operation  $\mathbf{Pool}_{\mathbf{s}}^w: \mathbb{R}^d \to \mathbb{R}^{\frac{d}{w}}$  mapping from  $\mathbf{x}$  to  $\mathbf{y}$  where  $\mathbf{y} = \mathcal{F}^{-1}(\mathbf{Y})$  follows
$$
\mathbf {Y} [ k ] = \mathcal {F} \left(\operatorname {P o o l} ^ {\mathrm {w}} \left(\mathcal {F} ^ {- 1} \left(\mathbf {X} [ - w | k |: w | k | ]\right)\right)\right) [ k ] \quad \forall k \leq \frac {d}{2 w}. \tag {18}
$$
Observe that this pooling layer satisfies Claim 1 by construction. Similar to non-linearity, we can enforce the equivariance over the set  $\mathcal{R}$  following the same formulation in Eq. (17).
Note that as pooling reduces the size of the output by a factor  $w$ , the operation is only scale-equivariant at every  $w^{\mathrm{th}}$  resolution. When the input size is not a multiple of  $w$  there is a truncation of the input.
Time Complexity. We now provide the time complexity of our scale-equivariant Fourier layer and compare it with standard group convolutions. Let's consider a 1D signal of length  $N$  and a kernel of length  $K$ . Our proposed model involves:
- A transformation of local filter to global with time complexity  $O(KN)$
- A convolution using Fourier transform with time complexity  $O(N \log(N))$
- Our scale equivariant non-linearity depends on the size of the group. Let  $A$  be the set of group actions. The time complexity of the proposed scale-equivariant non-linearity is  $O(|A|N\log (N))$ , where  $|A|$  denotes the cardinality of the set  $A$ .
So, the time complexity for each layer becomes
$$
O (| A | N \log (N) + K N).
$$
As a comparison, the time complexity of regular group convolution is  $O(KN|A|)$  in the first layer and  $O(KN|A|^2)$  for all intermediate layers, assuming the cost of group action is a negligible constant [12].
Considering the time complexity of the intermediate layers of group convolutions, our proposed method is more efficient when
$$
| A | N \log (N) + K N <   | A | ^ {2} K N \Longrightarrow \log (N) + \frac {K}{| A |} <   | A | K.
$$
So, when  $K << |A|$  and  $\log(N) < |A|K$ , i.e., assuming the set of group actions of moderate size, then our method is faster than group convolutions.
Modern GPUs are specifically optimized for regular convolution operations that can be performed in place. In contrast, the FFT algorithm does not fully capitalize on GPUs' advantages, primarily due to unique memory access patterns and moderate arithmetic intensities. Consequently, our approach is unable to harness the full potential of GPUs. When executed on a GPU, regular group convolutions implemented as standard convolutions might exhibit comparable or even shorter running times than our approach.
4.2 Classifier for equivariant features
A truly scale-invariant, defined in Eq. (9), the model's performance is limited by the lowest resolution as the prediction needs to be the same. In the extreme, the prediction can only depend on a single mean pixel. Instead of invariance, we believe that it is more desirable to ensure that a high-resolution image achieves a better performance than its down-scaled version, i.e., the performance is scale "consistent". To achieve this property, we propose a suitable classifier architecture and training scheme.