Original: Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networks
Revised: Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networks

Original: Abstract
Revised: Abstract

Original: We propose learnable polyphase sampling (LPS), a pair of learnable down/upsampling layers that enable truly shift-invariant and equivariant convolutional networks.
Revised: We propose learnable polyphase sampling (LPS), a pair of learnable down/upsampling layers that enable truly shift-invariant and equivariant convolutional networks.

Original: LPS can be trained end-to-end from data and generalizes existing handcrafteddownsampling layers.
Revised: LPS can be trained end-to-end from data and generalizes existing handcrafted downsampling layers.

Original: It is widely applicable as it can be integrated into any convolutional network by replacing down/upsampling layers.
Revised: It is widely applicable as it can be integrated into any convolutional network by replacing down/upsampling layers.

Original: We evaluate LPS on imageclassiﬁcation and semantic segmentation.
Revised: We evaluate LPS on image classiﬁcation and semantic segmentation.

Original: Experiments show that LPS is on-par withor outperforms existing methods in both performance and shift consistency.
Revised: Experiments show that LPS is on-par with or outperforms existing methods in both performance and shift consistency.

Original: For theﬁrst time, we achieve true shift-equivariance on semantic segmentation (PASCAL VOC), i e., 100% shift consistency, outperforming baselines by an absolute 3.3%
Revised: For the ﬁrst time, we achieve true shift-equivariance on semantic segmentation (PASCAL VOC), i.e. , 100% shift consistency, outperforming baselines by an absolute 3.3%.

Revised: Our project page and code are available at https://raymondyeh07.github.io/learnable_polyphase_sampling/

Original:  Introduction
Revised:  Introduction

Original: For tasks like image classiﬁcation, shifts of an object do not change the corresponding object label,i.e ., the task is shift-invariant.
Revised: For tasks like image classiﬁcation shifts of an object do not change the corresponding object label i.e . the task is shift-invariant.

Original: This shift-invariance property has been incorporated into deep-netsyielding convolutional neural nets (CNN).
Revised: This shift-invariance property has been incorporated into deep-nets yielding convolutional neural nets (CNN).

Original: Seminal works on CNNs [15, 24] directly attribute themodel design to shift-invariance.
Revised: Seminal works on CNNs [15, 24] directly attribute the model design to shift-invariance.

Original: For example, Fukushima [15] states “the network has an abilityof position-invariant pattern recognition” and LeCun et al.
Revised: For example, Fukushima [15] states “the network has an ability of position-invariant pattern recognition” and LeCun et al.

Original: [24] motivate CNNs by stating that they“ensure some degree of shift invariance.”
Revised: [24] motivate CNNs by stating that they “ensure some degree of shift invariance

Original: CNNs have evolved since their conception.
Revised: CNNs have evolved since their conception.

Original: Modern deep-nets contain more layers, use differentnon-linearities and pooling layers.
Revised: Modern deep-nets contain more layers, use different non-linearities and pooling layers.

Original: Re-examining these modern architectures, Zhang [49] surprisinglyﬁnds that modern deep-nets are not shift-invariant.
Revised: Re-examining these modern architectures, Zhang [56] surprisingly ﬁnds that modern deep-nets are not shift-invariant.

Original: To address this, Zhang [49] and Zou et al.
Revised: To address this, Zhang [56] and Zou et al.

Original: [50]propose to perform anti-aliasing before each downsampling layer, and found it to improve thedegree of invariance.
Revised: [57] propose to perform anti-aliasing before each downsampling layer, and found it to improve the degree of invariance.

Original: More recently, Chaman and Dokmanic [5] show that deep-nets can be “trulyshift-invariant,” i.e .,
Revised: More recently, Chaman and Dokmanic [5] show that deep-nets can be “truly shift-invariant,” i.e .,

Original: a model’s output is identical for given shifted inputs.
Revised: a model’s output is identical for given shifted inputs.

Original: For this, they replace alldownsampling layers with their adaptive polyphase sampling (APS) layer.
Revised: For this, they replace all downsampling layers with their adaptive polyphase sampling (APS) layer.

Original: While APS achieves true shift-invariance by selecting the max-norm polyphase component (a handcrafted downsampling scheme), an important question arises: are there more effective downsamplingschemes that can achieve true shift-invariance?
Revised: While APS achieves true shift-invariance by selecting the max-norm polyphase component (a handcrafted downsampling scheme), an important question arises: are there more effective downsampling schemes that can achieve true shift-invariance?

Original: Consider an extreme case, a handcrafted deep-netthat always outputs zeros is truly shift-invariant, but does not accomplish any task.
Revised: Consider an extreme case, a handcrafted deep-net that always outputs zeros is truly shift-invariant, but does not accomplish any task.

Original: This motivates tostudy how truly shift-invariant downsampling schemes can be learned from data.
Revised: This motivates to study how truly shift-invariant downsampling schemes can be learned from data.

Original: For this we propose Learnable Polyphase Sampling (LPS), a pair of down/upsampling layers thatyield truly shift-invariant/equivariant deep-nets and can be trained in an end-to-end manner. 
Revised: For this we propose Learnable Polyphase Sampling (LPS), a pair of down/upsampling layers that yield truly shift-invariant/equivariant deep-nets and can be trained in an end-to-end manner. For ∗ Equal contribution.

Original: Fordownsampling, LPS can be easily integrated into existing deep-net architectures by swapping out thepooling/striding layers.
Revised: downsampling, LPS can be easily integrated into existing deep-net architectures by swapping out the pooling/striding layers.

Original: Theoretically, LPS generalizes APS to downsampling schemes that cannot berepresented by APS.
Revised: Theoretically, LPS generalizes APS to downsampling schemes that cannot be represented by APS.

Original: Hence, LPS’s ideal performance is never worse than that of APS.
Revised: Hence, LPS’s ideal performance is never worse than that of APS.

Original: For upsampling,
Revised: For upsampling,

Original: LPS guarantees architectures that are truly shift-equivariant, i.e ., the output shifts accordingly whenthe input shifts.
Revised: LPS guarantees architectures that are truly shift-equivariant, i.e ., the output shifts accordingly when the input shifts.

Original: This is desirable for tasks like semantic image segmentation.
Revised: This is desirable for tasks like semantic image segmentation.

Original: To validate the proposed LPS, we conduct extensive experiments: (a) image classiﬁcation on CIFAR10 [21] and ImageNet [12]; (b) semantic segmentation on PASCAL VOC [13].
Revised: To validate the proposed LPS, we conduct extensive experiments: (a) image classiﬁcation on CIFAR10 [21] and ImageNet [12]; (b) semantic segmentation on PASCAL VOC [13].

Original: We observe thatthe proposed approach outperforms APS and further improves anti-aliasing methods on both modelperformance and shift consistency.
Revised: We observe that the proposed approach outperforms APS and further improves anti-aliasing methods on both model performance and shift consistency.

Original: Our contributions are as follows:
Revised: Our contributions are as follows:

Original: • We propose learnable polyphase sampling (LPS), a pair of novel down/upsampling layers, andprove that they yield truly shift-invariant/-equivariant deep-nets.
Revised: • We propose learnable polyphase sampling (LPS), a pair of novel down/upsampling layers, and prove that they yield truly shift-invariant/-equivariant deep-nets.

Original: Different from prior works, oursampling scheme is trained from end-to-end and not handcrafted.• We theoretically prove that LPS (downsampling) is a generalization of APS.
Revised: Different from prior works, our sampling scheme is trained from end-to-end and not handcrafted. • We theoretically prove that LPS (downsampling) is a generalization of APS.

Original: Hence, in theory,LPS improves upon APS.
Revised: Hence, in theory, LPS improves upon APS.

Original: • We conduct extensive experiments demonstrating the effectiveness of LPS on image classiﬁcationand segmentation over three datasets comparing to APS and anti-aliasing approaches.Code is
Revised: • We conduct extensive experiments demonstrating the effectiveness of LPS on image classiﬁcation and segmentation over three datasets comparing to APS and anti-aliasing approaches.

Original: provided in the supplemental material and will be made publicly available.

Original:  Related Work
Revised:  Related Work

Original: In this section, we brieﬂy discuss related work, including shift invariant/equivariant deep-nets andpooling layers.
Revised: In this section, we brieﬂy discuss related work, including shift invariant/equivariant deep-nets and pooling layers.

Original: Additional necessary concepts are reviewed in Sec. 3.
Revised: Additional necessary concepts are reviewed in Sec. 3.

Original: Shift Invariant/equivariant convolutional networks.
Revised: Shift Invariant/equivariant convolutional networks.

Original: Modern convolutional networks use stridingor pooling to reduce the amount of memory and computation in the model [17, 22, 37, 41].
Revised: Modern convolutional networks use striding or pooling to reduce the amount of memory and computation in the model [17, 22, 40, 45].

Original: Aspointed out by Azulay and Weiss [1] and Zhang [49], these pooling/striding layers break the shiftinvariance property of deep-nets.
Revised: As pointed out by Azulay and Weiss [1] and Zhang [56], these pooling/striding layers break the shiftinvariance property of deep-nets.

Original: To address this issue, Zhang [49] proposed to perform anti-aliasing,i.e ., lowpass ﬁltering (LPF) before each downsampling, a canonical signal processing technique formulti-rate systems [43].
Revised: To address this issue, Zhang [56] proposed to perform anti-aliasing, i.e ., lowpass ﬁltering (LPF) before each downsampling, a canonical signal processing technique for multi-rate systems [47].

Original: We illustrate this approach in Fig.
Revised: We illustrate this approach in Fig.

Original: 1 (left).
Revised: 1 (left).

Original: Zou et al.
Revised: Zou et al.

Original: [50] further improvedthe LPF technique by using adaptive ﬁlters which better preserve edge information.
Revised: [57] further improved the LPF technique by using adaptive ﬁlters which better preserve edge information.

Original: While anti-aliasing ﬁlters are effective, Chaman and Dokmanic [5] show that true shift-invariance, i.e .,100% shift consistency, can be achieved without anti-aliasing.
Revised: While anti-aliasing ﬁlters are effective, Chaman and Dokmanic [5] show that true shift-invariance, i.e ., 100% shift consistency, can be achieved without anti-aliasing.

Original: Speciﬁcally, they propose Adaptive
Revised: Speciﬁcally, they propose Adaptive

Original: Polyphase Sampling (APS) which selects the downsampling indices, i.e ., polyphase components,based on the (cid:96) p -norm of the polyphase components; a handcrafted rule, as illustrated in Fig.
Revised: Polyphase Sampling (APS) which selects the downsampling indices, i.e ., polyphase components, based on the (cid:96) p -norm of the polyphase components; a handcrafted rule, as illustrated in Fig.

Original: 1 (right).
Revised: 1 (right).

Original: In a follow up technical report [4], APS is extended to upsampling using unpooling layers [2, 48],where the downsampling indices are saved to place values back to their corresponding spatial locationduring upsampling.
Revised: In a follow up technical report [4], APS is extended to upsampling using unpooling layers [2, 55], where the downsampling indices are saved to place values back to their corresponding spatial location during upsampling.

Original: Our work presents a novel pair of shift-invariant/equivariant down/upsamplinglayers which are trainable, in contrast to APS’s handcrafted selection rule.
Revised: Our work presents a novel pair of shift-invariant/equivariant down/upsampling layers which are trainable, in contrast to APS’s handcrafted selection rule.

Original: We note that generalizations of equivariance beyond shifts have also been studied [3, 7, 34, 36, 42, 44]and applied to various domains, e.g ., sets [16, 29, 32, 35, 46], graphs [10, 11, 19,  28, 40], sphericalimages [8, 9, 20], volumetric data [45], etc .
Revised: We note that generalizations of equivariance beyond shifts have also been studied [3, 7, 37, 39, 43, 46, 48, 52] and applied to various domains, e.g ., sets [16, 31, 35, 38, 50, 53], graphs [10, 11, 19, 27, 28, 30, 32, 44, 51], spherical images [8, 9, 20], volumetric data [49], etc .

Original: In this work, we focus solely on shift-equivariance forimages with CNNs.
Revised: In this work, we focus solely on shift-equivariance for images with CNNs.

Original: Pooling layers.
Revised: Pooling layers.

Original: Many designs for better downsampling or pooling layers have been proposed.
Revised: Many designs for better downsampling or pooling layers have been proposed.

Original: Popularchoices are Average-Pooling [23] and Max-Pooling [33].
Revised: Popular choices are Average-Pooling [23] and Max-Pooling [36].

Original: Other generalizations also exists, e.g ., L PPooling [39] which generalizes pooling to use different norms.
Revised: Other generalizations also exists, e.g ., L P - Pooling [42] which generalizes pooling to use different norms.

Original: The effectiveness of different poolinglayers has also been studied by Scherer et al.
Revised: The effectiveness of different pooling layers has also been studied by Scherer et al.

Original: More similar to our work is Stochastic-Pooling [47]and Mixed Max-Average Pooling [25].
Revised: More similar to our work is Stochastic-Pooling [54] and Mixed Max-Average Pooling [25].

Original: Stochastic-Pooling constructs a probability distribution bynormalizing activations within a window and sampling during training.
Revised: Stochastic-Pooling constructs a probability distribution by normalizing activations within a window and sampling during training.

Original: In our work, we present a novel design which learns the sampling distribution.
Revised: In our work, we present a novel design which learns the sampling distribution.

Original: Mixed Max-Average Pooling learns a single scalar topermit a soft-choice between Max- and Average-Pooling.
Revised: Mixed Max-Average Pooling learns a single scalar to permit a soft-choice between Max- and Average-Pooling.

Original: In contrast, our LPS has shift-equivarianceguarantees while being end-to-end trainable.
Revised: In contrast, our LPS has shift-equivariance guarantees while being end-to-end trainable.

Original:  Preliminaries
Revised:  Preliminaries

Original: We provide a brief review on equivariant and invariant functions to establish the notation.
Revised: We provide a brief review on equivariant and invariant functions to establish the notation.

Original: Forreadability, we use one-dimensional data to illustrate these ideas.
Revised: For readability, we use one-dimensional data to illustrate these ideas.

Original: In practice, these concepts aregeneralized to multiple channels and two-dimensional data.
Revised: In practice, these concepts are generalized to multiple channels and two-dimensional data.

Original: Shift invariance and equivariance.
Revised: Shift invariance and equivariance.

Original: The concept of equivariance , a generalization of invariance,describes how a function’s output is transformed given that the input is transformed in a predeﬁnedway.
Revised: The concept of equivariance , a generalization of invariance, describes how a function’s output is transformed given that the input is transformed in a predeﬁned way.

Original: For example, shift equivariance describes how the output is shifted given that the input is alsoshifted: think of image segmentation, if an object in the image is shifted then its corresponding maskis also shifted.
Revised: For example, shift equivariance describes how the output is shifted given that the input is also shifted: think of image segmentation, if an object in the image is shifted then its corresponding mask is also shifted.

Original: A function f : R N (cid:55)→ R M is T N , { T M , I } -equivariant (shift-equivariant) if and only if (iff) 
Revised: A function f : R N (cid:55)→ R M is T N , { T M , I } -equivariant (shift-equivariant) if and only if (iff) [Equation]

Original: [Equation]
Revised: [Equation]

Original: where T N x [ n ] (cid:44) x [( n +1) mod N ] ∀ n ∈ Z denotes a circular shift, [ · ] denotes the indexing operator,and I denotes the identity function.
Revised: where T N x [ n ] (cid:44) x [( n +1) mod N ] ∀ n ∈ Z denotes a circular shift, [ · ] denotes the indexing operator, and I denotes the identity function.

Original: This deﬁnition of equivariance handles the ambiguity that ariseswhen shifting by one and downsampling by two.
Revised: This deﬁnition of equivariance handles the ambiguity that arises when shifting by one and downsampling by two.

Original: Ideally, a shift by one at the input should result in a0.5 shift in the downsampled signal, which is not achievable on the integer grid.
Revised: Ideally, a shift by one at the input should result in a 0.5 shift in the downsampled signal, which is not achievable on the integer grid.

Original: Hence, this deﬁnitionconsiders either a shift by one or a no shift at the output as equivariant.
Revised: Hence, this deﬁnition considers either a shift by one or a no shift at the output as equivariant.

Original: Following the equivariance deﬁnition, invariance can be viewed as a special case where the transformation at the output is an identity function, I .
Revised: Following the equivariance deﬁnition, invariance can be viewed as a special case where the transformation at the output is an identity function, I .

Original: Concretely, a function f : R N (cid:55)→ R M is T N , { I } -equivariant (shift-invariant) iff
Revised: Concretely, a function f : R N (cid:55)→ R M is T N , { I } - equivariant (shift-invariant) iff

Original: [Equation]
Revised: [Equation]

Original: To obtain shift-invariance from shift-equivariant functions it is common to use global pooling.
Revised: To obtain shift-invariance from shift-equivariant functions it is common to use global pooling.

Original: Observethat
Revised: Observe that

Original: [Equation]
Revised: [Equation]

Original: is shift-invariant if f is shift-equivariant, as summation is an orderless operation.
Revised: is shift-invariant if f is shift-equivariant, as summation is an orderless operation.

Original: Note that thecomposition of shift-equivariant functions maintains shift-equivariance.
Revised: Note that the composition of shift-equivariant functions maintains shift-equivariance.

Original: Hence, f can be a stack ofequivariant layers, e.g ., a composition of convolution layers.
Revised: Hence, f can be a stack of equivariant layers, e.g ., a composition of convolution layers.

Original: While existing deep-nets [17, 26, 37] do use global spatial pooling, these architectures are not shiftinvariant.
Revised: While existing deep-nets [17, 26, 40] do use global spatial pooling, these architectures are not shiftinvariant.

Original: This is due to pooling and downsampling layers, which are not shift-equivariant as wereview next.
Revised: This is due to pooling and downsampling layers, which are not shift-equivariant as we review next.

Original: Downsampling and pooling layers.
Revised: Downsampling and pooling layers.

Original: A downsampling-by-two layer D : R N (cid:55)→ R (cid:98) N/ 2 (cid:99) is deﬁned as
Revised: A downsampling-by-two layer D : R N (cid:55)→ R (cid:98) N/ 2 (cid:99) is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: which returns the even indices of the input x .
Revised: which returns the even indices of the input x .

Original: As a shift operator makes the odd indices even, adownsampling layer is not shift-equivariant/invariant.
Revised: As a shift operator makes the odd indices even, a downsampling layer is not shift-equivariant/invariant.

Original: Commonly used average or max pooling can be viewed as an average or max ﬁlter followed bydownsampling, hence pooling is also not shift-equivariant/invariant.
Revised: Commonly used average or max pooling can be viewed as an average or max ﬁlter followed by downsampling, hence pooling is also not shift-equivariant/invariant.

Original: To address this issue, Chamanand Dokmanic [5] propose adaptive polyphase sampling (APS) which is an input dependent (adaptive)selection of the odd/even indices.
Revised: To address this issue, Chaman and Dokmanic [5] propose adaptive polyphase sampling (APS) which is an input dependent (adaptive) selection of the odd/even indices.

Original: Adaptive polyphase sampling.
Revised: Adaptive polyphase sampling.

Original: Proposed by Chaman and Dokmanic [5], adaptive polyphase sampling (APS) returns whether the odd or even indices, i.e ., the polyphase components, based on theirnorms.
Revised: Proposed by Chaman and Dokmanic [5], adaptive polyphase sampling (APS) returns whether the odd or even indices, i.e ., the polyphase components, based on their norms.

Original: Formally, APS : R N (cid:55)→ R (cid:98) N/ 2 (cid:99) is deﬁned as:
Revised: Formally, APS : R N (cid:55)→ R (cid:98) N/ 2 (cid:99) is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original:  where x ∈ R N is the input and Poly ( x ) i denotes the polyphase components, i.e .,
Revised: otherwise [Equation] where x ∈ R N is the input and Poly ( x ) i denotes the polyphase components, i.e .,

Original: Poly ( x ) 

Revised: Poly ( x )

Original: [ n ] = x [2 n ] and Poly ( x ) 1 [ n ] = x
Revised: [ n ] = x [2 n ] and Poly ( x ) 1 [ n ] = x

Original: [2 n + 1] .
Revised: [2 n + 1] .

Original: [Equation]

Original: While this handcrafted selection rule achieves a consistent selection of the polyphase components, itis not the only way to achieve it, e.g ., returning the polyphase component with the smaller norm.
Revised: While this handcrafted selection rule achieves a consistent selection of the polyphase components, it is not the only way to achieve it, e.g ., returning the polyphase component with the smaller norm.

Original: Inthis work, we study a family of shift-equivariant sampling layers and propose how to learn them in adata-driven manner.
Revised: In this work, we study a family of shift-equivariant sampling layers and propose how to learn them in a data-driven manner.

Original:  Approach
Revised:  Approach

Original: [Figure]

Original: Our goal is to design a learnable down/upsampling layer that is shift-invariant/equivariant.
Revised: Our goal is to design a learnable down/upsampling layer that is shift-invariant/equivariant.

Original: Weformulate down/upsampling by modeling the conditional probability of selecting each polyphasecomponent given an input.
Revised: We formulate down/upsampling by modeling the conditional probability of selecting each polyphase component given an input.

Original: For this we use a small neural network.
Revised: For this we use a small neural network.

Original: This enables the sampling scheme to be trained end-to-end from data, hence the name learnable polyphase sampling (LPS).
Revised: This enables the sampling scheme to be trained end-to-end from data, hence the name learnable polyphase sampling (LPS).

Original: In Sec. 4.1, we introduce learnable polyphase downsampling (LPD), discuss how to train it end-to-end,and show that it generalizes APS.
Revised: In Sec. 4.1, we introduce learnable polyphase downsampling (LPD), discuss how to train it end-to-end, and show that it generalizes APS.

Original: In Sec. 4.2, we propose a practical layer design of LPD.
Revised: In Sec. 4.2, we propose a practical layer design of LPD.

Original: Lastly,in Sec. 4.3, we discuss how to perform LPS for upsampling, namely, learnable polyphase upsampling(LPU).
Revised: Lastly, in Sec. 4.3, we discuss how to perform LPS for upsampling, namely, learnable polyphase upsampling (LPU).

Original: For readability, we present the approach using one dimensional data, i.e ., a row in an image.
Revised: For readability, we present the approach using one dimensional data, i.e ., a row in an image.

Original: 4.1 Learnable Polyphase Downsampling
Revised: 4.1 Learnable Polyphase Downsampling

Original: We propose learnable polyphase downsampling (LPD) to learn a shift-equivariant downsamplinglayer.
Revised: We propose learnable polyphase downsampling (LPD) to learn a shift-equivariant downsampling layer.

Original: Given an input feature map x ∈ R C × N , LPD spatially downsamples the input to produce anoutput in R C ×(cid:98) N/ 2 (cid:99) viaLPD ( x ) [Equation] [Equation]
Revised: Given an input feature map x ∈ R C × N , LPD spatially downsamples the input to produce an output in R C ×(cid:98) N/ 2 (cid:99) via

Original: [Equation]
Revised: [Equation]

Original: where k (cid:63) = arg max k ∈{ 0 , 1 } p θ ( k = k | x ) and Poly ( x ) k (cid:63) denotes the k (cid:63) -th polyphase component.
Revised: where k (cid:63) = arg max k ∈{ 0 , 1 } p θ ( k = k | x ) and Poly ( x ) k (cid:63) denotes the k (cid:63) -th polyphase component.

Original: We model a conditional probability p θ ( k | x ) for selecting polyphase components, i.e ., k denotes therandom variable of the polyphase indices.
Revised: We model a conditional probability p θ ( k | x ) for selecting polyphase components, i.e ., k denotes the random variable of the polyphase indices.

Original: For 1D data, there are only two polyphase components.
Revised: For 1D data, there are only two polyphase components.

Original: Critically, not all p θ lead to an equivariant downsampling layer.
Revised: Critically, not all p θ lead to an equivariant downsampling layer.

Original: For example, p θ ( k = 0 | x ) = 1results in the standard down-sampling which always returns values on even indices for 1D signals.
Revised: For example, p θ ( k = 0 | x ) =results in the standard down-sampling which always returns values on even indices for 1D signals.

Original: We will next examine which family of p θ achieves a shift-equivariant downsampling layer.
Revised: We will next examine which family of p θ achieves a shift-equivariant downsampling layer.

Original: Shift-permutation equivariance of p θ .
Revised: Shift-permutation equivariance of p θ .

Original: Consider the example in Fig.
Revised: Consider the example in Fig.

Original: We can see that a circularshift in the spatial domain induces a permutation in the polyphase components.
Revised: We can see that a circular shift in the spatial domain induces a permutation in the polyphase components.

Original: Observe that thetop-row of the polyphase component containing the blue circle and orange square are permuted to thesecond row when the input is circularly shifted.
Revised: Observe that the top-row of the polyphase component containing the blue circle and orange square are permuted to the second row when the input is circularly shifted.

Original: We now state this formally.
Revised: We now state this formally.

Original: Lemma 1.
Revised: Lemma 1.

Original: Polyphase shift-permutation property
Revised: Polyphase shift-permutation property

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By deﬁnition, Poly ( T N x ) k
Revised: By deﬁnition, Poly ( T N x ) k

Original: [ n ]
Revised: [ n ]

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation] From Lemma 1, we observe that to achieve an equivariant downsampling layer a spatially shiftedinput should lead to a permutation of the selection probability (Claim 1).
Revised: From Lemma 1, we observe that to achieve an equivariant downsampling layer a spatially shifted input should lead to a permutation of the selection probability (Claim 1).

Original: We note that p θ is saidto be shift-permutation-equivariant if
Revised: We note that p θ is said to be shift-permutation-equivariant if

Original: [Equation]
Revised: [Equation]

Original: where π denotes a permutation on the polyphase indices, i.e ., a “swap” of indices is characterized byπ ( k ) , i.e ., π (0)
Revised: where π denotes a permutation on the polyphase indices, i.e ., a “swap” of indices is characterized by π ( k ) , i.e ., π (0)

Original: = 1 and π (1) = 0 .
Revised: = 1 and π (1) = 0 .

Original: Claim 1.
Revised: Claim 1.

Original: If p θ is shift-permutation-equivariant, deﬁned in Eq.
Revised: If p θ is shift-permutation-equivariant, deﬁned in Eq.

Original: (11) , then LPD deﬁned in Eq.
Revised: (11) , then LPD deﬁned in Eq.

Original: (7) is a shift-equivariant downsampling layer.
Revised: (7) is a shift-equivariant downsampling layer.

Original: Proof.
Revised: Proof.

Original: Let ˆ x (cid:44) T N x be a shifted version of x ∈ R N .
Revised: Let ˆ x (cid:44) T N x be a shifted version of x ∈ R N .

Original: Recall LPD ( x ) and LPD (ˆ x ) are deﬁned as:
Revised: Recall LPD ( x ) and LPD (ˆ x ) are deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: From Lemma 1, LPD ( T N x ) can be expressed as:
Revised: From Lemma 1, LPD ( T N x ) can be expressed as:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: As p θ is the shift-permutation-equivariant,
Revised: As p θ is the shift-permutation-equivariant,

Original: [Equation]
Revised: [Equation]

Original: Finally, combining Eq.
Revised: Finally, combining Eq.

Original: (14) and Eq.
Revised: (14) and Eq.

Original: [Equation]

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: showing that LPD satisﬁes the shift-equivariance deﬁnition reviewed in Eq.
Revised: showing that LPD satisﬁes the shift-equivariance deﬁnition reviewed in Eq.

Original: Here, we parameterize p θ with a small neural network.
Revised: Here, we parameterize p θ with a small neural network.

Original: The exact construction of a shift-permutationequivariant deep-net architecture is deferred to Sec.
Revised: The exact construction of a shift-permutation equivariant deep-net architecture is deferred to Sec.

Original: We next discuss how to train the distributionparameters θ in LPD.
Revised: We next discuss how to train the distribution parameters θ in LPD.

Original: End-to-end training of LPD.
Revised: End-to-end training of LPD.

Original: At training time, to incorporate stochasticity and compute gradients,we parameterize p θ using Gumbel Softmax [18, 27].
Revised: At training time, to incorporate stochasticity and compute gradients, we parameterize p θ using Gumbel Softmax [18, 29].

Original: To backpropagate gradients to θ , we relax theselection of polyphase components as a convex combination, i.e .,
Revised: To backpropagate gradients to θ , we relax the selection of polyphase components as a convex combination, i.e .,

Original: [Equation]
Revised: [Equation]

Original: 17) where z corresponds to a selection variable, i.e ., (cid:80) k
Revised: (17) where z corresponds to a selection variable, i.e ., (cid:80) k

Original: z k = 1 and z k ∈ [0 , 1] .
Revised: z k = 1 and z k ∈ [0 , 1] .

Original: Note the slight abuse ofnotation as p θ ( k | x ) denotes a probability over polyphase indices represented in a one-hot format.
Revised: Note the slight abuse of notation as p θ ( k | x ) denotes a probability over polyphase indices represented in a one-hot format.

Original: We further encourage the Gumbel Softmax to behave more like an argmax by decaying its temperatureτ during training as recommended by Jang et al.
Revised: We further encourage the Gumbel Softmax to behave more like an argmax by decaying its temperature τ during training as recommended by Jang et al.

Original: LPD generalizes APS.
Revised: LPD generalizes APS.

Original: A key advantage of LPS over APS is that it can learn from data, potentiallyleading to a better sampling scheme than a handcrafted one.
Revised: A key advantage of LPS over APS is that it can learn from data, potentially leading to a better sampling scheme than a handcrafted one.

Original: Here, we show that APS is a special caseof LPD.
Revised: Here, we show that APS is a special case of LPD.

Original: Therefore, LPD should perform at least as well as APS if parameters are trained well.
Revised: Therefore, LPD should perform at least as well as APS if parameters are trained well.

Original: Claim 2.
Revised: Claim 2.

Original: APS is a special case of LPD, i.e ., LPD can represent APS’s selection rule.
Revised: APS is a special case of LPD, i.e ., LPD can represent APS’s selection rule.

Original: Proof.
Revised: Proof.

Original: Consider a parametrization of p θ as follows,
Revised: Consider a parametrization of p θ as follows,

Original: [Equation]
Revised: [Equation]

Original: As the exponential is a strictly increasing function we have
Revised: As the exponential is a strictly increasing function we have

Original: [Equation]
Revised: [Equation]

Original: Eq.
Revised: Eq.

Original: (18) is a softmax with input (cid:107) Poly ( x ) k (cid:107) , as such a function exists, LPD generalizes APS.
Revised: (18) is a softmax with input (cid:107) Poly ( x ) k (cid:107) , as such a function exists, LPD generalizes APS.

Original: 4.2 Practical LPD Design
Revised: 4.2 Practical LPD Design

Original: We aim for a conditional distribution p θ that is shift-permutation equivariant to obtain a shiftequivariant pooling layer.
Revised: We aim for a conditional distribution p θ that is shift-permutation equivariant to obtain a shiftequivariant pooling layer.

Original: Let the conditional probability be modeled as:
Revised: Let the conditional probability be modeled as:

Original: [Equation]
Revised: [Equation]

Original: where f θ : R C × H (cid:48) × W (cid:48)
Revised: where f θ : R C × H (cid:48) × W (cid:48)

Original: (cid:55)→ R is a small network that extracts features from polyphase component
Revised: (cid:55)→ R is a small network that extracts features from polyphase component

Original: Poly ( x ) k .
Revised: Poly ( x ) k .

Original: We ﬁrst show that p θ is shift-permutation equivariant if f θ is shift invariant.
Revised: We ﬁrst show that p θ is shift-permutation equivariant if f θ is shift invariant.

Original: Claim 3.
Revised: Claim 3.

Original: In Eq.
Revised: In Eq.

Original: (20) , if f θ is shift invariant then p θ is shift-permutation equivariant (Eq.
Revised: (20) , if f θ is shift invariant then p θ is shift-permutation equivariant (Eq.

Original: Proof.
Revised: Proof.

Original: Denote a feature map x and its shifted version ˆ x (cid:44) T N x .
Revised: Denote a feature map x and its shifted version ˆ x (cid:44) T N x .

Original: By deﬁnition,
Revised: By deﬁnition,

Original: [Equation]
Revised: [Equation]

Original: With a shift-invariant f θ and using Lemma 1,
Revised: With a shift-invariant f θ and using Lemma 1,

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Based on the result in Claim 3, we now present a convolution based meta-architecture that satisﬁes theshift-permutation property.
Revised: Based on the result in Claim 3, we now present a convolution based meta-architecture that satisﬁes the shift-permutation property.

Original: The general design principle: share parameters across polyphase indices,just as convolution achieves shift equivariance by sharing parameters, plus averaging over the spatialdomain.
Revised: The general design principle: share parameters across polyphase indices, just as convolution achieves shift equivariance by sharing parameters, plus averaging over the spatial domain.

Original: An illustration of the proposed meta-architecture is shown in Fig.
Revised: An illustration of the proposed meta-architecture is shown in Fig.

Original: Fully convolutional model.
Revised: Fully convolutional model.

Original: Logits are extracted from the polyphase components via fullyconvolutional operations followed by averaging along the channel and the spatial domain.
Revised: Logits are extracted from the polyphase components via fullyconvolutional operations followed by averaging along the channel and the spatial domain.

Original: Followingthis, f conv θ is denoted as:
Revised: Following this, f conv θ is denoted as:

Original: [Equation]
Revised: [Equation]

Original: where ˜ f conv θ : R C × M (cid:55)→ R C × M is a CNN model (without pooling layers) and M = (cid:98) N/ 2 (cid:99) .
Revised: where ˜ f conv θ : R C × M (cid:55)→ R C × M is a CNN model (without pooling layers) and M = (cid:98) N/ 2 (cid:99) .

Original: Theshift equivariance property of ˜ f conv θ guarantees that f conv θ is shift-invariant due to the global pooling.
Revised: The shift equivariance property of ˜ f conv θ guarantees that f conv θ is shift-invariant due to the global pooling.

Original: 4.3 Learnable Polyphase Upsampling (LPU)
Revised: 4.3 Learnable Polyphase Upsampling (LPU)

Original: Beyond shift invariant models, we extend the theory from downsampling to upsampling, whichpermits to design shift-equivariant models.
Revised: Beyond shift invariant models, we extend the theory from downsampling to upsampling, which permits to design shift-equivariant models.

Original: The main idea is to place the features obtained after downsampling back to their original spatial location.
Revised: The main idea is to place the features obtained after downsampling back to their original spatial location.

Original: Given a feature map y ∈ R C ×(cid:98) N/ 2 (cid:99) downsampledvia LPD from x , the upsampling layer outputs u ∈ R C × N are deﬁned as follows:
Revised: Given a feature map y ∈ R C ×(cid:98) N/ 2 (cid:99) downsampled via LPD from x , the upsampling layer outputs u ∈ R C × N are deﬁned as follows:

Original: [Equation]
Revised: [Equation]

Original: We name this layer learnable polyphase upsampling (LPU), i.e ., LPU ( y , p θ ) (cid:44) u .
Revised: We name this layer learnable polyphase upsampling (LPU), i.e ., LPU ( y , p θ ) (cid:44) u .

Original: We now show that
Revised: We now show that

Original: LPU and LPD achieve shift-equivariance.
Revised: LPU and LPD achieve shift-equivariance.

Original: Claim 4.
Revised: Claim 4.

Original: If p θ is shift-permutation equivariant, as deﬁned in Eq.
Revised: If p θ is shift-permutation equivariant, as deﬁned in Eq.

Original: (11) , then LPU ◦ LPD is shift-equivariant.
Revised: (11) , then LPU ◦ LPD is shift-equivariant.

Original: Proof.
Revised: Proof.

Original: We prove this claim following deﬁnitions of LPU, LPD and Lemma 1.
Revised: We prove this claim following deﬁnitions of LPU, LPD and Lemma 1.

Original: The complete proof isdeferred to Appendix Sec. A1.
Revised: The complete proof is deferred to Appendix Sec. A1.

Original: End-to-end training of LPU.
Revised: End-to-end training of LPU.

Original: As in downsampling, we also incorporate stochasticity via GumbelSoftmax.
Revised: As in downsampling, we also incorporate stochasticity via GumbelSoftmax.

Original: To backpropagate gradients to p θ , we relax the hard selection into a convex combination, i [Table] e .,
Revised: To backpropagate gradients to p θ , we relax the hard selection into a convex combination, i.e .,

Original: Poly (

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Anti-aliasing for upsampling.
Revised: Anti-aliasing for upsampling.

Original: While LPU provides a shift-equivariant upsampling scheme, it introduces zeros in the output which results in high-frequency components.
Revised: While LPU provides a shift-equivariant upsampling scheme, it introduces zeros in the output which results in high-frequency components.

Original: This is known as aliasing ina multirate system [43].
Revised: This is known as aliasing in a multirate system [47].

Original: To resolve this, following the classical solution, we apply a low-pass ﬁlterscaled by the upsampling factor after each LPU.
Revised: To resolve this, following the classical solution, we apply a low-pass ﬁlter scaled by the upsampling factor after each LPU.

Original:  Experiments
Revised:  Experiments

Original: [Figure]

Original: [Table]
Revised: [Table]

Original: We conduct experiments on image classiﬁcation following prior works.
Revised: We conduct experiments on image classiﬁcation following prior works.

Original: We report on thesame architectures and training setup.
Revised: We report on the same architectures and training setup.

Original: We report both the circular shift setup in APS [5]and the standard shift setup in LPF [49].
Revised: We report both the circular shift setup in APS [5] and the standard shift setup in LPF [56] [Table]

Original: We also evaluate on semantic segmentation,considering the circular shift, inspired by
Revised: We also evaluate on semantic segmentation, considering the circular shift, inspired by

Original: APS, and the standard shift setup following
Revised: APS, and the standard shift setup following

Original: DDAC [50].
Revised: DDAC [57].

Original: For circular shift settings, thetheory exactly matches the experiment hencetrue equivariance is achieved.
Revised: For circular shift settings, the theory exactly matches the experiment hence true equivariance is achieved.

Original: To our knowledge, this is the ﬁrst truly shift equivariantmodel reported on PASCAL VOC.
Revised: To our knowledge, this is the ﬁrst truly shift equivariant model reported on PASCAL VOC.

Original: 5.1 ImageClassiﬁcation (Circular Shift)
Revised: 5.1 Image

Revised: Classiﬁcation (Circular Shift)

Original: Experiment & implementation details.
Revised: Experiment & implementation details.

Original: Following APS, all the evaluated pooling and anti-aliasing models use the ResNet-18 [17] architecturewith circular padding on CIFAR10 [21] and ImageNet [12].
Revised: Following APS, all the evaluated pooling and anti-aliasing models use the ResNet-18 [17] architecture with circular padding on CIFAR10 [21] and ImageNet [12].

Original: Anti-alias ﬁlters are applied after eachdownsampling layer following LPF [49] and DDAC [50].
Revised: Anti-alias ﬁlters are applied after each downsampling layer following LPF [56] and DDAC [57].

Original: We also replace downsampling layers with
Revised: We also replace downsampling layers with

Original: APS [5] and our proposed LPS layer.
Revised: APS [5] and our proposed LPS layer.

Original: We provide more experimental details in Appendix Sec. A4.
Revised: We provide more experimental details in Appendix Sec. A4.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: We report classiﬁcation accuracy to quantify the model performance on theoriginal dataset without any shifts.
Revised: We report classiﬁcation accuracy to quantify the model performance on the original dataset without any shifts.

Original: To evaluate shift-invariance, following APS, we report circularconsistency (C-Cons.) which computes the average percentage of predicted labels that are equal undertwo different circular shifts, i.e .,ˆ y [Equation]
Revised: To evaluate shift-invariance, following APS, we report circularconsistency (C-Cons.) which computes the average percentage of predicted labels that are equal under two different circular shifts, i.e .,

Original: [Equation]
Revised: [Equation]

Original: where ˆ y ( I ) denotes the predicted label for an input image I and h 1 , w 1 , h 2 , w 2 are uniformlysampled from 0 to 32.
Revised: where ˆ y ( I ) denotes the predicted label for an input image I and h 1 , w 1 , h 2 , w 2 are uniformly sampled from 0 to 32.

Original: We report the average over ﬁve random seeds.
Revised: We report the average over ﬁve random seeds.

Original: CIFAR10 results.
Revised: CIFAR10 results.

Original: Tab. 1 shows the classiﬁcation accuracy and circular consistency on CIFAR10.
Revised: Tab. 1 shows the classiﬁcation accuracy and circular consistency on CIFAR10.

Original: We report the mean and standard deviation over ﬁve runs with different random initialization ofthe ResNet-18 model.
Revised: We report the mean and standard deviation over ﬁve runs with different random initialization of the ResNet-18 model.

Original: We observe that the proposed LPS improves classiﬁcation accuracy over allbaselines while achieving 100% circular consistency.
Revised: We observe that the proposed LPS improves classiﬁcation accuracy over all baselines while achieving 100% circular consistency.

Original: In addition to attaining perfect shift consistency,we observe that combining anti-aliasing with LPS further improves performance.
Revised: In addition to attaining perfect shift consistency, we observe that combining anti-aliasing with LPS further improves performance.

Original: ImageNet results.
Revised: ImageNet results.

Original: We conduct experiments on ImageNet with circular shift using ResNet-18.
Revised: We conduct experiments on ImageNet with circular shift using ResNet-18.

Original: In Tab. 2, we compare with APS’s best model using a box ﬁlter (Rectangle-2), as reported by Chamanand Dokmanic [5].
Revised: In Tab. [Table] we compare with APS’s best model using a box ﬁlter (Rectangle-2), as reported by Chaman and Dokmanic [5].

Original: While both APS and LPS achieve 100% circular consistency, our proposed LPSimproves on classiﬁcation accuracy in all scenarios, highlighting its advantages.
Revised: While both APS and LPS achieve 100% circular consistency, our proposed LPS improves on classiﬁcation accuracy in all scenarios, highlighting its advantages

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: 5.2 Image Classiﬁcation (Standard Shift)
Revised: 5.2 Image Classiﬁcation (Standard Shift)

Original: Experiment & implementation details.
Revised: Experiment & implementation details.

Original: To directly compare with results from LPF and DDAC,we conduct experiments on ImageNet using the ResNet-50 and ResNet-101 architectures followingtheir setting, i.e ., training with standard shifts augmentation and using convolution layers withzero-padding.
Revised: To directly compare with results from LPF and DDAC, we conduct experiments on ImageNet using the ResNet-50 and ResNet-101 architectures following their setting, i.e ., training with standard shifts augmentation and using convolution layers with zero-padding.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: Shift consistency (S-Cons.) computes the average percentage of
Revised: Shift consistency (S-Cons.) computes the average percentage of

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where h 1 , w 1 , h 2 , w 2 are uniformly sampled from the interval { 0 , . .
Revised: where h 1 , w 1 , h 2 , w 2 are uniformly sampled from the interval { 0 , . .

Original: To avoid padding at theboundary, following LPF [49], we perform a shift on an image then crop its center 224 × 224 region.
Revised: To avoid padding at the boundary, following LPF [56], we perform a shift on an image then crop its center 224 × 224 region.

Original: We note that, due to the change in content at the boundary, perfect shift consistency is not guaranteed.
Revised: We note that, due to the change in content at the boundary, perfect shift consistency is not guaranteed.

Original: ImageNet results.
Revised: ImageNet results.

Original: In Tab. 3, we compare to the best anti-aliasing result as reported in LPF, DDACand DDAC ∗ which is trained from the authors’ released code using hyperparameters speciﬁed in therepository.
Revised: In Tab. 3, we compare to the best anti-aliasing result as reported in LPF, DDAC and DDAC ∗ which is trained from the authors’ released code using hyperparameters speciﬁed in the repository.

Original: Note, in standard shift setting LPS no longer achieves true shift-invariance due to paddingat the boundaries.
Revised: Note, in standard shift setting LPS no longer achieves true shift-invariance due to padding at the boundaries.

Original: Despite this gap from the theory, LPS achieves improvements in both performanceand shift-consistency over the baselines.
Revised: Despite this gap from the theory, LPS achieves improvements in both performance and shift-consistency over the baselines.

Original: When compared to LPF, both ResNet-50 and ResNet-101architecture achieved improved classiﬁcation accuracy and shift-consistency.
Revised: When compared to LPF, both ResNet-50 and ResNetarchitecture achieved improved classiﬁcation accuracy and shift-consistency.

Original: When compared to
Revised: When compared to

Original: DDAC, LPS achieves comparable accuracy with higher shift-consistency.
Revised: DDAC, LPS achieves comparable accuracy with higher shift-consistency.

Original: 5.3 Additional Analysis
Revised: 5.3 Trainable Parameters and Inference Time

Revised: While LPD is a data-driven downsampling layer, we show that the additional trainable parameters introduced by it are marginal with respect to the classiﬁcation architecture. Tab. 4 shows the number of trainable parameters required by the ResNet-101 models. For each method, we report the absolute number of trainable parameters, which includes both classiﬁer and learnable pooling weights.

Revised: We also include the relative number of trainable parameters, which only considers the learnable pooling weights and the percentage it represents with respect to the default ResNet-101 architecture weights.

Original: LPD learns sampling schemes different from APS.
Revised: For comparison purposes, we also include the inference time required by each model to evaluate their computational overhead. Mean and standard deviation of the inference time is computed for [Table] each method on 100 batches of size 32 .

Revised: Following ImageNet default settings, the image dimensions corrrespond to 224 × 224 × 3 .

Revised: Results show our proposed LPD method introduces approximately 1% additional trainable parameters on the ResNet-101 architecture

Revised: [Table]

Revised: and increases the inference time roughly by 14 . 89 ms over the LPF anti-aliasing method (the less computationally expensive of the evaluated techniques).

Revised: On the other hand, most of the overhead comes from DDAC, which increases the number of trainable parameters by approximately 4% and the inference time by approximately 55 .

Revised: 97 ms. Overall, our comparison shows that, by equipping a classiﬁer with LPD layers, the computational overhead is almost trivial.

Revised: Despite increasing the number of trainable parameters, we empirically show that our LPD approach outperforms classiﬁers with signiﬁcantly more parameters.

Revised: Please refer to Sec.

Revised: A4.3 for additional experiments comparing the performance of our ResNet-101 + LPD model against the much larger

Revised: ResNet-152 classiﬁer.

Revised: LPD learns sampling schemes different from APS.

Original: To further analyze LPD, we replace all the
Revised: To further analyze LPD, we replace all the

Original: LPD layers with APS for a ResNet-101 model trained on ImageNet.
Revised: LPD layers with APS for a ResNet-101 model trained on ImageNet.

Original: We observe a critical drop intop-1 classiﬁcation accuracy from 78 .
Revised: We observe a critical drop in top-1 classiﬁcation accuracy from 78 .

Original: 8% to 0 .
Revised: 8% to 0 .

Original: 1% , indicating that LPD did not learn a downsamplingscheme equivalent to APS.
Revised: 1% , indicating that LPD did not learn a downsampling scheme equivalent to APS.

Original: We also counted how many times (across all layers) LPD selects themax-norm.
Revised: We also counted how many times (across all layers) LPD selects the max-norm.

Original: On the ImageNet validation set, LPD selects the max (cid:96) 2 -norm polyphase component only20 .
Revised: On the ImageNet validation set, LPD selects the max (cid:96) 2 -norm polyphase component only 20 .

Original: 57% of the time.
Revised: 57% of the time.

Original: These show LPD learned a selection rule that differs from the handcrafted APS.
Revised: These show LPD learned a selection rule that differs from the handcrafted APS.

Original: Qualitative study on LPD.
Revised: Qualitative study on LPD.

Original: In Fig.
Revised: In Fig.

Original: 4 we show the selected activations, at the fourth layer, of a
Revised: 4 we show the selected activations, at the fourth layer, of a

Original: ResNet-50 model with LPD.
Revised: ResNet-50 model with LPD.

Original: Each column describes the ﬁrst 8 channels of the four possible polyphasecomponents k ∈ { 0 , . .
Revised: Each column describes the ﬁrst 8 channels of the four possible polyphase components k ∈ { 0 , . .

Original: The component selected by LPD, denoted as k (cid:63) , is boxed in blue.
Revised: The component selected by LPD, denoted as k (cid:63) , is boxed in blue.

Original: Forcomparison purposes, we also boxed the component that maximizes the (cid:96) 2 -norm in red.
Revised: For comparison purposes, we also boxed the component that maximizes the (cid:96) 2 -norm in red.

Original: We observe[Table] [Table] that LPD is distinct from APS as they select a different set of polyphase components.
Revised: We observe that LPD is distinct from APS as they select a different set of polyphase components.

Original: However, wedid not observe a speciﬁc pattern that can explain LPD’s selection rule.
Revised: However, we did not observe a speciﬁc pattern that can explain LPD’s selection rule.

Original: 5.4 Semantic Segmentation (Circular Shift)
Revised: 5.4 Semantic Segmentation (Circular Shift)

Original: Experiment & implementation details.
Revised: Experiment & implementation details.

Original: We evaluate LPS’s down/upsampling layers on semanticsegmentation.
Revised: We evaluate LPS’s down/upsampling layers on semantic segmentation.

Original: As in DDAC [50], we evaluate using the PASCAL VOC [13] dataset.
Revised: As in DDAC [57], we evaluate using the PASCAL VOC [13] dataset.

Original: Following DDAC,we use DeepLabV3+ [6] as our baseline model.
Revised: Following DDAC, we use DeepLabV3+ [6] as our baseline model.

Original: We use the ResNet-18 backbone pre-trained on the
Revised: We use the ResNet-18 backbone pre-trained on the

Original: ImageNet (circular shift) reported in Sec. 5.1.
Revised: ImageNet (circular shift) reported in Sec. 5.1.

Original: We experiment with using only the LPD backbone andthe full LPS, i.e ., both LPD and LPU. 
Revised: We experiment with using only the LPD backbone and the full LPS, i.e ., both LPD and LPU. [Table]

Revised: [Table]

Revised: We also evaluate the performance using APS, which corresponds to a hand-crafted downsampling scheme, in combination with the default bilinear interpolation strategy from DeepLabV3+.

Revised: Note that, while our LPS approach consists of both shift equivariant down and upsampling schemes (LPD and LPU, respectively), APS only operates on the downsampling process.

Revised: Thus, the latter does not guarantee a circularly shift equivariant segmentation.

Original: Evaluation metric.
Revised: Evaluation metric.

Original: We report mean intersection over union (mIoU) to evaluate segmentation performance.
Revised: We report mean intersection over union (mIoU) to evaluate segmentation performance.

Original: To quantify circular-equivariance, we report mean Average Segmentation Circular Consistency(mASCC) which computes the average percentage of predicted (per-pixel) labels that remained thesame under two different circular shifts. 
Revised: To quantify circular-equivariance, we report mean Average Segmentation Circular Consistency (mASCC) which computes the average percentage of predicted (per-pixel) labels that remained the same under two different circular shifts. I [Table]

Original: I.e ., a shifted image is passed to a model to make a segmentation prediction.
Revised: e ., a shifted image is passed to a model to make a segmentation prediction.

Original: This prediction is then “unshifted” for comparison.
Revised: This prediction is then “unshifted” for comparison.

Original: We report ﬁve random shiftpairs for each image.
Revised: We report ﬁve random shift pairs for each image.

Original: Results.
Revised: Results.

Original: We report the results for PASCAL VOC in Tab. 4.
Revised: We report the results for PASCAL VOC in Tab. 5.

Original: Overall, we observe that LPD only and
Revised: Overall, we observe that LPD only and

Original: LPS achieve comparable results to DDAC and APS in mIoU.
Revised: LPS achieve comparable results to DDAC and APS in mIoU.

Original: Notably, LPS achieves 100% mASCC,matching the theory.
Revised: Notably, LPS achieves 100% mASCC, matching the theory.

Original: This conﬁrms that both the proposed LPD and LPU layers are necessary and areable to learn effective down/up sampling schemes for semantic segmentation.
Revised: This conﬁrms that both the proposed LPD and LPU layers are necessary and are able to learn effective down/up sampling schemes for semantic segmentation.

Original: 5.5 Semantic Segmentation (Standard Shift)
Revised: 5.5 Semantic Segmentation (Standard Shift)

Original: Experiment & implementation details.
Revised: Experiment & implementation details.

Original: For the standard shift setting, we directly follow the experimental setup from DDAC.
Revised: For the standard shift setting, we directly follow the experimental setup from DDAC.

Original: We use DeepLabV3+ with a ResNet-101 backbone pre-trained on
Revised: We use DeepLabV3+ with a ResNet-101 backbone pre-trained on

Original: ImageNet as reported in Sec.
Revised: ImageNet as reported in Sec.

Original: Evaluation metric.
Revised: Evaluation metric.

Original: To quantify the shift-equivariance, following DDAC, we report the mean Average
Revised: To quantify the shift-equivariance, following DDAC, we report the mean Average

Original: Semantic Segmentation Consistency (mASSC) which is a linear-shift version of mASCC describedin Sec.
Revised: Semantic Segmentation Consistency (mASSC) which is a linear-shift version of mASCC described in Sec.

Original: 5.4 except boundary pixels are ignored.
Revised: 5.4 except boundary pixels are ignored.

Original: Results.
Revised: Results.

Original: In Tab. 5, we compare mIoU and mASSC of LPS to various baselines.
Revised: In Tab. 6, we compare mIoU and mASSC of LPS to various baselines.

Original: We observe that
Revised: We observe that

Original: LPS achieves improvements in mIoU and consistency when compared to DDAC ∗ .
Revised: LPS achieves improvements in mIoU and consistency when compared to DDAC ∗ .

Original: We note that
Revised: We note that

Original: DDAC [50] did not release their code for mASSC.
Revised: DDAC [57] did not release their code for mASSC.

Original: For a fair comparison, we report the performanceof their released checkpoint using our implementation of mASSC, indicated with DDAC ∗ .
Revised: For a fair comparison, we report the performance of their released checkpoint using our implementation of mASSC, indicated with DDAC ∗ .

Original: Despitethe gap in theory and practice due to non-circular padding at the boundary, our experiments show
Revised: Despite the gap in theory and practice due to non-circular padding at the boundary, our experiments show

Original: LPS remains an effective approach to improve both shift consistency and model performance.
Revised: LPS remains an effective approach to improve both shift consistency and model performance.

Original:  Conclusion
Revised:  Conclusion

Original: We propose learnable polyphase sampling (LPS), a pair of shift-equivariant down/upsampling layers.
Revised: We propose learnable polyphase sampling (LPS), a pair of shift-equivariant down/upsampling layers.

Original: LPS’s design theoretically guarantees circular shift-invariance and equivariance while beingend-to-end trainable.
Revised: LPS’s design theoretically guarantees circular shift-invariance and equivariance while being end-to-end trainable.

Original: Additionally, LPS retains superior consistency on standard shifts where theoretical assumptions are broken at image boundaries.
Revised: Additionally, LPS retains superior consistency on standard shifts where theoretical assumptions are broken at image boundaries.

Original: Finally, LPS captures a richer family ofshift-invariant/equivariant functions than APS.
Revised: Finally, LPS captures a richer family of shift-invariant/equivariant functions than APS.

Original: Through extensive experiments on image classiﬁcation and semantic segmentation, we demonstrate that LPS is on-par with/exceeds APS, LPF and
Revised: Through extensive experiments on image classiﬁcation and semantic segmentation, we demonstrate that LPS is on-par with/exceeds APS, LPF and

Original: DDAC in terms of model performance and consistency.
Revised: DDAC in terms of model performance and consistency.

Revised: Acknowledgments: We thank Greg Shakhnarovich & PALS at TTI-Chicago for the thoughtful discussions