According to Lemma 2.6, and since the convolution product commutes with translations, we have

$$ (4.3) \qquad (F_{\mathcal{T}_u X} * \bar{\Psi}_W)(\boldsymbol{x}) = \mathcal{T}_{su}(F_X * \bar{\Psi}_W)(\boldsymbol{x}). $$

Then, using (3.51), the above expression becomes

$$ (4.4) \qquad \mathbf{M}_{\mathcal{T}_u X}(\boldsymbol{x}) \times \mathbf{Z}_{\mathcal{T}_u X}(\boldsymbol{x}) = (\mathcal{T}_{su}\mathbf{M}_X)(\boldsymbol{x}) \times (\mathcal{T}_{su}\mathbf{Z}_X)(\boldsymbol{x}). $$

Therefore, we necessarily have (4.1). On the one hand, if $\mathbf{M}_{\mathcal{T}_u X}(\boldsymbol{x}) > 0$, then (4.2) is satisfied, by uniqueness of the magnitude-phase decomposition. On the other hand, if $\mathbf{M}_{\mathcal{T}_u X}(\boldsymbol{x}) = 0$, then (3.54) also guarantees (4.2), by design.

Finally, we remind that

$$ (4.5) \qquad \mathcal{T}_{su}\mathbf{M}_X(\boldsymbol{x}) = \mathbf{M}_X(\boldsymbol{x} - s\boldsymbol{u}) \quad \text{and} \quad \mathcal{T}_{su}\mathbf{Z}_X(\boldsymbol{x}) = \mathbf{Z}_X(\boldsymbol{x} - s\boldsymbol{u}). $$

Then, considering hypotheses Hypotheses 3.7 and 3.8 with $\boldsymbol{x} \leftarrow \boldsymbol{x} - s\boldsymbol{u}$ yields the result. ■

We are now ready to state the main result about shift invariance of $\mathbb{R}$Max outputs.

**Theorem 4.2 (Shift invariance of $\mathbb{R}$Max).** We assume that the requirements stated in Theorem 3.11 are satisfied. Besides, given a translation vector $\boldsymbol{u} \in \mathbb{R}^2$, we consider the following random variable:

$$ (4.6) \qquad \tilde{\mathbf{R}}_{\boldsymbol{u}, \boldsymbol{x}} := \|U_{m,q}^{\max}(\mathcal{T}_{\boldsymbol{u}}X) - U_{m,q}^{\max}X\|_2 / \|U_m^{\text{mod}}X\|_2. $$

Then, under condition (2.53), we have

$$ (4.7) \qquad \mathbb{E}[\tilde{\mathbf{R}}_{\boldsymbol{u}, \boldsymbol{x}}] \le 2 (\beta_q(m\kappa) + \gamma_q(m\theta)) + \alpha(\kappa\boldsymbol{u}), $$

where $\alpha, \beta_q$ and $\gamma_q$ are defined in (2.14), (3.37) and (3.63), respectively.

*Proof.* Using the triangle inequality, we compute

$$ (4.8) \qquad \begin{aligned} & \|U_{m,q}^{\max}(\mathcal{T}_{\boldsymbol{u}}X) - U_{m,q}^{\max}X\|_2 \\ & \le \|U_m^{\text{mod}}(\mathcal{T}_{\boldsymbol{u}}X)\|_2 \tilde{\mathbf{P}}_{\mathcal{T}_{\boldsymbol{u}}X} + \|U_m^{\text{mod}}X\|_2 \tilde{\mathbf{P}}_{\mathbf{X}} + \|U_m^{\text{mod}}(\mathcal{T}_{\boldsymbol{u}}X) - U_m^{\text{mod}}X\|_2, \end{aligned} $$

where $\tilde{\mathbf{P}}_{\mathbf{X}}$ and $\tilde{\mathbf{P}}_{\mathcal{T}_{\boldsymbol{u}}X}$ are defined in (3.58). According to (2.53), we can apply Proposition 2.10 on the first term of (4.8):

$$ (4.9) \qquad \|U_m^{\text{mod}}(\mathcal{T}_u X)\|_2 = \|U_m^{\text{mod}} X\|_2. $$

Moreover, we can apply Theorem 2.9 to the third term of (4.8):

$$ (4.10) \qquad \|U_m^{\text{mod}}(\mathcal{T}_u X) - U_m^{\text{mod}} X\|_2 \le \alpha(\kappa u) \|U_m^{\text{mod}} X\|_2. $$

We therefore get

$$ (4.11) \qquad \|U_{m,q}^{\max}(\mathcal{T}_u X) - U_{m,q}^{\max} X\|_2 \le [\tilde{\mathbf{P}}_{\mathcal{T}_u X} + \tilde{\mathbf{P}}_{\mathbf{X}} + \alpha(\kappa \boldsymbol{u})] \|U_m^{\text{mod}} X\|_2. $$
The sum-integral interchange is possible because X has a finite support. Then:

$$ (F_X * \bar{\Psi}_W)(msn) = \sum_{p \in \mathbb{Z}^2} X[p] \iint_{\mathbb{R}^2} \bar{\Psi}_W(x) \Phi^{(s)}(s(mn - p) - x) d^2 x \quad (149) $$

$$ = \sum_{p \in \mathbb{Z}^2} X[p] (\bar{\Psi}_W * \Phi^{(s)})(s(mn - p)) \quad (150) $$

Since {$\Phi_n^{(s)}$}$_{n \in \mathbb{Z}^2}$ is an orthonormal basis of $\mathcal{V}^{(s)}$, the definition of $\Psi_W$ in (24) implies, for any $p' \in \mathbb{Z}^2$,

$$ \bar{W}[p'] = \langle \Psi_W, \Phi_{-p'}^{(s)} \rangle = (\bar{\Psi}_W * \Phi^{(s)})(sp'), \quad (151) $$

because $\Phi^{(s)}$ is even. Therefore, plugging (151) with $p' \leftarrow (mn - p)$ into (150) yields

$$ (F_X * \bar{\Psi}_W)(msn) = \sum_{p \in \mathbb{Z}^2} X[p] \bar{W}[mn - p] = (X * \bar{W})[mn], \quad (152) $$

hence the result.

□

## A.5 Proof of Lemma 3

*Proof.* Let $u \in \mathbb{R}^2$. By definition of $F_{\mathcal{T}_u X}$ and $\mathcal{T}_u X$,

$$ F_{\mathcal{T}_u X} = s \sum_{n \in \mathbb{Z}^2} \mathcal{T}_{su} F_X (sn) \Phi_n^{(s)}. \quad (153) $$

On the other hand, $F_X \in \mathcal{V}^{(s)}$ by construction. Therefore, $\mathcal{T}_{su} F_X \in \mathcal{V}^{(s)}$. Then, according to Shannon's sampling theorem (Mallat, 2009, Theorem 3.11, p. 81), we get

$$ \mathcal{T}_{su} F_X = s \sum_{n \in \mathbb{Z}^2} \mathcal{T}_{su} F_X (sn) \Phi_n^{(s)}, \quad (154) $$

which concludes the proof.

□

## A.6 Proof of Lemma 4

*Proof.* Let us write:

$$ \sum_{n \in \mathbb{Z}^2} |\mathcal{T}_h F_0(s'n) - F_0(s'n)|^2 = \sum_{n \in \mathbb{Z}^2} |F_1(s'n)|^2 = \frac{1}{s'^2} \|X_1\|_2^2, \quad (155) $$

where we have denoted, for any $n \in \mathbb{Z}^2$,

$$ F_1 := \mathcal{T}_h F_0 - F_0 \quad \text{and} \quad X_1[n] := s' F_1(s'n). \quad (156) $$

According to Proposition 2 (25), $\Psi_W \in \mathcal{V}(\theta/s, \kappa/s)$. Therefore, according to Lemma 1,

$$ \text{supp} \widehat{F}_0 \subset B_\infty \left( \frac{\kappa}{2s} \right). \quad (157) $$

Moreover, by hypothesis, $\kappa \le \pi/m$; thus,

$$ B_\infty \left( \frac{\kappa}{2s} \right) \subset B_\infty \left( \frac{\pi}{s'} \right), \quad (158) $$

which yields (35), and $F_1 \in \mathcal{V}^{(s')}$. Then, according to Lemma 2 (23) with $X \leftarrow X_1$, $F \leftarrow F_1$ and $s \leftarrow s'$,

$$ \|X_1\|_2 = \|F_1\|_{L^2} = \|\mathcal{T}_h F_0 - F_0\|_{L^2}. \quad (159) $$
Then, by linearity of $\mathbb{E}$, we get

$$ (4.12) \qquad \mathbb{E}[\tilde{\mathbb{R}}_{\mathbf{X}, \mathbf{u}}] \le \mathbb{E}[\tilde{\mathbb{P}}_{\mathcal{T}_{\mathbf{u}}\mathbf{X}}] + \mathbb{E}[\tilde{\mathbb{P}}_{\mathbf{X}}] + \alpha(\kappa\mathbf{u}), $$

where $\tilde{\mathbb{R}}_{\mathbf{X}, \mathbf{u}}$ has been introduced in (4.7).

For any stochastic process $X'$ satisfying Hypotheses 3.7 and 3.8, Theorem 3.11 and Jensen's inequality yield:

$$ (4.13) \qquad \mathbb{E}[\tilde{\mathbb{P}}_{X'}] \le \beta_q(m\kappa) + \gamma_q(m\theta). $$

According to Lemma 4.1, Hypotheses 3.7 and 3.8 are also satisfied for $X \leftarrow \mathcal{T}_{\mathbf{u}}\mathbf{X}$. Therefore, (4.13) is valid for both $X' \leftarrow X$ and $X' \leftarrow \mathcal{T}_{\mathbf{u}}\mathbf{X}$, and plugging it into (4.12) concludes the proof. ■

In the bound established in (4.7), the sum $\beta_q(m\kappa) + \gamma_q(m\theta)$ accounts for the discrepancy between $\mathbb{R}Max$ and $\mathbb{C}Mod$ outputs, as stated in Theorem 3.11, whereas the term $\alpha(\kappa\mathbf{u})$ characterizes the stability of $\mathbb{C}Mod$ outputs, as stated in Theorem 2.9. If $\kappa$ is sufficiently small, then $\alpha(\kappa\mathbf{u})$ and $\beta_q(m\kappa)$ become negligible with respect to $\gamma_q(m\theta)$, and the bound can be approximated by $2\gamma_q(m\theta)$. Theorem 4.2 therefore provides a validity domain for shift invariance of $\mathbb{R}Max$ operators, as illustrated in Figure 5 with $q = 1$.

**Remark 4.3.** The stochastic discrepancy introduced in (4.6) is estimated relatively to the $\mathbb{C}Mod$ output. This choice is motivated by the perfect shift invariance of its norm, as shown in Proposition 2.10.

**Remark 4.4.** In practice, most of the time max pooling is performed on a grid of size $3 \times 3$; therefore $q = 1$. For the sake of conciseness, we shall sometimes drop $q$ in the notations, which implicitly means $q = 1$.

**5. Adaptation to Multichannel Convolution Operators.** In this section, we adapt Theorems 2.9, 3.11, and 4.2 to multichannel inputs (e.g., RGB images), employed in conventional CNNs such as AlexNet or ResNet.

First, we define multichannel $\mathbb{R}Max$ and $\mathbb{C}Mod$ operators relatively to (1.1) and (1.5). We denote by $K$ and $L \in \mathbb{N} \setminus \{0\}$ the number of input and output channels, respectively. Besides, we consider a multichannel convolution tensor

$$ (5.1) \qquad \mathbf{W} := (W_{lk})_{l \in \{0..L-1\}, k \in \{0..K-1\}} \in (l_{\mathbb{C}}^2(Z^2))^{L \times K}. $$

Multichannel $\mathbb{R}Max$ and $\mathbb{C}Mod$ operators take as input images, denoted by

$$ (5.2) \qquad \mathbf{X} := (X_k)_{k \in \{0..K-1\}} \in (l_{\mathbb{R}}^2(Z^2))^K. $$

They are defined, for any given output channel $l \in \{0..L-1\}$, by

$$ (5.3) \qquad U_{m,q,l}^{\max}[\mathbf{W}] : \mathbf{X} \mapsto \mathrm{MaxPool}_q \left( \sum_{k=0}^{K-1} (X_k * \mathrm{Re} \, \overline{W}_{lk}) \downarrow m \right); $$

$$ (5.4) \qquad U_{m,l}^{\mathrm{mod}}[\mathbf{W}] : \mathbf{X} \mapsto \left| \sum_{k=0}^{K-1} (X_k * \overline{W}_{lk}) \downarrow (2m) \right|, $$
Therefore, plugging (159) into (155) yields (36).

Furthermore, according again to Lemma 2,

$$ \|F_0\|_{L^2}^2 = \|X_0\|_2^2, \qquad (160) $$

where we have defined, for any $n \in \mathbb{Z}^2$,

$$ X_0[n] := s'F_0(s'n). \qquad (161) $$

Then,

$$ \|X_0\|_2^2 = s'^2 \sum_{n \in \mathbb{Z}^2} |(F_X * \bar{\Psi}_W)(s'n)|^2 \qquad (\text{acc. to (34)}) $$

$$ = s'^2 \sum_{n \in \mathbb{Z}^2} |(X * \bar{W}) \downarrow (2m)[n]|^2 \qquad (\text{acc. to Proposition 2 with } m \leftarrow 2m) $$

$$ = s'^2 \|U_m^{\text{mod}} X\|_2^2. \qquad (\text{acc. to (5)}) $$

Finally, plugging this result into (160) yields (37) and concludes the proof. □

## A.7 Proof of Theorem 1

*Proof.* As in Lemma 4, we consider the low-frequency function $F_0$ satisfying (34), and denote $s' := 2ms$. We can write

$$ |F_X * \bar{\Psi}_W| = |F_0| \quad \text{and} \quad |\mathcal{T}_{su} F_X * \bar{\Psi}_W| = |\mathcal{T}_{su} F_0|. \qquad (162) $$

Recall that $U_m^{\text{mod}}X = |(X * \bar{W}) \downarrow (2m)|$, such as defined in (5). According to Proposition 2 (27) and Corollary 1 (32) with $m \leftarrow 2m$, we therefore get

$$ U_m^{\text{mod}} X[n] = |F_0(s'n)|; \qquad (163) $$

$$ U_m^{\text{mod}}(\mathcal{T}_u X)[n] = |(\mathcal{T}_{su} F_0)(s'n)|. \qquad (164) $$

Then, using (163), (164) and the reverse triangle inequality,

$$ \begin{aligned} \|U_m^{\text{mod}}(\mathcal{T}_u X) - U_m^{\text{mod}} X\|_2^2 &= \sum_{n \in \mathbb{Z}^2} \left| |(\mathcal{T}_{su} F_0)(s'n)| - |F_0(s'n)| \right|^2 \\ &\le \sum_{n \in \mathbb{Z}^2} \left| |(\mathcal{T}_{su} F_0)(s'n) - F_0(s'n)| \right|^2. \end{aligned} $$

Since condition (38) is satisfied, we can use Lemma 4 (36) with $h \leftarrow su$:

$$ \|U_m^{\text{mod}}(\mathcal{T}_u X) - U_m^{\text{mod}} X\|_2^2 \le \frac{1}{s'^2} \|\mathcal{T}_{su} F_0 - F_0\|_{L^2}^2 \qquad (165) $$

Next, according to Proposition 1 with $\varepsilon \leftarrow \kappa/s$ and $h \leftarrow su$, we then get the following bound:

$$ \|U_m^{\text{mod}}(\mathcal{T}_u X) - U_m^{\text{mod}} X\|_2^2 \le \frac{\alpha(\kappa u)^2}{s'^2} \|F_0\|_{L^2}^2. \qquad (166) $$

Finally, using Lemma 4 (37) yields (39), which completes the proof. □

## A.8 Proof of Proposition 3

*Proof.* Let $X \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$ and $s > 0$. We consider $F_0 \in L_{\mathbb{C}}^2(\mathbb{R}^2)$ as the “low-frequency” function satisfying (34). Again, we introduce $s' := 2ms$ and $X_0 \in l_{\mathbb{C}}^2(\mathbb{Z}^2)$ satisfying (161). Moreover, for any $Y \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$, we denote by $F_Y^{(s')}$ the Shannon interpolation of Y parameterized by $s'$, analogously to (24):

$$ F_Y^{(s')} := \sum_{n \in \mathbb{Z}^2} Y[n] \Phi_n^{(s')}. \qquad (167) $$
where $m, q \in \mathbb{N} \setminus \{0\}$ respectively denote a subsampling factor and the max pooling grid half-size. Analogously to (3.9) for single-channel inputs, we now consider

$$ (5.5) \qquad Y_l^{\max} := U_{m,q,l}^{\max}[\mathbf{W}](\mathbf{X}) \quad \text{and} \quad Y_l^{\text{mod}} := U_{m,l}^{\text{mod}}[\mathbf{W}](\mathbf{X}). $$

Again, in what follows we omit the parameter between square brackets. To apply Theorems 2.9, 3.11, and 4.2 to the current setting on the $l$-th output channel, we need the following hypotheses.

**Hypothesis 5.1 (Monochrome filters).** Let

$$ (5.6) \qquad \widetilde{W}_l := \frac{1}{K} \sum_{k=0}^{K-1} W_{lk} $$

denote the mean kernel of the $l$-th output channel. Then, there exists $\mu_l \in \mathbb{R}^K$ such that

$$ (5.7) \qquad \forall k \in \{0..K-1\}, W_{lk} = \mu_{lk} \widetilde{W}_l. $$

**Hypothesis 5.2 (Gabor-like filters).** There exists a bandwidth $\kappa > 0$ satisfying $\kappa \le \pi/m$ and a frequency vector $\theta_l \in [-\pi, \pi]^2$ such that

$$ (5.8) \qquad \widetilde{W}_l \in \mathcal{J}(\theta_l, \kappa). $$

Note that the bandwidth $\kappa$ is not indexed by $l$, because it shall later be assumed to be shared across the output channels. Then, under Hypothesis 5.1, $Y_l^{\max}$ and $Y_l^{\text{mod}}$ are the outputs of single-channel $\mathbb{R}Max$ and $\mathbb{C}Mod$ operators, as introduced in (1.1) and (1.5):

$$ (5.9) \qquad Y_l^{\max} = U_{m,q}^{\max}[\widetilde{W}_l](X_l^{\text{lum}}) \quad \text{and} \quad Y_l^{\text{mod}} = U_m^{\text{mod}}[\widetilde{W}_l](X_l^{\text{lum}}), $$

where $X_l^{\text{lum}} \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$ ("luminance" image) is defined as the following linear combination:

$$ (5.10) \qquad X_l^{\text{lum}} := \sum_{k=0}^{K-1} \mu_{lk} X_k. $$

The results established for single-channel inputs can therefore be extended to multichannel operators. Specifically, we get the following corollaries to Theorems 2.9, 3.11, and 4.2.

**Corollary 5.3 (Shift invariance of CMod).** For a given output channel $l \in \{0..L-1\}$, we postulate Hypotheses 5.1 and 5.2. Then, for any input image $\mathbf{X} \in (l_{\mathbb{R}}^2(\mathbb{Z}^2))^K$ with finite support and any translation vector $\mathbf{u} \in \mathbb{R}^2$,

$$ (5.11) \qquad \| U_{m,l}^{\text{mod}}(\mathcal{T}_{\mathbf{u}}\mathbf{X}) - U_{m,l}^{\text{mod}}\mathbf{X} \|_2 \le \alpha(\kappa\mathbf{u}) \| U_{m,l}^{\text{mod}}\mathbf{X} \|_2, $$

where $\alpha$ has been defined in (2.14).
On the one hand, Lemma 4 provides (37). On the other hand, we seek a similar result with $X \leftarrow \mathcal{T}_u X$. For this purpose, (164) can be rewritten

$$U_m^{\text{mod}}(\mathcal{T}_u X)[n] = |\mathcal{T}_{s'u'} F_0(s'n)|, \qquad (168)$$

with $u' := u/(2m)$. Furthermore, according to Lemma 4 (35), $F_0 \in \mathcal{V}^{(s')}$. Therefore, Shannon's sampling theorem (Mallat, 2009, Theorem 3.11, p. 81) with $s \leftarrow s'$ yields

$$ \begin{aligned} F_0 &= s' \sum_{n \in \mathbb{Z}^2} F_0(s'n) \Phi_n^{(s')} \\ &= \sum_{n \in \mathbb{Z}^2} X_0[n] \Phi_n^{(s')} = F_{X_0}^{(s')}, \end{aligned} $$

where we have used the notations introduced in (161) and (167). Then, using Lemma 3 with $X \leftarrow X_0$, $u \leftarrow u'$ and $s \leftarrow s'$, we get

$$F_{\mathcal{T}_{u'}X_0}^{(s')} = \mathcal{T}_{s'u'}F_{X_0}^{(s')} = \mathcal{T}_{s'u'}F_0. \qquad (169)$$

Moreover, (26) (from Proposition 2) with $X \leftarrow \mathcal{T}_{u'}X_0$ and $s \leftarrow s'$ becomes

$$\mathcal{T}_{u'}X_0[n] = s' F_{\mathcal{T}_{u'}X_0}^{(s')} (s'n), \qquad (170)$$

and inserting (169) into (170) yields

$$\mathcal{T}_{u'}X_0[n] = s' \mathcal{T}_{s'u'}F_0(s'n). \qquad (171)$$

Therefore, (168) and (171) imply

$$\|U_m^{\text{mod}}(\mathcal{T}_u X)\|_2 = \frac{1}{s'} \|\mathcal{T}_{u'} X_0\|_2. \qquad (172)$$

Moreover, since $F_0 \in \mathcal{V}^{(s')}$, and according to (171), we can use Lemma 2 with $s \leftarrow s'$, $F \leftarrow \mathcal{T}_{s'u'}F_0$ and $X \leftarrow \mathcal{T}_{u'}X_0$. We get

$$\|\mathcal{T}_{u'}X_0\|_2 = \|\mathcal{T}_{s'u'}F_0\|_{L^2} = \|F_0\|_{L^2}, \qquad (173)$$

and plugging (173) into (172) yields

$$\|U_m^{\text{mod}}(\mathcal{T}_u X)\|_2 = \frac{1}{s'} \|F_0\|_{L^2}. \qquad (174)$$

Finally, considering Lemma 4 (37) concludes the proof. $\square$

### A.9 Proof of Proposition 4

*Proof.* Let us write:

$$ \begin{aligned} & (F * \text{Re} \bar{\Psi})(\mathbf{x} + \mathbf{h}) - |(F * \bar{\Psi})(\mathbf{x})| G(\mathbf{x}, \mathbf{h}) \\ &= \text{Re}((F * \bar{\Psi})(\mathbf{x} + \mathbf{h})) - |(F * \bar{\Psi})(\mathbf{x})| \text{Re}(e^{-i\langle\mathbf{v}, \mathbf{h}\rangle} e^{iH(\mathbf{x})}) \\ &= \text{Re}((F * \bar{\Psi})(\mathbf{x} + \mathbf{h})) - \text{Re}(|(F * \bar{\Psi})(\mathbf{x})| e^{iH(\mathbf{x})} e^{-i\langle\mathbf{v}, \mathbf{h}\rangle}) \\ &= \text{Re}((F * \bar{\Psi})(\mathbf{x} + \mathbf{h})) - \text{Re}((F * \bar{\Psi})(\mathbf{x}) e^{-i\langle\mathbf{v}, \mathbf{h}\rangle}) \\ &= \text{Re}((F * \bar{\Psi})(\mathbf{x} + \mathbf{h}) - (F * \bar{\Psi})(\mathbf{x}) e^{-i\langle\mathbf{v}, \mathbf{h}\rangle}). \end{aligned} $$

Therefore,

$$ \begin{aligned} & |(F * \text{Re} \bar{\Psi})(\mathbf{x} + \mathbf{h}) - |(F * \bar{\Psi})(\mathbf{x})| G(\mathbf{x}, \mathbf{h})| \\ & \le |(F * \bar{\Psi})(\mathbf{x} + \mathbf{h}) - (F * \bar{\Psi})(\mathbf{x}) e^{-i\langle\mathbf{v}, \mathbf{h}\rangle}| \\ & = |F_0(\mathbf{x} + \mathbf{h}) e^{-i\langle\mathbf{v}, \mathbf{x}+\mathbf{h}\rangle} - F_0(\mathbf{x}) e^{-i\langle\mathbf{v}, \mathbf{x}+\mathbf{h}\rangle}|, \end{aligned} $$

which yields (44) and concludes the proof. $\square$
**Corollary 5.4 (MSE between CMod and RMax).** As in Corollary 5.3, we postulate *Hypotheses* 5.1 and 5.2. Again, we assume that condition (2.53) is satisfied: $\kappa \le \pi/m$. Besides, we consider $\mathbf{X}$ as a stack of K discrete stochastic processes, and assume *Hypotheses* 3.4, 3.7, and 3.8 with $X \leftarrow X_l^{\text{lum}}$ and $W \leftarrow \tilde{W}_l$. Then,

$$ (5.12) \qquad \mathbb{E}[\tilde{\mathbf{P}}_{\mathbf{X},l}^2] \le (\beta_q(m\kappa) + \gamma_q(m\theta_l))^2, $$

where we have defined the following random variable:

$$ (5.13) \qquad \tilde{\mathbf{P}}_{\mathbf{X},l} := \frac{\|U_{m,l}^{\text{mod}}\mathbf{X} - U_{m,l}^{\text{max}}\mathbf{X}\|_2}{\|U_{m,l}^{\text{mod}}\mathbf{X}\|_2}. $$

**Corollary 5.5 (Shift invariance of RMax).** We assume that the requirements stated in Corollary 5.4 are satisfied. Then, for any translation vector $\mathbf{u} \in \mathbb{R}^2$,

$$ (5.14) \qquad \mathbb{E}[\tilde{\mathbf{R}}_{\mathbf{X}, \mathbf{u}, l}] \le 2 (\beta_q(m\kappa) + \gamma_q(m\theta_l)) + \alpha(\kappa\mathbf{u}), $$

where we have defined the following random variable:

$$ (5.15) \qquad \tilde{\mathbf{R}}_{\mathbf{X}, \mathbf{u}, l} := \frac{\|U_{m,l}^{\text{max}}(\mathcal{T}_{\mathbf{u}}\mathbf{X}) - U_{m,l}^{\text{max}}\mathbf{X}\|_2}{\|U_{m,l}^{\text{mod}}\mathbf{X}\|_2}. $$

**Remark 5.6.** In the above results, we used a translation operator on multichannel tensors, obtained by applying $\mathcal{T}_{\mathbf{u}}$, as defined in (2.34), to each channel $X_k$.

**6. A Case Study Implementing the Dual-Tree Complex Wavelet Packet Transform.** In this section, we experimentally validate the results stated in Theorems 2.9, 3.11, and 4.2. To this end, we consider a fully-deterministic scenario implementing the dual-tree complex wavelet packet transform (DT-CWPT), which exhibit characteristics akin to those observed in the initial convolution layer of freely-trained CNNs such as AlexNet or ResNet. In particular, as stated in subsection 6.1, DT-CWPT achieves subsampled convolutions with oriented band-pass filters tiling the Fourier domain into overlapping square windows. As such, it provides a convenient framework to experimentally validate our theoretical findings in a controlled environment. Then, in subsection 6.2, we build CMod and RMax operators based on DT-CWPT convolution kernels.

**6.1. Main Properties.** In what follows, we outline the principal characteristics of DT-CWPT. A detailed description of the transform itself is provided in Appendix B.1, whereas the results presented hereafter are formally established in Appendices B.2 and B.3.

For a given decomposition depth $J \in \mathbb{N}\setminus\{0\}$, DT-CWPT achieves subsampled convolutions with $4 \times 4^J$ oriented band-pass filters that tile the Fourier domain into overlapping square windows of size

$$ (6.1) \qquad \kappa_J := \pi/m_J, \quad \text{with} \quad m_J := 2^{J-1}. $$

More specifically, considering an input image $X \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$, it produces a set of $4 \times 4^J$ output feature maps

$$ (6.2) \qquad \mathbf{D}^{(J)} := (D_l^{\uparrow(J)}, D_l^{\downarrow(J)}, D_l^{\leftarrow(J)}, D_l^{\rightarrow(J)})_{l \in \{0..4^J-1\}}, $$
## A.10 Proof of Lemma 5

*Proof.* We apply Proposition 4 with $h \leftarrow h_n^{\max}$ and $h \leftarrow h_n^{\prime \max}$, respectively:

$$A_X^{\max}(\mathbf{x}_n) \le \tilde{A}_X(\mathbf{x}_n, h_n^{\max}) + |F_0(\mathbf{x}_n + h_n^{\max}) - F_0(\mathbf{x}_n)|; \quad (175)$$

$$\tilde{A}_X^{\max}(\mathbf{x}_n) \le A_X(\mathbf{x}_n, h_n^{\prime \max}) + |F_0(\mathbf{x}_n + h_n^{\prime \max}) - F_0(\mathbf{x}_n)|. \quad (176)$$

By construction, we have, for any $\mathbf{p} \in \{-q, \dots, q\}^2$,

$$\tilde{A}_X(\mathbf{x}_n, h_p) \le \tilde{A}_X^{\max}(\mathbf{x}_n) \quad \text{and} \quad A_X(\mathbf{x}_n, h_p) \le A_X^{\max}(\mathbf{x}_n). \quad (177)$$

Moreover, by definition, there exists $\mathbf{p}$ and $\mathbf{p}' \in \{-q, \dots, q\}^2$ such that $h_n^{\max} = h_p$ and $h_n^{\prime \max} = h_p'$. Therefore, (175) and (176) yield, respectively,

$$A_X^{\max}(\mathbf{x}_n) \le \tilde{A}_X^{\max}(\mathbf{x}_n) + |F_0(\mathbf{x}_n + h_n^{\max}) - F_0(\mathbf{x}_n)|; \quad (178)$$

$$\tilde{A}_X^{\max}(\mathbf{x}_n) \le A_X^{\max}(\mathbf{x}_n) + |F_0(\mathbf{x}_n + h_n^{\prime \max}) - F_0(\mathbf{x}_n)|, \quad (179)$$

which yields (68) and concludes the proof. $\square$

## A.11 Proof of Proposition 5

*Proof.* Let us write:

$$\begin{aligned} \|U_m^{\text{mod}}X - U_{m,q}^{\text{mod}}X\|_2^2 &= \sum_{n \in \mathbb{Z}^2} (U_m^{\text{mod}}X[n] - U_{m,q}^{\text{mod}}X[n])^2 \\ &= \sum_{n \in \mathbb{Z}^2} \left( U_m^{\text{mod}}X[n] - U_m^{\text{mod}}X[n] \max_{\|\mathbf{p}\|_{\infty} \le q} G_X(\mathbf{x}_n, h_p) \right. \\ & \qquad \left. + U_m^{\text{mod}}X[n] \max_{\|\mathbf{p}\|_{\infty} \le q} G_X(\mathbf{x}_n, h_p) - U_{m,q}^{\text{mod}}X[n] \right)^2 \\ &= \sum_{n \in \mathbb{Z}^2} \left( \delta_{m,q}X[n] + \tilde{A}_X^{\max}(\mathbf{x}_n) - A_X^{\max}(\mathbf{x}_n) \right)^2, \end{aligned}$$

according to (60), (65) and (66). Then, using the triangle inequality, we get

$$\|U_m^{\text{mod}}X - U_{m,q}^{\text{mod}}X\|_2 \le \|\delta_{m,q}X\|_2 + \left( \sum_{n \in \mathbb{Z}^2} \left( \tilde{A}_X^{\max}(\mathbf{x}_n) - A_X^{\max}(\mathbf{x}_n) \right)^2 \right)^{1/2}. \quad (180)$$

Furthermore, Lemma 5 yields

$$\sum_{n \in \mathbb{Z}^2} \left( \tilde{A}_X^{\max}(\mathbf{x}_n) - A_X^{\max}(\mathbf{x}_n) \right)^2 \le \sum_{n \in \mathbb{Z}^2} \max_{\mathbf{h} \in \{\mathbf{h}_n^{\max}, \mathbf{h}_n^{\prime \max}\}} |F_0(\mathbf{x}_n + \mathbf{h}) - F_0(\mathbf{x}_n)|^2 \quad (181)$$

$$\le \sum_{n \in \mathbb{Z}^2} |F_0(\mathbf{x}_n + \mathbf{h}_0) - F_0(\mathbf{x}_n)|^2, \quad (182)$$

according to Hypothesis 1. Next, since (38) is satisfied, we can use Lemma 4 (36) with $h \leftarrow h_0$. We get

$$\begin{aligned} \sum_{n \in \mathbb{Z}^2} \left( \tilde{A}_X^{\max}(\mathbf{x}_n) - A_X^{\max}(\mathbf{x}_n) \right)^2 &\le \frac{1}{4m^2s^2} \|\mathcal{T}_{h_0} F_0 - F_0\|_{L^2}^2 \\ &\le \alpha(\kappa h_0/s)^2 \frac{1}{4m^2s^2} \|F_0\|_{L^2}^2 \quad \text{(acc. to Proposition 1)} \\ &= \alpha(\kappa h_0/s)^2 \|U_m^{\text{mod}}X\|_2^2. \quad \text{(acc. to Lemma 4 (37))} \end{aligned}$$
where each arrow points to the Fourier quadrant where the feature map's energy is con-
centrated. Moreover, as stated in Proposition B.2, for any $l \in \{0..4^J - 1\}$, there exists
$W_l^{\perp(J)} \in l_{\mathbb{C}}^2(\mathbb{Z}^2)$ such that

$$
(6.3) \qquad D_l^{\perp(J)} = (X * \overline{W}_l^{\perp(J)*}) \downarrow 2^J.
$$

An interesting property is that each kernel $W_l^{\perp(J)}$ approximately satisfies

$$
(6.4) \qquad W_l^{\perp(J)} \in \mathcal{J}(\theta_l^{(J)}, \kappa_J)
$$

for a certain characteristic frequency $\theta_l^{(J)} \in [0, \pi]^2$. In other words, it approximately behaves
as a Gabor-like filter in the discrete framework (2.5). Moreover, each kernel corresponds to
a different frequency, thereby covering the top-right quadrant of the Fourier domain. Similar
results can be established for the other three Fourier quadrants. Graphical representations
of $\mathbf{W}^{\perp(J)} := (W_l^{\perp(J)})_{l \in \{0..4^{J-1}\}}$ and $\mathbf{W}^{\perp(J)} := (W_l^{\perp(J)})_{l \in \{0..4^{J-1}\}}$ are provided in Figure 6
with $J = 2$ (Figure 6a, 32 filters) and $J = 3$ (Figure 6b, 128 filters).

The $\mathbb{R}Max$ and $\mathbb{C}Mod$ operators implemented in our experiments respectively satisfy (1.1) and (1.5) with $W \leftarrow W_l^{\perp(J)}$ or $W_l^{\perp(J)}$, and $m \leftarrow m_J$. Note that increasing the decomposition depth $J$, and therefore the subsampling factor $m_J$, results in a decreased Fourier support size $\kappa_J$, therefore matching the condition stated in (2.53) $\kappa \leftarrow \kappa_J$ and $m \leftarrow m_J$.

Remark 6.1. Because X is real-valued, the feature maps $D_l^{\perp(J)}$ and $D_l^{\perp(J)}$ are the respective complex conjugates of $D_l^{\perp(J)}$ and $D_l^{\perp(J)}$, and thus do not need to be explicitly computed. Then, we can easily show that $W_l^{\perp(J)}$ and $W_l^{\perp(J)}$ are also the complex conjugates of $W_l^{\perp(J)}$ and $W_l^{\perp(J)}$, respectively.

**6.2. DT-CWPT-Based RMax and CMod Operators.** According to (6.1), (6.3), and (6.4), we can apply Theorems 2.9, 3.11, and 4.2 to the dual-tree framework. More precisely, for any output channel $l \in \{0..4^J - 1\}$, we consider the following $\mathbb{R}Max$ and $\mathbb{C}Mod$ operators:

$$
(6.5) \qquad U_l^{\max \perp} : X \mapsto \mathrm{MaxPool} \left( \left( X * \mathrm{Re} \, \overline{W}_l^{\perp(J)} \right) \downarrow 2^{J-1} \right);
$$

$$
(6.6) \qquad U_l^{\mathrm{mod} \perp} : X \mapsto \left| \left( X * \overline{W}_l^{\perp(J)} \right) \downarrow 2^J \right|.
$$

Using the notations introduced in (1.5) and (1.1), we have

$$
(6.7) \qquad U_l^{\max \perp} = U_{m_J}^{\max} [W_l^{\perp(J)}] \quad \text{and} \quad U_l^{\mathrm{mod} \perp} := U_{m_J}^{\mathrm{mod}} [W_l^{\perp(J)}],
$$

where we have defined $m_J := 2^{J-1}$. Note that, following Remark 4.4, we have omitted the
grid half-size $q$, which is equal to 1 (max pooling operates on a grid of size $3 \times 3$). Furthermore,
for the sake of brevity, we have omitted the depth $J$ in the above notations.

Remark 6.2. Both $U_l^{\max \perp}$ and $U_l^{\mathrm{mod} \perp}$ are implemented using DT-CWPT with $J$ decomposition stages. However, in (6.5), the subsampling factor is equal to $2^{J-1}$, instead of $2^J$, as stated in (6.3). In order to accommodate this property of $\mathbb{R}Max$ operators, the last stage of
Since, according to Hypothesis 1, $\|h_0\|_2 = \sqrt{2qms}$, it comes that $\|h_0\|_1 = 2qms$. Therefore,

$$ \alpha(\kappa h_0/s)^2 = \frac{\kappa^2 \|h_0\|_1^2}{4s^2} = (qm\kappa)^2, \qquad (183) $$

which yields

$$ \sum_{n \in \mathbb{Z}^2} \left( \tilde{A}_{\mathbf{X}}^{\max}(\mathbf{x}_n) - A_{\mathbf{X}}^{\max}(\mathbf{x}_n) \right)^2 \le \beta_q(m\kappa)^2 \|U_m^{\mathrm{mod}}\mathbf{X}\|_2^2. \qquad (184) $$

Finally, plugging (184) into (180) concludes the proof. □

## A.12 Proof of Proposition 7

*Proof.* We suppose that Hypothesis 3 is satisfied and we consider $\mathbf{x} \in \mathbb{R}^2$. For a given $n \in \mathbb{N} \setminus \{0\}$, we introduce the random variable

$$ \tilde{S}_{\mathbf{X}, n} := \sqrt{\sum_{\|\mathbf{p}\|_{\infty} \le n} \mathbf{M}_{\mathbf{X}}(\mathbf{x}_{\mathbf{p}})^2}. \qquad (185) $$

According to Hypothesis 3, $Z_X(\mathbf{x})$ is jointly independent of $M_X(\mathbf{x}_p)$ for $\mathbf{p} \in \{-n, \dots, n\}^2$. Therefore, by composition, $Z_X(\mathbf{x})$ is also independent of $\tilde{S}_{\mathbf{X}, n}$. Moreover, according to (83) and (111), $\tilde{S}_{\mathbf{X}, n}$ converges almost surely towards $\tilde{S}_{\mathbf{X}}$, which proves independence between $Z_X(\mathbf{x})$ and $\tilde{S}_{\mathbf{X}}$.

Next, we prove conditional independence between $Z_X(\mathbf{x})$ and $M_X(\mathbf{x})$ given $\tilde{S}_{\mathbf{X}}$. According to Hypothesis 3,

$$ (M_X(\mathbf{x}), \tilde{S}_{\mathbf{X}, n}) \perp Z_X(\mathbf{x}), \qquad (186) $$

where $\perp$ stands for independence. This is because $\tilde{S}_{\mathbf{X}, n}$ only depends on a finite number of $M_X(\mathbf{x}_p)$. Therefore,

$$ Z_X(\mathbf{x}) \perp M_X(\mathbf{x}) | \tilde{S}_{\mathbf{X}, n}. \qquad (187) $$

Finally, since $\tilde{S}_{\mathbf{X}, n}$ converges almost surely towards $\tilde{S}_{\mathbf{X}}$, it comes that $Z_X(\mathbf{x})$ and $M_X(\mathbf{x})$ are conditionally independent given $\tilde{S}_{\mathbf{X}}$. □

## A.13 Proof of Theorem 2

*Proof.* We consider $n \in \mathbb{Z}^2$. By construction, $Q_X(\mathbf{x}_n) := 1 - G_X^{\max}(\mathbf{x}_n)$ only depends on $Z_X(\mathbf{x}_n)$. Therefore, under Hypothesis 3, Proposition 7 implies

$$ Q_X(\mathbf{x}_n) \perp M_X(\mathbf{x}_n) | \tilde{S}_X^2 \quad \text{and} \quad Q_X(\mathbf{x}_n) \perp \tilde{S}_X^2. \qquad (188) $$

Additionally, we introduce

$$ \tilde{\Delta}_{\mathbf{X}} := \|\delta_{m,q}X\|_2, \qquad (189) $$

where $\delta_{m,q}X$ is defined in (87). Then, using the linearity of $\mathbb{E}$, we get

$$ \begin{aligned} \mathbb{E}[\tilde{\Delta}_{\mathbf{X}}^2 | \tilde{S}_{\mathbf{X}}^2 = \sigma] &= \sum_{n \in \mathbb{Z}^2} \mathbb{E}[\delta_{m,q}[n]^2 | \tilde{S}_{\mathbf{X}}^2 = \sigma] \\ &= \sum_{n \in \mathbb{Z}^2} \mathbb{E}[U_{m,l}^{\mathrm{mod}} X[n]^2 (1 - G_{\mathbf{X}}^{\max}(\mathbf{x}_n))^2 | \tilde{S}_{\mathbf{X}}^2 = \sigma] \\ &= \sum_{n \in \mathbb{Z}^2} \mathbb{E}[M_X(\mathbf{x}_n)^2 Q_X(\mathbf{x}_n)^2 | \tilde{S}_{\mathbf{X}}^2 = \sigma] \quad (\text{acc. to (83) and (88)}) \\ &= \sum_{n \in \mathbb{Z}^2} \mathbb{E}[M_X(\mathbf{x}_n)^2 | \tilde{S}_{\mathbf{X}}^2 = \sigma] \mathbb{E}[Q_X(\mathbf{x}_n)^2] \quad (\text{acc. to (188)}). \end{aligned} $$
<figure><img src="image_14.png" /><figcaption>(a) $J = 2$</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>(b) $J = 3$</figcaption></figure>

Figure 6. Real part of the convolution kernels $W^{\nwarrow(J)}$, $W^{\to(J)}$, with $J = 2$ (32 filters, $m_J = 2$) and $J = 3$ (128 filters, $m_J = 4$), respectively. The kernels have been computed using Q-shift orthogonal QMFs of length 10 [19]. The kernels have been respectively cropped to size 11 × 11 and 19 × 19, for the sake of legibility. Note that the filters displayed in (a) and (b) share similarities with those found in, respectively, ResNet ($m = 2$) and AlexNet ($m = 4$), after training with ImageNet.

DT-CWPT decomposition is carried out without subsampling, resulting in higher redundancy. This is similar to the concept of stationary wavelet transform as described by Nason and Silverman [31]. Furthermore, only the real component of the wavelet feature maps is preserved. On the other hand, $U_l^{\text{mod}\nearrow}$ implements a fully-decimated wavelet packet transform, and keeps both real and imaginary parts. Figure 7 illustrates these technical details.

**6.3. Experiments and Results.** We implemented the $\mathbb{R}$Max and $\mathbb{C}$Mod operators $U_l^{\text{max}\nearrow}$ and $U_l^{\text{mod}\nearrow}$, as introduced in (6.5) and (6.6), with both $J = 2$ and 3 stages of wavelet packet decomposition. To cover the whole frequency plane, we also implemented similar operators, denoted by $U_l^{\text{max}\searrow}$ and $U_l^{\text{mod}\searrow}$. They are associated with the convolution filters $W_l^{\nwarrow(J)}$, introduced in Proposition B.2, with energy being located in the bottom-right quadrant. However, as explained in Remark 6.1, we did not need to deal with the two other quadrants (negative $x$-values). Using the validation set of ImageNet-1K [39], ($N := 50000$ images), we measured the mean discrepancy between $\mathbb{R}$Max and $\mathbb{C}$Mod outputs, and evaluated the shift invariance
Using the monotone convergence theorem, we get

$$ \mathbb{E}[\tilde{\Delta}_X^2 | \tilde{S}_X^2 = \sigma] = \mathbb{E}\left[\sum_{n \in \mathbb{Z}^2} \mathbf{M}_X(x_n)^2 \middle| \tilde{S}_X^2 = \sigma\right] \mathbb{E}[\mathbf{Q}_X(x_n)^2]. \quad (190) $$

According to (83) and (111), we have

$$ \sum_{n \in \mathbb{Z}^2} \mathbf{M}_X(x_n)^2 = \|U_m^{\text{mod}}X\|_2^2 = \tilde{S}_X^2. \quad (191) $$

Therefore, we get

$$ \begin{aligned} \mathbb{E}[\tilde{\Delta}_X^2 | \tilde{S}_X^2 = \sigma] &= \mathbb{E}[\tilde{S}_X^2 | \tilde{S}_X^2 = \sigma] \mathbb{E}[\mathbf{Q}_X(x_n)^2] \\ &= \sigma \cdot \mathbb{E}[\mathbf{Q}_X(x_n)^2]. \end{aligned} $$

Under Hypothesis 2, Proposition 6 yields

$$ \mathbb{E}[\tilde{\Delta}_X^2 | \tilde{S}_X^2 = \sigma] = \sigma \cdot \gamma_q(m\theta)^2. \quad (192) $$

Moreover, we can reformulate $\tilde{\mathbf{Q}}_X$ such as defined in (89): $\tilde{\mathbf{Q}}_X = \tilde{\Delta}_X/\tilde{S}_X$. Therefore,

$$ \mathbb{E}[\tilde{\mathbf{Q}}_X^2 | \tilde{S}_X^2 = \sigma] = \frac{1}{\sigma} \mathbb{E}[\tilde{\Delta}_X^2 | \tilde{S}_X^2 = \sigma] = \gamma_q(m\theta)^2. \quad (193) $$

According to (193), the conditional expected value of $\tilde{\mathbf{Q}}_X^2$ remains the same whatever the outcome of $\tilde{S}_X^2$. Thus, the law of total expectation states that

$$ \mathbb{E}[\tilde{\mathbf{Q}}_X^2] = \mathbb{E}[\mathbb{E}[\tilde{\mathbf{Q}}_X^2 | \tilde{S}_X^2]] = \gamma_q(m\theta)^2. \quad (194) $$

Since we have assumed Hypothesis 1, we can apply Proposition 5. Using the definition of $\tilde{\mathbf{P}}_X$ (86) and $\tilde{\mathbf{Q}}_X$ (89), we get

$$ \tilde{\mathbf{P}}_X \le \tilde{\mathbf{Q}}_X + \beta_q(m\kappa). \quad (195) $$

Then,

$$ \mathbb{E}[\tilde{\mathbf{P}}_X^2] \le \mathbb{E}[\tilde{\mathbf{Q}}_X^2] + 2\beta_q(m\kappa) \mathbb{E}[\tilde{\mathbf{Q}}_X] + \beta_q(m\kappa)^2. \quad (196) $$

According to Jensen's inequality,

$$ \mathbb{E}[\tilde{\mathbf{Q}}_X] \le \sqrt{\mathbb{E}[\tilde{\mathbf{Q}}_X^2]} = \gamma_q(m\theta). \quad (197) $$

Thus,

$$ \mathbb{E}[\tilde{\mathbf{P}}_X^2] \le \gamma_q(m\theta)^2 + 2\beta_q(m\kappa)\gamma_q(m\theta) + \beta_q(m\kappa)^2, \quad (198) $$

which yields (112). □

### A.14 Proof of Lemma 6

*Proof.* First, we show that, for any $x \in \mathbb{R}^2$,

$$ \mathbf{M}_{\mathcal{T}_u X}(x) = \mathcal{T}_{su} \mathbf{M}_X(x); \quad (199) $$

$$ \mathbf{Z}_{\mathcal{T}_u X}(x) = \mathcal{T}_{su} \mathbf{Z}_X(x). \quad (200) $$

According to Lemma 3, and since the convolution product commutes with translations, we have

$$ (F_{\mathcal{T}_u X} * \bar{\Psi}_W)(x) = \mathcal{T}_{su}(F_X * \bar{\Psi}_W)(x). \quad (201) $$

Then, using (81), the above expression becomes

$$ \mathbf{M}_{\mathcal{T}_u X}(x) \times \mathbf{Z}_{\mathcal{T}_u X}(x) = (\mathcal{T}_{su} \mathbf{M}_X)(x) \times (\mathcal{T}_{su} \mathbf{Z}_X)(x), \quad (202) $$

which yields (199) and (200), by uniqueness of the magnitude-phase decomposition. Finally, we remind that

$$ \mathcal{T}_{su} \mathbf{M}_X(x) = \mathbf{M}_X(x - su) \quad \text{and} \quad \mathcal{T}_{su} \mathbf{Z}_X(x) = \mathbf{Z}_X(x - su). \quad (203) $$

Then, considering hypotheses Hypotheses 2 and 3 with $x \leftarrow x - su$ yields the result. □
<figure><img src="image_16.png" /><figcaption>Figure 7. Detailed illustration of the RMax (a) and CMod (b) operators based on DT-CWPT, with $J = 3$ decomposition stages. The numbers between modules correspond to the number of feature maps, height and width. The orange modules represent subsampled convolutions using one of the four 2D filter banks $G^{[0-3]}$, such as introduced in (B.1). The FB index is indicated between square brackets. The RMax model (a) only computes the real part of the dual-tree coefficients, and the last stage of decomposition is performed without subsampling (red modules). Additionally, the blue modules represent linear combinations of feature maps such as described in (B.6).</figcaption></figure>

of both models. Dual-tree decompositions have been performed with Q-shift orthogonal filters of length 10 [19].

**6.3.1. MSE between RMax and CMod.** Each image $n \in \{0..N-1\}$ in the dataset was converted to grayscale, from which a center crop of size $224 \times 224$ was extracted. We denote by $X_n \in l_R^2(Z^2)$ the resulting input feature map. For any $l \in \{0..4^J-1\}$, we denote by

$$ (6.8) \qquad Y_{nl}^{\max \nearrow} := U_l^{\max \nearrow}(X_n) \quad \text{and} \quad Y_{nl}^{\text{mod} \nearrow} := U_l^{\text{mod} \nearrow}(X_n) $$

the outputs of the $l$-th RMax and CMod operators as defined in (6.5) and (6.6), respectively. We adopt similar notations for the bottom-right Fourier quadrant. Then, the normalized mean squared error between $Y_{nl}^{\max \nearrow}$ and $Y_{nl}^{\text{mod} \nearrow}$ was computed. It is defined by the square of

$$ (6.9) \qquad \rho_{nl}^{\nearrow} := \| Y_{nl}^{\text{mod} \nearrow} - Y_{nl}^{\max \nearrow} \|_2 / \| Y_{nl}^{\text{mod} \nearrow} \|_2. $$

Finally, for each output channel $l$, an empirical estimate for $E[\tilde{P}_X^2]$, introduced in (3.58), was obtained by averaging $\rho_{nl}^{\nearrow 2}$ over the whole dataset. We denote by $\tilde{\rho}_l^{\nearrow 2}$ the corresponding quantity.

Since $U_l^{\max \nearrow}$ and $U_l^{\text{mod} \nearrow}$ are parameterized by $W_l^{\nearrow(J)}$, it follows that $\tilde{\rho}_l^{\nearrow 2}$ depends on the filter's characteristic frequency $\theta_l^{(J)}$ (6.4). According to Proposition B.4, these frequencies form a regular grid in the top-right quadrant of Fourier domain. This provides a visual representation of $\tilde{\rho}_l^{\nearrow 2}$, as shown in Figure 8. This figure also displays $\tilde{\rho}_l^{\nearrow 2}$, corresponding to
### A.15 Proof of Theorem 3

*Proof.* Using the triangle inequality, we compute

$$ \|U_{m,q}^{\max}(\mathcal{T}_u X) - U_{m,q}^{\max} X\|_2 \le \|U_m^{\mathrm{mod}}(\mathcal{T}_u X)\|_2 \tilde{\mathcal{P}}_{\mathcal{T}_u X} + \|U_m^{\mathrm{mod}} X\|_2 \tilde{\mathcal{P}}_X + \|U_m^{\mathrm{mod}}(\mathcal{T}_u X) - U_m^{\mathrm{mod}} X\|_2, \quad (204) $$

where $\tilde{\mathcal{P}}_X$ and $\tilde{\mathcal{P}}_{\mathcal{T}_u X}$ are defined in (86). According to (38), we can apply Proposition 3 on the first term of (204):

$$ \|U_m^{\mathrm{mod}}(\mathcal{T}_u X)\|_2 = \|U_m^{\mathrm{mod}} X\|_2. \quad (205) $$

Moreover, we can apply Theorem 1 to the third term of (204):

$$ \|U_m^{\mathrm{mod}}(\mathcal{T}_u X) - U_m^{\mathrm{mod}} X\|_2 \le \alpha(\kappa u) \|U_m^{\mathrm{mod}} X\|_2. \quad (206) $$

We therefore get

$$ \|U_{m,q}^{\max}(\mathcal{T}_u X) - U_{m,q}^{\max} X\|_2 \le [\tilde{\mathcal{P}}_{\mathcal{T}_u X} + \tilde{\mathcal{P}}_X + \alpha(\kappa u)] \|U_m^{\mathrm{mod}} X\|_2. \quad (207) $$

Then, by linearity of $\mathbb{E}$, we get

$$ \mathbb{E}[\tilde{\mathcal{R}}_{X, u}] \le \mathbb{E}[\tilde{\mathcal{P}}_{\mathcal{T}_u X}] + \mathbb{E}[\tilde{\mathcal{P}}_X] + \alpha(\kappa u), \quad (208) $$

where $\tilde{\mathcal{R}}_{X, u}$ has been introduced in (113).

For any stochastic process $X'$ satisfying Hypotheses 2 and 3, Theorem 2 and Jensen's inequality yield:

$$ \mathbb{E}[\tilde{\mathcal{P}}_{X'}] \le \beta_q(m\kappa) + \gamma_q(m\theta). \quad (209) $$

According to Lemma 6, Hypotheses 2 and 3 are also satisfied for $X \leftarrow \mathcal{T}_u X$. Therefore, (209) is valid for both $X' \leftarrow X$ and $X' \leftarrow \mathcal{T}_u X$, and plugging it into (208) concludes the proof. □

## B Appendix – Theoretical Foundations for our Hypotheses

In this section, we provide theoretical arguments for justifying Hypotheses 2 and 3. As will be discussed, these hypotheses rely on some degree of shift invariance for input images, which implies the notions of stationarity and phase-shift-equivariance for stochastic processes.

### B.1 Stationary and Local Stationarity

Given $n \in \mathbb{N} \setminus \{0\}$, we define n-th order stationarity of a given stochastic process F as stated by Park & Park (2018, p. 152): For any $n' \le n$, $(x_0, \dots, x_{n'-1}) \in (\mathbb{R}^2)^{n'}$ and $h \in \mathbb{R}^2$, the joint distribution of $(F(x_0), \dots, F(x_{n'-1}))$ is identical to the one of $(\mathcal{T}_h F(x_0), \dots, \mathcal{T}_h F(x_{n'-1}))$. Additionally, strict-sense stationarity is defined as n-th order stationarity for any $n \in \mathbb{N} \setminus \{0\}$.

Strict-sense stationarity suggests that any translated version of a given image is equally likely. However, this property is seldom fully achieved for images (Tygert et al., 2016). First, X has fixed boundaries. Consequently, any realization of $F_X(x)$ quickly vanishes as $\|x\|$ tends to $\infty$. Furthermore, depending on which category the image belongs to, the pixel distribution is likely to vary across various regions. For instance, we can expect the main subject to be located at the center of the image. We refer readers to Torralba & Oliva (2003) for more details on statistical properties of images from natural versus man-made objects. For this reason, we introduce a notion of local stationarity as follows.

**Definition 1.** Given $\tau \ge 0$, a stochastic process F is $\tau$-locally stationary to the n-th order if, for any $0 < n'' \le n' \le n$, any $(x_0, \dots, x_{n'-1}) \in (\mathbb{R}^2)^{n'}$, any displacement vector $h \in \mathbb{R}^2$, and any pair of measurable sets $\mathcal{X}, \mathfrak{Y} \subset \mathbb{R}^{n''} \times \mathbb{R}^{n'-n''}$,

$$ \left| \mathbb{P} \left\{ \left( \mathcal{T}_h F(x_0), \dots, \mathcal{T}_h F(x_{n''-1}) \right)^\top \in \mathcal{X} \mid \left( \mathcal{T}_h F(x_{n''}), \dots, \mathcal{T}_h F(x_{n'-1}) \right)^\top \in \mathfrak{Y} \right\} - \mathbb{P} \left\{ \left( F(x_0), \dots, F(x_{n''-1}) \right)^\top \in \mathcal{X} \mid \left( F(x_{n''}), \dots, F(x_{n'-1}) \right)^\top \in \mathfrak{Y} \right\} \right| \le \tau \|h\|_2. \quad (210) $$
<figure><img src="image_17.png" /></figure>

Figure 8. Empirical estimates of the normalized mean squared error between $\mathbb{R}Max$ and $\mathbb{C}Mod$ outputs, computed on ImageNet-1K (validation set). For each channel $l \in \{0..4^J - 1\}$, $\tilde{\rho}_l^{\max\perp 2}$ is plotted as a grayscale pixel centered in $\theta_l^{(J)}$ such as introduced in (6.4) (top-right quadrant). Similarly, $\tilde{\rho}_l^{\perp 2}$ is plotted in the bottom-right quadrant. Finally, the bottom- and top-left quadrants ($\tilde{\rho}_l^{\perp 2}$ and $\tilde{\rho}_l^{\max\perp 2}$) are simply obtained by symmetrizing the figures. Since the subsampling factor $m_J$ is equal to $2^{J-1}$, these experimental results can be compared with the left and right parts of Figure 5. Note that the low-pass filters have been discarded because they are outside the scope of this study.

<figure><img src="image_18.png" /></figure>

Figure 9. Shift invariance of $\mathbb{R}Max$ and $\mathbb{C}Mod$ outputs, computed on ImageNet 2012 (validation set). For each $l \in \{0..4^J - 1\}$, $\tilde{\rho}_l^{\max\perp}$ (Figure 9a) and $\tilde{\rho}_l^{\text{mod}\perp}$ (Figure 9b) are plotted by applying the same procedure as in Figure 8.
Moreover, $\mathbb{F}$ is strict-sense $\tau$-locally stationary if this property is satisfied for any $n \in \mathbb{N}$.

**Remark 10.** The special case where $\tau = 0$ corresponds to the standard definition of n-th order stationarity. Additionally, the use of conditional probabilities is essential for defining local stationarity, as it helps prevent instabilities that may arise when conditioning on low-probability events.

In what follows, we consider the following narrowband stochastic process:

$$F_{1,X} : x \mapsto (F_X * \bar{\Psi}_W)(x). \qquad (211)$$

We assume that $F_X$, and therefore $F_{1,X}$, is nearly shift-invariant for displacement vectors that are much smaller than the image “characteristic” size in the continuous domain, which is equal to $sN$, where, as a reminder, $N \in \mathbb{N}$ denotes the support size of input images and $s > 0$ denotes the sampling interval. More formally, $F_{1,X}$ is assumed to be strict-sense $\tau$-locally stationary with

$$\tau := \frac{1}{sN}. \qquad (212)$$

## B.2 Translations and Phase Shifts

We consider the following stochastic processes:

$$F_{0,X} : x \mapsto (F_X * \bar{\Psi}_W)(x) e^{i\langle\nu, x\rangle} \quad \text{and} \quad F_{1,X} : x \mapsto (F_X * \bar{\Psi}_W)(x). \qquad (213)$$

To justify our hypotheses, we also need to characterize $\mathcal{T}_h F_{1,X}$ as a function of $F_{1,X}$. Having:

$$\mathcal{T}_h F_{1,X}(x) = \mathcal{T}_h F_{0,X}(x) e^{-i\langle\nu, x\rangle} e^{i\langle\nu, h\rangle}, \qquad (214)$$

we get

$$|\mathcal{T}_h F_{1,X}(x) - F_{1,X}(x) e^{i\langle\nu, h\rangle}| = |\mathcal{T}_h F_{0,X}(x) - F_{0,X}(x)|. \qquad (215)$$

According to Lemma 1, the support of $\hat{F}_{0,X}$ is contained within the ball $B_\infty(\frac{\kappa}{2s})$. Intuitively, we can define a “minimal wavelength” $\lambda := 2\pi s/\kappa$ such that, if $\|h\|_2 \ll \lambda$, we can approximate $\mathcal{T}_h F_{0,X}(x) \approx F_{0,X}(x)$, and therefore

$$\mathcal{T}_h F_{1,X}(x) \approx F_{1,X}(x) e^{i\langle\nu, h\rangle}. \qquad (216)$$

If the two terms were strictly identical for any $x \in \mathbb{R}^2$, we would get, for any $n \in \mathbb{N}$, $(x_0, \dots, x_n) \in (\mathbb{R}^2)^n$, and any measurable set $\mathcal{X} \subset \mathbb{R}^n$,

$$(F_{1,X}(x_0), \dots, F_{1,X}(x_{n-1}))^\top \in \mathfrak{A} \iff (\mathcal{T}_h F_{1,X}(x_0), \dots, \mathcal{T}_h F_{1,X}(x_{n-1}))^\top \in e^{i\langle\nu, h\rangle}\mathfrak{A}, \qquad (217)$$

and therefore,

$$\mathbb{P}\{(F_{1,X}(x_0), \dots, F_{1,X}(x_{n-1}))^\top \in \mathfrak{A}\} = \mathbb{P}\{(\mathcal{T}_h F_{1,X}(x_0), \dots, \mathcal{T}_h F_{1,X}(x_{n-1}))^\top \in e^{i\langle\nu, h\rangle}\mathfrak{A}\}. \qquad (218)$$

Instead, we relax the above equality (218) and introduce the following definition.

**Definition 2.** Given $\tau \ge 0$, a stochastic process $\mathbb{F}$ is $\tau$-locally phase-shift-equivariant to the n-th order with respect to the frequency vector $\nu \in \mathbb{R}^2$ if, for any $0 < n'' \le n' \le n$, any $(x_0, \dots, x_{n'-1}) \in (\mathbb{R}^2)^{n'}$, any displacement vector $h \in \mathbb{R}^2$, and any pair of measurable sets $\mathcal{X}, \mathfrak{Y} \subset \mathbb{R}^{n''} \times \mathbb{R}^{n'-n''}$,

$$\left| \mathbb{P} \left\{ \left( \mathcal{T}_h F(x_0), \dots, \mathcal{T}_h F(x_{n''-1}) \right)^\top \in e^{i\langle\nu, h\rangle} \mathcal{X} \middle| \left( \mathcal{T}_h F(x_{n''}), \dots, \mathcal{T}_h F(x_{n'-1}) \right)^\top \in e^{i\langle\nu, h\rangle} \mathfrak{Y} \right\} - \mathbb{P} \left\{ \left( F(x_0), \dots, F(x_{n''-1}) \right)^\top \in \mathcal{X} \middle| \left( F(x_{n''}), \dots, F(x_{n'-1}) \right)^\top \in \mathfrak{Y} \right\} \right| \le \tau \|h\|_2. \qquad (219)$$

Moreover, $\mathbb{F}$ is strict-sense $\tau$-locally phase-shift-equivariant if this property is satisfied for any $n \in \mathbb{N}$.

In what follows, we will assume that the stochastic process $F_{1,X}$ is nearly phase-shift-equivariant for displacement vectors that are much smaller than the minimal wavelength $\lambda$. More formally, $F_{1,X}$ is assumed to be strict-sense $\tau'$-locally phase-shift-equivariant with

$$\tau' := 1/\lambda = \frac{\kappa}{2\pi s}. \qquad (220)$$
the bottom-right quadrant. The half-plane of negative x-values has simply been symmetrized, following Remark 6.1. We can observe a regular pattern of dark spots. More precisely, high discrepancies between max pooling and modulus seem to occur when the energy of $W_l^{\lambda^{(J)}}$ or $W_l^{\lambda^{(J)}}$ overlaps a dark region of Figure 5. This result corroborates Theorem 4.2, which states that high discrepancies are expected for certain pathological frequencies, due to the search for a maximum value over a discrete grid.

**6.3.2. Shift invariance.** For each input image previously converted to grayscale, two crops of size 224 × 224 were extracted, such that the corresponding sequences $X_n$ and $X'_n$ are shifted by one pixel along the x-axis. From these inputs, the following quantity was then computed:

$$ (6.10) \qquad \rho_{nl}^{\max\lambda} := \|Y_{nl}^{\max'\lambda} - Y_{nl}^{\max\lambda}\|_2 / \|Y_{nl}^{\mathrm{mod}\lambda}\|_2, $$

where $Y_{nl}^{\max'\lambda}$ satisfies (6.7) with $X_n \leftarrow X'_n$. Finally, for each output channel $l \in \{0..4^J - 1\}$, an empirical estimate for $\mathbb{E}[\hat{R}_{X,u}]$, satisfying (4.6) with $u = (1, 0)^T$, was obtained by averaging $\rho_{nl}^{\max\lambda}$ over the whole dataset. We denote by $\tilde{\rho}_l^{\max\lambda}$ the corresponding quantity. We point out that shift invariance is measured relatively to the norm of the CMod output, as explained in Remark 4.3.

On the other hand, the same procedure was applied to the CMod operators:

$$ (6.11) \qquad \rho_{nl}^{\mathrm{mod}\lambda} := \|Y_{nl}^{\mathrm{mod}'\lambda} - Y_{nl}^{\mathrm{mod}\lambda}\|_2 / \|Y_{nl}^{\mathrm{mod}\lambda}\|_2, $$

and $\tilde{\rho}_l^{\mathrm{mod}\lambda}$ was obtained as before by averaging $\rho_{nl}^{\mathrm{mod}\lambda}$ over the whole dataset.

A visual representation of $\tilde{\rho}_l^{\max\lambda}$ and $\tilde{\rho}_l^{\mathrm{mod}\lambda}$ are provided in Figure 9 (as well as the other Fourier quadrants). Two observations can be drawn here. (1) When the filter is horizontally oriented, the corresponding output is highly stable with respect to horizontal shifts. This can be explained by noticing that such kernels perform low-pass filtering along the x-axis. The exact transposed phenomenon occurs for vertical shifts. (2) Elsewhere, we observe that high discrepancies between $\mathbb{R}$Max and CMod outputs (Figure 8) are correlated with shift instability of $\mathbb{R}$Max (Figure 9, top). This is in line with (3.87) and (4.7) in Theorems 3.11 and 4.2. Note that CMod outputs are nearly shift invariant regardless the characteristic frequency $\theta_l^{(J)}$ (Figure 9, bottom), as predicted by Theorem 2.9 (2.54).

**7. Conclusion.** In this paper, we explored the shift invariance properties captured by the max pooling operator, when applied on top of a convolution layer with Gabor-like kernels. We established a validity domain for near-shift invariance and confirmed our predictions through an experimental setting based on the dual-tree complex wavelet packet transform. Our results indicate that the CMod operator can serve as a stable proxy for $\mathbb{R}$Max, extracting comparable features, except for certain filter frequencies, for which potential degeneracies can arise after max pooling. This suggests a promising approach for improving shift invariance in CNNs while preserving high-frequency information. This is the main focus of [25], in which we apply these principles to real-life architectures.

A link was missing between real- and complex-valued convolutions in CNNs. By comparing the outputs of CMod and $\mathbb{R}$Max operators, we established a connection between these two worlds, creating opportunities for extensions of the results obtained for complex wavelet
## B.3 Justification for Hypothesis 2

We then consider the following proposition, which states that the probability measure of $Z_X(x)$ is nearly invariant with respect to phase shifts.

**Proposition 8.** We assume that $F_{1,X}$ is $\tau$-locally stationary (Definition 1) and $\tau$'-locally phase-shift-equivariant with respect to $\nu$ (Definition 2), both to the first order. Then, for any measurable set $\mathfrak{A} \subset S^1$,

$$ \forall \omega \in [0, 2\pi], \quad |\mu(\mathfrak{A}) - \mu(e^{i\omega\mathfrak{A}})| \le 2\pi(\tau + \tau')/\|\nu\|_2. \quad (221) $$

where $\mu: \mathfrak{A} \mapsto \mathbb{P}\{Z_X(x) \in \mathfrak{A}\}$ denotes the probability measure of $Z_X(x)$.

*Proof.* Let $\omega \in [0, 2\pi]$. We compute:

$$ \begin{aligned} |\mu(\mathfrak{A}) - \mu(e^{i\omega\mathfrak{A}})| &= |\mathbb{P}\{Z_X(x) \in \mathfrak{A}\} - \mathbb{P}\{Z_X(x) \in e^{i\omega\mathfrak{A}}\}| \\ &= |\mathbb{P}\{Z_X(x) \in \mathfrak{A}\} - \mathbb{P}\{Z_X(x) \in e^{i\langle\nu, h\rangle\mathfrak{A}}\}|, \end{aligned} $$

where we have denoted $h := \omega\nu / \|\nu\|_2^2$. Then, using the triangle inequality,

$$ \begin{aligned} |\mu(\mathfrak{A}) - \mu(e^{i\omega\mathfrak{A}})| &\le |\mathbb{P}\{Z_X(x) \in \mathfrak{A}\} - \mathbb{P}\{\mathcal{T}_h Z_X(x) \in e^{i\langle\nu, h\rangle}\mathfrak{A}\}| \\ &\quad + |\mathbb{P}\{Z_X(x) \in e^{i\langle\nu, h\rangle}\mathfrak{A}\} - \mathbb{P}\{\mathcal{T}_h Z_X(x) \in e^{i\langle\nu, h\rangle}\mathfrak{A}\}| \\ &= |\mathbb{P}\{F_{1,X}(x) \in u^{-1}(\mathfrak{A})\} - \mathbb{P}\{\mathcal{T}_h F_{1,X}(x) \in u^{-1}(e^{i\langle\nu, h\rangle}\mathfrak{A})\}| \\ &\quad + |\mathbb{P}\{F_{1,X}(x) \in u^{-1}(e^{i\langle\nu, h\rangle}\mathfrak{A})\} - \mathbb{P}\{\mathcal{T}_h F_{1,X}(x) \in u^{-1}(e^{i\langle\nu, h\rangle}\mathfrak{A})\}|, \end{aligned} $$

where we have denoted $u: z \mapsto e^{i\angle(z)}$. We can easily show that $u^{-1}(e^{i\langle\nu, h\rangle}\mathfrak{A}) = e^{i\langle\nu, h\rangle}u^{-1}(\mathfrak{A})$. Therefore, the above expression can be re-written:

$$ \begin{aligned} |\mu(\mathfrak{A}) - \mu(e^{i\omega\mathfrak{A}})| &\le |\mathbb{P}\{F_{1,X}(x) \in \mathfrak{X}\} - \mathbb{P}\{\mathcal{T}_h F_{1,X}(x) \in e^{i\langle\nu, h\rangle}\mathfrak{X}\}| \\ &\quad + |\mathbb{P}\{F_{1,X}(x) \in \mathfrak{X}'\} - \mathbb{P}\{\mathcal{T}_h F_{1,X}(x) \in \mathfrak{X}'\}|, \quad (222) \end{aligned} $$

where we have denoted $\mathfrak{X} := u^{-1}(\mathfrak{A})$ and $\mathfrak{X}' := u^{-1}(e^{i\langle\nu, h\rangle}\mathfrak{A})$. Then, by applying (210) and (219) with $n' = 1$ to (222), we get:

$$ |\mu(\mathfrak{A}) - \mu(e^{i\omega\mathfrak{A}})| \le (\tau + \tau') \|h\|_2. \quad (223) $$

Finally, using the definition of $h$ and bounding $\omega$ by $2\pi$ yields the result. $\square$

If we replace $\tau$ and $\tau'$ in (221) by their respective values in (212) and (220), we get,

$$ \forall \omega \in [0, 2\pi], \quad |\mu(\mathfrak{A}) - \mu(e^{i\omega\mathfrak{A}})| \le \frac{2\pi}{sN} \times \|\nu\|_2^{-1} + \frac{\kappa}{s} \times \|\nu\|_2^{-1}. \quad (224) $$

We recall that $\nu := \theta/s$. Then, by applying the constraints stated in (84) and (85), we get

$$ |\mu(\mathfrak{A}) - \mu(e^{i\omega\mathfrak{A}})| \ll 1. \quad (225) $$

Therefore, $\mu$ is almost invariant to phase shifts. Since the only probability measure satisfying the phase-shift invariance property is the uniform probability measure,<fnref n="3" /> we deduce that $Z_X(x)$ follows a near-uniform distribution on $S^1$. For the sake of simplicity, in Hypothesis 2 we have assumed a strictly-uniform distribution.

<footnote n="3">³Any probability measure defined on $S^1$ is a Radon measure. Therefore, according to Haar's theorem (Halmos, 2013), there exists a unique probability measure on $S^1$ satisfying the phase-shift invariance property, and it turns out that the uniform probability measure is one such candidate.</footnote>
transform. To paraphrase Tygert et al. [47], the correspondence between standard real-valued CNNs (using max pooling) and complex wavelets is no longer “just a vague analogy.”

## Appendix A. Theoretical Foundations for our Hypotheses.

In this section, we provide theoretical arguments for justifying Hypotheses 3.7 and 3.8. Given $n \in \mathbb{N} \setminus \{0\}$, we define n-th order stationarity of a given stochastic process F as stated by Park et al. [34, p. 152]: for any $n' \in \{0..n-1\}$, $(x_1, ..., x_{n'}) \in (\mathbb{R}^2)^{n'}$ and $h \in \mathbb{R}^2$, the joint distribution of $(F(x_1), ..., F(x_{n'}))$ is identical to the one of $(F(x_1+h), ..., F(x_{n'}+h))$. Besides, strict-sense stationarity is defined as n-th order stationarity for any $n \in \mathbb{N} \setminus \{0\}$.

We recall that $\nu := \theta/s$. We then state the following results.

**Proposition A.1.** We assume that $F_X$ is first-order stationary. If, for any $x \in \mathbb{R}^2$ and any $h \in B_2(2\pi/\|\nu\|_2)$,

$$ (A.1) \qquad (\mathcal{T}_h F_X * \bar{\Psi}_W)(x) = e^{i\langle\nu, h\rangle} (F_X * \bar{\Psi}_W)(x), $$

then Hypothesis 3.7 is satisfied.

*Proof.* Let $x \in \mathbb{R}^2$. By design (see Remark 3.6), $Z_X(x)$ follows a uniform conditional probability distribution on $\mathbb{S}^1$, given $M_X(x) = 0$. In any other cases, we show that the conditional probability measure of $Z_X(x)$ given $M_X(x) > 0$ is invariant with respect to phase shifts, and is therefore equal to the uniform probability measure on $\mathbb{S}^1$. Specifically, we show that, for any measurable set $\mathfrak{A} \subset \mathbb{S}^1$,

$$ (A.2) \qquad \forall \omega \in [0, 2\pi], \mu(\mathfrak{A}) = \mu(e^{i\omega\mathfrak{A}}), $$

where we have denoted

$$ (A.3) \qquad \mu : \mathfrak{A} \mapsto \mathbb{P}\{Z_X(x) \in \mathfrak{A} \mid M_X(x) > 0\}. $$

Let $h \in B_2(2\pi/\|\nu\|_2)$. According to (A.1), and assuming $M_X(x) > 0$, we get

$$ (A.4) \qquad Z_X(x) \in \mathfrak{A} \iff \mathcal{T}_h Z_X(x) \in e^{i\langle\nu, h\rangle}\mathfrak{A}. $$

Therefore,

$$ (A.5) \qquad \mathbb{P}\{Z_X(x) \in \mathfrak{A} \mid M_X(x) > 0\} = \mathbb{P}\{\mathcal{T}_h Z_X(x) \in e^{i\langle\nu, h\rangle}\mathfrak{A} \mid M_X(x) > 0\}. $$

Since $F_X$ is first-order stationary, $Z_X(x)$ and $\mathcal{T}_h Z_X(x)$ have the same conditional probability distribution given $M_X(x) > 0$. Thus we get

$$ (A.6) \qquad \mathbb{P}\{Z_X(x) \in \mathfrak{A} \mid M_X(x) > 0\} = \mathbb{P}\{Z_X(x) \in e^{i\langle\nu, h\rangle}\mathfrak{A} \mid M_X(x) > 0\}. $$

Let $\omega \in [0, 2\pi]$. Considering $h := \omega \nu / \|\nu\|_2^2$, we have

$$ (A.7) \qquad h \in B_2(2\pi/\|\nu\|_2) \quad \text{and} \quad \langle\nu, h\rangle = \omega. $$

Therefore,

$$ (A.8) \qquad \forall \omega \in [0, 2\pi], \mathbb{P}\{Z_X(x) \in \mathfrak{A} \mid M_X(x) > 0\} = \mathbb{P}\{Z_X(x) \in e^{i\omega\mathfrak{A}} \mid M_X(x) > 0\}, $$
## B.4 Justification for Hypothesis 3

Let $n \in \mathbb{N} \setminus \{0\}$ and $\mathbf{x}, \mathbf{y}_0, \dots, \mathbf{y}_{n-1} \in \mathbb{R}^2$. To simplify notations, we consider the random vector

$$ \mathbf{M} := (\mathrm{M}_X(\mathbf{y}_0), \dots, \mathrm{M}_X(\mathbf{y}_{n-1}))^\top \quad (226) $$

with outcomes in $\mathbb{R}_+^n$. This section is organized as follows. Using reasoning similar to that in Proposition 8, we show that, for any measurable subset $\mathcal{S} \subset \mathbb{R}_+^n$, $Z_X$ follows a near-uniform probability distribution conditionally to $\mathbf{M} \in \mathcal{S}$. Since we already assumed that $Z_X$ follows an unconditional uniform distribution, we deduce that $Z_X$ and $\mathbf{M}$ are nearly independent.

**Proposition 9.** We assume that $\mathrm{F}_{1,X}$ is $\tau$-locally stationary (Definition 1) and $\tau'$-locally phase-shift-equivariant with respect to $\nu$ (Definition 2), both in the strict sense. Then, for any measurable sets $\mathfrak{A} \subset \mathbb{S}^1$ and $\mathcal{S} \subset \mathbb{R}_+^n$,

$$ \forall \omega \in [0, 2\pi], \quad |\mu_{\mathcal{S}}(\mathfrak{A}) - \mu_{\mathcal{S}}(e^{i\omega\mathfrak{A}})| \le 2\pi(\tau + \tau')/\|\nu\|_2. \quad (227) $$

where $\mu_{\mathcal{S}} : \mathfrak{A} \mapsto \mathbb{P}\{Z_X(\mathbf{x}) \in \mathfrak{A} \mid \mathbf{M} \in \mathcal{S}\}$ denotes the conditional probability measure of $Z_X(\mathbf{x})$.

*Proof.* Let $\omega \in [0, 2\pi]$. We compute:

$$ \begin{aligned} |\mu_{\mathcal{S}}(e^{i\omega\mathfrak{A}}) - \mu_{\mathcal{S}}(\mathfrak{A})| &= |\mathbb{P}\{Z_X(\mathbf{x}) \in e^{i\omega\mathfrak{A}} \mid \mathbf{M} \in \mathcal{S}\} - \mathbb{P}\{Z_X(\mathbf{x}) \in \mathfrak{A} \mid \mathbf{M} \in \mathcal{S}\}| \\ &= |\mathbb{P}\{Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathbf{M} \in \mathcal{S}\} - \mathbb{P}\{Z_X(\mathbf{x}) \in \mathfrak{A} \mid \mathbf{M} \in \mathcal{S}\}|, \end{aligned} $$

where we have denoted

$$ \mathbf{h} := \omega \nu / \|\nu\|_2^2 \quad (228) $$

Then, using the triangle inequality,

$$ \begin{aligned} |\mu_{\mathcal{S}}(e^{i\omega\mathfrak{A}}) - \mu_{\mathcal{S}}(\mathfrak{A})| &\le |\mathbb{P}\{\mathcal{T}_h Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathcal{T}_h \mathbf{M} \in \mathcal{S}\} - \mathbb{P}\{Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathbf{M} \in \mathcal{S}\}| \\ &\quad + |\mathbb{P}\{\mathcal{T}_h Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathcal{T}_h \mathbf{M} \in \mathcal{S}\} - \mathbb{P}\{Z_X(\mathbf{x}) \in \mathfrak{A} \mid \mathbf{M} \in \mathcal{S}\}|, \end{aligned} \quad (229) $$

where we have denoted

$$ \mathcal{T}_h \mathbf{M} := (\mathcal{T}_h \mathrm{M}_X(\mathbf{y}_0), \dots, \mathcal{T}_h \mathrm{M}_X(\mathbf{y}_{n-1}))^\top. \quad (230) $$

Let us split this expression for the sake of readability. The first term after the $\le$ sign can be equivalently written as follows:

$$ \begin{aligned} &|\mathbb{P}\{\mathcal{T}_h Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathcal{T}_h \mathbf{M} \in \mathcal{S}\} - \mathbb{P}\{Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathbf{M} \in \mathcal{S}\}| \\ &= |\mathbb{P}\{\mathcal{T}_h \mathrm{F}_{1,X}(\mathbf{x}) \in \mathcal{X}' \mid (\mathcal{T}_h \mathrm{F}_{1,X}(\mathbf{y}_0), \dots, \mathcal{T}_h \mathrm{F}_{1,X}(\mathbf{y}_{n-1}))^\top \in \mathfrak{Y}\} \\ &\quad - \mathbb{P}\{\mathrm{F}_{1,X}(\mathbf{x}) \in \mathcal{X}' \mid (\mathrm{F}_{1,X}(\mathbf{y}_0), \dots, \mathrm{F}_{1,X}(\mathbf{y}_{n-1}))^\top \in \mathfrak{Y}\}|, \end{aligned} \quad (231) $$

where we have denoted

$$ \mathcal{X}' := u^{-1}(e^{i\langle\nu, h\rangle\mathfrak{A}}) \quad \text{and} \quad \mathfrak{Y} := v^{-1}(\mathcal{S}), \quad (232) $$

with

$$ u : z \mapsto e^{i\langle z, h\rangle} \quad \text{and} \quad v : (z_0, \dots, z_{n-1})^\top \mapsto (|z_0|, \dots, |z_{n-1}|)^\top. \quad (233) $$

Therefore, according to the local stationarity hypothesis, we apply (210) to (231), which yields

$$ |\mathbb{P}\{\mathcal{T}_h Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathcal{T}_h \mathbf{M} \in \mathcal{S}\} - \mathbb{P}\{Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathbf{M} \in \mathcal{S}\}| \le \tau \|\mathbf{h}\|_2. \quad (234) $$
which yields (A.2).

Any probability measure defined on $\mathbb{S}^1$ is a Radon measure. Therefore, according to Haar's theorem [15], there exists a unique probability measure on $\mathbb{S}^1$ satisfying (A.2). Since the uniform probability measure is also invariant to phase shifts, we deduce that $Z_X(x)$ is uniformly distributed on $\mathbb{S}^1$, conditionally to $M_X(x) > 0$, which concludes the proof. ■

**Proposition A.2.** We assume the conditions of Proposition A.1 are met. If, moreover, $F_X$ is strict-sense stationary, then Hypothesis 3.8 is satisfied.

*Proof.* Let $n \in \mathbb{N} \setminus \{0\}$ and $x, y_0, \dots, y_{n-1} \in \mathbb{R}^2$. To alleviate notations, we consider the random vector $\mathbf{M} = (M_X(y_0), \dots, M_X(y_{n-1}))^\top$ with outcomes in $\mathbb{R}_+^n$. According to (3.53), $Z_X(x)$ is conditionally independent of $\mathbf{M}$ given $M_X(x) = 0$. Therefore, it remains to prove conditional independence given $M_X(x) > 0$.

The proof is organized as follows. Using a similar reasoning as Proposition A.1, we show that, for any measurable subset $\mathcal{S} \subset \mathbb{R}_+^n$, $Z_X$ follows a uniform probability distribution conditionally to $\mathbf{M} \in \mathcal{S}$ and $M_X(x) > 0$. Since we already know that $Z_X$ follows a uniform distribution conditionally to $M_X(x) > 0$ alone, we deduce that $Z_X$ and $\mathbf{M}$ are conditionally independent given $M_X(x) > 0$.

Let $\mathfrak{A} \subset \mathbb{S}^1$ and $\mathcal{G} := (\mathcal{G}_i)_{i \in \{0..n-1\}} \subset \mathbb{R}_+^n$ denote measurable sets. According to (A.1), and assuming $M_X(x) > 0$, we get, for any $h \in B_2(2\pi/\|\nu\|_2)$,

$$ (A.9) \qquad Z_X(x) \in \mathfrak{A} \iff \mathcal{T}_h Z_X(x) \in e^{i\langle\nu, h\rangle}\mathfrak{A}; $$

$$ (A.10) \qquad M_X(y_i) \in \mathcal{G}_i \iff \mathcal{T}_h M_X(y_i) \in \mathcal{G}_i \quad \forall i \in \{0 \dots n-1\}. $$

Therefore,

$$ (A.11) \qquad \begin{aligned} & \mathbb{P}\{(Z_X(x) \in \mathfrak{A}) \ \& \ (\mathbf{M} \in \mathcal{G}) \ | \ M_X(x) > 0\} \\ & = \mathbb{P}\{(\mathcal{T}_h Z_X(x) \in e^{i\langle\nu, h\rangle}\mathfrak{A}) \ \& \ (\mathcal{T}_h \mathbf{M} \in \mathcal{G}) \ | \ M_X(x) > 0\}. \end{aligned} $$

Since $F_X$ is strict-sense stationary, the joint conditional probability density of

$$ (A.12) \qquad \mathcal{T}_h Z_X(x), \mathcal{T}_h M_X(y_0), \dots, \mathcal{T}_h M_X(y_{n-1}) $$

is identical to the one of

$$ (A.13) \qquad Z_X(x), M_X(y_0), \dots, M_X(y_{n-1}). $$

Therefore we get

$$ (A.14) \qquad \begin{aligned} & \mathbb{P}\{(Z_X(x) \in \mathfrak{A}) \ \& \ (\mathbf{M} \in \mathcal{G}) \ | \ M_X(x) > 0\} \\ & = \mathbb{P}\{(Z_X(x) \in e^{i\langle\nu, h\rangle}\mathfrak{A}) \ \& \ (\mathbf{M} \in \mathcal{G}) \ | \ M_X(x) > 0\}. \end{aligned} $$

We assume that $\mathbb{P}(\mathbf{M} \in \mathcal{G}) > 0$. According to the above expression, and similarly to the proof of Proposition A.1, we get,

$$ (A.15) \qquad \begin{aligned} & \forall \omega \in [0, 2\pi], \mathbb{P}\{Z_X(x) \in \mathfrak{A} \ | \ (\mathbf{M} \in \mathcal{G}) \ \& \ (M_X(x) > 0)\} \\ & = \mathbb{P}\{Z_X(x) \in e^{i\omega}\mathfrak{A} \ | \ (\mathbf{M} \in \mathcal{G}) \ \& \ (M_X(x) > 0)\}. \end{aligned} $$
Next, the second term after the $\le$ sign in (229) can be equivalently written as follows:

$$ \begin{aligned} & \left| \mathbb{P}\{\mathcal{T}_h Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathcal{T}_h \mathbf{M} \in \mathcal{S}\} - \mathbb{P}\{Z_X(\mathbf{x}) \in \mathfrak{A} \mid \mathbf{M} \in \mathcal{S}\} \right| \\ & = \left| \mathbb{P}\{\mathcal{T}_h \mathbf{F}_{1,x}(\mathbf{x}) \in u^{-1}(e^{i\langle\nu, h\rangle\mathfrak{A}}) \mid (\mathcal{T}_h \mathbf{F}_{1,x}(\mathbf{y}_0), \dots, \mathcal{T}_h \mathbf{F}_{1,x}(\mathbf{y}_{n-1}))^\top \in \mathbf{v}^{-1}(\mathcal{S})\} \\ & \quad - \mathbb{P}\{\mathbf{F}_{1,x}(\mathbf{x}) \in u^{-1}(\mathfrak{A}) \mid (\mathbf{F}_{1,x}(\mathbf{y}_0), \dots, \mathbf{F}_{1,x}(\mathbf{y}_{n-1}))^\top \in \mathbf{v}^{-1}(\mathcal{S})\} \right|. \end{aligned} \quad (235) $$

As explained in the proof of Proposition 8, we have

$$ u^{-1}(e^{i\langle\nu, h\rangle\mathfrak{A}}) = e^{i\langle\nu, h\rangle}u^{-1}(\mathfrak{A}). \quad (236) $$

Moreover, we can easily show that

$$ \mathbf{v}^{-1}(\mathcal{S}) = e^{i\langle\nu, h\rangle} \mathbf{v}^{-1}(\mathcal{S}). \quad (237) $$

Therefore, (235) can be re-written:

$$ \begin{aligned} & \left| \mathbb{P}\{\mathcal{T}_h Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathcal{T}_h \mathbf{M} \in \mathcal{S}\} - \mathbb{P}\{Z_X(\mathbf{x}) \in \mathfrak{A} \mid \mathbf{M} \in \mathcal{S}\} \right| \\ & = \left| \mathbb{P}\{\mathcal{T}_h \mathbf{F}_{1,x}(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid (\mathcal{T}_h \mathbf{F}_{1,x}(\mathbf{y}_0), \dots, \mathcal{T}_h \mathbf{F}_{1,x}(\mathbf{y}_{n-1}))^\top \in e^{i\langle\nu, h\rangle\mathfrak{A}}\} \\ & \quad - \mathbb{P}\{\mathbf{F}_{1,x}(\mathbf{x}) \in \mathfrak{A} \mid (\mathbf{F}_{1,x}(\mathbf{y}_0), \dots, \mathbf{F}_{1,x}(\mathbf{y}_{n-1}))^\top \in \mathfrak{A}\} \right|, \end{aligned} \quad (238) $$

where we have denoted

$$ \mathfrak{A} := u^{-1}(\mathfrak{A}) \quad \text{and} \quad \mathfrak{A} := \mathbf{v}^{-1}(\mathcal{S}). \quad (239) $$

Therefore, according to the local phase-shift equivariance hypothesis, we apply (219) to (238), which yields

$$ \left| \mathbb{P}\{\mathcal{T}_h Z_X(\mathbf{x}) \in e^{i\langle\nu, h\rangle\mathfrak{A}} \mid \mathcal{T}_h \mathbf{M} \in \mathcal{S}\} - \mathbb{P}\{Z_X(\mathbf{x}) \in \mathfrak{A} \mid \mathbf{M} \in \mathcal{S}\} \right| \le \tau' \|\mathbf{h}\|_2. \quad (240) $$

Finally, plugging (234) and (240) into (229), using the definition of $\mathbf{h}$ in (228), and bounding $\omega$ by $2\pi$, yields the result. $\square$

By applying the constraints sated in (84) and (85), we get (see Appendix B.3),

$$ \forall \omega \in [0, 2\pi], \quad |\mu_{\mathcal{S}}(\mathfrak{A}) - \mu_{\mathcal{S}}(e^{i\omega\mathfrak{A}})| \ll 1. \quad (241) $$

Therefore, $Z_X(\mathbf{x})$ follows a near-uniform conditional distribution on $\mathbb{S}^1$ given $\mathbf{M} \in \mathcal{S}$.

Furthermore, according to Appendix B.3, $Z_X(\mathbf{x})$ also follows a near-uniform unconditional distribution. Therefore, we get, for any measurable sets $\mathfrak{A} \subset \mathbb{S}^1$ and $\mathcal{S} \subset \mathbb{R}^n_+$,

$$ \mathbb{P}\{Z_X(\mathbf{x}) \in \mathfrak{A} \mid (\mathbf{M} \in \mathcal{S})\} \approx \mathbb{P}\{Z_X(\mathbf{x}) \in \mathfrak{A}\}. \quad (242) $$

We deduce that $Z_X(\mathbf{x})$ and $\mathbf{M}$ are nearly independent. For the sake of simplicity, in Hypothesis 3 we have assumed strict independence.

# C Appendix – Details on DT-CWPT

A description of the transform itself is provided in Appendix C.1. Then, Appendix C.2 shows that DT-CWPT performs convolutions with a subsampling factor $m_J$ which depends on the decomposition depth $J$. Finally, the Gabor-like nature of the convolution kernels is established in Appendix C.3.

## C.1 Background

We provide a brief overview of the classical, real-valued 2D wavelet packet transform (WPT) algorithm (Mallat, 2009, p. 377), before introducing the redundant, complex-valued and oriented DT-CWPT (Bayram & Selesnick, 2008).
Then, the above conditional probability measure satisfies phase shift invariance (A.2). Therefore, as in the proof of Proposition A.1, Haar's theorem implies that $Z_X(x)$ follows a uniform conditional distribution given $M \in \mathcal{G}$ and $M_X(x) > 0$.

Moreover, strict-sense implies first-order stationarity, and thus, according to the proof of Proposition A.1, $Z_X(x)$ follows a uniform distribution conditionally to $M_X(x) > 0$. Therefore we get, for any measurable sets $\mathfrak{A} \subset \mathbb{S}^1$ and $\mathcal{G} \subset \mathbb{R}_+^n$ such that $\mathbb{P}(M \in \mathcal{G}) > 0$,

$$ (A.16) \qquad \mathbb{P}\{Z_X(x) \in \mathfrak{A} \mid (M \in \mathcal{G}) \& (M_X(x) > 0)\} = \mathbb{P}\{Z_X(x) \in \mathfrak{A} \mid M_X(x) > 0\}, $$

which proves conditional independence between $Z_X(x)$ and $M$ given $M_X(x) > 0$, and concludes the proof. ■

**Remark A.3 (Stationarity hypothesis).** Strict-sense stationarity suggests that any translated version of a given image is equally likely. In reality, this statement is too strong, for several reasons. First, by construction, X has all its realizations in $L_{\mathbb{R}}^2(\mathbb{R}^2)$. In that context, a stationary process yields outcomes which are zero almost everywhere. Besides, depending on which category the image belongs to, the pixel distribution is likely to vary across various regions. For instance, we can expect the main subject to be located at the center of the image. More details on statistical properties of images from natural versus man-made objects can be found in a paper by Torralba and Oliva [46]. Nevertheless, this hypothesis will be considered as a reasonable approximation if the shift is much smaller than the image “characteristic” size in the continuous domain; i.e., if

$$ (A.17) \qquad \|h\|_2 \ll sN, $$

where, as a reminder, $N$ denotes the support size of input images. We refer the reader to [47] for a related notion of local stationarity. As it turns out, the proofs of Propositions A.1 and A.2 only requires shifts with $\|h\|_2 \le 2\pi/\|\nu\|_2$. Therefore, the constraint on $\|\theta\|_2$ stated in (3.56) implies (A.17), and the stationarity hypothesis holds.

**Remark A.4 (Justification for (A.1)).** We consider

$$ (A.18) \qquad \Phi_W : x \mapsto \Psi_W(x) e^{-i\langle \nu, x \rangle}. $$

Similarly to Lemma 2.2, we can show that $\Phi_W$ is a low-pass filter, with $\text{supp} \hat{\Phi}_W \subset B_\infty(\varepsilon/2)$. For all $h \in \mathbb{R}^2$ such that $\|h\|_2 \le 2\pi/\|\nu\|_2$, we have

$$ \begin{aligned} (\mathcal{T}_h \mathbf{F}_X * \bar{\Psi}_W)(x) &= \iint_{\mathbb{R}^2} \mathcal{T}_h \mathbf{F}_X(x - y) \bar{\Phi}_W(y) e^{-i\langle \nu, y \rangle} \mathrm{d}^2 y \\ &= e^{i\langle \nu, h \rangle} \iint_{\mathbb{R}^2} \mathbf{F}_X(x - y') \bar{\Phi}_W(y' - h) e^{-i\langle \nu, y' \rangle} \mathrm{d}^2 y'. \end{aligned} $$

Since $\text{supp} \hat{\Phi}_W \subset B_\infty(\frac{\kappa}{2s})$, we can define a “minimal wavelength” $\lambda_{\Phi_W} := 2\pi s/\kappa$. Then, if $\|h\|_2 \ll \lambda_{\Phi_W}$, we can approximate $\bar{\Phi}_W(y' - h) \approx \bar{\Phi}_W(y')$. This sufficient condition is actually met, because $\|h\|_2 \le 2\pi/\|\nu\|_2$ and, according to (3.57), $\|\nu\|_2 \gg \kappa/s$. Therefore,

$$ (A.19) \qquad (\mathcal{T}_h \mathbf{F}_X * \bar{\Psi}_W)(x) \approx e^{i\langle \nu, h \rangle} (\mathbf{F}_X * \bar{\Psi}_W)(x). $$
### C.1.1 Discrete Wavelet Packet Transform

Given a pair of low- and high-pass 1D orthogonal filters h, g ∈ $l_2^R(Z)$ satisfying a quadrature mirror filter (QMF) relationship, we consider a separable 2D filter bank (FB), denoted by $G := (G_l)_{l \in \{0, ..., 3\}}$, defined by

$$G_0 = h \otimes h; \quad G_1 = h \otimes g; \quad G_2 = g \otimes h; \quad G_3 = g \otimes g. \quad (243)$$

Let $X \in l_2^R(Z)$. The decomposition starts with $D_0^{(0)} = X$. Given $j \in \mathbb{N}$, suppose that we have computed $4^j$ sequences of wavelet packet coefficients at stage $j$, denoted by $D_l^{(j)} \in l_2^R(Z)$ for each $l \in \{0, ..., 4^j - 1\}$. They are referred to as feature maps.

At stage $j + 1$, we compute a new representation of X with increased frequency resolution—and decreased spatial resolution. It is obtained by further decomposing each feature map $D_l^{(j)}$ into four sub-sequences, using subsampled (or strided) convolutions with kernels $G_k$, for each $k \in \{0, ..., 3\}$:

$$\forall k \in \{0, ..., 3\}, \quad D_{4l+k}^{(j+1)} = (D_l^{(j)} * \overline{G_k}) \downarrow 2. \quad (244)$$

The algorithm stops after reaching the desired number of stages $J > 0$—referred to as decomposition depth. Then,

$$\mathbf{D}^{(J)} := (\mathbf{D}_l^{(J)})_{l \in \{0, ..., 4^J - 1\}} \quad (245)$$

constitutes a multichannel representation of X in an orthonormal basis, from which the original image can be retrieved.

### C.1.2 Dual-Tree Complex Wavelet Packet Transform

Despite having interesting properties such as sparse signal representation, WPT is unstable with respect to small shifts and suffers from a poor directional selectivity. To overcome this, Kingsbury (2001) designed a new type of discrete wavelet transform, where images are decomposed in a redundant frame of nearly-analytic, complex-valued waveforms. It was later extended to the wavelet packet framework by Bayram & Selesnick (2008). The latter operation, referred to as dual-tree complex wavelet packet transform (DT-CWPT), is performed as follows.

Let $(h^{[0]}, g^{[0]})$ and $(h^{[1]}, g^{[1]})$ denote two pairs of QMFs as defined in Appendix C.1.1, satisfying the half-sample delay condition:

$$\forall \omega \in [-\pi, \pi], \quad \hat{h}^{[1]}(\omega) = e^{-i\omega/2} \hat{h}^{[0]}(\omega). \quad (246)$$

Then, for any $k \in \{0, ..., 3\}$, we build a 2D FB $G_k := (G_{k,l})_{l \in \{0, ..., 3\}}$ similarly to (243):

$$G_{k,0} = h_i \otimes h_j; \quad G_{k,1} = h_i \otimes g_j; \quad G_{k,2} = g_i \otimes h_j; \quad G_{k,3} = g_i \otimes g_j, \quad (247)$$

where $i, j \in \{0, 1\}$ are defined such that $k = 2 \times i + j$.<fnref n="4" />

Let $J > 0$ denote a decomposition depth. Using each of the four FBs $G_{0-3}$ as defined above, we assume that we have decomposed an input image X into four multichannel WPT representations $D_{0-3}^{(J)}$, each of which satisfies (244) and (245). Then, for any $l \in \{0, ..., 4^J - 1\}$, the following complex feature maps are computed:

$$\begin{pmatrix} D_l^{\downarrow(J)} \\ D_l^{\uparrow(J)} \end{pmatrix} = \begin{pmatrix} 1 & -1 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} D_l^{[0](J)} \\ D_l^{[3](J)} \end{pmatrix} - i \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} D_l^{[2](J)} \\ D_l^{[1](J)} \end{pmatrix}. \quad (248)$$

As explained in Appendix C.3, the feature maps of dual-tree coefficients have their Fourier transform restricted to a compact region of the frequency plane, and as such can be considered as Gabor-like coefficients.

<footnote n="4">⁴Actually, the FB design requires some technicalities which are not described here.</footnote>
As explained in Remarks A.3 and A.4, the sufficient conditions outlined in Propositions A.1 and A.2 are not strictly met. Nevertheless, we consider that Hypotheses 3.7 and 3.8 still provide a reasonable description of the distribution from which input images are drawn.

**Appendix B. Details on DT-CWPT.** A description of the transform itself is provided in Appendix B.1. Then, Appendix B.2 shows that DT-CWPT performs convolutions with a subsampling factor $m_J$ which depends on the decomposition depth $J$. Finally, the Gabor-like nature of the convolution kernels is established in Appendix B.3.

**B.1. Background.** We provide a brief overview of the classical, real-valued 2D wavelet packet transform (WPT) algorithm [27, p. 377], before introducing the redundant, complex-valued and oriented DT-CWPT [4].

**B.1.1. Discrete Wavelet Packet Transform.** Given a pair of low- and high-pass 1D orthogonal filters $h, g \in l_{\mathbb{R}}^2(\mathbb{Z})$ satisfying a *quadrature mirror filter* (QMF) relationship, we consider a separable 2D filter bank (FB), denoted by $G := (G_l)_{l \in \{0..3\}}$, defined by

$$ (B.1) \qquad G_0 = h \otimes h; \quad G_1 = h \otimes g; \quad G_2 = g \otimes h; \quad G_3 = g \otimes g. $$

Let $X \in l_{\mathbb{R}}^2(\mathbb{Z})$. The decomposition starts with $D_0^{(0)} = X$. Given $j \in \mathbb{N}$, suppose that we have computed $4^j$ sequences of wavelet packet coefficients at stage $j$, denoted by $D_l^{(j)} \in l_{\mathbb{R}}^2(\mathbb{Z})$ for each $l \in \{0..4^j - 1\}$. They are referred to as *feature maps*.

At stage $j+1$, we compute a new representation of $X$ with increased frequency resolution— and decreased spatial resolution. It is obtained by further decomposing each feature map $D_l^{(j)}$ into four sub-sequences, using subsampled (or strided) convolutions with kernels $G_k$, for each $k \in \{0..3\}$:

$$ (B.2) \qquad \forall k \in \{0..3\}, \quad D_{4l+k}^{(j+1)} = (D_l^{(j)} * \overline{G_k}) \downarrow 2. $$

The algorithm stops after reaching the desired number of stages $J > 0$—referred to as *decomposition depth*. Then,

$$ (B.3) \qquad \mathbf{D}^{(J)} := (\mathbf{D}_l^{(J)})_{l \in \{0..4^J-1\}} $$

constitutes a multichannel representation of $X$ in an orthonormal basis, from which the original image can be retrieved.

**B.1.2. Dual-Tree Complex Wavelet Packet Transform.** Despite having interesting properties such as sparse signal representation, WPT is unstable with respect to small shifts and suffers from a poor directional selectivity. To overcome this, Kingsbury [18] designed a new type of discrete wavelet transform, where images are decomposed in a redundant frame of nearly-analytic, complex-valued waveforms. It was later extended to the wavelet packet framework by Bayram and Selesnick [4]. The latter operation, referred to as *dual-tree complex wavelet packet transform* (DT-CWPT), is performed as follows.

Let $(h^{[0]}, g^{[0]})$ and $(h^{[1]}, g^{[1]})$ denote two pairs of QMFs as defined in Appendix B.1.1, satisfying the *half-sample delay* condition:

$$ (B.4) \qquad \forall \omega \in [-\pi, \pi], \quad \hat{h}^{[1]}(\omega) = e^{-i\omega/2} \hat{h}^{[0]}(\omega). $$
In the above expression, the arrow points to the Fourier quadrant where energy is concentrated. Furthermore, in the specific case where input images are real-valued, $D_l^{\lambda(J)}$ and $D_l^{\kappa(J)}$ are defined as the complex conjugates of the above feature maps, and therefore do not need to be explicitly computed. Then,

$$ \mathbf{D}^{(J)} := (D_l^{\lambda(J)}, D_l^{\kappa(J)}, D_l^{\lambda(J)}, D_l^{\kappa(J)})_{l \in \{0, \dots, 4^J - 1\}} \quad (249) $$

constitutes a complex-valued, four-time redundant multichannel representation of X from which the original image can be reconstructed.

## C.2 Convolution Operators

We now show that DT-CWPT performs subsampled convolutions with Gabor-like filters, whose characteristics will be specified. First, we state the following lemma concerning the real-valued WPT algorithm, such as introduced in Appendix C.1.1. It is a simple reformulation of the well-known result that two successive convolutions can be written as another convolution with a wider kernel.

**Lemma 7.** For any $l \in \{0, \dots, 4^J - 1\}$, there exists $V_l^{(J)} \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$ such that

$$ D_l^{(J)} = (X * \bar{V}_l^{(J)}) \downarrow 2^J. \quad (250) $$

*Proof.* We introduce the upsampling operator: $(X \uparrow m)[n] := X[n/m]$ if $n/m \in \mathbb{Z}^2$, and 0 otherwise. We also consider the “identity” filter $I \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$ such that $I[0] = 1$ and $I[n] = 0$ otherwise. First, for any $U, V \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$ and any $s, t \in \mathbb{N}^*$, we have

$$ ((U \downarrow s) * V) \downarrow t = (U * (V \uparrow s)) \downarrow (st). \quad (251) $$

Then, a simple reasoning by induction yields the result, with

$$ V_0^{(0)} := I; \quad V_{4l+k}^{(j+1)} := V_l^{(j)} * (G_k \uparrow 2^j) \quad (252) $$

for any $l \in \{0, \dots, j-1\}$ and any $k \in \{0, \dots, 3\}$. $\square$

Based on Lemma 7, the following proposition introduces complex kernels characterizing DT-CWPT.

**Proposition 10.** For any $l \in \{0, \dots, 4^J - 1\}$, there exists $W_l^{\lambda(J)} \in l_{\mathbb{C}}^2(\mathbb{Z}^2)$ such that (132) is satisfied. Identical results are obtained with the three other Fourier quadrants.

*Proof.* For each of the four filter banks $m \in \{0, \dots, 3\}$, and any channel $l \in \{0, \dots, 4^J - 1\}$, Lemma 7 provides a convolution kernel $V_l^{[m](J)} \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$ such that

$$ D_l^{[m](J)} = (X * \bar{V}_l^{[m](J)}) \downarrow 2^J. \quad (253) $$

Then, the result is obtained by plugging (253) into (248) for all $m \in \{0, \dots, 3\}$, and by denoting

$$ \begin{pmatrix} W^{\lambda(J)}l \\ W^{\kappa(J)}l \end{pmatrix} = \begin{pmatrix} 1 & -1 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} V_l^{[0](J)} \\ V_l^{[3](J)} \end{pmatrix} + i \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} V_l^{[2](J)} \\ V_l^{[1](J)} \end{pmatrix}. \quad (254) $$

<figure><img src="image_11.png" /></figure>

**Remark 11.** DT-CWPT, computed on a discrete image X, approximates the decomposition of a continuous 2D signal $F \in L_{\mathbb{R}}^2(\mathbb{R}^2)$ into a tight frame

$$ \Psi_{\mathbb{C}}^{(J)} := \bigcup_{l=0}^{4^J-1} (\Psi_{l, n}^{\lambda(J)}, \Psi_{l, n}^{\kappa(J)}, \Psi_{l, n}^{\lambda(J)}, \Psi_{l, n}^{\kappa(J)})_{n \in \mathbb{Z}^2}. \quad (255) $$
Then, for any $k \in \{0..3\}$, we build a 2D FB $G_k := (G_{k,l})_{l \in \{0..3\}}$ similarly to (B.1):

$$ (B.5) \qquad G_{k,0} = h_i \otimes h_j; \quad G_{k,1} = h_i \otimes g_j; \quad G_{k,2} = g_i \otimes h_j; \quad G_{k,3} = g_i \otimes g_j, $$

where $i, j \in \{0, 1\}$ are defined such that $k = 2 \times i + j$.<fnref n="3" />

Let $J > 0$ denote a decomposition depth. Using each of the four FBs $G_{0-3}$ as defined above, we assume that we have decomposed an input image X into four multichannel WPT representations $D_{0-3}^{(J)}$, each of which satisfies (B.2) and (B.3). Then, for any $l \in \{0..4^J - 1\}$, the following complex feature maps are computed:

$$ (B.6) \qquad \begin{pmatrix} D_l^{\uparrow(J)} \\ D_l^{\downarrow(J)} \end{pmatrix} = \begin{pmatrix} 1 & -1 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} D_l^{[0](J)} \\ D_l^{[3](J)} \end{pmatrix} - i \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} D_l^{[2](J)} \\ D_l^{[1](J)} \end{pmatrix}. $$

As explained in Appendix B.3, the feature maps of dual-tree coefficients have their Fourier transform restricted to a compact region of the frequency plane, and as such can be considered as Gabor-like coefficients. In the above expression, the arrow points to the Fourier quadrant where energy is concentrated. Furthermore, in the specific case where input images are real-valued, $D_l^{\uparrow(J)}$ and $D_l^{\downarrow(J)}$ are defined as the complex conjugates of the above feature maps, and therefore do not need to be explicitly computed. Then,

$$ (B.7) \qquad \mathbf{D}^{(J)} := (D_l^{\uparrow(J)}, D_l^{\downarrow(J)}, D_l^{\uparrow(J)}, D_l^{\downarrow(J)})_{l \in \{0..4^J - 1\}} $$

constitutes a complex-valued, four-time redundant multichannel representation of X from which the original image can be reconstructed.

**B.2. Convolution Operators.** We now show that DT-CWPT performs subsampled convolutions with Gabor-like filters, whose characteristics will be specified. First, we state the following lemma concerning the real-valued WPT algorithm, such as introduced in Appendix B.1.1. It is a simple reformulation of the well-known result that two successive convolutions can be written as another convolution with a wider kernel.

**Lemma B.1.** For any $l \in \{0..4^J - 1\}$, there exists $V_l^{(J)} \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$ such that

$$ (B.8) \qquad D_l^{(J)} = (X * \bar{V}_l^{(J)}) \downarrow 2^J. $$

*Proof.* We introduce the upsampling operator: $(X \uparrow m)[n] := X[n/m]$ if $n/m \in \mathbb{Z}^2$, and 0 otherwise. We also consider the “identity” filter $I \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$ such that $I[0] = 1$ and $I[n] = 0$ otherwise. First, for any $U, V \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$ and any $s, t \in \mathbb{N}^*$, we have

$$ (B.9) \qquad ((U \downarrow s) * V) \downarrow t = (U * (V \uparrow s)) \downarrow (st). $$

Then, a simple reasoning by induction yields the result, with

$$ (B.10) \qquad V_0^{(0)} := I; \quad V_{4l+k}^{(j+1)} := V_l^{(j)} * (G_k \uparrow 2^j) $$

for any $l \in \{0..j-1\}$ and any $k \in \{0..3\}$.

<figure><img src="image_19.png" /></figure>

<footnote n="3">³Actually, the FB design requires some technicalities which are not described here.</footnote>
In this context, the feature maps of dual-tree wavelet packet coefficients satisfy

$$D_l^{\lambda^{(J)}}[\mathbf{n}] \approx (F * \bar{\Psi}_l^{\lambda^{(J)*}})(2^J \mathbf{n}), \quad \text{with} \quad \Psi_l^{\lambda^{(J)}} := \Psi_{l, \mathbf{0}}^{\lambda^{(J)}}. \qquad (256)$$

Expression (256) is only an approximation because of implementation technicalities that occur in practice. A “perfect” dual-tree transform should be initialized with four different inputs $X^{[0-3]}$. Instead, all four WPT decompositions operate on the same input image X, leading to non-analytic outputs for small values of J. In order to counterbalance this shortcoming, the first stage of DT-CWPT decomposition must be performed with a special set of filters that satisfy the one-sample delay condition. We refer to Selesnick et al. (2005) for more details on this matter.

### C.3 Gabor-Like Convolution Kernels

In this section, we show that the convolution kernels $W_l^{\lambda^{(J)}}$ and $W_l^{\lambda^{(J)*}}$, introduced in (132), approximately behave as Gabor-like filters, as defined in (11). To begin with, we assume that $h^{[0]}$ is a Shannon filter, which is associated with a sinc scaling function (Shannon, 1949). Let $J \in \mathbb{N} \setminus \{0\}$ denote the number of decomposition stages. The following proposition states that DT-CWPT tiles the frequency plane with square windows.

**Proposition 11.** There exists a permutation $(\sigma_l^{(J)})_{l \in \{0, \dots, 4^J - 1\}}$ of $\{0, \dots, 2^J - 1\}^2$ such that, for any $l \in \{0, \dots, 4^J - 1\}$,

$$\Psi_l^{\lambda^{(J)}} \in \mathcal{V}(\theta_l^{(J)}, \kappa_J), \qquad (257)$$

where $\Psi_l^{\lambda^{(J)}}$ has been introduced in Remark 11, and where we have defined

$$\theta_l^{(J)} := \left( \sigma_l^{(J)} + \frac{1}{2} \right) \frac{\pi}{2^J} \quad \text{and} \quad \kappa_J := \frac{\pi}{2^J}. \qquad (258)$$

We remind the reader that $\mathcal{V}(\nu, \varepsilon)$, defined in (8), denotes a space of Gabor-like filters in the continuous framework.

*Proof.* The atoms $\Psi_l^{\lambda^{(J)}}$ of the wavelet packet tight frame $\Psi_{\mathbb{C}}^{(J)}$ can be written as the tensor product of two 1D wavelet packets:

$$\Psi_l^{\lambda^{(J)}} = \psi_{l_1}^{(J)} \otimes \psi_{l_2}^{(J)}, \qquad (259)$$

for some indices $l_1$ and $l_2 \in \{0, \dots, 2^J - 1\}$. Moreover, for any $l' \in \{0, \dots, 2^J - 1\}$, we have

$$\psi_{l'}^{(J)} = \psi_{l'}^{[0](J)} + i \psi_{l'}^{[1](J)}, \qquad (260)$$

where $\psi_{l'}^{[0](J)} \in L_{\mathbb{R}}^2(\mathbb{R})$ is an atom of the standard Shannon wavelet packet orthonormal basis, and $\psi_{l'}^{[1](J)}$ is the one-dimensional Hilbert transform of $\psi_l^{[0](J)}$. Therefore, since the Hilbert transform suppresses negative frequencies, we get

$$\hat{\psi}_{l'}^{(J)} = 2 \hat{\psi}_{l'}^{[0](J)} 1_{\mathbb{R}_+}. \qquad (261)$$

Consequently, according to the Coifman-Wickerhauser theorem (Mallat, 2009, pp. 384-385), there exists $k \in \{0, \dots, 2^J - 1\}$ such that

$$\text{supp } \hat{\psi}_{l'}^{(J)} \subset \left[ \frac{k\pi}{2^J}, \frac{(k+1)\pi}{2^J} \right]. \qquad (262)$$

Finally, the tensor product (259) yields the result. □

According to Proposition 11, each atom $\Psi_l^{\lambda^{(J)}}$, for $l \in \{0, \dots, 4^J - 1\}$, is supported in a square window of size $\kappa_J \times \kappa_J$ included in the top-right quadrant of the Fourier domain. Similar results can be obtained for the three remaining quadrants, with $\Psi_l^{\lambda^{(J)*}}$, $\Psi_l^{\lambda^{(J)}}$ and $\Psi_l^{\lambda^{(J)}}$. We would like to deduce from Proposition 11 that the discrete filter $W_l^{\lambda^{(J)}} \in l_{\mathbb{C}}^2(\mathbb{Z}^2)$ satisfies the Gabor property (133). However, as mentioned in Remark 11,
Based on Lemma B.1, the following proposition introduces complex kernels characterizing DT-CWPT.

**Proposition B.2.** For any $l \in \{0..4^J - 1\}$, there exists $W_l^{\lambda^{(J)}} \in l_{\mathbb{C}}^2(\mathbb{Z}^2)$ such that (6.3) is satisfied. Identical results are obtained with the three other Fourier quadrants.

*Proof.* For each of the four filter banks $m \in \{0..3\}$, and any channel $l \in \{0..4^J - 1\}$, Lemma B.1 provides a convolution kernel $V_l^{[m](J)} \in l_{\mathbb{R}}^2(\mathbb{Z}^2)$ such that

$$ (B.11) \qquad D_l^{[m](J)} = (X * \bar{V}_l^{[m](J)}) \downarrow 2^J. $$

Then, the result is obtained by plugging (B.11) into (B.6) for all $m \in \{0..3\}$, and by denoting

$$ (B.12) \qquad \begin{pmatrix} W^{\lambda^{(J)}} l \\ W^{\lambda^{(J)}} \bar{l} \end{pmatrix} = \begin{pmatrix} 1 & -1 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} V_l^{[0](J)} \\ V_l^{[3](J)} \end{pmatrix} + i \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} V_l^{[2](J)} \\ V_l^{[1](J)} \end{pmatrix}. \qquad \blacksquare $$

*Remark B.3.* DT-CWPT, computed on a discrete image X, approximates the decomposition of a continuous 2D signal $F \in L_{\mathbb{R}}^2(\mathbb{R}^2)$ into a tight frame

$$ (B.13) \qquad \Psi_{\mathbb{C}}^{(J)} := \bigcup_{l=0}^{4^J-1} (\Psi_{l, \mathbf{n}}^{\lambda^{(J)}}, \Psi_{l, \mathbf{n}}^{\lambda^{(J)}}, \Psi_{l, \mathbf{n}}^{\lambda^{(J)}}, \Psi_{l, \mathbf{n}}^{\lambda^{(J)}})_{\mathbf{n} \in \mathbb{Z}^2}. $$

In this context, the feature maps of dual-tree wavelet packet coefficients satisfy

$$ (B.14) \qquad D_l^{\lambda^{(J)}}[\mathbf{n}] \approx (F * \bar{\Psi}_l^{\lambda^{(J)}*})(2^J \mathbf{n}), \quad \text{with} \quad \Psi_l^{\lambda^{(J)}} := \Psi_{l, \mathbf{0}}^{\lambda^{(J)}}. $$

Expression (B.14) is only an approximation because of implementation technicalities that occur in practice. A “perfect” dual-tree transform should be initialized with four different inputs $X^{[0-3]}$. Instead, all four WPT decompositions operate on the same input image X, leading to non-analytic outputs for small values of J. In order to counterbalance this shortcoming, the first stage of DT-CWPT decomposition must be performed with a special set of filters that satisfy the one-sample delay condition. We refer to [41] for more details on this matter.

**B.3. Gabor-Like Convolution Kernels.** In this section, we show that the convolution kernels $W_l^{\lambda^{(J)}}$ and $W_l^{\lambda^{(J)}}$, introduced in (6.3), approximately behave as Gabor-like filters, as defined in (2.5). To begin with, we assume that $h^{[0]}$ is a Shannon filter, which is associated with a sinc scaling function [42]. Let $J \in \mathbb{N} \setminus \{0\}$ denote the number of decomposition stages. The following proposition states that DT-CWPT tiles the frequency plane with square windows.

**Proposition B.4.** There exists a permutation $(\sigma_l^{(J)})_{l \in \{0..4^J-1\}}$ of $\{0..2^J - 1\}^2$ such that, for any $l \in \{0..4^J - 1\}$,

$$ (B.15) \qquad \Psi_l^{\lambda^{(J)}} \in \mathcal{V}(\theta_l^{(J)}, \kappa_J), $$

where $\Psi_l^{\lambda^{(J)}}$ has been introduced in Remark B.3, and where we have defined

$$ (B.16) \qquad \theta_l^{(J)} := \left( \sigma_l^{(J)} + \frac{1}{2} \right) \frac{\pi}{2^J} \quad \text{and} \quad \kappa_J := \frac{\pi}{2^J}. $$
<table><caption>Table 1. Energy concentration of the DT-CWPT filters within a Fourier window of size $κ_J × κ_J$, with $κ_J := π/2^{J-1}$.</caption><thead><tr><td>DEPTH J</td><td>BANDWIDTH κ<sub>J</sub></td><td>MEAN</td><td>STD</td></tr></thead><tbody><tr><td>2</td><td>π/2</td><td>0.98</td><td>0.00</td></tr><tr><td>3</td><td>π/4</td><td>0.95</td><td>0.02</td></tr></tbody></table>

(256) is only an approximation. In fact, the Fourier support of $W_l^{\perp(J)}$ is contained in four square regions of size $κ_J$ (one in each quadrant), its energy becoming negligible outside the top-right quadrant when $J$ increases. Nevertheless, employing, in the first stage, a specific pair of low-pass filters satisfying the one-sample delay condition (Selesnick et al., 2005) yields near-analytic solutions even for small values of $J$. We therefore assume that (133) is a reasonable approximation if $J ≥ 2$.

**Remark 12.** Proposition 11 tiles the top-right Fourier quadrant with $4^J$ square cells of size $κ_J := π/2^J$. However, the Shannon wavelet is poorly suited for sparse image representations, because of its slow decay rate. Moreover, it deviates from what is typically observed in freely-trained CNNs, because $W_l^{\perp(J)}$ must be approximated with very large filters to avoid numerical instabilities. Practical implementations of DT-CWPT use fast-decaying filters such as these associated to Meyer wavelets (Meyer, 1985), or finite-length filters that approximate the half-sample delay condition (Selesnick et al., 2005). Therefore, energy is leaking outside the square cells tiling the Fourier domain. To counterbalance this, we increase the window size up to

$$κ_J := \frac{\pi}{2^{J-1}} = \pi/m_J, \qquad (263)$$

and suggest that (133) remains a reasonable approximation. Therefore, the conditions to apply Theorems 1 to 3 are approximately satisfied in this context.

In order to numerically assess this assumption, we measured the maximum percentage of energy within a square window of size $κ_J × κ_J$ in the Fourier domain:

$$ρ_l^{\perp} := \frac{\max_{\theta \in [-\pi, \pi]^2} \left\| \mathbb{1}_{B_{\infty}(\theta, \kappa_J/2)} \widehat{W}_l^{\perp(J)} \right\|_{L^2}^2}{\left\| \widehat{W}_l^{\perp(J)} \right\|_{L^2}^2}, \qquad (264)$$

where the $l^\infty$-ball $B_\infty(\theta, \kappa_J/2)$ is defined in the quotient space $[-\pi, \pi]^2/(2\pi\mathbb{Z}^2)$, as explained in Remark 2. If (133) is perfectly satisfied, then $ρ_l^{\perp} = 1$. The statistics computed over the collection $(ρ_l^{\perp}, ρ_l^{\perp})_{l \in \{0, ..., 4^J-1\}}$ are reported in table 1.

**Remark 13.** For “boundary filters”, i.e., when $||θ_l^{(J)}||_\infty = (1 - 2^{-(J+1)}) \pi$, Remark 2 states that a small fraction of the filter’s energy remains located at the far end of the Fourier domain—see also Bayram & Selesnick (2008). Therefore, these filters do not strictly comply with the conditions of Theorems 1 to 3. We nevertheless include them in our experiments.
We remind the reader that $V(\nu, \varepsilon)$, defined in (2.2), denotes a space of Gabor-like filters in the continuous framework.

*Proof.* The atoms $\Psi_l^{\lambda(J)}$ of the wavelet packet tight frame $\Psi_C^{(J)}$ can be written as the tensor product of two 1D wavelet packets:

$$ (B.17) \qquad \Psi_l^{\lambda(J)} = \psi_{l_1}^{(J)} \otimes \psi_{l_2}^{(J)}, $$

for some indices $l_1$ and $l_2 \in \{0..2^J - 1\}$. Moreover, for any $l' \in \{0..2^J - 1\}$, we have

$$ (B.18) \qquad \psi_{l'}^{(J)} = \psi_{l'}^{[0](J)} + i \psi_{l'}^{[1](J)}, $$

where $\psi_{l'}^{[0](J)} \in L_{\mathbb{R}}^2(\mathbb{R})$ is an atom of the standard Shannon wavelet packet orthonormal basis, and $\psi_{l'}^{[1](J)}$ is the one-dimensional Hilbert transform of $\psi_{l'}^{[0](J)}$. Therefore, since the Hilbert transform suppresses negative frequencies, we get

$$ (B.19) \qquad \hat{\psi}_{l'}^{(J)} = 2 \hat{\psi}_{l'}^{[0](J)} 1_{\mathbb{R}_+}. $$

Consequently, according to the Coifman-Wickerhauser theorem [27, pp. 384-385], there exists $k \in \{0..2^J - 1\}$ such that

$$ (B.20) \qquad \mathrm{supp} \, \hat{\psi}_{l'}^{(J)} \subset \left[ \frac{k\pi}{2^J}, \frac{(k+1)\pi}{2^J} \right]. $$

Finally, the tensor product (B.17) yields the result.

According to Proposition B.4, each atom $\Psi_l^{\lambda(J)}$, for $l \in \{0..4^J - 1\}$, is supported in a square window of size $\kappa_J \times \kappa_J$ included in the top-right quadrant of the Fourier domain. Similar results can be obtained for the three remaining quadrants, with $\Psi_l^{\lambda(J)}$, $\Psi_l^{\lambda(J)}$ and $\Psi_l^{\lambda(J)}$. We would like to deduce from Proposition B.4 that the discrete filter $W_l^{\lambda(J)} \in l_C^2(\mathbb{Z}^2)$ satisfies the Gabor property (6.4). However, as mentioned in Remark B.3, (B.14) is only an approximation. In fact, the Fourier support of $W_l^{\lambda(J)}$ is contained in four square regions of size $\kappa_J$ (one in each quadrant), its energy becoming negligible outside the top-right quadrant when $J$ increases. Nevertheless, employing, in the first stage, a specific pair of low-pass filters satisfying the one-sample delay condition [41] yields near-analytic solutions even for small values of $J$. We therefore consider (6.4) as a reasonable approximation if $J \ge 2$.

*Remark B.5.* Proposition B.4 tiles the top-right Fourier quadrant with $4^J$ square cells of size $\kappa_J := \pi/2^J$. However, the Shannon wavelet is poorly suited for sparse image representations, because of its slow decay rate. Moreover, it deviates from what is typically observed in freely-trained CNNs, because $W_l^{\lambda(J)}$ must be approximated with very large filters to avoid numerical instabilities. Practical implementations of DT-CWPT use fast-decaying filters such as these associated to Meyer wavelets [30], or finite-length filters that approximate the half-sample delay condition [41]. Therefore, energy is leaking outside the square cells tiling the Fourier domain. To counterbalance this, we increase the window size up to

$$ (B.21) \qquad \kappa_J := \frac{\pi}{2^{J-1}} = \pi/m_J, $$
<table><caption>Table 1. Energy concentration of the DT-CWPT filters within a Fourier window of size $κ_J × κ_J$, with $κ_J := π/2^{J-1}$.</caption><thead><tr><td>DEPTH J</td><td>BANDWIDTH κ<sub>J</sub></td><td>MEAN</td><td>STD</td></tr></thead><tbody><tr><td>2</td><td>π/2</td><td>0.98</td><td>0.00</td></tr><tr><td>3</td><td>π/4</td><td>0.95</td><td>0.02</td></tr></tbody></table>

and consider that (6.4) remains a reasonable approximation. Therefore, the conditions to apply Theorems 2.9, 3.11, and 4.2 are approximately satisfied in this context.

In order to numerically assess this assumption, we measured the maximum percentage of energy within a square window of size $κ_J × κ_J$ in the Fourier domain:

$$ (B.22) \qquad \rho_l^{\uparrow} := \frac{\max_{\theta \in [-\pi, \pi]^2} \left\| \mathbb{1}_{B_{\infty}(\theta, \kappa_J/2)} \widehat{W}_l^{\uparrow(J)} \right\|_{L^2}^2}{\left\| \widehat{W}_l^{\uparrow(J)} \right\|_{L^2}^2}, $$

where the $l^\infty$-ball $B_\infty(\theta, \kappa_J/2)$ is defined in the quotient space $[-\pi, \pi]^2/(2\pi\mathbb{Z}^2)$, as explained in Remark 2.1. If (6.4) is perfectly satisfied, then $\rho_l^{\uparrow} = 1$. The statistics computed over the collection $(\rho_l^{\uparrow}, \rho_l^{\downarrow})_{l \in \{0..4^{J-1}\}}$ are reported in Table 1.

**Remark B.6.** For “boundary filters”, i.e., when $\|\theta_l^{(J)}\|_\infty = (1 - 2^{-(J+1)}) \pi$, Remark 2.1 states that a small fraction of the filter’s energy remains located at the far end of the Fourier domain—see also [4]. Therefore, these filters do not strictly comply with the conditions of Theorems 2.9, 3.11, and 4.2. We nevertheless include them in our experiments.

REFERENCES

[1] K. B. ATHREYA AND S. N. LAHIRI, *Measure Theory and Probability Theory*, vol. 19, Springer, 2006.

[2] A. AZULAY AND Y. WEISS, *Why do deep convolutional networks generalize so poorly to small image transformations?*, Journal of Machine Learning Research, 20 (2019), pp. 1–25.

[3] R. BALAN, M. SINGH, AND D. ZOU, *Lipschitz Properties for Deep Convolutional Networks*, Contemporary Mathematics, 706 (2018), pp. 129–151.

[4] I. BAYRAM AND I. W. SELESNICK, *On the Dual-Tree Complex Wavelet Packet and M-Band Transforms*, IEEE Transactions on Signal Processing, 56 (2008), pp. 2298–2310.

[5] A. BIETTI, *Approximation and Learning with Deep Convolutional Models: A Kernel Perspective*, in International Conference on Learning Representations, 2022.

[6] A. BIETTI AND J. MAIRAL, *Invariance and Stability of Deep Convolutional Representations*, in Advances in Neural Information Processing Systems, 2017, p. 1622.

[7] A. BIETTI AND J. MAIRAL, *Group invariance, stability to deformations, and complexity of deep convolutional representations*, The Journal of Machine Learning Research, 20 (2019), pp. 876–924.

[8] A. BIETTI AND J. MAIRAL, *On the Inductive Bias of Neural Tangent Kernels*, in Advances in Neural Information Processing Systems, vol. 32, Curran Associates, Inc., 2019.

[9] J. BRUNA AND S. MALLAT, *Invariant Scattering Convolution Networks*, IEEE Transactions on Pattern Analysis and Machine Intelligence, 35 (2013), pp. 1872–1886.

[10] F. COTTER AND N. KINGSBURY, *A Learnable Scatternet: Locally Invariant Convolutional Layers*, in 2019 IEEE International Conference on Image Processing (ICIP), 2019, pp. 350–354.
[11] W. CZAJA AND W. LI, *Analysis of time-frequency scattering transforms*, Applied and Computational Harmonic Analysis, 47 (2019), pp. 149–171.

[12] W. CZAJA AND W. LI, *Rotationally Invariant Time-Frequency Scattering Transforms*, Journal of Fourier Analysis and Applications, 26 (2020), p. 4.

[13] F. GAMA, A. RIBEIRO, AND J. BRUNA, *Stability of Graph Scattering Transforms*, in Advances in Neural Information Processing Systems, vol. 32, Curran Associates, Inc., 2019.

[14] S. GAUTHIER, B. THÉRIEN, L. ALSÈNE-RACICOT, M. CHAUDHARY, I. RISH, E. BELILOVSKY, M. EICKENBERG, AND G. WOLF, *Parametric Scattering Networks*, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5749–5758.

[15] P. R. HALMOS, *Measure Theory*, Springer, 2013.

[16] J. HAVLICEK, J. HAVLICEK, AND A. BOVIK, *The analytic image*, in Proceedings of International Conference on Image Processing, vol. 2, 1997, pp. 446–449 vol.2.

[17] K. HUANG AND S. AVIYENTE, *Wavelet feature selection for image classification*, IEEE Transactions on Image Processing, 17 (2008), pp. 1709–1720.

[18] N. KINGSBURY, *Complex wavelets for shift invariant analysis and filtering of signals*, Applied and computational harmonic analysis, 10 (2001), pp. 234–253.

[19] N. KINGSBURY, *Design of Q-shift complex wavelets for image processing using frequency domain energy minimization*, in Proceedings International Conference on Image Processing, vol. 1, 2003, pp. I-1013.

[20] N. KINGSBURY AND J. MAGAREY, *Wavelet Transforms in Image Processing*, in Signal Analysis and Prediction, Applied and Numerical Harmonic Analysis, Birkhäuser, Boston, MA, 1998, pp. 27–46.

[21] A. LAINE AND J. FAN, *Texture Classification by Wavelet Packet Signatures*, IEEE Transactions on Pattern Analysis and Machine Intelligence, 15 (1993), pp. 1186–1191.

[22] Y. LECUN, Y. BENGIO, AND G. HINTON, *Deep learning*, Nature, 521 (2015), pp. 436–444.

[23] Y. LECUN, L. BOTTOU, Y. BENGIO, AND P. HAFFNER, *Gradient-based learning applied to document recognition*, Proceedings of the IEEE, 86 (1998), pp. 2278–2323.

[24] H. LETERME, K. POLISANO, V. PERRIER, AND K. ALAHARI, *Modélisation Parcimonieuse de CNNs avec des Paquets d'Ondelettes Dual-Tree*, in ORASIS, 2021.

[25] H. LETERME, K. POLISANO, V. PERRIER, AND K. ALAHARI, *From CNNs to Shift-Invariant Twin Models Based on Complex Wavelets*, 2023, https://arxiv.org/abs/2212.00394.

[26] B. LIAO AND F. PENG, *Rotation-invariant texture features extraction using Dual-Tree Complex Wavelet Transform*, in 2010 International Conference on Information, Networking and Automation (ICINA), vol. 1, 2010.

[27] S. MALLAT, *A Wavelet Tour of Signal Processing: The Sparse Way*, Academic Press, 2009.

[28] S. MALLAT, *Group invariant scattering*, Communications on Pure and Applied Mathematics, 65 (2012), pp. 1331–1398.

[29] S. MALLAT, *Understanding deep convolutional networks*, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374 (2016), p. 20150203.

[30] Y. MEYER, *Principe d'incertitude, bases hilbertiennes et algèbres d'opérateurs*, in Séminaire Bourbaki, vol. 662, Société Mathematique de France, 1985.

[31] G. P. NASON AND B. W. SILVERMAN, *The Stationary Wavelet Transform and some Statistical Applications*, in Wavelets and Statistics, Lecture Notes in Statistics, Springer, 1995, pp. 281–299.

[32] E. OYALLON, *Analyzing and Introducing Structures in Deep Convolutional Neural Networks*, doctoral thesis, Paris Sciences et Lettres, 2017.

[33] E. OYALLON, E. BELILOVSKY, S. ZAGORUYKO, AND M. VALKO, *Compressing the Input for CNNs with the First-Order Scattering Transform*, in Proceedings of the European Conference on Computer Vision (ECCV), 2018.

[34] K. I. PARK AND M. PARK, *Fundamentals of Probability and Stochastic Processes with Applications to Communications*, Springer, 2018.

[35] J. C. PÉREZ, M. ALFARRA, G. JEANNERET, A. BIBI, A. THABET, B. GHANEM, AND P. ARBELÁEZ, *Gabor Layers Enhance Network Robustness*, in Proceedings of the European Conference on Computer Vision (ECCV), 2020.

[36] M. PERLMUTTER, F. GAO, G. WOLF, AND M. HIRN, *Geometric Wavelet Scattering Networks on Compact Riemannian Manifolds*, in Proceedings of The First Mathematical and Scientific Machine Learning Conference, PMLR, 2020, pp. 570–604.
[37] S. PITTNER AND S. V. KAMARTHI, Feature extraction from wavelet coefficients for pattern recognition tasks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 21 (1999), pp. 83–88.

[38] M. RAI AND P. RIVAS, *A Review of Convolutional Neural Networks and Gabor Filters in Object Recognition*, in 2020 International Conference on Computational Science and Computational Intelligence (CSCI), 2020, pp. 1560–1567.

[39] O. RUSSAKOVSKY, J. DENG, H. SU, J. KRAUSE, S. SATHEESH, S. MA, Z. HUANG, A. KARPATHY, A. KHOSLA, M. BERNSTEIN, A. C. BERG, AND L. FEI-FEI, *ImageNet Large Scale Visual Recognition Challenge*, International Journal of Computer Vision, 115 (2015), pp. 211–252.

[40] M. SCETBON AND Z. HARCHAOUI, *Harmonic Decompositions of Convolutional Networks*, in International Conference on Machine Learning, PMLR, 2020, pp. 8522–8532.

[41] I. W. SELESNICK, R. BARANIUK, AND N. KINGSBURY, *The dual-tree complex wavelet transform*, IEEE Signal Processing Magazine, 22 (2005), pp. 123–151.

[42] C. SHANNON, *Communication in the Presence of Noise*, Proceedings of the IRE, 37 (1949), pp. 10–21.

[43] L. SIFRE AND S. MALLAT, *Rotation, Scaling and Deformation Invariant Scattering for Texture Discrimination*, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 1233–1240.

[44] A. SINGH AND N. KINGSBURY, *Dual-Tree wavelet scattering network with parametric log transformation for object classification*, in ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2017.

[45] C. SZEGEDY, W. ZAREMBA, I. SUTSKEVER, J. BRUNA, D. ERHAN, I. GOODFELLOW, AND R. FERGUS, *Intriguing properties of neural networks*, 2014, https://arxiv.org/abs/1312.6199.

[46] A. TORRALBA AND A. OLIVA, *Statistics of natural image categories*, Network: Computation in Neural Systems, 14 (2003), pp. 391–412.

[47] M. TYGERT, J. BRUNA, S. CHINTALA, Y. LECUN, S. PIANTINO, AND A. SZLAM, *A Mathematical Motivation for Complex-Valued Convolutional Networks*, Neural Computation, 28 (2016), pp. 815–825.

[48] M. VETTERLI, *Wavelets, approximation, and compression*, IEEE Signal Processing Magazine, 18 (2001), pp. 59–73.

[49] A. VIRMAUX AND K. SCAMAN, *Lipschitz regularity of deep neural networks: Analysis and efficient estimation*, in Advances in Neural Information Processing Systems, 2018.

[50] I. WALDSPURGER, *Wavelet Transform Modulus : Phase Retrieval and Scattering*, doctoral thesis, Ecole normale supérieure, Paris, 2015.

[51] T. WATOWSKI AND H. BÖLCSKEI, *A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction*, IEEE Transactions on Information Theory, 64 (2018), pp. 1845–1866.

[52] G. G. YEN, *Wavelet packet feature extraction for vibration monitoring*, IEEE Transactions on Industrial Electronics, 47 (2000), pp. 650–667.

[53] J. YOSINSKI, J. CLUNE, Y. BENGIO, AND H. LIPSON, *How transferable are features in deep neural networks?*, in Advances in Neural Information Processing Systems, 2014, pp. 3320–3328.

[54] J. ZARKA, F. GUTH, AND S. MALLAT, *Separation and Concentration in Deep Networks*, in International Conference on Learning Representations, 2021.

[55] J. ZARKA, L. THIRY, T. ANGLES, AND S. MALLAT, *Deep Network Classification by Scattering and Homotopy Dictionary Learning*, in International Conference on Learning Representations, 2020.

[56] R. ZHANG, *Making Convolutional Networks Shift-Invariant Again*, in International Conference on Machine Learning, 2019.

[57] D. ZOU, R. BALAN, AND M. SINGH, *On Lipschitz Bounds of General Convolutional Neural Networks*, IEEE Transactions on Information Theory, 66 (2020), pp. 1738–1759.

[58] D. ZOU AND G. LERMAN, *Graph convolutional neural networks via scattering*, Applied and Computational Harmonic Analysis, 49 (2020), pp. 1046–1074.

[59] X. ZOU, F. XIAO, Z. YU, Y. LI, AND Y. J. LEE, *Delving Deeper into Anti-Aliasing in ConvNets*, International Journal of Computer Vision, 131 (2023), pp. 67–81.