Repository: kymatio/kymatio

Issue:
BUG 1D filter resolutions introduces energy issue at octaves
While developing JTFS, we observed some artefacts that appeared around octave boundaries. Moreover, we observed that the energy of a signal that should have constant energy across `n1` decays with increasing `j`. 

This holds for `Scattering1D` as well as JTFS and is exemplified by visualization of the S1 of an dirac or chirp:

```
from kymatio.torch import Scattering1D
import librosa 

def plot_spec(Sx, S, sr=22050):
    x_coords = librosa.times_like(Sx, hop_length=S.T)
    y_coords = [psi["xi"]*sr for psi in S.psi1_f]

    plt.figure(figsize=(8, 4))
    librosa.display.specshow(Sx[1:,:].numpy(), sr=sr,
        x_coords=x_coords, x_axis="time",
        y_coords=y_coords, y_axis="cqt_hz",
        cmap="magma")
    plt.xlabel("Time (seconds)")
    plt.ylabel("Frequency (Hz)")
    plt.ylim(0, sr // 2)
    # plt.yticks([100, 200, 400, 800, 1600, 3200, 6400])
    plt.xticks(np.arange(0, 1.2, 0.2))
    plt.minorticks_off()

N = 2**15
kwargs = {'shape': N, 'Q': (12, 1), 'J': 8}
S = Scattering1D(**kwargs, max_order=1)

x = torch.zeros(N)
x[N // 2] = 1

Sx = S(x)

plot_spec(Sx, S, sr=2**15)
```

![image](https://user-images.githubusercontent.com/10761938/211219386-4863763e-5918-4f2f-90a1-41ee13f9efa8.png)

![image](https://user-images.githubusercontent.com/10761938/211219482-fefbed08-fb3c-415e-b6f1-dd436b695dc8.png)

And the spectrum at the time slice centred around the impulse:

![image](https://user-images.githubusercontent.com/10761938/211219419-036ffa6b-2d2d-44a3-9ef7-364745175efd.png)

Which appears to decay by a factor of 2 at octave boundaries ... an energy decay by roughly `2**k1`

This discontinuity at octave boundaries did not cause problems at second-order for `Scattering1D`, but is an issue for second-order of JTFS because we do wavelet conv across octaves. 

The chirp below covers 3 octaves: 

![image](https://user-images.githubusercontent.com/10761938/211219647-1e6d2d56-6d40-4018-897b-4a77d5fa5d34.png)

What results are some energy artifacts exactly at the octave boundaries:

![image](https://user-images.githubusercontent.com/10761938/211219661-af734f3a-256b-4e22-9706-2ed92e8a8153.png)


We identified some relationship between this and `T`. We tried normalizing the wavelet filters to have constant peak. This worked for computing U1, but not S1. 

We realzing that when we create resolutions of `psi` and `phi`, we subsample by taking a mean, but across different signal lengths. This introduces an energy decay correlated with `2**level`.
https://github.com/kymatio/kymatio/blob/954c44990ca2edcd9b4cc12c20321dd20d27cce6/kymatio/scattering1d/filter_bank.py#L382 
https://github.com/kymatio/kymatio/blob/954c44990ca2edcd9b4cc12c20321dd20d27cce6/kymatio/scattering1d/filter_bank.py#L395 

#983 proposes to solve this by changing the `mean` to a `sum` to preserve l1 norm. we get a nice response at `S1` for a dirac:

![image](https://user-images.githubusercontent.com/10761938/211219812-fe1d271c-48ff-4633-8579-e3448d9658f4.png)

![image](https://user-images.githubusercontent.com/10761938/211219917-d12f6865-6998-4637-b992-a22fe180cd33.png) 

The difference between `n1` and `n1_max` is not huge and the decay is smooth rather than these abrupt and steep jumps between octaves.


Discussion:

Resolving patch:
diff --git a/kymatio/scattering1d/filter_bank.py b/kymatio/scattering1d/filter_bank.py
--- a/kymatio/scattering1d/filter_bank.py
+++ b/kymatio/scattering1d/filter_bank.py
@@ -319,7 +319,7 @@ def spinned_fn(J, Q, **kwargs):
     return spinned_fn, filterbank_kwargs
 
 
-def scattering_filter_factory(N, J, Q, T, filterbank):
+def scattering_filter_factory(N, J, Q, T, filterbank, _reduction=np.mean):
     """
     Builds in Fourier the Morlet filters used for the scattering transform.
 
@@ -351,6 +351,8 @@ def scattering_filter_factory(N, J, Q, T, filterbank):
         **filterbank_kwargs as optional keyword arguments.
         Corresponds to the self.filterbank property of the scattering object.
         As of v0.3, only anden_generator is supported as filterbank_fn.
+    _reduction : callable
+        either np.mean (default) or np.sum.
 
     Returns
     -------
@@ -379,7 +381,8 @@ def scattering_filter_factory(N, J, Q, T, filterbank):
             # Resample to smaller resolutions if necessary (beyond 1st layer)
             # The idiom min(previous_J, j, 1+log2_T) implements "j1 < j2"
             for level in range(1, min(previous_J, j, 1+log2_T)):
-                psi_level = psi_levels[0].reshape(2 ** level, -1).mean(axis=0)
+                psi_level_reshaped = psi_levels[0].reshape(2**level, -1)
+                psi_level = _reduction(psi_level_reshaped, axis=0)
                 psi_levels.append(psi_level)
             psi_f.append({'levels': psi_levels, 'xi': xi, 'sigma': sigma, 'j': j})
             max_j = max(j, max_j)
@@ -392,7 +395,8 @@ def scattering_filter_factory(N, J, Q, T, filterbank):
     sigma_low = filterbank_kwargs["sigma0"] / T
     phi_levels = [gauss_1d(N, sigma_low)]
     for level in range(1, max(previous_J, 1+log2_T)):
-        phi_level = phi_levels[0].reshape(2 ** level, -1).mean(axis=0)
+        phi_level_reshaped = phi_levels[0].reshape(2**level, -1)
+        phi_level = _reduction(phi_level_reshaped, axis=0)
         phi_levels.append(phi_level)
     phi_f = {'levels': phi_levels, 'xi': 0, 'sigma': sigma_low, 'j': log2_T,
         'N': N}
diff --git a/kymatio/scattering1d/frontend/base_frontend.py b/kymatio/scattering1d/frontend/base_frontend.py
--- a/kymatio/scattering1d/frontend/base_frontend.py
+++ b/kymatio/scattering1d/frontend/base_frontend.py
@@ -25,6 +25,7 @@ def __init__(self, J, shape, Q=1, T=None, max_order=2,
         self.oversampling = oversampling
         self.out_type = out_type
         self.backend = backend
+        self._reduction = np.mean
 
     def build(self):
         """Set up padding and filters
@@ -91,7 +92,7 @@ def build(self):
     def create_filters(self):
         # Create the filters
         self.phi_f, self.psi1_f, self.psi2_f = scattering_filter_factory(
-            self._N_padded, self.J, self.Q, self.T, self.filterbank)
+            self._N_padded, self.J, self.Q, self.T, self.filterbank, self._reduction)
         ScatteringBase._check_filterbanks(self.psi1_f, self.psi2_f)
 
     def scattering(self, x):
@@ -535,6 +536,7 @@ def __init__(self, *, J, J_fr, shape, Q, T=None, oversampling=0,
         self.F = F
         self.oversampling_fr = oversampling_fr
         self.format = format
+        self._reduction = np.sum
 
     def build(self):
         super(TimeFrequencyScatteringBase, self).build()
@@ -574,9 +576,10 @@ def build(self):
 
     def create_filters(self):
         phi0_fr_f,= scattering_filter_factory(self._N_padded_fr,
-            self.J_fr, (), self.F, self.filterbank_fr)
+            self.J_fr, (), self.F, self.filterbank_fr, self._reduction)
         phi1_fr_f, psis_fr_f = scattering_filter_factory(self._N_padded_fr,
-            self.J_fr, self.Q_fr, 2**self.J_fr, self.filterbank_fr)
+            self.J_fr, self.Q_fr, 2**self.J_fr, self.filterbank_fr,
+            self._reduction)
         self.filters_fr = (phi0_fr_f, [phi1_fr_f] + psis_fr_f)
 
         # Check for absence of aliasing


Test patch: