Repository: speechbrain/speechbrain

Issue:
spkrec-ecapa-voxceleb-mel-spec model modifies mel spectrum in place when used with CPU
### Describe the bug

Thanks for your wonderful project. I would love to make my first speechbrain contribution and help fix this issue :). when I encode a mel spectrogram using a CPU using MelSpectrogramEncoder, the underlying mel spectrogram data is modified

### Expected behaviour

Calculating the embedding should not modify the original data

### To Reproduce

test_in_place.py
```python
from speechbrain.pretrained import MelSpectrogramEncoder
import torch
from torch.nn.functional import mse_loss
import torchaudio

# On CPU
waveform, sr = torchaudio.load("tests/samples/TTS/LJ050-0131.wav")
waveform = torchaudio.functional.resample(waveform, sr, 16000)
spk_emb_encoder = MelSpectrogramEncoder.from_hparams(source="speechbrain/spkrec-ecapa-voxceleb-mel-spec", savedir="spk_emb_encoder_checkpoints", run_opts={"device": "cpu"})
mel = spk_emb_encoder.mel_spectogram(waveform)
mel_copy = torch.clone(mel)
spkr_emb = spk_emb_encoder.encode_mel_spectrogram(mel_copy)
print(mse_loss(mel, mel_copy))

# On CUDA device
waveform, sr = torchaudio.load("tests/samples/TTS/LJ050-0131.wav")
waveform = torchaudio.functional.resample(waveform, sr, 16000)
spk_emb_encoder = MelSpectrogramEncoder.from_hparams(source="speechbrain/spkrec-ecapa-voxceleb-mel-spec", savedir="spk_emb_encoder_checkpoints", run_opts={"device": "cuda"})
mel = spk_emb_encoder.mel_spectogram(waveform)
mel_copy = torch.clone(mel)
spkr_emb = spk_emb_encoder.encode_mel_spectrogram(mel_copy)
print(mse_loss(mel, mel_copy))
```
python tests/test_in_place.py 
tensor(33.1424)
tensor(0.)

### Environment Details

speechbrain '0.5.16'

### Relevant Log Output

_No response_

### Additional Context

It only happens with CPU


Discussion:
Thanks for the convenient repro. I would note that this is not really specific to the CPU device: The bug seems to be in-place modification of the input tensor.

Since you're passing a CPU tensor in the second example, it gets migrated to CUDA and thus effectively cloned. If you `.to("cuda")` the waveform yourself the bug also reproduces.
The bug is in `InputNormalization` when the `norm_type` is either `"sentence"` or `"speaker"`, in which case it performs in-place assignments. I will make a PR soon.

Resolving patch:
diff --git a/speechbrain/processing/features.py b/speechbrain/processing/features.py
--- a/speechbrain/processing/features.py
+++ b/speechbrain/processing/features.py
@@ -1082,6 +1082,11 @@ def forward(self, x, lengths, spk_ids=torch.tensor([]), epoch=0):
         current_means = []
         current_stds = []
 
+        if self.norm_type == "sentence" or self.norm_type == "speaker":
+            # we will do in-place slice assignments over `out`
+            out = torch.empty_like(x)
+        # otherwise don't assign it yet
+
         for snt_id in range(N_batches):
             # Avoiding padded time steps
             actual_size = torch.round(lengths[snt_id] * x.shape[1]).int()
@@ -1095,7 +1100,7 @@ def forward(self, x, lengths, spk_ids=torch.tensor([]), epoch=0):
             current_stds.append(current_std)
 
             if self.norm_type == "sentence":
-                x[snt_id] = (x[snt_id] - current_mean.data) / current_std.data
+                out[snt_id] = (x[snt_id] - current_mean.data) / current_std.data
 
             if self.norm_type == "speaker":
                 spk_id = int(spk_ids[snt_id][0])
@@ -1141,14 +1146,14 @@ def forward(self, x, lengths, spk_ids=torch.tensor([]), epoch=0):
                         speaker_mean = current_mean.data
                         speaker_std = current_std.data
 
-                x[snt_id] = (x[snt_id] - speaker_mean) / speaker_std
+                out[snt_id] = (x[snt_id] - speaker_mean) / speaker_std
 
         if self.norm_type == "batch" or self.norm_type == "global":
             current_mean = torch.mean(torch.stack(current_means), dim=0)
             current_std = torch.mean(torch.stack(current_stds), dim=0)
 
             if self.norm_type == "batch":
-                x = (x - current_mean.data) / (current_std.data)
+                out = (x - current_mean.data) / (current_std.data)
 
             if self.norm_type == "global":
                 if self.training:
@@ -1175,9 +1180,11 @@ def forward(self, x, lengths, spk_ids=torch.tensor([]), epoch=0):
 
                     self.count = self.count + 1
 
-                x = (x - self.glob_mean.data.to(x)) / (self.glob_std.data.to(x))
+                out = (x - self.glob_mean.data.to(x)) / (
+                    self.glob_std.data.to(x)
+                )
 
-        return x
+        return out
 
     def _compute_current_stats(self, x):
         """Computes mean and std


Test patch: