How to normalize audio data in PyTorch?

Haad_Bhutta: <p>I have many .mp3 audio recordings of people saying the same sentence. These people have different vocal ranges. So my question is:</p>
<p>In the data preprocessing stage, is it necessary to “normalize” the audio samples to be in one vocal range? If so, how would I go about doing this in PyTorch? I couldn’t find anything relevant in the documentation other than the method <code>torchaudio.sox_effects.apply_effects_tensor()</code>. But I don’t know how to use it properly. I am completely new to PyTorch. Any help would be great.</p>

francescopapaleo: <p>Hi,<br>
I am also a newbie in Pytorch, and I’ve been facing the same questions.<br>
The best I could find is to apply a sequence of transforms while loading the data with my Dataset class, in this case <code>EgfxDataset</code>. Assuming that in your Dataset <code>__init__</code> you have included the transforms:</p>
<pre><code class="lang-auto">class EgfxDataset(Dataset):
    """Egfx dataset
    Args:
        data_dir (str): Path to the data directory
        transforms (list): List of transforms to apply to the audio samples
    
    Returns:
        torch.utils.data.Dataset: Dataset object containing tuples of dry and wet audio samples
    """
    def __init__(self,
                 data_dir,
                 transforms=None
                ):
        self.data_dir = Path(data_dir) / 'egfxset'
        self.transforms = transforms
</code></pre>
<p>You can define the transforms to reduce dynamic range, correct DC and peak normalize between -1.0 and +1.0:</p>
<pre><code class="lang-auto">def contrast (tensor):
    return torchaudio.functional.contrast(tensor, 50)

def correct_dc_offset(tensor):
    return tensor - torch.mean(tensor)

def peak_normalize(tensor):
    tensor /= torch.max(torch.abs(tensor))
    return tensor

# I create a list of the transforms I want to apply:
TRANSFORMS = [contrast, correct_dc_offset, peak_normalize]

# When you instanciate your Dataset:
dataset = EgfxDataset(data_dir=data_dir, transforms=TRANSFORMS)
</code></pre>
<p>If anyone has more tools to suggest or corrections to point out I am listening.<br>
Thanks</p>

J_Johnson: <p>When using <code>torchaudio.load</code>, the waveform data is already normalized between -1.0 and 1.0.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/4/1/4123ffdaf150688f9012a8b43531e546740e5f03.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/4123ffdaf150688f9012a8b43531e546740e5f03" title="IMG_20230921_165510"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/4123ffdaf150688f9012a8b43531e546740e5f03_2_382x500.jpeg" alt="IMG_20230921_165510" data-base62-sha1="9igaasEokuFuaxa8YYdQY5FpiYb" width="382" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/4123ffdaf150688f9012a8b43531e546740e5f03_2_382x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/4123ffdaf150688f9012a8b43531e546740e5f03_2_573x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/4/1/4123ffdaf150688f9012a8b43531e546740e5f03_2_764x1000.jpeg 2x" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">IMG_20230921_165510</span><span class="informations">1080×1411 156 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>From there you could apply further normalization, if you wish. Or you could just let the model handle it. In image processing, it is actually ideal to augment the data with various transforms in order to make the model more robust to various levels of tone, contrast, brightness, discoloration, etc.</p>
<p>You could apply a BatchNorm1d on the inputs. For example:</p>
<pre><code class="lang-auto">self.norm = nn.BatchNorm1d(1)

...

def forward(self, x):
    x = self.norm(x)
    ...
</code></pre>
<p><a href="https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm1d.html#torch.nn.BatchNorm1d" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.BatchNorm1d.html#torch.nn.BatchNorm1d</a></p>
<p>Additionally, if you have varying sample rates, you may want to consider conditionally adding an upsampler so they are all uniform sample rate.</p>
<p><a href="https://pytorch.org/docs/stable/generated/torch.nn.Upsample.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/generated/torch.nn.Upsample.html</a></p>

francescopapaleo: <p>Thanks a lot for pointing that out!<br>
Adding that line from the <code>audio_io_tutorial</code> to the <a href="https://pytorch.org/audio/stable/backend.html#torchaudio.backend.sox_io_backend.load" rel="noopener nofollow ugc">documentation</a> would clarify a lot.</p>
<p><code>BatchNorm1d</code> is very useful, but in some cases you can’t use large batch sizes and you want to reduce the global dynamic range of a dataset. The contrast transform helps a bit, working as a broadband compressor, another solution might be loudness normalization. There are packages that allow loudness normalization on <code>NumPy</code> arrays, but I have not yet had the opportunity to test whether it would significantly increase loading time. If there is a possibility to apply loudness normalization directly on tensors, I would be very interested in trying it.</p>