How to classify sounds using Pytorch
By: Soumo Chatterjee
Tags: Audio Classification, Python, Sound Classification, Pytorch, Deep Learning

In this article , I am going to talk about how we can prepare the sound or, audio dataset so that we can able to classify them . It may be a little long but if you can hold patience and read it thoroughly once, it will be highly beneficial.

First we will load the audio file. From the directory having n number of sounds files, we will try to load 2–3 out of them using torchaudio.load first. torchaudio supports sound files of format ‘.wav’ and ‘.mp3’ which is used to give waveform and sample rate of the sound file. Waveform consists of frequencies of the sound per frame in an array format whereas the sample rate determines the frequency at which the waveform can be represented.

import torchaudio

waveform, sample_rate = torchaudio.load("_PATH OF THE AUDIO FILE_")

If the audio file is in other format then try googling to extract out waveform, sample_rate of the audio file. For example, if you have ‘.flac’ audio file you can try this : →

import soundfile as sf

waveform, sample_rate = sf.read("_PATH OF THE AUDIO FILE IN FLAC_")

2. Normalize all the shape of waveforms to one size. After loading the file, check the shape of the waveform using waveform.size()[0] . If it’s value is more than 1 , then we will have to normalize it using

from pydub import AudioSegment waveform = AudioSegment.from_mp3(_PATH OF THE AUDIO FILE_)

waveform = waveform.set_channels(1)

waveform = waveform.get_array_of_samples()

waveform = torch.tensor(waveform, dtype = torch.float)

waveform = torch.reshape(waveform, (1,waveform.shape[0]))

3. Change the waveform to Spectrogram, Mel Spectrogram or, MFCC. Now we will change waveform into Spectrogram(a visual representation of the spectrum of frequencies of a signal as it varies with time) using

Spectrogram = torchaudio.transforms.Spectrogram()(waveform)

or, mel spectrogram(a representation of the short-term power spectrum of a sound, based on a linear cosine transform of a log power spectrum on a nonlinear mel scale of frequency) using

Mel_Spectrogram = torchaudio.transforms.MelSpectrogram()(waveform)

or, MFCC(Mel-frequency cepstral coefficients (MFCCs) are coefficients that collectively make up an mel-frequency cepstrum. Mel-frequency cepstrum is a representation of the short-term power spectrum of a sound, based on a linear cosine transform of a log power spectrum on a nonlinear mel scale of frequency) using

n_fft = 400.0

frame_length = n_fft / sample_rate * 1000.0

frame_shift = frame_length / 2.0



params = {

"channel": 0,

"dither": 0.0,

"window_type": "hanning",

"frame_length": frame_length,

"frame_shift": frame_shift,

"remove_dc_offset": False,

"round_to_power_of_two": False,

"sample_frequency": sample_rate,

}

mfcc = torchaudio.compliance.kaldi.mfcc(waveform, **params)

4. Finally we can create the dataset class using the above 3 points like this. #1#Define the dataset class name first .

class audio_classification(torch.utils.data.Dataset):

#2# Define the class constructor to define audio_ids , their classification class_ids in a list and applying augmentations to them

def __init__(self, ids, recording_id, class_id, required_columns, is_valid = 0):

self.ids = ids

self.audio_ids = audio_ids

self.class_id = class_id

self.required_columns = required_columns

self.is_valid = is_valid

if self.is_valid == 1:

self.aug = # transfoms for validation images

else:

self.aug = # transfoms for training images

#3# Define the __len__ function

def __len__(self):

return len(self.ids)

#4# Finally define the __getitem__ function

def __getitem__(self, index):

filename = "__PATH OF THE AUDIO__"+ self.audio_ids[index] +"__EXTENSION OF THE AUDIO FILE__" waveform , _ = torchaudio.load(filename) # reshape to 1 channel if required

waveform = torch.reshape(waveform, (1, waveform.shape[0])) # Change the waveform to Spectrogram, Mel-Spectrogram or,MFCC

specgram = torchaudio.transforms.Spectrogram()(waveform)



# convert 1 channel to 3 channels applying imagenet models

specgram = specgram.repeat(3, 1, 1)



# Applying audio augmentations by converting to numpy array

specgram = np.transpose(specgram.numpy(), (1,2,0))

specgram = self.aug(image = specgram)['image']



#As torch accepts channels first so applying this

specgram = np.transpose(specgram,(2,0,1)).astype(np.float32)



return {

'specgram' : torch.tensor(specgram, dtype = torch.float) 'label' : torch.tensor(self.class_id[index], dtype = torch.float)

}

After doing all these steps this will completely act as an image classification problem. And we can define dataloaders , models , loss function, optimizer and training & validation process after this and start training our model consequently like we do in Image classification problem.

I hope you have understood this article well , if there is any questions related to this article then please feel free to comment on this article, until then enjoy learning.