How to create a custom dataset for audio recognition

username: tiagoovieira
cooked: <p>Hi,</p>
<p>I have a question, I have a dataset of audiofiles that I’d like to convert into melspectogram and I want to use tourchaudio library to convert audio into a tensor directly. I’ve seen some people doing this by saving as an image, and I’d like to bypass that step, and train directly as a tensor. My question is, how should I do regarding, creating a Dataloader so that I can do this computational expensive operation and taking advantage of the fact that each folder is the label that contains all the training data for that class. The end goal is create a classification algorithm</p>
<p>Thanks so much!</p>

username: ptrblck
cooked: <p>You could create a custom <code>Dataset</code> as explained in <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">this tutorial</a> and apply the transformation on each sample in the <code>__getitem__</code> of this class. To create the class indices based on the folder structure you could reuse the logic from <a href="https://github.com/pytorch/vision/blob/5339e63148304ce32fd1cbd1e8bb74ea79458691/torchvision/datasets/folder.py#L107-L241"><code>DatasetFolder</code></a>.</p>

username: vincentqb
cooked: <p>Similarly to the previous answer, you can also checkout the <a href="https://pytorch.org/tutorials/intermediate/speech_command_recognition_with_torchaudio.html" rel="noopener nofollow ugc">audio classification tutorial</a> and update the line <code>tensors += [waveform]</code> in <code>collate_fn</code> to <code>tensors += [transform(waveform)]</code> where <code>transform</code> is whatever transform you want.</p>
<p>If your goal is to apply the transform, save the transformed waveform to disk to avoid recomputing it later, and then create a new dataset with this, then you could also try to cache dynamically your dataset using something like <a href="https://github.com/pytorch/audio/blob/master/torchaudio/datasets/utils.py#L235" rel="noopener nofollow ugc"><code>diskcache_iterator</code></a>.</p>

username: kkoutini
cooked: <p>you can use something simple like this:</p>
<pre><code class="lang-auto">from torch.utils.data import Dataset as TorchDataset

class SpectrogramDataset(TorchDataset):

    def __init__(self,file_label_ds,  process_func, audio_path=""):
        self.ds= file_label_ds
        self.process_func = process_func
        self.audio_path=audio_path
    def __getitem__(self, index):
        file,label=self.ds[index]
        x=self.process_func(self.audio_path+file)
        return x, file, label
    def __len__(self):
        return len(self.ds)
</code></pre>
<p><code>file_label_ds</code> is a dataset that gives you the file name and label.<br>
<code>process_func</code> is a function that takes the full audio path and return the spectrogram.<br>
PS: this helps you to extract the spectrograms in parallel using the CPU (if you have <code>num_workers&gt;0</code>),  this won’t work if you use the gpu to extract the spectrograms.</p>

username: tiagoovieira
cooked: <p>Great stuff, truy appreciate the pointers, I did a rough implementation based on your suggestions! I have a small issue, and confirm if I’m thinking about this in the right way! I saw some examples where in the pre-process step, converts audio into images, but I’d like to convert straight into a tensor. And also, keep the dataset with the fix size. I have audiofiles with different durations. How can I accomplish this in the context of spectrogram?</p>
<p>If I understood correctly, there are two different ways I can create this dataloader, one is by creating a class, and overriding the <strong>getitem</strong> method, and the other is passing a collate_fn method. Is this a fair assessment?</p>