Freesound Audio Tagging — Recognizing Sounds of Various Natures
By: Divyansh Jain
Tags: Machine Learning, Deep Learning, Data Science, Audio, Kaggle Competition

7. Modeling

Initially, we build our models using the curated training dataset only as these clips are quite clear and audible. Also, the data is less (~5000 clips) as compared to the noisy set (~20,000 clips) and thus the models will be trained faster.

Later on, we will try training the models on the complete dataset (Curated + Noisy) and check if there’s any significant increase in the model’s performance.

Since we have time-series data, we’ll try deep learning models as these are quite strong in extracting hidden features from raw data, given there is enough data.

7.1 Train-Validation split

We perform a 70–30 train-validation split which is not random, but, stratified such that both train and validation dataset contains approximately the same distribution of the number of labels per clip. The above strategy helps to reduce the train-validation set differences.

Stratified Train-Validation Split

We can verify if the split works as expected:

Get the percentage of clips having a given number of labels in the input dataframe

Distribution of the number of labels in train and validation dataframes

As we observe, the train and validation datasets have the same distribution of the number of labels per clip.

Since this is a Kaggle competition, we have a separate test dataset (~3,300 clips) which we will use to evaluate each model’s performance.

7.2 Data preparation for Model-1

After this, we read the contents of the preprocessed .wav files produced in Step 5 using scipy.io.read(). We can have multiple ground truth labels for each clip which we convert into multi-hot-encoded vectors.

All the data is converted into TensorFlow Datasets format. This allows proper utilization of both CPU and GPU so that the idle time for each of these is minimum (this link can be referred to for more advantages of TF Datasets).

The detailed code for conversion of the .wav files into TF datasets can be found on my Github Profile.

As mentioned in Step 5, we used a sampling rate of 16 kHz (16,000 samples per second) and sampled/padded each clip to the same length of 15 seconds. So, each clip/datapoint now contains 16,000 samples/second * 15 seconds = 2,40,000 samples in total. Each sample here becomes a feature for the ML model.

Example datapoint

7.3 Model-1 (1-D CNN)

We use a 1-dimensional convolutional model (architecture is inspired by this kernel). It performs temporal convolution (along one axis only) on raw time-series features. We prefer to use 1-D CNN over RNN/LSTM because these are much faster to train and give better results. Max-pooling layers are also used to downsample, so as to reduce overfitting. Appropriate dropouts are also used for the same purpose. ReLu activation function is used everywhere except the last layer where we use the Sigmoid activation function since we want independent probabilities for each class.

Since this is a multilabel classification task where multiple labels such as “Crowd”, “Cheering”, “People” can be present in the same clip, we use sigmoid and not softmax, the main difference between these two is, sigmoid predicts the probability of the clip belonging to each label independently, i.e., the clip can have a probability of belonging to class “Crowd” equal to 0.7 and at the same time, the probability of it belonging to the class “Cheering” can be 0.6, while, softmax assumes each clip belongs to only one label (that is, all probabilities are dependent on each other so that the total sum is 1). This is a very nice article explaining the differences between sigmoid and softmax.

We use the Categorical Crossentropy loss function, a batch size of 64 along with the Adam optimizer to train the model.

Model-1 (1-D CNN)

Architecture of Model-1

We train the model for 50 epochs, store the weights after each epoch, and observe the train and validation curves to be as follows: