Bird Song Classification using Siamese Networks and Dilated Convolutions
By: Aditya Dutt
Tags: Audio Classification, Dilated Convolution, Birds, Deep Learning, Siamese Networks

Introduction

Bioacoustics is very useful in studying the environment. It has been used for a long time to track submarines and whales. Birds help a lot to shape the plant life we see around us. Recognizing bird songs is very important for automatic monitoring of wildlife and studying the behavior of birds. It can help in tracking birds without disturbing them. We can also tell which birds species exist at a particular place. It can give us some information about their migration pattern. Every species of birds have their unique sounds. They use songs of varying length and complexity to attract mates, warn other birds of nearby danger, and mark their territory. Songbirds can have different dialects based on their geographical location. But sounds from non-songbirds do not change much based on geography. Using Deep learning methods, we can easily classify birds based on their calls and songs. We can use any of the neural network frameworks like CNNs, Siamese Networks, WaveNets, etc. for this task.

Problem Statement

Goal: We want to classify different birds species given their song/ call audio samples. We can extract spectrograms of the audio samples and use them as features for classification. We will use the British Birdsong Dataset available on Kaggle for this experiment. The dataset is described in the section Data Description.

A Quick Introduction to Siamese Networks and Dilated Convolutions

Siamese Networks

I have written an article on Siamese Network before. You can check it out to get a deeper understanding of its working and loss functions used for it. Code is also provided in that article. But, I will give a summary of the Siamese Network here.

A Siamese network is a class of neural networks that contains one or more identical networks. We feed a pair of inputs to these networks. Each network computes the features of one input. And, then the similarity of features is computed using their difference or the dot product. For same class input pairs, the target output is 1 and for different classes input pairs, the output is 0. Remember, both networks have same the parameters and weights. If not, then they are not Siamese.

Siamese Network basic structure

Different loss functions can be used for a Siamese network.

Contrastive Loss: In Contrastive loss, a pair of inputs is taken. For same class pairs, distance is less between them. For different pairs, distance is more. Although binary cross-entropy seems like a perfect loss function for our problem, contrastive loss does a better job differentiating between image pairs. Contrastive loss, L = Y * D² + (1-Y) * max(margin — D, 0)² Triplet Loss: Triplet loss was introduced by Google in 2015 for face recognition. Here, the model takes three inputs- anchor, positive, and negative. The anchor is a reference input. Positive input belongs to the same class as anchor input. Negative input belongs to a random class other than the anchor class. We have to minimize the distance between the anchor and the positive sample and simultaneously maximize the distance between the anchor and the negative sample.

We will use triplet loss for our experiment.

Dilated Convolutions

Dilated Convolutions are a type of convolution that “inflate” the kernel by inserting holes between the kernel elements. They are also called atrous convolutions.

The concept of Dilated Convolution came from the wavelet decomposition in which the mother wavelet is scaled or dilated by different scales to capture different frequencies.

(a) Standard 3 x 3 kernel, (b) Kernel with a dilation factor 2, (c) Kernel with a dilation factor of 4. Source Original paper

Dilated convolutions increase the receptive field with the same computation and memory costs and without the loss of resolution. It can capture context from the entire input with the same number of parameters.

Here is a nice article on Dilated Convolutions by Sik-Ho Tsang.

Data Description

We are using the British Birdsong Dataset available on Kaggle for this experiment. It is a small subset gathered from the Xeno Canto data collection to form a balanced dataset of 88 bird species in the United Kingdom.

We are only classifying 9 bird species here: Canada Goose, Carrion Crow, Coal Tit, Common Blackbird, Common Chaffinch, Common Chiffchaff, Common Linnet, Common Moorhen, and Common Nightingale.

There are very few samples in this dataset for each bird. The audio samples are around 40-60 seconds long. Some of them are a little noisy and sometimes there are other birds in the background. Clips of 2 seconds are extracted from each sample with 50% overlap to create new samples. This will create a sufficient number of samples for training the neural network. Data is divided into 60% for training, and 40% for testing.

Implementation

Feature Extraction

Librosa library in python is used for music and audio analysis. We can read audio files and extract spectrograms using it.

Step 1: Read the audio file using librosa. Normalize the time series between -1 and 1.

Step 2: Remove silence from audio.

Step 3: Split each audio file into 2 second long clips with a 50% overlap.

Step 4: Divide the data into training and testing. 60% is used for training and 40% for testing.

Step 5: Extract spectrograms from samples. A filter is applied to the spectrograms to get a frequency range between 1KHz and 8KHz as most bird songs have frequencies in that range. Now, standardize all the spectrograms (You can normalize them also between 0 and 1). Here, the shape of each spectrogram is 163 x 345.