# Deep Learning for Audio Event Detection and Tagging on Low-Resource Datasets
## 3 Training Methods  
We investigate three different methods to train the two intermediate tasks. One of them is the simple and usual approach of training each network independently for each task. Additionally, two multi-task learning (MTL) methods were tested, namely joint training and tied weights training,  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Layer** & **Size** & **\#Fmaps** & **Activation** & **12\_regularisation** \\ \hline Convolution 2D & 3 \(\times\) 3 & 64 & Linear & 0.001 \\ Batch Normalisation & - & - & - & - \\ Activation & - & - & ReLU & - \\ Convolution 2D & 3 \(\times\) 3 & 64 & Linear & 0.001 \\ Batch Normalisation & - & - & - & - \\ Activation & - & - & ReLU & - \\ Convolution 2D & 3 \(\times\) 3 & 64 & Linear & 0.001 \\ Convolution 2D & 3 \(\times\) 3 & 64 & Linear & 0.001 \\ Batch Normalisation & - & - & - & - \\ Convolution 2D & 3 \(\times\) 3 & 64 & Linear & 0.001 \\ Batch Normalisation & - & - & - & - \\ Activation & - & - & ReLU & - \\ Max Pooling & 1 \(\times\) 4 & - & - & - \\ Convolution 2D & 3 \(\times\) 3 & 64 & Linear & 0.001 \\ Batch Normalisation & - & - & - & - \\ Activation & - & - & ReLU & - \\ Max Pooling & 1 \(\times\) 2 & - & - & - \\ Global Average Pooling 2D & - & - & - & - \\ Dense & \#labels & - & Sigmoid & 0.001 \\ \hline \hline \end{tabular}
\end{table}
Table 2: WHO network architecture. Size refers to either kernel shape or number of units. #Fmaps is the number of feature maps in the layer. Activation denotes the activation used for the layer and 12_regularisation the amount of l2 kernel regularisation used in the layer.  
both of which follow a hard parameter sharing approach. All three different methods have advantages and disadvantages that will be compared in detail in Section 4.  
MTL [28] aims to improve the performance of multiple learning tasks by sharing useful information among them. MTL can be very useful when using low-resource datasets since it can exploit useful information from other related learning tasks to help alleviate the issue of limited data. Based on the assumption that the multiple tasks are related, MTL is empirically and theoretically found to lead to better performance than independent learning. MTL is similar to transfer learning [29] which also transfers knowledge from one task to another. However, the focus of transfer learning is to help a single target task by initially training on one or multiple tasks while MTL uses multiple tasks to help each other. Furthermore, MTL can be viewed as a generalization of multi-label learning [30] when different tasks in multi-task learning share the same training data.  
The motivation behind using MTL includes the implicit data augmentation, since a model that learns two tasks simultaneously is able to learn a more general representation. In addition, if data is limited, MTL can help the model focus its attention on those features that actually matter as other tasks will provide additional evidence for the relevance or irrelevance of those features. Finally, MTL acts as a regulariser by introducing an inductive bias that reduces the risk of overfitting. An overview of MTL can be found in [31].