# Unsupervised Adaptation of Deep Speech Activity Detection Models to Unseen Domains
## 2 Domain Adaptation
### Unsupervised Domain Adaptation Techniques
#### 2.3.3 Deep CORAL  
The recently proposed Correlation Alignment (CORAL) method [37] is an unsupervised adaptation technique that is performed by aligning the second-order statistics of the source and the target distributions. However, this technique relies on a linear transformation and is not end-to-end. In order to address those issues, an extension on the CORAL method named Deep CORAL was proposed [59] with the idea of incorporating the CORAL technique directly into deep neural networks by constructing a differentiable loss functions that minimises the difference between source and target correlations. The CORAL loss between two domains for a single feature layer is described in the following lines. Suppose a set of training examples coming from the labelled source domain, \(D_{s}\), described by \(\mathbf{U}_{s}=\{u_{s_{1}},\ldots,u_{s_{N}}\}\) with \(u\in\mathbb{R}^{d}\), and unlabelled target data \(\mathbf{V}_{t}=\{v_{t_{1}},\ldots,v_{t_{M}}\}\), with \(v\in\mathbb{R}^{d}\). The number of source and target data are \(N\) and \(M\) respectively. As described in the original paper, \(\mathbf{U}_{s}\) and \(\mathbf{V}_{t}\) represent the \(d\)-dimensional deep layer activations of a deep neural network model. Considering the provided definitions, the covariance matrices of the source and target data are given by the following equations:  
\[\mathbf{C_{s}}=\frac{1}{N-1}(\mathbf{U}_{s}^{T}\mathbf{U}_{s}-\frac{1}{N}( \mathbf{1}^{T}\mathbf{U}_{s})^{T}(\mathbf{1}^{T}\mathbf{U}_{s}))\,, \tag{4}\]  
\[\mathbf{C_{t}}=\frac{1}{M-1}(\mathbf{V}_{t}^{T}\mathbf{V}_{t}-\frac{1}{M}( \mathbf{1}^{T}\mathbf{V}_{t})^{T}(\mathbf{1}^{T}\mathbf{V}_{t}))\,, \tag{5}\]  
where \(\mathbf{1}\) is a column vector with all elements equal to 1. The CORAL loss is then defined as the distance between the second-order statistics of the source and target features. This is shown in Equation (6):  
\[L_{\text{CORAL}}=\frac{1}{4d^{2}}\|\mathbf{C_{s}}-\mathbf{C_{t}}\|_{F}^{2}\,, \tag{6}\]  
where \(\|\cdot\|_{F}^{2}\) denotes the squared matrix Frobenius norm. The idea behind Deep CORAL adaptation technique is to obtain a set of deep features that are both discriminative enough to train a strong classifier and invariant to the change observed between source and target domains. Minimising a classification loss by itself, as usually done in supervised learning approaches, may lead to overfitting on the source domain and a reduced performance on the target domain. By contrast, minimising the CORAL loss alone may lead to degenerated features. In order to match the conditions stated above, the final loss to be optimised is a combination of both a classification loss and  
Figure 1: Schematic description of the proposal for the knowledge distillation domain adaptation technique. the CORAL loss. The representation of the neural architecture needed to implement Deep CORAL adaptation technique is shown in Figure 2.  
As shown, source features are forwarded through the DNN model and then a classification loss is computed using source labels. Similarly, target features are used in combination with source features to compute CORAL loss. Network parameters are shared among the two DNN models. Considering the described architecture, the joint optimisation target of classification loss and CORAL loss is described in Equation (7):  
\[L=L_{\text{classif}}+\sum_{i=1}^{z}\lambda_{i}L_{\text{CORAL}_{i}}\,, \tag{7}\]  
where \(L_{\text{classif}}\) is any traditional classification loss function such as cross entropy, \(z\) is the number of CORAL loss layers in a deep network and \(\lambda_{i}\) is a weight that trades off adaptation and classification accuracy on the source domain. The sum term depicted aims to represent the possibility of incorporating the CORAL loss on additional layers of the DNN architecture. However, as described in the original paper, authors apply the CORAL loss only to the last classification layer in the DNN architecture.