# Accented Speech Recognition Based on End-to-End Domain Adversarial Training of Neural Networks
## 3 Accented Speech Recognition Based on End-to-End Domain Adversarial Training
### Domain Adversarial Neural Network  
A representative domain adaptation technique in neural network-based model training is DANN, which was first proposed in the field of computer vision [8]. Some speech recognition studies then adopted the technique to handle noisy speech data [14]. Compared with accent embedding, which was addressed in Section 2, DANN requires a lower computational cost, as it can be trained using only filter-bank features without the generation of extra features. In particular, DANN performs more sophisticated training procedures consisting of feature learning, domain adaptation, and label prediction, each of which is jointly unified within the architecture. Figure 3 shows the architecture of DANN, which comprises three sub-networks: the feature extractor, the domain classifier, and the label predictor. In this figure, the green-colored, red-colored, and blue-colored parts are the feature extractor \(G_{f}(x;\theta_{f})\), the domain classifier \(G_{d}(f;\theta_{d})\), and the label predictor \(G_{y}(f;\theta_{y})\), respectively.  
Figure 2: Effect of domain adaptation.  
First, the feature extractor aims to extract some useful features from the input data x to train the model. These features are then forwarded to the domain classifier and the label predictor. According to the backpropagation mechanism, the extractor reflects the gradient of the domain classifier \(\frac{\theta L_{d}}{\theta\theta_{d}}\) and the label predictor \(\frac{\theta L_{y}}{\theta\theta_{y}}\). Afterward, the extractor adjusts the features to be domain-invariant in a number of training steps.  
The domain classifier determines whether the domain of the input data is the source domain or the target domain. The goal of DANN is to reduce the distribution differences between the source and target domains, and it is achieved by making it difficult to distinguish between the source and target domains. The gradient reversal layer (GRL) plays an important role in this step. The GRL is positioned at the bottom of the domain classifier and reverses the gradient \(\frac{\theta L_{d}}{\theta\theta_{d}}\) by multiplying it by a scalar \(\lambda\) during backpropagation. With the help of the GRL, the feature extractor receives the reversed gradient \(-\lambda\frac{\theta L_{d}}{\theta\theta_{f}}\). Thus, the feature extractor can make the features become domain-invariant in the subsequent training steps. The GRL is only activated during backpropagation and it does not change any parameters during forward propagation.  
Lastly, the label predictor predicts a label \(\hat{y}\) and calculates the loss value between the predicted label and the original label y. The gradient \(\frac{\theta L_{y}}{\theta\theta_{y}}\) is also backpropagated into the bottom layers of the model. In summary, DANN has two main objects: to predict the label correctly and to reduce the distribution between the source and target domains. These objects are achieved by minimizing the loss of the label predictor (given by (1)) and maximizing the loss of the domain classifier (given by (2)).  
\[\theta_{y}=argmin_{\theta_{y}}E\big{(}\theta_{f},\theta_{y},\theta_{d}\big{)} \tag{1}\]  
\[\theta_{d}=argmax_{\theta_{d}}E\big{(}\theta_{f},\theta_{y},\theta_{d}\big{)} \tag{2}\]