# Accented Speech Recognition Based on End-to-End Domain Adversarial Training of Neural Networks
## 2 Conventional Studies on Accented Speech Recognition  
Up to the early 2010s, several adaptation techniques such as MAP and MLLR were mainly adopted for accented speech recognition [3; 4]. These techniques aimed to adapt the standard ASR model to accented speech data. Although the approaches successfully improved accented speech recognition performance, they operated on statistical pattern models such as the Gaussian Mixture Model (GMM)-Hidden Markov Model (HMM), which is a classic approach [9; 10; 11]. Thus, they have rarely been used in current studies that focus on neural network modeling.  
The authors of [5] proposed a multi-task learning network-based approach that performs ASR and accent classification simultaneously. They built a standalone network to produce accent embedding features, which were fed into the multi-task learning network, expecting that asking (accent classification) and telling (accent embedding) the network would improve the performance. In [6], the authors improved this approach by changing the inner structure of the ASR model from two time delay neural network (TDNN) [12] layers to a combination of two convolutional neural network (CNN) layers and four gated recurrent unit (GRU) [13] layers.  
The conventional approaches based on accent embedding build a multi-task learning network and a standalone network. Figure 1 describes the model architecture proposed in [5]. A multi-task learning network has two sub-networks ((A) and (B)), and each sub-network performs different functions. As shown in this figure, the acoustic model (A) is combined with the accent classifier (B), and these sub-networks share two TDNN layers. Based on two sub-networks, the multi-task learning network performs phone recognition and accent classification simultaneously. Next, the standalone network (C) produces accent embedding features, which are fed into the multi-task learning network. Thus,mel-frequency cepstral coefficients, i-vectors, and accent embedding features are used as input features in this approach.  
The accent embedding method is helpful for improving ASR performance; however, it has some limitations. To output the accent embedding features, the standalone network needs to be built and trained independently. Moreover, the standalone network should be constructed sophisticatedly, as ASR performance is considerably affected by the accent classification accuracy of the network.  
Accent classification accuracy has been reported according to the model specifications, as shown in Table 1. When the model has more nodes, the accuracy becomes higher. In other words, to achieve stable classification accuracy, a complex model that has high computational costs and large memory space is required.  
Furthermore, a number of parameters should be estimated in the multi-task learning network because additional feature vectors are added into the network. To accommodate these parameters, the network should be complex, similar to the standalone network.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Model Specifications** & **Accent Classification Accuracy** \\ \hline TDNN, 7 layers, 100-d nodes & 78.4\% \\ TDNN, 7 layers, 200-d nodes & 78.4\% \\ TDNN, 7 layers, 300-d nodes & 80.0\% \\ TDNN, 7 layers, 1024-d nodes & 82.6\% \\ \hline \hline \end{tabular}
\end{table}
Table 1: Accent classification accuracy of a standalone network.  
Figure 1: Architecture of the multi-task learning network and the standalone network in the accent embedding method. (**A**) Acoustic model; (**B**) Accent classifier; (**C**) Standalone network.