# Accented Speech Recognition Based on End-to-End Domain Adversarial Training of Neural Networks
## 5 Conclusions  
This study proposed an efficient accented speech recognition approach using end-to-end domain adversarial training of neural networks based on DANN. The goal of DANN is to learn domain-invariant features to reduce the distribution differences between the source and target domains. In this study, we proposed an efficient DANN model architecture to carefully handle accented speech recognition. Each of the three sub-networks of DANN was constructed with appropriate neural networks considering the characteristics of accented speech data. CNN was used for the feature extractor, DNN for the domain classifier, and BiGRU for the label predictor. In particular, we used a CTC-based end-to-end framework to construct the label predictor, which is a very important module in DANN, as the accuracy of the predicted labels directly affects the ASR results.  
To verify the efficiency of the proposed approach, we performed several experiments of accented speech recognition using the Common Voice corpus for four English accents (Australian, Canadian, England, and Indian accents). For all accents, the proposed DANN model outperformed the baseline model constructed according to a standard end-to-end speech recognition scheme. In addition, in experiments performed with varying amounts of target accent data, we observed that the accuracy of the proposed model improved significantly as the amount of the target domain data increased.  
In a further study, we will investigate an efficient DANN approach for an unsupervised accented speech recognition task in which data labels are not required. We expect that this further study will help to make larger datasets because unlabeled accented speech data are easier to collect. Furthermore, we will utilize the transformer architecture used for end-to-end speech recognition to enhance the end-to-end domain adversarial training. In particular, since the transformer-based decoder is applicable to the label prediction module of DANN, we expect that this further study will greatly improve accented speech recognition performance.  
Conceptualization, H.-J.N. and [PERSON]-[PERSON]; methodology, H.-J.N. and [PERSON]-S.P.; software, H.-J.N.; validation, H.-J.N. and J.-S.P.; formal analysis, H.-J.N. and [PERSON]-[PERSON]; writing--original draft preparation, H.-J.N.; writing--review and editing, J.-S.P.; supervision, J.-S.P.; project administration, J.-S.P.; funding acquisition, J.-S.P. All authors have read and agreed to the published version of the manuscript.  
This research was supported by Hankuk University of Foreign Studies Research Fund, the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No. NRF-2020R1A2C1013162), and the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2021-2016-0-00313) supervised by the IITP (Institute for Information and Communications Technology Planning and Evaluation).  
Not applicable.  
The speech data used for experiments is publicly available at [[https://commonvoice.mozilla.org/](https://commonvoice.mozilla.org/)]([https://commonvoice.mozilla.org/](https://commonvoice.mozilla.org/)) (accessed on 16 August 2021).  
The authors declare no conflict of interest.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Accent**} & \multicolumn{6}{c}{**Model**} \\ \cline{2-6}  & **DANN-EN1** & **DANN-EN2** & **DANN-EN3** & **DANN-EN4** & **DANN-EN5** \\ \hline EN & 21.60\%/54.67\% & 19.96\%/52.53\% & 18.65\%/49.91\% & 17.15\%/47.32\% & 16.35\%/45.68\% \\ \hline \hline \end{tabular}
\end{table}
Table 7: Performance (CER/WER) of DANN models according to the amount of EN accent data.