# Accented Speech Recognition Based on End-to-End Domain Adversarial Training of Neural Networks
## 4 Experiments and Results
### Experimental Results  
To verify the efficiency of the proposed approach, baseline and DANN models were built. The baseline model performed standard end-to-end speech recognition, and it comprised four CNN and four BiGRU layers, as shown in Figure 6. The DANN model was constructed according to Figure 4, illustrated in Section 3.3. The performance of the two model types was first investigated. As described in Table 5, two baseline models were compared with the DANN model. Baseline-src was trained using only the dataset of the source domain (US-160k), whereas baseline-src-tgt was trained using both the datasets of the source domain (US-160k) and each target domain (AU-20k, CA-20k, EN-20k, and IN-20k). DANN was also trained using the same datasets as baseline-src-tgt. The performance of the models was assessed using two measures: the character error rate (CER) and the word error rate (WER), which were measured by the edit distance based on the Levenshtein algorithm [23] between the predicted labels \(\hat{Y}=\{\hat{y}_{1},\hat{y}_{2},\dots,\hat{y}_{n}\}\) and the original labels \(Y=\{y_{1},y_{2},\dots,y_{n}\}\). The results are presented in pairs of CER and WER (CER/WER) in Table 5. As shown in the table, baseline-src-tgt achieved a better performance than baseline-src for all accents. Although the amount of target domain data was much smaller than that of the source domain data, the model applying the target domain data was effective for recognizing the test data corresponding to the target domain. Moreover, compared with the two baseline model types, DANN achieved significant performance improvements for all accents. In particular, compared with baseline-src-tgt, the proposed DANN approach showed a notable performance for EN and IN accents, with WER reductions of approximately 5% and 3%, respectively. Meanwhile, it had a very slight performance improvement for the CA accent, with a WER reduction of 0.53%. These results indicated that the linguistic differences between the source and target domain accents affected the performance. The source domain accent (US) has linguistically more similar characteristics to CA than the EN and IN accents. As a result, domain adversarial training provides better conditions for recognizing target accents that are linguistically different from the source domain accents. Although our proposed approach demonstrated superior performance compared with the baseline, it is necessary to examine whether the results are sufficiently meaningful. In particular, the CER and WER are quite high for practical purposes. In general, standard speech recognition systems providing stable performance use vast amounts of training data. The study in [16], which was based on end-to-end speech recognition, used over 10,000 h  
Figure 6: Architecture of the baseline model. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Accent**} & \multicolumn{3}{c}{**Model**} \\ \cline{2-4}  & **Baseline-src** & **Baseline-src-tgt** & **DANN** \\ \hline AU & 28.85\%/65.95\% & 25.09\%/61.06\% & 24.22\%/59.80\% \\ CA & 14.76\%/43.16\% & 13.77\%/40.68\% & 13.55\%/40.15\% \\ EN & 25.49\%/61.53\% & 24.43\%/59.78\% & 21.60\%/54.67\% \\ IN & 36.41\%/76.03\% & 30.52\%/69.41\% & 28.83\%/66.49\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Performance (CER/WER) comparison between the baseline and DANN models. of standard speech data to train acoustic models. However, it is a challenge to construct reliable models for different accent types, as it is difficult to collect a sufficient amount of speech data for each target accent. For this reason, most studies have concentrated on finding efficient methods under the conditions of limited amounts of accented speech data. In this study, we used approximately 200 h of source domain data and 25 h of target domain data to train the models. Nevertheless, it is necessary to compare the performance of our approach with that of conventional studies.