# Regularized Urdu Speech Recognition with Semi-Supervised Deep Learning
## 4 Results and Analysis
### Wer  
WER down to 22% is achieved by the proposed SSL-Neural Network (SSL-NN) model for speaker independent setup as compared to 25.42% achieved by HMM on the same corpus [3]. The WER comparison is presented below in Table 3. HMM has better performance for test speakers 0, 2, 5 and 8 while the proposed model predicts better for speakers 1, 2, 3, 4, 6, 7 and 9. Consequently the mean WER of the proposed model is lower than that of HMM. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Train Speakers**} & \multirow{2}{*}{**Test Speakers**} & \multicolumn{2}{c}{**Word Error Rate \%**} \\ \cline{3-4}  & & **Asad-HMM [3]** & **SSL-NN** \\ \hline
1–9 & 0 & 17.6 & 35.6 \\
0,2–9 & 1 & 20 & 13.6 \\
0–1,3–9 & 2 & 12.8 & 20 \\
0–2,4–9 & 3 & 36.4 & 17.2 \\
0–3,5–9 & 4 & 35 & 9.6 \\
0–4,6–9 & 5 & 13.6 & 33.2 \\
0–5,7–9 & 6 & 38 & 19.6 \\
0–6,8–9 & 7 & 24.8 & 20 \\
0–7,9 & 8 & 26.8 & 27.2 \\
0–8 & 9 & 29.2 & 24.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Word Error Rate (WER) comparison with Asad-Hidden Markov Models (HMMs) [3]. Figure 2: Proposed system model. The HMM model is tested with only 10% data reserved for validation. However, the proposed model is tested to calculate WER with a higher ratio of data from the same corpus reserved as the validation set. Up to 5 speakers are used to test on a model trained with 5 speakers for training. Even after using 2, 3, 4 and 5 speakers as test data, WER does not increase drastically due to unsupervised techniques being incorporated. Table 4 summarizes the mean WER recorded for 20%, 30%, 40% and 50% partition as test data. WER for different combinations of training and test data are recorded before and after incorporating SSL and model averaging. WER decreases after adding LLE dimensions, it further decreases after self-training using label propagation. Averaging over multiple models reduces WER further. While the lowest WER is obtained after self-training (ST) of ensemble models with LLE appended input data used for training and testing purposes. Table 5 shows WER details for the validation data set of 1 speaker with neural network trained on 9 speakers after using SSL and averaging. Table 6 shows the WER for the validation data set of 2 speakers and the training data of 8 speakers before and after using regularization techniques. \begin{table}
\begin{tabular}{c c} \hline \hline
**Test Set Size** & **WER \%** \\ \hline
10\% & 22.08 \\
20\% & 22.68 \\
30\% & 27.34 \\
40\% & 31.35 \\
50\% & 34.65 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Mean WER% for different validation dataset size. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Train** & **Test** & \multicolumn{5}{c}{**Word Error Rate \%**} \\ \cline{3-7}
**Speakers** & **Speakers** & **Features** & **Adding LLE** & **LLE + ST** & **Ensemble** & **Ensemble + ST** \\ \hline
1–9 & 0 & 42 & 40.4 & 37.2 & 37.6 & 35.6 \\
0,2–9 & 1 & 15.6 & 14.8 & 12.4 & 12.8 & 13.6 \\
0–1,3–9 & 2 & 28.4 & 20.8 & 21.6 & 22.8 & 20 \\
0–2,4–9 & 3 & 20.4 & 18.4 & 17.2 & 16.8 & 17.2 \\
0–3,5–9 & 4 & 12.4 & 10.4 & 10 & 9.2 & 9.6 \\
0–4,6–9 & 5 & 36 & 35.2 & 34.4 & 36.4 & 33.2 \\
0–5,7–9 & 6 & 20 & 21.2 & 22 & 18.8 & 19.6 \\
0–6,8–9 & 7 & 25.2 & 21.6 & 21.2 & 20.4 & 20 \\
0–7,9 & 8 & 31.2 & 30 & 30.8 & 29.2 & 27.2 \\
0–8 & 9 & 25.6 & 26.8 & 25.6 & 23.6 & 24.8 \\ \hline  & Mean & 25.68 & 23.96 & 23.24 & 22.76 & 22.08 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Word Error Rate for 10% data as test. Table 7 shows the WER over the validation data set of 3 speakers. It can be observed that with an increasing validation set size, WER recorded without SSL starts increasing considerably as the corpus data is limited; however, after incorporating SSL, the WER increase is less significant. Table 8 shows the WER over the validation data set of 4 speakers. It is visible that with an increasing validation data set, the improvement in WER by SSL gets more significant as compared to the smaller validation data partition. Table 9 shows the WER for 5 speakers with the neural network trained on 5 speakers.