# Regularized Urdu Speech Recognition with Semi-Supervised Deep Learning
## 3 System Model  
The proposed model consists of neural network with input data extracted using MFCC and delta speech features. The features matrix is projected on lower dimensions using LLE, and the compressed matrix is used to aid input data. The initially trained neural network is further trained using confidently predicted samples by label propagation. The primary objective of the proposed model is to utilize recently proposed neural network techniques in order to achieve lower WER as compared to the benchmark achieved by HMM [3]. A smaller portion of the same corpus is used as training data to analyze the impact of SSL for a scarce labeled-data availability scenario. The Urdu corpus used for testing is the same as that used by [PERSON] et al. and consists of 250 isolated words spoken by 10 different speakers. Lower 13 MFCC, corresponding delta and 26 logfbank coefficients are extracted from speech samples. The sliding window length to extract each set of coefficients is set as 25 milliseconds. This sliding window is applied every 10 ms. Delta or gradients of MFCC coefficients are computed with respect to 30 frames ahead and backwards. This gives a feature matrix for each speech sample with 52 rows and a number of columns equal to the number of feature extraction time frames for that sample. Feature matrices are flattened to obtain single dimensional vectors for each sample. Speech signals in the corpus have variable lengths, so in order to warp them in time they are scaled through linear interpolation to a fixed length chosen to be the minimum feature vector length. The features matrix is compressed to 972 dimensions for each sample using LLE with the 21 nearest neighbors for each point. This LLE vector for each sample is concatenated to its original feature vector, so instead of reducing, dimensions of input vector are actually increased to the original plus the LLE dimensions. This adds useful information about vectors, obtained by incorporating all of the data set including test samples. Table 1 summarizes the input data set obtained for the experiments. The Adam optimization algorithm is used for optimization [6] of neural network to try to minimize categorical cross entropy as the cost function for 150 iterations using stochastic gradient descent on a mini-batch size of 250 samples. Figure 1 illustrates the proposed neural network architecture graphically. The left-most layer is input layer 'I' with dropout ratio 0.5 and dimensions T. H represents the first hidden layer with 2000 neurons and Relu activation. M represents the second hidden layer with Maxout activation and 4 affine maps. O represents the output layer with 250 Softmax units, each representing one of 250 spoken words. \begin{table}
\begin{tabular}{l l} \hline \hline \multicolumn{1}{c}{**Input Data**} \\ \hline No. of words = 250, speakers = 10 & Input linear interpolation: R\({}^{2500\times\text{Lf}}\)\(\rightarrow\) R\({}^{2500\times\text{Lm}}\) \\ No. of samples in corpus = 250 × 10 = 2500 & Input LLE transformation: R\({}^{2500\times\text{Lm}}\)\(\rightarrow\) R\({}^{2500\times\text{972}}\) \\ Interval of extracting feature window, 10 ms & For 2500 points and 21 neighbors \\ Length of each speech sample, Ls milliseconds & Features matrix = X = R\({}^{2500\times\text{Lm}}\) \\ No. of 52 feature columns per sample, Nf = Ls/10 & Neighbor matrix of X = B=R\({}^{2500\times 21\times\text{Lm}}\) \\ Length of flattened feature vector, Lf = 52 × f & Weight matrix = W = R\({}^{2500\times\text{21}}\), satisfying min |X-WB| \\ Scaled length of feature vectors, Lm = min (Lf) & Neighbor matrix of Y = C = R\({}^{2500\times\text{21}\times\text{972}}\) \\ Target LLE dimensions = 972 & LLE output = Y = R\({}^{2500\times\text{972}}\), satisfying min |Y-WC| \\ Length of concatenated vectors, T = Lm + 972 & Input data = I = (X appended with Y) \\  & = I (2500 X T) = R\({}^{2500\times\text{T}}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Input data set. Table 2 illustrates the architecture of the proposed neural network model mathematically.