## 4 Proposed DNN-Based VAD Method for Low Latency Decoding  
Viterbi decoding involves two processes: one estimates probabilities of states in all possible paths, and the other finds an optimal path by backtracking the states with the highest probability. The ASR system yields the results only after both processes are completed, usually at the end of an utterance.  
In an online ASR system recognizing long utterances, the end point of an utterance is not known in advance and deciding the back-tracking point affects user experience in terms of response time. If the backtracking is performed infrequently, the user will receive a delayed response, and in the opposite case, the beam search will not find the optimal path that reflects the language model contexts.  
In our system, VAD based on an acoustic model for ASR is used to detect short pauses in continuous utterance, which will trigger backtracking. Especially, the acoustic model is built with a deep neural network; hence, we call it DNN-VAD. Here, DNN includes not only a fully connected DNN but also all types of deep models, including LSTM and BLSTM. As explained in previous sections, in our ASR system, BLSTM is used to compute a posterior probability of each state of triphone for each frame. By re-using these values, we can also estimate the probability of non-silence for a given frame with little additional computational cost.  
Each output node of the DNN model can be mapped into states of nonsilence or silence phones. Let the output of the DNN model in the \(i\)-th node be \(o_{i}\). Then, the speech probability of the given frame is computed as  
\[\log P_{\text{nonsil}} = \max_{i}o_{i}\text{, where }i\in\text{non-silence states} \tag{9}\] \[\log P_{\text{sil}} = \max_{i}o_{i}\text{, where }i\in\text{silence states}\] (10) \[\text{LLR}_{\text{nonsil}} = \log P_{\text{nonsil}}-\log P_{\text{sil}}(t)\text{,} \tag{11}\]  
where LLR is log likelihood ratio. Each frame at time \(t\) is decided to be a silence frame if \(\text{LLR}_{nonsil}\) is smaller than the predefined threshold.  
\[s(t)=\begin{cases}1&\text{if }\text{LLR}_{\text{nonsil}}(t)<\mathcal{T}_{l}\\ 0&\text{otherwise.}\end{cases} \tag{12}\]  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Transmission** & **Min. Frequency** & **Max. Frequency** & **Size** \\ \hline from CPU to GPU & \(\mathcal{N}_{channel}/\mathcal{N}_{parallel}^{GPU}\) & \(\mathcal{N}_{channel}\) & \(\mathcal{N}_{bat}\times k\) \\ from GPU to CPU & \(\mathcal{N}_{channel}/\mathcal{N}_{parallel}^{GPU}\) & \(\mathcal{N}_{channel}\) & \(\mathcal{N}_{node}\times k\) \\ \hline \hline \end{tabular}
\end{table}
Table 4: Summary of the frequency and transmission sizes between CPU and GPU for each feature with a duration of \(\mathcal{T}_{bat}\) if the number of concurrent clients is \(\mathcal{N}_{channel}\) and the number of decoded speech feature vectors is \(k\leq GPU_{parallel}\), when using the proposed fast multichannel parallel acoustic score computation.  
In addition, for smoothing purpose, the ratio of silence frames in a window of length \((2W+1)\) is computed and compared to the predefined threshold \(\mathcal{T}_{r}\).  
\[\hat{s}(t)=\begin{cases}1&\text{if }\sum_{w=-W}^{W}s(t+w)/(2W+1)>\mathcal{T}_{r}\\ 0&\text{otherwise.}\end{cases} \tag{13}\]  
All computations in from Equation (9) to Equation (13) are performed within each minibatch, and the frame at \(t\) when \(\hat{s}(t-1)=1\) and \(\hat{s}(t)=0\) is regarded as the short pause in an utterance. As will be explained in the Section 5.3, the frequent backtracking reduces the response time but also degrades the recognition accuracy. Thus, a minimal interval is set between the detection of short pauses to control the trade-off.