## 3 Proposed Fast Multichannel Parallel Acoustic Score Computation
\begin{table}
\begin{tabular}{l c c} \hline \hline
**Thread** & **Name** & **Device** & **Size** \\ \hline Worker & \(\mathcal{M}_{feat\_vc\_f}^{CPU(W)}\) & CPU & \(\mathcal{N}_{bat}\times\mathcal{N}_{channel}\) \\ Worker & \(\mathcal{M}_{feat\_vc}^{CPU(W)}\) & CPU & \(\mathcal{N}_{bat}\times\mathcal{N}^{GPU}\) _parallel_ \\ Worker & \(\mathcal{M}_{feat\_vc}^{GPU}\) & GPU & \(\mathcal{N}_{bat}\times\mathcal{N}_{parallel}^{GPU}\) \\ Worker & \(\mathcal{M}_{feat\_mat}^{GPU}\) & GPU & \(\mathcal{N}_{win}\times\mathcal{N}_{seg}\times\mathcal{N}_{parallel}^{GPU}\) \\ Worker & \(\mathcal{M}_{prob\_vc}^{GPU}\) & GPU & \(\mathcal{N}_{node}\times\mathcal{N}_{parallel}^{GPU}\) \\ Worker & \(\mathcal{M}_{prob\_vc}^{CPU(W)}\) & CPU & \(\mathcal{N}_{node}\times\mathcal{N}_{Channel}^{GPU}\) \\ \hline Child\({}_{i}\) & \(\mathcal{M}_{prob\_vc}^{CPU}\) & CPU & \(\mathcal{N}_{node}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Summarization of the allocated run-time memories at CPU and GPU of the proposed fast multichannel parallel acoustic score computation. the worker thread, \(\mathcal{M}^{CPU(W)}_{feat\_vec\_f}\). For instance of an \(i\)-th client, the vector is stored at \(\mathcal{M}^{CPU(W)}_{feat\_vec\_f}\) with the offset index of  
\[\mathcal{N}_{bat}\times i. \tag{5}\]  
Then, the decoder thread waits for the acoustic scores to be calculated by the worker thread. On the other hand, whenever the worker thread is in idle state and there are buffered speech feature vectors in \(\mathcal{M}^{CPU(W)}_{feat\_vec\_f}\), the worker thread pops \(k\) speech feature vectors from \(\mathcal{M}^{CPU(W)}_{feat\_vec\_f}\) in first-in, first-out (FIFO) order where \(k\) is the number of feature vectors to be decoded at a time and the maximum is \(\mathcal{N}^{GPU}_{parallel}\). The obtained \(k\) vectors are merged at \(\mathcal{M}^{GPU}_{feat\_vec}\), and then transmitted from CPU to GPU (\(\mathcal{M}^{GPU}_{feat\_vec}\)). On GPU, the transmitted vectors are reformed into CSC-based vectors, which are merged into a CSC-based matrix in the cascaded form of the CSC-based matrix of each speech feature vector, as follows:  
\[ \begin{bmatrix} CSC^{(1)}\\ \vdots\\ CSC^{(k)}\end{bmatrix} = \begin{bmatrix} CSC^{(1)}_{1,1}&\cdots& CSC^{(1)}_{1,i}&\cdots& CSC^{(1)}_{1,N_{ain}}\\ \vdots&\vdots&&\vdots\\ CSC^{(1)}_{Nag,1}&\cdots& CSC^{(1)}_{Nag,i}&\cdots& CSC^{(1)}_{Nag,N_{ain}}\\ &&\vdots&&\vdots\\ \left[\begin{array}{cccc} CSC^{(k)}_{1,1}&\cdots& CSC^{(k)}_{1,i}&\cdots& CSC^{(k)}_{1,N_{ain}}\\ \vdots&&\vdots&&\vdots\\ CSC^{(k)}_{Nag,1}&\cdots& CSC^{(k)}_{Nag,i}&\cdots& CSC^{(k)}_{Nag,N_{ain}} \end{array}\right],\end{bmatrix}  \tag{6}\]  
where \(CSC^{(i)}\) indicates the CSC-based matrix of the \(i\)-th speech feature vector of \(\mathcal{M}^{GPU}_{feat\_vec}\). Then, the matrix is normalized using an LDA-based transform [37] and the acoustic scores are calculated into \(\mathcal{M}^{GPU}_{prob\_vec}\) using the CSC BLSTM AM. The acoustic scores are in the following cascaded form:  
\[ \begin{bmatrix}prob^{1}&\cdots&prob^{k}\end{bmatrix} =\\ \begin{bmatrix}[prob^{(1)}_{1}&\cdots&prob^{(1)}_{i}&\cdots&prob^{(1)}_{N_{ undo.}}\end{bmatrix} &\cdots& \begin{bmatrix}prob^{(k)}_{1}&\cdots& prob^{(k)}_{i}&\cdots&prob^{(k)}_{N_{ undo.}}\end{bmatrix} \end{bmatrix}^{\prime}, \tag{7}\]  
where \(prob^{i}\) means the acoustic scores of the \(i\)-th speech feature vector of \(\mathcal{M}^{GPU}_{feat\_vec}\). Next, the acoustic scores are transmitted from GPU to CPU. For instance of \(prob^{i}\), if it is for the \(m\)-th client, \(prob^{i}\) is stored at \(\mathcal{M}^{CPU(W)}_{prob\_vec}\) with an offset index of  
\[\mathcal{N}_{node}\times m. \tag{8}\]  
If the waiting decoder thread detects the acoustic scores at \(\mathcal{M}^{CPU(W)}_{prob\_vec}\) with the corresponding offset index of Equation (8), the decoder thread copies them into its local memory (\(\mathcal{M}^{CPU}_{prob\_vec}\)) and proceeds the subsequent steps as the baseline method. The descried procedures are shown in Figure 4c.