# Real-Time Communication Aid System for Korean Dysarthrie Speech
## 4 Experiments
### Integrated Communication Aid Systems  
Finally, the trained ASR and TTS models are combined to build the entire communication system. The entire system operates as a single program, and during execution,  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **CER (\%)** & **WER (\%)** & **ERR(CER) (\%)** & **ERR(WER) (\%)** \\ \hline Hybrid CTC/attention transformer model [27] & 15.6 & 18.8 & - & - \\ \hline Proposed conformer-based GTN-CTC model & 9.1 & 11.6 & 41.7 & 38.3 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance evaluation results of the proposed ASR model.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**TTS Model** & **MOS (1–5)** \\ \hline JETS [37] & 4.64 \(\pm\) 0.06 \\ \hline Tacotron2 [41] & 3.03 \(\pm\) 0.07 \\ \hline Transformer TTS [42] & 2.51 \(\pm\) 0.09 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Evaluate MOS comparisons for different TTS models.  
it receives speech input from the user via the user interface. As the length of the input increases, the time to obtain the final output also increases. However, if the user provides speech input lasting 5-10 s, the final output is obtained approximately 2-3 s after input. This means that the conversion and output are completed within 1-2 s after voice input, assuming that the utterances used in daily life are generally short utterances of 3-5 s. Therefore, this system has the problem of a long delay time in the case of a long voice input of 10 s or more, but the delay time is reduced to around 1 s for short utterances used for simple communication in daily life, indicating that it is not unreasonable to be applied in real-time in real life.  
Figure 7 shows a visualization of the Korean dysarthric speech used as input and the synthetic utterance generated by the entire system. In Figure 6(a), corresponding to the dysarthric speech, we can see that there are frequent silences in the middle of the utterance. This is due to the nature of dysarthria, which is a speech blockage caused by the dysregulation of the articulatory system during speech, which causes great confusion for the listener in understanding the speech. When such dysarthric speech is fed into a communication aid system, all the silent intervals caused by speech blockages are removed and resynthesized into a natural rhyming utterance, Figure 6(b).  
Figure 8 shows the input and output results for a similar case to Figure 7, but for a slightly longer speech: a dysarthric speech of about 9.5 s has been significantly shortened to about 3.5, and the long pauses in the middle of the speech have been eliminated. This process of speech recognition and synthesis makes the relatively long speech shorter and clearer, allowing the user to produce natural speech.  
Figure 7: Visualization of the system output for the Korean dysarthric speech input: (Meaning: “At dawn, it thundered and rained”) and (English phonetic transcription: “saebyeoge cheondungi chigo biga neryeossoyo”). The gray areas in the graph indicate silent segments within the speech, which were caused by dysarthric speech disfluencies.  
Since the system is provided to users through a web server, the performance of the system is dependent on the performance of the server computer that deploys the system on the web server. Therefore, the device for running the system has the advantage of not having any problems with system performance even on a device with limited resources, as long as it is provided with an Internet network that can reliably connect to the web server. In addition, the recognition speed, which accounts for a large part of the performance of this system, has been measured based on CPU, and if it is converted to a GPU-based recognition method or optimized as a cloud-based service in the future, performance improvement and a more efficient system deployment can be expected.