# Real-Time Communication Aid System for Korean Dysarthrie Speech
## 1 Introduction
[PERSON] and [PERSON] (2020) [22] developed the necessity for a tailored dysarthic ASR system based on a hybrid model combining Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), because ASR systems trained on normal speech are not effective for dysarthic language. In [23], [PERSON] et al. (2022) used a multi-speaker end-to-end Text-to-Speech (TTS) system, capable of synthesizing high-quality speech with diverse prosodies, to generate realistic dysarthic speech for data augmentation. This allowed the inclusion of speech reflecting varying degrees of dysarthria, thus improving the performance of the ASR system. [PERSON] et al. (2023) proposed dysarthic ASR for Arabic speech [24]. Despite advancements in ASR, especially for Arabic, it is difficult to obtain dysarthic speech data. In [24], the first ASR model for Arabic dysarthic speech was developed with simulated dysarthic speech artificially modified from healthy Arabic speech, which contributes significantly to the Arabic ASR field. Table 1 summarizes previous studies on assistive tools for the treatment of dysarthria, focusing exclusively on software-related research, excluding those utilizing physical devices. These prior studies exhibit several limitations in terms of practical communication support for individuals with dysarthria. First, except for VIVOCA [10], none of the other studies provide a framework for real-life communication assistance for dysarthic speakers. As a result, additional efforts are required to apply these research outputs to real-world  
scenarios, which poses a significant limitation. Moreover, all of the above studies rely on English datasets, such as UA-Speech [25] and Torgo [26]. This presents a major challenge for other languages, particularly those with sparse resources like [25], where research on ASR models for dysarthria remains scarce. Additionally, most studies report speech recognition performance below 70%. While the AAC system achieved the highest performance, this result was derived from a combination of specific word-level speech recognition and message reconstruction processes, making it incomparable to other studies under the same evaluation criteria. Furthermore, for ASR models to be effectively utilized in real-life communication assistance, sentence-level speech recognition is essential. Therefore, studies conducted at the word level face clear limitations in supporting real-time communication across diverse scenarios. Individuals with dysarthria often experience unpredictable variations in specific pronunciations, and these speech patterns exhibit distinct characteristics unique to each person. Moreover, there is a higher uncertainty at phoneme boundaries compared to standard speech. These peculiar speech patterns and the increased phoneme uncertainty are more pronounced depending on the language, indicating that dysarthric ASR systems must be custom-trained for each language. However, the majority of ongoing research is conducted using English datasets, with very little research on dysarthric ASR for minority languages, including Korean. The ultimate goal of this paper is to develop a communication aid system that combines a dysarthric ASR model and a Korean TTS model to transform the speech of individuals with dysarthria in real-time and output it as normal speech in their daily lives. By providing a framework for real-life communication assistance based on a high-performance  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Reference**} & \multirow{2}{*}{**Type**} & \multirow{2}{*}{**Method**} & **Speech** & \multirow{2}{*}{**Dataset**} & **Performance** \\  & & & **Recognition** & & **(WRA)** \\ \hline [PERSON] et al.