D y PCL : Dynamic Phoneme-level Contrastive Learning for Dysarthric Speech Recognition (2025)

Abstract: Dysarthric speech recognition often suffers from performance degradation due to the intrinsic diversity of dysarthric severity and extrinsic disparity from normal speech. To bridge these gaps, we propose a Dynamic Phoneme-level Contrastive Learning (DyPCL) method, which leads to obtaining invariant representations across diverse speakers. We decompose the speech utterance into phoneme segments for phoneme-level contrastive learning, leveraging dynamic connectionist temporal classification alignment. Unlike prior studies focusing on utterance-level embeddings, our granular learning allows discrimination of subtle parts of speech. In addition, we introduce dynamic curriculum learning, which progressively transitions from easy negative samples to difficult-to-distinguishable negative samples based on phonetic similarity of phoneme. Our approach to training by difficulty levels alleviates the inherent variability of speakers, better identifying challenging speeches. Evaluated on the UASpeech dataset, DyPCL outperforms baseline models, achieving an average 22.10% relative reduction in word error rate (WER) across the overall dysarthria group.

## introduction accurate recognition of dysarthric speech, which is slurred and difficult to understand, is critical for assisting effective communication for individuals with speech impairments @xcite . however, due to the inherent diversity of severity levels and substantial differences compared to normal speech, dysarthric speech recognition (dsr) poses significant challenges. previous studies mainly focused on data augmentation @xcite @xcite and speaker-adaptive training @xcite @xcite . however, relying on addi-* equally contributed tional augmentation techniques or feature extraction methods limits the practical applicability. contrastive learning has been explored in dsr to learn invariant representations by using healthy speech as a stable reference point @xcite . the model can more effectively capture the underlying linguistic content in dysarthric speech by anchoring the learning process on phonetic embeddings from healthy speakers despite surface-level variations. for instance, @xcite applied pyramid pooling to distinguish words within audio, while @xcite used word-level contrastive learning with entire audio segments. however, word-level embeddings fail to achieve fine-grained recognition, which is crucial for dysarthric speakers with distinct pronunciation challenges. in this paper, we propose a dynamic phonemelevel contrastive learning (dypcl) framework, integrating phoneme-level speech embedding with contrastive learning. dypcl incorporates twoway dynamic approaches: dynamic connectionist temporal classification (ctc) alignment and dynamic curriculum learning. first, we introduce a dynamic ctc alignment method that accurately aligns speech embeddings with phoneme labels for phoneme-level contrastive learning. unlike previous approaches that rely on external alignment modules for phoneme-level contrastive learning @xcite , dynamic ctc alignment simultaneously learns robust feature representations. it aligns speech sequences with their corresponding phonemes during training, eliminating the need for explicit frame-level annotations. in addition, we introduce dynamic curriculum learning, which dynamically organizes negative samples based on difficulty, determined by the similarity distance of the anchor and negative phoneme in pcl. this phonetic approach further enhances dsr performance by effectively distinguishing between similar-sounding phonemes, a critical factor in dysarthric speech. evaluated on the uaspeech dataset, a representative dsr benchmark, our method shows substantial improvements over baseline models. specifically, in the lowest intelligibility group, dypcl reduces the word error rate (wer) from 58.49% to 49.45%, while overall wer across all dysarthria groups drops from 25.97% to 20.23%. extensive analysis and ablation studies highlight the robustness of the proposed strategies. contrastive learning contrastive learning in speech recognition has proven effective in various atypical scenarios, such as noisy environments @xcite and accented speech datasets @xcite , due to its ability to learn robust speech representations by reducing speech embedding variability. for dsr, contrastive learning has been applied to reduce the distance between healthy utterance and dysarthric utterance representations @xcite using the word or utterance fragments. to consider subtle segments during training, phoneme-level contrastive learning has been suggested for accented speech recognition @xcite , but it has yet to be explored in dsr. to perform phoneme-level contrastive learning, sophisticated phoneme alignment for a given user utterance is required. previous research explored various alignment methods @xcite @xcite to match given phonemes to audio frames closely. notably, using ctc forced alignment has shown great alignment accuracy @xcite . this work proposes phonemelevel dynamic ctc alignment that dynamically adapts during optimizing contrastive learning. in addition, considering the importance of hard negative sampling for contrastive learning @xcite , we dynamically select negative samples across varying difficulty groups and phoneme distance levels, integrating with the curriculum learning process; thus, the model can learn invariant representations. ## conclusion this paper introduced the dynamic phoneme-level contrastive learning (dypcl) framework to im-prove dysarthric speech recognition. dypcl effectively tackles phoneme alignment challenges and accounts for phonetic difficulty through dynamic ctc alignment and curriculum learning. our experiments on the uaspeech dataset demonstrated the effectiveness of dypcl, which reduced the wer from 58.49% to 49.45% in the very low (vl) intelligibility group and the overall wer across all dysarthria groups from 25.97% to 20.23%. these results underscore dypcl's capability to capture subtle phonetic variations, significantly enhancing speech recognition accuracy across all levels of dysarthria severity