Severity Classification and Dysarthric Speech Detection using Self-Supervised Representations (2024)

Abstract: Automatic detection and classification of dysarthria severity from speech provides a non-invasive and efficient diagnostic tool, offering clinicians valuable insights to guide treatment and therapy decisions. Our study evaluated two pre-trained models—wav2vec2-BASE and distilALHuBERT, for feature extraction to build speech detection and severity-level classification systems for dysarthric speech. We conducted experiments on the TDSC dataset using two approaches: a machine learning model (support vector machine, SVM) and a deep learning model (convolutional neural network, CNN). Our findings showed that features derived from distilALHuBERT significantly outperformed those from wav2vec2-BASE in both dysarthric speech detection and severity classification tasks. Notably, the distilALHuBERT features achieved 99% accuracy in automatic detection and 95% accuracy in severity classification, surpassing the performance of wav2vec2 features.

## introduction dysarthria is a motor speech disorder caused by neurological damage that affects the mechanisms responsible for speech production. @xcite . this condition is the result of damage to the nervous system, which can occur due to either an acquired or congenital neurological illness. common causes of dysarthria include cerebral palsy, brain tumors, traumatic brain injuries, and strokes. additionally, it can develop as a result of neurodegenerative diseases such as parkinson's disease, amyotrophic lateral sclerosis (als), or huntington's disease, which progressively impair the body's motor functions. a range of abnormalities in different elements of speech production typically marks speech affected by dysarthria. these can include difficulties with phonation (the ability to produce vocal sound), resonance (the quality of the voice), articulation (the clarity of speech sounds), and prosody (the rhythm and intonation patterns of speech). these combined deficits often reduce speech intelligibility, making it difficult for listeners to understand individuals with dysarthria @xcite . speech intelligibility is traditionally evaluated by speech-language pathologists in voice clinics using standardized intelligibility tests @xcite . augmentative and alternative speech aids, like those in @xcite , show promise in enhancing dysarthric communication using confusion transducers to correct errors. however, these subjective tests are often expensive, time-consuming, and can be influenced by the pathologist's familiarity with the patient and their speech disorder, leading to potential bias @xcite . consequently, there is a growing need for an objective method to assess dysarthric speech. the evaluation process typically involves two key steps: (1) classifying the dysarthric speech from the acoustic speech signal,(2) determining the severity of the condition. this research aims to automate the detection of dysarthria and classify its severity level based on speech signals. in recent years, there has been significant interest in the automatic detection and severity classification of dysarthria from speech, driven by advancements in signal processing, machine learning (ml), and deep learning (dl). dysarthria detection methods typically involve a two-stage pipeline system, which includes feature extraction followed by classification. these systems are trained in a supervised fashion using labeled speech data, where the labels (e.g., healthy vs. dysarthric) are derived from evaluations performed by speech-language pathologists. numerous studies have explored various feature extraction techniques aimed at capturing the key characteristics of dysarthric speech production. @xcite investigated sentence-level features to examine abnormal variations in pronunciation, prosody, and voice quality. in the studies by @xcite and @xcite , single frequency filtering-based features were examined for the detection of dysarthric speech and its classification into four intelligibility levels (very low, low, medium, and high). in studies by @xcite and @xcite , linear weighted combinations of articulation, phonation, and prosody features were investigated for assessing the intelligibility of dysarthric speech. glottal source features, together with opensmile features @xcite , were effectively used by @xcite to improve classification performance in both dysarthric speech detection and intelligibility classification, utilizing support vector machines (svm) as the classifier. @xcite explored the usability of the extended geneva minimalistic acoustic parameter set (egemaps) @xcite in predicting phoneme intelligibility in dysarthric speech. recently, numerous studies have investigated various spectro-temporal representations, such as spectrograms, mel-spectrograms, and mfccs, combined with different deep learning (dl) classifiers like squeeze-and-excitation (se) networks, convolutional neural networks (cnns), residual neural networks (resnets), gated recurrent units (grus), and long short-term memory networks (lstms), for estimating the intelligibility of dysarthric speech studied the effectiveness of a multi-head attention mechanism to identify severity-emphasizing periods in spectrograms, alongside a multi-task learning approach for classifying dysarthria severity levels. a comprehensive systematic review of current studies on automatic classification of dysarthria severity levels was given by @xcite . self-supervised representation learning has gained significant interest in the field of paralinguistics, primarily due to the smaller size of datasets in this area compared to those used for tasks like automatic speech recognition (asr). this involves training a model in an unsupervised manner on large speech datasets, enabling it to learn speech representations directly from raw audio, which can then be applied to various tasks. notable examples of pre-trained models include wav2vec2 @xcite and distilalhubert @xcite , which have demonstrated strong performance across several speech-related tasks, such as asr, emotion recognition, speaker and language identification, and voice pathology detection. in this study, we investigate the effectiveness of two pre-trained models-wav2vec2 and distilalhubert-for the detection of dysarthria and the classification of its severity levels. our preliminary research on dysarthria detection and severity classification using the wav2vec2 model, applied to the ua-speech database @xcite , produced promising results. this initial success has inspired the current extended study. the pre-trained wav2vec2 and distilalhubert models utilized in this research are available on huggingface @xcite . the key contributions of this study are: a comprehensive comparison of wav2vec2 and distilalhubert embeddings, using them as features for two primary tasks: ## the detection and severity level classification systems this study investigates two classification tasks: binary and multi-class classification. the existing systems employ a two-stage pipeline approach that includes a feature extraction phase followed by a classification phase, as illustrated in fig. 1 . fig. 1 (a) depicts the system designed to differentiate dysarthric speech from healthy speech (i.e., the detection task), while fig. 1(b ) presents the system used to classify dysarthria severity into three categories: mild, moderate, and severe. in both tasks, the feature extraction component leverages two widely-used pre-trained models-wav2vec2-base @xcite and hubert @xcite -to derive feature vectors from raw speech signals. the classification phase utilizes both svm and cnn to predict the output labels. ## results this section outlines the results obtained from the features extracted using two pre-trained models (wav2vec2-base and distilalhubert) in combination with svm and cnn classifiers. the detection experiment results are presented first in section 4.1, followed by the severity classification experiment results in section 4.2.