Pronunciation scoring for dysarthric speakers with DNN - HMM based goodness of pronunciation ( G o P ) measure (2024)

Abstract: Dysarthria is a neurological motor disorder caused by cranial damage that interferes with the muscles involved in the correct pronunciation of sounds and intelligible speech. Computer Aided Pronunciation training (CAPT) systems traditionally used for the pronunciation assessment of L2 language learners can offer a method to detect and score mispronounced sounds in dysarthric speakers as a way of evaluation without human intervention. In this work, a phonetic level DNN-HMM based Goodness of Pronunciation (GoP) for pronunciation scoring, on native Tamil Dysarthric speakers corpus is presented. The scores are calculated using the posteriors of the subphonemic elements called senones with a focus on their prevalence across phones and their transitions across HMM states. The phonetic-level scores obtained for speakers of different levels of severity help establish speaker-specific trends in pronunciation through an objective log-likelihood metric, in contrast to subjective evaluations by Speech Language Therapists (SLTs).

## introduction damage caused to the brain upon injury or through a developmental disorder results in dysarthria, a neuromotor speech disorder leading to the discoordination of the speech production muscles which results in dysarthric speech sounding nasal, breathy, jerky, raspy, harsh and incorrect @xcite , and suffering reduced intelligibility. @xcite and @xcite note that current methods of assessment of dysarthria are performed by speech language therapists (slts) through case histories, non-speech examinations of speed, strength and steadiness of speech musculature, and examinations of the phonation, articulation and prosody. treatment comprises general exercises to strengthen respiration, phonation and articulation to improve the range and strength of muscles involved in speech production (en-derby, 2013) administered by slts. previous work @xcite has discussed the development of a speech communicative aid to differentiate pronunciation and system errors, and subsequently correct dysarthric speech electronically. however, this work focuses on analysing speakerspecific weaknesses in their speech production system using place of articulation analysis which can guide slts to tailor customized treatment for patients and track their progress over time. computer aided pronunciation training (capt) systems have been long used for phone-level pronunciation assessment among l2 language learners. such systems use automatic speech recognizers (asrs) to decode the phonemes uttered, followed by an evaluation or feedback system to score the learner's pronunciation of the words @xcite . consistent research has depicted the superiority of deep neural networks-based asrs (dnn) over gaussian mixture model-asrs, with improved word error rates (wers) @xcite . @xcite proposed the goodness of pronunciation (gop) metric based on a log-likelihood ratio to evaluate segment-level pronunciation. following various reformulations and revisions especially suited to dnn-based approach, the evaluation of the gop using the sub-phonemic posterior probabilities along with state transition probabilities (stps) as outlined by @xcite has been used in the mispronunciation detection and scoring task of the ssnce tamil dysarthric speech corpus @xcite with a modification to account for the empirical senone distribution as actually appearing in the corpus. section 2 of this paper outlines the design of the mispronunciation evaluation system, explaining the development of the asr, the goodness of pronunciation formulation and the data used in the work. section 3 describes the experimental setup and implementation aspects. the results and subsequent analyses are presented in section 4. ## methods the mispronunciation evaluation system consists of two main parts, the automatic speech recognition (asr) phase and the gop evaluation phase. ## results figure 1 illustrates the spectrogram for an utterance. presented below are the overall frame-level gop scores for every phone. the frame-level gop scores are smoothened to obtain the phone-level gops, based on the number of frames allotted to each canonical phone during the forced-alignment phase, which are depicted in the third plot. a comparison of phone-level gop scores is shown in the time-normalised plots figure 2 , presenting the variations in scores across contiguous phones of the same utterance between a mild and moderate dysarthric speaker. it can be seen that for the mild speaker (blue), the variations in pronunciation within an utterance are lesser than that of a moderate speaker (green). further analysis of speaker-specific phonetic-level pronunciation is performed for all 20 dysarthric speakers for five classes of phones based on their places of articulation. the phone scores based on the proposed approach, for selected speakers in the three dysarthric categories are presented in the scores offer insights into speaker-specific differences in the articulation of different phones and inter-speaker pronunciation variations within the same dysarthric category. figure 2: a comparison of a mild (blue) and a moderate (green) speakers' phone-level scores for the utterance "kadxawulxai wanxanggeu"