Spectral modification for recognition of children’s speech undermismatched conditions (2021)

Abstract: In this paper, we propose spectral modification by sharpening formants and by reducing the spectral tilt to recognize children’s speech by automatic speech recognition (ASR) systems developed using adult speech. In this type of mismatched condition, the ASR performance is degraded due to the acoustic and linguistic mismatch in the attributes between children and adult speakers. The proposed method is used to improve the speech intelligibility to enhance the children’s speech recognition using an acoustic model trained on adult speech. In the experiments, WSJCAM0 and PFSTAR are used as databases for adults’ and children’s speech, respectively. The proposed technique gives a significant improvement in the context of the DNN-HMM-based ASR. Furthermore, we validate the robustness of the technique by showing that it performs well also in mismatched noise conditions.

## introduction recent advances in asr have impacted many applications in various fields, such as education, entertainment, home automation, and medical assistance @xcite . these applications can benefit children in their daily life, in playing games, reading tutors @xcite , and learning both native and foreign languages @xcite . the task of speech parameterization for the front-end aims at a compact representation that captures the relevant information in the speech signal by using short-time feature vectors. the two commonly used feature sets are mel-frequency cepstral coefficients (mfcc) @xcite and the perceptual linear prediction cepstral coefficients (plpcc) @xcite . speech of adults and children have large acoustic and linguistic differences @xcite @xcite . both the mel-filterbank and plp coefficients are better suited for adults as they provide better resolution for low-frequency contents while a greater degree of averaging happens in the highfrequency range @xcite . in the case of children's speech, more relevant information is available in the high-frequency range. therefore, to enhance the system performance, a better resolution needs to be used for the high-frequency range. previous studies have also shown that formant sharpening is helpful for increasing speech intelligibility @xcite @xcite . motivated by these observations, we suggest to modify the speech spectrum by formant sharpening and spectral tilt reduction. in @xcite @xcite , it was shown that the word error rate (wer) in recognition of children's speech is much higher than that of adult speech and specifically under mismatched and noisy conditions. the problems are due to higher inter-speaker variance caused by the development of the vocal tract, leading to different formant locations and spectral distribution @xcite , and due to the inaccuracy in pronunciation and grammar caused by language acquisition. most importantly, the insufficient training data limits the performance because collecting large speech databases of children's speech is hard. adult speech corpora normally contain hun-dreds or thousands of hours of data, while most publicly available corpora for children's speech have less than 100 hours of data @xcite . therefore, it is necessary that asr systems built for children are robust for various mismatched conditions. in this paper, a spectral sharpening and tilt reduction method is proposed to enhance the intelligibility of children's speech to boost the asr system performance under mismatched conditions. spectral sharpening and spectral tilt reduction have been used in enhancement of speech intelligibility in noise @xcite . in this study, it is shown that the mfcc and plpcc features computed after the spectral modification (referred to as ss-mfcc and ss-plpcc) are found to outperform the conventional mfcc and plpcc features. this is demonstrated by both the spectral analyses and experimental evaluations in this paper. the robustness of the technique is further validated by showing that it performs well in mismatched noise conditions also. the remaining of this paper is presented as follows: in section 2, the proposed spectral sharpening and tilt reduction technique is discussed. in section 3, the speech corpora and asr specifications are described. the results of the proposed method are presented in section 4. in section 5, the effects of noisy environment on the proposed method are discussed. finally, the paper is concluded in section 6. ## the spectral modification method the proposed spectral modification technique consists of formant sharpening and spectral tilt reduction as described below and depicted in the block diagram in fig 1 . from the spectral examples shown in fig 2 and spectrograms shown in fig 3 , we can observe that the proposed method enhances formant peaks and the level of higher frequencies. ## data and experimental setup this section describes the speech corpora (adult and children), front-end speech features and specifications of asr system. ## results and discussion the baseline wers for children's test set with respect to the dnn-hmm-based asr systems trained using the mfcc and plpcc features are given in for further analysis, the children test data was divided into three different test sets based on age groups: 4 -6 years, 7 -9 years, and 10 -13 years. to further validate the effectiveness of the proposed modification method, another dnn-hmmbased asr system was developed by pooling together speech data from training sets of both adults and children. for children's speech, the training set derived from pf-star consisted of 8.3 hours of speech by 122 speakers. the total number of utterances in this training set was equal to 856 with a total of 46974 words. the training set of adult speakers consisted of 15.5 hours of speech from 92 speakers (both male and female). further, the training set comprised 132, 778 words and the total number of utterances was 7852. the developed asr system exhibits a lower degree of acoustic/linguistic mismatch due to the pooling of children's speech into training. as a result, the baseline wers for the developed system (given in ## experiments in noisy conditions to further validate the proposed technique, noise robustness of the spectral modification technique was studied. four different noises (babble, white, factory and volvo noise) extracted from noisex-92 @xcite were added to the test data under varying snr levels. the noisy test sets were then decoded using the acoustic models trained with clean speech. wers in the case of adult/child mismatched testing are given in ## conclusion this work explores spectral modification (sharpening of formants and reduction of spectral tilt) to achieve robust recognition of children's speech under mismatched conditions. the explored spectral modification technique is observed to enhance asr of children's speech for both the mfcc and plpcc features. also, asr results are analyzed for different age-groups and it was found that for all the age-groups there exists an improvement with the proposed approach compared to baseline. further, improvements were also observed in mismatch conditions caused by additive noise.