Autism Speech Analysis using Acoustic Features (2019)

Abstract: Autism speech has distinct acoustic patterns, different from normal speech. Analyzing acoustic features derived from the speech of children affected with autism spectrum disorder (ASD) can help its early detection. In this study, a comparative analysis of the discriminating acoustic characteristics is carried out between ASD affected and normal children speech, from speech production point of view. Datasets of English speech of children affected with ASD and normal children were recorded. Changes in the speech production characteristics are examined using the excitation source features F0 and strength of excitation (SoE), the vocal tract filter features formants (F1 to F5) and dominant frequencies (FD1, FD2), and the combined source-filter features signal energy and zero-crossing rate. Changes in the acoustic features are compared in the five vowels’ regions of the English language. Significant changes in few acoustic features are observed for ASD affected speech as compared to normal speech. The differences between the mean values of the formants and dominant frequencies, for ASD affected and normal children, are highest for vowel /i/. It indicates that ASD affected children have possibly more difficulty in speaking the words with vowel /i/. This study can be helpful towards developing systems for automatic detection of ASD.

## introduction asd is a pervasive developmental disorder, defined clinically by observing the abnormalities in three areas: communication, social reciprocity, and hyperfocus or reduced behavioral flexibility @xcite @xcite . study shows, at least 50% of the total population of asd tends to show atypical acoustic patterns in their speech, and it persists throughout the improvement of other language aspects @xcite @xcite . in fact, the exact characteristics of autism and its underlying mechanisms are also unclear @xcite . according to study, 1 in 150 individuals with autism was reported in 2002, which became 1 in 68 in 2014 @xcite . it is reported that there are tens of millions of individuals with asd worldwide, and it is affecting approximately 1.5% of our total population @xcite . communication impairments, abnormal voice quality and disturbances of prosody are some of the most important aspects among individuals with asd who speak @xcite . individuals with asd speak with distinctive acoustic patterns in their speech, and as a result they face social interaction deficits @xcite . the reason behind the language impairment in autism is the result of primary linguistic disorder with a focus on pragmatic impairments @xcite . besides, the speech signal of the children with asd is reported as improperly modulated, wooden, and dull @xcite . in fact, in many cases, a significant spoken language delay and repetitive language can also be encountered @xcite . in general, normal children start establishing their vocabularies at the age of two years, whereas the children with asd may not be able to do the same @xcite . previous studies mostly based on either speech prosody or unusual suprasegmental features of speech production of children with asd @xcite . like, in @xcite , authors had reported the segmental and suprasegmental speech features of individuals with high-functioning autism (hfa). also, some studies used objective measures to quantify speech related issues in autism @xcite . some of the most significant analyses based on pitch features of individuals with asd were reported in @xcite , @xcite , etc., where in each study authors had reported different result from others. for instance, in @xcite , authors had reported higher pitch value for asd children as compared with normal children. on the other hand, in @xcite , authors had reported lower pitch value for asd children as compared with normal children. besides, in the case of the intensity based analyses, some of the studies indicated no significant differences between asd and normal children @xcite . likewise, based on duration (syllable duration, utterance duration, etc.), voice patterns, speech rate, etc., researchers had done some significant analyses on individuals with asd @xcite @xcite . but, none of the previous studies had done only on english vowels, especially pronounced by non-native indian english speakers with asd. also, many robust speech features like dominant frequencies (fd1, fd2), strength of excitation (soe), etc., had not been considered in previous studies. therefore, in this study, we have considered all these mentioned points. this paper analyzed the autism speech, i.e., the speech signal of the children with asd, by differentiating them from the normal children. differences are made in terms of the speech production features of the asd and the normal children. here, only english vowels, i.e., /a/, /e/, /i/, /o/, and /u/ are taken into consideration, because of their relatively longer duration in the case of children with asd. also, the production of vowels sounds by an individual is not a random process; hence it is important to find characteristics of the speech production mechanism of children with asd during the pronunciation of vowels sounds. this study on analyzing the speech production characteristics of the children with asd has high importance, because it may play a vital role in improving the communication impairments associated with asd. in addition, current diagnostic criteria for asd do not include any atypical vocalizations @xcite . hence, this study can be utilized as a diagnostic marker to identify this study consists of four major steps. firstly, two speech signal datasets were collected, by recording the sound files of the asd and the normal children. secondly, unwanted signal parts were removed, and the speech signal files were arranged in two different databases for the asd and the normal children. thirdly, speech signal processing methods were applied on the collected datasets to extract the selected production features. finally, results were made by differentiating between the asd and the normal children in terms of their speech production features. the rest of the paper is organized as follows. details about the two collected datasets of the asd and the normal children are discussed in section 2. next, the signal processing methods and features used for analyses are discussed in section 3. section 4 presents key results and observations on results. then, section 5 discusses the analyses of observed results in speech production point of view. section 6 represents key contributions. lastly, section 7 presents conclusions, along with the scope of future work on this topic. ## speech datasets of asd and normal children two speech signal datasets in the english language were recorded for this study, where one dataset contains the speech samples of 13 children with asd, and another dataset contains the speech samples of 20 normal children. details of both the datasets are given in the age of 3 years when a child begins to show delays in developmental milestones @xcite . another reason was that the current study only focused on verbal children. besides, in the case of the children with asd, it was made sure by a well-experienced doctor and a psychologist that the children considered were diagnosed with asd. the children with asd considered for the data collection met the dsm-iv diagnostic criteria @xcite . furthermore, all the children with asd considered here had distinctive acoustic patterns in their speech, during the entire period of data collection. however, the normal children did not have any such issues and were living a normal life. speech samples were recorded every week (once or twice), for a period of over 1 year. recordings took place in a noise-free empty room, which did not have any object that could distract the children. also, the neutral emotional state of the children was affirmed during all the data collection sessions. the asd and the normal children were asked to name in english a set of 25 specifically selected daily life pictures, shown to them along with each picture's name in english on a laptop. the pictures consisted of animals, vegetables, flowers, and english numbers. all the children were asked to pronounce only the object's name as a word, presented to them in the form of a picture. the children's first response was confronted by asking them to pronounce the picture's name. then, we kept changing the pictures one by one, while the children named the object shown as a picture. each child was asked to name the same set of pictures over each of the recording sessions. five different pictures were selected for each of five english vowels, and the names of all the pictures were either in consonant-vowel-consonant (cvc) or consonant-vowel-vowel-consonant (cvvc) word format. the total utterances of 25 words by each child (5 vowels × 5 words) were recorded in each of the two such sessions, in a day. roland r-26 digital audio recorder was used with 48 khz sampling rate to record the speech samples. the distance of 25 cm was maintained between the recorder and the speaker's mouth. our collected datasets have immense impor-tance because of several reasons. firstly, all the children considered here were non-native indian english speakers. whereas, in previous studies like @xcite , @xcite , @xcite , @xcite , etc., authors had not considered non-native indian english speakers(children) with asd. secondly, in previous studies datasets were mostly collected from social interaction @xcite , constrained production @xcite and spontaneous production @xcite . but, here the datasets were recorded differently, as described earlier in this section. ## signal processing methods and features the production characteristics of speech signal of the asd and the normal children are differentiated by examining changes in the source features, vocal tract system features and combined sourcefilter features. the source features f0 and strength of excitation (soe), and the vocal tract filter features dominant frequencies (fd1, fd2) and first five formants (f1 to f5) are examined. the combined source-filter features signal energy (e) and zero-crossing rate (zcr) are also examined. here, for each speech feature, the mean (µ) or average values are computed. the mean values are computed for each english vowel by taking the average of all the calculated values of a particular speech feature, and this procedure is followed for each speaker. besides, the µ soe , µ e and µ zcr values are multiplied by 100, 1000, and 1000, respectively, for a better understanding. ## results and observations the obtained results indicate higher µ f 0 values for the children with asd as compared with the normal children, and this statement is true for all english vowels. besides, according to the tongue position, female children with asd have the highest µ f 0 value for mid-vowel /e/ and have the lowest µ f 0 value for low-vowel /a/ as compared with other english vowels. but, in the case of the normal female children, high-vowel /i/ gives the highest and mid-vowel /o/ gives the lowest µ f 0 values as compared with other english vowels. however, in the case of the male children with asd, such results have not been found. it is observed that male children with asd follow a similar µ f 0 trend with the normal male children for all english vowels. these results can be analyzed from like µ f 0 , in the case of µ e also, the children with asd have higher values for all the five english vowels as compared with the normal children. also, for all the five english vowels, the female children with asd have higher µ e values as compared with the male children with asd, but this is vice versa for the normal children. besides, in the case of the children with asd, the same vowel /e/ has the lowest µ e values for both male and female children, whereas this is not the same for the normal male and female children. likewise, in the case of the normal children, the same vowel /o/ has the highest µ e values for both male and female children, whereas this is not true for the male and female children with asd. these statements can be observed from µ e values in regarding µ soe , only front vowels /e/ and /i/ indicate lower values for the children with asd as compared with the normal children. but, in the case of mid and rear vowels, i.e., /a/, /o/, and /u/, µ soe indicate higher values for the children with asd than the normal children. besides, in the case of both the normal male and female children, the same vowel /i/ has the highest µ soe values as compared with other english vowels. but, this statement is not true in the case of the children with asd. again, in the case of both the male and female children with asd, the same vowel /a/ has the lowest µ soe values as compared with other english vowels, whereas this is not the case with the normal children. all these results can be observed from µ soe values, tabulated in the µ zcr have lower values for the children with asd as compared with the normal children, and it is true for all english vowels. this observation is graphically represented in figure 1(g ) and 1(h). also, in the case of the front and mid vowels, i.e., /a/, /e/, and /i/, the male children with asd have higher µ zcr values as compared with the female. but, it is vice versa in the case of the normal children. besides, in the case of both male and female children with asd, the same vowel /e/ has the lowest µ zcr values as compared with other english vowels, whereas this is not the case with the normal children. these results can be observed from µ zcr values, given in the µ f 2 values are higher for all english vowels in the case of the children with asd as compared with the normal children. also, the µ f 2 values for all the five english vowels of both the male and female children with asd follow a similar trend, whereas there is no such trend observed in the case of the normal children. besides, according to the tongue position, both the male and female children with asd have the highest µ f 2 values for the mid vowel /e/ as compared with the high and low vowels. but, in the case of the normal children, as compared with the mid and low vowels the high vowels /i/ and /u/ give the highest µ f 2 values for both the male and female children, respectively. all these results can be analyzed from µ f 2 values tabulated in as compared with the normal children, the children with asd have higher µ f 4 values for the front and mid vowels only. next, according to the tongue position, in the case of both the male and female children with asd, the µ f 4 gives the highest values for the mid vowel /o/ as compared with the high and low vowels. but, this is not the case for the normal children. the µ f 4 results can be analyzed from the figure 2 (g) and 2(h), also from the µ f 3 values, tabulated in the µ f 5 indicates higher values for all the five english vowels in the case of the children with asd as compared with the normal children, depicted in figure 2 (i) and 2(j). also, both the male and female normal children have the lowest µ f 5 values for the mid vowel /o/ as compared with the high and low vowels. but, this statement is not true in the case of the asd children. the µ f 5 values are tabulated in all the five english vowels have higher µ f d1 values for the children with asd as compared with the normal children, depicted in figure 3(a) and 3(b). according to the tongue position, both the male and female normal children have the lowest µ f d1 values for the high vowel /i/ as compared with the mid and low vowels. but, in the case of the asd children, as compared with the high and low vowels the mid vowels /e/ and /o/ indicate the lowest µ f d1 values for both the female and male, respectively. the µ f d2 results can be analyzed from in the case of µ f d2 , only the front vowel /e/ and /i/ have higher values for the children with asd as compared with the normal children, graphically shown in figure 3 (c) and 3(d). in addition, according to the tongue position, both the male and female normal children have the highest µ f d2 values for the low vowel /a/ as compared with the mid and high vowels. on the other hand, as compared with other english vowels the high vowel /u/ has the highest µ f d2 value for the male asd group and the mid vowel /e/ has the highest µ f d2 value for the female asd group. the µ f d2 values are tabulated in ## analyses of results this section describes the observed results in speech production point of view. firstly, the f0 which reveals the source characteristics of the speech production system, the result infers that in the case of all the five english vowels, the male and female children with asd have a higher vo-cal fold vibration rate than the normal male and female children. this statement is true for all the five english vowels. furthermore, in the case of female children with asd, mid-vowel /e/ has the highest and low-vowel /a/ has the lowest vocal fold vibration rate as compared with other english vowels. on the other hand, in the case of the normal female children, high-vowel /i/ has the highest and mid-vowel /o/ has the lowest vocal fold vibration rate as compared with other english vowels. these observations can be analyzed from figure 1 (a) and 1(b). in the case of e which gives the information about the combined source-system characteristics of the speech production system, the result implies that the children with asd have louder speech and put more vocalization effort than the normal children. also, in the case of all english vowels the female children with asd put more vocalization effort than the male children with asd, but this is vice versa in the case of the normal group. these results can be analyzed from µ e values graphically depicted in figure 1(c ) and 1(d). the observed soe result infers that in the case of the front vowels the strength of impulse-like excitation is lower during the glottal activity (vibration of vocal folds) of the children with asd as compared with the normal children. but, in the case of mid and rear vowels the strength of impulse-like excitation is higher for the asd children than the normal children. this result can be analyzed from figure 1(e ) and 1(f). the f1 result implies that in the case of all five english vowels, the children with asd have a lesser oral constriction in the front half of the oral section of the vocal tract as compared with the normal children. again, in terms of pharyngeal constriction, it can be stated that during the pronunciation of all the five english vowels the pharyngeal constriction is greater for the children with asd as compared with the normal children. the f1 observed result also implies that both the male and female children with asd have the greatest pharyngeal constriction for the low-vowel /a/ as compared with the mid and high vowels. but, the normal male and female children have the greatest pharyngeal constriction for the mid-vowel /o/ as compared with the high and low vowels. furthermore, during the pronunciation of all english vowels the children with asd increase their tongue higher than the normal children. because the f1 value increases with increasing the tongue position higher. the f1 values for all english vowels are graphically depicted in figure 2(a) and 2(b). the f2 result implies that in the case of all english vowels the back tongue constriction is lesser and the front tongue constriction is greater for the children with asd than the normal children. furthermore, it can be stated from the observed result that both the male and female children with asd have the least back tongue constriction and the greatest front tongue constriction for the mid vowel /e/ as compared with the high and low vowels. on the other hand, the normal male children have the least back tongue constriction and the greatest front tongue constriction for the highvowel /i/ as compared with the mid and low vowels, and the normal female children have the least back tongue constriction and the greatest front tongue constriction for the high-vowel /u/ as compared with the mid and low vowels. this observation can be analyzed from figure 2(c) and 2(d ). the f3 result implies that in the case of the children with asd lip-rounding is lesser during the pronunciation of all english vowels. hence, the constriction is least and as a result all english vowels give higher µ f 3 frequency values for the children with asd as compared with the normal children. the results are graphically depicted in figure 2 (e) and 2(f). also, the results of the first three formants (f1, f2 and f3) indicate that the length of the pharyngeal-oral tract is shorter in the case of the children with asd as compared with the normal children. because, the formants values of vowels are inversely proportional to the pharyngeal-oral tract, and here the children with asd have higher µ f 1 , µ f 2 and µ f 3 values for all english vowels as compared with the normal children. also, in terms of the lip-rounding, the f1, f2, f3 and f5 results imply that the children with asd have a lesser liprounding as compared with the normal group. in the case of formants frequencies and dominant frequencies, the differences between the asd and the normal children are highest for vowel /i/. it implies that asd children have probably more difficulty in pronouncing the words with vowel /i/. ## key contributions the key contributions of this study are as follows: ## conclusions the aim of this study is to analyze differences in various speech production features of the children with asd as compared with the normal children. only english vowels sounds are used in this study. an autism speech dataset and a normal childrens speech dataset are recorded separately for this research purpose. then, differences between the children with asd and the normal children are analyzed by observing the source characteristics (f0 and soe), system characteristics (dominant frequencies and formants), and combined characteristics (zcr and e). it is observed that there are significant differences between the asd and the normal children, in terms of their speech production characteristics in english vowels regions. in the case of most of the speech production features, the asd children have significantly higher values than the normal children. these acoustic characteristics of the children with asd can be used as markers to identify asd. but, we did not find any single speech feature that can be utilized as a diagnostic marker for asd. a small size of speech data for female asd children is a limitation of this study. in future studies, we will try to find a single speech feature that can be utilized as an acoustic marker to identify asd.