Introduction
Parkinson's disease (PD) is a neurodegenerative disorder characterized by the progressive loss of dopaminergic neurons in the mid-brain producing several motor and non-motor impairments in the patients BIBREF0. Motor symptoms include among others, bradykinesia, rigidity, resting tremor, micrographia, and different speech impairments. The speech impairments observed in PD patients are typically grouped as hypokinetic dysarthria, and include symptoms such as vocal folds rigidity, bradykinesia, and reduced control of muscles and limbs involved in the speech production. The effects of dysarthria in the speech of PD patients include increased acoustic noise, reduced intensity, harsh and breathy voice quality, increased voice nasality, monopitch, monoludness, speech rate disturbances, imprecise articulation of consonants BIBREF1, and involuntary introduction of pauses BIBREF2. Clinical observations in the speech of patients can be objectively and automatically measured by using computer aided methods supported in signal processing and pattern recognition with the aim to address two main aspects: (1) to support the diagnosis of the disease by classifying healthy control (HC) subjects and patients, and (2) to predict the level of degradation of the speech of the patients according to a specific clinical scale.
Most of the studies in the literature to classify PD from speech are based on computing hand-crafted features and using classifiers such as support vector machines (SVMs) or K-nearest neighbors (KNN). For instance, in BIBREF3, the authors computed features related to perturbations of the fundamental frequency and amplitude of the speech signal to classify utterances from 20 PD patients and 20 HC subjects, Turkish speakers. Classifiers based on KNN and SVMs were considered, and accuracies of up to 75% were reported. Later, in BIBREF4 the authors proposed a phonation analysis based on several time frequency representations to assess tremor in the speech of PD patients. The extracted features were based on energy and entropy computed from time frequency representations. Several classifiers were used, including Gaussian mixture models (GMMs) and SVMs. Accuracies of up to 77% were reported in utterances of the PC-GITA database BIBREF5, formed with utterances from 50 PD patients and 50 HC subjects, Colombian Spanish native speakers. The authors from BIBREF6 computed features to model different articulation deficits in PD such as vowel quality, coordination of laryngeal and supra-laryngeal activity, precision of consonant articulation, tongue movement, occlusion weakening, and speech timing. The authors studied the rapid repetition of the syllables /pa-ta-ka/ pronounced by 24 Czech native speakers, and reported an accuracy of 88% discriminating between PD patients and HC speakers, using an SVM classifier. Additional articulation features were proposed in BIBREF7, where the authors modeled the difficulty of PD patients to start/stop the vocal fold vibration in continuous speech. The model was based on the energy content in the transitions between unvoiced and voiced segments. The authors classified PD patients and HC speakers with speech recordings in three different languages (Spanish, German, and Czech), and reported accuracies ranging from 80% to 94% depending on the language; however, the results were optimistic, since the hyper-parameters of the classifier were optimized based on the accuracy on the test set. Another articulation model was proposed in BIBREF8. The authors considered a forced alignment strategy to segment the different phonetic units in the speech utterances. The phonemes were segmented and grouped to train different GMMs. The classification was performed based on a threshold of the difference between the posterior probabilities from the models created for HC subjects and PD patients. The model was tested with Colombian Spanish utterances from the PC-GITA database BIBREF5 and with the Czech data from BIBREF9. The authors reported accuracies of up to 81% for the Spanish data, and of up to 94% for the Czech data.
In addition to the hand-crafted feature extraction models, there is a growing interest in the research community to consider deep learning models in the assessment of the speech of PD patients BIBREF10, BIBREF11, BIBREF12. Deep learning methods have the potential to extract more abstract and robust features than those manually computed. These features could help to improve the accuracy of different models to classify pathological speech, such as PD BIBREF13. A deep learning based articulation model was proposed in BIBREF11 to model the difficulties of the patients to stop/start the vibration of the vocal folds. Transitions between voiced and unvoiced segments were modeled with time-frequency representations and convolutional neural networks (CNNs). The authors considered speech recordings of PD patients and HC speakers in three languages: Spanish, German, and Czech, and reported accuracies ranging from 70% to 89%, depending on the language. However, in a language independent scenario, i.e., training the CNN with utterances from one language and testing with the remaining two, the results were not satisfactory (accuracy$<60\%$).
The classification of PD from speech in different languages has to be carefully conducted to avoid bias towards the linguistic content present in each language. For instance, Czech and German languages are richer than Spanish language in terms of consonant production, which may cause that it is easier to produce consonant sounds by Czech PD patients than by Spanish PD patients. Despite these language dependent issues, the results in the classification of PD in different languages could be improved using a transfer learning strategy among languages, i.e., to train a base model with utterances from one language, and then, to perform a fine-tuning of the weights with utterances from the target language BIBREF14. Similar approaches based on transfer learning have been recently considered to classify PD using handwriting BIBREF15. In the present study, we propose a methodology to classify PD via a transfer learning strategy with the aim to improve the accuracy in different languages. CNNs trained with utterances from one language are used to initialize a model to classify speech utterances from PD patients in a different language. The models are evaluated with speech utterances in Spanish, German, and Czech languages. The results suggest that the use of a transfer learning strategy improved the accuracy of the models over 8% with respect to those obtained when the model is trained only with utterance from the target language.
Materials and methods ::: Data
Speech recordings of patients in three different languages are considered: Spanish, German, and Czech. All of the recordings were captured in noise controlled conditions. The speech signals were down-sampled to 16 kHz. The patients in the three datasets were evaluated by a neurologist expert according to the third section of the movement disorder society, unified Parkinson's disease rating scale (MDS-UPDRS-III) BIBREF16. Table TABREF5 summarizes the information about the patients and healthy speakers.
Materials and methods ::: Data ::: Spanish
The Spanish data consider the PC-GITA corpus BIBREF5, which contains utterances from 50 PD patients and 50 HC, Colombian Spanish native speakers. The participants were asked to pronounce a total of 10 sentences, the rapid repetition of /pa-ta-ka/, /pe-ta-ka/, /pa-ka-ta/, /pa/, /ta/, and /ka/, one text with 36 words, and a monologue. All patients were in ON state at the time of the recording, i.e., under the effect of their daily medication.
Materials and methods ::: Data ::: German
Speech recordings of 88 PD patients and 88 HC speakers from Germany are considered BIBREF17. The participants performed four speech task: the rapid repetition of /pa-ta-ka/, 5 sentences, one text with 81 words, and a monologue.
Materials and methods ::: Data ::: Czech
A total of 100 native Czech speakers (50 PD, 50 HC) were considered BIBREF18. The speech tasks performed by the participants include the rapid repetition of the syllables /pa-ta-ka/, a read text with 80 words, and a monologue.
Materials and methods ::: Segmentation
Speech signals are analyzed based on the automatic detection of onset and offset transitions, which model the difficulties of the patients to start/stop the movement of the vocal folds. The detection of the transitions is based on the presence of the fundamental frequency of speech in short-time frames, as it was shown in BIBREF7. The border between voiced and unvoiced frames is detected, and 80 ms of the signal are taken to the left and to the right, forming segments with 160 ms length. The transition segments are modeled with two different approaches: (1) a baseline model based on hand-crafted features, which are classified using an SVM, and (2) a model based on time-frequency representations used as input to train a CNN, which then will be used for the transfer learning strategy. Further details are given in the following subsections.
Materials and methods ::: Baseline model
The features extracted from the transitions include 12 Mel-Frequency Cepstral Coefficients (MFCCs) with their first and second derivatives, and the log energy of the signal distributed into 22 Bark bands. The total number of descriptors corresponds to 58. Four statistical functionals (mean, standard deviation, skewness, and kurtosis) are computed for each descriptor, obtaining a 232-dimensional feature-vector per utterance. The classification of PD patients and HC speakers is performed with a radial basis SVM with margin parameter $C=10$ and a Gaussian kernel with parameter $\gamma =0.0001$. The SVM is tested following a 10-fold Cross-Validation strategy, speaker independent.
Materials and methods ::: CNN model
Time frequency representations based on the short-time Fourier transform (STFT) are used as input to a CNN, which extract the most suitable features to discriminate between PD patients and HC subjects. The STFT with 256 frequency bins is computed for each segmented transition, for a window length of 16 ms and a step-size of 4 ms, forming 41 time frames per transition. The obtained spectrogram is transformed into the Mel-scale using 80 filters, forming an spectrogram with a size of 80$\times $41, which is used to train the CNNs. The architecture of the implemented CNN is summarized in Table TABREF9. It consists of four convolutional and max-pooling layers, dropout to regularize the weights, and two fully connected layers followed by the output layer to make the final decision using a softmax activation function. The number of feature maps on each convolutional layer is twice the previous one in order to get more detailed representations of the input space in the deeper layers. The CNN is trained using the the cross-entropy as the loss function, using an Adam optimizer BIBREF19.
Materials and methods ::: Transfer learning
Transfer learning allows to use a neural network trained for one task to be used in another domain. We use transfer learning to classify patients and healthy speakers in three different languages. The CNN architecture described before is used to train a CNN with utterances from one language. Then, the pre-trained model is used as a base to initialize two different models with the remaining languages. Figure FIGREF11 summarizes this procedure.
Experiments and results
The experiments are divided as follows: First, the baseline and the CNN models are trained considering each language individually. Then, the trained CNNs for each language are used as a base model in the transfer learning strategy in order to improve the accuracy in the other two languages. All speech exercises performed by the participants were considered for the classification strategy. The final decision for each speaker was obtained by a majority voting strategy among the different speech exercises.
Experiments and results ::: Baseline and individual CNN models
Table TABREF13 shows the results obtained for the baseline and the CNNs trained for each language individually. Similar accuracies are obtained between the baseline and the CNN model for Spanish language, which also exhibit the highest accuracy among the three languages. Note that the highest accuracy for German language was obtained with the baseline model. Conversely, for Czech language the CNN produces the highest accuracy. Note also that for the three languages, the results are unbalanced towards one of the two classes according to the specificity and sensitivity values. The difference in the results obtained among the three languages can be explained considering the information provided in Table TABREF5. For the patients in the Spanish language, the average MDS-UPDRS-III score is higher compared with the German and Czech patients, i.e, there are patients with higher disease severity in the Spanish data compared to German and Czech patients.
Experiments and results ::: Transfer language among languages
The results with the transfer learning strategy among languages are shown in Table TABREF15. A CNN trained with utterances from the base language is fine-tuned with utterances from the target language. Note that the accuracy improved considerably when the target languages are German and Czech, with respect to the results observed for baseline and the CNN in Table TABREF13. The accuracy improved over 8% for German (from 69.3% in the baseline to 77.3% when the model is fine-tuned from Spanish), and over 4.1% for Czech language (from 68.5% with the initial CNN to 72.6% when the model is fine-tuned from Spanish). Particularly, the highest accuracy for German and Czech languages is obtained when the base language is Spanish. This can be explained considering that Spanish speakers have the best initial separability, thus, the other two languages benefit from the best initial model. The results obtained with the transfer learning strategy among languages are also more balanced in terms of the specificity and sensitivity than the observed in the baseline and with the initial CNNs. The standard deviation of the transfered CNNs is also lower, which leads to an improvement in the generalization of the models.
The receiver operating characteristic (ROC) curves from Figure FIGREF16 show with more detail the effect of the transfer learning strategy in the performance of the CNNs to classify PD speakers in different languages. The area under the ROC curve (AUC) when the target language is Spanish (Figure FIGREF16A) is slightly higher when the base language is Czech. When the target languages are German and Czech (Figure FIGREF16B and Figure FIGREF16C) the highest AUC is obtained when the base model is trained with Spanish utterances.
Conclusion
This study proposed the use of a transfer learning strategy based on fine-tuning to classify PD from speech in three different languages: Spanish, German, and Czech. The transfer learning among languages aimed to improve the accuracy when the models are initialized with utterances from a different language than the one used for the test set. Mel-scale spectrograms extracted from the transitions between voiced and unvoiced segments are used to train a CNN for each language. Then, the trained models are used to fine-tune a model to classify utterances in the remaining two languages.
The results indicate that the transfer learning among languages improved the accuracy of the models in up to 8% when a base model trained with Spanish utterances is used to fine-tune a model to classify PD German utterances. The results obtained after the transfer learning are also more balanced in terms of specificity-sensitivity and have a lower variance. In addition, the transfer learning among languages scheme was accurate to improve the accuracy in the target language only when the base model was robust enough. This was observed when the model trained with Spanish utterances was used to initialize the models for German and Czech languages.
Further experiments will include the development of more robust base models using hyper-parameter optimization strategies like those based on Bayesian optimization. In addition, the base models will be trained considering two of the languages instead of only one of them. The trained models will also be evaluated to classify the speech of PD patients in several stages of the disease based on the MDS-UPDRS-III score, or based on their dysarthria severity BIBREF20. Further experiments will also include transfer learning among diseases, for instance training a base model with utterances to classify PD, and use such a model to initialize another one to classify other neurological diseases such as Hungtinton's disease.
Acknowledgments
The work reported here was financed by CODI from University of Antioquia by grant Numbers 2017–15530 and PRG2018–23541. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Grant Agreement No. 766287. T. Arias-Vergara is also under grants of Convocatoria Doctorado Nacional-785 financed by COLCIENCIAS.

Question: Is dataset balanced in terms of available data per language?
Answer: Yes

No

Question: What datasets are used?
Answer: PC-GITA corpus, 88 PD patients and 88 HC speakers from Germany, 100 native Czech speakers (50 PD, 50 HC) 

the PC-GITA corpus BIBREF5, BIBREF17, BIBREF18