EEG Analysis Towards Evaluating Synthesized
Speech Quality
Ivan Halim Parmonangan1, Hiroki Tanaka1, Sakriani Sakti1,2, Shinnosuke Takamichi3,Satoshi Nakamura1,2
1Division of Information Science, Nara Institute of Science and Technology, Japan; e-mail: ivan.halim_parmonangan.ia4@is.naist.jp.
2Center of Advanced Intelligence Project, RIKEN, Japan

3Graduate School of Information Science and Technology, The University of Tokyo, Japan

Background

MethodUsing almost the same scenario, we tried to find out if the difference in speech quality can be seen at thebeginning of the stimuli in certain electrodes.
53 Japanese sentences areprepared. Each sentence
has original record and 4synthesized types.

10
Record EEG while listening the speech• Text to speech system generates voice from text input automatically.
10TTS

However, sometimes it sounds
unnatural.
Therefore, it needs evaluation.

Listen to the speech and give score 1~5• One approach is to use computer program and evaluate the synthesized speech acoustic
features objectively.
Despite of the speed, the relation
between acoustic features andhuman perception is yet to beunderstood[2].
• Another way to evaluate the quality subjectively is to collect Mean Opinion Score (MOS)
from subjects.

Figure of stimuli presentation during EEG recording
Fisher score of 𝑗th feature can be calculated as:
Where (𝜎 ) = ∑𝑛 (𝜎 ) .

Let 𝜎 and 𝜇 be the standard
deviation and mean of 𝑘 th class.

However, not only it is time
consuming, but also only provides
overall impression[1].

𝐹 𝑥=∑𝑛 (𝜇 − 𝜇 )(𝜎 )The recorded EEG thenpreprocessed and analyzed using
generalized Fisher score[4].

The collected opinion score from10 people then averaged and usedas the baseline.
In this research, we want to see
how human brain reacts towards
different synthesized speech
quality given that they alreadyknow how the original sounds like.
Let 𝜇 and 𝜎 be the mean andstandard deviation of the wholedataset corresponding to the 𝑗thfeature.

Results
EEG topology
used in thisstudy• Previously, a physiological approach was conducted using EEG to predict speech quality[3].
• This approach used full length of English speech and all electrodes combined.
However, all of theaforementioned
subjective evaluation
approaches use fulllength of the speech.

The Mean OpinionScore from 10subjects

Research Question
1. Are there any electrodes that show difference in activity when listening to different
types of synthesized speech quality?
2. How fast human brain can notice the difference between synthesized speech quality?
[1] R. Kubichek, “Mel-cepstral distance measure for objective speech quality
assessment,” in Proc. of IEEE PacRim,1993.
[2] C. Mayo, R.A. Clark, and S. King, “Listeners weighting of acoustic cues tosynthetic speech naturalness: A multidimensional scaling analysis,” SpeechCommunication, 2011.
[3] H. Maki, S. Sakti, H. Tanaka, and S. Nakamura, “Quality prediction ofsynthesized speech based on tensor structured EEG signals,” PLOS ONE, 2018.
[4] Q. Gu, Z. Li, and J. Han, “Generalized Fisher score for feature selection,” inProc. of UAI, USA, 2011.

Figures of Fisher score result between natural and each synthesized speech type in alpha band.

Conclusion

Discussion• Left parietal and occipital electrodes show small but different neuralactivities between 300ms and 700ms.
• However, the largest difference happens right after 1000ms until 3000ms.

• The result shows how fast human brain can distinguish the difference
between natural and each synthesized speech in short time.
• It needs further evaluation whether human can notice the quality
difference between all speech types with total random order.