Automatic Detection of Very Early Stage of Dementia
through Spoken Dialog with Computer Avatars
Hiroki Tanaka1 , Hiroyoshi Adachi2 , Norimichi Ukita1 , Takashi Kudo2 , and Satoshi Nakamura1
Abstract— This paper proposes a new approach to detecting
very early stage of dementia automatically. We develop a computer avatar with spoken dialog functionalities that produces
natural spoken queries. As initial analysis, audio visual dataof spoken dialogues from 18 participants are recorded, andfeatures of the number of token, type token ratio, the numberof fillers, speech rate, and pauses before new turn are extracted.
The results of the analyses showed that support vector machines(SVM) can classify two groups with 0.84 (> 0.70) detection
performance.

I. INTRODUCTION
Dementia is broadly defined as deterioration in memory,
thinking and behaviour that decreases a person’s ability tofunction independently [1]. The detection of dementia ischallenging, especially in its very early stages [2]. Previous
works have attempted to detect dementia from their speechand language attributes [3], [4]. However, most of themused non-interactive data such as picture description task andnarrative1 .
This paper proposes a new approach to detecting very
early stage of dementia automatically. We develop a computer avatar with spoken dialog functionalities that produces
natural spoken queries referring to Mini Mental State Examination (MMSE), Wechsler Memory Scale-Revised (WMS-R)
and other related questions.
II. S POKEN D IALOG WITH C OMPUTER AVATARS
We used MMDAgent2 as a computer avatar. The system
works on a regular laptop, which processes the audio input.
The system was adopted to elderly people by displayingsubtitles and slower speaking rate. We briefly describe thedialogue procedures as follows:• Self-introduction: The system introduces him/her-self.
• Gaze: The system displays a small dot on a computer
screen, and users are directed to gaze the moving dot.
• Reading: The system displays a document, and users
read aloud the sentence.
• Q&A: The system asks a total of 8 queries. The first
three queries are fixed (referring to the MMSE), and theother five queries are randomly produced.
*This work was supported by Yanmar Innovation Lab. 2112
1 H. Tanaka, N. Ukita, and S. Nakamura are with Graduate School ofInformation Science, Nara Institute of Science and Technology, Ikoma-shi,Nara, Japan {hiroki-tan,ukita,s-nakamura}@is.naist.jp
2 H. Adachi, and T. Kudo are with Department of Psychiatry, Osaka University Health Care Center, Toyonaka, Osaka, Japan{hadachi,kudo}@psy.med.osaka-u.ac.jp
1 https://talkbank.org/DementiaBank/
2 http://www.mmdagent.jp/

Fig. 1.
•ROC curve.

Retelling: The system read aloud a document of theWMS-R, and users retell the sentence.

III. I NITIAL A NALYSISWe recorded audiovisual data of spoken dialogues from 18
participants (9 very early stage of dementia and 9 healthycontrols). We extracted features of the number of tokens,
type token ratio, the number of fillers, speech rate, and pauses
before new turns from the answers of the three fixed queries.
Here, we denote values of pauses before new turns as time
between the end of the avatar’s question and the start of theuser’s answer.
As shown in Fig. 1, we confirmed that SVM with sigmoid
kernel classified two groups with 0.84 (> 0.70) detection
performance as measured by areas under ROC curve.
IV. C ONCLUSION
We proposed a new approach to detecting very early stageof dementia through spoken dialog with computer avatars. As
initial analysis, we found that our system has possibilities todetect very early stage of dementia.
R EFERENCES[1] G. M. McKhann, D. S. Knopman, H. Chertkow, B. T. Hyman, C. R.
Jack, C. H. Kawas, W. E. Klunk, W. J. Koroshetz, J. J. Manly,
R. Mayeux et al., “The diagnosis of dementia due to alzheimer’s disease: Recommendations from the national institute on aging-alzheimers association workgroups on diagnostic guidelines for alzheimer’sdisease,” Alzheimer’s & dementia, vol. 7, no. 3, pp. 263–269, 2011.
[2] V. Taler and N. A. Phillips, “Language performance in alzheimer’sdisease and mild cognitive impairment: a comparative review,” Journalof Clinical and Experimental Neuropsychology, vol. 30, no. 5, pp. 501–556, 2008.
[3] M. Lehr, E. T. Prud’hommeaux, I. Shafran, and B. Roark, “Fullyautomated neuropsychological assessment for detecting mild cognitiveimpairment.” in INTERSPEECH, 2012, pp. 1039–1042.
[4] K. C. Fraser, J. A. Meltzer, and F. Rudzicz, “Linguistic features
identify alzheimer s disease in narrative speech,” Journal of Alzheimer’s
Disease, vol. 49, no. 2, pp. 407–422, 2015.