Detection of dementia from responses to atypical questions asked byembodied conversational agents
112

34

2

Tsuyoki Ujiro , Hiroki Tanaka , Hiroyoshi Adachi , Hiroaki Kazui , Manabu Ikeda , Takashi Kudo , Satoshi Nakamura
1) Graduate School of Information Science, Nara Institute of Science and Technology, Japan
3) Department of Neuropsychiatry, Kochi Medical School, Japan
E-mail: ujiro.tsuyoki.uq5@is.naist.jp

2) Health and Counseling Center, Osaka University, Japan
4) Graduate school of Medicine, Osaka University, Japan

ResultIntroduction
Overview

1High detection performance

Aim
Early detection of dementia (Don’t need to go to hospital!)

Input (classification model)

AUC

Accuracy

Proposal

Gap only (SVM)

0.690.63

 During interaction with agents
 Tools that can be used repeatedly for daily use

MMSE (SVM)

0.85

0.83

Speech features (SVM)

0.85

0.83

Speech features (LR)

0.90

0.83

Speech and language features (LR)

0.92

0.92

Detection of dementia from responses to atypical questions

Suspected of dementia

Typical question:
No suspected of dementia

Atypical question:

Neuropsychological tests

Based on neuropsychological tests(e.g. MMSE; What’s the date today?)Not based on neuropsychological test

Related work

[Roark et al. 2011], [Aramaki et al. 2016], [Tanaka et al. 2017]

Some previous studies show possibilities of detecting
dementia from speech features and language features
Same interaction patterns and questions
These studies are not suitable for daily use

Frequently used screening for dementia(e.g. MMSE [Folstein et al. 1975], WMS-R [Wechsler 1997])

Experts are necessary for these tests
Need to go to hospital

MethodEmbodied conversational agents

GroupN

5 Verbs

Non-dementia

12 74.5 (4.3)

27.5 (1.8)

Dementia

12 75.9 (7.6)

21.2 (5.1)

This work!

The dementia group is slow to respond to questions than non-dementia group(two-tailed Mann-Whitney U test) 𝑝 = 0.04 (n = 24)(Effect size) 𝐶𝑜ℎ𝑒𝑛′ 𝑠 𝑑 = 0.98
“Gap” is the time difference from the end of the agent’s question to the startof the participant’s response

No difference between the two groups Q5, Q6, Q7
About the past story unlike other questions
For our participants, memory disorder possibly
has not been progressed

Question set

These questions are based on consultations
with neuropsychologists

0.92

Differences of ”Gap” at each questions

SystemAgent’sutterance

0.95

mean (SD) mean (SD)

Diagnosis is based on DSM-IV-TR

Content
Q1 Please tell me about your family.
Please tell me something that you feel isQ2
stressful in your life.
Q3 What is your hobby?
Q4 What is your favorite song?
Q5 Please tell me about Yujiro Ishihara.
Q6 Please tell me about Shigeo Nagashima.
Q7 Please tell me about Hibari Misora.
Q8 Who is Japan's Prime Minister?
Q9 What season is it now?
Q10 What year is it?
Q11 Are you left-handed or right-handed?
Q12 Do you sleep well?
Q13 How is your appetite?

Speech and language features (SVM)[Tanaka et al. 2017]

Analysis of “Gap”

TOP51 Gap

4 mean value of Pause
MMSE

0.83

Analysis of feature weight in the logistic regression

3 f0 max
Age

0.93

Previous work

Normalized the features (Mean: 0, SD: 1)
Model evaluation: Leave-one-participant-out, ROC Curve

2 f0 range

Detail of participants

Speech, language
and image features (SVM)

Frequently usedneuropsychological
tests

User’sutterance

Agent’sutterance

 Q11, Q12, Q13
Closed-ended questions are not effective
Easy to answer with short sentences

User’sutterance

Conclusion
TimeFeature extraction
Speech features
Language features
Answer time, Gap, Power, Speech rate, Tokens, Fillers,Pause, f0
POS (noun, verb, adjective, adverb)
 The agent randomly asks five questions from question set
 Analyze the user’s utterance

 Dementia can be detected even when using
speech features and language features withatypical questions(AUC: 0.95, Unweighted accuracy: 0.92)
 The result will change depending on questiontypes
 Possibility of measuring the degree ofdementia from memory disorder

Future work
 Analyze image featurese.g. facial expressions, eye movements
 Confirm the relationship between memory disorderand degree of dementia

References
H. Tanaka, H. Adachi, N. Ukita, M. Ikeda, H. Kazui, T. Kudo, and S. Nakamura, “Detecting dementia throughinteractive computer avatars,” IEEE journal of translational engineering in health and medicine, vol. 5, pp. 1–11, 2017.
E. Aramaki, S. Shikata, M. Miyabe, and A. Kinoshita, “Vocabulary size in speech may be an early indicator ofcognitive impairment,” PloS one, vol. 11, no. 5, p. e0155195, 2016.

B. Roark, M. Mitchell, J.-P. Hosom, K. Hollingshead, and J. Kaye, “Spoken language derived measures for detecting
mild cognitive impairment,” IEEE transactions on audio, speech, and language processing, vol. 19, no. 7, pp. 2081–2090, 2011.