Proceedings of iiWAS2020

iiWAS2020 Papers

Evaluating Video Playing Application for Elderly People at
Home by Facial Expression Sensing Service
Kosuke Hirayama1 , Sachio Saiki1 , Masahide Nakamura1,2
1 Graduate School of System Informatics, Kobe University, Japan
2 Riken AIP, Tokyo

hirayama@ws.cs.kobe-u.ac.jp,sachio@carp.kobe-u.ac.jp,masa-n@cs.kobe-u.ac.jp

ABSTRACT
We have been developing “Facial expression sensing service” foremotional analysis and quantitative evaluation of care based on
subtle facial movements and conducted a preliminary experiment
about its practicality. In this research, focusing both obtaining facial
expression data and searching for an efficient care method, to elderly
people can activate themselves and relieve their stress, we havedeveloped “Video player service” that can easily play videos andautomatically collect facial expression data. After developing theservice, we have asked people who engage in elderly care to try it
and obtained feedback. As a result, we received favorable comments
for the usefulness of the service, and we were able to get facial
expression data for four people.

CCS CONCEPTS• Human-centered computing → Visualization systems andtools; • Applied computing → Health care information systems.

KEYWORDS
Scientific long-term care, Reminiscence therapy, Facial expression
analysis, Care effect
ACM Reference Format:
Kosuke Hirayama1 , Sachio Saiki1 , Masahide Nakamura1,2 . 2020. EvaluatingVideo Playing Application for Elderly People at Home by Facial Expression Sensing Service. In The 22nd International Conference on Information
Integration and Web-based Applications & Services (iiWAS ’20), November 30December 2, 2020, Chiang Mai, Thailand. ACM, New York, NY, USA, 7 pages.
https://doi.org/10.1145/3428757.3429113

1

INTRODUCTION

Japan is facing a super-aged society. The number of elderly people
who need care is increasing, which leads to a chronic lack of careresources. Under this circumstance, the Japan Government has
declared the practice of “scientific long-term care” as a national
policy[3]. It aims at optimal use of care resources by corroborating
the effect of the care by scientific evidence. To achieve scientificlong-term care, it is essential to evaluate the effect of the careobjectively and quantitatively.
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
iiWAS ’20, November 30-December 2, 2020, Chiang Mai, Thailand© 2020 Association for Computing Machinery.
ACM ISBN 978-1-4503-8922-8/20/11. . . $15.00
https://doi.org/10.1145/3428757.3429113

We have been researching and developing “Facial expression
sensing service” [4]. This service focuses on facial expressions as an
objective indicator of the care effects, and to capture subtle changes
of facial expression. Also, we have attempted an experiment of
this service to obtain data of facial expression changes of elderly
people who are receiving care and discussed whether the service
can acquire useful data for estimating the emotions of the object
person. However, at present, our experiment is only preliminary
by using videos and data of the other past experiments.
To examine facial expression data in more detail, we have tocollect face images of the people who are receiving care. To obtainaccurate data, we have to collect stable face images. Then it isconsidered unsuitable to try to capture the face during the careof moving the body dynamically like bodily exercise. Besides, it isunclear whether care recipients behave as usual under the conditionof being pointed by a camera to take facial photos. However, hiding
the camera is a bad idea considering privacy issues. Therefore, wehave to let care recipients be immersed in the care and weaken
the existence of a camera. To collect facial data smoothly, we haveto consider a care method that can overcome such constraints
naturally.
In this study, to meet the above-mentioned conditions, to obtainfacial expression data efficiently, and to be enjoyable for care recipients, we have decided to let them view videos on a computer or atablet device. Then we have developed a dedicated “Video player
service”. With this service, the care recipient can easily play videos
set by the caregiver in advance. In addition, face images can be
automatically collected while the care recipient is watching videos.
In this research, we have developed Video player service. Also,
we have asked people who are involved in elderly care to try thecreated service and to make feedback on the usability and usefulnessof the service. As a result, we received favorable comments for theusefulness of the service. Also, we were able to get facial expression
data for four people.

2 PRELIMINARIES
2.1 The current situation around scientificlong-term care
As we described in Chapter 1, to evaluate the effect of care quantitatively and objectively is important for scientific long-term care.
However, the evaluation methods of care tend to depend on a subjective scale, like observations and questionnaires.
Therefore, it is difficult to use the obtained data as scientificevidence. Moreover, these assessment methods compel a heavy
burden on care evaluators, care recipients, or both.

21

Proceedings of iiWAS2020

iiWAS2020 Papers
Kosuke Hirayama1 , Sachio Saiki1 , Masahide Nakamura1,2

iiWAS ’20, November 30-December 2, 2020, Chiang Mai, Thailand

Besides, since scientific care is premised on using large-scaledata, efficient data collection is required. In the past, to measurecare effects objectively, an experiment that measured the changes
in the facial expressions of object people undergoing care havebeen conducted[10]. In this experiment, facial features such as theheight of eyebrows and the opening of eyes are measured fromimages based on the method called “facial expression analysis” of
P. Ekman et al[2] and so on. However, they measured the featuresfrom the recorded video manually. Also, in the report, they havebeen concluded that “...Currently, we have not generalized thisstudy successfully because of the limited number of cases. ... To
establish an objective evaluation method, We have to collect andanalyze both the data of the objective people and the caregiver
more.”
From the above, to realize practicing scientific long-term care, it
is important to develop a computer-assisted service that enables
us to automatically collect data that contributes to the quantitative
assessment of care.

2.2

Facial expression sensing service

We have been researching and developing “Facial expression sensing service” focusing “scientific long-term care”. This service focuses on facial expressions as an objective indicator of the careeffects, and to capture subtle changes of facial expression. This
service measures movements of facial parts as “feature values” toquantify the care effect.
One of the ways of estimating facial expressions from face images is to use a cognitive API constructed using machine learning.
However, the existing API is for general purpose. Therefore, forexample, when the facial expression changes of a care recipient
are weakened by functional and/or cognitive impairment, the API
may be impossible to capture the subtle changes. Also, the emotion
analysis of existing APIs is a black box. Therefore, caregivers cannot
trace why and how the results from API are output. As a result, we
may face difficulty when we are going to use the data from API as
the evidence for the effect of care.
Facial expression sensing service extracts the coordinates ofcharacteristic parts of the face (= feature points) such as eyebrows,
eyes, and mouth from the face image. Then, the service measuresthe length connecting two feature points as a feature value andrecords as time-series data. By tracking the change of these values,
we can measure the degree of facial expression change from facial
movement. As a result, even when the object person shows only
subtle facial expression change, the service will be able to measure
and record them without overlooking. We aim to contribute to
explicable emotion analysis and quantitative evaluation of care byexamining the degree of facial expression change as clear numericalvalues and inferring the emotional changes associated with the
movement of facial expression.

2.3

Reminiscence therapy“Reminiscence therapy” is one of non-drug therapy for dementia.
It aims at the object person’s psychological stability by offeringhim/her a time to take a look back one’s past experience and responding empathically and receptively. It is used to control the progression of dementia and prevent depression of elderly people[1].

Facial expression sensing service
Measurefeature value
API

Send
face image

Feature value
measurement
moduleMeasurement result,face image

Register info
APIs
Register
object person

Access todatabase

Read/Write
various information

Database for
Facial expression
sensing service
Measurement
results
Featurevalue
definitions
Face images

Registerfeature value
definition

Details ofobjectpersonGet

Get info measurement
APIs
resultsGet
feature value
definition

Get detailof objectpersonGet
face image

Figure 1: Conceptual diagram of Facial expression sensing
service
In some cases, old images and music which make an object person
feels nostalgic are used supplementarily so that the object person
can recall one’s memory easily[13]. There is also a report of reminiscence therapy with using YouTube (video sharing service)[8].

3SUMMARY AND IMPLEMENTATION OFSERVICES

In this chapter, we explain the implementation of Facial expression sensing service stated in Chapter 2.2 and the summary andimplementation of Video player service we mentioned in Chapter 1.
We have implemented each service as a web service so that users
can use them from a web browser.

3.1

Summary of Facial expression sensing
service

Facial expression sensing service provides various functions as APIs
so that external applications can access them via the Internet. The
service has a dedicated external database to store the information
about object people and the measurement results of the feature
values, and so on. Figure 1 shows a conceptual diagram of theservice.
To measure facial feature values, users must specify the combination of feature points and define the feature values in advance. This
service has various APIs to register various kinds of information,and by sending a request to them, users can register and store thedata in the database. For the measurement, the service receives face
images from measure feature value API and calculates the feature
values based on the created definitions. The measurement data
is recorded in the database with face images. Users can retrieve
various data stored in the database by using the get information
APIs.
Object people information registered in Facial expression sensing
service is used to specify the person when measuring feature values.
In addition, this object people information will also be used byexternal services that provide care with this service to specify the
person who receives the care.

22

Proceedings of iiWAS2020

iiWAS2020 Papers
Evaluating Video Playing Application for Elderly People at Home by Facial Expression Sensing Service

Measurefeature value
API

Facialexpression
sensing
serviceiiWAS ’20, November 30-December 2, 2020, Chiang Mai, Thailand

DB for facial
expression
sensing service

Get info APIs
Send face
imagesMovie play+Get face image

Movie player service

Details ofobject person

Send images

Management
screenManagemovies

Display
movie player

Manageplaylists

Movie playscreenCare giver

Watch

Read from/Write to database

Care recipientMoviesPlaylists

DB formovie player service

Figure 3: Video player service, playlist create screen

Figure 2: Conceptual diagram of Video player service
This service itself is an API and has no user interface. Therefore,
we have created a separate Web application for managing the API,
which wraps functions of the service and can be used from a browser
with a graphical interface.

3.2

Summary of Video player service

The purpose of Video player service is to relieve object people’sstress and make them feel better through watching videos. Developing this service, we referred to the concept of reminiscence therapy
described in Chapter 2.3. Figure 2 shows the conceptual diagram ofVideo player service. The usage of this service is divided into the
caregiver’s side and the care recipient’s side.
First, caregivers have to register videos which they want toshow care recipients. When selecting videos, it is assumed thatcaregivers will refer to the preferences of the target care recipient.
The preferences are, for example, investigated by listening directlyor using a questionnaire to the care recipient or his/her family.
This service uses the video-sharing service “YouTube” as the video
source. After registering videos, caregivers put together multiple
videos and make a playlist, and link to the care recipient who
is taken care of them. This service uses the information of carerecipients registered in Facial expression sensing service. Whenmaking playlists process is completed, a URL link to play videos
is created for each care recipient. Caregivers have to present it totheir care recipients so that they can access it.
The care recipient accesses the video player page from the linkpresented by the caregiver. Then, the video play start screen isdisplayed, and the list of playlists prepared by the caregiver isshowed. Selecting one of them, the service will start playing theplaylist. With playing videos, the service gets face images of the carerecipient who is watching videos at fixed time intervals, and sends
them to Facial expression sensing service, and measures the feature
values. Besides, we have prepared no-sensing mode in which theservice only plays videos and does not take face images.

Figure 4: Video player service, movie play start screen

Figure 5: Video player service, movie player screen
As mentioned above, since we have implemented this service
as a Web service, users can use this service anywhere if a Web
browser and a Web camera (when measuring feature values) are23

Proceedings of iiWAS2020

iiWAS2020 Papers
Kosuke Hirayama1 , Sachio Saiki1 , Masahide Nakamura1,2

iiWAS ’20, November 30-December 2, 2020, Chiang Mai, Thailand

3
1

Table 1: Detail of the qestionnaire

4
10

2

5

(1)
6

7

8

(2)
(3)
(4)
(5)
(6)

1+ 2
= ─ : Eyebrows height2
3+ 4
2 = ─ : Eyebrows lifting
2
5+ 6
3 = ─ : Eyes opening
2
7+ 8
4 = ─ : Mouth corner lowering2
9 : Mouth width
10 : Eyebrows distanceQ1 Please write your name.

1

Q2 Do you think you could operate the service without confusion?
Q3 Do you think you could play videos without any problems?
Q4 Do you think you could make playlists smoothly?
Q5

Q6 Do you think this service will be useful for elderly care?
Q7 Please write the reason why you thought so in Q6.
Q8 Do you think any caregiver can handle this service?

9

Figure 6: Used feature values for the experiment

Do you think the system's usability (feeling to use,simplicity of screen,etc.) was good?

Q9

Do you think any care recipient can use this service without any
problems?

Q10 Did you encounter troubles when using the service?
Q11 (If you answered “Yes” to Q10) Please report the detail of the troubles.
Q12 Do you think the manual we sent was easy to understand?

available. Therefore, care recipients can access the service and play
the playlist created by the caregiver even at home, and we canacquire the feature values.

3.3

Q13

If you have any other feedback, requests, suggestions for improvement,comments, etc., please write.

Table 2: Result of the questionnaire

implementation

We have used the following technologies to implement the API of
Facial expression sensing service.

Q2
Q3
Q4
Q5
Q6
Q8
Q9
Q12

• Development languages: Python3• Libraries and tools: Django, OpenCV, dlib[6], Celery, Redis,
PostgreSQL• External database: MySQL

We have used the following technologies to implement the management application of Facial expression sensing service and Video
player service.
• Development languages: Java 1.8, HTML, JavaScript• Libraries and tools: Apache Tomcat, Gradle, Spring Boot,
jQuery, Bootstrap, Plotly, DataTables• External database: MySQL

Figure 3 shows the playlist create screen, Figure 4 shows themovie play start screen, and Figure 5 shows the movie player screen.
We designed the screens used by care recipients (Fig. 4, Fig. 5) withan easy-to-understand user interface in consideration of the casewhere an elderly person who is not very familiar with digital devices.
On the movie player screen, embedded player of YouTube1 is used
to play videos defined in the playlist. A care recipient can change
a playing video by clicking the bottom “Go to the previous video”
and “Go to the next video” buttons.

4 EVALUATION EXPERIMENT
4.1 Purpose of the experiment
By asking other people to use implemented Video player service andcollecting feedback, we have validated the usefulness of the service
and checked an area for improvement. In addition, we have collectedfacial expression data (=feature values) during the experiment andbriefly examined them.
1 https://developers.google.com/youtube/iframe_api_reference

Agree

Somewhatagree

SomewhatdisagreeDisagree

2
2
1
-

1
1
1
1
1
2

1
1
-

1
1
-

Yes

No

1

2

Q10 (whether encountered troubles)

4.2

Summary of experiment

The experiment was conducted with the cooperation of three people, and they used Video player service. Of the participants, two are
medical personnel engaged in nursing care for the elderly, and one
is a researcher. We asked each participant to perform a sequenceof operations of the service, creating playlists, and playing movies
themselves or showing movies to other people, referring to themanual sent in advance. Besides, take face images of the person
watching the video, and measure feature values every about 3 seconds using Facial expression sensing service. We have measured 6
feature values, which are shown in Figure 6. In this experiment, it
was difficult to acquire data from the actual care recipient due toprivacy restrictions. Then we asked participants to show videos toa person close to them.
After having used the service for a certain period, we asked theparticipants to answer the questionnaire shown in Table 1 to get
feedback. Q2 to Q6, Q8, Q9, and Q12 were evaluated in 4 levels:“Disagree,” “Somewhat disagree,” “Somewhat agree,” and “Agree.”
We use SQuaRE[5], an international standard for software quality,
as a reference to create this questionnaire.

24

Proceedings of iiWAS2020

iiWAS2020 Papers
Evaluating Video Playing Application for Elderly People at Home by Facial Expression Sensing Service

After collecting the feature value data of the person watching
videos, we examined them briefly.
The experiment has been approved by the Research Ethics Committee of Kobe University Graduate School of System Informatics(No. R01-02).

caregiver side. Taking this result, We realized the necessity to brush
up the service operability on the premise that people with various
PC skills will operate this service not only on the care recipientside but also on the caregiver side.
We have got another feedback that “I think it would be good toprepare some examples that will make object people happy by age
group, gender, occupation, and so on.” From this, we recognized that
it is important for the experiment efficiency to prepare documents
to assist to select videos, looking ahead to expand the experimentin the future. Of course, it would be difficult to present contents
that are acceptable to everyone, because each person has different favorite things. However, We still would like to indicate someguides and make selecting movies smoothly. Regarding this, we
received various opinions from the participants of this experiment.
A participant who had the experience to show videos to elderly
people before this experiment provided the following findings.

5 EXPERIMENT RESULTS
5.1 Result of questionnaire
Table 2 shows the results of the questionnaire.
In Q2 to Q5 which are questions about usability, there weremany “Agree” and “Somewhat agree” answers, but also “Somewhat
disagree” in Q2 and Q5, and “Disagree” in Q4. Regarding this, we
got the feedback that “caregiver’s operations such as registering
videos are complicated, and I felt it is difficult for people who are
not familiar with PC to complete these tasks.” The usefulness of theservice in Q6 was generally well-received, and the following opinions were obtained in the following Q7 (excerpt from the original
text, partly modified).

• I thought it would be good to be able to provide videos according to the interests of the person so that seasonal eventsand one’s hobbies could be reflected. When I showed movies
to my mother, she enjoyed watching videos of scenery,dance, etc.
• By using this service, it could be possible to show the carerecipient’s favorite video at any time.
• I think that providing video contents according to the interests of the elderly is a new perspective in elderly care.
Since it is possible to ensure individuality and create a video
program according to the purpose, I think it is effective forreminiscence methods, recreation programs, and also reducing the care burden of the family of dementia patients.













• The news, TV commercials, popular songs, etc. of the period
when the object person was young or made a home seems
effective when watching the video with him/her.
• Monotonuse images of landscapes can’t draw much attention.
• I have tried to show elderly comic chats or song talks. However, it seems that videos contain fast-paced conversation
makes them confused.
• Magic trice gave a good impression surprisingly. Many
people watched magic videos earnestly.



In addition, I got the following comments from the other participant.
If the object person is who have a mild to moderate degree of
dementia, an approach of reminiscence therapy is effective.
But when it comes to dealing with those who have severe
dementia, this approach often becomes invalid because they
forget their old memories. In that case, I think it would begood to use simple images, like baby and animal, moving
landscape from car windows, or sparkling images such as
fireworks.

Q8 and Q9 are questions from the viewpoint of whether anyone canuse this service. There is no “Agree”, while “Somewhat disagree” or“Disagree” were noticeable. From Q12, we proved that the manual
obtained some evaluation. In Q10, we asked reports on troubles
during use, and got a report from one person, which is “I cannot
confirm whether the camera function is working well from a remotelocation.”

5.2

iiWAS ’20, November 30-December 2, 2020, Chiang Mai, Thailand

Discussion about the result ofquestionnaire

Through the questionnaire, we have received good feedbacks that
Video player service will be useful. In particular, they highly evaluated regarding the service can create playlists according to individual interests and care purposes. What contributed to this evaluation
might be the advantage of using a video sharing service with hugevideo resources for care. Also, since Q3 was “Agree” with all 3 participants, we have found that the service had worked without causing
serious trouble in the overall operations from creating playlists toplay videos.
On the other hand, there were some complaints about the operability of service. In particular, most of them were comments
and feedback regarding the complexity of the user interface on the

To improve the quality of the service entirely, we would like tocreate a guide to select contents using such knowledge as references,in parallel with the improvement of service implementation.

5.3

Obtained facial data and discussion

In this experiment, we measured the feature values of five people,
which are three participants and, with the cooperation of the father
and mother of a participant, additional two people. We obtained
four person’s data successfully through measuring five people. The
service has failed to get face images and measure the feature values
for remaining one person. Figure 7 shows a part of the obtainedfeatures of one of the object people. The horizontal axis shows theelapsed time from the start of playing video, and the vertical axis
shows the feature value. The black dotted line is the average value.
Figure 8 is an image showing the feature points that were extracted
from the face image and were used to measure the feature values.
Here feature value means the ratio of length based on the distance25

Proceedings of iiWAS2020

iiWAS2020 Papers
Kosuke Hirayama1 , Sachio Saiki1 , Masahide Nakamura1,2

iiWAS ’20, November 30-December 2, 2020, Chiang Mai, Thailand

Mouth width

1.10

1st movie

2nd movie

3rd movie

1.00
0.90
0.80
0.70
0.60
0:00
1.40
1.30
1.20
1.10
1.00
0.90
0.80
0:00

2:00

4:00

6:00

8:00

10:00

12:00

14:00

Mouth corner lowering
1st movie

2:00

4:00

2nd movie

6:00

8:00

10:00

3rd movie

12:00

14:00

Eyes opening
1st movie

2nd movie

3rd movie

0.15
0.10
0.05
0:00

In this experiment, she watched three videos, and the contents
were Manzai (Japanese stand-up comedy), dance, and scenery inorder from the first. Among them, in the feature values in Fig. 7, theabove-mentioned signs of happiness were mostly seen in the latter
half of the first video. Therefore, we can analyze that she might
like to watch Manzai videos.
As above, by examining the obtained feature values in detail,we may analyze care recipients’ emotions based on numerical datarather than vague and subjective observations. In addition, by examining the obtained feature values in detail, it might be possible
to guess what content a care recipient likes.
As in this experiment, when users use the service in their home,
the quality of obtained face images depends on the environment
around them (room lighting and brightness, camera performance,etc.). This time there were no problems caused by room environment, but we have realized that it is necessary to present somerecommended environment because the environment for taking
images could affect the reliability of feature values measurement.
Also, in this experiment, we have faced a case of failed to get face
images and the feature values. In consideration of the reported
trouble mentioned in Q10 of 5.1, when the service does not worknormally, we need to indicate clearly to the user about the trouble.

6
2:00

4:00

6:00

8:00

10:00

12:00

14:00

Figure 7: A part of the obtained features

Figure 8: A part of feature points extraction result
connecting the centers of both eyes. When making a graph, we
excluded obvious outliers that came from failures of face detection.
Changes in “Mouth width” and “Mouth corner lowering” seemto be relatively good hints to consider whether an object person enjoys watching a video. When we checked face images when “Mouth
width” was high and mouth corners were raised (= when “Mouth
corner lowering” was low), we could see smiling faces. By the way,even in reference [2] regarding the facial expression analysis thatwas stated in Chapter 2.1, the characteristic expression of happiness is defined as “edges of the lip are pulled backward and slightly
raised” and “cheeks are lifted”. In addition, “Eyes opening” could be
evidence to judge whether they feel fun since the cheeks should risewhen laughing, then a bottom eyelid should be raised accordingly,
and as a result, the opening of eyes should be narrowed. Furthermore, it might be possible to consider whether they concentrate on
videos when “Eyes opening” stays low for a long time. However, if
it is too low, the video might be boring and they feel sleepy.

RELATED WORK

Various techniques have ever been devised for computer technology
intervention in elderly care or dementia care. For example, there isa research that shows images such as paintings and photographs
to patients with dementia using a tablet device and measures theeffect[12]. To offer some stimulation to patients, the images wereselected not likely to trigger for reminiscing about a specific time
period or event. They have investigated the care effects by providing
questionnaires and interviews with people with dementia and theircaregivers. As part of that, the art-viewing app used in care asked
about the user’s mood and collected data before and after the care.
As a research to evaluate care effects from facial expressions, wecite “Face Emotion Tracker” which is one of the previous studiesof our research group[9]. In this research, Microsoft Face API is
used to analyze emotions while a person with dementia was receiving care. The researchers adopted a computer-based method
as a care method, and people with dementia talked with a virtual
agent[11]. As a result, various facial expression signs were able
to be captured. However, for people with weakened facial expression changes caused by weakening facial muscles or euphoria, theemotion of the person became hard to analyze.
Besides, as a study to evaluate care quality, there is a researchthat focuses on how caregivers give care[7]. Caregivers performedcare with wearing a head-mounted camera to their head and the
distance and angle of the caregiver’s face toward the cared person
are measured from captured images. Then, based on the differencein the data between experts and amateurs, they extract care posturesthat are peculiar to experts. The aim of this is to enable caregivers
to know their care skill from feedback and to improve their qualityof care. From the viewpoint of care evaluation, this research havein common with this research and the related research mentioned
above, but the focus is not on creating a new care methods but
rather on improving the quality of existing care methods.

26

Proceedings of iiWAS2020

iiWAS2020 Papers
Evaluating Video Playing Application for Elderly People at Home by Facial Expression Sensing Service

7

CONCLUSION

In this paper, we have developed Video player service that canefficiently obtain facial expression data and provide care that carerecipients can enjoy, and conduct an evaluation experiment withpeople who engaged in care for the elderly people.
As a result of the experiment, we have been able to obtain a goodevaluation regarding the usefulness of the service. In addition, we
have been confirmed that facial data could be acquired and briefly
examined.
As future works, we would like to improve the usability of theservice and add functions with a view to expanding the scale ofexperiments. In the future, we would like to experiment with elderly people who are actual care recipients and analyzing facial
expression data.

ACKNOWLEDGMENTS
This research was partially supported by JSPS KAKENHI Grant
Numbers JP19H01138, JP18H03242, JP18H03342, JP19H04154,
JP19K02973, JP20K11059, JP20H04014, JP20H05706 and Tateishi
Science and Technology Foundation (C) (No.2207004).

REFERENCES[1] Elder Care Alliance. [n.d.]. Benefits of Reminiscence Therapy. https://eldercarealliance.org/blog/benefits-reminiscence-therapy/. accessed August
15, 2020.
[2] P. Ekman and W. V Friesen. 1975. Unmasking the Face. Malor Books.
[3] Headquarters for Japan’s Economic Revitalization (No.26). [n.d.]. Future investment strategy (2017). http://www.kantei.go.jp/jp/singi/keizaisaisei/dai26/siryou.
pdf.

iiWAS ’20, November 30-December 2, 2020, Chiang Mai, Thailand[4] Kosuke Hirayama, Sachio Saiki, Masahide Nakamura, and Kiyoshi Yasuda. 2020.
Capturing User-defined Facial Features for Scientific Evidence of Elderly Care. In
The 2020 International Workshop on Pervasive Information Flow (PerFlow’20), Heldin conjunction with the 18th Annual IEEE International Conference on Pervasive
Computing and Communications (PerCom 2020). 35–40.
[5] International Organization for Standardization. 2011. Systems and software
Quality Requirements and Evaluation (SQuaRE). https://www.iso.org/standard/
35733.html/.
[6] Davis E. King. 2009. Dlib-ml: A Machine Learning Toolkit. Journal of Machine
Learning Research 10 (2009), 1755–1758.
[7] A. Nakazawa and M. Honda. 2019. First-Person Camera System to Evaluate Tender Dementia-Care Skill. In 2019 IEEE/CVF International Conference on Computer
Vision Workshop (ICCVW). 4435–4442.
[8] Julia O’Rourke, Fiona Tobin, Susan O’Callaghan, Rebecca Sowman, and DR
Collins. 2011. ‘YouTube’: a useful tool for reminiscence therapy in dementia?
Age and Ageing 40, 6 (08 2011), 742–744. https://doi.org/10.1093/ageing/afr100
arXiv:https://academic.oup.com/ageing/article-pdf/40/6/742/98976/afr100.pdf[9] Arashi Sako, Sachio Saiki, Masahide Nakamura, and Kiyoshi Yasuda. 2018. Developing Face Emotion Tracker for Quantitative Evaluation of Care Effects. In
Digital Human Modeling 2018 (DHM 2018), Held as Part of HCI International 2018,
Vol. LNCS 10917. 513–526. Las Vegas, USA.
[10] Kazunori Sato, Kazuyuki Sugino, and Takashi Hayashi. 2010. Development ofan objective evaluation method for the nursing-care education based on facial
expression analysis. Medical and health science research : Bulletin of Tsukuba
International University 1 (2010), 163–170.
[11] Seiki Tokunaga, Kazunari Tamamizu, Sachio Saiki, Masahide Nakamura, andKiyoshi Yasuda. 2016. VirtualCareGiver: Personalized Smart Elderly Care. International Journal of Software Innovation (IJSI) 5, 1 (Oct 2016), 30–43. http://www.igiglobal.com/journals/abstract-announcement/158780.
[12] Charles Tyack, Paul M. Camic, Michael James Heron, and Sabina Hulbert.
2017. Viewing Art on a Tablet Computer: A Well-Being Intervention forPeople With Dementia and Their Caregivers. Journal of Applied Gerontology 36, 7 (2017), 864–894.
https://doi.org/10.1177/0733464815617287
arXiv:https://doi.org/10.1177/0733464815617287 PMID: 26675353.
[13] Bob Woods, Laura O’Philbin, Emma M Farrell, Aimee E Spector, and Martin
Orrell. 2018. Reminiscence therapy for dementia. The Cochrane database of
systematic reviews 3, 3 (2018).

27