Integration Analysis of Heterogeneous
Data on Mind Externalization of Elderly
People at Home
Sinan Chen1(B) , Hayato Ozono1 , and Masahide Nakamura1,2
1

Graduate School of System Informatics, Kobe University,
1-1 Rokkodai-cho, Nada, Kobe 657-8501, Japan{chensinan,ozono}@ws.cs.kobe-u.ac.jp,
masa-n@cs.kobe-u.ac.jp
2
RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku,
Tokyo 103-0027, Japan

Abstract. As the aging population around the world, ﬁguring out thereason for changes in the health status of elderly adults at home ispressing. In our research group, to comprehend the scientiﬁc self-careof elderly adults at home, a core concept called “mind externalization”
that retrieves the elders’ thoughts as much as possible using spoken dialogue agent technology has developed swiftly. The purpose of this paper
is to consider an approach to elucidating reasons for health changes inthe elderly at home. Our key idea is to merge the results of health status, dialogue logs, and emotional values (recognized from images andaudio during the spoken dialogue) into a time series. More speciﬁcally,
we describe an approach for extracting the features of changes in healthdata (i.e., heart rate, stress, sleep quality, step, and activity level fromthe wearable device). It intends to add health data retrieved from a wearable device and unite heterogeneous data (i.e., health data and dialoguedata). Based on the integration between health data and dialogue data
(i.e., text logs, audio, and images), we discuss an approach to estimating
the reasoning context before and after the period. In this way, assisting elderly adults at home by grasping their daily living in detail canbe appreciated. Meanwhile, executing personalized self-management is
promising.
Keywords: Elderly at home · Mind externalization · Spoken
dialogue · Wearable device · Smart healthcare

1

Introduction

Population aging is currently a big issue in Japan, Germany, and Italy. Speciﬁcally, the number of people aged 65 years has reached 28.4% in Japan. It exceeds
21% of the national total, and the country is confronting a super-aging society1 . As the population and disease structure change, the demand for nursing
1

https://www8.cao.go.jp/kourei/english/annualreport/2020/pdf/2020.pdf.

c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022

Q. Gao and J. Zhou (Eds.): HCII 2022, LNCS 13331, pp. 197–209, 2022.
https://doi.org/10.1007/978-3-031-05654-3_13

198

S. Chen et al.

care is expanding year by year. In particular, many chronic diseases (e.g., mild
cognitive impairment, disease syndrome) require long-term care. However, theshortage of facilities and humans for medical welfare and nursing care is becoming more severe. As the trend toward conversion from institutional care tohome care continues, the number of elderly adults (requiring nursing care) athome is expanding more swiftly. Furthermore, the number of individual cases isalso expanding, such as one elderly adult caring for another, and elder adults
living alone.
Encouraging self-care among elderly adults within diﬀerences between
households is a big challenge. For example, the degree of completion of activities
of daily living (ADL), the degree of achieving a regular rhythm of life, and thedegree of physical and mental health are important indicators. In recent years,
research on assistive technology applying engineering technology to support the
lives of the elderly has been spreading around the world. A representative technology is to monitor elderly adults at home. It mainly utilizes the internet ofthings (IoT) and information and communication technology (ICT). A machinemonitors the situation (called context) of the elderly adult instead of the family
caregiver and communicates with the family caregiver only when required. In
this way, enhancing the quality of life (QoL) of the elderly and decreasing theburden on their families is promising.
Our research group is studying various systems and services from two perspectives: the “environment” and “humans” in houses. Related research fromthe perspective of the environment includes environmental sensing [20], behavior
estimation [21], and context recognition [5]. On the other hand, related researchfrom the perspective of the elderly includes facial expression recognition [10],health monitoring [19], and mental externalization through a virtual agent (VA)
listening service [14]. However, in the VA listening service conducted in previous
research, various kinds of heterogeneous data (e.g., text, image, and voice data
in a time series) that acquired and accumulated remain. We have not yet beenable to conduct an integrated analysis of the health status (e.g., heartrate, stress) and the “mental state” of the elderly.
The purpose of this paper is to consider a method to elucidate the reasons forcausing changes in the health status of elderly adults at home. Figure 2 shows
an image diagram of integration analysis of health data and dialogue data forelderly people at home. As an approach, we add health data retrieved from awearable health device (i.e., an activity meter worn on the arm) and consider
an integrated analysis of heterogeneous data (health data and dialogue data) in
a time series. Our key idea is to merge the results of health status, dialogue logs,
and emotional values in a time series. In our approach, we consider the following
ﬁve concrete steps:– Step 1: VA Listening Service and Wearable Health Installation– Step 2: Acquisition and Storage of Heterogeneous Data
– Step 3: Extraction of Health Feature Changes in Time Series– Step 4: Analysis of Interrelationships Among Changing Features– Step 5: Context Estimation Based on Past Feature Analysis

Integration Analysis of Heterogeneous Data
Value

199

Reasoning in Dialogue Data

t1

Health Data

Wearable Device

Time

Analysis

An Elderly Person

t1

Time

Dialogue Data

Dialogue Agent

Fig. 1. Integration analysis of health data and dialogue data for elderly people at home.

In the discussion, the advantage of the research approach is that it provides
a more multifaceted analysis of the elderly at home by utilizing heterogeneous
data. The abundance of available data has likewise improved, allowing for amore granular understanding of the daily living conditions of the elderly athome. As for a limitation of the research approach, we have not yet resolved the
problem of dealing with missing data in time series. Furthermore, we anticipate
the interrelationships among diﬀerent types of data. They have not evaluatedfor individual elderly adults. In particular, the correctness of the acquired datarequires investigation. It also requires checking the eﬀects of these data in casestudies.

2

Previous Study: Monitoring Assistance for Elderly
People at Home

This section depicts the background surrounding the current situation of theelderly at home. It presents previous studies aimed at understanding the mindof elderly adults.
2.1

Mental Externalization of the Elderly at Home

The “externalization of the mind” of the elderly refers to the act of blurtingout to the outside world as much as possible of what is on the “mind” of theelderly. In recent years, with the aging of the world’s population, the shortageof nursing care facilities and human resources [16] has become a social
problem. In Japan, which is confronting a super-aging society, the number of
elderly people at home is expanding every year as the structure of the population

200

S. Chen et al.

and diseases alter in line with the shift from conventional institutional care
to home care2 . The number of elderly people living at home is expanding year
by year. While caring for the elderly in their own homes is habituated, it places
a heavy burden on family members, and the problems of “elderly couple careeach other,” “people with dementia care each other,” and elderly people living
alone at home are becoming more severe.
The implementation of independent living for the elderly at home is not aneasy task. The decline of physical and psychological functions, nursing care, andlong-term medical treatment have limitations on self-care, and assistive technologies are required. One of the typical technologies is to monitor elderly people
at home. In the previous study, we studied the two aspects of “physical” and“psychological” monitoring. For monitoring the physical aspects, we estimateddaily life behaviors [21], recognized contexts [5], and measured and analyzed
physical activities [6]. On the other hand, for observing mental aspects, there
are virtual agent listening services, microservice execution by voice dialogue, andmultimodal diary services [4]. In this paper, we focus mostly on psychologicalmonitoring and support technologies.
2.2

Virtual Agent Listening Service

Virtual Agent (VA) technology refers to a technology that is equipped with asoftware (i.e., virtual) interface. It acts as an intermediary to promote smoothinteractions between machines and humans. In our previous study, we proposed
a VA listening service with voice interaction function to listen to the “feelings”
of elderly people in their daily lives. Figure 2 shows the components of the VA
listening service in the edge and cloud environments.
In the VA listening service, the components are described in detail from two
aspects: hardware and software. The hardware component consists of IoT sensors (i.e., Phidgets 3 motion and pressure sensor modules) and a general-purpose
computer (i.e., including built-in speakers, cameras, and microphones). On theother hand, we explain the details of the software components from two perspectives: local and cloud. First, the local component consists of a sensor driver, a
VA (i.e., MMDAgent [13]: including speech synthesis and character materials), a
local server (i.e., Apache Tomcat [15]), and a Web browser (i.e., including speech
recognition by Web Speech API [1] and display of dialogue contents by Web userinterface). Next, the cloud-side component consists of a cloud server (i.e., WebSocket and Pub/Sub [12]) and a database (i.e., dialogue scenarios (including
gestures, time periods, and questions) and dialogue logs).
By linking the IoT sensor with a general-purpose computer, it is possible todrive the VA Listening Service in two modes: “passive mode” and “active mode”.
In the “passive mode,” the VA listening service is automatically activated when
an elderly person at home approaches the motion sensor. Based on the userdeﬁned time of day and the questions to be asked, the VA listening service can2

https://www.mhlw.go.jp/english/policy/care-welfare/care-welfare-elderly/dl/
establish e.pdf.
3
https://www.phidgets.com/?tier=0&catid=3&pcid=8.

Integration Analysis of Heterogeneous Data

201

Cloud
Home
Server

Database

Display
SpokenDialogue

Speaker

Camera Mic

Elderly Person

Motion Pressure
IoT Sensors

Keyboard
General Computer

Fig. 2. Component structure of virtual agent listening service

ask about the person’s mood, physical condition, eating and drinking status, etc.
In the “active mode,” the VA listening service is automatically activated whenthe elderly person at home presses the pressure sensor. When the elderly person
tells the VA any tale by voice, the VA can give a random response (e.g., “Is thatso?”, “Yes”, etc.) by voice. Through interaction between the VA listening service
and the elderly person at home, thoughts and concepts in daily life, as well as
answers to speciﬁc questions, can be extracted, recorded, and stored externally(i.e., externalization of the elderly adult at home).
2.3

Microservice Execution Through Voice Interaction

The use of various microservices, such as contents and services on the Web, as
well as the externalization of the mind through voice dialogue between elderly
people at home and VA is also promising. In our previous study, we aimed to help
elderly people who are not accustomed to operating smart devices (e.g., computers, smartphones, etc.) to smoothly use Web services and information based on
the spoken dialogue function described in Sect. 2.2. The key idea is to apply theapplication programming interface (API) to microservices in Web services (e.g.,calendar, video watching, to-do management, etc.) and incorporate the appropriate content into the VA dialogue scenario. The approach includes two parts:(1) User information and microservice management. (2) Conﬁguring VA behavior at API runtime. To demonstrate the eﬀectiveness of the proposed framework,
we linked the “ToDo management service” and the “YouTube watching service”

202

S. Chen et al.

with user information. This allows the elderly to control the execution and stopping of microservices through voice interaction.
2.4

Multi-modal Diary Service

Although VA listening services have made it possible to record and accumulatedialogue data, the diversity of dialogue data and the lack of analysis and reuse
still pose challenges. In a previous study, we proposed a multimodal diary service
that records and analyzes dialogue data from a variety of perspectives. The key
idea is to retrieve not only text logs, but also audio and images during the dialogue as dialogue data, and to perform multimodal visualization. In the proposed
method, we executed heterogeneous data extraction, Web API development,
multimodal diary generation, and heterogeneous data analysis. To extend theheterogeneous data, we also considered text-based negative judgment, speechbased emotion detection, and image-based emotion estimation approaches. Using
the proposed method, the elderly can better recall the past for themselves. Hence,a more precise improvement in self-care and health management of individual
elderly adults is promising.

3

Development of Advancing Technology: Wearable
HealthThis section describes the challenges and measurement items for wearable healthdevices for the elderly at home, based on smart healthcare technology.
3.1

Smart Healthcare of Elderly People at HomeThe use of information and communication technology (ICT), internet of things(IoT), and artiﬁcial intelligence (AI) technologies to support the traditional
healthcare of the elderly at home (e.g., visiting hospitals, managing health,receiving care from family members, etc.) has been accelerating. The introduction of smart sensors (i.e., environmental sensors, opening/closing sensors, etc.),
smart speakers, and 360-degree cameras to make the homes of the elderly into an
environment similar to that of a facility has enabled safety conﬁrmation [11], status conﬁrmation [17], and automatic operation of home appliances [2]. However,these smart devices and approaches are only capable of sensing external environmental conditions, including the externalization of the elderly as describedin Secti. 2, and are not capable of monitoring the health of individual elderly
people, including changes in heart rate and stress, distance traveled, and numberof steps taken.

Integration Analysis of Heterogeneous Data

3.2

203

Emergence of Wearable Health Devices

Wearable health devices are emerging to monitor the health of the elderly athome. Smart apparel (i.e., clothing-type devices) include Xenoma4 and Hamon5 .
On the other hand, arm-worn smart watches (i.e., activity meters) include
Garmin6 and Fitbit7 . Through the use of these devices, smart healthcare technology can be deployed in various areas such as ﬁtness, sleep, and nursing care.
By collaborating with local facilities and hospitals, it is possible to provide preventive medicine or telemedicine by utilizing health data. However, wearable
health devices may not be suitable for bedridden elderly people and may beinvasive to their daily lives. In the case of clothing-type devices, the user has
to change and wash the related clothing regularly, which increases the burdenof daily life.
3.3

Collection of Health Data by Activity Tracker

In order to minimize the existing challenges, we focus on the collectionof health data by arm-worn activity tracker. Speciﬁcally, here is an example ofhandling health data from the following six items:– Number of Steps: Measuring the number of steps and the intensity of
exercise leads to the prevention of lifestyle-related diseases and the extensionof healthy life span.
– Sleep Quality: The quality of sleep (i.e., REM/non-REM sleep) is expressedby analyzing heart rate, heart rate variability (i.e., HRV, the change in thelength of the heartbeat with each beat), and activity level data.
– Stress: Estimates the factors that inﬂuence stress, such as training and physical activity, by measuring heart rate variability.
– Heart Rate: The heart rate is measured continuously for 24 hours using an
optical heart rate monitor.
– Calories burned: The total number of calories burned during exercise/rest.
– Body Battery: This unique Garmin feature measures remaining physicalenergy by analyzing heart rate variability, stress levels, sleep quality, andactivity levels.

4

Considering Approach: Integration Analysis
of Heterogeneous Data

4.1

Technical Challenges

We are considering the following two technical challenges to elucidate the factorsthat cause changes in the health status of the elderly at home.
4

https://xenoma.com/products/eskin-sleep-lounge/.
https://www.mitsufuji.co.jp/en/service/.
6
https://www.garmin.co.jp/minisite/health/guide/ (in Japanese).
7
https://healthsolutions.ﬁtbit.com/.
5

204

S. Chen et al.

The ﬁrst is the lack of richness in data types. In our research on support
systems for elderly people living at home, we have studied the extraction of
biometric data (e.g., heart rate, blood pressure, amount of activity, etc.) using
sensor devices and dedicated devices, the collection of multimedia data (e.g.,text, voice, images, etc.) using IoT devices, and the collection of environmental
data (e.g., temperature, illumination, humidity, etc.). Sensing of environmental
data (e.g., temperature, illumination, humidity). However, many studies havebeen conducted using each of these methods independently, and there have beenfew studies using diﬀerent types of data together.
Second, there is a lack of integration analysis among the data used. Inorder to elucidate the factors that contribute to changes in the health of theelderly at home, it is not enough to collect and analyze each piece of data, but
an integrated analysis (meta-analysis) among the data is necessary. Integration
analysis refers to the integration of the results of multiple studies and analysis
from a higher perspective, or methods and statistical analysis for this purpose8 .
In particular, by supplementing and verifying the results obtained from heterogeneous data, multifaceted relationships can be clariﬁed. For example, heart ratevariability in the elderly may be related to temperature.
4.2

Goal and Key Idea

The purpose of this study is to examine methods to elucidate the factors thatcause changes in the health status of elderly people at home, based on the previous studies described in Sect. 2. The key idea is to integrate and analyze healthdata, dialogue data, and the results of further analysis of dialogue data in a timeseries.
4.3

Overall Architecture

Figure 3 shows the overall architecture of the study approach, which consistsof ﬁve steps: In Step 1, the elderly person wears a wearable health device andthe VA listening service with voice interaction is set up. In Step 2, health dataand interaction data are acquired and stored through the wearable health device
and the VA listening service. In Step 3, we extract the typical changes in healthdata over time. In Step 4, we focus on dialogue data during health changes
and analyze the interrelationships among diﬀerent features. In Step 5, we searchfor personal rhythms from the previous feature analysis and consider context
estimation from the latest data. More speciﬁcally, each step is described below.
4.4

Flows of Considering Approach

Step 1: VA Listening Service and Wearable Health Installation
In Step 1, we ﬁrst deploy the VA listening service to the homes of the elderly
based on previous studies. In addition, the elderly will wear a wearable health8

https://en.wikipedia.org/wiki/Meta-analysis.

Integration Analysis of Heterogeneous Data

205

Further Analysis from Dialogue Data
Home
Emotion Recognition and Sentiment Analysis
Spoken Dialogue

Finding Explainable Changes

Dialogue Data
Integration
Analysis

VA Listening Service
Audio

Image

Text

Step

Sleep

Health Data
An Elderly
PersonWearable Health
Step 1Stress Heart Rate

Step 2

Activity

Changing Point Extraction

Step 3t1

TimeFig. 3. Overall architecture of considering approach.

device on their arms. In this research, we chieﬂy focus on arm-worn activity
meters (e.g., Garmin vivosmart 49 ).
Step 2: Acquisition and Storage of Heterogeneous Data
In Step 2, we ﬁrst acquire voice data, text data recognized from the voice,and image data taken at one-second intervals during the voice dialogue process
between the elderly person and the agent based on the VA Listening Service,and send them to the cloud database. AI techniques such as emotion analysis
are applied to these data, and the recognized results are also sent to the cloud
database. With regard to health data, activity meters are generally linked to a
dedicated smartphone application via Bluetooth, and by manually opening thededicated application, health data is transmitted and sent to a cloud database.
In order to transmit health data to the cloud database on a regular basis, we
also introduce a dialogue scenario that reminds the user to open the dedicatedapplication, and the agent asks the user to do so once a day.
Step 3: Extraction of Health Feature Changes in Time Series
In Step 3, we ﬁrst develop a Web Application Programming Interface (API)that can retrieve health data for each item (see Sect. 3.3) to the computer local(edge). GET or POST/date={yyyy-MM-dd}/data={data} in the format of REST
API [3] to retrieve the relevant health data by specifying the date and data item.
Although each health data has a diﬀerent measurement interval, the following
ﬁve measures of feature change are deﬁned to create a feature extraction programfrom past health data.
– Maximum value: m1 = max{x1 , x2 , · · · , xn } (n = count(xi ))– Minimum value: m2 = min{x1 , x2 , · · · , xn }9https://www.garmin.co.jp/products/wearables/vivosmart-4-gray-r/.

206

S. Chen et al.

– Diﬀerence value: di = |(xi − xi+1 )| (1 ≤ i ≤ n)n
– Average value: x̄ = n1 (x1 + x2 + x3 + · · · + xn ) = n1 i=1 xi– Variancevalue: s2 = n1 (x1 − x̄)2 + (x2 − x̄)2 + · · · + (xn − x̄)2n12i=1 (xi − x̄)n=Step 4: Analysis of Interrelationships Among Changing Features
In Step 4, we utilize the following three computational approaches to evaluate the interrelationship between the values of the health change features andthe dialogue recognition results. For example, we consider whether the negativescore of the dialogue content rises or not when the stress increases swiftly. We
likewise examine the change in the score of emotion analysis using facial expression and voice during the dialogue. Unlike real-time health data acquisition, it
is challenging to cope with missing dialogue data.
– Correlation coeﬃcients: rxy =n xi −x̄ yi −ȳ1
i=1 sx · sy
n

sxy
sx sy

= √n

n

i=1 (xi −x̄)(yi −ȳ)n22i=1 (yi −ȳ)i=1 (xi −x̄)

r

−r

√

=

r

xz yz
– Partial correlation coeﬃcient: rxy·z = √ xy 2 √21−rxz 1−ryzn1– Covariance: sxy = n i=1 (xi − x̄)(yi − ȳ)

Step 5: Context Estimation Based on Past Feature Analysis
In Step 5, we ﬁrst consider the interrelated feature data from the past as training
data. Next, we estimate the context by applying lightweight machine learning(e.g., supervised learning) to them. Context refers to an individual’s emotional
state and negative/positive changes. Through context estimation, we expect tounderstand the individual’s emotional state and negative/positive changes byanalyzing and estimating the health change characteristics obtained from thewearable health device even during the time when the VA listening service is
not used.

5

Discussion

This section summarizes each of the main points regarding the advantages andlimitations of the approaches considered in Sect. 4, and discusses the speciﬁcs.
5.1

Advantage

The following two advantages of the research approach are considered:– Utilization of heterogeneous data: Unlike previous related research, wewill utilize diﬀerent devices and approaches to collect and warehouse data.
This allows us to enhance the richness of the data types utilized. It can also
be used to enhance the scalability of existing systems.
– Expansion of multifaceted analysis: In addition to the utilization of heterogeneous data, the analysis of interrelationships can be anticipated to infer
the factors behind changes in the health status of the elderly at home from amore multifaceted perspective. Furthermore, it can promote and enhance theimplementation of self-care for the elderly at home.

Integration Analysis of Heterogeneous Data

5.2

207

Limitation

We examine the following two limitations of the research approach:– Trouble in dealing with missing data: There are times when the wearer
is not wearing the wearable health device (e.g., recharging, taking a bath,etc.), and there are times when the wearer is not utilizing the VA listening
service. During these times, data acquisition is not feasible, and it is hard todeal with missing data.
– Problems with data acquisition accuracy: Depending on the tightnessof the wearable health device’s belt, there are subtle diﬀerences in the data
values such as heart rate attained. There are also occasional problems with
mis-recognition of dialogue text gained by speech recognition, negative judgments, and emotion analysis.

6

Related Work

Two researches related to the approach considered in this study are described
below. Research [7] proposed a web-based medical data integration and management platform that collects heterogeneous types of health-related medical
records and real-time lifelog data. Unlike the wearable health used in this study,
the proposed platform in this study provides the ability to manage real-timedata such as heart rate, blood pressure, and activity information extracted frommedical devices and send them to a server. It also applies machine learning
tools to analyze risks based on domain knowledge and individual diﬀerences,and dynamically visualizes the results to patients and doctors based on how the
information is simpliﬁed.
Research [8] described all these important aspects of smart healthcare wearable sensors, body domain sensors, advanced pervasive healthcare systems, andnew IoT technologies for big data analytics. Unlike the focus of this work on integrated analysis in heterogeneous data, we identify new perspectives and focus
on issues such as scalability, interoperability, device-network-human interface,and security. We also present the results of an evaluation of the applicabilityof knowledge in the ﬁeld of CAD, such as large-scale analysis and optimization
methods, to key eHealth problems.

7

Conclusion

In this paper, we focused on the self-help of the elderly at home in the super-agedsociety, and we organized and discussed the ﬂow of considering the approach of
integrating and analyzing heterogeneous data in order to elucidate the factorsof health status change.
However, in order to obtain health data, the wearable health device to be used
still has the challenge of being invasive to daily life. The wearable remote healthmonitoring device [9] is expected to provide a portable health care system and208

S. Chen et al.

facilitate the provision of decentralized health care, as opposed to the traditional
centralized clinical care. The correlation between non-verbal data (e.g., facial
expression, posture, etc.) and health status of the elderly also needs to be veriﬁed
in evaluation experiments. In research [18], facial features that convey a sense offamiliarity were associated with a decrease in patients’ pain perception.
As future work, we will conduct case studies and evaluation experiments
using the study approach to clarify the correlation between actual interaction
data and health data of elderly people at home. In addition, we will utilize dataon daily behavior, daily rhythms, and environmental conditions to examine thecorrelation with changes in health status.
Acknowledgements. This research was partially supported by JSPS KAKENHI
Grant Numbers JP19H01138, JP18H03242, JP18H03342, JP19H04154, JP19K02973,
JP20K11059, JP20H04014, JP20H05706 and Tateishi Science and Technology Foundation (C) (No. 2207004).

References
1. Adorf, J.: Web speech API. KTH Royal Institute of Technology (2013)
2. Arun Francis, G., Lexmitha, S., Aruna Devi, N., Swathika, C.: Embedded system
based smart automation for elderly and disabled people. Ann. Roman. Soc. Cell
Biol. 25, 9909–9917 (2021)
3. Atlidakis, V., Godefroid, P., Polishchuk, M.: Checking security properties of cloudservice rest APIs. In: 2020 IEEE 13th International Conference on Software Testing, Validation and Veriﬁcation (ICST), pp. 387–397. IEEE (2020)
4. Chen, S., Nakamura, M.: Generating personalized dialogues based on conversation
log summarization and sentiment analysis. In: The 23nd International Conferenceon Information Integration and Web-based Applications & Services (iiWAS2021),
pp. 221–226, November 2021
5. Chen, S., Saiki, S., Nakamura, M.: Recognizing ﬁne-grained home contexts using
multiple cognitive APIs. In: 2019 International Conference on Cyber-Enabled
Distributed Computing and Knowledge Discovery (CyberC), pp. 360–366. IEEE
(2019)
6. Chen, S., Saiki, S., Nakamura, M.: Nonintrusive ﬁne-grained home care monitoring: characterizing quality of in-home postural changes using bone-based humansensing. Sensors 20(20), 5894 (2020)
7. Choi, A., Shin, H.: Longitudinal healthcare data management platform of healthcare IoT devices for personalized services. J. Univers. Comput. Sci. 24(9), 1153–
1169 (2018)
8. Firouzi, F., Farahani, B., Ibrahim, M., Chakrabarty, K.: Keynote paper: from EDA
to IoT eHealth: promises, challenges, and solutions. IEEE Trans. Comput. Aid.
Desi. Integr. Circ. Syst. 37(12), 2965–2978 (2018)
9. Ghosh, R., et al.: Micro/nanoﬁber-based noninvasive devices for health monitoring
diagnosis and rehabilitation. Appl. Phys. Rev. 7(4), 041309 (2020)
10. Hirayama, K., Chen, S., Saiki, S., Nakamura, M.: Toward capturing scientiﬁc evidence in elderly care: Eﬃcient extraction of changing facial feature points. Sensors
21(20), 6726 (2021)

Integration Analysis of Heterogeneous Data

209

11. Jan, H., Yar, H., Iqbal, J., Farman, H., Khan, Z., Koubaa, A.: Raspberry PI assisted
safety system for elderly people: an application of smart home. In: 2020 FirstInternational Conference of Smart Systems and Emerging Technologies (SMARTTECH), pp. 155–160. IEEE (2020)
12. Kul, S., Sayar, A.: A survey of publish/subscribe middleware systems for microservice communication. In: 2021 5th International Symposium on Multidisciplinary
Studies and Innovative Technologies (ISMSIT), pp. 781–785. IEEE (2021)
13. Lee, A., Oura, K., Tokuda, K.: MMDAgent-a fully open-source toolkit for voiceinteraction systems. In: 2013 IEEE International Conference on Acoustics, Speechand Signal Processing, pp. 8382–8385. IEEE (2013)
14. Maeda, H., Saiki, S., Nakamura, M., Yasuda, K.: Rule-based inquiry service toelderly at home for eﬃcient mind sensing. In: Proceedings of the 21st International
Conference on Information Integration and Web-Based Applications & Services,
pp. 664–668 (2019)15. Manelli, L., Zambon, G.: Introducing JSP and Tomcat. In: Beginning Jakarta EE
Web Development, pp. 1–53. Apress, Berkeley (2020). https://doi.org/10.1007/
978-1-4842-5866-8 1
16. Marć, M., Bartosiewicz, A., Burzyńska, J., Chmiel, Z., Januszewicz, P.: A nursing
shortage-a prospect of global and local policies. Int. Nurs. Rev. 66(1), 9–16 (2019)17. Martins, H., Gupta, N., Reis, M.J.C.S.: A non-intrusive IoT-based real-time alert
system for elderly people monitoring. In: Paiva, S., Lopes, S.I., Zitouni, R., Gupta,
N., Lopes, S.F., Yonezawa, T. (eds.) SmartCity360 2020. LNICST, vol. 372, pp.
339–357. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-76063-2 24
18. Mattarozzi, K., et al.: Pain and satisfaction: healthcare providers’ facial appearance
matters. Psychol. Res. 85, 1706–1712 (2021)
19. Miura, C., Saiki, S., Nakamura, M., Yasuda, K.: Implementing and evaluating feedback feature of mind monitoring service for elderly people at home. In: Proceedingsof the 22nd International Conference on Information Integration and Web-based
Applications & Services, pp. 390–395 (2020)
20. Sakakibara, S., Saiki, S., Nakamura, M., Matsumoto, S.: Implementing autonomous
environmental sensing in smart city with IoT based sensor box and cloud services.
Inform. Eng. Express 4(1), 1–10 (2018)
21. Tamamizu, K., Sakakibara, S., Saiki, S., Nakamura, M., Yasuda, K.: Capturing
activities of daily living for elderly at home based on environment change andspeech dialog. In: Duﬀy, V.G. (ed.) DHM 2017. LNCS, vol. 10287, pp. 183–194.
Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58466-9 18