International Journal of Informatics Society, VOL.15, NO.2 (2023) 103-111

103

Regular Paper

Prototype Development and Evaluation of an Avatar Remote Communication
System for ALS Patients Using Eye-tracking Interface
Nagisa Kokubu *, Takayo Namba **, and Keiichi Abe ****Kanagawa Institute of Technology, Atsugi City, Kanagawa, Japan
Kawasaki City College of Nursing, Kawasaki City, Kanagawa, Japan
abe@he.kanagawa-it.ac.jp

Abstract -ALS patients with severe disease have difficultyin walking, going out, and talking with people. The authors
believe that there is a need for a system that enables people
with severe ALS to communicate with many people at homewhile maintaining contact with society. A typical example
of a solution to this problem is an alter-ego robot. This robot
is placed in a place where ALS patients cannot go, and it cancommunicate with the other person remotely by gestures.
However, the robot needs to be set up at the place where thepatient is going to talk with the other person, and the robot
needs to be removed from the place after the conversation.
In this study, we propose a system that enables ALS patientswith severe symptoms who have difficulty in going out totalk with many people through avatars such as computer
graphics (CG), which are alter ego characters, easily andremotely by the patients themselves. We also describe indetail the development and evaluation of a prototype of our
proposed system.
Keywords: Remote Communication, ALS, online conferencing tool.

1

INTRODUCTION

Amyotrophic lateral sclerosis (ALS) is a disease in which
the nerves that control voluntary movements (motor
neurons) are damaged, resulting in the loss of muscles
throughout the body and those necessary for conversation.
In severe cases, the patient becomes bedridden and isolated,with no contact with other people, and it is difficult for thepatient to move his/her body, although he/she is conscious.
A previous study aiming to solve this problem is an avatar
robot [1]. This avatar robot is placed in a place where ALS
patients cannot go, and it can communicate with a person
remotely by gestures. However, it is necessary to set up theavatar robot and remove it after the conversation. Therefore,
the system requires help from other people besides the ALS
patient.
In this study, we propose a system that enables ALS
patients to communicate with many people by themselves inorder to keep contact with society and enjoy communication
with various people. In this study, we propose a system thatenables ALS patients to talk with many people at their own
will by using eye control technology, avatar technology, andonline conferencing tools [2]. Furthermore, the systemallows ALS patients to choose their favorite avatar character

during the conversation. We developed and evaluated a
prototype of the proposed system. In order to evaluate theeffectiveness of the proposed system from various
viewpoints, we performed three questionnaires: one is aquestionnaire for ALS patients, two is a questionnaire for
medical professionals who actually work with ALS patients.
and the other is a questionnaire for able-bodied persons. The
details are described below.

2

RELARED WORKS

When ALS patients become severely ill, it becomesdifficult for them to move their bodies and to speak.
Conversation aids are an existing technology to help
severely ill ALS patients communicate their intentions. The
first representative product of portable conversational aids isa keyboard input type conversational aid [3]. This keyboardinput type conversation aid uses the keyboard to create
sentences, which are then spoken by a voice synthesizer. It
also has functions for registering frequently used words in
categories and communicating using images. However, thiskeyboard input type conversation aid is difficult to use forseverely ill ALS patients who have quadriplegia and havedifficulty moving their bodies because the device requires
the user to input data by hand. On the other hand, there is agaze-input conversation aid [4]. Gaze input typeconversation aids are devices that use a gaze input instead ofa keyboard to create sentences and have them spoken byspeech synthesis. Gaze-input conversation aids use a special
gaze-input interface which is expensive and increases thesystem introduction cost. Common to these two products is
the issue of not being able to converse with many people atonce because they are face-to-face conversational types. As
a previous study to solve these problems, there is an avatar
robot developed by Oly Research Institute. This robot is
installed in a place where ALS patients cannot go, and it cantalk with a person whom the patient wants to talk withremotely by gestures. However, the robot needs to be set up
at the place where there is the person whom the ALS patientwants to talk to, and the robot needs to be removed after theconversation is over, which requires the help of others
besides the ALS patient. Another avatar technology otherthan robots is the Metaverse [6]. The avatars used in the
metaverse are computer graphics, and users can selectvarious characters of their choice. This technology allows
people who have difficulty communicating in the real world

ISSN1883-4566 © 2023 - Informatics Society and the authors. All rights reserved.

104

N. Kokubu et al. / Prototype Development and Evaluation of an Avatar Remote Communication System for ALS Patients Using Eye-tracking Interface

to freely enjoy conversations and other activities in thevirtual space through their favorite avatars. In this metaverse,
there is no need to set up an avatar robot in advance at theplace where you want to have a conversation, nor is it
necessary to remove the avatar after the conversation is over.
However, the interface currently used in the metaverse is atechnology for able-bodied people, and it is difficult forpeople with physical disabilities, such as ALS patients, who
cannot speak. On the other hand, a prior study of CG avatars
is the Avatar Customer Service [5], which has begun to be
introduced in convenience stores. This technology wasdeveloped to realize a new way of working that is not
restricted by time, place, age, gender, or various otherobstacles, and has been installed in actual convenience
stores. The system displays a CG avatar on the display
screen and uses gestures and hand gestures to converse withthe customer. This technology does not burden others asavatar robots do because it is operated remotely using apersonal computer, and it allows users to communicate
remotely via avatars without questioning the location of theuser. However, these prior technologies have issues thatmake it difficult to use in cases such as ALS patients withphysical disabilities, because they are only for able-bodied
people.
Therefore, in related work on the past，to the best of ourknowledge, remote video communication for ALS patientswith the help of eye-tracking does not exist. To solve thisproblem, we propose a remote video communication usingeye- tracking for ALS patients who have difficulty going out.
additionally, we propose a system that allows you to have apleasant conversation with many people remotely using
avatars, which are your favorite spoken dialogue agents.

3

PROPOSED SYSTEM

Section 3 provides a detailed overview of the avatar remotecommunication system for ALS patients using videoconferencing tools proposed in this paper.

3.1 Outline of the Proposed System
Figure 1 shows a schematic diagram of the proposedsystem. The proposed system uses an online conferencing
tool [2] as a telecommunication tool for the patient and theother party to have a conversation. There are two reasons forusing an online conferencing tool in this paper. One is that
ZOOM is widely used around the world as an online
conferencing tool and is compatible with various platforms.
The second reason is that ALS patients can easily
communicate with their counterparts at home or in thehospital, regardless of their location, as long as the otherparty has an information terminal that can use the online
conferencing tool and a network connection is available. We
also thought that if the other party to the conversation used asmartphone or similar device, there would be no burden ofremoving the device after the conversation, and both parties
could talk without burdening the other. Next, the proposedsystem uses an eye tracking interface [7] for the ALS
patient's side. We adopted the Eye Tracking interface
because it is less physically demanding than keyboard

Figure 1: ARCS-ALS overview

Figure 2: ARCS-ALS Prototype System Overview
operation, since severely ill ALS patients have significantly
reduced physical abilities. In this system, the characters
input by the eye tracking interface are directly output by
text-to-speech to realize conversation. In addition, thesystem allows the user to freely add and select a characterfor the spoken dialogue agent that serves as the alter ego of
the ALS patient by utilizing the function of adding andselecting spoken dialogue agents of a previous study [8]. As
in the previous study, the ALS patient's favorite voiceinteraction agent can be used as an avatar by adding GIF(Graphics Interchange Format) files of his/her favorite
cartoon character in conversation and in standby mode to thesystem, using FaceRig [9] or other software. The systemallows users to use their favorite characters as avatars. By
adding these two files to the system, a live-action video of the
ALS patient himself can be used as a voice interaction agent.
By adding a voice model for each character and configuring
detailed settings such as voice, speed of speech, and endings,
the system can output different voices for each character. The
system allows users to remotely communicate with theirconversation partners via the spoken dialogue agent bytransmitting the display screen of the spoken dialogue agent
selected by the user on the shared screen of an online
conference tool. The system also has a branch speaker
output so that the user can talk not only remotely but alsowith a person nearby. In this paper, we call the proposedsystem Avatar Remote Communication System for ALS
Patients (ARCS-ALS).

International Journal of Informatics Society, VOL.15, NO.2 (2023) 103-111

105

4 PROTOTYPE DEVELOPMENT
Section 4 details the development of the ARCS-ALS
prototype system proposed in Section 3.

4.1 Outline of Prototype System
Figure 2 shows an overview of the prototype system. In
this prototype, distributed processing is performed by aWindows PC and a Raspberry PI3B+ (hereafter referred toas "RPI"). The Windows PC is in charge of the mainprocessing, and the RPI is in charge of the sub-processing.
The main processing on the Windows PC is the input
processing of the Eye Tracking interface (model
EyeTracke4C/Tobii) [7] used by the ALS patient to operate
the ARCS-ALS, the switching processing of the video instandby and conversational mode according to the speechsynthesis of the spoken dialogue agent that talks on behalfof the ALS patient, the input processing of the video instandby and conversational mode according to the speechsynthesis of the spoken dialogue agent that talks on behalfof the ALS patient. The RPI is in charge of the subprocessing, and is in charge of the video switching between
standby and conversation, as well as the remotecommunication with the other party via ZOOM. On theother hand, RPI, which is in charge of sub-processing,
outputs the input from the Windows PC to the voice of the
spoken dialogue agent and instructs the Windows PC whento switch between the live-action video in standby modeand the live-action video during conversation. The text-tospeech output from the sub-processing (RPI) was input tothe microphone on the Windows PC side of the mainprocessing via the speaker output of the RPI. This allows
the synthesized voice to be output to the other party via
ZOOM. In addition, the system is designed to output thesynthesized voice from the sub-processing (RPI) to the
speaker on the ALS patient's side through a branch. This
enables the patient to talk not only online but also withpeople in the vicinity. Open JTalk [10] was used for the
speech synthesis of the spoken dialogue agent.

4.2 Application Screen
In the prototype of the proposed system, ARCS-ALS, wedeveloped a voice dialogue application to be used by ALS
patients when conversing with each other. Figure 3 is a
general view of the prototype system developed for thisproject, and Fig. 4 shows the screen of the application
developed this time. As shown in Fig. 4, the large characterin the middle of the screen is the selected character. In otherwords, it will be a voice dialogue agent that will carry out
conversations on behalf of the ALS patient. When thepatient wants to change the selected agent to suit his/her
preference, he/she can click on the character display on theright side of the screen to change the character
automatically.
In this prototype, we prepared four characters: the user's
own image, a child character, and animal characters of a
dog and a cat. Two methods were prepared for the input of

Figure 3: Prototype system developed for this project

Figure 4: Application screen for ARCS-ALS
the conversation. In the first method, frequently usedconversations such as replies, and greetings are registered inadvance and can be spoken at the touch of a button. The
second method is to use the keyboard, which is one of thefunctions of the eye control input interface, to input text. In
this method, the ALS patient inputs what he/she wants to
say using the keyboard and presses the "send" button to
speak. Although it takes some time to input the content of
the conversation, any kind of conversation can be sent.

4.3

Eye Tracking Interface

ARCS-ALS uses the Eye Tracking interface [7] as amethod for ALS patients to talk. The reason for using thisinterface is that even ALS patients who have limb
disabilities and have difficulty moving their bodies can use
the Eye Tracking interface for eye control if they do nothave eye problems. In our proposed system, we used an offthe-shelf eye tracking interface and the eye tracking
function [7] that comes standard in Windows 10. To use thesystem, the Eye Tracking interface is connected to a
Windows PC via a USB interface, and the main unit of the
Eye Tracking interface is attached to the bottom of themonitor. After installing the driver, adjusting the position of
the Eye Tracker 4C, and performing simple setups such as avisibility test, the system is able to detect the user's line ofsight on the Windows PC screen. Next, by activating theeye control function that comes standard with Windows 10
and later, the user can click, use the mouse, and performkeyboard input, so even the physically challenged canoperate the PC using only their line of sight. The Eye
Tracking interface should be installed at an appropriate
distance from the user, which is approximately 90 cm. If the

106

N. Kokubu et al. / Prototype Development and Evaluation of an Avatar Remote Communication System for ALS Patients Using Eye-tracking Interface

Figure 5: Overview of Eye Tracking interface
distance is too close or too far, the camera cannot becalibrated accurately and may not operate properly.
Figure 5 shows the eye tracking interface used in the
prototype and the eye control function. The eye tracking
interface attached to the lower part of the monitor (circled
in red) recognizes the user's line of sight, and the eye
control function (circled in green) is used to control theoperation. By looking at the desired function for a certain
period of time, the user can instruct the eye tracking
function to display icons to select the desired location for
clicking and a keyboard for inputting text, as shown in Fig.
5. At first, it may be a little difficult to operate the function
you want to use, but as you gradually become accustomed
to it, you will be able to operate it at will.

4.4 Voice Dialogue AgentIn the ARCS-ALS system, a voice dialogue agent is usedwhich talks to the other party instead of the patient. The
voice dialogue agent was created by applying our previous
studies. In this prototype, we prepared four characters: acharacter created from the ALS patient's own image, a childcharacter, and animal characters (a dog and a cat). This
allows the user to choose one of the characters according tohis or her preference and environment, such as those whowant to show their healthy self when talking to otherpersons or those who want to appear to be a substitutecartoon character instead of their own image. The voice
dialogue agent can be created in two main ways: as a CG
character or as a live-action character. Figure 6 shows asummary of each method. First, CG characters were createdusing FaceRig [9] software, which allows the user tobecome any character using a webcam, and two GIF images
were created: one in the standby mode (as if the user islistening to a conversation) and one in the conversationalmode (as if the user is talking). By displaying the two GIF
images in accordance with the speech state of the voice
dialogue agent using text-to-speech output and switching thecharacter, the mouth movements are synchronized with thevideo as if the character is actually speaking. The live-action
character is an application of our previous work on creating
CG characters. To create a live-action character, two liveaction videos were actually shot using video equipment, one
in the standby mode and the other in the conversational
mode.

Figure 6 : How to Create Voice Dialogue Agents

Figure 7: Speech synthesis output mechanism
As in the previous study, a live-action character is easily
created by switching between the two GIF images inaccordance with the speech state. In this case, it is necessary
to shoot a short movie of 3 to 5 seconds and change themovie file to a GIF file. This makes it possible to repeat andplay back the short video, which allows for longer text
conversations. This method has the advantage that, once theuser becomes accustomed to it, a live-action character canbe created in about 10 minutes, making it possible to createa character of the person to be reproduced at a low cost. In
addition, it is considered to be better to create the image of
the ALS patient before the disease becomes severe, because
it is considered to be practically difficult for the ALS patient
to film himself after the disease has become severe Voicesynthesis output.
The voice of the voice dialogue agent, which is the avatarof the ALS patient, is synthesized by Open J Talk [10], andits output is shown in Fig. 7. In this system, the ALS patientuses the eye control input interface to input the content of
the conversation in text using the application developed inthis study. The input conversation content is output as textto-speech by Open J Talk [10], and the voice dialogue agent
speaks the content. Acoustic models are prepared for each of
the voice dialogue agents selected by the user, and the voice,speech style, and endings are changed for each character tospeak. The dog CG character would end with "bow-wow"
and the cat CG character would say "meow". On the other
hand, when using a live-action character created from the
ALS patient's own video, we prepared an acoustic modelthat closely resembled the patient's own voice, and thenmanually adjusted the parameters using Open J Talk tomake the voice sound similar to the patient's own voice.

International Journal of Informatics Society, VOL.15, NO.2 (2023) 103-111

107

Specifically, the voice quality, which can be changed tofeminine or masculine by changing related parameter values,
the pitch shift, which changes the tone of the voice, and the
speech speed, which changes the speed of speech, were
adjusted to match the patient's voice. The values of thesevoice parameters were used because they can be fine-tuned,which actually takes time but produces highly reproducible
voice. Other speech synthesis technologies include thosethat utilize AI technology to learn voices. Coestation [11][12] are representative of theseand LYREBIRD
technologies. With these technologies, it is easy to produce avoice synthesizer that is close to the user's own voice byreading a few examples of sentences and having the AI learn
those sentences. However, since the example sentences need
to be learned repeatedly, it is not possible to train the AI for
ALS patients who have difficulty in speech. Therefore, thissystem uses Open J Talk, which allows the user to manually
adjust the parameters of the acoustic model based on theoriginal voice.

evaluation of able-bodied persons was conducted on 20
male subjects in their 20s. The survey method consisted of
explaining the outline of the proposed system and theprototype to the subjects, having them watch ademonstration video of the actual use of the prototype(about 5 minutes), and then asking them for answering thequestionnaire. The survey consisted of five major questions.
These of evaluation procedure will be basically the same,
just with different subjects.

5

EVALUATIONIn this Section 5, a questionnaire evaluation was
conducted to evaluate the prototype of the proposed system.
This time, in addition to the questionnaire evaluation
targeting ALS patients, the evaluation was conducted fromthree viewpoints targeting medical professionals who haveknowledge about ALS patients, able-bodied persons. From
the questionnaire evaluation, we investigated theeffectiveness of the proposed system ARCS-ALS as acommunication tool that enables ALS patients to enjoy
talking with many people by their own power, and thefunctions and improvements that are required for this system.

5.1 Method of Evaluation
First, we conducted a questionnaire evaluation of one
ALS patient through an acquaintance who had a close
relationship with the patient. Face-to-face evaluation was
difficult this time because of the coronary disaster. Although
it was difficult to have the participants actually use theprototype, we asked them to understand the proposed systemand to answer a questionnaire based on the assumption thatthey had used the prototype. This time, we evaluated thequestionnaire on the assumption that the prototype wasactually used, but we asked ALS patients who cooperated inthis survey to fill out the questionnaire because they had
experience using eye tracking interfaces in the past, and wethought they would give us their opinions. In the second
evaluation, a questionnaire was sent to medical personnel
involved with ALS patients. This time, a questionnaireevaluation was conducted on 26 medical personnel atseveral home health care nursing stations in Kanagawa
Prefecture. In this survey, we created a video (about 9
minutes long) explaining the proposed system and showing
an actual conversation using the prototype, so that the
medical personnel could answer the questionnaire at any
time according to their work schedule. The video waswatched during breaks, etc., and the participants were asked
to answer a questionnaire afterward. The Third evaluation
was conducted with able-bodied persons. The questionnaire

5.2

Evaluation Results of ALS Patients

Table 1 shows the results of the questionnaire for ALS
patients. The answer was "Yes" to the question 1 "Would
you like to use the system you saw in the video? Yes" to thequestion "Would you like to use the system shown in thevideo? This indicates that the proposed system is usefulfrom the viewpoint of ALS patients. In response to question
2, "In what ways would you like to use the system? The
answers to all the questions were "I can use it by myself," "I
can communicate easily," "I can communicate with others as
my favorite character," and "I can talk with many people
even if I cannot go out. In addition, "It is easy to listen to
because of the voice synthesis and pronunciation intonation"and "Predictive text conversion is easy to use. These results
indicate that the proposed system has achieved theobjectives of the proposed system. Next, to question 3,"What would you like to use as a character for the spoken
dialogue agent in this system? the answers were "My current
live-action video," "My favorite animal (dog, cat, etc.)
character," "My favorite animation," and so on. Thissuggests that the respondents would like to use not only
their current live-action video, but also their favorite animals
and favorite animation characters for conversation. In
response to question 4, "What would you like to use as thevoice of this system? the respondents answered "myvoice","favorite star's voice", "favorite anime", etc.,
suggesting that they would like to select a voice thatmatches the avatar character they are using. Next, There
were no responses to the question 5, "What improvements
do you think are needed in the layout of the application
screen of the system? In response to question 6, "In addition
to the remote conversation function, what other service
functions would you like to see in this system? was "moretypes of keyboards" as other opinions. Next, There was noanswer to the question 7, "What do you think needs to be
improved in the future? Finally, to the question 8, "Pleasefeel free to give us any other comments or requests you may
have. The last question was "Please feel free to give us anyother comments or requests. I use the internet, YouTube,
Facebook, LINE, and recently google spreadsheets insteadof Excel. The rest I rarely use, so I thought it would bebetter to create a well-balanced system. Please let me makea decision after actually using the system. In summary, for
ALS patients who responded to this survey, in addition to
the avatar remote communication function, it would be
useful to have various functions that would help them intheir daily lives, such as remote control of home appliances,
health management, and a doorbell.

108

N. Kokubu et al. / Prototype Development and Evaluation of an Avatar Remote Communication System for ALS Patients Using Eye-tracking Interface

Table1:Results of Questionnaire Evaluation of ALS Patient.

but however, considering the cost at the time of installation,
we feel that a minimal number of functions is sufficient. In
other words, in addition to the proposed remotecommunication function, it would be good if the user could
watch YouTube videos on the Internet or send messages via
SNS.

5.3

Evaluation Results of Medical Personnel

Figure 8 shows the results of the questionnaire forhealthcare professionals involved with ALS patients. Thistime, the questionnaire was sent to 26 medical professionals
involved with ALS patients. The gender of the respondents
was 22 females and 4 males, and the age breakdown was asfollows: 1 respondent was in her 20s, 8 were in their 30s, 10were in their 40s, 5 were in their 50s, and 2 were in their 60s
or older. To the question 1, "Do you feel that the system you
saw in the demo video is a system that you would
recommend to ALS patients? 96% of all respondents
answered "Yes" and 4% answered "No". This indicates that
more than 90% of the medical professionals highly
evaluated the proposed system.

Figure 8: Questionnaire evaluation results for medical
professionals.

International Journal of Informatics Society, VOL.15, NO.2 (2023) 103-111

109

Next, in response to question 2, "For those who answered
yes to question 1, in what ways did you think it was good? "The most common answer was "Because I can talk withmany people without going out" was 19 responses, followed
by "Because it is easy to communicate" was 18 responses,
and then "Because it can be used by the patient alone" was 9
responses. These results indicate that the proposed system's
purpose of communicating with many people without
having to go outside and its ease of use were wellunderstood and appreciated by many medical professionals.
Next, in response to question 3 of "If you answered no toquestion 1, I would like to ask you. What did you think was
wrong with it?" one respondent answered "Because it isdifficult for one patient to use it alone. Next, in response toquestion 4 of "Do you think this system would be a burden
for ALS patients to use?" about 81% of all respondents
answered "Yes," and about 19% of all respondents answered"No". In response to Question 5 of " Those who answered
yes to question 4. Why?" , the most common answer was "Ittakes practice to get used to it" was 17 responses, "Stressfulin operation" was the second most common was 10
responses, and "Stressful because it is hard to hear thevoice" and "Eyes get tired" were the third most common
was 5 responses. This indicates that many respondents
thought that the system requires practice before they getused to it, and that many healthcare professionals thoughtthat the system would be stressful to operate. Next, in
response to question 6, "What needs to be improved in thissystem?" the most common response was "voice quality"with 17 responses.
Other responses were "additional applications" was 4
responses, "screen fixation," "size of equipment," and"screen size" was 3 responses, and "starting up this system"
was 2 responses. This suggests that the current system isdifficult for ALS patients to listen to due to the poor voice
quality of the avatar character speaking. Finally, to question
7, "If you were to evaluate this system overall, which would
you rate it in terms of...?" 7% of all respondents answered"very good (5)," 58% answered "good (4)," 31% answered"average (3)," 4% answered "not good (2)," and 0%
answered "not so good (1)." The total of "very good" and"good" responses amounted to 65% overall. The averagescore on a 5-point scale from "very good: 5" to "not sogood: 1" was about 3.69, which is a reasonably high rating．
Therefore, although the proposed system needs to be
improved in terms of "voice quality" and "requires practice
until the user becomes familiar with the operation," thesystem is considered to be reasonably useful for ALS
patients from the viewpoint of medical professionals who
are usually involved with ALS patients.

5.4 Evaluation Results of Able-bodied Persons
Figure 9 shows the results of the questionnaire with
able-bodied persons. All respondents answered "Yes" to thefirst question, "Do you feel that you would recommend thissystem to ALS patients?" For those who answered "yes," thesecond question was asked: "In what ways do you think thissystem is recommended for ALS patients?”

Figure 9: Questionnaire evaluation results for able-bodied
persons.

110

N. Kokubu et al. / Prototype Development and Evaluation of an Avatar Remote Communication System for ALS Patients Using Eye-tracking Interface

The most popular answer was “Because they cancommunicate autonomously (71%)”. The next mostcommon answer was "Because it seems easy tocommunicate" (14%), followed by "Because it seems easy
to operate" (10%), and "Because I can become my favorite
character" (0%). The other responses were "the cost of
introduction is low" and "it is easy to use”. Next, Question 3
was asked: "Which of the voice dialogue agent characterswould you like to use in this system?” The most common
answer was "Live-action video of ALS patients in the pastwhen they were healthy" (12 responses), and "Live-action
video of ALS patients in the present" (10 responses), both ofwhich were answered by more than one half of all the
respondents. Among the other responses, "Favorite charactersuch as anime" was selected by 8 respondents, "Favorite
character such as animal" by 5 respondents, and "Youngmale/female character" by 3 respondents, indicating that
"Favorite character such as anime" was the most soughtafter character for ALS patients other than live-action videos.
As for Question 4, "What functions would you like to see inthis system?”, the two most popular responses were “addcanned text” (12) and “display of the content of the otherparty’s conversation” (11). The next most common
responses were "remote control of home appliances" (7),"provision of daily life information" (5), "healthmanagement function" (3), and "increase in the number ofcharacters" (2). Finally, to Question 5, “What do you thinkneeds to be improved in the future?”, the most common
response was "waiting time during text input" (12 responses),followed by "speed of conversation" (11 responses), both ofwhich were cited by more than one half of the respondents
as an area that needs improvement. Next were
"awkwardness of response" and "operability" (5 responses
each), and "appearance" (1 response).

5.5 Summary of Evaluation Results
In this prototype evaluation, the proposed system wasevaluated from various perspectives by ALS patients,
medical professionals working with ALS patients and ablebodied persons. As a result, more than 90% of the
respondents answered that they would recommend the
proposed system to ALS patients, and the system was highly
evaluated. The reasons for this are as follows: "I can use thesystem by myself," "I can communicate easily," "I cancommunicate with others as my favorite character," and "I
can talk with many people even if I cannot go out,"
according to the questionnaire results from the ALS patients'side.
On the other hand, the most common response frommedical professionals involved with ALS patients in thequestionnaire was "I like the fact that I can talk with many
people without going out", followed by "I can communicate
easily" and "The patient can use the system alone". This
indicates that the medical professionals involved with ALS
patients understood and highly evaluated the features of the
proposed system. On the other hand, however, when
medical professionals were asked, "Do you think this system
will be a burden for ALS patients? 81% of the respondents
answered that the system would be a burden for ALS

patients. The reason for this is that it requires practice until
they become accustomed to its operation. Also, some
respondents felt that the proposed system is burdensome interms of operation. This point is considered to be an issue
for the future of this system. Next, we asked, "Are there any
functions you would like to see in the proposed system? The
ALS patient's answers were "a function to display
conversations," "a function to remotely operate home
appliances," "a function to measure and manage the user's
health (temperature, pulse, SpO2)," "a function to provide
daily life information such as weather and time," etc., aswell as "a doorbell and zoom function.
However, in consideration of the cost at the time of
installation, ALS patients' opinions indicated that, as aminimum, in addition to the proposed remote avatar
communication function, they would like to be able to watch
YouTube videos on the Internet and send messages via SNS.
Furthermore, when asked, "What do you think needs to be
improved in the future?" the ALS patients answered that
there were no areas that needed improvement, while themedical professionals most frequently asked for
improvement in "voice quality. This indicates that thequality of the voice output from the avatar is something thatneeds to be improved in the future. In addition, the medical
professionals who responded to the questionnaire gave theproposed system a five-point overall rating, with an averagescore of 3.69, indicating that they gave the system areasonably high rating.

6

CONCLUSION

In this paper, we proposed a remote video communication
using eye-tracking for ALS patients who have difficulty
going out. additionally, we proposed a system that allows
you to have a pleasant conversation with many people
remotely using avatars, which are your favorite spoken
dialogue agents. The proposed system utilizes an online
conferencing tool as a method of conversation with otherparties, and uses an interface that can be operated by the
ALS patient himself/herself by using an eye tracking
interface. In addition, we have developed a new dedicatedapplication for ALS patients to conduct a conversation. This
enables ALS patients to easily converse with many people attheir own will without the help of others. In addition, wedeveloped an actual prototype of the proposed system andevaluated the prototype. In the evaluation, the prototype was
evaluated from two perspectives: a questionnaire evaluation
for ALS patients themselves and a questionnaire evaluation
for medical professionals involved with ALS patients. The
evaluation results confirmed the effectiveness of the
proposed system, ARCS-ALS. We were also able to confirm
the required functions of the system and points to be
improved.

7

FUTURE WORK

Through the development and evaluation of the prototypeof the proposed system, we have learned some details thatneed to be improved, and we plan to improve this prototype
for practical use in the future.

International Journal of Informatics Society, VOL.15, NO.2 (2023) 103-111

ACKNOWLEDGEMENTS
We would like to express our sincere gratitude to the
TOWN NURSE Home-visit Nurse Station, KAWASAKI
Home-visit Nursing Station, HIBARINOMORI Home-visit
Nurse Station, and KAWASAKIK DAISHID Home-visit
Nurse Station for their cooperation in this study. and We
would like to express our sincere gratitude to Mr. Riku
Sugimoto and Mr. Shouma Hamada for their cooperation inthis study.
This study was approved by the Ethical Review Board forthe use of human subjects of Kanagawa Institute ofTechnology (No.20220715-01).

REFERENCES[1] "Orihime Product Introduction ", < https://orihime.orylab.com/>, (Referred May.2022)．
[2] “Zoom”，< https://zoom.us/jp-jp/meetings.html>，(Referred May.2022)．
[3] “TALKING AID+” ， <https://www.talkingaid.net/
products/ta-plus>，(Referred May.2022).
[4] "LUCY Double Giken " ， <http://www.j-d.co.jp/
fukushikiki-lucy.html>，(Referred May.2022).
[5] "LawsonLaboratory",<https://www.lawson.co.jp/
lab/tsuushin/art/1458363_4659.html>,(Referred May.
2022).
[6] "Metaverse Japan", <https://metaverse-japan.org/>,(Referred May.2022)．
[7] Eye Tracker Product List|Tobii Technology<https://www.tobiipro.com/ja/product-listing>,( Referred May.2022).
[8] N. Kokubu, T.Mukai, and K.Abe, " A Proposal for a
Low-Cost voice Dialogue System Using Live-Action
Video Contents ", 2021 IEEE Global Conference on
Consumer Electronics Proceedings (GCCE2021) ，pp.340-341, Oct. (2021).
[9] FaceRig Live2D Module, <https://www.live2d.
com/in-terviews/facerig>, (Referred May.2022).
[10] Open J Talk, < http://open-jtalk.sp.nitech.ac.jp>,(Referred May.2022).
[11] "coestation", <https://coestation.jp/consumer/coestationapp/>, (Referred May.2022).
[12] "Lyrebird", <https://www.descript.com/lyrebird>,(Referred May.2022).
(Received: December 8, 2022)(Accepted: October 4, 2023)

111

Nagisa Kokubu received the M. E. degreefrom Kanagawa Institute of Technology in2023. His research interests include Avatar
Remote Communication System for ALS patients, Voice Dialogue System. He is currently an sales engineer in TOKYO ELECTRIC POWER COMPANY HOLDINGS.
INC.

Takayo Namba received a Ph.D. from Nagano College of Nursing in 2009. She was aprofessor of the faculty of nursing, Kanagawa
Institute of Technology from Apr.2017 toMar.2019. She was a professor of Teikyo
Heisei University from Apr.2021 to Mar.2022.
She has been a professor of Kawasaki City
College of Nursing since Apr.2022. Her research interests about Elderly Abuse and Codependency, Educational program and Visiting nurses. She is a member of the Japan
Academy of Nursing Science and the Japan
Society for Health and Welfare etc.
Keiichi Abe received a Ph. D. in informatics
from Shizuoka University in 2012. He has beena professor of the Faculty of Creative Engineering, Kanagawa Institute of Technology.
His research interests include smart system using IOT technology, Interactive system using
Robot and CG, and Victims Information Management System At large scale disaster, HEMS.
He is a member of IPSJ，IEICE, ACM andIEEE.