Development of a prototype simulation model for mealtime
assistance and a spoon navigation application
Hirokazu Kawana

Masumi Muramatsu

Atsushi Mitani

gekitetz Inc.
Digital Hollywood University Graduate
SchoolTokyo, Japan
access@gekitetz.com

School of Nursing, Sapporo City
University
Sapporo, Japanm.muramatsu@scu.ac.jp

School of Design, Sapporo City
University
Sapporo, Japana.mitani@scu.ac.jp

Abstract—Japan has a substantial problem in regard to its
rapidly aging society. To realize the goal of a “100-year life” setby the Ministry of Health, Labour and Welfare, it is crucial for
older adults to maintain healthy oral functions as long as possible .
Mealtime assistance helps older adults take food by mouth and istherefore essential in nursing and caregiver education. However,students have few opportunities to receive direct coaching from
instructors because of low student/teacher ratios. Recently,
simulation education has attracted attention as an effective
process for nursing and caregiver students before clinical practicein hospitals and care facilities. With the aim of improving the
educational environment in oral care and the effectiveness of
simulation education, we have developed several oral caresimulation models, including a prototype mealtime assistancesimulation model consisting of denture and tongue models with asensor system to detect spoon movement. Here, we describe thedevelopment of a prototype mealtime assistant simulation modelshowing an improved humanoid appearance with a jaw joint
driven by a servo motor and an improved tongue model with asensor board embedded with three photo reflectors. We also
developed a spoon navigation algorithm for the tongue part basedon our former model and a spoon movement navigation
application, and evaluated the function in a series of experiments.
Keywords—simulation education; oral care; simulation
model; mealtime assistance; sensing; interaction design

I. INTRODUCTION
An aging society poses a significant problem for manynations such as Japan. To achieve the goal of a “100-year life”,
the Japanese Ministry of Health, Labour and Welfare haspromoted strategies for enhancing the quality of life andmaintaining the health and oral functions of older adults aslong as possible. Oral functions deteriorate owing to an agerelated loss of muscle strength and particular diseases that
affect older adults, such as Parkinson’s disease and dementia,
which usually lead to dysphasia. Therefore, mealtime
assistance skills are essential in nursing and caregiver
education.
In clinical practice at hospitals and care facilities, students
in nursing and caregiver education also have few opportunities
to practice on actual patients and are limited to observing how
well-trained caregivers provide care to patients with poor oral
status, such as older adults with dementia.
In recent years, simulation education has been attracting
increasing attention as an effective exercise process thatallows students in nursing and caregiver education to learn
fundamental clinical care skills and procedures in the clinicalsetting before clinical practice in hospitals and care facilities.
Simulation education started in the US in the 1970s and has

been systematized as an essential part of the curriculum forstudents in medical, nursing, dental hygiene, and caregiver
education. In Japan, simulation education was introduced in
the 2000s as an effective process to cultivate the practicalabilities required of a specialist [1].
Focusing on the insufficient educational environment ineducational facilities, as well as the effectiveness of
simulation education, we attempted to develop oral caresimulation models suitable for an education program. We thendeveloped a prototype mealtime assistance simulation model[2] consisting of denture and tongue models with a sensor
system to detect spoon movement. Next, we evaluated thesensor system in spoon detection experiments and devised aspoon navigation algorithm that enabled students to move thespoon to the target position and apply a pressing force tostimulate the hypoglossal nerve and provoke oral closure.
The present study aimed to develop a prototype simulation
model for mealtime assistance. This model has an improved
appearance closer to an actual human head accompanied by ajaw joint driven by a servo motor. Based on fundamental
experiments and previous work, we also developed a spoon
navigation algorithm on the tongue model as well as a spoon
movement navigation application, the interface and functionof which are introduced in this paper.
II. RELATED WORKS
Several simulation models and related research topics canbe found in the simulation education area. However, examples
such as a foot gait simulator [3] and a defecation simulator [4]were not designed for simulation education in nursing school.
SimMan (Laerdal Medical) [5] is a robot patient simulator thatsimulates various body parts, but was not designed for use inoral care education. Other examples used in Japan include
Seiketsu-Kun (Kyoto Kagaku Co.), a commercial oral caresimulation model [6], DR-H Ni:Mo (Nissin), a practicephantom with a denture model [7] targeted for use in dental
hygiene school education, and Manabot (Nissin), an oral
function management training tool [8] consisting of an oralcavity with denture and tongue models that replicates an older
adult. These models can be used to practice some oral care
procedures, but not mealtime assistance.
Some research topics have been targeted for simulation
education in nursing school. A robot simulator [9] has beenused to simulate the oral care of bedridden patients, but not for
mealtime assistance. Although case studies of simulation
education using SimMan have been reported in nursing andmedicine [10,11], no reports regarding oral care simulation
education have been published. Other simulators, such as anarrhythmia simulator [12] and a remote oral diagnosis system[13], were not designed for oral care simulation.
Information technology has been integrated into
simulation education and several research topics have been
explored, including a computer-aided auscultation learning
system [14], clinical prostate training [15], a haptic feedback
system for nasotracheal suctioning [16], a virtual reality
auscultation learning system [17], and a dental patient robot
for dental students [18]. However, none of these systems coveroral care or mealtime assistance.
Therefore, the present study aimed to develop an
appropriate oral care simulation model including mealtime
assistance by applying robot and information technologies.
III. REQUIREMENT TO DETECT SPOON MOVEMENT IN
MEALTIME ASSISTANCE
Mealtime assistance consists of all processes that can help
older adults who cannot eat independently, such as oraltraining, massaging the tongue, wetting the oral mucosa,
getting into a proper seating posture at a table or while in awheelchair, and oral care with a toothbrush or sponge brush
after eating. In the case of mealtime assistance using a spoon,
the caregiver must move the spoon so that it stimulates nerves
such as the hypoglossal nerve (left side of Figure 1) andtrifacial nerve to not only place foods on the tongue, but also
improve swallowing function (right side of Figure 1). Thus,
the mealtime assistance simulator developed in the present
study was designed to be able to evaluate whether the spoon
would have stimulated the target nerves correctly [19,20]. To
realize this, sensors for detecting spoon movement were
installed in the tongue and the upper lip parts of the simulator.
In the present work, STEP 1 can be performed as shown in
Figure 1, but STEP 2 will be implemented in the future.
Therefore, the present simulation model has the following
functions:
1. Detects spoon movement with sensors embedded in
the tongue modelUnits
2. Closes the mouth via a servo motor-driven jaw
joint model

Figure 2. Prototype model And System flow.

V. SPOON MOVEMENT DETECTION SENSOR SYSTEM FOR
MEALTIME ASSISTANCE
5.1. Development of the Prototype of Tongue Model
Figure 3(a) shows our newly developed sensor board to
detect spoon movement. In our previous work, the sensor
board for the tongue model consisted of four sensors with two
types: a force sensor to detect the pressing force on the centerof the tongue, and three photo reflectors (PRs) to estimate thepressing position by detecting deformation in the tongue
model [2]. In the present work, three reflection sensors wereused to estimate both the pressing force and position. Figure3(b) shows a molded tongue model covered with a softurethane gel designed to approximate the feeling of humantissue, called “Hitohada Gel”, product of Exseal Corporation.
VI. FUNDAMENTAL EXPERIMENTS FOR SPOON MOVEMENT
DETECTION AND NAVIGATION
6.1 Spoon Identification for Mealtime Assistance
Figure 3 shows a typical spoon for mealtime assistance.
Based on previous work [2], we defined the target values asfollows:
1. Detects spoon movement with sensors embedded in
the tongue modelUnits
2.

Figure 1. Spoon movement for mealtime assistance: STEP 1 can be
performed as shown, whereas STEP 2 is still under development.

IV. MEALTIME ASSISTANCE SIMULATION MODEL
With reference to the operating mechanism of the jaw joint,a humanoid upper body model (above the shoulder) was
newly designed using three-dimensional (3D) computer-aided
design software and output using a 3D printer. The chassis had
an exterior part and a denture model part with a jaw joint. The
model is controlled by a PC and microcomputer (Figure 2).

Pressing force: 100 –150 gf

6.2 Experiments for the Tongue Model
Based on the tongue coordinate system Ot-XtYt shown in
Figure 3(b), we considered the relationship between thepressing force and sensor signals from the three PRs. In
Figure 3(b), the origin Ot indicates the position of the
hypoglossal nerve and the coordinates of each PR are thesame as those in our previous work [2], denoted as follows(unit: mm): pr1 = (–7.5, 10), pr2 = (7.5, 10), and pr3 = (0, –7.5).
We prepared an analog-to-digital converter (AIO160802AY-USB; Contec Inc., Osaka, Japan) to record output
signals from the PRs via a USB interface. The pressing force
was imposed using a weight on a spoon Figure 3(c).
Figure 3(c) shows the experimental results for spoonpressing forces of 0, 50, 100, 150, and 200 gf on the origin
Ot. The results indicate that the sensor outputs from each PR
increased with increases in the pressing force from 0 to 200

gf. Thus, the pressing force can be estimated as follows fromthe output of each sensor.
Smaller: pr1<2.7, pr2 <2.8, and pr3 <2.17
Appropriate: 2.7 < pr1 < 2.85, 2.85 < pr2 <3.27, and 2.17 < pr3 < 2.28
Larger: pr1 > 2.8, pr2 > 3.27, and pr3 > 2.28

Next, we considered the estimation of pressing position
from the sensor signals. Based on previous work, the signal
outputs from each PR are the largest when the spoon presses
just above each sensor position, and become smaller as the
distance from the sensor to the pressing position increases.
Therefore, the spoon can be positioned close to the origin Ot
along both the Xt and Yt axes. Figure3(a) shows the sensor
signal trajectory of each PR when the spoon moves fromnegative to positive on the Yt axis with a pressing force of 100
gf. As shown in the figure, pr1 and pr2 peaked at y = 10,
perpendicular to the positions of pr1 and pr2, and pr3 peakedat y = –7.5. This means that the appropriate spoon position canbe indicated using the same algorithm as that in our previous
work [2].

Phase 1.
Move the spoon into the oral cavity andplace it on the center of the tongue.
Phase 2.
Maintain the position of the spoon andapply an appropriate force.
Phase 3.
Remove the spoon from the oral cavity
while pressing against the upper lip.
To realize this spoon navigation procedure, we developed asoftware-based navigation algorithm.
7.3. Validity of the Navigation Algorithm in Phase 1
The value P0 is transferred from the tongue sensor board
value, while P1 is transferred from the sensor detecting theappropriate applied force value. The relationship between P0
and P1 is described in the following equation, with P_offset
defined as the allocated silicon-reacted value:
P1 = P0 / P_offset

F0 is defined as the normal condition when the spoon does
not apply any force on the tongue. In the case of F0, the rangeof P1 can be described as follows:
0.8 < P1 < 1.0f

By using the F0 condition equation, the software application
can estimate in which area the spoon is located in the position
chart. F1, F2, and F3 in the conditional expressions are thevalues of pr1, pr2, and pr3 on the sensor board.
A judgment is then made based on one of the following
equations when the spoon is placed on the tongue sensor. If
the spoon is not placed in the appropriate center position, theuser is urged to move the spoon to the correct position. Figure
4 shows the navigation results when the spoon is located ineach area. All navigation arrows indicate the direction to Area
5.
If the spoon is not navigated appropriately, this indicatesthat the spoon should be moved to the proper center position.
Figure 3. Experiments for the Tongue Model.

VII. DEVELOPMENT OF THE SPOON NAVIGATIONAPPLICATION
7.1. Application Interface
Based on the studies described earlier, we developed an
application to enable the user to learn how to move the spoon
appropriately. This application records the sensor output
values sent from the sensors mounted on the simulation
model and visualizes the spoon movements based on thesevalues.
7.2. Software-Based Navigation Algorithm for Spoon
Movement
Based on the details described in Section 3, the procedurenecessary for mealtime assistance can be defined ascomprising the following phases:

Left position

Area 1 = F1 £ 1.0f & F2 ³ 1.2f & F3 ³ 1.2f

Right position

Area 2 = F1 ³ 1.2f & F2 £ 1.0f & F3 ³ 1.2f

Front position

Area 3 = F1 ³ 1.2f & F2 ³ 1.2f & F3 £ 1.0f

Back position

Area 4 = F1 £ 1.0f & F2 £ 1.0f & F3 ³ 1.2f

Center position

Area 5 = F1 ³ 1.2f & F2 ³ 1.2f & F3 ³ 1.2f

7.5. Algorithm to Control the Timing of the Jaw Joint
Three movements were set up to move the patient’s jaw joint
during dietary care. A servo motor was used for the
movement of the jaw joint. The angle of the servo motor canbe changed according to the navigation application, allowing
the dietary movements to be controlled.
By performing the movements of the jaw joint inconjunction with the 3D model of the oral cavity in thesoftware, the system makes it possible for the user tounderstand the timing of dietary care and the flow of
movements (e.g., mastication) in a way that corroborates withboth actual and simulated learning environments.

Figure 5. Opening and closing of the jaw

VIII.PRELIMINARY EXPERIMENTS

Figure 4. Position chart for the navigation application and Navigation resultsin each area.

7.4. Algorithm for Simulation of the Spoon Movement
The three movements listed in Section 6.2 were
implemented using the following equations. The value of the
movement to press the spoon on the indicated tongue position
is denoted as S1. The time of the movement to press the spoonon the indicated tongue position is denoted as t1, and thedesignated time as t2. The movement to fix the spoon at thecurrent position and keep pressing it until the indicated timeis denoted as S2. If the value of the upper lip sensor is denoted
as P3 and the movement to remove the spoon from the oralcavity with the spoon is fixed as S3, the equations are as
follows:
1: S1 = Area 5
2: S2 = S1 & t1 > t2
3: S3 = F0 & P3 > 1.0f

When the user presses the spoon on the correct position or
maintains a fixed position, the navigation indicates that theuser should continue such correct movements by showing thedesignated time on the screen. In addition, if the user
performs the procedure correctly and moves to the nextscreen, the application displays the term “Correct”; if the user
does not perform the procedure correctly, the applicationurges the user to follow the correct movements by providing
instruction on how to apply force properly, etc.

Before the experiments were conducted, we obtained theapproval of the ethics committee of Sapporo City University(No. 2022-01). However, under the circumstances producedby the coronavirus disease 2019 pandemic, we had difficultyfinding agreement between the experiments in nursing school
and care facilities, and we could not obtain a sufficientnumber of examinees to analyze the simulation model
statistically. Therefore, this section provides the following
comments from six experts who were working in the
rehabilitation section of a hospital:“Visualization is an essential function of the simulator.”“The simulation model showed reactions according to spoon
movements.”“Patients sometimes refuse to open their mouth, and some havetongue paralysis or cancer, or missing teeth and dentures. The
simulation model should include functions appropriate for suchclinical situations.”

IX. DISCUSSION
Based on the comments in Section 8 obtained after
conducting examinee experiments targeted for students, thesimulation model appears to be adequate for learning
fundamental mealtime assistance skills and procedures using
a spoon. On the other hand, further improvements are needed
to improve its effectiveness. Although the present oral cavity
model is similar to a healthy actual one, a certain proportionof patients are expected to have some disorders. Therefore,various types of oral cavities, such as those with tongue
paralysis, tongue cancer, choking, and tooth pain, which
inhibit chewing and swallowing function, should be
developed to replicate clinical situations more accurately.

X. CONCLUSION
In the present study, we developed and described the detailsof a prototype simulation model for mealtime assistance fornursing students and caregivers, including an upper lip modeland improved tongue model, a spoon navigation application,
and a navigation algorithm. To assess the developed
simulation model, we conducted experiments targeting
caregiving experts. The results suggested that the simulator iseffective for teaching fundamental mealtime assistance skills
and procedures.
In the future, we plan to conduct further examinee
experiments targeting students and experts in various
facilities to assess the simulation model. We also plan to
improve the simulation model and its application based on theadvice and comments obtained in the experiments, and todevelop a mealtime assistance simulation education program.
Acknowledgements: This work was supported in part by
JSPS KAKENHI (18H03114) and the Collaborative ResearchFund, Sapporo City University, Japan.
Acknowledgements:
[1]

[2]

[3]

[4]

[5]
[6]

[7]

[8]

Ministry of Health, Labour and Welfare. [Report of a review meetingon the contents and methods of nursing education] (in Japanese), Feb.
28, 2011. https://www.mhlw.go.jp/stf/shingi/2r98520000013l6yatt/2r98520000013lbh.pdf (accessed July 7, 2023)
Mitani, A.; Muramatsu, M. Development of human tongue model for
mealtime assistant training using oral care simulation model.
International Journal of Automation Technology 2019, 13, 499–505.
Hashimoto, T.; Takakura, Y.; Hamada, T.; Akazawa, T.; Yamamoto,
M. Development of a foot gait simulator for presenting environment toeach user. Journal of Advanced Computational Intelligence andIntelligent Informatics 2011, 15, 554–562.
Tokoro, K.; Hashimoto, T.; Kobayashi, H. Development of a robotic
defecation simulator. Journal of Robotics and Mechatronics 2014, 26,
377–387.
SimMan. https://www.laerdal.com/us/doc/86/SimMan (accessed May
11, 2023).
Oral Care Simulator Seiketsu-Kun.
https://www.kyotokagaku.com/en/products_data/m89 (accessed May
11, 2023)
NISSIM Dental Simulator Phantom. http://www.nissindental.net/products/DentalTrainingProducts/DentalSimulator/nissim2/
index.html (accessed May 11, 2023).
MANABOT. http://www.nissindental.net/products/DentalTrainingProducts/DentalSimulator/MANA
BOT/index.html (accessed May 11, 2023).

[9]

Takahashi, Y.; Yatsumonji, T. Going out experience robot for
bedridden people by remote control system. Procs. 39th SICE Annual
Conference International Session Papers (IEEE Cat. No.00TH8545),
28 July 2000, Iizuka, Japan, pp. 175–178.
[10] Changping, S. Application of SimMan universal patient simulator in
the teaching of medical nursing. Procs. 2011 International Conferenceon Human Health and Biomedical Engineering, 19–22 Aug 2011,
Jilin, China, pp. 1172–1174.
[11] Torregrosa, T.; et al. Fabrication of a nursing manikin overlay for
simulation of chest drainage management. Procs. 2015 IEEE Great
Lakes Biomedical Conference (GLBC), 14–17 May 2015,
Milwaukee, WI, USA, pp. 1–4.
[12] Clavario, P.; et al. Use of an arrhythmia simulator for the evaluationof nurses’ knowledge and as a teaching tool. Procs. 1995 Computers
in Cardiology, 10–13 Sept 1995, Vienna, Austria, pp. 621–624.
[13] Xiao-Lin Lu, System design and development for a CSCW based
remote oral medical diagnosis system, Procs. 2005 International
Conference on Machine Learning and Cybernetics, Vol. 6, 36983703, 2005.
[14] Hou, C.J.; et al. Computer-aided auscultation learning system fornursing technique instruction. Procs. 30th Annual International
Conference of the IEEE Engineering in Medicine and Biology
Society, 20–25 Aug 2008, Vancouver, BC, Canada, pp. 1575–1578.
[15] Gerling, G.J.; et al. The design and evaluation of a computerized andphysical simulator for training clinical prostate exams. IEEE
Transactions on Systems, Man, and Cybernetics - Part A: Systems
and Humans 2009, 39, 388–403.
[16] Mudura, L.; et al. A virtual nursing simulator with haptic feedback fornasotracheal suctioning. Procs. 2006 IEEE/RSJ International
Conference on Intelligent Robots and Systems, 9–15 Oct 2006,
Beijing, China, pp. 698–703.
[17] Chiba, M.; Hamamoto, K. Development of a learning support systemon auscultation for nurses using virtual reality. Procs. 2018 2ndInternational Conference on Biomedical Engineering (IBIOMED), 24–
26 July 2018, Kuka, Indonesia, pp. 34–38.
[18] Takanobu, H., et al. Dental patient robot as a mechanical humansimulator. Procs. 2007 IEEE International Conference on Mechatronics,
Jilin, China, 2007, pp. 1–6. DOI: 10.1109/ICMECH.2007.428[19] Florie, M.G.M.H.; Pilz, W.; et al. The effect of cranial nerve
stimulation on swallowing: a systematic review. Dysphagia 2021, 36,
216–230. DOI: 0.1007/s00455-020-10126-x[20] Tanaka, Y. [Spoon technique that brings out oral function. Dementia
Care Cases Journal 2015, 8(2), 135–143. (in Japanese)