# Virtual Co-Eating: Making Solitary Eating Experience More Enjoyable

Monami Takahashi, Hiroki Tanaka, Hayato Yamana, Tatsuo Nakajima

► To cite this version:

Monami Takahashi, Hiroki Tanaka, Hayato Yamana, Tatsuo Nakajima. Virtual Co-Eating: Making Solitary Eating Experience More Enjoyable. 16th International Conference on Entertainment Computing (ICEC), Sep 2017, Tsukuba City, Japan. pp.460-464, 10.1007/978-3-319-66715-7_63 hal-01771257

HAL Id: hal-01771257

https://inria.hal.science/hal-01771257

Submitted on 19 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Virtual Co-Eating: Making Solitary Eating Experience More Enjoyable

Monami Takahashi¹, Hiroki Tanaka¹, Hayato Yamana¹, Tatsuo Nakajima¹

¹Department of Computer Science and Engineering, Waseda University, Tokyo
{m.takahashi, tatsuo}@dcl.cs.waseda.ac.jp, {mizurin, yamana}@yama.info.waseda.ac.jp

**Abstract.** Recently, a research on eating habits of Japanese college students revealed that they have a highly desire to communicate with others through co-eating. Even though better eating experience through co-eating is important, they often tend to be alone even more because of some reasons like small households, living alone, and having no time to find others for co-eating. Therefore, we believe that it may improve eating experience by incorporating a fictional character into the real space as a partner to eat together. For validating the idea, we have developed a *virtual co-eating* system for solving issues caused from solitary eating, and show some insights from its user study.

**Keywords:** Co-EatingSolitary · Eating · Augmented Reality

## 1 Introduction

Recently, Japanese college students tend to eat alone even more because of some reasons like small households, living alone and having no time to find others for co-eating. However, they have a highly desire to communicate with others through co-eating [1]. Co-eating gives them better eating experience that cannot gain from solitary eating, and recent researches showed that good eating experience through co-eating closely related to positive emotion such as ‘enjoyable’, ‘family’, ‘talking’, ‘together’, ‘friend’, and ‘smile’ [1]. Especially, ‘family’ and ‘talking’ are strongly related to good experience [2]. Therefore, we believe that a partner to eat together is essential as a factor for improving solitary eating experience.

Typically, good eating experience is comprehensively obtained by combining not only the taste of food but also the environment, memory and experience such as a condition, feelings and atmosphere while eating. Therefore, there are several investigations on whether it is possible to improve eating experience by changing a variety of information other than the taste of food. For example, in a past study to report a system that enhances eating experience, a feeling of a full stomach is maintained due to changing the meal amount, if its appearance size is not changed by an augmented reality technology [3].

In this study, we introduce an alternative approach named *virtual co-eating* to improve eating experience by allowing a user to talk with a virtual partner and dissolve his/her dull eating experience for college students who frequently eat food alone.
## 2 Virtual Co-Eating System

In order to improve eating experience, the virtual co-eating system is designed by focusing on the two factors: "*a preferable eating partner*" and "*enjoyable conversation*". For considering the first factor; we adopted a 3D virtual character named Kurei Kei (Pro-Nama-Chan, https://github.com/kureikei), who is typically preferable from the main target users of the virtual co-eating system who are Japanese computer science course students as a virtual partner because the character is very popular in young Japanese information technology communities. This character can easily guide to an enjoyable conversation with various common topics with the main target users. Also, in order to increase a feeling of the reality on the character, we decided to use the mirroring effect, which is the effect of having favor with others who do the same actions as a user. As shown in Fig. 1 (a), for example, in the virtual co-eating system, the user's smile is detected while talking with the virtual partner, and the partner also smiles as a feedback.

<figure><img src="image_2.png" /><figcaption>Fig. 1. An Overview of Virtual Co-Eating System.</figcaption></figure>

For considering the second factor: enjoyable conversation, the virtual partner answers appropriate responses when a user speaks to her as shown in Fig. 1 (b) (c). On the other hand, when the user is silent, the partner asks the user a question. The partner talks three topics; common topics with users, topics related to the meal, and typical daily conversation. Also, in order to direct the atmosphere to a pleasant dining table and to guide a user to an enjoyable conversation experience, the system offers a background music taken from a video game for making eating atmosphere calm.

In the virtual co-eating system, the partner's behavior is generated in software that can create 3D character's motions called MMD. In addition, the voice of the partner is
generated by a voice synthesizing software using 500 kinds of voice materials from
actual voice actors. The character's 3D model, motion materials and voice materials are
loaded into Unity, which is a game development platform. We wrote C# scripts and to
control the playback timing of the partner's motion, expressions and voice.

If the system requires to wear a wearable device such as a head-mounted display, it
makes a user hard to eat and to see, then it makes difficult to improve eating experience.
Therefore, the system uses a projection technique to present the virtual partner in the
real space. The screen of the mesh fabric is adopted for the projection to make the
partner to be more three-dimensional in the real space as shown in Fig. 2.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Projecting the Virtual Partner</figcaption></figure>

In addition, facial expression recognition is performed with OpenCV written in Py-
thon, and the recognition results can be transmitted to Unity through WSocket. By
using OpenCV's haar-like feature classifier in two stages, we cut out the face, then we
implemented the smile recognition function by detecting the feature quantity of the
smile from among the cut out faces. Finally, the system controls the behavior of the
partner by sending the recognition results to Unity through WSocket.

3 Preliminary User Study

We conducted an experiment to investigate whether it is possible to improve eating experience by changing solitary eating experience to co-eating experience through the dialogue between participants who join to the experiment and the virtual partner. The main purpose of the user study is to find potential pitfalls of the current design not to validate the effectiveness of the design. This experiment was carried out by five participants; 20's male Japanese computer science course students who like animation and game content characters and who are usually eating alone. In the experiment, each participant first silently eats white rice balls with no good taste alone. Then, he uses this system to have the conversation with the virtual partner and eats the same rice balls.
Finally, we analyzed the evaluation of each factors designed in the system through the questionnaires and the interview with him.

From the results of the questionnaires, when comparing with the eating experience to eat food alone and with the virtual partner, 4 out of 5 participants answered that their eating experience was improved. From the interviews, three participants said "I felt it was more delicious than when I was eating alone." In addition, a participant claimed "It were not tasty because I were hungry at the time when eating alone.", and also commented "I did not matter for me.." when eating with the system. On the other hand, some participants who did not change eating experience with the system answered "Because I were talking with the partner, I did not pay an attention to meal." Thus, it is essential how the system supports the conversation with the partner.

More than half participants answered "Whichever" for the background music and talking common topics in the conversation with the virtual partner. Regarding common topics, in particular, three participants commented "It is better to have a daily conversation than to talk about programming as common topics during eating." In addition, one participant commented "I want to share feelings of delicious food through conversation."

Regarding a background music, one participant answered "There is no background music in my actual daily life.", and two participants answered "I did not feel loneliness when there is a background music." On the other hand, two participants answered, "It would be better to use environmental sounds such as talking sounds and tableware sounds, not a background music from a video game."

Regarding a smile feedback, four participants answered "I felt that my partner was smiling if I laughed, then I became more smiling." Thus, this approach seems that participants were able to make the virtual partner feel more preferable. On the other hand, one participant commented "It looked she was always laughing."

## 4 Conclusion

In this study, we developed a virtual co-eating system to improve eating experience. The system provides better eating experience through a conversation with an empathetic virtual partner. The current system focuses on improving eating experience of 20's Japanese computer science course students. We conducted an experiment to extract pitfalls of the current design, and extracted some insights for future improvement of the current prototype system.

## References

1. Nakagawa, M., Nagatsuka, M., 2010. *The function and the possibility of "Kyoshoku" (eating together)*, Journal of HortResearch, Vol. 64, pp. 55-65.

2. Iitsuka. Y., 2014. Analysis of the Psychological and Behavioural Pattern in Eating with Others and Eating Alone I, Shimane University.

3. Narumi, T., Ban, Y., Kajinami, T., Tanikawa, T., Hirose, M., 2012, *Augmented perception of satiety: controlling food consumption by changing apparent size of food with augmented reality*, In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 109-118.