# Artificial Intelligence and Entertainment Science Workshop: Towards Empathic Entertainment Technology

Mohd Khalid, Hiroyuki Iida, Umi Kalsom Yusof, Ruzinoor Che Mat

► To cite this version:

Mohd Khalid, Hiroyuki Iida, Umi Kalsom Yusof, Ruzinoor Che Mat. Artificial Intelligence and Entertainment Science Workshop: Towards Empathic Entertainment Technology. 20th International Conference on Entertainment Computing (ICEC), Nov 2021, Coimbra, Portugal. pp.475-481, 10.1007/978-3-030-89394-1_41. hal-04144364

HAL Id: hal-04144364

https://inria.hal.science/hal-04144364

Submitted on 28 Jun 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Artificial Intelligence and Entertainment Science
Workshop: Towards Empathic Entertainment
Technology

Mohd Nor Akmal Khalid<fnref n="1" />, Hiroyuki Iida<fnref n="1" />,
Umi Kalsom Yusof<fnref n="2" />, and Ruzinoor Che Mat<fnref n="3" />

**Abstract.** The intersection of artificial intelligence and entertainment science via games had led to fruitful development in both fields. Empathic technology in the entertainment medium is a promising trend that bridges both worlds. This workshop provides the discussion space for those interested in its application, societal issues, risks, and best practices.

**Keywords:** Artificial intelligence · Entertainment · Technology · Empathy

## 1 Background

Billions of people have been enlightening with the joy of the entertainment industry via movies, games, books, and music. Today's production and marketing budgets for entertainment products often exceed $100 million and can reach up to $500 million for a single new movie or video game [5]. *Entertainment Science* built on the assumption of the modern world with almost unlimited data and great computational resources, the combination of intelligent analytics and powerful theories would provide valuable insights in supporting decision making.

The notion of *robust AI* had been advocated, which involves applying its knowledge to a wide range of problems systematically and reliably, synthesizing knowledge from a variety of sources such that it can reason flexibly and dynamically about the world, able to transfer what it learns from one context to another and build the sense of trust [13]. An overview of the game design and study of human-based phenomena that induce emotional reactions and people's motivation in digital games had been explored by [12]. Similarly, emotionally sentient systems would also enable AI agents to perform complex tasks more effectively, making better decisions, and offered more practical and effective services [14].

<footnote n="1">¹ School of Information Science, JAIST, Ishikawa, Japan
{akmal,iida}@jaist.ac.jp</footnote>

<footnote n="2">² School of Computer Sciences, University of Science Malaysia, Penang, Malaysia
umiyusof@usm.my</footnote>

<footnote n="3">³ School of Creative Industry Management and Performing Arts (SCIMPA), Universiti Utara Malaysia (UUM), Kedah, Malaysia
ruzinoor@uum.edu.my</footnote>
With the recent rise of artificial emotional intelligence, empathy becomes the new buzzword term. With the establishment of *Empathic Computing* framework [3], a machine can recognize human feelings and states, understand user intents, and respond to user needs dynamically [22]. Such a framework becomes the enabler for prominent applications such as the empathic conversations in healthcare [7], social [22], and both [4]. Another study utilized such frameworks in mediating judgment, explored the effect of metaphors on evaluating AI agents while managing expectation and perception [9]. In addition, combinations of sympathetic actions and explanation types would assist AI agents to adapt in multiple human-computer interaction scenarios [8].

Based on the advancements outlined by the previous works, empathic technology is currently a nascent field. The general trend focuses on the development of emotional AI agents and technologies surrounding it [14], describing the AI performance and its implications to interaction design [15], realizing computational creativity and humor [20], and bridging physical and virtual sensations and interactions [11,2]. However, empathic computing was adopted alongside AI technology while its influence and interactions, physically, emotionally, or virtually, remained unexplored from the entertainment science perspective. As such, empathic technology from an entertainment point-of-view could potentially produce robust, humorous, explainable, and perceptive AI systems. Such niche is the primary motivation of this workshop, which could observe and bring a heightened experience of interactions, collaborations, and co-creations, in human-machine relations and beyond.

## 2 Workshop Objective

Artificial intelligence (AI) had enabled the establishment of a platform for the development of intelligent search techniques [19] and general language description [16] in competitive settings. However, such research studies emphasize functional utility and performance optimization. In contrast, the motion in mind concept [6] synthesized the foundation of entertainment science by considering the analogy of physics and motions with the game-playing process by aggregating value functions and reward systems [21,10]. Nevertheless, such a concept lacks experimental validation and empirical verification because of its subjective properties.

A recent study showed that AI search applied in a game environment provided the utility to identify entertaining elements as a new form of scientific venture [17]. Bringing together AI and entertainment science, establishing common interests is vital to understand better how those fields interact. In particular, in what ways does AI becomes entertaining? How can entertainment benefit from AI applications? Nevertheless, such questions address one side of the equation. More importantly, how can both fields come together as beneficial to humanity? In such a situation, having a common interest that resonates in both fields requires understanding the needs and challenges faced by the experts of the fields and the users.
Dale Carnegie argued that a person's success is impacted more by the ability to "deal with people" than by professional knowledge, as outlined in his classic book *How to Win Friends and Influence People* [1]. However, such "people skills" are not easy to come by. For many, it takes years of practice and learning to get along with others, especially when the people's interests are not aligned with ourselves [15]. This situation describes the notion of "empathy," and its importance in human interactions, which is an essential component in the coming age of *artificial emotional intelligence* [18]. In essence, this workshop would be interested in technology-related development and encourages multi-disciplinary efforts while being neither purely technical driven nor purely qualitative research.

This workshop aims to bring together researchers, designers, and practitioners under the umbrella of the workshop organized alongside the 20th International Federation for Information Processing–International Conference on Entertainment Computing (IFIP-ICEC 2021). The workshop aimed to establish a panel of discussion at the intersections of AI and entertainment science areas by utilizing entertainment computing activities such as video games, digital arts, or film media. The niche area of empathic entertainment, which focuses on intertwining empathic computing and entertainment science, provides a unique approach to humanizing AI applications. A novel entertainment-based empathic technology can be developed and established, which is centralized on the notion of empathy and how it influences technological advancements that benefit people and their surrounding community. Best-practice experience and successful initiatives can be identified, while empirical outcomes and novel designs can be determined to build up and strengthen the community of interest in empathic entertainment technology. In addition to building an international community, the workshop aims to identify challenges and opportunities related to empathic entertainment technology in games or non-game contexts.

## 3 Deliverable and Organization

Participants will be asked to submit a two (2) pages extended abstract to present their work, and the organizers will invite them to present such works as a short or poster presentation. An extended version of their research works will be considered for publication in a special issue journal. The materials and results of the workshop will be made available to all participants. Other intended results are the formation of networks for project proposals or further common activities. As a follow-up, the workshop organizers will summarize the workshop's outcomes in a paper, preferably published in the *IFIP-ICEC 2021* or the *Entertainment Computing* journal. We also want to collect relevant literature and successful initiatives to be shared on the workshop's website. Follow-up activities also include joint research activities and subsequent publications.

The profiles of the expected participants were from the background of computer science, engineering, and technology-related fields. A minimum number of 10 participants and a maximum of 20 participants will be expected. The scope of the topics include, but are not limited to:
1. **Theoretical contributions** that lead to or deliver empathic entertainment.

2. **Presentation and experience** of empathic AI agent and empathic simulation.

3. **Perception and acceptance** related to empathic experience and its entertainment context.

4. **Specific aspects** of human-AI interactions and empathic play in games or non-game context.

5. **Examples** of entertainment medium, designed and developed for better empathic experience.

6. **Examples** of empathic game design or processes that integrate empathetic design.

7. **Examples** of empathic AI-based support tools for creating an entertaining experience.

## 4 Program Structure and Important Dates

The workshop will be organized as a **full day session** (about **8 hours**). The program includes the keynote speech from two invited speakers and an expected 10 participants with short presentations (each given about 10 minutes presentation and 5 minutes question and answering session). The program starts with an opening speech by the organizers (about 15-30 minutes), followed by the first keynote speech (about 1 hour and 15 minutes). Then, the participant's presentation will be conducted in two sessions (totaled into two and a half hours) and finally followed by the second keynote speech (about 1 hour and 15 minutes) before summarizing the program and ending the workshop. In the event of more participants, parallel sessions will be conducted for the participants. Meanwhile, in the event of fewer participants, plenary speakers will potentially be conducted by one or more program committee members. More information is outlined in https://aies.info/.

**Submission deadline:** 30th September 2021

**Notifications of acceptance:** 7th October 2021

**Workshop program:** 2nd November 2021

## 5 Organizers

Mohd Nor Akmal Khalid is an assistant professor in the School of Information Science at the Japan Advanced Institute of Science and Technology (JAIST) and a part-time Research Fellow in the School of Computer Sciences, University of Science Malaysia (USM), Malaysia. His work focuses specifically on methods and developments in the fields of entertainment technology and operational research. His topics of interest include but are not limited to manufacturing systems, advanced scheduling and planning, artificial intelligence techniques, game analytic and informatics, search algorithms, bio-inspired optimization techniques, and machine learning methods. He was also involved in organizing scientific, social
activities such as research writing workshops, student colloquiums, and academic
talks.

Hiroyuki Iida is a Japanese computer scientist and computer game researcher focusing on game refinement theory, opponent-model search, and computer shogi. Hiroyuki Iida is the Trustee and Vice President for educational and student affairs at the Japan Advanced Institute of Science and Technology (JAIST), Director of the Global Communication Center, and head of the Iida laboratory. He is a member of the ICGA as Secretary-Treasurer and a Section Editor of the ICGA Journal. Previously, he was affiliated with Shizuoka University, Hamamatsu, and was a guest researcher at Maastricht University. He is a professional 7-dan shogi player, coauthor of the shogi program Tacos, and a four-time gold medal winner at the ICGA Computer Olympiad. He also had the pleasure of becoming the organizer of several International Computer Games Association (ICGA) in 2002, 2010, and 2013, and the program committee of 2021 International Joint Conferences on Artificial Intelligence (IJCAI), as well as the founder of the International Conference of Entertainment Computing (ICEC). His research interests include artificial intelligence, game informatics, game theory, mathematical models, search algorithms, game refinement theory, game tree search, and entertainment science.

Umi Kalsom Yusof currently an Associate Professor and a Senior Lecturer with the School of Computer Sciences, USM. She has previously worked in Petronas, Toyota, ASE Electronics, and Motorola before joining the academia in 2008. She also co-organized several local conferences from 2007 to 2011, and international conferences from 2013 to 2015, while actively involved as a reviewer in local and international journals. Her research interests are related to artificial intelligence, machine learning, computational intelligence, multi-objective optimization, evolutionary computing, Web engineering, manufacturing optimization, crowd behavior in an emergency evacuation, and health-related and global warming effect studies. She has published research articles in national and international journals, conference proceedings, and book chapters.

Ruzinoor Che Mat is an Associate Professor at the School of Creative Industry Management and Performing Arts (SCIMPA), Universiti Utara Malaysia (UUM). He has developed numerous products on an online 3D system for agriculture, teaching and learning, mobile Augmented (AR), and Virtual Reality (VR). He was also actively involved in many research competitions at local and international which were awarded 23 medals. He has 18 years of experience in teaching and has taught about 30 subjects in various disciplines from four different faculties and invited to teach at Open University Malaysia (OUM) and Universiti Malaysia Sabah (UMS) as a part-time lecturer. He had also successfully procured funds, leading six research projects and 15 others as a team member. He is also an expert in 3D visualization and game engine, where he had published scientific papers in refereed journals, proceedings, academic books, and actively involved as a reviewer of several high impact journals such as Remote Sensing and GIS. His research interests are on 3D GIS, terrain visualization, game
engines, remote sensing, GIS application, Virtual Reality, Augmented Reality, Gamification, computer graphics, and visualization.

## References

1. Carnegie, D.: How to win friends & influence people. e-artnow (2017)

2. Dey, A., Piumsomboon, T., Lee, Y., Billinghurst, M.: Effects of sharing physiological states of players in a collaborative virtual reality gameplay. In: Proceedings of the 2017 CHI conference on human factors in computing systems. pp. 4045–4056 (2017)

3. Fung, P., Bertero, D., Wan, Y., Dey, A., Chan, R.H.Y., Siddique, F.B., Yang, Y., Wu, C.S., Lin, R.: Towards empathetic human-robot interactions. In: International Conference on Intelligent Text Processing and Computational Linguistics. pp. 173–193. Springer (2016)

4. Goel, R., Vashisht, S., Dhanda, A., Susan, S.: An empathetic conversational agent with attentional mechanism. In: 2021 International Conference on Computer Communication and Informatics (ICCCI). pp. 1–4. IEEE (2021)

5. Hennig-Thurau, T., Houston, M.B., et al.: Entertainment science

6. Iida, H., Khalid, M.N.A.: Using games to study law of motions in mind. IEEE Access 8, 138701–138709 (2020)

7. James, J., Balamurali, B., Watson, C.I., MacDonald, B.: Empathetic speech synthesis and testing for healthcare robots. International Journal of Social Robotics pp. 1–19 (2020)

8. Kampik, T., Nieves, J.C., Lindgren, H.: Explaining sympathetic actions of rational agents. In: International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems. pp. 59–76. Springer (2019)

9. Khadpe, P., Krishna, R., Fei-Fei, L., Hancock, J.T., Bernstein, M.S.: Conceptual metaphors impact perceptions of human-ai collaboration. Proceedings of the ACM on Human-Computer Interaction 4 (CSCW2), 1–26 (2020)

10. Khalid, M.N.A., Iida, H.: Objectivity and subjectivity in games: Understanding engagement and addiction mechanism. IEEE Access 9, 65187–65205 (2021). https://doi.org/10.1109/ACCESS.2021.3075954

11. Lee, Y., Masai, K., Kunze, K., Sugimoto, M., Billinghurst, M.: A remote collaboration system with empathy glasses. In: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). pp. 342–343. IEEE (2016)

12. Lopes, P., Boulic, R.: Towards designing games for experimental protocols investigating human-based phenomena. In: International Conference on the Foundations of Digital Games. pp. 1–11 (2020)

13. Marcus, G.: The next decade in ai: four steps towards robust artificial intelligence. arXiv preprint arXiv:2002.06177 (2020)

14. McDuff, D., Czerwinski, M.: Designing emotionally sentient agents. Communications of the ACM 61(12), 74–83 (2018)

15. Oudah, M., Rahwan, T., Crandall, T., Crandall, J.: How ai wins friends and influences people in repeated games with cheap talk. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 32 (2018)

16. Piette, É., Soemers, D.J.N.J., Stephenson, M., Sironi, C.F., Winands, M.H.M., Browne, C.: Ludii - the ludemic general game system. CoRR abs/1905.05013 (2019), http://arxiv.org/abs/1905.05013
17. Primanita, A., Khalid, M.N.A., Iida, H.: Computing games: Bridging the gap between search and entertainment. IEEE Access 9, 72087-72102 (2021). https://doi.org/10.1109/ACCESS.2021.3079356

18. Schuller, D., Schuller, B.W.: The age of artificial emotional intelligence. Computer 51(9), 38-46 (2018)

19. Song, Z., Iida, H., van den Herik, H.J.: Probability based proof number search. In: ICAART (2). pp. 661-668 (2019)

20. Veale, T.: Sympathetic magic in ai and the humanities. Journal of Artificial Intelligence Humanities 2, 9-38 (2018)

21. Xiaohan, K., Khalid, M.N.A., Iida, H.: Player satisfaction model and its implication to cultural change. IEEE Access 8, 184375-184382 (2020)

22. Zhou, L., Gao, J., Li, D., Shum, H.Y.: The design and implementation of xiaoice, an empathetic social chatbot. Computational Linguistics 46(1), 53-93 (2020)