Reviewerly/exHarmony huggingface dataset:
MRespond – An Innovative and Flexible MR Training System for First Responders

First responders are often lacking opportunities to train for dangerous situations such as big fires or chemical accidents, because these scenarios are often either too dangerous to simulate in real life or associated with high costs or personnel expenditure. The MRespond project addresses this challenge by providing a Mixed Reality (MR) solution that allows first responders such as fire fighters or paramedics to train hazardous situations in a multi-story building. Virtual hazards such as fire or chemical substances can be blended into the real world. The MR system offers free movement of trainees within a multi-story building and an outdoor area. First responders can train with their usual equipment. Objects and equipment such as doors, fire nozzles or emergency manikins are tracked and integrated into the system, so that the trainees can interact with the training scenario as they are used to. Real objects can interact vith virtual objects so that e.g., the fire propagation changes by opening doors and windows. Virtual injuries can be projected onto real emergency manikins so that trainees can train triage scenarios and still have the tactile feedback if they e.g., apply a bandage. Further, an exercise instructor interface provides the possibility to adapt the scenario in real-time by e.g., placing fires or changing the vital parameters of a patient. Thus, MRespond offers a highly adaptable training system.
Towards Co-design with Day Care Teachers Based on In-Situ Behavioral Data: A Case Study of a Workshop for Reflection Based on Video Recordings

In designing products and services that improve the quality of childcare, it is necessary for designers to empathize deeply with people involved in childcare (i.e., day care teachers) to understand context-dependent know-how. For that purpose, co-design, a collaborative design activity involving designers and non-designers, is essential. To conduct co-design, in-situ data can provide a common understanding for discussion. However, the types of data that are useful and how they can support co-design for childcare is unclear. Therefore, the objective of this study is to explore the potential usage of behavioral data in day care centers for co-design. We conducted a two-hour workshop with day care teachers. The results of the workshop showed that the video recordings effectively supported the teachers’ reflections. In addition, they also pointed out that video reflection can be supported by supplemental data such as children’s feelings and the context of the scenes. Overall, the results imply that behavioral data has huge potential to be used for co-design. The next step of this research is to clarify the issues and benefits of using behavioral data for co-design by conducting a workshop among day care teachers and designers.
Mapping XR Platforms: Analyzing Immersion from the Designer’s Perspective

Understanding humans are the key to developing optimal design solutions for product-service systems. In this sense, the experiential approach is in line but might go beyond typical Human Centered Design (HCD) methods in that it focuses on generating positive experiences that contribute directly to human well-being. Extended Reality (XR) showed the potential to replicate or simulate experience as a whole and gained attention from design communities. XR platforms confused design practitioners due to their fast-advancing amounts and relevant experiences. Hence, this study introduced two surveys on XR platforms to clarify which experiences they could provide and when to implement them into HCD processes. Survey 1 categorized XR platforms according to their key attributes and mapped them into the Experience Matrix. Survey 2 invented two designer personas and a fictional project to analyze barriers and strategies to implement XR platforms into design processes. Eighty-eight XR platforms were categorized into nineteen clusters, where creation and simulation had the highest numbers. Regarding implementing XR in design practices, the cost is still the key concern and there's a trade-off between software cost and assets purchased for different types of designers.
Incorporating the Dynamics of Climate Change into the Deep Dive Virtual Reality Underwater Site Prediction System

The procedural content of a virtual reality system is a key contributor to its success. However, it is often the case that the content needs to be adjusted. This may take place for a variety of reasons. This research investigates the possibility of the use of machine learning technology to facilitate the modification of a games content. Here, the Deep Dive system was designed originally to predict ancient site locations. Recently it was repurposed to be used as an educational tool to facilitate aspects of STEM education. This required the modification of the content to support this novel use. An evolutionary learning algorithm, Cultural Algorithm, is employed to facilitate the addition of the new content required for the educational application.
A Systematic Literature Review on Image Captioning

Image captioning is a technology that generates textual descriptions of images by integrating computer vision and natural language processing. This review aims to provide a comprehensive overview of current state-of-the-art image captioning models and techniques, summarize the critical approaches, evaluate their performance on benchmark datasets, highlight main challenges and limitations, and provide insights into future research directions. The systematic literature review followed a PRISMA-based approach and formulated three research questions (RQs) to construct the review. The study extracted and synthesized data from 49 eligible papers on visual and language models, datasets employed, and evaluation mechanisms used to measure performance. We found that it is evident that large-scale models and pre-training strategies have significantly advanced the field of image captioning. Specific sub-tasks for image captioning in certain applications have also contributed to the development of unique datasets. Future research directions in image captioning can be focused on solving the issues of expensive and scarce paired datasets, as well as shifting towards user-centric sub-tasks from generalization. This review can serve as a valuable resource for researchers, practitioners, and enthusiasts interested in image captioning.
Development of VR Education System for Media Exchange in Cell Culture Operation

In recent years, Japan has seen an increase in cases of fraudulent manufacturing of pharmaceutical products. Therefore, there is a growing need for education on Good Manufacturing Practice (GMP), which is the standard for manufacturing and quality control of pharmaceutical products. However, GMP education has problems, including a long training period and maintenance costs of facilities and equipment. For such issues, some challenges have already been made to conduct chemical experiments using HMD-type VR, and products have been developed that enable users to learn compliance rules for aseptic manipulation in VR. However, these systems only provide text feedback on compliance rules. Therefore, they are insufficient to improve the skills of beginners. In this study, we developed an educational system that provides feedback when violations are committed in cell culture operations and evaluated its effectiveness based on questionnaires and operation data. Therefore, it was suggested that this system has the potential to improve the skills of beginners.
Survey of Services that Store Passwords in a Recoverable Manner

Passwords entered by users in web services and applications are essential and confidential information. Therefore, it is ideal for difficulty storing them to decipher in case of unauthorized intrusion from the outside. As a typical example, passwords are converted into hash values using the SHA2 algorithm and stored. However, not all web services and applications implement the ideal storage method. There have been many incidents in which personal information has been leaked. In some cases, the passwords were not stored correctly on the server-side but in plain text or encrypted in a reversible form. The passwords were leaked when there was an unauthorized intrusion or other damage. This research aims to clarify the actual situation of how services and applications store users' passwords in plaintext or reversible form on the server-side through external observation surveys. The method is to list the survey targets for each service or application and conduct the survey for each service or application. As a result of the survey, there were no services or apps that were confirmed to have implemented inappropriate storage methods in both the top sites in the Alexa ranking and the top apps in the Google Play ranking, and the survey revealed that there were not many services that returned plain text in general.
Towards Effective Conversational Agents: A Prototype-Based Approach for Facilitating Their Evaluation and Improvement

Using conversational agents (CAs) has become increasingly popular for organizations for various applications, such as customer service and healthcare. However, user satisfaction and engagement with CA adoption stay behind expectations. In this context, quality criteria (e.g., regarding dialog flow or representation) can serve as a basis to evaluate CAs and improve their effectiveness to consequently affect user satisfaction and engagement. Hereby, we contribute to emerging research of quality criteria by proposing a prototype-based approach to facilitate the evaluation and improvement of CAs design and effectiveness. Our approach involves deriving scenarios from synthesized CA criteria from preliminary work, creating prototypes, and evaluating them in expert interviews. By comparing prototypes against each other, the influence on CA effectiveness can be measured. Our results demonstrate an approach for applying the criteria and offer a promising direction towards designing, developing, and operating CAs.
A Metamodel for Immersive Gaming Experiences in Virtual Reality

Due to the increasing adoption of specialized hardware, the development of novel gaming experiences in virtual reality (VR) has gained traction. Despite the popularity that VR has achieved so far, it is still not clear how to address the design of immersion, which is critical for providing meaningful, enjoyable, and engaging experiences in VR video games. Aiming to bridge this gap, in this paper we reviewed the recent literature in this domain and formalized it as a conceptual metamodel for analyzing and designing meaningful gaming experiences in VR. The main dimensions addressed in this metamodel interrelate user, software, and hardware components, explicitly addressing immersion and player involvement. We argue that the proposed metamodel can be useful to both researchers and designers interested in conceiving immersive, innovative, and engaging VR gaming experiences.
Customer Review Classification Using Machine Learning and Deep Learning Techniques

The online e-commerce industry has become very competitive today and continues to increase. Companies generate a lot of data that contain customer feedback data like reviews about their products and services. In addition, online customer reviews play a significant role in helping the company improve its sales and increase its customer base. Customer reviews are feedback given to businesses and Retailers based on the customer's experience with their organization. Companies use them to improve upon their existing service or the product they are selling. For example, in an e-commerce-driven world, where people have no physical access to the goods they wish to purchase, many customers will turn to online reviews to get an opinion on what to buy. E-commerce is changing how people buy products and services. For example, many businesses look at past data to increase customer count. In this context, companies can use customer reviews on their products to promote their high sales and gain a competitive edge in the market. This paper focuses on classifying the sentiment of customer reviews using machine learning approaches. The models are evaluated using various metrics like confusion matrix, AUROC curve, and classification report.
A Framework for Contextual Recommendations Using Instance Segmentation

Due to the restrictive measures to prevent COVID-19 from spreading, an increasingly large number of viewers are eschewing traditional television programs, resorting to streaming and on-demand platforms. This rapid change in audience preference, combined with the great appeal of streaming services, has constituted a form of "threat" for traditional advertising, causing advertisers and advertising agencies to adapt by participating in content that is, among others, supported by online advertising and streaming platforms. In this work, a novel framework for contextual recommendations using instance segmentation in movies is presented. The proposed service employs deep learning and computer vision algorithms to automatically detect objects in real-time on video streams. The experiments conducted offered satisfactory results regarding both the mAP (mean average precision) for the bounding box and the masks and the continuous decrease of the loss, as well as the correctly detected objects in real time.
Instructional Design of a VR-Based Empathy Training Program to Primary School Children

There are a variety of empathy training programs have been designed to explicitly teach empathy. These programs are generally based on perspective-taking, and aim to feel the others' emotions, to understand them, and to regulate one's own feelings. Virtual reality (VR) allows users to have an "embodied experience". Embodied technology may be a key feature of VR that allows users to practice and improve their cognitive empathy skills, specifically perspective taking. Research that has included the use of VR interventions has found increased empathy towards people with special needs. In the present study, we used VR to develop an empathy curriculum for primary school students. The study applied instructional design principles of ADDIE model, including analysis, design, development, implementation, and evaluation phases, to develop an empathy training program to facilitate educators to meet learning objectives through iterative users' feedback and instructional designers' reflection. The study recruited seven interdisciplinary team members who have experiences in empathy curriculum design, class teaching, academic research, school counseling, and instructional design. The main instructional content includes eight VR videos presenting four common school conflicts in first-person and bystander view, and an interactive function at the end of the VR videos for students to choose a voluntary response in a particular conflict situation. The class instruction is constructed by four sections including VR experiencing, self-reflection, group discussion, and a group activity respectively. The results show that the VR-based empathy training program highly engage students and effectively facilitate students to change perspectives and raise empathic concern. The design of this program sheds light on how to integrate VR technology into classroom teaching with to meet instructional goals.
BLEU Skies for Endangered Language Revitalization: Lemko Rusyn and Ukrainian Neural AI Translation Accuracy Soars

Accelerating global language loss, associated with elevated incidence of illicit substance use, type 2 diabetes, binge drinking, and assault, as well as sixfold higher youth suicide rates, poses a mounting challenge for minority, Indigenous, refugee, colonized, and immigrant communities. In environments where intergenerational transmission is often disrupted, artificial intelligence neural machine translation systems have the potential to revitalize heritage languages and empower new speakers by allowing them to understand and be understood via instantaneous translation. Yet, artificial intelligence solutions pose problems, such as prohibitive cost and output quality issues. A solution is to couple neural engines to classical, rule-based ones, which empower engineers to purge loanwords and neutralize interference from dominant languages. This work describes an overhaul of the engine deployed at LemkoTran.com to enable translation into and out of Lemko, a severely endangered, minority lect of Ukrainian genetic classificability indigenous to borderlands between Poland and Slovakia (where it is also referred to as Rusyn). Dictionary-based translation modules were fitted with morphologically and syntactically informed noun, verb, and adjective generators fueled by 877 lemmata together with 708 glossary entries, and the entire system was riveted by 9,518 automatic, codification-referencing, must-pass quality-control tests. The fruits of this labor are a 23% improvement since last publication in translation quality into English and 35% increase in quality translating from English into Lemko, providing translations that outperform every Google Translate service by every metric, and score 396% higher than Google’s Ukrainian service when translating into Lemko.
Conversation-Driven Refinement of Knowledge Graphs: True Active Learning with Humans in the Chatbot Application Loop

The value of knowledge-grounded cognitive agents is often limited by a lack of high-quality knowledge. Although advances in natural language processing have substantially improved knowledge-extraction capabilities, the demand for different types of knowledge fragments and the potential for error in extraction processes has created a next generation of the knowledge acquisition bottleneck. Instead of waiting for a perfect knowledge base, we propose a design for an agent that is aware of these issues and that actively seeks feedback from users in conversations to improve its knowledge and extraction processes. This approach allows for imperfection and incompleteness, and for the agent to improve over time. Any feedback provided by the users in this conversational application loop is used to not only refine the underlying knowledge graph, but also to improve the knowledge extraction processes. Eventually, the agent’s knowledge and the quality of its answers rises while talking to its users.
The Usability Assessment of Meta Quest2 and Recommendations for Improving

With the development of Virtual Reality (VR) technology in recent years, the VR market is expanding rapidly, from the commercial side to the user side gradually. VR headset devices have begun to popularize. This study focusses on Meta Quest2, a mainstream VR headset device, to analyze its usability by using the Leventhal and Barnes’ usability model and conducting qualitative research. Based on the analysis, this study compares Meta Quest2 with another VR headset Pico 4 and gives recommendations for future improvement.
The Use of Immersive Technologies While Ageing in the Digitally Mediated Society

The ageing population is increasing at an exponential rate, with senior citizens representing a disadvantaged group in threat of both social and digital marginalization. The potential of gerontechnology combined with immersive technologies, such as virtual reality, augmented reality, and mixed reality, may enable active and healthy ageing while enhancing the possibility of encouraging information retention, engagement with the activity, stimulation of cognitive ability, and behavior change. In fact, using immersive gerontechnology to assist senior citizens with age-related impairments to escape from their typically constrained realities may offer a great opportunity that might decrease monotony, disengagement, sadness, stress, and loneliness while also enhancing overall wellness. The aim of this paper is to present and analyze research on VR, AR, and MR for senior citizens, outlining case studies that show how the use of immersive gerontechnology has contributed to or will continue to contribute to active and healthy ageing.
Dialogue-Based User Needs Extraction for Effective Service Personalization

The research of service personalization is flourishing due to the development of machine learning and natural language processing. Despite the prevalence of prior research based on deep learning and dialogue, it remains challenging to reconcile the disadvantages of machine learning, such as explainability, with the strength of utilizing big data. This research proposes a user needs model that incorporates three elements: user readability, ease of extraction through dialogue, and the potential for advancement in machine learning. Additionally, a voice dialogue-based extraction method is designed and constructed to extract the proposed needs. Specifically, by adopting the 6W1H format for the needs model, a simple yet powerful dialogue flow is achieved and enables a comparison of existing services and needs simultaneously. The main modules of the system are a voice dialogue agent, a dialogue system, and a natural language processing-based needs extraction API. Through designing, implementing, and integrating each module, this study realizes a needs extraction system in Japanese. Furthermore, by operating the realized system, a simple evaluation of the needs model and the system is carried out. As a result of this research, both the user and the system can extract needs that are highly readable, contributing to the realization of user-friendly and effective service personalization.
Proposal for a Semi-subjective Learning Support System with Operation Indices Targeting Vectors

It is important that learners engage in trial and error when solving mathematical problems. However, when learning ordinary mathematics, learners often cannot sufficiently perform trial and error because the correct answer is given to them by the textbook or the teacher after they have solved the problem. Therefore, we aim to encourage trial and error by producing a visualization of the learner's solution and having them interact with it. In this study, we give an overview of how we developed a learning support system that visualizes errors made by learners studying vector addition.
Don’t Miss the Fairytale Forest for the Trees: Integrating Location-Based Authoring in Interactive Story Development for Immersive Augmented Reality

Authoring in location-based Interactive Storytelling (IS) for Augmented Reality (AR) requires a combination of, at one end, process-intensive development of assets and complex conditional storylines on a desktop computer, and at the other end, the creation of situational aspects in the field. While there are occasional solutions for hand-held AR to support such a distributed workflow, the topic has not yet been explored in depth for immersive AR using a head-mounted display. We therefore propose to combine an existing IS-tool with Unity3D to bridge this gap. We describe our development process and discuss our findings. Early results show that the combination of Ink and Unity3D has the potential to support authors by providing a more integrated development workflow as they alternate between editing phases on the desktop computer and on-site.