Reviewerly/exHarmony huggingface dataset:
The Effects of Dual Display Configuration and Users’ Experience on Performance and Preference in Complex Spatial-Verbal Tasks

Working from home has become popular during Covid-19 pandemic and it promoted the use of dual displays to handle complex information coming with multi-tasking. Previous studies have shown that the use of dual displays can produce better performance; however, the effects of different dual display configurations on users' performance in complex spatial-verbal tasks have not been systematically discussed. Furthermore, whether the user is experienced in utilizing dual displays may affect their performance and preference such as assignment of software windows. The current study investigated task completion time and user behaviors (window assignment, head turning counts and the number of window switches) in four display configurations (preferred primary-secondary screens, swapped primary-secondary screens, unified screens, and a single screen) via a realistic experimental task where participants operated 3D street view on Google Map (spatial task) to answer relevant questions in MS Word (verbal task). The results from 30 participants showed that the swapped primary-secondary screens (where the window assignment was different from the participant's preference) and the unified screen caused significant more window switching and head turning; the task completion time also increased insignificantly. The participants experienced in using dual screen performed better than the non-experienced ones in all configurations but the unified screens. The experienced participants subjectively reported that they preferred to perform longer and more complicated operations on the primary display while the non-experienced ones said the opposite. Through this study the user behaviors in spatial-verbal tasks using dual displays were understood and guidelines were provided to improve user performance.
Connecting Historic Photographs with the Modern Landscape

We present a mobile application that is bringing to life historic photographs of Heraklion by mapping them in historical routes within the city and offering the visitor a dialogue with its social and historical context. At the same time while visiting the public library where these historic photographs are exhibited, the mobile app enhances the provided information by virtually replacing the photographs with alternative views from different historic periods.
Identification of Evaluation Items in Consumer Reviews Using Natural Language Processing Models with Social Media Information

In recent years, customer reviews have been attracting attention as data representing real consumer feedback. On the other hand, it is said that it is difficult to identify the detailed contents from the text. In this study, we aim to understand the detailed contents of customer reviews from the data of customer reviews of accommodation facilities. Specifically, we collected tweets using keywords related to accommodation reviews as search queries, and used them to create a sentence classification model by fine-tuning a previously developed Japanese version of BERT model. Sentence classification was conducted by applying this model to the customer review data, which was separated into sentences. In addition, sentiment analysis was conducted to identify whether the sentences were positive or negative.
Studying Multi-modal Human Robot Interaction Using a Mobile VR Simulation

The task of designing unequivocal communication between autonomous vehicles and humans is important, considering the supposed daily interaction which might take place in the future. Using vehicles which are still in development for a study, can be a security risk and might even endanger the subjects. At the same time, it can be a waste of resources to integrate an interaction model which might not work well. It is therefore crucial to create a safe but realistic environment, in which studies can be held. In this paper we study the interaction between humans and an autonomous vehicle called CityBot, by using self-developed VR and web-applications. We describe two conducted studies, which were focused on speech, gestures and audio signals as means of communication between both parties. We discuss the problems, as well as differences between gestures and speech for controlling the CityBot. Furthermore, we propose a prefered usage of speech and how to handle ambigious gestures.
Exploring Design Principles for Speech-to-Visualization Data Entry Interfaces

We present an interface for automatic transcription and simultaneous visualization of spoken narratives on a timeline. We discuss the results of initial laboratory testing of the interface and interviews with prospective users of the approach in two different domains. Speech transcription and entity recognition errors inherent in machine-learning-based approaches to natural language processing place special requirements on the system-user interaction. We outline the design principles for this kind of interface based on the results of testing and interviews with potential users. The presented approach differs from other state-of-the-art text-to-visualization systems in that it constructs a visualization from a speech narrative, while automatically identifying data of interest, as opposed to building plot visualization for user-specified data in structured form based on user's spoken instructions.
Situation-Aware Adaptations for Individualized User Experience and Task Management Optimization

The SitAdapt system is an architecture and runtime system for building adaptive interactive applications. The system is integrated into the PaMGIS framework for pattern- and model-based user interface construction and generation. This paper focuses on the different types of adaptations and the timeslots for controlling the options for changing the user interface and the interactive behavior. During sessions the system is monitoring the user and collecting visual, bio-physical, and emotional data that may vary over time. Based on these data situations can be recognized and adaptation rules that activate the PaMGIS models and resources can be triggered at runtime. The operation of the system is demonstrated with examples from an adaptive travel-booking application. The paper also discusses, how task accomplishment can be controlled by mapping real user activities during a session to the task models that are made available through the PaMGIS model repositories.
“Should My Best Prove Insufficient, We Will Find Another Way”: Time Loop Mechanics as Expressions of Hope in Digital Games

Many digital games use mechanics, user interface design, and interaction design to explore themes of science fiction and technology, including the ramifications of time travel and time manipulation. Time loop games, in which a character repeats the same time period or sequence of experiences indefinitely, are surprisingly optimistic, even when the game world, content, or scenario is emotionally challenging. While all digital games allow players to manipulate time at a basic level, time loop games emphasize mastery-through-repetition in both gameplay and narrative. Unlike time travel narratives in other media, time loop games cannot help but express optimism, as the game's mechanics and interface encourage players to continually manipulate the game world until the best of all possible outcomes is achieved. It follows that time loop games are a clear example of narrative expressed through mechanics and interface, as well as a way to express hope in difficult times. This paper uses a humanities-based critical methodology to explore how the structure of time loop games reinforces feelings of optimism and hope through iterative improvement and eventual mastery of game systems, and examines both seminal and recent time loop games including The Legend of Zelda: Majora's Mask, Returnal, Deathloop, and Outer Wilds.
Is It Possible to Measure Real-Time VR Motion Sickness with EDA Wavelet Features?

Virtual reality (VR) has emerged as a more commercial mainstream, and the number of users complaining of VR motion sickness is increasing rapidly. As a result, many methods, physiological indicators: electrogastrography (EGG), heart rate (HR), temperature, electrodermal activity (EDA), subjective measurement methods, simulator sickness questionnaire (SSQ), virtual reality symptom questionnaire (VRSQ), fast motion sickness score (FMS), have been studied to quantify VR motion sickness. Consequently, this study attempted to track motion sickness using EDA (sweat). The study group consisted of ten female and ten male participants who have access to VR (average age: 21.2 years; standard deviation: 2.07 years) with no physical or visual health problems. The VR environment was designed by wearing a head-mounted display (HMD) through a simple operation method. Subjects wore an EDA sensor (Empatica E4) in a VR environment and were asked to respond to SSQ, VRSQ, and FMS questionnaires. For all items of FMS and SSQ (nausea, oculomotor, disorientation, and total) and Total of VRSQ, the coefficient of determination was high in the order of 1st trial > 2nd trial > Rest. Overall, the motion sickness score was rated the lowest during the rest period. The coefficient of determination of EDA (sweat) in rest was low in all except Disorientation in VRSQ. And VR motion sickness may have different symptoms complaining to the user depending on the VR environment. Hence, this study suggests that the EDA (sweat) signal may be suitable depending on the user’s immersion interval and the symptoms.
VisRef: A Reflection Support System Using a Fixed-Point Camera and a Smartwatch for Childcare Fields

We propose VisRef, a video-based reflective support system incorporating a fixed-point camera and a smartwatch, and investigate its application for improved retrospection in childcare fields. When a childcare worker wearing a smartwatch encounters a situation he or she hopes to reflect upon later, the worker can press a button on the smartwatch to record a video of the immediately preceding few minutes. Depending on the button pressed, a preset text tag or transcripted voice tag is added to the video when they are posted to a chat system, shared with the relevant parties, or stored as a private record. We conducted experiments using the proposed application at a nursery school and at a daycare center for children with disabilities in Japan. The efficacy of this system in improving different field activities was evaluated through questionnaires and interviews. The results indicate that the acceptance of the system varied considerably depending on the differences in the facility environments. We identified several points can be explored in the future.
Discussion on the User Experience of Intelligent Travel Applications for Electric Bicycles

This research mainly discussed the influence of interface design on the user experience of intelligent travel applications for electric bicycles. Three representative electric bicycle applications were selected as the experimental samples. Thirty subjects were asked to complete three tasks, including locking and unlocking an electric bicycle, finding the functions of authorizing others to use the electric bicycle and contacting for maintenance. The operation time of each task was measured. Participants were also required to fill out system usability scale (SUS) questionnaire to collect subjective data. The research results can be summarized as follows: (1) Using a sliding button to lock and unlock an electric bicycle can improve users’ subjective evaluations of usability and their efficiency. (2) The main functional icons need to be integrated into a clear interface and have good affordance to improve users’ efficiency. (3) It is recommended to use fewer colors on the interfaces of intelligent travel applications.
Relevance-Aware Question Generation in Non-task-Oriented Dialogue Systems

In recent years, there has been a growing interest in dialogue systems as spoken agents and communication robots become available for practical use. Much research has been conducted on non-task oriented dialogues, in which the goal is for the dialog system to interact naturally with a person. However, at present, it cannot be said that the system and people are sufficiently able to have a natural dialogue. Therefore, this study focuses on question generation and aims to develop natural dialogue between the system and people by generating questions that take into account the relevance of the dialogue. Here, the topic of an utterance is extracted, and related words are obtained on the basis of information associated with the extracted topic and their literal co-occurrence. Interrogative word is selected with the utterance information considering the context, and a question is generated using the acquired related word and interrogative word. The experimental results show the effectiveness of the proposed method in generating contextualized question.
Virtual Reality and the Interactive Design of Elevated Public Spaces: Cognitive Experience vs VR Experience

VR and visualization are visual aids, but can give much more than sight, such as a wider sensation of being in the 3D world. For the past ten years, researchers and organizations have been trying to find ways to get the public more involved in social project design and execution. Technologies such as Visual Simulation (VS) and Virtual Reality (VR) could make it possible to provide final stakeholders and consumers with closer access to the design and implementation processes of social projects. This paper analyzed two recent developments in London, Sky Garden and Crossrail Place. The study investigates how technologies such as VR can enhance the design experience of elevated urban spaces. The main aim of this study is to explore the difference and similarities between human interaction in the real cognitive experience and the virtual world. The method used in this study is walk-long interviews with the visitors of both gardens and a VR exploratory experiment, followed by semi-structured interviews with the participants. This methodology brings the participant as an active element of the project, able to explore the space and propose changes. The paper discussed (i) the design quality of elevated social spaces such as accessibility, circulation, activities, design concerns and design features, (ii) the participant's experience and behaviour during the study and (iii) the effectiveness of the design used tools. The results showed a positive impact on enhancing user involvement and allowed users to produce and test different design alternatives in real time. The research findings include a study of human activities in the physical and the virtual world, pedestrian modelling for the selected case studies, and an analysis of the potential need for new rules and regulations relating to the use of such spaces. The research also highlighted the limitations and potentials of using these new methods as a co-design approach for designing public and social spaces in the city.
A Contextual Design Approach for Creating a Holistic Fitness Application

With the rising popularity of fitness and the growing need to record workouts for analytical or social purposes, there is an increasing demand for fitness application(s) that would cater to the requirements of the emerging lifestyle. Looking at the available applications, there are only a few of them, namely, Garmin Connect, Strava, RunKeeper, Map My Ride, Zombie Run and Zwift. From the reviews received on their websites or applications, multiple issues were identified. A few of them are syncing issues, notification issues, GPS issues, fitness goal update issues, and many more. Motivated by these shortcomings in a market of high demand, a contextual design approach was used to design a new fitness application that would benefit the fitness enthusiast without these issues. Intermediary steps in achieving this goal were - experience model construction, grand vision development, identification of persons, product concepts and their cool drilldowns, storyboarding, Generalized Transition Network (GTN), wireframing, prototyping, creating Usability Aspect Report (UAR) based on persona, and improvements made according to the UARs. Along with these, a few suggestions for further improvement have been made.
Newbie Guides for Omnidirectional Guidance in Head-Mounted-Device-Based Museum Applications

Novel digital museum guides using omnidirectional cues thr-ough head-mounted devices (HMDs) have been presented in previous research. However, due to the novel interaction technologies of HMDs, today's visitors are not yet familiar with these types of guides. We developed and tested instructions ("newbie guides") to help first-time users learn novel interaction patterns of HMDs, inspired by those used in website design or game tutorial design. First, we created three basic example patterns to direct attention using visual, auditory, and a combination of visual and auditory cues. Second, we created a newbie guide as an example showcase for each guidance pattern based on seven design criteria. The newbie guides were tested and evaluated in an exhibition situation, and the results indicate that the seven design criteria can assist HMD-based AR developers in creating effective newbie guides to support first-time users learn omnidirectional guidance patterns in the museum and exhibition context.
Toward a Gesture System Architecture in Extended Reality Based on a Multi-dimensional Taxonomy of Gestures

Gesture-based interactions have been widely applied in Human-Computer Interaction areas. As Extended Reality technology (XR) has grown rapidly, gesture has also grown in importance as a way of interaction in XR. Although there have been many efforts on gesture taxonomy in HCI, there is a lack of works that classify gestures focusing on their applications and roles in XR domains. We have been working on a taxonomy that incorporates important previous works on gesture taxonomies, modifies, and injects new dimensions for the taxonomy to better reflect the wide variety of contexts and scenarios in XR. This paper presents a gesture system architecture for Extended Reality based on our ongoing work on a multi-dimensional taxonomy of gesture in XR applications. This gesture system architecture can be used to build general-purpose gesture systems that will be easy for both designers and users to choose the right gesture for the situation at hand.
Dashcam Video Curation for Generating Memorial Movies on Tourism Using Multiple Measures of “Tourist Spot Likeness”

Recently, a method for the automatic generation of driving sightseeing memorial route movies has been proposed by performing curation on videos captured by a dashcam. Existing methods curate videos by scoring video frames with a single specific measure, however, it is hard to generate the appropriate memorial route movies due to the differences in important perspectives for different users. To solve this problem, we propose the method that represents “tourist spot likeness” with a combination of common measures derived by procedures used in the field of Affective Engineering and curates dashcam videos based on multiple measures. In this paper, we surveyed these representative words to identify the measures that compose the measure of “tourist spot likeness” and construct a model to estimate the score of each measure using dashcam video data as input. As the results of the experiment by using dashcam videos taken in Okinawa, we have derived six measures for curating the dashcam videos and confirmed the proposed models, which estimate each measure scored with a 7-level Likert scale achieved mean MAE of 0.64 (best: 0.52, worst: 0.73).
Applying PBL Model to Technological Learning Strategies for Older Adults

With the aim of integrating the concepts of PBL learning philosophies and learning communities, this study discussed and worked out a technological teaching strategy suitable for older adults. It was expected that this study could explore and deepen the learning experience of older adults through personally experiencing the situation and through problem exchange to assist them in improving their technological literacy and the ability to learn independently. This study held a participatory workshop and invited 18 older adults to participate in a technological learning course to further probe into the learning effectiveness and the opinions of older adults. This study showed that when learning in technological courses, the participants had great individual differences, were interest- and practical-oriented, had a lack of self-confidence, had a community effect, and had requirements for immediate discussion and assistance. Assisted by two kinds of PBL models, the participants received significant positive effects on the desire for technical learning, the perceived usefulness, and their psychological well-being. Moreover, the participants also had the thought and motivation to continue pursuing independent learning.
A Human or a Computer Agent: The Social and Cognitive Effects of an e-Learning Instructor’s Identity and Voice Cues

An instructor in an e-learning video can identify as a human or a computer agent. Relatedly, they can project a human-recorded voice or a machine-voice generated from a classical text-to-speech engine. This study examines the effects of an e-learning instructor's identity and voice cues on an instructor's social ratings, learners' cognitive load, and learning performance. A between-subjects laboratory experiment was conducted where university undergraduates (n = 108) interacted with either one of the four e-learning videos featuring different pairings of an instructor's identity (human versus agent) and voice (human-voice versus machine-voice) cues that delivered a lesson on programming algorithms. The findings affirmed the voice effects in multimedia learning in that the human-voice enhanced social and learning outcomes more than the machine-voice, irrespective of the identity cues. Credibility ratings were diminished when an instructor identified as a computer agent projected a human-voice than a machine-voice—additionally, endowing a human-identified instructor with a machine-voice prompted learners to assign lower intrinsic cognitive load. These observations imply the congruence/incongruence effects of identity-voice cue pairings on social and cognitive load outcomes. Theoretical and practical implications are discussed in this paper.
Semi-automatic Basketball Jump Shot Annotation Using Multi-view Activity Recognition and Deep Learning

Statisticians introduce subjectivity while recording game statistics. This research aims to reduce this problem using a multi-view pose classification model, starting with the jump shot location event annotation. Basketball simulations will be conducted to determine if the proposed model can be more objective than a human statistician recording the same jump shot event. To this end, the Exhaustive Basketball System (EBS) was developed. EBS is a web application that allows customizable courts and game rules variations, enabling storing in-game event data. Allowing the extraction of the necessary jump shot coordinates data recorded by the statistician during the simulations for analyses. By controlling the number of players, game time duration, and an agility index grouping technique proposed for the basketball simulations, their impact on the coordinates data will be analyzed in an ANOVA 3*2*3 factorial design with three repetitions. The response variable is the average distance of the jump shot attempts event annotation regarding the ground truth location. While other researchers have worked on jump shot recognition using a single view of the court, our research attempts to contribute to this concept but with multiple synchronized viewing angles in addition to subjectivity reduction. We expect to prove an ideal game statistics generation technique to register objective statistics. Moreover, be a pathway for objective game statistics and present recommendations for future work related to other sports or fields that could benefit from the proposed technique.