Reviewerly/exHarmony huggingface dataset:
Inter-Subject EEG Synchronization during a Cooperative Motor Task in a Shared Mixed-Reality Environment

Mixed-reality (MR) environments, in which virtual objects are overlaid on the real environment and shared with peers by wearing a transparent optical head-mounted display, are considered to be well suited for collaborative work. However, no studies have been conducted to provide neuroscientific evidence of its effectiveness. In contrast, inter-brain synchronization has been repeatedly observed in cooperative tasks and can be used as an index of the quality of cooperation. In this study, we used electroencephalography (EEG) to simultaneously measure the brain activity of pairs of participants, a technique known as hyperscanning, during a cooperative motor task to investigate whether inter-brain synchronization would be also observed in a shared MR environment. The participants were presented with virtual building blocks to grasp and build up an object cooperatively with a partner or individually. We found that inter-brain synchronization in the cooperative condition was stronger than that in the individual condition (F(1, 15) = 4.70, p &lt; 0.05). In addition, there was a significant correlation between task performance and inter-brain synchronization in the cooperative condition (rs = 0.523, p &lt; 0.05). Therefore, the shared MR environment was sufficiently effective to evoke inter-brain synchronization, which reflects the quality of cooperation. This study offers a promising neuroscientific method to objectively measure the effectiveness of MR technology.
Designing for In-Home Long-Term Family-Robot Interactions: Family Preferences, Connection-Making, and Privacy

Participatory approaches to designing technology with families, for families allow designers to have a first-hand understanding of needs, desires, and preferences of families toward new technology. For example, in my prior work, I conducted participatory design studies with children and their families in order to design social robots that could facilitate long term interactions, with multiple users, in their homes. Our design sessions identified that children and families preferred an in-home robot to have the role of a companion or an assistant, be able to hold group interactions and participate in shared recreation activities, and had concerns about the robot's ability to follow conversational privacy norms (i.e., within a family, what information is shared with whom). In a long-term field deployment, I explored how children engaged in dyadic interactions with a reading companion robot in their homes and how interactions with multiple family members formed with and around the robot. However, family-centered insights are limited in the Human-Robot Interaction field. In my research I aim to further explore long-term family-robot interactions in group settings, with social companion robots that serve as a playmate or a confidant. I seek to design social robots that can facilitate connection-making between family members and mitigate communication privacy conflicts in these group interactions.
Potential and Challenges of DIY Smart Homes with an ML-intensive Camera Sensor

Sensors and actuators are crucial components of a do-it-yourself (DIY) smart home system that enables users to construct smart home features successfully. In addition, machine learning (ML) (e.g., ML-intensive camera sensors) can be applied to sensor technology to increase its accuracy. Although camera sensors are often utilized in homes, research on user experiences with DIY smart home systems employing camera sensors is still in its infancy. This research investigates novel user experiences while constructing DIY smart home features using an ML-intensive camera sensor in contrast to commonly used IoT sensors. Thus, we conducted a seven-day field diary study with 12 families who were given a DIY smart home kit. Here, we assess the five characteristics of the camera sensor as well as the potential and challenges of utilizing the camera sensor in the DIY smart home and discuss the opportunities to address existing DIY smart home issues.
GVQA: Learning to Answer Questions about Graphs with Visualizations via Knowledge Base

Graphs are common charts used to represent the topological relationship between nodes. It is a powerful tool for data analysis and information retrieval tasks involve asking questions about graphs. In formative study, we found that questions for graphs are not only about the relationship of nodes but also about the properties of graph elements. We propose a pipeline to answer natural language questions about graph visualizations and generate visual answers. We first extract the data from graphs and convert them into GML format. We design data structures to encode graph information and convert them into an knowledge base. We then extract topic entities from questions. We feed questions, entities and knowledge bases into our question-answer model to obtain the SPARQL queries for textual answers. Finally, we design a module to present the answers visually. A user study demonstrates that these visual and textual answers are useful, credible and and transparent.
Photovoltaic inverter anomaly detection method based on LSTM serial depth autoencoder

Abstract To ensure the safety of the massive growth of distributed photovoltaic grid-connected inverters and the security of backhaul data in the context of new power systems, research on anomaly detection has also been put on the agenda. The data of the photovoltaic grid-connected inverter has complex time dependence and uncertainty, and the data security problem is prone to occur in the process of data transmission, and the existing anomaly detection means do not comprehensively consider these factors, and cannot timely find the abnormal phenomena caused by the synthesis of data security and their faults, which leads to the tampering of power data and the increase of operation and maintenance costs. To solve the above problems, an anomaly detection method integrating a long short-term memory network (LSTM) and serial depth autoencoder (DAE) is proposed based on edge computing, characterized by the power and voltage of the device, the length, and the delay of the data. Firstly, the method performed time series transformation processing on the original training dataset through the sliding window. Then, the model was trained using LSTM and DAE. Finally, the decoding error of the test data was compared with the threshold obtained by the training. The example verification shows that the model has achieved good performance in the recall rate, accuracy rate and F1 score, and can effectively detect the equipment that produces its fault or data return abnormality in the photovoltaic inverter. So, the model has good industrial practical value.
IcaGCN: Model Intents via Coactivated Graph Convolution Network for Recommendation

In this era of information overload, to better provide personalized content services to users, recommendation systems have greatly improved the efficiency of information distribution. Graph Convolution Network(GCN), which is one of the representative works of graph structure aggregation processing, works by node convolution with the help of the Laplacian matrix of the graph and weighted combination of neighbor node information according to the outgoing and incoming degrees of neighbor nodes to obtain the representation of the current node. However, the mainstream GCN models nowadays do not take into account data augmentation of metadata and the fact that each node plays different roles with different importance and weights, thus making the recommendation performance limited. To better solve the above problems, we propose the IcaGCN model, which can perform data augmentation and calculate node weights in modules, and is a convenient plug-and-play method. Finally, extensive experimental results on four real-world datasets have shown the effectiveness and robustness of the proposed model. Especially on the Amazon-Book dataset, our IcaGCN has improved by 6.32%, 42.29%, and 12.38% in Recall@20, MRR@20, and NDCG@20, respectively, compared to other existing state-of-the-art models. We also provide source code and data to reproduce the experimental results <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>a</i></sup> .
Enhanced Auditoriums for Attending Talks in Social Virtual Reality

Replicating traditional auditorium layouts for attending talks in social virtual reality often results in poor visibility of the presentation and a reduced feeling of being there together with others. Motivated by the use case of academic conferences, we therefore propose to display miniature representations of the stage close to the viewers for enhanced presentation visibility as well as group table arrangements for enhanced social co-watching. We conducted an initial user study with 12 participants in groups of three to evaluate the influence of these ideas on audience experience. Our results confirm the hypothesized positive effects of both enhancements and show that their combination was particularly appreciated by audience members. Our results therefore strongly encourage us to rethink conventional auditorium layouts in social virtual reality.
Fine-Tuning of Pre-Trained Deep Face Sketch Models Using Smart Switching Slime Mold Algorithm

There are many pre-trained deep learning-based face recognition models developed in the literature, such as FaceNet, ArcFace, VGG-Face, and DeepFace. However, performing transfer learning of these models for handling face sketch recognition is not applicable due to the challenge of limited sketch datasets (single sketch per subject). One promising solution to mitigate this issue is by using optimization algorithms, which will perform a fine-tuning and fitting of these models for the face sketch problem. Specifically, this research introduces an enhanced optimizer that will evolve these models by performing automatic weightage/fine-tuning of the generated feature vector guided by the recognition accuracy of the training data. The following are the key contributions to this work: (i) this paper introduces a novel Smart Switching Slime Mold Algorithm (S2SMA), which has been improved by embedding several search operations and control rules; (ii) the proposed S2SMA aims to fine-tune the pre-trained deep learning models in order to improve the accuracy of the face sketch recognition problem; and (iii) the proposed S2SMA makes simultaneous fine-tuning of multiple pre-trained deep learning models toward further improving the recognition accuracy of the face sketch problem. The performance of the S2SMA has been evaluated on two face sketch databases, which are XM2VTS and CUFSF, and on CEC’s 2010 large-scale benchmark. In addition, the outcomes were compared to several variations of the SMA and related optimization techniques. The numerical results demonstrated that the improved optimizer obtained a higher level of fitness value as well as better face sketch recognition accuracy. The statistical data demonstrate that S2SMA significantly outperforms other optimization techniques with a rapid convergence curve.
Subgame Optimal Strategies in Finite Concurrent Games with Prefix-Independent Objectives

Abstract We investigate concurrent two-player win/lose stochastic games on finite graphs with prefix-independent objectives. We characterize subgame optimal strategies and use this characterization to show various memory transfer results: 1) For a given (prefix-independent) objective, if every game that has a subgame almost-surely winning strategy also has a positional one, then every game that has a subgame optimal strategy also has a positional one; 2) Assume that the (prefix-independent) objective has a neutral color. If every turn-based game that has a subgame almost-surely winning strategy also has a positional one, then every game that has a finite-choice (notion to be defined) subgame optimal strategy also has a positional one. We collect or design examples to show that our results are tight in several ways. We also apply our results to Büchi, co-Büchi, parity, mean-payoff objectives, thus yielding simpler statements.
T-Force: Exploring the Use of Typing Force for Three State Virtual Keyboards

Three state virtual keyboards which differentiate contact events between released, touched, and pressed states have the potential to improve overall typing experience and reduce the gap between virtual keyboards and physical keyboards. Incorporating force sensitivity, three-state virtual keyboards can utilize a force threshold to better classify a contact event. However, our limited knowledge of how force plays a role during typing on virtual keyboards limits further progress. Through a series of studies we observe that using a uniform threshold is not an optimal approach. Furthermore, the force being applied while typing varies significantly across the keys and among participants. As such, we propose three different approaches to further improve the uniform threshold. We show that a carefully selected non-uniform threshold function could be sufficient in delineating typing events on a three-state keyboard. Finally, we conclude our work with lessons learned, suggestion for future improvements, and comparisons with current methods available.
CHI Course: Introduction to HCI

This course introduces the field of human-computer interaction (HCI). In the first unit, we present the characteristics of the field, discussing concepts and goals that distinguish it from other fields. In the second unit, we illustrate how to understand people, conduct user research, describe interaction, distinguish types of user interface, and design, engineer, and evaluate interactive systems. The course is based on an upcoming book (Introduction to Human-Computer Interaction) by the authors.
A Pipeline for Story Visualization from Natural Language

Generating automatic visualization from natural language texts is an important task for promoting language learning and literacy development for young children and language learners. However, translating a text into a coherent visualization matching its relevant keywords is a challenging problem. To tackle this issue, we proposed a robust story visualization pipeline ranging from NLP and relation extraction to image sequence generation and alignment. First, we applied a shallow semantic representation of the text where we extracted concepts including relevant characters, scene objects, and events in an appropriate format. We also distinguished between simple and complex actions. This distinction helped to realize an optimal visualization of the scene objects and their relationships according to the target audience. Second, we utilized an image generation framework along with different versions to support the visualization task efficiently. Third, we used CLIP similarity function as a semantic relevance metric to check local and global coherence to the whole story. Finally, we validated the scene sequence to compose a final visualization using the different versions for various target audiences. Our preliminary results showed considerable effectiveness in adopting such a pipeline for a coarse visualization task that can subsequently be enhanced.
Taxonomy of outlier detection methods for power system measurements

Abstract The new emerging technologies utilize various sensors, deployed in an ad‐hoc manner to reduce energy consumption in data communication. The data collected from these sensors is huge and have a high possibility of being polluted by outliers. Therefore, researchers are trying to develop better and faster outlier detection techniques that can handle large amount of data. In this paper, the research works from the year 2000 to 2022 have been reviewed. Several fundamental and latest outlier detection methods are discussed and categorized on the basis of statistical properties, density, distance, and clustering. The other methods discussed in this paper are ensemble methods and learning‐based methods. The definitions, causes of outliers, and different methods of outlier detection are discussed. Further, one of the efficient methods from each category of the method is implemented on synthetic data of the IEEE 13‐bus distribution system. The IEEE 13‐bus system is assumed to have a Multi‐Function Meter (MFM) at each line in the system. The data captured is injected with a fixed number of outliers at a given instant. Thereafter, the performance of all the methods is tested based on the number of outliers being detected.
A Logical Framework with Higher-Order Rational (Circular) Terms

Abstract Logical frameworks provide natural and direct ways of specifying and reasoning within deductive systems. The logical framework LF and subsequent developments focus on finitary proof systems, making the formalization of circular proof systems in such logical frameworks a cumbersome and awkward task. To address this issue, we propose $$ \text {CoLF} $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mtext>CoLF</mml:mtext> </mml:math> , a conservative extension of LF with higher-order rational terms and mixed inductive and coinductive definitions. In this framework, two terms are equal if they unfold to the same infinite regular Böhm tree. Both term equality and type checking are decidable in $$ \text {CoLF} $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mtext>CoLF</mml:mtext> </mml:math> . We illustrate the elegance and expressive power of the framework with several small case studies.
Towards Intermediated Workflows for Hybrid Telemedicine

The growing platformization of health has spurred new avenues for healthcare access and reinvigorated telemedicine as a viable pathway to care. Telemedicine adoption during the COVID-19 pandemic has surfaced barriers to patient-centered care that call for attention. Our work extends current Human-Computer Interaction (HCI) research on telemedicine and the challenges to remote care, and investigates the scope for enhancing remote care seeking and provision through telemedicine workflows involving intermediation. Our study, focused on the urban Indian context, involved providing doctors with videos of remote clinical examinations to aid in telemedicine. We present a qualitative evaluation of this modified telemedicine experience, highlighting how workflows involving intermediation could bridge existing gaps in telemedicine, and how their acceptance among doctors could shift interaction dynamics between doctors and patients. We conclude by discussing the implications of such telemedicine workflows on patient-centered care and the future of care work.
Understanding Gender Transition Tracking Habits and Technology

Personal health tracking has long been a topic of investigation in the HCI community. There is an emerging class of apps that support gender transition, which we term transition-tracking apps. However, little work has been done examining the use and impact of such apps. We aimed to address this gap by conducting an interview study with sixteen participants who are currently undergoing different forms of gender transition. We provide an understanding of transition tracking habits, the usage and potential of transition-tracking apps in the context of transition support technologies, and provide design suggestions and open areas of research.
Futuring (CHI)Art: Building a Collective Common Future

Human-Computer Interaction and the Arts have a relatively recent history, but computers have opened the world to new forms of expression and wide audiences. The way in which we think of art has changed, from webcomics and digital painting to AI-generated imagery and NFTs. However, these changes have also caused tension, and a disparity between what art is, and how it reconciles with research. This workshop aims to bring together artists and researchers to explore the current worldview of HCI and Art and speculate as to its possible, plausible, and probable futures, via discussion, collaboration, and the formation of new expressive artworks, objects, and performances. Through this process, we will also create a roadmap for a common future and imagine new ways of sharing this dual practice with the SIGCHI community and beyond.
Classifying minerals and their related names in a relational database

Abstract The categorisation of minerals and their related names, such as synonyms, obsolete or historical names, varieties or mixtures, is an asset for designing an interoperable and consistent mineralogical data warehouse. An enormous amount of this data, provided by mindat.org and other resources, was reviewed and analysed during the research. The analysis indicates the existence of several categories of (1) the abstract titles or designations representing the link to the original material or a group of names or substances without actual physical representation, and (2) the unique names representing actual physical material, compounds, or an aggregate of one or more minerals. A revision of the dependency between the categories attributes stored in a database (e.g. chemical properties, physical properties) and their classification status assigned allowed us to design a robust prototype for maintaining database integrity and consistency. The proposed scheme allows standardisation and structuring of officially regulated and maintained species, e.g. IMA-approved, and, in addition, unregulated ones.
AESOP: Adjustable Exhaustive Search for One-Pixel Attacks in Deep Neural Networks

Deep neural networks have achieved remarkable performance in various fields such as image recognition and natural language processing. However, recent research has revealed that even a small imperceptible perturbation can confound well-trained neural network models and yield incorrect answers. Such adversarial examples are regarded as a key hazard to the application of machine learning techniques to safety-critical systems, such as unmanned vehicle navigation and security systems. In this study, we propose an efficient technique for searching one-pixel attacks in deep neural networks, which are recently reported as an adversarial example. Using exhaustive search, our method can identify one-pixel attacks which existing methods cannot detect. Moreover, the method can adjust exhaustiveness to reduce the search space dramatically. However, it still identifies most attacks. We present our experiment using the MNIST data set to demonstrate that our adjustable search method efficiently identifies one-pixel attacks in well-trained deep neural networks, including convolutional layers.
Is “Categorical Imperative” Metaversal?: A Kantian Ethical Framework for Social Virtual Reality

The increasing adoption of social virtual reality (VR) environments for socializing and collaborating with others has led to a growing concern about ethical issues in these immersive environments. Beyond the introduction of some practical guidelines, theoretical work on this topic has been scant. In this paper, we propose an ethical framework for social VR based on Kant's Theory of Morality. In so doing, we argue that the Kantian concept of categorical imperative does apply to social VR, that the reality of VR is not different from the reality of real life, and that what is morally unacceptable in real life is and should be unacceptable in social VR. In our framework, we provide three principles that can aid users and developers of social VR environments in reasoning about ethical issues in social VR, while advocating for more theoretical approaches to addressing the issue of VR ethics in human-computer interaction circles.