Reviewerly/exHarmony huggingface dataset:
µMe: Exploring the Human Microbiome as an Intimate Material for Living Interfaces

Our bodies are home to an unseen ecosystem of microbes that live in symbiosis with us. In this work, we extend the “human” in Human-Computer Interaction (HCI) to include these microbes. Specifically, we explore the skin microbiome as an intimate material for interaction design. Viewing the body as a microbial interface, we start by presenting a method to grow our microbiome such that it becomes visible to the human eye. We then present a design space that explores how different environmental parameters, such as temperature and growth media, can be controlled to influence the color of the microbiome. We further investigate how our interactions in a daily uncontrolled environment (e.g., exercising, hugging, typing) can impact the microbiome. We demonstrate several wearable applications that reveal and control the microbiome. Lastly, we address the challenges and opportunities of working with the microbiome as an intimate, living material for interaction design.
Introducing computers in Indian schools: institutional resistances and the making of a digital divide

The introduction of computers in Indian schools runs parallel with the development of a digital divide in the country. In addition to the formal ways in which government policies shape access to computers in schools, deeper informal cultures of the state and schools shape opinion about these technologies. In this paper, we examine three levels of these informal cultures: political cultures and their perceptions of technology; bureaucratic cultures, which are distanced from schooling realities; and classroom teaching-learning practices, where digital inequalities add to pre-existing educational inequalities. For examining the interplay of the informal cultures, we scale down from the national level to examine the case of West Bengal, a state in eastern India. The state has been one of the strongholds of Left politics in India. Drawing upon its experiences, we explore the linkage between political ideology and the social embedding of resistance to computers.
Co-Speculating on Dark Scenarios and Unintended Consequences of a Ubiquitous(ly) Augmented Reality

The vision of a ‘metaverse’ may soon bring a ubiquitous(ly) Augmented Reality (UAR) delivering context-aware, geo-located, and continuous blends of real and virtual elements into reach. This paper draws on speculative design to explore, question, and problematize consequences of AR becoming pervasive. Elaborating on Desjardin et al.’s bespoke booklets, we co-speculate together with 12 globally dispersed participants. Each participant received a custom-made design workbook containing pictures of their immediate surroundings, which they elaborated on in situated brainstorming activities. We present an integration of their speculative ideas and lived experiences in 3 overarching themes from which 7 ‘dark’ scenarios caused by UAR were formed. The Scenarios are indicative of deceptive design patterns that can (and likely will be) devised to misuse UAR, and anti-patterns that could cause unintended consequences. These contributions enable the timely discussion of potential antidotes and to which extent they can mitigate imminent harms of UAR.
Metamorphonic: A Reflective Design Inquiry into Human-Silkworm Relationship

This work explores an alternative human-silkworm relationship through crafting a triad of interactive research artifacts that serve as a temporary habitat for domestic silk moths, Bombyx mori. In the first part of the paper, we present the design journey from acquiring the eggs to harvesting movement in orchestrating the improvised ambient music and elaborate on the aspects of materiality and temporality that crafting the habitability involved. In the second part, we reflect on the alternative relationship as intended and embodied during the four months of co-habitation and discuss the ethical and other emergent issues informed by designer-researcher’s autoethnography.
Exploring the Information Needs of Somaliland Pastoralists: Design Considerations for Digital Climate Adaptation Services

Pastoralists in Somaliland are suffering the impacts of climate change, and need relevant, timely, and practical information about water resources and pastures. However, these communities are sparsely distributed and distant from decision-making centres, which impedes rapid dissemination of climate-related information that could allow them to build resilience to climate impacts. Although smartphone adoption rates are low, mobile services are widespread across Somaliland, presenting opportunities for supporting communities through mobile technologies. We interviewed 30 Somali pastoralists to understand their climate information needs, access to technology, and the potential for technology support. We also conducted a focus group with four Somali development experts to explore the cultural context and possible impacts of different design choices. Our results highlight and explore tensions around different design directions, and suggest possible ways to mitigate these. Designing systems that navigate such considerations can contribute to much-needed climate resilience for pastoralist communities in Somaliland and beyond.
Negotiating Dyadic Interactions through the Lens of Augmented Reality Glasses

Augmented Reality (AR) glasses separate dyadic interactions on different sides of the lens, where the person wearing the glasses (primary user) sees an AR world overlaid on their partner (secondary actor). The secondary actor interacts with the primary user understanding they are seeing both physical and virtual worlds. We use grounded theory to study interaction tasks, participatory design sessions, and in-depth interviews of 10 participants and explore how AR real-time modifications affect them. We observe a power imbalance attributed to the: (1) lack of transparency of the primary user’s view, (2) violation of agency over self-presentation, and (3) discreet recording capabilities of AR glasses. This information asymmetry leads to a negotiation of behaviors to reach a silently understood equilibrium. This paper addresses underlying design issues that contribute to power imbalances in dyadic interactions and offers nuanced insights into the dynamics between primary users and secondary actors.
Designing Wearable Technology for Opera

“RUR: A Torrent of Light” is an opera combining contemporary stage production with wearable electronic devices to create an immersive experience for the audience. In this paper we share the design and prototyping methods used to create the wearable devices, detail the variety of collaborations required to incorporate wearables into this kind of production, and reflect on the overall process as well as the results. We then share a set of recommendations based on our findings aimed to support future creatives and technologists developing wearable technology projects for a stage environment.
Identification and Classification of Human Body Exercises on Smart Textile Bands by Combining Decision Tree and Convolutional Neural Networks

In recent years, human activity recognition (HAR) has gained significant interest from researchers in the sports and fitness industries. In this study, the authors have proposed a cascaded method including two classifying stages to classify fitness exercises, utilizing a decision tree as the first stage and a one-dimension convolutional neural network as the second stage. The data acquisition was carried out by five participants performing exercises while wearing an inertial measurement unit sensor attached to a wristband on their wrists. However, only data acquired along the z-axis of the IMU accelerator was used as input to train and test the proposed model, to simplify the model and optimize the training time while still achieving good performance. To examine the efficiency of the proposed method, the authors compared the performance of the cascaded model and the conventional 1D-CNN model. The obtained results showed an overall improvement in the accuracy of exercise classification by the proposed model, which was approximately 92%, compared to 82.4% for the 1D-CNN model. In addition, the authors suggested and evaluated two methods to optimize the clustering outcome of the first stage in the cascaded model. This research demonstrates that the proposed model, with advantages in terms of training time and computational cost, is able to classify fitness workouts with high performance. Therefore, with further development, it can be applied in various real-time HAR applications.
Incorporating logic rules with textual representations for interpretable knowledge graph reasoning

Reasoning on knowledge graphs (KGs) is significant for downstream applications, such as question answering and information extraction. On the basis of using factual triples in KGs, learning logic rules has been explored to handle KG reasoning in an interpretable pattern. Some rule learning methods simply apply thresholds to reduce the exponential complexity caused by the vast candidate rules derived from KGs. Others are designed to prune candidates on the basis of embedding facts in the KG. However, these methods only consider the factual triples, which mainly contain structural information, and ignore sufficient semantic information in real-world KGs. This situation weakens the methods in searching for high-quality rules. To this end, we propose a joint method by incorporating logic rules with textual representations for interpretable KG reasoning, named LoTus. Firstly, LoTus obtains the structural embeddings with only factual triples, and the textual embeddings with descriptions of entities, which are later integrated into the representations of entities and relations. Secondly, top candidate rules are selected by a proposed joint pruning strategy incorporating structural and textual representations in KGs. Finally, LoTus captures high-quality rules and removes low-quality ones by a filtering process with metrics of rules. It reduces the time complexity on KG reasoning by learning rules and provides interpretability simultaneously. Extensive experiments on different datasets show that LoTus obtains competitive performance comparing to baselines on KG reasoning metrics Hits@1, Hits@10 and mean reciprocal rank (MRR), and achieves significant superiority among the recent rule learning methods.
Visualizing the Risks of De-anonymization in High-Dimensional Data

De-anonymization is a type of privacy breach in which an adversary identifies an individual in an anonymized dataset, thus gaining access to the individual’s private and sensitive data. To that end, data controllers who own such data and wish to distribute it to third parties have to be aware of the risks associated with their dataset and implement the appropriate anonymization measures. In order to conduct de-anonymization risk analyses, data controllers are in need of methods that can highlight risks, aid in regulatory compliance, and guide them through the anonymization process. The main drawbacks of existing de-anonymization analysis solutions are their insufficiency to deal with high-dimensional data as well as their limited visualization possibilities for users to evaluate the results of the analyses. The presented work introduces four novel risk analysis methods for four types of complex, high-dimensional data through intuitive and interactive visualizations while also improving on existing ones.
Causal reinforcement learning based on Bayesian networks applied to industrial settings

The increasing amount of real-time data collected from sensors in industrial environments has accelerated the application of machine learning in decision-making. Reinforcement learning (RL) is a powerful tool to find optimal policies for achieving a given goal. However, RL's typical application is risky and insufficient in environments where actions can have irreversible consequences and require interpretability and fairness. While new trends in RL may provide guidance based on expert knowledge, they do not often consider uncertainty or include prior knowledge in the learning process. We propose a causal reinforcement learning alternative based on Bayesian networks (RLBNs) to address this challenge. The RLBN simultaneously models a policy and takes advantage of the joint distribution of the state and action space, reducing uncertainty in unknown situations. We propose a training algorithm for the network's parameters and structure based on the reward function and likelihood of the effects and measurements taken. Our experiment with the CartPole benchmark and industrial fouling using ordinary differential equations (ODEs) demonstrates that RLBNs are interpretable, secure, flexible, and more robust than their competitors. Our contributions include a novel method that incorporates expert knowledge into the decision-making engine. It uses Bayesian networks with a predefined structure as a causal graph and a hybrid learning strategy that considers both likelihood and reward. This would avoid losing the virtues of the Bayesian network.
Complementary double pulse-width-modulation for 3D shape measurement of complex surfaces

Pulse-width modulation has the advantages of fast projection speed and high precision. However, when it is used to measure the three-dimensional (3D) shape of complex surfaces, it suffers from harmonic interference caused by slight defocusing and is sensitive to pixel oversaturation. To achieve high-speed and accurate 3D measurement of complex surfaces using this method, in this paper, a complementary double pulse-width-modulation (CDPWM) is proposed and used to generate ideal phase-shift fringe patterns. In our method, a triangular carrier signal with a zero phase is first used to generate a set of double pulse-width-modulation (DPWM) patterns, and then a triangular carrier signal with a phase of π/2 is used to generate another set of DPWM patterns. Then, the CDPWM is obtained by fusing these two sets of DPWM patterns. We demonstrate that CDPWM can tremendously suppress harmonic interference so that in practical applications, a smaller defocus can be used to robustly reconstruct complex surfaces. Moreover, it is proved that the patterns corresponding to the π/2 phase are the compensation patterns of the patterns corresponding to the zero phase, which enables CDPWM to eliminate or reduce the pixel oversaturation effect caused by a surface with a high dynamic range. Through experiments on simulated and real data, the proposed method is shown to substantially improve the phase quality and measurement accuracy over a wide range of fringe periods (e.g., from 18 to 912 pixels), especially when the defocus level is low. Different camera exposure times are used to demonstrate the robustness of the proposed method to oversaturated pixels. Furthermore, we measured a set of ice surfaces and a metal surface containing both low- and high-reflection areas. The projector was set to a slight defocus, which better preserved the binary characteristics of the patterns it projected. The measurement results proved our method can significantly improve the measurement accuracy of surfaces with complex reflection characteristics.
Expressing Robot’s Understanding of Human Preference Based on Successive Estimations during Dialog

AbstractAbstractConversational recommendation systems are crucial for making recommendations agreeable to the user. To reach an agreeable recommendation, this study proposes a dialog strategy that represents a reasonable order of items and elicits the current estimation by the wording of utterances based on the subjective preference estimation. We developed two dialog functions for the topic and word choice based on the history of preference estimations. The human impression of a robot's diligence, understanding capability, and satisfaction were evaluated through a conversation with a virtual robot using a crowdsourcing platform. We compared six conditions that differed based on two topics and three wording patterns. The experimental results indicated that the main effect of the wording patterns, whereas one of the topic choices was not found to be significant. Further analysis showed that accurate estimation improves the robot's impression when demonstrating its diligence.Keywords: Human-computer interactionDialog robotDialog strategyPreference estimation Disclosure statementThe authors report there are no competing interests to declare.Additional informationFundingThis work was supported by JST ERATO Grant Number JPMJER1401, JSPS KAKENHI Grant Number JP19H05691.Notes on contributorsKazuki SakaiKazuki Sakai received a Ph.D. degree in engineering science from Osaka University, Japan, in 2020. He is currently a specially appointed assistant professor of the Department of Systems Innovation in the Graduate School of Engineering Science at Osaka University. His research interests include human-robot interaction and dialogue systems.Yutaka NakamuraYutaka Nakamura received his Ph.D. from Nara Institute of Science and Technology in 2004. He held positions at Osaka University and other institutions and has been a team leader of Guardian Robot Project, RIKEN, Japan from 2020. His research focuses on the fields of machine learning and robotics.Yuichiro YoshikawaYuichiro Yoshikawa received the Ph.D. degree in engineering from Osaka University, Japan, in 2005. From 2010, he has been an associate professor of Graduate School of Engineering Science, Osaka University. He is interested in the study of human-robot communication, especially about multiple-robots coordination and psychiatric or educational applications.Shingo KanoShingo Kano received the Bachelor degree in engineering from Osaka University, Japan, 2018. He is interested in the human-robot interaction.Hiroshi IshiguroHiroshi Ishiguro received a D. Eng. in systems engineering from the Osaka University, Japan in 1991. He is currently Professor of Department of Systems Innovation in the Graduate School of Engineering Science at Osaka University (2009-). He is also visiting Director (2014-) of ATR Hiroshi Ishiguro Laboratories.Notes1 https://wordnet.princeton.edu/publications
FC-ResNet: A Multilingual Handwritten Signature Verification Model Using an Improved ResNet with CBAM

Offline signature verification is a widely used biometric method in finance, law, and administrative procedures. However, existing deep convolutional neural network models perform poorly on signature datasets that span different regions and ethnic people, while also suffering from problems such as large parameter counts and slow inference speeds. To address these issues, we propose an improved residual network model (FC-ResNet). This model introduces a convolutional block attention module into the classical residual network to adapt to the diversity and variability of signatures, while also compressing the model for lightweight deployment. Due to the lack of public, offline handwritten signature datasets for ethnic people, we collected a large-scale offline handwritten signature dataset, including genuine signatures and forged signatures in Chinese, Uyghur, Kazakh, and Kirgiz, totaling 38,400 images. Our FC-ResNet model achieved an accuracy of over 96% for each language in our self-built dataset, as well as accuracy rates of 96.21%, 98.42%, and 97.28% on the public datasets CEDAR, BHSig-B, and BHSig-H, respectively. Based on the above experimental results, our proposed model demonstrates great potential for both public and self-built signature datasets, while also exhibiting significant advantages in lightweight model deployment. We believe that this work can provide a feasible solution for ethnic people signature verification.
A Fast Density Peak Clustering Method for Power Data Security Detection Based on Local Outlier Factors

The basic work of power data research is anomaly detection. It is necessary to find a method suitable for processing current power system data. Research proposes an algorithm of fast density peak clustering with Local Outlier Factor (LOF). The algorithm has poor performance in processing datasets with irregular shapes and significant local density changes, and has the disadvantage of strong dependence on truncation distance. This study provides the decision rules for outliers incorporating the idea of LOF. The improved algorithm can fully consider the characteristics of power data and reduce the dependence on truncation distance. In anomaly detection based on the simulation of real power data, the classification accuracy of the improved CFSFDP algorithm is 4.87% higher than that of the traditional algorithm, and the accuracy rate is 97.41%. The missed and false detection rates of the LOF-CFSFDP algorithm are decreased by 2.23% and 2.64%, respectively, compared to the traditional algorithm, and it is ultimately able to reach rates of 1.26% and 1.33%. These results indicate that the algorithm proposed in this study can better describe the characteristics of power data, making the features of outliers and cluster center points more obvious.
Proof of Evaluation-based energy and delay aware computation offloading for Digital Twin Edge Network

The increasing availability of Internet of Things (IoT) applications has led to the development of new technologies. Specifically, the deployment of edge servers close to IoT devices has strengthened the edge computing paradigm. With the collaboration of Mobile Edge Computing (MEC) and cloud computing, delay-sensitive and computation-intensive tasks can be offloaded to the edge/cloud servers to improve system performance in terms of the delay and energy consumption of IoT devices. However, there is a need to schedule the computation tasks for an efficient management. More importantly, the task scheduling strategy can face data tampering attacks to deliberately modify, destroy or manipulate the decisions. To solve the above problems, in this paper, we newly propose to integrate digital twin and blockchain into the edge networks. However, it is unclear (i) how energy and delay-aware computation should be combined, and (ii) which mining computations should be executed for a secure task scheduling. The state-of-the-art focuses on task scheduling and blockchain mining, separately. Therefore, we propose a novel blockchain-based digital twin-edge network architecture where our proposed algorithm solves these two challenges at the same execution. We design a three-layer system architecture, composed of physical entity layer, digital twin edge layer and blockchain layer. In the physical entity layer, we formulate an energy and delay-aware task scheduling problem. In the digital twin edge layer, we propose a novel Proof of Evaluation (PoE)-based secure energy and delay-aware task scheduling algorithm where optimization is executed by the genetic algorithm implementation of Warehouse Location Problem (WLP). In the blockchain layer, the best-found solutions are shared with the topology in a blockchain. Here, each block includes the hash of the previous block, a genetic algorithm-based solution, nonce value, and a hash of whole blocks in the blockchain. Thus, we aim to execute the computation tasks with an acceptable delay in an energy-efficient manner and prevent data tampering attacks against the optimal computation decisions. We validate the outcomes of our PoE-based secure digital twin-edge network model with extensive evaluations. Since the proposed model distributes the task not only to the local device but also to the MEC and cloud server for delay awareness, it reduces the delay but consumes more energy. Nevertheless, the additional energy consumption can be neglected against the delay reduction. The proposed scheme is also more scalable to compare with the conventional solution. The numerical results clearly show that the proposed model provides energy and delay awareness, maintaining both data integrity and trustworthiness at the same execution of algorithm.