Reviewerly/exHarmony huggingface dataset:
AI Student: A Machine Reading Comprehension System for the Korean College Scholastic Ability Test

Machine reading comprehension is a question answering mechanism in which a machine reads, understands, and answers questions from a given text. These reasoning skills can be sufficiently grafted into the Korean College Scholastic Ability Test (CSAT) to bring about new scientific and educational advances. In this paper, we propose a novel Korean CSAT Question and Answering (KCQA) model and effectively utilize four easy data augmentation strategies with round trip translation to augment the insufficient data in the training dataset. To evaluate the effectiveness of KCQA, 30 students appeared for the test under conditions identical to the proposed model. Our qualitative and quantitative analysis along with experimental results revealed that KCQA achieved better performance than humans with a higher F1 score of 3.86.
Discriminator Based Resilient Multi-agent Deep Deterministic Policy Gradient Under Uncertain Faulty Agents

Abstract In recent years, Multi-agent reinforcement learning (MARL) is widely applied in various of fields, to achieve a global goal in a centralized or distributed manner. However, during its application it is crucial to be fault-tolerance as some agents behave abnormal. In this paper, we propose a Resilient Multi-gent Deep Deterministic Policy Gradient (RMADDPG) algorithm to achieve a cooperative task in the presence of faulty agents via centralized training decentralized execution. At training stage, each normal agent observes and records information only from other normal ones, without access to the faulty ones. Meanwhile, a discriminator is generated based on the well-trained actor network to identify each faulty agent via supervised learning. Followed by executing stage, each normal agent selects its action based on local observation according to its actor network and its discriminator, so as to achieve certain system goal. Specifically, RMADDPG offers a scheme to train agents for improved resilience against arbitrary number of faulty agents. Finally, a cooperative navigation experiment is provided to validate the effectiveness of the proposed algorithm.
Towards Equitable Futures in Frontline Health:Design of Intelligent Systems for Supporting (Gendered) Care Work in Resource-Constrained Settings

Frontline health workers in many countries are responsible for filling gaps in essential primary health infrastructure, as witnessed during the COVID-19 pandemic. Their work increasingly involves the use of purportedly “intelligent” systems or data collection for such systems, to support diagnosis, disease forecasting, and information delivery. My research aims to inform the design of data-driven and automated systems in frontline health work, particularly for women workers in low-level and precarious roles in the Global South. Drawing from literature in the fields of human-computer interaction (HCI), gender and development studies, and health informatics, I will critically examine health workers’ experiences and relationships with “intelligent” systems, and engage in the participatory design of technology that might better serve worker needs while strengthening the frontline health ecology overall.
LoFi Sketch: A Large Scale Dataset of Smartphone Low Fidelity Sketches

Recent research on automating the transformation of low fidelity (LoFi) sketches to code using Deep Neural Networks require a large-scale dataset for generalizable results. This paper introduces the LoFi sketch dataset, an open-access dataset of hand-drawn smartphone LoFi sketches, to facilitate this research domain. It contains 4,527 LoFi sketches annotated with 21 categories of UI elements. Through annotations, it provides the category, position, and location of 41,560 constituent UI elements. We collected these sketches from 361 participants from 76 countries through questionnaires and by acquiring premade LoFi sketches to ensure the dataset’s diversity and ecological validity. These sketches were drawn using pen, pencil, mouse, touch, and stylus as the input medium and available in raster and vector format. This dataset enables further research on several avenues of AI assistance in UI design, such as UI element detection, sketch completion, UI layout refinement, and Sketch-based image retrieval.
Bridging Cultural Differences with Critical Design in a Globalized World

Is simplicity and minimalism the universal standard for interaction design? How can we avoid stereotyping with personas in design practices? What AI algorithms and design mechanism made “digital blackface” phenomenon on social media so popular? This interactive course teaches participants to reconsider some commonly held design beliefs and routine design practices with a lens of cultural differences. Illustrated with design case studies, it introduces strategies and techniques to turn differences into design resources for inclusivity. Participants will learn essential critical design skills of creating engaging and empowering designs in a globalized world at a divisive time.
Union is Strength: Improving face sketch synthesis by fusing outcomes of Fully-Convolutional-Networks and Random Sampling Locality Constraint

Automated formulation of sketches from face photos has seen successive growth since the work of Wang and Tang in recent years. Each new methodology is, however, able to partially achieve its objective of sketch synthesis while using pairs of photos and viewed sketches as a training medium. The viewed sketches are also used as a testing medium to determine the success of those methodologies. Resulting sketches do not fully capture all features of the training photos and viewed sketches. Their similarity value to respective sketch is also around 30 – 50%. One technique may produce sketches with sharp edges, but they do not bear completeness of facial features. Another technique produces sketches with the completeness of facial elements, but they are not well-focused. Second limitation of existing techniques is attributable to face-recognition procedure which is used as a validation step for these methodologies. Face-recognition process with help of synthesized sketches delivers reliable results over datasets with a limited diversity of age, ethnicity, and light intensities. We propose a novel and cost-effective approach to fuse resulting sketches of two test techniques. The two techniques are merged to yield a better sketch containing well-defined features, sharp contours, and less noise. Secondly, fusion suppresses limitations of the component methodologies reaching the resulting sketch. To test this idea of combining sketch-synthesis methods, we experiment with the most basic techniques of image fusion including simple (arithmetic), PCA, and Wavelet based fusions. The proposed setup considered FCN (complete features but less sharpness) and Fast-RSLCR (sharp edges but missing contours) as candidate techniques. It is tested on two datasets namely CUFS and CUFSF. Second dataset incorporates variations of age, ethnicity, light intensities, and slightly deformed features between photos and viewed sketches. Our results indicate achievement of 60.29% SSIM score (enhancement by 3.84%) and 79.03% face-recognition score (enhancement by 5.62%) as compared to Fast-RSLCR.
“How much connection did you get?” Drinking Journaling Data through Hot Cocoa

What if you could get a taste of your own thoughts? We propose a new system in HCI that uses computational tools to map text onto flavors and thus allow users to ”eat their words” in the form of a custom hot cocoa blend created from written journal entries, helping people engage in reflection. Our more open-ended approach to computer-augmented reflection explores new tangible mediums and methods to facilitate a sense of playful fluidity in machine interactions. A deep learning pipeline is developed that finds meaning within user journals by identifying similar poems/stories/texts in a pre-collected corpus. A dispenser outputs a drink mix with flavors based on these discovered meanings, and prints a receipt to help users make sense of their experience. To encourage disclosure and create a reflective mood, we model our experience based on journaling at an intimate cafe. Our preliminary evaluation demonstrates strong promise in this approach.
A novel deep learning-based recommender system for Arabic content

In the era of big data, recommender systems (RSs) have become growing essential tools. They represent important machine learning solutions that mainly contribute to keeping users engaged with personalized content in e-platforms. Several RSs have been proposed in the literature, and most of them have focused on English content. However, for content in other languages like Arabic, very restricted works have been done to develop RSs. In recent times, the Arabic content on the Web has increased significantly because of the growing number of Arabic web users. This highlights the need for building RSs for Arabic content. To better handle this challenge, we decided to provide the research community with a novel deep learning (DL)-based RS devoted to Arabic content. The main goal of the proposed RS is to predict user preferences from textual reviews written in the Arabic language. This is achieved by combining two independent DL techniques into one system: a convolutional neural network (CNN)-text processor for representing users and items; and a neural network, in particular, a multi-layer perceptron (MLP) to estimate interactions between user-item pairs. Extensive experiments on four large-scale Arabic datasets demonstrate that our proposed system can achieve better prediction accuracy than other state-of-the-art alternatives. Notably, it improves the MSE between 0.84% and 16.96%, and the MAE between 0.14% and 13.71%. This work is the first attempt designed to deal with a large volume of data in the Arabic context, opening up new research possibilities for future developments of Arabic RSs.
“A Voice that Suits the Situation”: Understanding the Needs and Challenges for Supporting End-User Voice Customization

Although there is a potential demand for customizing voices, most customization is limited to the visual appearance of a figure (e.g., avatars). To better understand the users’ need, we first conducted an online survey with 104 participants. Then we conducted a semi-structured interview with a prototype with 14 participants to identify design considerations for supporting voice customization. The results show that there is a desire for voice customization especially for non-face-to-face conversations with someone unfamiliar. In addition, the findings revealed that different voices are favored for different contexts from a better version of one’s own voice for improving delivery to a completely different voice for securing identity. As future work, we plan to extend this study by investigating voice synthesis techniques for end-users who wish to design their own voices for various contexts.
A Method based on Multi-agent Systems and Passive Replication Technique for Predicting Failures in Cloud Computing

Background: Cloud computing refers to the computing capacities of remote computers, where the user has considerable computing power without having to own power units. The probability of failures, which can occur during execution, increases in the number of nodes. Since failures cannot be completely avoided, one solution is to use failure tolerance mechanisms. Predicting failures has become a major task for engineers and software developers, as failure increases resource usage costs. Objectives and Methods: This paper presents a hybrid method of predicting failures in a cloudcomputing environment based on the passive replication technique and multi-agent systems. It detects failures, improves the average response time and minimizes lost time. This method makes it possible to efficiently and transparently guarantee the continuity of cloud computing services in the presence of failures. Results: The results show that the proposed method performs well in the presence of failures, improves the response time and minimizes the additional costs caused by the failures. Conclusion: This paper proposes a hybrid method of predicting failures in cloud-computing based on the passive replication technique and multi-agent systems to detect failures and minimize lost time. The replication technique works by duplicating some system components, which are deployed simultaneously across different resources. This technique aims to make the system robust, increase availability and guarantee the execution of jobs. In addition, it is suitable for long-running tasks.
The design of a complex weather environment simulation system for evaluating imaging detection efficiency

Common electronic imaging can be easily affected by some weather factors. In order to explore the influence of several main weather factors in Lab circumstance, it is necessary to develop a weather simulation system for evaluating imaging detection efficiency. In this paper, a design and manufacture of a relatively complex weather environment simulation system for evaluating imaging detection operation is introduced. The proposed system can be used to simulate several common weather conditions such as rain, fog, air flow, and temperature variance, in a small spatial size of 60×45×45cm, which covers a large spectral range from the visible to infrared wavelength bands of 0.38μm~14μm and can be utilized to conveniently explore the adverse effects of the weather factors mentioned above on imaging detection. Firstly, the feasibility for simulating different factors including rain, fog, air flow, and temperature variance, is demonstrated. The overall system is designed and realized based on the corresponding electrical and optical and mechanical parameters. The velocity distribution of air flow is simulated by Fluent. The components of the system are made and assembled by traditional processing methods such as Computer Numerical Control. Finally, the overall system is tested, and the results show that the design goals have been attained well. The system can be applied to simulate imaging detection experiments.
Personas

Personas has evolved since Alan Cooper coined the term in 1999, moving into new domains, new ways of collecting data, and with novel ways of presenting the persona profiles. From the beginning, personas was linked to software design, expressing the need for empathy with end-users. This is still the case today, but we want to show how this is executed in different domains, not only in software, and how different forms of presentation relate to empathy. Thus, the persona course investigates the relationship between data collection, the representation of data as persona profiles, and empathy.
BoldMove: Enabling IoT Device Control on Ubiquitous Touch Interfaces by Semantic Mapping and Sequential Selection

Recent advances in ultra-low-power ubiquitous touch interfaces make touch inputs possible anytime, anywhere. However, their functions are usually pre-determined, i.e., one button is only associated with one fixed function. BoldMove enables spontaneous and efficient association of touch inputs and IoT device functions with semantic-based function filtering and a wait-confirm sequential selection strategy. In this way, such touch interfaces become ubiquitous IoT device controllers. We proposed the semantic-based IoT function filtering to improve control efficiency, then designed the sequential selection mechanism for interfaces with constrained input and output resources. We implemented BoldMove on a custom-built touch interface with capacitive button inputs and a smartwatch display. We then conducted a user study to determine the design parameters for the sequential selection method. At last, we validated that BoldMove only takes 3.25 seconds to complete a selection task if the target function appears within the Top-3 displayed item. Even if the assumption is relaxed to Top-10, BoldMove is still estimated to be more efficient than the conventional selection method with device-based filtering and menu-navigated selection.
A self-supervised feature selection algorithm for spectral embedding based on block HSIC lasso

Abstract With the rapid development of information technology, a large amount of unlabeled high-dimensional data has been generated. To be able to better handle these data, we propose a new self-supervised feature selection algorithm for spectral embedding based on block HSIC lasso (FSSBH). It innovatively applies the HSIC theoretical approach to unlabeled scenarios for feature importance assessment, and performs feature selection by self-supervised learning with the pseudo-label matrix formed by the spectral embedding. The algorithm is compared with five popular algorithms for classification experiments on seven publicly available datasets. The results show the superior performance of the FSSBH algorithm.
Interactive Research Artifacts

New technologies and digitization have the potential to vastly alter our knowledge infrastructures. Specifically, this work focuses on the effects of interactive technologies on research practices, referred as “interactive research artifacts.” Current research investigates the communicative affordances of such technologies, but there exists minimal work which critically examines the creative ways scholars are engaging with these artifacts. Through in-depth interviews with 14 scholars, and Design Studies literature, this work arrives at an understanding of interactive research artifacts as creative knowledge creation tools, rather than simply communicative tools. As such, to design for a future where interactive research artifacts become commonly used scholarly tools, an in-depth understanding of the ways these artifacts are used as knowledge creation tools is critical.
A Case Study on the Rapid Development of Natural and Synergistic Multimodal Interfaces for XR Use-Cases

Multimodal Interfaces (MMIs) supporting the synergistic use of natural modalities like speech and gesture have been conceived as promising for spatial or 3D interactions, e.g., in Virtual, Augmented, and Mixed Reality (XR for short). Yet, the currently prevailing user interfaces are unimodal. Commercially available software platforms like the Unity or Unreal game engines simplify the complexity of developing XR applications through appropriate tool support. They provide ready-to-use device integration, e.g., for 3D controllers or motion tracking, and according interaction techniques such as menus, (3D) point-and-click, or even simple symbolic gestures to rapidly develop unimodal interfaces. A comparable tool support is yet missing for multimodal solutions in this and similar areas. We believe that this hinders user-centered research based on rapid prototyping of MMIs, the identification and formulation of practical design guidelines, the development of killer applications highlighting the power of MMIs, and ultimately a widespread adoption of MMIs. This article investigates potential reasons for the ongoing uncommonness of MMIs. Our case study illustrates and analyzes lessons learned during the development and application of a toolchain that supports rapid development of natural and synergistic MMIs for XR use-cases. We analyze the toolchain in terms of developer usability, development time, and MMI customization. This analysis is based on the knowledge gained in years of research and academic education. Specifically, it reflects on the development of appropriate MMI tools and their application in various demo use-cases, in user-centered research, and in the lab work of a mandatory MMI course of an HCI master’s program. The derived insights highlight successful choices made as well as potential areas for improvement.
Unsupervised Video Summarization Based on An Encoder-Decoder Architecture

Abstract The purpose of video summarization is to facilitate large-scale video browsing. Video summarization is a short and concise synopsis of original video. It usually composed of a set of representative video frames from the original video. This paper solves the problem of unsupervised video summarization by developing a Video Summarization Network (VSN) to summarize videos, which is formulated as selecting a sparse subset of video frames that best represents the input video. VSN predicts a probability for each video frame, which indicates the possibility of a frame being selected, and then takes actions to select frames according to the probability distribution to form a video summary. We designed a novel loss function which takes into account the diversity and representativeness of the generated summarization without labels or user interaction.
Jointly Optimizing Expressional and Residual Models for 3D Facial Expression Removal

This article proposes a facial expression removal method to recover a 3D neutral face from a single 3D expressional or non-neutral face. We treat a 3D non-neutral face as the sum of its neutral one and the residual. This can be satisfied if the correspondence between 3D vertices of expressional faces and those of neutral faces is established. We propose a non-rigid deformation method to establish the correspondence between 3D faces. Then, according to algebra inequality, the minimization of a neutral face model can be replaced by the minimization of its upper bound, i.e., the errors of an expressional face model and a residual model. Thus, we co-optimize the representation errors of the latter two models and build the relationship between the representation coefficients of the two models. Given an expressional face as the input, its corresponding neutral face can be inferred by the associative representation parameters in these two models. In the testing stage, we use an iterative joint fitting scheme to obtain a more accurate recovery. Extensive experiments are conducted to evaluate our method. The results show that our method obtains considerably better performance than existing methods in terms of average root mean square errors and recognition rates, and also better visual effects.