Reviewerly/exHarmony huggingface dataset:
Walk The Line: Leveraging Lateral Shifts of the Walking Path as an Input Modality for Head-Mounted Displays

Recent technological advances have made head-mounted displays (HMDs) smaller and untethered, fostering the vision of ubiquitous interaction in a digitally augmented physical world. Consequently, a major part of the interaction with such devices will happen on the go, calling for interaction techniques that allow users to interact while walking. In this paper, we explore lateral shifts of the walking path as a hands-free input modality. The available input options are visualized as lanes on the ground parallel to the user's walking path. Users can select options by shifting the walking path sideways to the respective lane. We contribute the results of a controlled experiment with 18 participants, confirming the viability of our approach for fast, accurate, and joyful interactions. Further, based on the findings of the controlled experiment, we present three example applications.
Privacy and Power: Acknowledging the Importance of Privacy Research and Design for Vulnerable Populations

Privacy researchers and designers must take into consideration the unique needs and challenges of vulnerable populations. Normative and privileged lenses can impair conceptualizations of identities and privacy needs, as well as reinforce or exacerbate power structures and struggles-and how they are formalized within privacy research methods, theories, designs, and analytical tools. The aim of this one-day workshop is to facilitate discourse around alternative ways of thinking about privacy and power, as well as ways for researching and designing technologies that not only respect the privacy needs of vulnerable populations but attempt to empower them. We will work towards developing best practices to help academics and industry folks, technologists, researchers, policy makers, and designers do a better job of serving the privacy needs of vulnerable users of technology.
Knowledge mobilization of human–computer interaction for development research: core issues and domain questions

(2020). Knowledge mobilization of human–computer interaction for development research: core issues and domain questions. Information Technology for Development: Vol. 26, Outrage and Anger in a Global Pandemic: Flipping the Script on Healthcare, pp. 551-576.
Quantification of Users' Visual Attention During Everyday Mobile Device Interactions

We present the first real-world dataset and quantitative evaluation of visual attention of mobile device users in-situ, i.e. while using their devices during everyday routine. Understanding user attention is a core research challenge in mobile HCI but previous approaches relied on usage logs or self-reports that are only proxies and consequently do neither reflect attention completely nor accurately. Our evaluations are based on Everyday Mobile Visual Attention (EMVA) – a new 32-participant dataset containing around 472 hours of video snippets recorded over more than two weeks in real life using the front-facing camera as well as associated usage logs, interaction events, and sensor data. Using an eye contact detection method, we are first to quantify the highly dynamic nature of everyday visual attention across users, mobile applications, and usage contexts. We discuss key insights from our analyses that highlight the potential and inform the design of future mobile attentive user interfaces.
Automation of the Northwestern Narrative Language Analysis System

Purpose Analysis of spontaneous speech samples is important for determining patterns of language production in people with aphasia. To accomplish this, researchers and clinicians can use either hand coding or computer-automated methods. In a comparison of the two methods using the hand-coding NNLA (Northwestern Narrative Language Analysis) and automatic transcript analysis by CLAN (Computerized Language Analysis), Hsu and Thompson (2018) found good agreement for 32 of 51 linguistic variables. The comparison showed little difference between the two methods for coding most general (i.e., utterance length, rate of speech production), lexical, and morphological measures. However, the NNLA system coded grammatical measures (i.e., sentence and verb argument structure) that CLAN did not. Because of the importance of quantifying these aspects of language, the current study sought to implement a new, single, composite CLAN command for the full set of 51 NNLA codes and to evaluate its reliability for coding aphasic language samples. Method Eighteen manually coded NNLA transcripts from eight people with aphasia and 10 controls were converted into CHAT (Codes for the Human Analysis of Talk) files for compatibility with CLAN commands. Rules from the NNLA manual were translated into programmed rules for CLAN computation of lexical, morphological, utterance-level, sentence-level, and verb argument structure measures. Results The new C-NNLA (CLAN command to compute the full set of NNLA measures) program automatically computes 50 of the 51 NNLA measures and generates the results in a summary spreadsheet. The only measure it does not compute is the number of verb particles. Statistical tests revealed no significant difference between C-NNLA results and those generated by manual coding for 44 of the 50 measures. C-NNLA results were not comparable to manual coding for the six verb argument measures. Conclusion Clinicians and researchers can use the automatic C-NNLA to analyze important variables required for quantification of grammatical deficits in aphasia in a way that is fast, replicable, and accessible without extensive linguistic knowledge and training.
Similarity-based analyses on software applications: A systematic literature review

In empirical studies on processes, practices, and techniques of software engineering, automation and machine learning are gaining popularity. In order to extract knowledge from existing software projects, a sort of similarity analysis is often performed using different methodologies, data and metadata. This systematic literature review focuses therefore on existing approaches of similarity-, categorization- and relevance-based analysis on software applications. In total, 136 relevant publications and patents were identified between 2002 and 2019 according to the established inclusion and exclusion criteria, which perform a calculation of software similarity in general or to support certain software engineering phases.
Resource efficient sensor fusion by knowledge-based network pruning

The perception of the environment is key for autonomous driving applications. To increase the accuracy of perception in different environmental contexts vehicles can rely on both camera and LiDAR sensors that provide complementary information about the same features. Therefore, a sensor fusion method can improve the detection accuracy by combining the information of both sensors. Recently, many sensor fusion methods have been proposed that rely on deep neural networks that typically require a lot of resources to be executed in real-time. Therefore, we propose a resource efficient sensor fusion approach with a new neural network optimization method called knowledge-based pruning. The general principle is to prune the neural network guided by the location of the knowledge within the network that is unveiled with explainable AI methods. More specifically, in this work we propose a pruning method that uses layer-wise relevance propagation (LRP) to localize the network knowledge. The considered sensor fusion method uses off-the-shelve pretrained networks which we optimize for our application using the LRP pruning method. This can be used as a form of transfer learning as a pretrained model is optimized to be applied for a subset of the tasks it was originally trained for.
Weighted Adaptive Image Super-Resolution Scheme Based on Local Fractal Feature and Image Roughness

Image super-resolution aims to reconstruct a high-resolution image from the known low-resolution version. During this process, it should keep the degree of image roughness non-decreasing, which reflects various texture features and appearance. However, this point is not well addressed in the current work. This work argues that reducing roughness during image super-resolution is the key reason causing various problems such as artificial texture and/or edge blur. In this work, keeping the image roughness non-decreasing during super-resolution is being well investigated for the first time to our best knowledge. Image super-resolution is cast as an optimization problem to keep image roughness non-decreasing. In order to tackle this problem, the image super-resolution is approached based on the theory of fractal, where adaptive fractal interpolation function is proposed. In this way, the rational fractal interpolation model is adaptive to every local region. Thus, the roughness of every image region can be best maintained while super-resolution is carried out through fractal interpolation. In this work, the image roughness is reflected by the fractal dimension, which is a key element affecting the construction of fractal interpolation model. That is, the image roughness is measurable using fractal dimension. Mathematically, the overall image super-resolution process can be converted into a fractal interpolation optimization problem where the local fractal dimension is maintained. Although adaptive super-resolution on image segments may best maintain image roughness using the proposed method, it still generates unnecessary block artifacts. To tackle this problem, this work proposes a fine-grained pixel-wise fractal function. Our extensive experimental results demonstrate that the proposed method achieves encouraging performance with the state-of-the-art super-resolution algorithms.
Designing and Evaluating 'In the Same Boat', A Game of Embodied Synchronization for Enhancing Social Play

Social closeness is important for health and well-being, but is difficult to maintain over a distance. Games can help connect people by strengthening existing relationships or creating new ones through shared playful experiences. We present the design and evaluation of 'In the Same Boat' (ITSB), a two-player infinite runner designed to foster social closeness in distributed dyads. ITSB leverages the synchronization of both players' input to steer a canoe down a river and avoid obstacles. We created two versions: embodied controls, which use players' physiological signals (breath rate, facial expressions), and standard keyboard controls. Results from a study with 35 dyads indicate that ITSB fostered affiliation, and while embodied controls were less intuitive, people enjoyed them more. Further, photos of the dyads were rated as happier and closer in the embodied condition, indicating the potential of embodied controls to foster social closeness in synchronized play over a distance.
HCI Across Borders and Sustainable Development Goals

As HCI Across Borders aspires to celebrate its fifth year at CHI, and the CHI 2020 venue of Hawaii signifies a coming together of four continents, the goal of the 2020 symposium is to bring our focus to themes that unify and foster solidarity across borders. Thus we select the United Nations' Sustainable Development Goals as our object of study. Many communities within CHI focus on the constrained and ephemeral nature of resources, including the HCI for Development (HCI4D), Sustainable HCI (SHCI), and Crisis Informatics (CI) communities, among several others. We contend that it is time for these communities to come together in addressing issues of global relevance and impact, and for many more to care. Additionally, as the venue for CHI shifts to Asia in 2021, we aspire to prepare the conference and its participants to grapple with themes that might offer a different and novel perspective when engaged within the Global South.
Gender based face aging with cycle-consistent adversarial networks

Face aging is a task which referred to image synthesis, and the challenge comes from the training dataset, most existing face aging works require paired face images which is difficult to collect. Face images with various ages of the same person can be considered as unpaired images which come from different domains. The degree of aging effect can be influenced by age, gender, race and some other factors. In this paper, we are committed to studying the impact of gender on face aging problem, which involves the processing and modeling of face images. It has been proved that Generative Adversarial Networks(GANs) is competitive in realistic image synthesis, and many works employed Cycle-Consistent Adversarial Networks(CycleGANs) have shown the high performance in unpaired image-to-image translation. To overcome current difficulties and improve the performance of existing models on the face aging tasks, we proposed an innovative Gender-based training method using CycleGAN by pairwise training CycleGAN over several age groups which are grouped by age and gender. We build a constraint model based on gender discrimination to better simulate the expected aging effect of face images. To evaluate our works using subjective method, we have set up a quantitative evaluation mechanism with participants involved in. Compared with other similar subjective evaluation methods, our method is more objective in the demonstration of experimental results. The experimental results show that our method has a more realistic and excellent performance compared to those using CycleGAN for face age synthesis directly.
An Efficient Attribute-Based Multi-Keyword Search Scheme in Encrypted Keyword Generation

With the growing popularity of cloud computing in recent years, data owners (DOs) now prefer to outsource their data to cloud servers and allow the specific data users (DUs) to retrieve the data. Searchable encryption is an important tool to provide secure search over the encrypted cloud data without infringing data confidentiality and data privacy. In this work, we consider a secure search service providing fine-grained and search functionality, called attribute-based multiple keyword search (ABMKS), which can be seen as an extension of searchable encryption. In the existing ABMKS schemes, the computation operations in the encrypted keyword index generation are time-consuming modular exponentiation, and the number of which is linearly growing with the factor $m$ . Here $m$ is the number of keywords embedded in a file. To reduce the computation overhead, in this paper, we propose an ABMKS with only multiplication operations in encrypted keyword index generation. As a result, the computation cost of the encrypted keyword index generation is more efficient than the existing schemes. In addition, the encrypted keyword indexes are aggregated into one item, which is regardless of the number of underlying keywords in a file data. Finally, the security and the performance analysis demonstrate that our scheme is both efficient and secure.
Billion-scale Recommendation with Heterogeneous Side Information at Taobao

In recent years, embedding models based on skip-gram algorithm have been widely applied to real-world recommendation systems (RSs). When designing embedding-based methods for recommendation at Taobao, there are three main challenges: scalability, sparsity and cold start. The first problem is inherently caused by the extremely large numbers of users and items (in the order of billions), while the remaining two problems are caused by the fact that most items have only very few (or none at all) user interactions. To address these challenges, in this work, we present a flexible and highly scalable Side Information (SI) enhanced Skip-Gram (SISG) framework, which is deployed at Taobao. SISG overcomes the drawbacks of existing embedding-based models by modeling user metadata and capturing asymmetries of user behavior. Furthermore, as training SISG can be performed using any SGNS implementation, we present our production deployment of SISG on a custom-built word2vec engine, which allows us to compute item and SI embedding vectors for billion-scale sets of products in a join semantic space on a daily basis. Finally, using offline and online experiments we demonstrate the significant superiority of SISG over our previously deployed framework, EGES, and a well-tuned CF, as well as present evidence supporting our scalability claims.
Modular Graphical Ontology Engineering Evaluated

Ontology engineering is traditionally a complex and time-consuming process, requiring an intimate knowledge of description logic and predicting non-local effects of different ontological commitments. Pattern-based modular ontology engineering, coupled with a graphical modeling paradigm, can help make ontology engineering accessible to modellers with limited ontology expertise. We have developed CoModIDE, the Comprehensive Modular Ontology IDE, to develop and explore such a modeling approach. In this paper we present an evaluation of the CoModIDE tool, with a set of 21 subjects carrying out some typical modeling tasks. Our findings indicate that using CoModIDE improves task completion rate and reduces task completion time, compared to using standard Protégé. Further, our subjects report higher System Usability Scale (SUS) evaluation scores for CoModIDE, than for Protégé. The subjects also report certain room for improvements in the CoModIDE tool – notably, these comments all concern comparatively shallow UI bugs or issues, rather than limitations inherent in the proposed modeling method itself. We deduce that our modeling approach is viable, and propose some consequences for ontology engineering tool development.
SpherePHD: Applying CNNs on 360 Images With Non-Euclidean Spherical PolyHeDron Representation

Omni-directional images are becoming more prevalent for understanding the scene of all directions around a camera, as they provide a much wider field-of-view (FoV) compared to conventional images. In this work, we present a novel approach to represent omni-directional images and suggest how to apply CNNs on the proposed image representation. The proposed image representation method utilizes a spherical polyhedron to reduce distortion introduced inevitably when sampling pixels on a non-Euclidean spherical surface around the camera center. To apply convolution operation on our representation of images, we stack the neighboring pixels on top of each pixel and multiply with trainable parameters. This approach enables us to apply the same CNN architectures used in conventional euclidean 2D images on our proposed method in a straightforward manner. Compared to the previous work, we additionally compare different designs of kernels that can be applied to our proposed method. We also show that our method outperforms in monocular depth estimation task compared to other state-of-the-art representation methods of omni-directional images. In addition, we propose a novel method to fit bounding ellipses of arbitrary orientation using object detection networks and apply it to an omni-directional real-world human detection dataset.
A Fast Side-Channel Leakage Simulation Technique Based on IC Chip Power Modeling

Hardware-based secure integrated circuit (IC) chips with implemented cryptographic algorithms are vulnerable to practical attacks analyzing side-channel (SC) leakage such as electromagnetic radiation. Correlation power analysis (CPA) is a real threat to secret key crypto ICs. Therefore, the information leakage risks need to be evaluated at the design stage of secure devices. This letter proposes a fast power leakage simulation method for hardware-implemented cryptographic ICs. The power delivery network (PDN) of cryptographic hardware including a silicon substrate is modeled by a chip power model (CPM) and a chip package system (CPS) board model. The proposed method was applied to the advanced encryption standard (AES) test chip implemented by the flip-chip ball grid array (FC-BGA) assembly technology. The simulation results are evaluated by CPA for the SC leakage during plaintext encryption propagating through PDN and silicon substrate. The measurement and simulation successfully find 128-bit secret key on CPA attack. The proposed simulation technique reduces the time required in the exploration of SC leakage risks for secure device designs.
Progressive Cross-Stream Cooperation in Spatial and Temporal Domain for Action Localization

Spatio-temporal action localization consists of three levels of tasks: spatial localization, action classification, and temporal localization. In this work, we propose a new progressive cross-stream cooperation (PCSC) framework that improves all three tasks above. The basic idea is to utilize both spatial region (resp., temporal segment proposals) and features from one stream (i.e., the Flow/RGB stream) to help another stream (i.e., the RGB/Flow stream) to iteratively generate better bounding boxes in the spatial domain (resp., temporal segments in the temporal domain). In this way, not only the actions could be more accurately localized both spatially and temporally, but also the action classes could be predicted more precisely. Specifically, we first combine the latest region proposals (for spatial detection) or segment proposals (for temporal localization) from both streams to form a larger set of labelled training samples to help learn better action detection or segment detection models. Second, to learn better representations, we also propose a new message passing approach to pass information from one stream to another stream, which also leads to better action detection and segment detection models. By first using our newly proposed PCSC framework for spatial localization at the frame-level and then applying our temporal PCSC framework for temporal localization at the tube-level, the action localization results are progressively improved at both the frame level and the video level. Comprehensive experiments on two benchmark datasets UCF-101-24 and J-HMDB demonstrate the effectiveness of our newly proposed approaches for spatio-temporal action localization in realistic scenarios.
Embedding-Based Recommendations on Scholarly Knowledge Graphs

The increasing availability of scholarly metadata in the form of Knowledge Graphs (KG) offers opportunities for studying the structure of scholarly communication and evolution of science. Such KGs build the foundation for knowledge-driven tasks e.g., link discovery, prediction and entity classification which allow to provide recommendation services. Knowledge graph embedding (KGE) models have been investigated for such knowledge-driven tasks in different application domains. One of the applications of KGE models is to provide link predictions, which can also be viewed as a foundation for recommendation service, e.g. high confidence “co-author” links in a scholarly knowledge graph can be seen as suggested collaborations. In this paper, KGEs are reconciled with a specific loss function (Soft Margin) and examined with respect to their performance for co-authorship link prediction task on scholarly KGs. The results show a significant improvement in the accuracy of the experimented KGE models on the considered scholarly KGs using this specific loss. TransE with Soft Margin (TransE-SM) obtains a score of 79.5% Hits@10 for co-authorship link prediction task while the original TransE obtains 77.2%, on the same task. In terms of accuracy and Hits@10, TransE-SM also outperforms other state-of-the-art embedding models such as ComplEx, ConvE and RotatE in this setting. The predicted co-authorship links have been validated by evaluating profile of scholars.