Reviewerly/exHarmony huggingface dataset:
CNN-RNN-CTC Based End-to-end Mispronunciation Detection and Diagnosis

This paper focuses on using Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and Connection-ist Temporal Classification (CTC) to build an end-to-end speech recognition for Mispronunciation Detection and Diagnosis (MDD) task. Our approach is end-to-end models, while phonemic or graphemic information, or forced alignment between different linguistic units, are not required. We conduct experiments that compare the proposed CNN-RNN-CTC approach with alternative mispronunciation detection and diagnoses (MDD) approaches. The F-measure of our approach is 74.65%, which significantly outperforms the Extended Recognition Network (ERN) (S-AM) by 44.75% and State-level Acoustic Model (S-AM) by 32.28% relatively. The relative improvement in F-measure when over Acoustic-Phonemic Model (APM), Acoustic-Graphemic Model (AGM) and Acoustic-Phonemic-Graphemic Model (APGM) are 9.57%, 5.04% and 2.77% respectively.
Researcher introspection for experience-driven design research

We challenge the unquestioning pursuit of the appearance of objectivity and ingrained designer-user dualism in human-centred design research and propose a resurrection of introspection as a valid approach to investigating subjective experiences. Through comparing epistemic perspectives and reviewing the histories of introspection in several disciplines, we liberate the research field of experience-driven design from a long-lasting doubt about and the disguised and unsystematic use of this method. To establish a foundation for the further development of introspective methods, we focus on its most controversial type (i.e. researcher introspection) and discuss its strengths and weaknesses, preconditions of use, diverse ways to practise for different suitable experience-driven design research purposes, and useful techniques and tools.
Measuring and Understanding Photo Sharing Experiences in Social Virtual Reality

Millions of photos are shared online daily, but the richness of interaction compared with face-to-face (F2F) sharing is still missing. While this may change with social Virtual Reality (socialVR), we still lack tools to measure such immersive and interactive experiences. In this paper, we investigate photo sharing experiences in immersive environments, focusing on socialVR. Running context mapping (N=10), an expert creative session (N=6), and an online experience clustering questionnaire (N=20), we develop and statistically evaluate a questionnaire to measure photo sharing experiences. We then ran a controlled, within-subject study (N=26 pairs) to compare photo sharing under F2F, Skype, and Facebook Spaces. Using interviews, audio analysis, and our questionnaire, we found that socialVR can closely approximate F2F sharing. We contribute empirical findings on the immersiveness differences between digital communication media, and propose a socialVR questionnaire that can in the future generalize beyond photo sharing.
Multi-Correlation Filters With Triangle-Structure Constraints for Object Tracking

Correlation filters (CFs) have been extensively used in tracking tasks due to their high efficiency although most of them regard the tracked target as a whole and are minimally effective in handling partial occlusion. In this study, we incorporate a part-based strategy into the framework of CFs and propose a novel multipart correlation tracker with triangle-structure constraints. Specifically, we train multiple CFs for the global object and local parts, which are then jointly applied to obtain the correlation response of any candidate during tracking. The tracker is robust in handling partial occlusion because of the use of part-based representation. The remaining global representation can contribute reliable cues in cases wherein several local filters drift away in a specific scene. We further propose a triangle-structure model to measure the structural similarity of candidates. The model employs multiple triangles to determine the spatial relationship among parts and helps constrain the location of the target. Moreover, we introduce an effective part selection scheme based on energy and integrity, which is generally applicable to part-tracking models. Extensive experiments on two public benchmarks demonstrate the superiority of the proposed method over the state-of-the-art approaches.
Co-Design Beyond Words

Existing co-design methods support verbal children on the autism spectrum in the design process, while their minimally-verbal peers are overlooked. We describe Co-Design Beyond Words (CDBW), an approach which merges existing co-design methods with practice-based methods from Speech and Language Therapy which are child-led and interests-based. These emphasise the rich detail that can be conveyed in the moment, through recognising occurrences of, for example, Joint Attention, Turn Taking and Imitation. We worked in an autism-specific primary school over 20 weeks with ten children, aged 5 to 8. We co-designed a playful prototype, the TangiBall, using the three iterative phases of CDBW; the Foundation Phase (preparation for interaction), the Interaction Phase (designing-and-reflecting in the moment) and the Reflection Phase (reflection-on-action). We contribute a novel co-design approach and present moments of interaction, the micro instances in design in which minimally-verbal children on the spectrum can convey meaning beyond words, through their actions, interactions, and attentional foci. These moments of interaction provide design insight, shape design direction, and reveal unique strengths, interests, and abilities.
Ethical Mediation in UX Practice

HCI scholars have become increasingly interested in describing the complex nature of UX practice. In parallel, HCI and STS scholars have sought to describe the ethical and value-laden relationship between designers and design outcomes. However, little research describes the ethical engagement of UX practitioners as a form of design complexity, including the multiple mediating factors that impact ethical awareness and decision-making. In this paper, we use a practice-led approach to describe ethical complexity, presenting three varied cases of UX practitioners based onin situ observations and interviews. In each case, we describe salient factors relating to ethical mediation, including organizational practices, self-driven ethical principles, and unique characteristics of specific projects the practitioner is engaged in. Using the concept of mediation from activity theory, we provide a rich account of practitioners' ethical decision making. We propose future work on ethical awareness and design education based on the concept of ethical mediation.
Do We Care About Diversity in Human Computer Interaction

In Human-Computer Interaction (HCI) research, awareness for the relevance of diversity of users is increasing. In this work, we analyze whether the articulated need for more diversity-sensitive research led indeed to a higher consideration of diversity in HCI research. Based on a comprehensive collection of diversity dimensions, we present results of a quantitative content analysis of articles accepted in the Proceedings of the Conference on Human Factors in Computing Systems 2006, 2011, and 2016. Results demonstrate how many and how intensively diversity dimensions were considered, and moreover highlight those dimensions that have so far received less attention. Uncovering continuous and discontinuous trends across time and differences between subfields of research, we identify research gaps and aim at contributing to a comprehensive understanding of diversity supporting diversity-sensitive research in HCI.
Deep-feature encoding-based discriminative model for age-invariant face recognition

Facial aging variation is a major problem for face recognition systems due to large intra-personal variations caused by age progression. A major challenge is to develop an efficient, discriminative feature representation and matching framework, which is robust to facial aging variations. In this paper, we propose a robust deep-feature encoding-based discriminative model for age-invariant face recognition. Our method learns high-level deep features using a pre-trained deep-CNN model. These features are then encoded by learning a codebook, which converts each of the features into a discriminant S-dimensional codeword for image representation. By incorporating the locality information in the whole learning process, a closed-form solution is obtained for both the codebook-updating and encoding stages. As the features of the same person at different ages should have certain correlations, canonical correlation analysis is utilized to fuse the pair of training features, for two different ages, to make the codebook discriminative in terms of age progression. In the testing stage, the gallery and query image's features are encoded using the learned codebook. Then, linear mapping based on linear regression is employed for face matching. We evaluate our method on three publicly available challenging facial aging datasets, FGNET, MORPH Album 2, and Large Age-Gap (LAG). Experimental results show that our proposed method outperforms various state-of-the-art age-invariant face recognition methods, in terms of the rank-1 recognition accuracy.
Assessing the Accuracy of Point &amp; Teleport Locomotion with Orientation Indication for Virtual Reality using Curved Trajectories

Room-scale Virtual Reality (VR) systems have arrived in users' homes where tracked environments are set up in limited physical spaces. As most Virtual Environments (VEs) are larger than the tracked physical space, locomotion techniques are used to navigate in VEs. Currently, in recent VR games, point & teleport is the most popular locomotion technique. However, it only allows users to select the position of the teleportation and not the orientation that the user is facing after the teleport. This results in users having to manually correct their orientation after teleporting and possibly getting entangled by the cable of the headset. In this paper, we introduce and evaluate three different point & teleport techniques that enable users to specify the target orientation while teleporting. The results show that, although the three teleportation techniques with orientation indication increase the average teleportation time, they lead to a decreased need for correcting the orientation after teleportation.
Hackathons as Participatory Design

Breastfeeding is not only a public health issue, but also a matter of economic and social justice. This paper presents an iteration of a participatory design process to create spaces for re-imagining products, services, systems, and policies that support breastfeeding in the United States. Our work contributes to a growing literature around making hackathons more inclusive and accessible, designing participatory processes that center marginalized voices, and incorporating systems- and relationship-based approaches to problem solving. By presenting an honest assessment of the successes and shortcomings of the first iteration of a hackathon, we explain how we re-structured the second "Make the Breast Pump Not Suck" hackathon in service of equity and systems design. Key to our re-imagining of conventional innovation structures is a focus on experience design, where joy and play serve as key strategies to help people and institutions build relationships across lines of difference. We conclude with a discussion of design principles applicable not only to designers of events, but to social movement researchers and HCI scholars trying to address oppression through the design of technologies and socio-technical systems.
A novel GA-ELM model for patient-specific mortality prediction over large-scale lab event data

Patient-specific mortality prediction models are an essential component of Clinical Decision Support Systems developed for caregivers in Intensive Care Units (ICUs), that enable timely decisions towards effective patient care and optimized ICU resource management. While high prediction accuracy is a fundamental requirement for any mortality prediction application, being able to so with minimal patient-specific data is a major plus point that can help in improving care delivery and cost optimization. Most existing scoring techniques and prediction models utilize a multitude of lab tests and patient events to predict mortality and also suffer from reduced performance when available patient data is less. In this paper, a Genetic Algorithm based Wrapper Feature Selection technique is proposed for determining most-optimal lab events that contribute predominantly to mortality, even for large-scale patient cohorts. Using this, an Extreme Learning Machine (ELM) based neural network is designed for predicting patient-specific ICU mortality. The proposed GA-ELM model was benchmarked against four popular traditional mortality scores and also state-of-the-art machine learning models for experimental validation. The GA-ELM model achieved promising results as it outperformed the traditional scoring systems by 11%–29% and state-of-the-art models by up to 14%, in terms of AUROC.
Deep Objective Quality Assessment Driven Single Image Super-Resolution

Single-image super-resolution (SISR) is a classic problem in the image processing community, which aims at generating a high-resolution image from a low-resolution one. In recent years, deep learning based SISR methods emerged and achieved a performance leap than previous methods. However, because the evaluation metrics of SISR methods is peak signal-to-noise ratio (PSNR), previous methods usually choose L2-norm as the loss function. This leads to a significant improvement in the final PSNR value but little improvement in perceptual quality. In this paper, in order to achieve better results in both perceptual quality and PSNR values, we propose an objective quality assessment driven SISR method. First, we propose a novel full-reference image quality assessment approach for SISR and employ it as a loss function, namely super-resolution image quality assessment (SR-IQA) loss. Then, we combine SR-IQA loss with L2-norm to guide our proposed SISR method to achieve better results. Besides that, our proposed SISR method consists of several proposed highway units. Furthermore, in order to verify the generalization ability of our new kind of loss function, we integrate SR-IQA loss to generative adversarial networks based SR method and achieve better perceptual quality. Experimental results prove that our proposed SISR method achieves better performance than other methods both qualitatively and quantitatively in most of the cases.
Adversarial Examples for CNN-Based Malware Detectors

The convolutional neural network (CNN)-based models have achieved tremendous breakthroughs in many end-to-end applications, such as image identification, text classification, and speech recognition. By replicating these successes to the field of malware detection, several CNN-based malware detectors have achieved encouraging performance without significant feature engineering effort in recent years. Unfortunately, by analyzing their robustness using gradient-based algorithms, several studies have shown that some of these malware detectors are vulnerable to the evasion attacks (also known as adversarial examples). However, the existing attack methods can only achieve quite low attack success rates. In this paper, we propose two novel white-box methods and one novel black-box method to attack a recently proposed malware detector. By incorporating the gradient-based algorithm, one of our white-box methods can achieve a success rate of over 99%. Without prior knowledge of the exact structure and internal parameters of the detector, the proposed black-box method can also achieve a success rate of over 70%. In addition, we consider adversarial training as a defensive mechanism in order to resist evasion attacks. While proving the effectiveness of adversarial training, we also analyze its security risk, that is, a large number of adversarial examples can poison the training dataset of the detector. Therefore, we propose a pre-detection mechanism to reject adversarial examples. The experiments show that this mechanism can effectively improve the safety and efficiency of malware detection.
Coordinated path planning for an unmanned aerial-aquatic vehicle (UAAV) and an autonomous underwater vehicle (AUV) in an underwater target strike mission

Unmanned system has become more and more popular as it can adapt to diverse environments and has prospective applications. Especially, the coordination among heterogeneous vehicles is capable of completing complicated tasks, which is often beyond the ability of homogeneous vehicles. In this paper, the underwater target strike mission is concentrated, and the mission is completed by the coordination between a UAAV and an AUV. UAAV and AUV are deployed in this mission because UAAV has strong search ability in air and can communicate with AUV directly after it dives into water. Firstly, to decompose the problem, the mission is divided into two phases, i.e., single flying of UAAV and underwater coordination between UAAV and AUV. In the coordinated path planning model, the motion of vehicles, the constraints in different media and the optimization index in each phase are all formulated into mathematical forms. Based on the particle swarm optimization (PSO) algorithm, the collocation points are used to determine the locations of control variables. Those points can reduce the computation load and improve the solution quality, and they are distributed by height and moment according to the forms of constraints in each phase. Besides, the strategy of addressing infeasible solutions is generated to guarantee the normal operation of PSO-based algorithm. Simulation results demonstrate that the proposed two-phase coordinated path planning method can generate coordinated paths, and the obtained results is very close to the optimal solution in theory. Compared to the whole method, the two-phase method can better deal with the complicated constraints in each phase.
An Intelligent Arabic Sign Language Recognition System Using a Pair of LMCs With GMM Based Classification

In this paper, we propose a dual leap motion controllers (LMC)-based Arabic sign language recognition system. More specifically, we introduce the concept of using both front and side LMCs to cater for the challenges of finger occlusions and missing data. For feature extraction, an optimum set of geometric features is selected from both controllers, while for classification, we used both a Bayesian approach with a Gaussian mixture model (GMM) and a simple linear discriminant analysis (LDA) approach. To combine the information from the two LMCs, we introduce an evidence-based fusion approach; namely, the Dempster-Shafer (DS) theory of evidence. Data were collected from two native adult signers, for 100 isolated Arabic dynamic signs. Ten observations were collected for each of the signs. The proposed framework uses an intelligent strategy to handle the case of missing data from one or both controllers. A recognition accuracy of about 92% was achieved. The proposed system outperforms state-of-the-art glove-based systems and single-sensor-based techniques.
A novel methodology for optimal single mobile robot scheduling using whale optimization algorithm

One of the fundamental requirements for creating an intelligent manufacturing environment is to develop a reliable, efficient and optimally scheduled material transport system. Besides traditional material transport solutions based on conveyor belts, industrial trucks, or automated guided vehicles, nowadays intelligent mobile robots are becoming widely used to satisfy this requirement. In this paper, the authors analyze a single mobile robot scheduling problem in order to find an optimal way to transport raw materials, goods, and parts within an intelligent manufacturing system. The proposed methodology is based on biologically inspired Whale Optimization Algorithm (WOA) and is aimed to find the optimal solution of the nondeterministic polynomial-hard (NP-hard) scheduling problem. The authors propose a novel mathematical model for the problem and give a mathematical formulation for minimization of seven fitness functions (makespan, robot finishing time, transport time, balanced level of robot utilization, robot waiting time, job waiting time, as well as total robot and job waiting time). This newly developed methodology is extensively experimentally tested on 26 benchmark problems through three experimental studies and compared to five meta-heuristic algorithms including genetic algorithm (GA), simulated annealing (SA), generic and chaotic Particle Swarm Optimization algorithm (PSO and cPSO), and hybrid GA–SA algorithm. Furthermore, the data are analyzed by using the Friedman statistical test to prove that results are statistically significant. Finally, generated scheduling plans are tested by Khepera II mobile robot within a laboratory model of the manufacturing environment. The experimental results show that the proposed methodology provides very competitive results compared to the state-of-art optimization algorithms.
SketchGAN: Joint Sketch Completion and Recognition With Generative Adversarial Network

Hand-drawn sketch recognition is a fundamental problem in computer vision, widely used in sketch-based image and video retrieval, editing, and reorganization. Previous methods often assume that a complete sketch is used as input; however, hand-drawn sketches in common application scenarios are often incomplete, which makes sketch recognition a challenging problem. In this paper, we propose SketchGAN, a new generative adversarial network (GAN) based approach that jointly completes and recognizes a sketch, boosting the performance of both tasks. Specifically, we use a cascade Encode-Decoder network to complete the input sketch in an iterative manner, and employ an auxiliary sketch recognition task to recognize the completed sketch. Experiments on the Sketchy database benchmark demonstrate that our joint learning approach achieves competitive sketch completion and recognition performance compared with the state-of-the-art methods. Further experiments using several sketch-based applications also validate the performance of our method.
A novel chaotic system and its application in a color image cryptosystem

In this paper, a new chaotic system in the form of the power-exponential structure is designed. The results of three types of analyses demonstrate that the proposed system can obtain many one-dimensional chaotic maps with a large chaotic range and uniform distribution. Then, a new color image cryptosystem is introduced using the four integrated chaotic maps that are based on the former system. Both the confusion and diffusion phases in our cryptosystem are related to those chaotic maps and the plain color image. Furthermore, we conduct many simulations and security analyses, and the results demonstrate that the proposed scheme is highly secure against common types of attacks.