Reviewerly/exHarmony huggingface dataset:
Nurse care activity recognition

Human activity recognition is a challenging task due to complexity and variations of human movements while performing activities by different subjects. Extracting features to model the temporal evolution of different movements plays an important role in this task. In this paper, we present the approach followed by our team, Dark_Shadow, to recognize complex nurse activities in the "Nurse Care Activity Recognition Challenge" [1]. We present a deep learning method to capture the movements of essential body parts from time series of human activity data collected by sensors and then classify them. Deep learning approaches have provided satisfactory results in various human activity recognition tasks. In this work, we propose a Gated Recurrent Unit (GRU) model with attention mechanism to recognize the nurse activities. We obtain approximately 66.43% accuracy for person-wise one leave out cross validation.
Dynamic Gesture Recognition Using Inertial Sensors-based Data Gloves

Gesture recognition systems provide a natural interface of interaction between humans and computational systems. This study proposes a system for dynamic gesture recognition using inertial sensors-based data gloves. The proposed data gloves that consist of thirty-six inertial measurement units capture the motion of two arms and hands. The multimodal dataset included the sign language information of data gloves and skeletons is built. Then the convolutional neural network structure for sign language recognition named SLRNet is designed. It mainly consists of a convolutional layer, a batch normalization layer, and a fully connected layer. Finally dynamical gesture recognition experiments are implemented to prove the effectiveness of the proposed method.
Secure multiparty learning from the aggregation of locally trained models

In many applications, multiple parties would benefit from a precise learning model trained from the aggregated dataset. However, the trivial method that all the data is aggregated into one datacenter and processed centrally is not appropriate when data privacy is a significant concern. In this paper, we propose a new framework for secure multi-party learning and construct a concrete scheme by incorporating aggregate signature and proxy re-encryption techniques. Unlike the previous solutions for multi-party privacy-preserving machine learning, we don't use encryption algorithm to encrypt the whole dataset or the intermediate values during the training process. In our scheme, secure verifiable computation delegation is utilized to privately label a public dataset from the aggregation of locally trained models. Using these newly generated labeled data items, the participants can update their learning models with great accuracy improvement. Further, we prove that the proposed scheme satisfies the desired security properties, and the experimental analysis on MNIST and HAM10000 shows that it is highly efficient.
Microwave Radiometer Data Superresolution Using Image Degradation and Residual Network

Microwave radiometers are the key sensors to globally monitor environmental parameters; however, it suffers from its low and nonuniform spatial resolution. In this paper, a superresolution (SR) technique based on image degradation and residual network is proposed to enhance the spatial resolution of microwave radiometer data. Specifically, an improved degradation model is proposed to construct pairs of high-resolution (HR) and low-resolution (LR) data for training and testing. In addition, a new residual network connected by the SR main and gradient auxiliary branches in parallel is designed to achieve SR reconstructions, where eight-channel gradient maps extracted from LR data are input into the auxiliary branch to help to reconstruct. SR results are eventually generated by the trained SR network. Experiments executed on both simulated and actual data demonstrate the soundness and the superiority of the proposed SR technique.
Enabling Robust Grammatical Error Correction in New Domains: Data Sets, Metrics, and Analyses

Until now, grammatical error correction (GEC) has been primarily evaluated on text written by non-native English speakers, with a focus on student essays. This paper enables GEC development on text written by native speakers by providing a new data set and metric. We present a multiple-reference test corpus for GEC that includes 4,000 sentences in two new domains ( formal and informal writing by native English speakers) and 2,000 sentences from a diverse set of non-native student writing. We also collect human judgments of several GEC systems on this new test set and perform a meta-evaluation, assessing how reliable automatic metrics are across these domains. We find that commonly used GEC metrics have inconsistent performance across domains, and therefore we propose a new ensemble metric that is robust on all three domains of text.
Improving Mispronunciation Detection of Mandarin Tones for Non-Native Learners With Soft-Target Tone Labels and BLSTM-Based Deep Tone Models

We investigate the effectiveness of soft-target tone labels and sequential context information for mispronunciation detection of Mandarin lexical tones pronounced by second language (L2) learners whose first language (L1) is of European origin. In conventional approaches, prosodic information (e.g., F0 and tone posteriors extracted from trained tone models) is used to calculate goodness of pronunciation (GOP) scores or train binary classifiers to verify pronunciation correctness. We propose three techniques to improve detection of mispronunciation of Mandarin tones for non-native learners. First, we extend our tone model from a deep neural network (DNN) to a bidirectional long short-term memory (BLSTM) network in order to more accurately model the high variability of non-native tone productions and the contextual information expressed in tone-level co-articulation. Second, we characterize ambiguous pronunciations where L2 learners' tone realizations are between two canonical tone categories by relaxing hard target labels to soft targets with probabilistic transcriptions. Third, segmental tone features fed into verifiers are extracted by a BLSTM to exploit sequential context information to improve mispronunciation detection. Compared to DNN-GOP trained with hard targets, the proposed BLSTM-GOP framework trained with soft targets reduces the tones' averaged equal error rate (ERR) from 7.58% to 5.83% and the averaged area under ROC curve (AUC) is increased from 97.85% to 98.31%. By utilizing BLSTM-based verifiers the EER further decreases to 5.16%, and the AUC is increased to 98.47%.
Attention‐based spatial–temporal hierarchical ConvLSTM network for action recognition in videos

Human action recognition in videos is an important research topic in computer vision due to its wide applications. Actions naturally contain both spatial and temporal information. The key to action recognition is to model the spatial and temporal structures of actions. In this study, the authors propose an attention‐based spatial–temporal hierarchical convolutional long short‐term memory (ST‐HConvLSTM) network to model the structures of actions in the spatial and temporal domains. The ST‐HConvLSTM consists of two parts: a spatial–temporal attention module and a novel LSTM‐like architecture named hierarchical ConvLSTM (HConvLSTM). The HConvLSTM can model the spatial and temporal structures of actions. The spatial–temporal attention module can figure out which part of the video is more discriminative for action recognition and makes the HConvLSTM focus on it. In addition, a weighted fusion strategy is proposed to fuse the appearance information and motion information of the video. The proposed ST‐HConvLSTM is evaluated on UCF101, HMDB51 and Kinetics datasets. Experimental results show that the authors’ proposed ST‐HConvLSTM achieves state‐of‐the‐art performance compared with other recent LSTM‐like architectures and attention‐based methods.
A contextual learning model for developing interactive e-books to improve students’ performances of learning the Analects of Confucius

The Analects of Confucius has been identified as an important and influential philosophy around the globe. However, in traditional instruction, it is difficult to express the spirit of this philosophy. Therefore, many students consider it as a big challenge to study the Analects of Confucius. To cope with this problem, in this study, an interactive e-book system has been developed based on the contextual learning model to interpret the Analects of Confucius in an interactive way. A quasi-experiment was conducted to evaluate the performance of the proposed approach. The participants were 38 fifth graders from 2 classes in an elementary school located in northern Taiwan, who were divided into an experimental and a control group. The experimental group used an interactive e-book learning mode to learn the Analects of Confucius course while the control group used a conventional technology enhanced teaching mode. The results showed that adopting the interactive e-book teaching mode to study the Analects of Confucius in the elementary school course can promote students’ learning achievement and motive. It was also found that the interactive e-book learning mode can trigger learners’ deep motive, and hence promote their learning achievement.
Activity recognition using ST-GCN with 3D motion data

For the Nurse Care Activity Recognition Challenge, an activity recognition algorithm was developed by Team TDU-DSML. A spatial-temporal graph convolutional network (ST-GCN) was applied to process 3D motion capture data included in the challenge dataset. Time-series data was divided into 20-second segments with a 10-second overlap. The recognition model with a tree-structure graph was then created. The prediction result was set to one-minute segments on the basis of a majority decision from each segment output. Our model was evaluated by using leave-one-subject-out cross-validation methods. An average accuracy of 57% for all six subjects was achieved.
Blindfolded Evaluation of Random Forests with Multi-Key Homomorphic Encryption

Decision tree and its generalization of random forests are a simple yet powerful machine learning model for many classification and regression problems. Recent works propose how to privately evaluate a decision tree in a two-party setting where the feature vector of the client or the decision tree model (such as the threshold values of its nodes) is kept secret from another party. However, these works cannot be extended trivially to support the outsourcing setting where a third-party who should not have access to the model or the query. Furthermore, their use of an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interactive</i> comparison protocol does not support branching program, hence requires interactions with the client to determine the comparison result before resuming the evaluation task. In this paper, we propose the first secure protocol for collaborative evaluation of random forests contributed by multiple owners. They outsource evaluation tasks to a third-party evaluator. Upon receiving the client's encrypted inputs, the cloud evaluates obliviously on individually encrypted random forest models and calculates the aggregated result. The system is based on our new secure comparison protocol, secure counting protocol, and a multi-key somewhat homomorphic encryption on top of symmetric-key encryption. This allows us to reduce communication overheads while achieving round complexity lower than existing work.
Towards Differential Privacy for Symbolic Systems

In this paper, we develop a privacy implementation for symbolic control systems. Such systems generate sequences of non-numerical data, and these sequences can be represented by words or strings over a finite alphabet. This work uses the framework of differential privacy, which is a statistical notion of privacy that makes it unlikely that privatized data will reveal anything meaningful about underlying sensitive data. To bring differential privacy to symbolic control systems, we develop an exponential mechanism that approximates a sensitive word using a randomly chosen word that is likely to be near it. The notion of “near” is given by the Levenshtein distance, which counts the number of operations required to change on string into another. We then develop a Levenshtein automaton implementation of our exponential mechanism that efficiently generates privatized output words. This automaton has letters as its states, and this work develops transition probabilities among these states that give overall output words obeying the distribution required by the exponential mechanism. Numerical results are provided to demonstrate this technique for both strings of English words and runs of a deterministic transition system, demonstrating in both cases that privacy can be provided in this setting while maintaining a reasonable degree of accuracy.
Coherence manipulation under incoherent operations

Coherence manipulation is fundamental in the study of the resource theory of quantum coherence. It is aimed at identifying when it is possible to convert a given coherent state to another using only incoherent operations. In this paper, it is shown that a finite number of measure conditions are not sufficient to characterize coherence manipulation on general mixed states. Further, a finite number of measure conditions to classify coherence manipulation on subspace-independent states are found. For the input pure state, we also furnish a structural characterization of coherence manipulation in terms of a finite number of measure conditions.
Fuzzy Removing Redundancy Restricted Boltzmann Machine: improving learning speed and classification accuracy

To improve the feature extraction ability and shorten the learning time, fuzzy removing redundancy restricted Boltzmann machine (F3RBM) is developed. The features extracted by F3RBM with unsupervised learning are imported into support vector machine (SVM) to establish F3RBM-SVM model, which achieves fast and high-precision automatic classification of different samples. To expand the feature extraction capability of restricted Boltzmann machine (RBM), the deterministic parameters of control model are replaced by fuzzy numbers in view of the superiority of fuzzy idea and the redundancy removal mechanism is introduced. Comparing the feature similarity of hidden units with the threshold value, if the similarity is greater than the threshold value, they are considered to be redundant units with the same features. The redundant units are removed to achieve further dimension reduction. Finally, the learning speed, feature extraction ability, and classification accuracy of different models are compared in MINIST handwritten dataset, Fashion MNIST dataset, and Olivetti Face dataset. The experimental results show that the feature extraction capability of FRBM and F3RBM is better than that of RBM. When there are a large number of hidden units, the learning speed of F3RBM is obviously faster than that of FRBM. The features extracted from F3RBM are imported into the SVM to build F3RBM-SVM model, which improves the classification accuracy and learning speed than general classifier. When adding other noises, F3RBM-SVM has better robustness than other models.
User-Defined Voice and Mid-Air Gesture Commands for Maneuver-based Interventions in Automated Vehicles

For highly automated vehicles (AVs), new interaction concepts need to be developed. Even in AVs, the driver might want to intervene and override the automation from time to time. To create the possibility of control, we explore vehicle control through maneuver-based interventions (MBI). Thereby, we focus on explicit, contact-less interaction, which could be beneficial in future AV designs, where the driver is not necessarily bound to classical controls. We propose a set of freehand gestures and keywords for voice control derived in a user-centered design process. Further, we discuss properties, applicability and user impressions of both interaction modalities. Voice control seems to be an efficient way to select a maneuver and free-hand gestures could be used, if voice channel is blocked, e.g., through conversation with passengers.
IoT Avatars: Mixed Reality Hybrid Objects for CoRe Ambient Intelligent Environments

The Internet of Things (IoT) continues its growth, adoption, toward ubiquitous usage but is not without the inevitable communication bandwidth challenge. Human-computer-interaction in this space must account for the multiple facets of human-in-the-loop considerations in IoT, yet current mechanisms are at present limited by display dimensions and unclear indicators. Mixed Reality (MR) may be a solution to this human communication bandwidth problem, as smart glasses and other head mounted displays could provide an ideal interface platform for IoT human-computer interaction, while handheld mobile MR can be used as a testbed. To bring MR interfaces to the IoT, this work contributes; i) a new IoT-Avatar architectural framework; ii) a bi-directional communication approach between an IoT system and a virtual avatar character representation; and iii) an early descriptive exploration of how such systems could be explored in future applications toward MR interfaces in ambient intelligent environments.
Towards End-to-End Speech Recognition with Deep Multipath Convolutional Neural Networks

Approaches to deep learning have been used all over in connection to Automatic Speech Recognition (ASR), where they have achieved a high level of accuracy. This has mostly been seen in Convolutional Neural Network (CNN) which has recently been investigated in ASR. Due to the fact that CNN has an increased network’s depth on one branch, and may not be wide enough to work on capturing adequate features on signals of human speech. We focus on a proposal for an architecture that is deep and wide in CNN referred to as Multipath Convolutional Neural Network (MCNN). MCNN-CTC combines three additional paths with Connectionist Temporal Classification (CTC) objective function, and can be defined as an end-to-end system that has the ability to fully exploit spectral and temporal structures related to speech signals simultaneously. Results from the experiments show that the newly proposed MCNN-CTC structure enables a reduction in the error rate arising from the construction of end-to-end acoustic model. In the absence of a Language Model (LM), our proposed MCNN-CTC acoustic model has a relative reduction of 1.10%–12.08% comparing to the traditional HMM-based or DCNN-CTC-based models with strong generalization performance.
Interpretability in Intelligent Systems – A New Concept?

The very active community for interpretable machine learning can learn from the rich 50+ year history of explainable AI. We here give two specific examples from this legacy that could enrich current interpretability work: First, Explanation desiderata were we point to the rich set of ideas developed in the ‘explainable expert systems’ field and, second, tools for quantification of uncertainty of high-dimensional feature importance maps which have been developed in the field of computational neuroimaging.
Technical Mapping of the Grooming Anatomy Using Machine Learning Paradigms: An Information Security Approach

In the field of information security, there are several areas of study that are under development.Social engineering is one of them that addresses the multidisciplinary challenges of cyber security.Nowadays, the attacks associated with social engineering are diverse, including the so-called Advanced Persistent Threats (APTs).These have been the subject of numerous investigations; however, cybernetic attacks of similar nature as grooming have been excluded from these studies.In the last decade, various efforts have been made to understand the structure and approach of grooming from the field of computer science with the use of computational learning algorithms.Nevertheless, these studies are not aligned with information security.In this work, the study of grooming is formalized as a social engineering attack, contrasting its stages or phases with life cycles associated with APTs.To achieve this goal, we use a database of real cyberpedophile chats; this information was refined and the Latent Dirichlet Allocation (LDA) topic modeling was applied to determine the stages of the attack.Once the number of stages was determined, we proceed to give them a linguistic context, and with the use of machine learning, a linear model was trained to obtain 97.6% of training accuracy.With these results, it was determined that the study of grooming could support research associated with social engineering and contribute to new fields of information security.
Impact of Ultrasound Image Reconstruction Method on Breast Lesion Classification with Deep Learning

In this work we investigate the usefulness and robustness of transfer learning with deep convolutional neural networks (CNNs) for breast lesion classification in ultrasound (US). Deep learning models can be vulnerable to adversarial examples, engineered input image pixel intensities perturbations that force models to make classification errors. In US imaging, distribution of US image pixel intensities relies on applied US image reconstruction algorithm. We explore the possibility of fooling deep learning models for breast mass classification by modifying US image reconstruction method. Raw radio-frequency US signals acquired from malignant and benign breast masses were used to reconstruct US images, and develop classifiers using transfer learning with the VGG19, InceptionV3 and InceptionResNetV2 CNNs. The areas under the receiver operating characteristic curve (AUCs) obtained for each deep learning model developed and evaluated using US images reconstructed in the same way were equal to approximately 0.85, and there were no associated differences in AUC values between the models (DeLong test p-values > 0.15). However, due to small modifications of the US image reconstruction method the AUC values for the models utilizing the VGG19, InceptionV3 and InceptionResNetV2 CNNs significantly decreased to 0.592, 0.584 and 0.687, respectively. Our study shows that the modification of US image reconstruction algorithm can have significant negative impact on classification performance of deep models. Taking into account medical image reconstruction algorithms may help develop more robust deep learning computer aided diagnosis systems.
Understanding User Search Behavior Across Varying Cognitive Levels

The ubiquitous accessibility of the world-wide web has led people to increasingly use web search to learn or acquire new knowledge. Recent research efforts have targeted the optimization of web search to satisfy learning related needs. However, there is little known about how one's search interactions differ across varying cognitive levels that correspond to one's learning. In this paper, we address this knowledge gap by investigating how the search interactions of 150 users vary across 6 search tasks corresponding to distinct cognitive levels. We also analyze how users' knowledge gain varies across the cognitive levels. Our findings suggest that the cognitive learning level of a user in a search session has a significant impact on the user's search behavior and knowledge gain. Estimating the cognitive level of users during their interactions with search systems will allow us to construct and improve learning experiences for the users. For example, learners can be served content that corresponds to their current cognitive level within their learning process.