Reviewerly/exHarmony huggingface dataset:
The toolish hand illusion: embodiment of a tool based on similarity with the hand

Abstract A tool can function as a body part yet not feel like one: Putting down a fork after dinner does not feel like losing a hand. However, studies show fake body-parts are embodied and experienced as parts of oneself. Typically, embodiment illusions have only been reported when the fake body-part visually resembles the real one. Here we reveal that participants can experience an illusion that a mechanical grabber, which looks scarcely like a hand, is part of their body. We found changes in three signatures of embodiment: the real hand’s perceived location, the feeling that the grabber belonged to the body, and autonomic responses to visible threats to the grabber. These findings show that artificial objects can become embodied even though they bear little visual resemblance to the hand.
IoTHunter: IoT network traffic classification using device specific keywords

With the proliferation of IoT devices, network management and security monitoring are becoming a challenge. For the timely detection of IoT device status and their behaviour, traffic classification methods are used. Herein, IoTHunter, a Deep Packet Inspection based IoT traffic classifier, is described. It extracts unique keywords comprising domain names, device names etc. to identify flows belonging to a particular device. IoTHunter automates the keyword extraction using the frequency of occurrence of words belonging to flows of different devices. To further enhance the performance, IoTHunter combines device specific keywords with MAC address of device for subsequent flow labelling. A publicly available IoT dataset is experimented and a good classification accuracy of it over a range of IoT devices is demonstrated.
On the design of automatic voice condition analysis systems. Part III: review of acoustic modelling strategies

This is the third of a three-part series devoted to review the current state of the art of automatic voice condition analysis systems. A direct continuation to “On the design of automatic voice condition analysis systems. Part I: review of concepts and an insight to the state of the art” and to “On the design of automatic voice condition analysis systems. Part II: review of speaker recognition techniques and study on the effects of different variability factors” already published in this journal. The goal of this paper is to compile the most significant parameterisation approaches used in the literature for automatic voice condition analysis systems, along with a critical discussion about their usefulness, providing the user with a comprehensive review of the most important techniques used for acoustic modelling in the field. The paper presents the mathematical formulation and physical interpretation of a series of perturbation and fluctuation parameters, noise features, complexity based parameters, modulation spectra, morphological parameters, and spectral-cepstral coefficients; and is complemented with a library written in MATLAB®, which has been made available to the readers in an online software repository.
Dense Open-set Recognition with Synthetic Outliers Generated by Real NVP

Today's deep models are often unable to detect inputs which do not belong to the training distribution. This gives rise to confident incorrect predictions which could lead to devastating consequences in many important application fields such as healthcare and autonomous driving. Interestingly, both discriminative and generative models appear to be equally affected. Consequently, this vulnerability represents an important research challenge. We consider an outlier detection approach based on discriminative training with jointly learned synthetic outliers. We obtain the synthetic outliers by sampling an RNVP model which is jointly trained to generate datapoints at the border of the training distribution. We show that this approach can be adapted for simultaneous semantic segmentation and dense outlier detection. We present image classification experiments on CIFAR-10, as well as semantic segmentation experiments on three existing datasets (StreetHazards, WD-Pascal, Fishyscapes Lost & Found), and one contributed dataset. Our models perform competitively with respect to the state of the art despite producing predictions with only one forward pass.
Real-World Human-Robot Collaborative Reinforcement Learning

The intuitive collaboration of humans and intelligent robots (embodied AI) in the real-world is an essential objective for many desirable applications of robotics. Whilst there is much research regarding explicit communication, we focus on how humans and robots interact implicitly, on motor adaptation level. We present a real-world setup of a human-robot collaborative maze game, designed to be non-trivial and only solvable through collaboration, by limiting the actions to rotations of two orthogonal axes, and assigning each axes to one player. This results in neither the human nor the agent being able to solve the game on their own. We use deep reinforcement learning for the control of the robotic agent, and achieve results within 30 minutes of real-world play, without any type of pre-training. We then use this setup to perform systematic experiments on human/agent behaviour and adaptation when co-learning a policy for the collaborative game. We present results on how co-policy learning occurs over time between the human and the robotic agent resulting in each participant's agent serving as a representation of how they would play the game. This allows us to relate a person's success when playing with different agents than their own, by comparing the policy of the agent with that of their own agent.
AI-based smart and intelligent wheelchair

The differently abled and/or old-aged people require assistance for their movement. Generally, such assistant providing tool is wheelchair. Normal wheelchairs are manually operated and heavy to move adding burden to the suffered. Hence, automated wheelchairs that are equipped with sensors and a data processing unit constitute a special class of wheeled mobile robots, termed as “smart wheelchairs” in general. In the existing system, the wheelchair movement that is controlled by joystick uses buttons to start and stop the wheel. This is difficult for the differently abled to press the required button with precision. Although there are smart wheelchairs with gesture control, it lacks accuracy in the calculation of the location. The proposed system uses artificial intelligence for its working and proves to be a unique combination of wheelchair and health monitoring system. The wheelchair can be accessed both in manual and automatic modes. In the manual mode, the wheel is controlled using joystick whereas in the automated mode, MPU6050 sensor and accelerometer is used to control the direction by gesture. SPO2 sensor attached to the wheelchair is used to collect the health parameters. Thus, enabling the self-dependency of the person. Further, deep learning analysis of the data from the sensors and the wheelchair usage pattern is compared with the dataset to determine the stress level. The signal from the sensors is monitored and the vitals data is updated in the ThingSpeak website via Bluetooth module serving as a digital health chart.
An Approach To Mode and Anomaly Detection with Spacecraft Telemetry Data

This paper discusses a mixed method that combines unsupervised learning methods and human expert input for analyzing telemetry data from long-duration robotic space missions. Our goal is to develop more automated methods for detecting anomalies in time series data. Once anomalies are identified using unsupervised learning methods we use feature selection methods followed by expert input to derive the knowledge required for building on-line detectors. These detectors can be used in later phases of the current mission and in future missions for improving operations and overall safety of the mission. Whereas the primary focus in this paper is on developing data-driven anomaly detection methods, we also present a computational platform for data mining and analytics that can operate on historical data offline, as well as incoming telemetry data on-line.
Hierarchical Semantic Risk Minimization for Large-Scale Classification

Hierarchical structures of labels usually exist in large-scale classification tasks, where labels can be organized into a tree-shaped structure. The nodes near the root stand for coarser labels, while the nodes close to leaves mean the finer labels. We label unseen samples from the root node to a leaf node, and obtain multigranularity predictions in the hierarchical classification. Sometimes, we cannot obtain a leaf decision due to uncertainty or incomplete information. In this case, we should stop at an internal node, rather than going ahead rashly. However, most existing hierarchical classification models aim at maximizing the percentage of correct predictions, and do not take the risk of misclassifications into account. Such risk is critically important in some real-world applications, and can be measured by the distance between the ground truth and the predicted classes in the class hierarchy. In this work, we utilize the semantic hierarchy to define the classification risk and design an optimization technique to reduce such risk. By defining the conservative risk and the precipitant risk as two competing risk factors, we construct the balanced conservative/precipitant semantic (BCPS) risk matrix across all nodes in the semantic hierarchy with user-defined weights to adjust the tradeoff between two kinds of risks. We then model the classification process on the semantic hierarchy as a sequential decision-making task. We design an algorithm to derive the risk-minimized predictions. There are two modules in this model: 1) multitask hierarchical learning and 2) deep reinforce multigranularity learning. The first one learns classification confidence scores of multiple levels. These scores are then fed into deep reinforced multigranularity learning for obtaining a global risk-minimized prediction with flexible granularity. Experimental results show that the proposed model outperforms state-of-the-art methods on seven large-scale classification datasets with the semantic tree.
Document Plagiarism Detection Using a New Concept Similarity in Formal Concept Analysis

This paper proposes an algorithm for document plagiarism detection using the provided incremental knowledge construction with formal concept analysis (FCA). The incremental knowledge construction is presented to support document matching between the source document in storage and the suspect document. Thus, a new concept similarity measure is also proposed for retrieving formal concepts in the knowledge construction. The presented concept similarity employs appearance frequencies in the obtained knowledge construction. Our approach can be applied to retrieve relevant information because the obtained structure uses FCA in concept form that is definable by a conjunction of properties. This measure is mathematically proven to be a formal similarity metric. The performance of the proposed similarity measure is demonstrated in document plagiarism detection. Moreover, this paper provides an algorithm to build the information structure for document plagiarism detection. Thai text test collections are used for performance evaluation of the implemented web application.
CASTLE: Cluster-aided space transformation for local explanations

With Artificial Intelligence becoming part of a rapidly increasing number of industrial applications, more and more requirements about their transparency and trustworthiness are being demanded to AI systems, especially in military, medical and financial domains, where decisions have a huge impact on lives. In this paper, we propose a novel model-agnostic Explainable AI (XAI) technique, named Cluster-aided Space Transformation for Local Explanation (CASTLE), able to provide rule-based explanations based on both the local and global model's workings, i.e. its detailed "knowledge" in the neighborhood of the target instance and its general knowledge on the training dataset, respectively. The framework has been evaluated on six datasets in terms of temporal efficiency, cluster quality and model significance. Eventually, we asked 36 users to evaluate the explainability of the framework, getting as result an increase of interpretability of 6% with respect to another state-of-the-art technique, named Anchors.
Learning for Learning: Predictive Online Control of Federated Learning with Edge Provisioning

Operating federated learning optimally over distributed cloud-edge networks is a non-trivial task, which requires to manage data transference from user devices to edges, resource provisioning at edges, and federated learning between edges and the cloud. We formulate a non-linear mixed integer program, minimizing the long-term cumulative cost of such a federated learning system while guaranteeing the desired convergence of the machine learning models being trained. We then design a set of novel polynomial-time online algorithms to make adaptive decisions by solving continuous solutions and converting them to integers to control the system on the fly, based only on the predicted inputs about the dynamic and uncertain cloud-edge environments via online learning. We rigorously prove the competitive ratio, capturing the multiplicative gap between our approach using predicted inputs and the offline optimum using actual inputs. Extensive evaluations with real-world training datasets and system parameters confirm the empirical superiority of our approach over multiple state-of-the-art algorithms.
Smart Contract Vulnerabilities: Vulnerable Does Not Imply Exploited

In recent years, we have seen a great deal of both academic and practical interest in the topic of vulnerabilities in smart contracts, particularly those developed for the Ethereum blockchain. While most of the work has focused on detecting *vulnerable* contracts, in this paper, we focus on finding how many of these vulnerable contracts have actually been *exploited*. We survey the 23,327 vulnerable contracts reported by six recent academic projects and find that, despite the amounts at stake, only 1.98% of them have been exploited since deployment. This corresponds to at most 8,487 ETH (~1.7 million USD), or only 0.27% of the 3 million ETH (600 million USD) at stake. We explain these results by demonstrating that the funds are very concentrated in a small number of contracts which are *not exploitable* in practice.
Effective Named Entity Recognition with Boundary-aware Bidirectional Neural Networks

Named Entity Recognition (NER) is a fundamental problem in Natural Language Processing and has received much research attention. Although the current neural-based NER approaches have achieved the state-of-the-art performance, they still suffer from one or more of the following three problems in their architectures: (1) boundary tag sparsity, (2) lacking of global decoding information; and (3) boundary error propagation. In this paper, we propose a novel Boundary-aware Bidirectional Neural Networks (Ba-BNN) model to tackle these problems for neural-based NER. The proposed Ba-BNN model is constructed based on the structure of pointer networks for tackling the first problem on boundary tag sparsity. Moreover, we also use a boundary-aware binary classifier to capture the global decoding information as input to the decoders. In the Ba-BNN model, we propose to use two decoders to process the information in two different directions (i.e., from left-to-right and right-to-left). The final hidden states of the left-to-right decoder are obtained by incorporating the hidden states of the right-to-left decoder in the decoding process. In addition, a boundary retraining strategy is also proposed to help reduce boundary error propagation caused by the pointer networks in boundary detection and entity classification. We have conducted extensive experiments based on three NER benchmark datasets. The performance results have shown that the proposed Ba-BNN model has outperformed the current state-of-the-art models.
NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media

Online misinformation is a prevalent societal issue, with adversaries relying on tools ranging from cheap fakes to sophisticated deep fakes.We are motivated by the threat scenario where an image is used out of context to support a certain narrative.While some prior datasets for detecting image-text inconsistency generate samples via text manipulation, we propose a dataset where both image and text are unmanipulated but mismatched.We introduce several strategies for automatically retrieving convincing images for a given caption, capturing cases with inconsistent entities or semantic context.Our large-scale automatically generated NewsCLIPpings Dataset:(1) demonstrates that machine-driven image repurposing is now a realistic threat, and (2) provides samples that represent challenging instances of mismatch between text and image in news that are able to mislead humans.We benchmark several state-of-the-art multimodal models on our dataset and analyze their performance across different pretraining domains and visual backbones.
A Study of Automatic Metrics for the Evaluation of Natural Language Explanations

As transparency becomes key for robotics and AI, it will be necessary to evaluate the methods through which transparency is provided, including automatically generated natural language (NL) explanations.Here, we explore parallels between the generation of such explanations and the much-studied field of evaluation of Natural Language Generation (NLG).Specifically, we investigate which of the NLG evaluation measures map well to explanations.We present the ExBAN corpus: a crowd-sourced corpus of NL explanations for Bayesian Networks.We run correlations comparing human subjective ratings with NLG automatic measures.We find that embedding-based automatic NLG evaluation methods, such as BERTScore and BLEURT, have a higher correlation with human ratings, compared to word-overlap metrics, such as BLEU and ROUGE.This work has implications for Explainable AI and transparent robotic and autonomous systems.
Deep Learning based Anomaly Detection in Images: Insights, Challenges and Recommendations

Deep learning-based anomaly detection in images has recently been considered a popular research area with numerous applications worldwide.The main aim of anomaly detection (i.e., Outlier detection), is to identify data instances that deviate considerably from the majority of data instances.This paper offers a comprehensive analysis of previous works that have been proposed in the area of anomaly detection in images through deep learning generally and in the medical field specifically.Twenty studies were reviewed, and the literature selection methodology was defined based on four phases: keyword filter, publish filter, year filter, and abstract filter.In this review, we highlight the differences among the studies included by considering the following factors: methodology, dataset, preprocessing, results and limitations.Besides, we illustrate the various challenges and potential future directions relevant to anomaly detection in images.
Machine learning framework for quantum sampling of highly constrained, continuous optimization problems

In recent years, there is growing interest in using quantum computers for solving combinatorial optimization problems. In this work, we developed a generic, machine learning-based framework for mapping continuous-space inverse design problems into surrogate quadratic unconstrained binary optimization (QUBO) problems by employing a binary variational autoencoder and a factorization machine. The factorization machine is trained as a low-dimensional, binary surrogate model for the continuous design space and sampled using various QUBO samplers. Using the D-Wave Advantage hybrid sampler and simulated annealing, we demonstrate that by repeated resampling and retraining of the factorization machine, our framework finds designs that exhibit figures of merit exceeding those of its training set. We showcase the framework's performance on two inverse design problems by optimizing (i) thermal emitter topologies for thermophotovoltaic applications and (ii) diffractive meta-gratings for highly efficient beam steering. This technique can be further scaled to leverage future developments in quantum optimization to solve advanced inverse design problems for science and engineering applications.
Detection of Malpractice in E-exams by Head Pose and Gaze Estimation

Examination malpractice is a deliberate wrong doing contrary to official examina-tion rules designed to place a candidate at unfair advantage or disadvantage. The proposed system depicts a new use of technology to identify malpractice in E-Exams which is essential due to growth of online education. The current solu-tions for such a problem either require complete manual labor or have various vulnerabilities that can be exploited by an examinee. The proposed application en-compasses an end-to-end system that assists an examiner/evaluator in deciding whether a student passes an online exam without any probable attempts of mal-practice or cheating in e-exams with the help of visual aids. The system works by categorizing the student’s VFOA (visual focus of attention) data by capturing the head pose estimates and eye gaze estimates using state-of-the-art machine learn-ing techniques. The system only requires the student (test-taker) to have a func-tioning internet connection along with a webcam to transmit the feed. The exam-iner is alerted when the student wavers in his VFOA, from the screen greater than X, a predefined threshold of times. If this threshold X is crossed, the appli-cation will save the data of the person when his VFOA is off the screen and send it to the examiner to be manually checked and marked whether the action per-formed by the student was an attempt at malpractice or just momentary lapse in concentration. The system use a hybrid classifier approach where two different classifiers are used, one when gaze values are being read successfully (which may fail due to various reasons like transmission quality or glare from his specta-cles), the model falls back to the default classifier which only reads the head pose values to classify the attention metric, which is used to map the student’s VFOA to check the likelihood of malpractice. The model has achieved an accuracy of 96.04 percent in classifying the attention metric.