OpenRaiser/Intern-Atlas huggingface dataset:
Secondary Inputs for Measuring User Engagement in Immersive VR Education Environments
Venue: arXiv cs.HC 2019
Year: 2019

This paper presents an experiment to assess the feasibility of using secondary input data as a method of determining user engagement in immersive virtual reality (VR). The work investigates whether secondary data (biosignals) acquired from users are useful as a method of detecting levels of concentration, stress, relaxation etc. in immersive environments, and if they could be used to create an affective feedback loop in immersive VR environments, including educational contexts. A VR Experience was developed in the Unity game engine, with three different levels, each designed to expose the user in one of three different states (relaxation, concentration, stress). While in the VR Experience users physiological responses were measured using ECG and EEG sensors. After the experience users completed questionnaires to establish their perceived state during the levels, and to established the usability of the system. Next a comparison between the reported levels of emotion and the measured signals is presented, which show a strong correspondence between the two measures indicating that biosignals are a useful indicator of emotional state while in VR. Finally we make some recommendations on the practicalities of using biosensors, and design considerations for their incorporation in to a VR system, with particular focus on their integration in to task-based training and educational virtual environments.
On the security and privacy of Interac e-Transfers
Venue: arXiv cs.CR 2019
Year: 2019

Nowadays, the Interac e-Transfer is one of the most important remote payment methods for Canadian consumers. To the best of our knowledge, this paper is the very first to examine the privacy and security of Interac e-Transfers. Experimental results show that the notifications sent to customers via email and SMS contain sensitive private information that can potentially be observed by third parties. Anyone with illegitimate intent can use this information to carry out attacks, including the fraudulent redirection of Standard e-Transfers. Such an attack is shown to be possible at least in an experimental setup but likely also in reality. Recent news articles support this finding. Improvements to overcome these interconnected privacy and security problems are proposed and discussed.
360-Indoor: Towards Learning Real-World Objects in 360° Indoor Equirectangular Images
Venue: arXiv cs.CV 2019
Year: 2019

While there are several widely used object detection datasets, current computer vision algorithms are still limited in conventional images. Such images narrow our vision in a restricted region. On the other hand, 360° images provide a thorough sight. In this paper, our goal is to provide a standard dataset to facilitate the vision and machine learning communities in 360° domain. To facilitate the research, we present a real-world 360° panoramic object detection dataset, 360-Indoor, which is a new benchmark for visual object detection and class recognition in 360° indoor images. It is achieved by gathering images of complex indoor scenes containing common objects and the intensive annotated bounding field-of-view. In addition, 360-Indoor has several distinct properties: (1) the largest category number (37 labels in total). (2) the most complete annotations on average (27 bounding boxes per image). The selected 37 objects are all common in indoor scene. With around 3k images and 90k labels in total, 360-Indoor achieves the largest dataset for detection in 360° images. In the end, extensive experiments on the state-of-the-art methods for both classification and detection are provided. We will release this dataset in the near future.
Character Feature Engineering for Japanese Word Segmentation
Venue: arXiv cs.CL 2019
Year: 2019

On word segmentation problems, machine learning architecture engineering often draws attention. The problem representation itself, however, has remained almost static as either word lattice ranking or character sequence tagging, for at least two decades. The latter of-ten shows stronger predictive power than the former for out-of-vocabulary (OOV) issue. When the issue escalating to rapid adaptation, which is a common scenario for industrial applications, active learning of partial annotations or re-training with additional lexical re-sources is usually applied, however, from a somewhat word-based perspective. Not only it is uneasy for end-users to comply with linguistically consistent word boundary decisions, but also the risk/cost of forking models permanently with estimated weights is seldom affordable. To overcome the obstacle, this work provides an alternative, which uses linguistic intuition about character compositions, such that a sophisticated feature set and its derived scheme can enable dynamic lexicon expansion with the model remaining intact. Experiment results suggest that the proposed solution, with or without external lexemes, performs competitively in terms of F1 score and OOV recall across various datasets.
NeurReg: Neural Registration and Its Application to Image Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

Registration is a fundamental task in medical image analysis which can be applied to several tasks including image segmentation, intra-operative tracking, multi-modal image alignment, and motion analysis. Popular registration tools such as ANTs and NiftyReg optimize an objective function for each pair of images from scratch which is time-consuming for large images with complicated deformation. Facilitated by the rapid progress of deep learning, learning-based approaches such as VoxelMorph have been emerging for image registration. These approaches can achieve competitive performance in a fraction of a second on advanced GPUs. In this work, we construct a neural registration framework, called NeurReg, with a hybrid loss of displacement fields and data similarity, which substantially improves the current state-of-the-art of registrations. Within the framework, we simulate various transformations by a registration simulator which generates fixed image and displacement field ground truth for training. Furthermore, we design three segmentation frameworks based on the proposed registration framework: 1) atlas-based segmentation, 2) joint learning of both segmentation and registration tasks, and 3) multi-task learning with atlas-based segmentation as an intermediate feature. Extensive experimental results validate the effectiveness of the proposed NeurReg framework based on various metrics: the endpoint error (EPE) of the predicted displacement field, mean square error (MSE), normalized local cross-correlation (NLCC), mutual information (MI), Dice coefficient, uncertainty estimation, and the interpretability of the segmentation. The proposed NeurReg improves registration accuracy with fast inference speed, which can greatly accelerate related medical image analysis tasks.
Formula Transformers and Combinatorial Test Generators for Propositional Intuitionistic Theorem Provers
Venue: arXiv cs.LO 2019
Year: 2019

We develop combinatorial test generation algorithms for progressively more powerful theorem provers, covering formula languages ranging from the implicational fragment of intuitionistic logic to full intuitionistic propositional logic. Our algorithms support exhaustive and random generators for formulas of these logics. To provide known-to-be-provable formulas, via the Curry-Howard formulas-as-types correspondence, we use generators for typable lambda terms and combinator expressions. Besides generators for several classes of formulas, we design algorithms that restrict formula generation to canonical representatives among equiprovable formulas and introduce program transformations that reduce formulas to equivalent formulas of a simpler structure. The same transformations, when applied in reverse, create harder formulas that can catch soundness or incompleteness bugs. To test the effectiveness of the testing framework itself, we describe use cases for deriving lightweight theorem provers for several of these logics and for finding bugs in known theorem provers. Our Prolog implementation available at: https://github.com/ptarau/TypesAndProofs and a subset of formula generators and theorem provers, implemented in Python is available at: https://github.com/ptarau/PythonProvers. Keywords: term and formula generation algorithms, Prolog-based theorem provers, formulas-as-types, type inference and type inhabitation, combinatorial testing, finding bugs in theorem provers.
Learning Robust Representations with Graph Denoising Policy Network
Venue: arXiv cs.LG 2019
Year: 2019

Graph representation learning, aiming to learn low-dimensional representations which capture the geometric dependencies between nodes in the original graph, has gained increasing popularity in a variety of graph analysis tasks, including node classification and link prediction. Existing representation learning methods based on graph neural networks and their variants rely on the aggregation of neighborhood information, which makes it sensitive to noises in the graph. In this paper, we propose Graph Denoising Policy Network (short for GDPNet) to learn robust representations from noisy graph data through reinforcement learning. GDPNet first selects signal neighborhoods for each node, and then aggregates the information from the selected neighborhoods to learn node representations for the down-stream tasks. Specifically, in the signal neighborhood selection phase, GDPNet optimizes the neighborhood for each target node by formulating the process of removing noisy neighborhoods as a Markov decision process and learning a policy with task-specific rewards received from the representation learning phase. In the representation learning phase, GDPNet aggregates features from signal neighbors to generate node representations for down-stream tasks, and provides task-specific rewards to the signal neighbor selection phase. These two phases are jointly trained to select optimal sets of neighbors for target nodes with maximum cumulative task-specific rewards, and to learn robust representations for nodes. Experimental results on node classification task demonstrate the effectiveness of GDNet, outperforming the state-of-the-art graph representation learning methods on several well-studied datasets. Additionally, GDPNet is mathematically equivalent to solving the submodular maximizing problem, which theoretically guarantees the best approximation to the optimal solution with GDPNet.
Deep Q-Network for Angry Birds
Venue: arXiv cs.AI 2019
Year: 2019

Angry Birds is a popular video game in which the player is provided with a sequence of birds to shoot from a slingshot. The task of the game is to destroy all green pigs with maximum possible score. Angry Birds appears to be a difficult task to solve for artificially intelligent agents due to the sequential decision-making, non-deterministic game environment, enormous state and action spaces and requirement to differentiate between multiple birds, their abilities and optimum tapping times. We describe the application of Deep Reinforcement learning by implementing Double Dueling Deep Q-network to play Angry Birds game. One of our main goals was to build an agent that is able to compete with previous participants and humans on the first 21 levels. In order to do so, we have collected a dataset of game frames that we used to train our agent on. We present different approaches and settings for DQN agent. We evaluate our agent using results of the previous participants of AIBirds competition, results of volunteer human players and present the results of AIBirds 2018 competition.
Sparse Surface Constraints for Combining Physics-based Elasticity Simulation and Correspondence-Free Object Reconstruction
Venue: arXiv cs.GR 2019
Year: 2019

We address the problem to infer physical material parameters and boundary conditions from the observed motion of a homogeneous deformable object via the solution of an inverse problem. Parameters are estimated from potentially unreliable real-world data sources such as sparse observations without correspondences. We introduce a novel Lagrangian-Eulerian optimization formulation, including a cost function that penalizes differences to observations during an optimization run. This formulation matches correspondence-free, sparse observations from a single-view depth sequence with a finite element simulation of deformable bodies. In conjunction with an efficient hexahedral discretization and a stable, implicit formulation of collisions, our method can be used in demanding situation to recover a variety of material parameters, ranging from Young's modulus and Poisson ratio to gravity and stiffness damping, and even external boundaries. In a number of tests using synthetic datasets and real-world measurements, we analyse the robustness of our approach and the convergence behavior of the numerical optimization scheme.
Few-Shot Abstract Visual Reasoning With Spectral Features
Venue: arXiv cs.LG 2019
Year: 2019

We present an image preprocessing technique capable of improving the performance of few-shot classifiers on abstract visual reasoning tasks. Many visual reasoning tasks with abstract features are easy for humans to learn with few examples but very difficult for computer vision approaches with the same number of samples, despite the ability for deep learning models to learn abstract features. Same-different (SD) problems represent a type of visual reasoning task requiring knowledge of pattern repetition within individual images, and modern computer vision approaches have largely faltered on these classification problems, even when provided with vast amounts of training data. We propose a simple method for solving these problems based on the insight that removing peaks from the amplitude spectrum of an image is capable of emphasizing the unique parts of the image. When combined with several classifiers, our method performs well on the SD SVRT tasks with few-shot learning, improving upon the best comparable results on all tasks, with average absolute accuracy increases nearly 40% for some classifiers. In particular, we find that combining Relational Networks with this image preprocessing approach improves their performance from chance-level to over 90% accuracy on several SD tasks.
Vulnerability of Face Recognition to Deep Morphing
Venue: arXiv cs.CV 2019
Year: 2019

It is increasingly easy to automatically swap faces in images and video or morph two faces into one using generative adversarial networks (GANs). The high quality of the resulted deep-morph raises the question of how vulnerable the current face recognition systems are to such fake images and videos. It also calls for automated ways to detect these GAN-generated faces. In this paper, we present the publicly available dataset of the Deepfake videos with faces morphed with a GAN-based algorithm. To generate these videos, we used open source software based on GANs, and we emphasize that training and blending parameters can significantly impact the quality of the resulted videos. We show that the state of the art face recognition systems based on VGG and Facenet neural networks are vulnerable to the deep morph videos, with 85.62 and 95.00 false acceptance rates, respectively, which means methods for detecting these videos are necessary. We consider several baseline approaches for detecting deep morphs and find that the method based on visual quality metrics (often used in presentation attack detection domain) leads to the best performance with 8.97 equal error rate. Our experiments demonstrate that GAN-generated deep morph videos are challenging for both face recognition systems and existing detection methods, and the further development of deep morphing technologies will make it even more so.
Generating Relevant Counter-Examples from a Positive Unlabeled Dataset for Image Classification
Venue: arXiv cs.CV 2019
Year: 2019

With surge of available but unlabeled data, Positive Unlabeled (PU) learning is becoming a thriving challenge. This work deals with this demanding task for which recent GAN-based PU approaches have demonstrated promising results. Generative adversarial Networks (GANs) are not hampered by deterministic bias or need for specific dimensionality. However, existing GAN-based PU approaches also present some drawbacks such as sensitive dependence to prior knowledge, a cumbersome architecture or first-stage overfitting. To settle these issues, we propose to incorporate a biased PU risk within the standard GAN discriminator loss function. In this manner, the discriminator is constrained to request the generator to converge towards the unlabeled samples distribution while diverging from the positive samples distribution. This enables the proposed model, referred to as D-GAN, to exclusively learn the counter-examples distribution without prior knowledge. Experiments demonstrate that our approach outperforms state-of-the-art PU methods without prior by overcoming their issues.
Studying Breakdowns in Interactions with Smart Speakers
Venue: arXiv cs.HC 2019
Year: 2019

The popularity of voice-controlled smart speakers with intelligent personal assistants (IPAs) like the Amazon Echo and their increasing use as an interface for other Internet of Things (IoT) technologies in the home provides opportunities to study smart speakers as an emerging and ubiquitous IoT device. Prior research has studied how smart speaker usage has unfolded in homes and how the devices have been integrated into people's daily routines. In this context, researchers have also observed instances of smart speakers' 'black box' behaviour. In this position paper, we present findings from a study we conducted to specifically investigate such challenges people experience around intelligibility and control of their smart speakers, for instance, when the smart speaker interfaces with other IoT systems. Reflecting on our findings, we discuss new possible directions for smart speakers including physical intelligibility, situational physical interaction, and providing access to alternative interpretations in shared and non-shared contexts.
Zero Shot Learning on Simulated Robots
Venue: arXiv cs.RO 2019
Year: 2019

In this work we present a method for leveraging data from one source to learn how to do multiple new tasks. Task transfer is achieved using a self-model that encapsulates the dynamics of a system and serves as an environment for reinforcement learning. To study this approach, we train a self-models on various robot morphologies, using randomly sampled actions. Using a self-model, an initial state and corresponding actions, we can predict the next state. This predictive self-model is then used by a standard reinforcement learning algorithm to accomplish tasks without ever seeing a state from the "real" environment. These trained policies allow the robots to successfully achieve their goals in the "real" environment. We demonstrate that not only is training on the self-model far more data efficient than learning even a single task, but also that it allows for learning new tasks without necessitating any additional data collection, essentially allowing zero-shot learning of new tasks.
A Survey of Benchmarks to Evaluate Data Analytics for Smart-* Applications
Venue: arXiv cs.OH 2019
Year: 2019

The growth of ubiquitous sensor networks at an accelerating pace cuts across many areas of modern day life. They enable measuring, inferring, understanding and acting upon a wide variety of indicators, in fields ranging from agriculture to healthcare or to complex urban environments. The applications devoted to this task are designated as Smart-* Applications. They hide a staggering complexity, relying on multiple layers of data collection, transmission, aggregation, analysis and also storage, both at the network edge and on the cloud. Furthermore, Smart-* Applications raise additional specific challenges, such as the need to process and extract knowledge from diverse data, which is flowing at high velocity in near real-time or in the heavily distributed environment they rely on. How to assess the performance of such a complex stack, when faced with the specifics of \mbox{Smart-*} Applications, remains an open research question. In this article, the key specific characteristics and requirements of Smart-* Applications are initially detailed. Afterwards, for each of these requirements, there is a description of the benchmarks one can use to precisely evaluate the performance of the underlying systems and technologies. Finally, an identification of future research directions related to identified open issues for benchmarking Smart-* Applications is performed.
Dynamic data fusion using multi-input models for malware classification
Venue: arXiv cs.CR 2019
Year: 2019

Criminals use malware to disrupt cyber-systems. The number of these malware-vulnerable systems is increasing quickly as common systems, such as vehicles, routers, and lightbulbs, become increasingly interconnected cyber-systems. To address the scale of this problem, analysts divide malware into classes and develop, for each class, a specialized defense. In this project we classified malware with machine learning. In particular, we used a supervised multi-class long short term memory (LSTM) model. We trained the algorithm with thousands of malware files annotated with class labels (the training set), and the algorithm learned patterns indicative of each class. We used disassembled malware files (provided by Microsoft) and separated the constituent data into parsed instructions, which look like human-readable machine code text, and raw bytes, which are hexadecimal values. We are interested in which format, text or hex, is more valuable as input for classification. To solve this, we investigated four cases: a text-only model, a hexadecimal-only model, a multi-input model using both text and hexadecimal inputs, and a model based on combining the individual results. We performed this investigation using the machine learning Python package Keras, which allows easily configurable deep learning architectures and training. We hoped to understand the trade-offs between the different formats. Due to the class imbalance in the data, we used multiple methods to compare the formats, using test accuracies, balanced accuracies (taking into account weights of classes), and an accuracy derived from tables of confusion. We found that the multi-input model, which allows learning on both input types simultaneously, resulted in the best performance. Our finding expedites malware classification research by providing researchers a suitable deep learning architecture to train a tailored version to their malware.