OpenRaiser/Intern-Atlas huggingface dataset:
Training Deep Neural Networks Using Posit Number System
Venue: arXiv cs.LG 2019
Year: 2019

With the increasing size of Deep Neural Network (DNN) models, the high memory space requirements and computational complexity have become an obstacle for efficient DNN implementations. To ease this problem, using reduced-precision representations for DNN training and inference has attracted many interests from researchers. This paper first proposes a methodology for training DNNs with the posit arithmetic, a type- 3 universal number (Unum) format that is similar to the floating point(FP) but has reduced precision. A warm-up training strategy and layer-wise scaling factors are adopted to stabilize training and fit the dynamic range of DNN parameters. With the proposed training methodology, we demonstrate the first successful training of DNN models on ImageNet image classification task in 16 bits posit with no accuracy loss. Then, an efficient hardware architecture for the posit multiply-and-accumulate operation is also proposed, which can achieve significant improvement in energy efficiency than traditional floating-point implementations. The proposed design is helpful for future low-power DNN training accelerators.
SEdroid: A Robust Android Malware Detector using Selective Ensemble Learning
Venue: arXiv cs.CR 2019
Year: 2019

For the dramatic increase of Android malware and low efficiency of manual check process, deep learning methods started to be an auxiliary means for Android malware detection these years. However, these models are highly dependent on the quality of datasets, and perform unsatisfactory results when the quality of training data is not good enough. In the real world, the quality of datasets without manually check cannot be guaranteed, even Google Play may contain malicious applications, which will cause the trained model failure. To address the challenge, we propose a robust Android malware detection approach based on selective ensemble learning, trying to provide an effective solution not that limited to the quality of datasets. The proposed model utilizes genetic algorithm to help find the best combination of the component learners and improve robustness of the model. Our results show that the proposed approach achieves a more robust performance than other approaches in the same area.
A Convolutional Neural Network Approach Towards Self-Driving Cars
Venue: arXiv cs.RO 2019
Year: 2019

A convolutional neural network (CNN) approach is used to implement a level 2 autonomous vehicle by mapping pixels from the camera input to the steering commands. The network automatically learns the maximum variable features from the camera input, hence requires minimal human intervention. Given realistic frames as input, the driving policy trained on the dataset by NVIDIA and Udacity can adapt to real-world driving in a controlled environment. The CNN is tested on the CARLA open-source driving simulator. Details of a beta-testing platform are also presented, which consists of an ultrasonic sensor for obstacle detection and an RGBD camera for real-time position monitoring at 10Hz. Arduino Mega and Raspberry Pi are used for motor control and processing respectively to output the steering angle, which is converted to angular velocity for steering.
Deterministic Value-Policy Gradients
Venue: arXiv cs.LG 2019
Year: 2019

Reinforcement learning algorithms such as the deep deterministic policy gradient algorithm (DDPG) has been widely used in continuous control tasks. However, the model-free DDPG algorithm suffers from high sample complexity. In this paper we consider the deterministic value gradients to improve the sample efficiency of deep reinforcement learning algorithms. Previous works consider deterministic value gradients with the finite horizon, but it is too myopic compared with infinite horizon. We firstly give a theoretical guarantee of the existence of the value gradients in this infinite setting. Based on this theoretical guarantee, we propose a class of the deterministic value gradient algorithm (DVG) with infinite horizon, and different rollout steps of the analytical gradients by the learned model trade off between the variance of the value gradients and the model bias. Furthermore, to better combine the model-based deterministic value gradient estimators with the model-free deterministic policy gradient estimator, we propose the deterministic value-policy gradient (DVPG) algorithm. We finally conduct extensive experiments comparing DVPG with state-of-the-art methods on several standard continuous control benchmarks. Results demonstrate that DVPG substantially outperforms other baselines.
Unsupervised Domain Adaptation for Depth Prediction from Images
Venue: arXiv cs.CV 2019
Year: 2019

State-of-the-art approaches to infer dense depth measurements from images rely on CNNs trained end-to-end on a vast amount of data. However, these approaches suffer a drastic drop in accuracy when dealing with environments much different in appearance and/or context from those observed at training time. This domain shift issue is usually addressed by fine-tuning on smaller sets of images from the target domain annotated with depth labels. Unfortunately, relying on such supervised labeling is seldom feasible in most practical settings. Therefore, we propose an unsupervised domain adaptation technique which does not require groundtruth labels. Our method relies only on image pairs and leverages on classical stereo algorithms to produce disparity measurements alongside with confidence estimators to assess upon their reliability. We propose to fine-tune both depth-from-stereo as well as depth-from-mono architectures by a novel confidence-guided loss function that handles the measured disparities as noisy labels weighted according to the estimated confidence. Extensive experimental results based on standard datasets and evaluation protocols prove that our technique can address effectively the domain shift issue with both stereo and monocular depth prediction architectures and outperforms other state-of-the-art unsupervised loss functions that may be alternatively deployed to pursue domain adaptation.
Incremental learning of environment interactive structures from trajectories of individuals
Venue: arXiv cs.LG 2019
Year: 2019

This work proposes a novel method for estimating the influence that unknown static objects might have over mobile agents. Since the motion of agents can be affected by the presence of fixed objects, it is possible use the information about trajectories deviations to infer the presence of obstacles and estimate the forces involved in a scene. Artificial neural networks are used to estimate a non-parametric function related to the velocity field influencing moving agents. The proposed method is able to incrementally learn the velocity fields due to external static objects within the monitored environment. It determines whether an object has a repulsive or an attractive influence and provides an estimation of its position and size. As stationarity is assumed, i.e., time-invariance of force fields, learned observation models can be used as prior knowledge for estimating hierarchically the properties of new objects in a scene.
Lattice-Based Fuzzy Medical Expert System for Low Back Pain Management
Venue: arXiv cs.AI 2019
Year: 2019

Low Back Pain (LBP) is a common medical condition that deprives many individuals worldwide of their normal routine activities. In the absence of external biomarkers, diagnosis of LBP is quite challenging. It requires dealing with several clinical variables, which have no precisely quantified values. Aiming at the development of a fuzzy medical expert system for LBP management, this research proposes an attractive lattice-based knowledge representation scheme for handling imprecision in knowledge, offering a suitable design methodology for a fuzzy knowledge base and a fuzzy inference system. The fuzzy knowledge base is constructed in modular fashion, with each module capturing interrelated medical knowledge about the relevant clinical history, clinical examinations and laboratory investigation results. This approach in design ensures optimality, consistency and preciseness in the knowledge base and scalability. The fuzzy inference system, which uses the Mamdani method, adopts the triangular membership function for fuzzification and the Centroid of Area technique for defuzzification. A prototype of this system has been built using the knowledge extracted from the domain expert physicians. The inference of the system against a few available patient records at the ESI Hospital, Sealdah has been checked. It was found to be acceptable by the verifying medical experts.
The Trumpiest Trump? Identifying a Subject's Most Characteristic Tweets
Venue: arXiv cs.CL 2019
Year: 2019

The sequence of documents produced by any given author varies in style and content, but some documents are more typical or representative of the source than others. We quantify the extent to which a given short text is characteristic of a specific person, using a dataset of tweets from fifteen celebrities. Such analysis is useful for generating excerpts of high-volume Twitter profiles, and understanding how representativeness relates to tweet popularity. We first consider the related task of binary author detection (is x the author of text T?), and report a test accuracy of 90.37% for the best of five approaches to this problem. We then use these models to compute characterization scores among all of an author's texts. A user study shows human evaluators agree with our characterization model for all 15 celebrities in our dataset, each with p-value < 0.05. We use these classifiers to show surprisingly strong correlations between characterization scores and the popularity of the associated texts. Indeed, we demonstrate a statistically significant correlation between this score and tweet popularity (likes/replies/retweets) for 13 of the 15 celebrities in our study.
A Flexible Framework for Anomaly Detection via Dimensionality Reduction
Venue: arXiv cs.LG 2019
Year: 2019

Anomaly detection is challenging, especially for large datasets in high dimensions. Here we explore a general anomaly detection framework based on dimensionality reduction and unsupervised clustering. We release DRAMA, a general python package that implements the general framework with a wide range of built-in options. We test DRAMA on a wide variety of simulated and real datasets, in up to 3000 dimensions, and find it robust and highly competitive with commonly-used anomaly detection algorithms, especially in high dimensions. The flexibility of the DRAMA framework allows for significant optimization once some examples of anomalies are available, making it ideal for online anomaly detection, active learning and highly unbalanced datasets.
Towards Learning a Self-inverse Network for Bidirectional Image-to-image Translation
Venue: arXiv cs.CV 2019
Year: 2019

The one-to-one mapping is necessary for many bidirectional image-to-image translation applications, such as MRI image synthesis as MRI images are unique to the patient. State-of-the-art approaches for image synthesis from domain X to domain Y learn a convolutional neural network that meticulously maps between the domains. A different network is typically implemented to map along the opposite direction, from Y to X. In this paper, we explore the possibility of only wielding one network for bi-directional image synthesis. In other words, such an autonomous learning network implements a self-inverse function. A self-inverse network shares several distinct advantages: only one network instead of two, better generalization and more restricted parameter space. Most importantly, a self-inverse function guarantees a one-to-one mapping, a property that cannot be guaranteed by earlier approaches that are not self-inverse. The experiments on three datasets show that, compared with the baseline approaches that use two separate models for the image synthesis along two directions, our self-inverse network achieves better synthesis results in terms of standard metrics. Finally, our sensitivity analysis confirms the feasibility of learning a self-inverse function for the bidirectional image translation.
Structural Commutation Relations for Stochastic Labelled Graph Grammar Rule Operators
Venue: arXiv cs.FL 2019
Year: 2019

We show how to calculate the operator algebra and the operator Lie algebra of a stochastic labelled-graph grammar. More specifically, we carry out a generic calculation of the product (and therefore the commutator) of time-evolution operators for any two labelled-graph grammar rewrite rules, where the operator corresponding to each rule is defined in terms of elementary two-state creation/annihilation operators. The resulting graph grammar algebra has the following properties: (1) The product and commutator of two such operators is a sum of such operators with integer coefficients. Thus, the algebra and the Lie algebra occurs entirely at the structural (or graph-combinatorial) level of graph grammar rules, lifted from the level of elementary creation/annihilation operators (an improvement over [1], Propositions 1 and 2). (2) The product of the off-diagonal (state-changing) parts of two such graph rule operators is a sum of off-diagonal graph rule operators with non-negative integer coefficients. (3) These results apply whether the semantics of a graph grammar rule leaves behind hanging edges (Theorem 1), or removes hanging edges (Theorem 2). (4) The algebra is constructive in terms of elementary two-state creation/annihilation operators (Corollaries 3 and 8). These results are useful because dynamical transformations of labelled graphs comprise a general modeling framework, and algebraic commutators of time-evolution operators have many analytic uses including designing simulation algorithms and estimating their errors.
Sensitivity Analysis for Markov Decision Process Congestion Games
Venue: arXiv cs.GT 2019
Year: 2019

We consider a non-atomic congestion game where each decision maker performs selfish optimization over states of a common MDP. The decision makers optimize for their own expected costs, and influence each other through congestion effects on the state-action costs. We analyze on the sensitivity of MDP congestion game equilibria to uncertainty and perturbations in the state-action costs by applying an implicit function type analysis. The occurrence of a stochastic Braess paradox is defined, analyzed based on sensitivity of game equilibria and demonstrated in simulation. We further analyze how the introduction of stochastic dynamics affects the magnitude of Braess paradox in comparison to deterministic dynamics.
BERT-Based Arabic Social Media Author Profiling
Venue: arXiv cs.CL 2019
Year: 2019

We report our models for detecting age, language variety, and gender from social media data in the context of the Arabic author profiling and deception detection shared task (APDA). We build simple models based on pre-trained bidirectional encoders from transformers (BERT). We first fine-tune the pre-trained BERT model on each of the three datasets with shared task released data. Then we augment shared task data with in-house data for gender and dialect, showing the utility of augmenting training data. Our best models on the shared task test data are acquired with a majority voting of various BERT models trained under different data conditions. We acquire 54.72% accuracy for age, 93.75% for dialect, 81.67% for gender, and 40.97% joint accuracy across the three tasks.
NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

The problem of explaining deep learning models, and model predictions generally, has attracted intensive interest recently. Many successful approaches forgo global approximations in order to provide more faithful local interpretations of the model's behavior. LIME develops multiple interpretable models, each approximating a large neural network on a small region of the data manifold and SP-LIME aggregates the local models to form a global interpretation. Extending this line of research, we propose a simple yet effective method, NormLIME for aggregating local models into global and class-specific interpretations. A human user study strongly favored class-specific interpretations created by NormLIME to other feature importance metrics. Numerical experiments confirm that NormLIME is effective at recognizing important features.
Multimodal Embeddings from Language Models
Venue: arXiv cs.CL 2019
Year: 2019

Word embeddings such as ELMo have recently been shown to model word semantics with greater efficacy through contextualized learning on large-scale language corpora, resulting in significant improvement in state of the art across many natural language tasks. In this work we integrate acoustic information into contextualized lexical embeddings through the addition of multimodal inputs to a pretrained bidirectional language model. The language model is trained on spoken language that includes text and audio modalities. The resulting representations from this model are multimodal and contain paralinguistic information which can modify word meanings and provide affective information. We show that these multimodal embeddings can be used to improve over previous state of the art multimodal models in emotion recognition on the CMU-MOSEI dataset.
Automatic difficulty management and testing in games using a framework based on behavior trees and genetic algorithms
Venue: arXiv cs.AI 2019
Year: 2019

The diversity of agent behaviors is an important topic for the quality of video games and virtual environments in general. Offering the most compelling experience for users with different skills is a difficult task, and usually needs important manual human effort for tuning existing code. This can get even harder when dealing with adaptive difficulty systems. Our paper's main purpose is to create a framework that can automatically create behaviors for game agents of different difficulty classes and enough diversity. In parallel with this, a second purpose is to create more automated tests for showing defects in the source code or possible logic exploits with less human effort.
Cloud ArcGIS Online as an innovative tool for developing geoinformation competence with future geography teachers
Venue: arXiv cs.CY 2019
Year: 2019

The article dwells upon the scientifically relevant problem of using cloud-based GIS-technologies when training future geography teachers (based on ArcGIS Online application). The authors outline the basic principles for implementing ArcGIS Online in the educational process (interdisciplinary integration, the sequence of individualization in training, communicability, distance education and regional studies), and provide an example of an interactive map created with the help of the specified cloud GIS, since this kind of map is the most popular a form of research by geography students. In the article it is noted that integration of ArcGIS Online into the educational process allows the teacher to follow a clear pedagogical strategy, taking into account possible variants of its use (demonstration, direct mastering of GIS in a computer class and independent work in an individual mode). Considering cloud GIS as a new stage in the development of geoinformational education, the authors emphasize their key benefits (round-the-clock access, work with GIS package in the cloud, the ability to use other maps as well as the creation of their own maps and webapplications) and disadvantages (monetization of services, underestimation of the GIS role in the curriculum of the higher school, the lack of Ukrainian content, etc.).
An Overview of Open-Ended Evolution: Editorial Introduction to the Open-Ended Evolution II Special Issue
Venue: arXiv cs.NE 2019
Year: 2019

Nature's spectacular inventiveness, reflected in the enormous diversity of form and function displayed by the biosphere, is a feature of life that distinguishes living most strongly from nonliving. It is, therefore, not surprising that this aspect of life should become a central focus of artificial life. We have known since Darwin that the diversity is produced dynamically, through the process of evolution; this has led life's creative productivity to be called Open-Ended Evolution (OEE) in the field. This article introduces the second of two special issues on current research in OEE and provides an overview of the contents of both special issues. Most of the work was presented at a workshop on open-ended evolution that was held as a part of the 2018 Conference on Artificial Life in Tokyo, and much of it had antecedents in two previous workshops on open-ended evolution at artificial life conferences in Cancun and York. We present a simplified categorization of OEE and summarize progress in the field as represented by the articles in this special issue.
The Prevalence of Errors in Machine Learning Experiments
Venue: arXiv cs.LG 2019
Year: 2019

Context: Conducting experiments is central to research machine learning research to benchmark, evaluate and compare learning algorithms. Consequently it is important we conduct reliable, trustworthy experiments. Objective: We investigate the incidence of errors in a sample of machine learning experiments in the domain of software defect prediction. Our focus is simple arithmetical and statistical errors. Method: We analyse 49 papers describing 2456 individual experimental results from a previously undertaken systematic review comparing supervised and unsupervised defect prediction classifiers. We extract the confusion matrices and test for relevant constraints, e.g., the marginal probabilities must sum to one. We also check for multiple statistical significance testing errors. Results: We find that a total of 22 out of 49 papers contain demonstrable errors. Of these 7 were statistical and 16 related to confusion matrix inconsistency (one paper contained both classes of error). Conclusions: Whilst some errors may be of a relatively trivial nature, e.g., transcription errors their presence does not engender confidence. We strongly urge researchers to follow open science principles so errors can be more easily be detected and corrected, thus as a community reduce this worryingly high error rate with our computational experiments.
Natural Adversarial Sentence Generation with Gradient-based Perturbation
Venue: arXiv cs.IR 2019
Year: 2019

This work proposes a novel algorithm to generate natural language adversarial input for text classification models, in order to investigate the robustness of these models. It involves applying gradient-based perturbation on the sentence embeddings that are used as the features for the classifier, and learning a decoder for generation. We employ this method to a sentiment analysis model and verify its effectiveness in inducing incorrect predictions by the model. We also conduct quantitative and qualitative analysis on these examples and demonstrate that our approach can generate more natural adversaries. In addition, it can be used to successfully perform black-box attacks, which involves attacking other existing models whose parameters are not known. On a public sentiment analysis API, the proposed method introduces a 20% relative decrease in average accuracy and 74% relative increase in absolute error.