OpenRaiser/Intern-Atlas huggingface dataset:
SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards
Venue: arXiv cs.LG 2019
Year: 2019

Learning to imitate expert behavior from demonstrations can be challenging, especially in environments with high-dimensional, continuous observations and unknown dynamics. Supervised learning methods based on behavioral cloning (BC) suffer from distribution shift: because the agent greedily imitates demonstrated actions, it can drift away from demonstrated states due to error accumulation. Recent methods based on reinforcement learning (RL), such as inverse RL and generative adversarial imitation learning (GAIL), overcome this issue by training an RL agent to match the demonstrations over a long horizon. Since the true reward function for the task is unknown, these methods learn a reward function from the demonstrations, often using complex and brittle approximation techniques that involve adversarial training. We propose a simple alternative that still uses RL, but does not require learning a reward function. The key idea is to provide the agent with an incentive to match the demonstrations over a long horizon, by encouraging it to return to demonstrated states upon encountering new, out-of-distribution states. We accomplish this by giving the agent a constant reward of r=+1 for matching the demonstrated action in a demonstrated state, and a constant reward of r=0 for all other behavior. Our method, which we call soft Q imitation learning (SQIL), can be implemented with a handful of minor modifications to any standard Q-learning or off-policy actor-critic algorithm. Theoretically, we show that SQIL can be interpreted as a regularized variant of BC that uses a sparsity prior to encourage long-horizon imitation. Empirically, we show that SQIL outperforms BC and achieves competitive results compared to GAIL, on a variety of image-based and low-dimensional tasks in Box2D, Atari, and MuJoCo.
DockerizeMe: Automatic Inference of Environment Dependencies for Python Code Snippets
Venue: arXiv cs.SE 2019
Year: 2019

Platforms like Stack Overflow and GitHub's gist system promote the sharing of ideas and programming techniques via the distribution of code snippets designed to illustrate particular tasks. Python, a popular and fast-growing programming language, sees heavy use on both sites, with nearly one million questions asked on Stack Overflow and 400 thousand public gists on GitHub. Unfortunately, around 75% of the Python example code shared through these sites cannot be directly executed. When run in a clean environment, over 50% of public Python gists fail due to an import error for a missing library. We present DockerizeMe, a technique for inferring the dependencies needed to execute a Python code snippet without import error. DockerizeMe starts with offline knowledge acquisition of the resources and dependencies for popular Python packages from the Python Package Index (PyPI). It then builds Docker specifications using a graph-based inference procedure. Our inference procedure resolves import errors in 892 out of nearly 3,000 gists from the Gistable dataset for which Gistable's baseline approach could not find and install all dependencies.
Learning to Detect and Retrieve Objects from Unlabeled Videos
Venue: arXiv cs.CV 2019
Year: 2019

Learning an object detector or retrieval requires a large data set with manual annotations. Such data sets are expensive and time consuming to create and therefore difficult to obtain on a large scale. In this work, we propose to exploit the natural correlation in narrations and the visual presence of objects in video, to learn an object detector and retrieval without any manual labeling involved. We pose the problem as weakly supervised learning with noisy labels, and propose a novel object detection paradigm under these constraints. We handle the background rejection by using contrastive samples and confront the high level of label noise with a new clustering score. Our evaluation is based on a set of 11 manually annotated objects in over 5000 frames. We show comparison to a weakly-supervised approach as baseline and provide a strongly labeled upper bound.
Path Planning Problems with Side Observations-When Colonels Play Hide-and-Seek
Venue: arXiv cs.GT 2019
Year: 2019

Resource allocation games such as the famous Colonel Blotto (CB) and Hide-and-Seek (HS) games are often used to model a large variety of practical problems, but only in their one-shot versions. Indeed, due to their extremely large strategy space, it remains an open question how one can efficiently learn in these games. In this work, we show that the online CB and HS games can be cast as path planning problems with side-observations (SOPPP): at each stage, a learner chooses a path on a directed acyclic graph and suffers the sum of losses that are adversarially assigned to the corresponding edges; and she then receives semi-bandit feedback with side-observations (i.e., she observes the losses on the chosen edges plus some others). We propose a novel algorithm, EXP3-OE, the first-of-its-kind with guaranteed efficient running time for SOPPP without requiring any auxiliary oracle. We provide an expected-regret bound of EXP3-OE in SOPPP matching the order of the best benchmark in the literature. Moreover, we introduce additional assumptions on the observability model under which we can further improve the regret bounds of EXP3-OE. We illustrate the benefit of using EXP3-OE in SOPPP by applying it to the online CB and HS games.
Hierarchy of Transportation Network Parameters and Hardness Results
Venue: arXiv cs.DM 2019
Year: 2019

The graph parameters highway dimension and skeleton dimension were introduced to capture the properties of transportation networks. As many important optimization problems like Travelling Salesperson, Steiner Tree or $k$-Center arise in such networks, it is worthwhile to study them on graphs of bounded highway or skeleton dimension. We investigate the relationships between mentioned parameters and how they are related to other important graph parameters that have been applied successfully to various optimization problems. We show that the skeleton dimension is incomparable to any of the parameters distance to linear forest, bandwidth, treewidth and highway dimension and hence, it is worthwhile to study mentioned problems also on graphs of bounded skeleton dimension. Moreover, we prove that the skeleton dimension is upper bounded by the max leaf number and that for any graph on at least three vertices there are edge weights such that both parameters are equal. Then we show that computing the highway dimension according to most recent definition is NP-hard, which answers an open question stated by Feldmann et al. Finally we prove that on graphs $G=(V,E)$ of skeleton dimension $\mathcal{O}(\log^2 \vert V \vert)$ it is NP-hard to approximate the $k$-Center problem within a factor less than $2$.
Development of an Intuitive Foot-Machine Interface for Robotic Surgery
Venue: arXiv cs.RO 2019
Year: 2019

The human-machine interface is of critical importance for master-slave control of the robotic system for surgery, in which current systems offer the control or two robotic arms teleoperated by the surgeon's hands. To relax the need for surgical assistants and augment dexterity in surgery, it has been recently proposed to use a robot like a third arm that can be controlled seamlessly, independently from the natural arms, and work together with them. This report will develop and investigate this concept by implementing foot control of a robotic surgical arm. A novel passive haptic foot-machine interface system and analysis of its performances were introduced in this report. This interface using a parallel-serial hybrid structure with springs and force sensors, which allows intuitive control of a slave robotic arm with four degrees of freedom (dof). The elastic isometric design enables a user to control the interface system accurately and adaptively, with an enlarged sensing range breaking the physical restriction of the pedal size. A subject specific (independent component analysis, ICA) model is identified to map the surgeon's foot movements into kinematic parameters of the slave robotic arm. To validate the system and assess the performance it allows, 10 subjects carried out experiments to manipulate the foot-machine interface system in various movements. With these experimental data, the mapping models were built and verified. A comparison between different mapping models was made and analyzed proving the ICA algorithm is obviously dominant over other methods.
Decision-making with reference information
Venue: arXiv cs.GT 2019
Year: 2019

We often try to predict others' actions by obtaining supporting information that shows a preference index of surrounding people. In order to reproduce these situations, we propose a game named "One-sided Preference Game with Reference Information (OSPG-R)." We conducted experiments in which players who have similar preferences compete for objects in OSPG-R. In the experiment, we used three different types of objects: boxes, faces, and cars. Our results show that the most frequently selected object was not the most popular one. In order to gain deeper insights into the experiment's results, we constructed a decision-making model based on two assumptions: (1) players are rational and (2) are convinced that the other players' preference orders are equivalent to the preference index for the group. Compared to the choice behavior of the model, the experiment's results show that there was a tendency to take risks when the objects were faces, or the priority of that particular player was low.
Induction of Non-Monotonic Rules From Statistical Learning Models Using High-Utility Itemset Mining
Venue: arXiv cs.LG 2019
Year: 2019

We present a fast and scalable algorithm to induce non-monotonic logic programs from statistical learning models. We reduce the problem of search for best clauses to instances of the High-Utility Itemset Mining (HUIM) problem. In the HUIM problem, feature values and their importance are treated as transactions and utilities respectively. We make use of TreeExplainer, a fast and scalable implementation of the Explainable AI tool SHAP, to extract locally important features and their weights from ensemble tree models. Our experiments with UCI standard benchmarks suggest a significant improvement in terms of classification evaluation metrics and running time of the training algorithm compared to ALEPH, a state-of-the-art Inductive Logic Programming (ILP) system.
Document Embeddings vs. Keyphrases vs. Terms: An Online Evaluation in Digital Library Recommender Systems
Venue: arXiv cs.IR 2019
Year: 2019

Many recommendation algorithms are available to digital library recommender system operators. The effectiveness of algorithms is largely unreported by way of online evaluation. We compare a standard term-based recommendation approach to two promising approaches for related-article recommendation in digital libraries: document embeddings, and keyphrases. We evaluate the consistency of their performance across multiple scenarios. Through our recommender-as-a-service Mr. DLib, we delivered 33.5M recommendations to users of Sowiport and Jabref over the course of 19 months, from March 2017 to October 2018. The effectiveness of the algorithms differs significantly between Sowiport and Jabref (Wilcoxon rank-sum test; p < 0.05). There is a ~400% difference in effectiveness between the best and worst algorithm in both scenarios separately. The best performing algorithm in Sowiport (terms) is the worst performing in Jabref. The best performing algorithm in Jabref (keyphrases) is 70% worse in Sowiport, than Sowiport`s best algorithm (click-through rate; 0.1% terms, 0.03% keyphrases).
Radar-based Feature Design and Multiclass Classification for Road User Recognition
Venue: arXiv cs.LG 2019
Year: 2019

The classification of individual traffic participants is a complex task, especially for challenging scenarios with multiple road users or under bad weather conditions. Radar sensors provide an - with respect to well established camera systems - orthogonal way of measuring such scenes. In order to gain accurate classification results, 50 different features are extracted from the measurement data and tested on their performance. From these features a suitable subset is chosen and passed to random forest and long short-term memory (LSTM) classifiers to obtain class predictions for the radar input. Moreover, it is shown why data imbalance is an inherent problem in automotive radar classification when the dataset is not sufficiently large. To overcome this issue, classifier binarization is used among other techniques in order to better account for underrepresented classes. A new method to couple the resulting probabilities is proposed and compared to others with great success. Final results show substantial improvements when compared to ordinary multiclass classification
ImgSensingNet: UAV Vision Guided Aerial-Ground Air Quality Sensing System
Venue: arXiv cs.SY 2019
Year: 2019

Given the increasingly serious air pollution problem, the monitoring of air quality index (AQI) in urban areas has drawn considerable attention. This paper presents ImgSensingNet, a vision guided aerial-ground sensing system, for fine-grained air quality monitoring and forecasting using the fusion of haze images taken by the unmanned-aerial-vehicle (UAV) and the AQI data collected by an on-ground three-dimensional (3D) wireless sensor network (WSN). Specifically, ImgSensingNet first leverages the computer vision technique to tell the AQI scale in different regions from the taken haze images, where haze-relevant features and a deep convolutional neural network (CNN) are designed for direct learning between haze images and corresponding AQI scale. Based on the learnt AQI scale, ImgSensingNet determines whether to wake up on-ground wireless sensors for small-scale AQI monitoring and inference, which can greatly reduce the energy consumption of the system. An entropy-based model is employed for accurate real-time AQI inference at unmeasured locations and future air quality distribution forecasting. We implement and evaluate ImgSensingNet on two university campuses since Feb. 2018, and has collected 17,630 photos and 2.6 millions of AQI data samples. Experimental results confirm that ImgSensingNet can achieve higher inference accuracy while greatly reduce the energy consumption, compared to state-of-the-art AQI monitoring approaches.
A State Observer Design for Simultaneous Estimation of Charge State and Crossover in Self-Discharging Disproportionation Redox Flow Batteries
Venue: arXiv cs.SY 2019
Year: 2019

This paper presents an augmented state observer design for the simultaneous estimation of charge state and crossover flux in disproportionation redox flow batteries, which exhibits exponential estimation error convergence to a bounded residual set. The crossover flux of vanadium through the porous separator is considered as an unknown function of the battery states, model-approximated as the output of a persistently excited linear system. This parametric model and the simple isothermal lumped parameter model of the battery are combined to form an augmented space state representation suitable for the observer design, which is carried out via Lyapunov stability theory including the error-uncertainty involved in the approximation of the crossover flux. The observer gain is calculated by solving a polytopic linear matrix inequality problem via convex optimization. The performance of this design is evaluated with a laboratory flow battery prototype undergoing self-discharge.
A Survey of Adaptive Resonance Theory Neural Network Models for Engineering Applications
Venue: arXiv cs.NE 2019
Year: 2019

This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to modern ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers.
Label Universal Targeted Attack
Venue: arXiv cs.CR 2019
Year: 2019

We introduce Label Universal Targeted Attack (LUTA) that makes a deep model predict a label of attacker's choice for `any' sample of a given source class with high probability. Our attack stochastically maximizes the log-probability of the target label for the source class with first order gradient optimization, while accounting for the gradient moments. It also suppresses the leakage of attack information to the non-source classes for avoiding the attack suspicions. The perturbations resulting from our attack achieve high fooling ratios on the large-scale ImageNet and VGGFace models, and transfer well to the Physical World. Given full control over the perturbation scope in LUTA, we also demonstrate it as a tool for deep model autopsy. The proposed attack reveals interesting perturbation patterns and observations regarding the deep models.
Implicit Rugosity Regularization via Data Augmentation
Venue: arXiv cs.LG 2019
Year: 2019

Deep (neural) networks have been applied productively in a wide range of supervised and unsupervised learning tasks. Unlike classical machine learning algorithms, deep networks typically operate in the \emph{overparameterized} regime, where the number of parameters is larger than the number of training data points. Consequently, understanding the generalization properties and the role of (explicit or implicit) regularization in these networks is of great importance. In this work, we explore how the oft-used heuristic of \emph{data augmentation} imposes an {\em implicit regularization} penalty of a novel measure of the \emph{rugosity} or "roughness" based on the tangent Hessian of the function fit to the training data.
Lane-free Artificial-Fluid Concept for Vehicular Traffic
Venue: arXiv cs.SY 2019
Year: 2019

A novel paradigm for vehicular traffic in the era of connected and automated vehicles (CAVs) is proposed, which includes two combined principles: lane-free traffic and vehicle nudging, whereby vehicles are "pushing" (from a distance, using communication or sensors) other vehicles in front of them. This traffic paradigm features several advantages, including: smoother and safer driving; increase of roadway capacity; and no need for the anisotropy restriction. The proposed concept provides, for the first time since the automobile invention, the possibility to actively design (rather than describe) the traffic flow characteristics in an optimal way, i.e. to engineer the future CAV traffic flow as an efficient artificial fluid. Options, features, application domains and required research topics are discussed. Preliminary simulation results illustrate some basic features of the concept.
On Measuring Gender Bias in Translation of Gender-neutral Pronouns
Venue: arXiv cs.CL 2019
Year: 2019

Ethics regarding social bias has recently thrown striking issues in natural language processing. Especially for gender-related topics, the need for a system that reduces the model bias has grown in areas such as image captioning, content recommendation, and automated employment. However, detection and evaluation of gender bias in the machine translation systems are not yet thoroughly investigated, for the task being cross-lingual and challenging to define. In this paper, we propose a scheme for making up a test set that evaluates the gender bias in a machine translation system, with Korean, a language with gender-neutral pronouns. Three word/phrase sets are primarily constructed, each incorporating positive/negative expressions or occupations; all the terms are gender-independent or at least not biased to one side severely. Then, additional sentence lists are constructed concerning formality of the pronouns and politeness of the sentences. With the generated sentence set of size 4,236 in total, we evaluate gender bias in conventional machine translation systems utilizing the proposed measure, which is termed here as translation gender bias index (TGBI). The corpus and the code for evaluation is available on-line.
Optimizing Adaptive Video Streaming in Mobile Networks via Online Learning
Venue: arXiv cs.MM 2019
Year: 2019

In this paper, we propose a novel algorithm for video rate adaptation in HTTP Adaptive Streaming (HAS), based on online learning. The proposed algorithm, named Learn2Adapt (L2A), is shown to provide a robust rate adaptation strategy which, unlike most of the state-of-the-art techniques, does not require parameter tuning, channel model assumptions or application-specific adjustments. These properties make it very suitable for mobile users, who typically experience fast variations in channel characteristics. Simulations show that L2A improves on the overall Quality of Experience (QoE) and in particular the average streaming rate, a result obtained independently of the channel and application scenarios.
Extracting adverse drug reactions and their context using sequence labelling ensembles in TAC2017
Venue: arXiv cs.CL 2019
Year: 2019

Adverse drug reactions (ADRs) are unwanted or harmful effects experienced after the administration of a certain drug or a combination of drugs, presenting a challenge for drug development and drug administration. In this paper, we present a set of taggers for extracting adverse drug reactions and related entities, including factors, severity, negations, drug class and animal. The systems used a mix of rule-based, machine learning (CRF) and deep learning (BLSTM with word2vec embeddings) methodologies in order to annotate the data. The systems were submitted to adverse drug reaction shared task, organised during Text Analytics Conference in 2017 by National Institute for Standards and Technology, archiving F1-scores of 76.00 and 75.61 respectively.
Evaluating time series forecasting models: An empirical study on performance estimation methods
Venue: arXiv cs.LG 2019
Year: 2019

Performance estimation aims at estimating the loss that a predictive model will incur on unseen data. These procedures are part of the pipeline in every machine learning project and are used for assessing the overall generalisation ability of predictive models. In this paper we address the application of these methods to time series forecasting tasks. For independent and identically distributed data the most common approach is cross-validation. However, the dependency among observations in time series raises some caveats about the most appropriate way to estimate performance in this type of data and currently there is no settled way to do so. We compare different variants of cross-validation and of out-of-sample approaches using two case studies: One with 62 real-world time series and another with three synthetic time series. Results show noticeable differences in the performance estimation methods in the two scenarios. In particular, empirical experiments suggest that cross-validation approaches can be applied to stationary time series. However, in real-world scenarios, when different sources of non-stationary variation are at play, the most accurate estimates are produced by out-of-sample methods that preserve the temporal order of observations.