pwc-archive/papers-with-abstracts huggingface dataset:
Learning agile and dynamic motor skills for legged robots
Authors: Jemin Hwangbo, JoonHo Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, Marco Hutter
Date: 2019-01-24 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Legged robots pose one of the greatest challenges in robotics. Dynamic and
agile maneuvers of animals cannot be imitated by existing methods that are
crafted by humans. A compelling alternative is reinforcement learning, which
requires minimal craftsmanship and promotes the natural evolution of a control
policy. However, so far, reinforcement learning research for legged robots is
mainly limited to simulation, and only few and comparably simple examples have
been deployed on real systems. The primary reason is that training with real
robots, particularly with dynamically balancing systems, is complicated and
expensive. In the present work, we introduce a method for training a neural
network policy in simulation and transferring it to a state-of-the-art legged
system, thereby leveraging fast, automated, and cost-effective data generation
schemes. The approach is applied to the ANYmal robot, a sophisticated
medium-dog-sized quadrupedal system. Using policies trained in simulation, the
quadrupedal machine achieves locomotion skills that go beyond what had been
achieved with prior methods: ANYmal is capable of precisely and
energy-efficiently following high-level body velocity commands, running faster
than before, and recovering from falling even in complex configurations.
Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics
Authors: Antonin Raffin, Ashley Hill, René Traoré, Timothée Lesort, Natalia Díaz-Rodríguez, David Filliat
Date: 2019-01-24 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Representation Learning

Scaling end-to-end reinforcement learning to control real robots from vision presents a series of challenges, in particular in terms of sample efficiency. Against end-to-end learning, state representation learning can help learn a compact, efficient and relevant representation of states that speeds up policy learning, reducing the number of samples needed, and that is easier to interpret. We evaluate several state representation learning methods on goal based robotics tasks and propose a new unsupervised model that stacks representations and combines strengths of several of these approaches. This method encodes all the relevant features, performs on par or better than end-to-end learning with better sample efficiency, and is robust to hyper-parameters change.
Learning Independently-Obtainable Reward Functions
Authors: Christopher Grimm, Satinder Singh
Date: 2019-01-24 00:00:00

We present a novel method for learning a set of disentangled reward functions
that sum to the original environment reward and are constrained to be
independently obtainable. We define independent obtainability in terms of value
functions with respect to obtaining one learned reward while pursuing another
learned reward. Empirically, we illustrate that our method can learn meaningful
reward decompositions in a variety of domains and that these decompositions
exhibit some form of generalization performance when the environment's reward
is modified. Theoretically, we derive results about the effect of maximizing
our method's objective on the resulting reward functions and their
corresponding optimal policies.
Dynamic Measurement Scheduling for Event Forecasting using Deep RL
Authors: Chun-Hao Chang, Mingjie Mai, Anna Goldenberg
Date: 2019-01-24 00:00:00
Tasks: Deep Reinforcement Learning, ICU Mortality, Mortality Prediction, Reinforcement Learning, Reinforcement Learning (RL), Scheduling

Imagine a patient in critical condition. What and when should be measured to forecast detrimental events, especially under the budget constraints? We answer this question by deep reinforcement learning (RL) that jointly minimizes the measurement cost and maximizes predictive gain, by scheduling strategically-timed measurements. We learn our policy to be dynamically dependent on the patient's health history. To scale our framework to exponentially large action space, we distribute our reward in a sequential setting that makes the learning easier. In our simulation, our policy outperforms heuristic-based scheduling with higher predictive gain and lower cost. In a real-world ICU mortality prediction task (MIMIC3), our policies reduce the total number of measurements by $31\%$ or improve predictive gain by a factor of $3$ as compared to physicians, under the off-policy policy evaluation.
Ablation Studies in Artificial Neural Networks
Authors: Richard Meyes, Melanie Lu, Constantin Waubert de Puiseau, Tobias Meisen
Date: 2019-01-24 00:00:00

Ablation studies have been widely used in the field of neuroscience to tackle
complex biological systems such as the extensively studied Drosophila central
nervous system, the vertebrate brain and more interestingly and most
delicately, the human brain. In the past, these kinds of studies were utilized
to uncover structure and organization in the brain, i.e. a mapping of features
inherent to external stimuli onto different areas of the neocortex. considering
the growth in size and complexity of state-of-the-art artificial neural
networks (ANNs) and the corresponding growth in complexity of the tasks that
are tackled by these networks, the question arises whether ablation studies may
be used to investigate these networks for a similar organization of their inner
representations. In this paper, we address this question and performed two
ablation studies in two fundamentally different ANNs to investigate their inner
representations of two well-known benchmark datasets from the computer vision
domain. We found that features distinct to the local and global structure of
the data are selectively represented in specific parts of the network.
Furthermore, some of these representations are redundant, awarding the network
a certain robustness to structural damages. We further determined the
importance of specific parts of the network for the classification task solely
based on the weight structure of single units. Finally, we examined the ability
of damaged networks to recover from the consequences of ablations by means of
recovery training. We argue that ablations studies are a feasible method to
investigate knowledge representations in ANNs and are especially helpful to
examine a networks robustness to structural damages, a feature of ANNs that
will become increasingly important for future safety-critical applications.
A BERT Baseline for the Natural Questions
Authors: Chris Alberti, Kenton Lee, Michael Collins
Date: 2019-01-24 00:00:00
Tasks: Natural Questions, Question Answering

This technical note describes a new baseline for the Natural Questions. Our model is based on BERT and reduces the gap between the model F1 scores reported in the original dataset paper and the human upper bound by 30% and 50% relative for the long and short answer tasks respectively. This baseline has been submitted to the official NQ leaderboard at ai.google.com/research/NaturalQuestions. Code, preprocessed data and pretrained model are available at https://github.com/google-research/language/tree/master/language/question_answering/bert_joint.
Fair k-Center Clustering for Data Summarization
Authors: Matthäus Kleindessner, Pranjal Awasthi, Jamie Morgenstern
Date: 2019-01-24 00:00:00
Tasks: Clustering, Data Summarization, Fairness

In data summarization we want to choose $k$ prototypes in order to summarize a data set. We study a setting where the data set comprises several demographic groups and we are restricted to choose $k_i$ prototypes belonging to group $i$. A common approach to the problem without the fairness constraint is to optimize a centroid-based clustering objective such as $k$-center. A natural extension then is to incorporate the fairness constraint into the clustering problem. Existing algorithms for doing so run in time super-quadratic in the size of the data set, which is in contrast to the standard $k$-center problem being approximable in linear time. In this paper, we resolve this gap by providing a simple approximation algorithm for the $k$-center problem under the fairness constraint with running time linear in the size of the data set and $k$. If the number of demographic groups is small, the approximation guarantee of our algorithm only incurs a constant-factor overhead.
Automatic Parallel Corpus Creation for Hindi-English News Translation Task
Authors: Aditya Kumar Pathak, Priyankit Acharya, Dilpreet Kaur, Rakesh Chandra Balabantaray
Date: 2019-01-24 00:00:00
Tasks: Machine Translation, Multilingual NLP, Translation

The parallel corpus for multilingual NLP tasks, deep learning applications
like Statistical Machine Translation Systems is very important. The parallel
corpus of Hindi-English language pair available for news translation task till
date is of very limited size as per the requirement of the systems are
concerned. In this work we have developed an automatic parallel corpus
generation system prototype, which creates Hindi-English parallel corpus for
news translation task. Further to verify the quality of generated parallel
corpus we have experimented by taking various performance metrics and the
results are quite interesting.
AutoShuffleNet: Learning Permutation Matrices via an Exact Lipschitz Continuous Penalty in Deep Convolutional Neural Networks
Authors: Jiancheng Lyu, Shuai Zhang, Yingyong Qi, Jack Xin
Date: 2019-01-24 00:00:00
Tasks: Graph Matching

ShuffleNet is a state-of-the-art light weight convolutional neural network
architecture. Its basic operations include group, channel-wise convolution and
channel shuffling. However, channel shuffling is manually designed empirically.
Mathematically, shuffling is a multiplication by a permutation matrix. In this
paper, we propose to automate channel shuffling by learning permutation
matrices in network training. We introduce an exact Lipschitz continuous
non-convex penalty so that it can be incorporated in the stochastic gradient
descent to approximate permutation at high precision. Exact permutations are
obtained by simple rounding at the end of training and are used in inference.
The resulting network, referred to as AutoShuffleNet, achieved improved
classification accuracies on CIFAR-10 and ImageNet data sets. In addition, we
found experimentally that the standard convex relaxation of permutation
matrices into stochastic matrices leads to poor performance. We prove
theoretically the exactness (error bounds) in recovering permutation matrices
when our penalty function is zero (very small). We present examples of
permutation optimization through graph matching and two-layer neural network
models where the loss functions are calculated in closed analytical form. In
the examples, convex relaxation failed to capture permutations whereas our
penalty succeeded.
Petrophysical Property Estimation from Seismic Data Using Recurrent Neural Networks
Authors: Motaz Alfarraj, Ghassan AlRegib
Date: 2019-01-24 00:00:00
Tasks: BIG-bench Machine Learning, regression

Reservoir characterization involves the estimation petrophysical properties
from well-log data and seismic data. Estimating such properties is a
challenging task due to the non-linearity and heterogeneity of the subsurface.
Various attempts have been made to estimate petrophysical properties using
machine learning techniques such as feed-forward neural networks and support
vector regression (SVR). Recent advances in machine learning have shown
promising results for recurrent neural networks (RNN) in modeling complex
sequential data such as videos and speech signals. In this work, we propose an
algorithm for property estimation from seismic data using recurrent neural
networks. An applications of the proposed workflow to estimate density and
p-wave impedance using seismic data shows promising results compared to
feed-forward neural networks.
Learned Belief-Propagation Decoding with Simple Scaling and SNR Adaptation
Authors: Mengke Lian, Fabrizio Carpi, Christian Häger, Henry D. Pfister
Date: 2019-01-24 00:00:00
Tasks: Decoder

We consider the weighted belief-propagation (WBP) decoder recently proposed
by Nachmani et al. where different weights are introduced for each Tanner graph
edge and optimized using machine learning techniques. Our focus is on
simple-scaling models that use the same weights across certain edges to reduce
the storage and computational burden. The main contribution is to show that
simple scaling with few parameters often achieves the same gain as the full
parameterization. Moreover, several training improvements for WBP are proposed.
For example, it is shown that minimizing average binary cross-entropy is
suboptimal in general in terms of bit error rate (BER) and a new "soft-BER"
loss is proposed which can lead to better performance. We also investigate
parameter adapter networks (PANs) that learn the relation between the
signal-to-noise ratio and the WBP parameters. As an example, for the (32,16)
Reed-Muller code with a highly redundant parity-check matrix, training a PAN
with soft-BER loss gives near-maximum-likelihood performance assuming simple
scaling with only three parameters.
A Stable Combinatorial Particle Swarm Optimization for Scalable Feature Selection in Gene Expression Data
Authors: Hassen Dhrif, Luis G. Sanchez Giraldo, Miroslav Kubat, Stefan Wuchty
Date: 2019-01-24 00:00:00
Tasks: Diagnostic, Diversity, feature selection, General Classification

Evolutionary computation (EC) algorithms, such as discrete and
multi-objective versions of particle swarm optimization (PSO), have been
applied to solve the Feature selection (FS) problem, tackling the combinatorial
explosion of search spaces that are peppered with local minima. Furthermore,
high-dimensional FS problems such as finding a small set of biomarkers to make
a diagnostic call add an additional challenge as such methods ability to pick
out the most important features must remain unchanged in decision spaces of
increasing dimensions and presence of irrelevant features. We developed a
combinatorial PSO algorithm, called COMB-PSO, that scales up to
high-dimensional gene expression data while still selecting the smallest
subsets of genes that allow reliable classification of samples. In particular,
COMB-PSO enhances the encoding, speed of convergence, control of divergence and
diversity of the conventional PSO algorithm, balancing exploration and
exploitation of the search space. Applying our approach on real gene expression
data of different cancers, COMB-PSO finds gene sets of smallest size that allow
a reliable classification of the underlying disease classes.
Boosting Standard Classification Architectures Through a Ranking Regularizer
Authors: Ahmed Taha, Yi-Ting Chen, Teruhisa Misu, Abhinav Shrivastava, Larry Davis
Date: 2019-01-24 00:00:00
Tasks: Classification, General Classification, Triplet

We employ triplet loss as a feature embedding regularizer to boost classification performance. Standard architectures, like ResNet and Inception, are extended to support both losses with minimal hyper-parameter tuning. This promotes generality while fine-tuning pretrained networks. Triplet loss is a powerful surrogate for recently proposed embedding regularizers. Yet, it is avoided due to large batch-size requirement and high computational cost. Through our experiments, we re-assess these assumptions. During inference, our network supports both classification and embedding tasks without any computational overhead. Quantitative evaluation highlights a steady improvement on five fine-grained recognition datasets. Further evaluation on an imbalanced video dataset achieves significant improvement. Triplet loss brings feature embedding characteristics like nearest neighbor to classification models. Code available at \url{http://bit.ly/2LNYEqL}.
Deep Neural Linear Bandits: Overcoming Catastrophic Forgetting through Likelihood Matching
Authors: Tom Zahavy, Shie Mannor
Date: 2019-01-24 00:00:00
Tasks: Decision Making, Efficient Exploration, General Classification, Multi-Armed Bandits, regression, Sentiment Analysis, Sequential Decision Making

We study the neural-linear bandit model for solving sequential decision-making problems with high dimensional side information. Neural-linear bandits leverage the representation power of deep neural networks and combine it with efficient exploration mechanisms, designed for linear contextual bandits, on top of the last hidden layer. Since the representation is being optimized during learning, information regarding exploration with "old" features is lost. Here, we propose the first limited memory neural-linear bandit that is resilient to this phenomenon, which we term catastrophic forgetting. We evaluate our method on a variety of real-world data sets, including regression, classification, and sentiment analysis, and observe that our algorithm is resilient to catastrophic forgetting and achieves superior performance.
Perturbed Proximal Descent to Escape Saddle Points for Non-convex and Non-smooth Objective Functions
Authors: Zhishen Huang, Stephen Becker
Date: 2019-01-24 00:00:00

We consider the problem of finding local minimizers in non-convex and
non-smooth optimization. Under the assumption of strict saddle points, positive
results have been derived for first-order methods. We present the first known
results for the non-smooth case, which requires different analysis and a
different algorithm.
Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning
Authors: Sanjeevan Ahilan, Peter Dayan
Date: 2019-01-24 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We investigate how reinforcement learning agents can learn to cooperate.
Drawing inspiration from human societies, in which successful coordination of
many individuals is often facilitated by hierarchical organisation, we
introduce Feudal Multi-agent Hierarchies (FMH). In this framework, a 'manager'
agent, which is tasked with maximising the environmentally-determined reward
function, learns to communicate subgoals to multiple, simultaneously-operating,
'worker' agents. Workers, which are rewarded for achieving managerial subgoals,
take concurrent actions in the world. We outline the structure of FMH and
demonstrate its potential for decentralised learning and control. We find that,
given an adequate set of subgoals from which to choose, FMH performs, and
particularly scales, substantially better than cooperative approaches that use
a shared reward function.
A Zero-Shot Learning application in Deep Drawing process using Hyper-Process Model
Authors: João Reis, Gil Gonçalves
Date: 2019-01-24 00:00:00
Tasks: Zero-Shot Learning

One of the consequences of passing from mass production to mass customization
paradigm in the nowadays industrialized world is the need to increase
flexibility and responsiveness of manufacturing companies. The high-mix /
low-volume production forces constant accommodations of unknown product
variants, which ultimately leads to high periods of machine calibration. The
difficulty related with machine calibration is that experience is required
together with a set of experiments to meet the final product quality.
Unfortunately, all possible combinations of machine parameters is so high that
is difficult to build empirical knowledge. Due to this fact, normally trial and
error approaches are taken making one-of-a-kind products not viable. Therefore,
a Zero-Shot Learning (ZSL) based approach called hyper-process model (HPM) to
learn the relation among multiple tasks is used as a way to shorten the
calibration phase. Assuming each product variant is a task to solve, first, a
shape analysis on data to learn common modes of deformation between tasks is
made, and secondly, a mapping between these modes and task descriptions is
performed. Ultimately, the present work has two main contributions: 1)
Formulation of an industrial problem into a ZSL setting where new process
models can be generated for process optimization and 2) the definition of a
regression problem in the domain of ZSL. For that purpose, a 2-d deep drawing
simulated process was used based on data collected from the Abaqus simulator,
where a significant number of process models were collected to test the
effectiveness of the approach. The obtained results show that is possible to
learn new tasks without any available data (both labeled and unlabeled) by
leveraging information about already existing tasks, allowing to speed up the
calibration phase and make a quicker integration of new products into
manufacturing systems.
Combinational Q-Learning for Dou Di Zhu
Authors: Yang You, Liangwei Li, Baisong Guo, Weiming Wang, Cewu Lu
Date: 2019-01-24 00:00:00
Tasks: Atari Games, Card Games, Deep Reinforcement Learning, Q-Learning, Reinforcement Learning

Deep reinforcement learning (DRL) has gained a lot of attention in recent
years, and has been proven to be able to play Atari games and Go at or above
human levels. However, those games are assumed to have a small fixed number of
actions and could be trained with a simple CNN network. In this paper, we study
a special class of Asian popular card games called Dou Di Zhu, in which two
adversarial groups of agents must consider numerous card combinations at each
time step, leading to huge number of actions. We propose a novel method to
handle combinatorial actions, which we call combinational Q-learning (CQL). We
employ a two-stage network to reduce action space and also leverage
order-invariant max-pooling operations to extract relationships between
primitive actions. Results show that our method prevails over state-of-the art
methods like naive Q-learning and A3C. We develop an easy-to-use card game
environments and train all agents adversarially from sractch, with only
knowledge of game rules and verify that our agents are comparative to humans.
Our code to reproduce all reported results will be available online.
Bottom-up Broadcast Neural Network For Music Genre Classification
Authors: Caifeng Liu, Lin Feng, Guochao Liu, Huibing Wang, Shenglan Liu
Date: 2019-01-24 00:00:00
Tasks: Classification, Decision Making, General Classification, Genre classification, Music Genre Classification, Music Genre Recognition

Music genre recognition based on visual representation has been successfully
explored over the last years. Recently, there has been increasing interest in
attempting convolutional neural networks (CNNs) to achieve the task. However,
most of existing methods employ the mature CNN structures proposed in image
recognition without any modification, which results in the learning features
that are not adequate for music genre classification. Faced with the challenge
of this issue, we fully exploit the low-level information from spectrograms of
audios and develop a novel CNN architecture in this paper. The proposed CNN
architecture takes the long contextual information into considerations, which
transfers more suitable information for the decision-making layer. Various
experiments on several benchmark datasets, including GTZAN, Ballroom, and
Extended Ballroom, have verified the excellent performances of the proposed
neural network. Codes and model will be available at
"ttps://github.com/CaifengLiu/music-genre-classification".
OWA aggregation of multi-criteria with mixed uncertain fuzzy satisfactions
Authors: Yunjuan Wang, Yong Deng
Date: 2019-01-24 00:00:00
Tasks: Decision Making

We apply the Ordered Weighted Averaging (OWA) operator in multi-criteria
decision-making. To satisfy different kinds of uncertainty, measure based
dominance has been presented to gain the order of different criterion. However,
this idea has not been applied in fuzzy system until now. In this paper, we
focus on the situation where the linguistic satisfactions are fuzzy measures
instead of the exact values. We review the concept of OWA operator and discuss
the order mechanism of fuzzy number. Then we combine with measure-based
dominance to give an overall score of each alternatives. An example is
illustrated to show the whole procedure.