pwc-archive/papers-with-abstracts huggingface dataset:
Physical Symmetries Embedded in Neural Networks
Authors: M. Mattheakis, P. Protopapas, D. Sondak, M. Di Giovanni, E. Kaxiras
Date: 2019-04-18 00:00:00

Neural networks are a central technique in machine learning. Recent years have seen a wave of interest in applying neural networks to physical systems for which the governing dynamics are known and expressed through differential equations. Two fundamental challenges facing the development of neural networks in physics applications is their lack of interpretability and their physics-agnostic design. The focus of the present work is to embed physical constraints into the structure of the neural network to address the second fundamental challenge. By constraining tunable parameters (such as weights and biases) and adding special layers to the network, the desired constraints are guaranteed to be satisfied without the need for explicit regularization terms. This is demonstrated on upervised and unsupervised networks for two basic symmetries: even/odd symmetry of a function and energy conservation. In the supervised case, the network with embedded constraints is shown to perform well on regression problems while simultaneously obeying the desired constraints whereas a traditional network fits the data but violates the underlying constraints. Finally, a new unsupervised neural network is proposed that guarantees energy conservation through an embedded symplectic structure. The symplectic neural network is used to solve a system of energy-conserving differential equations and out-performs an unsupervised, non-symplectic neural network.
End-to-End Environmental Sound Classification using a 1D Convolutional Neural Network
Authors: Sajjad Abdoli, Patrick Cardinal, Alessandro Lameiras Koerich
Date: 2019-04-18 00:00:00
Tasks: Environmental Sound Classification, General Classification, Sound Classification

In this paper, we present an end-to-end approach for environmental sound
classification based on a 1D Convolution Neural Network (CNN) that learns a
representation directly from the audio signal. Several convolutional layers are
used to capture the signal's fine time structure and learn diverse filters that
are relevant to the classification task. The proposed approach can deal with
audio signals of any length as it splits the signal into overlapped frames
using a sliding window. Different architectures considering several input sizes
are evaluated, including the initialization of the first convolutional layer
with a Gammatone filterbank that models the human auditory filter response in
the cochlea. The performance of the proposed end-to-end approach in classifying
environmental sounds was assessed on the UrbanSound8k dataset and the
experimental results have shown that it achieves 89% of mean accuracy.
Therefore, the propose approach outperforms most of the state-of-the-art
approaches that use handcrafted features or 2D representations as input.
Furthermore, the proposed approach has a small number of parameters compared to
other architectures found in the literature, which reduces the amount of data
required for training.
TTS Skins: Speaker Conversion via ASR
Authors: Adam Polyak, Lior Wolf, Yaniv Taigman
Date: 2019-04-18 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, speech-recognition, Speech Recognition

We present a fully convolutional wav-to-wav network for converting between speakers' voices, without relying on text. Our network is based on an encoder-decoder architecture, where the encoder is pre-trained for the task of Automatic Speech Recognition, and a multi-speaker waveform decoder is trained to reconstruct the original signal in an autoregressive manner. We train the network on narrated audiobooks, and demonstrate multi-voice TTS in those voices, by converting the voice of a TTS robot.
Exploring the Limitations of Behavior Cloning for Autonomous Driving
Authors: Felipe Codevilla, Eder Santana, Antonio M. López, Adrien Gaidon
Date: 2019-04-18 00:00:00
Proceeding: ICCV 2019 10
Tasks: Autonomous Driving, Imitation Learning

Driving requires reacting to a wide variety of complex environment conditions
and agent behaviors. Explicitly modeling each possible scenario is unrealistic.
In contrast, imitation learning can, in theory, leverage data from large fleets
of human-driven cars. Behavior cloning in particular has been successfully used
to learn simple visuomotor policies end-to-end, but scaling to the full
spectrum of driving behaviors remains an unsolved problem. In this paper, we
propose a new benchmark to experimentally investigate the scalability and
limitations of behavior cloning. We show that behavior cloning leads to
state-of-the-art results, including in unseen environments, executing complex
lateral and longitudinal maneuvers without these reactions being explicitly
programmed. However, we confirm well-known limitations (due to dataset bias and
overfitting), new generalization issues (due to dynamic objects and the lack of
a causal model), and training instability requiring further research before
behavior cloning can graduate to real-world driving. The code of the studied
behavior cloning approaches can be found at
https://github.com/felipecode/coiltraine .
Intentional Computational Level Design
Authors: Ahmed Khalifa, Michael Cerny Green, Gabriella Barros, Julian Togelius
Date: 2019-04-18 00:00:00
Tasks: Diversity, Evolutionary Algorithms

The procedural generation of levels and content in video games is a
challenging AI problem. Often such generation relies on an intelligent way of
evaluating the content being generated so that constraints are satisfied and/or
objectives maximized. In this work, we address the problem of creating levels
that are not only playable but also revolve around specific mechanics in the
game. We use constrained evolutionary algorithms and quality-diversity
algorithms to generate small sections of Super Mario Bros levels called scenes,
using three different simulation approaches: Limited Agents, Punishing Model,
and Mechanics Dimensions. All three approaches are able to create scenes that
give opportunity for a player to encounter or use targeted mechanics with
different properties. We conclude by discussing the advantages and
disadvantages of each approach and compare them to each other.
VoteNet: A Deep Learning Label Fusion Method for Multi-Atlas Segmentation
Authors: Zhipeng Ding, Xu Han, Marc Niethammer
Date: 2019-04-18 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Deep learning (DL) approaches are state-of-the-art for many medical image segmentation tasks. They offer a number of advantages: they can be trained for specific tasks, computations are fast at test time, and segmentation quality is typically high. In contrast, previously popular multi-atlas segmentation (MAS) methods are relatively slow (as they rely on costly registrations) and even though sophisticated label fusion strategies have been proposed, DL approaches generally outperform MAS. In this work, we propose a DL-based label fusion strategy (VoteNet) which locally selects a set of reliable atlases whose labels are then fused via plurality voting. Experiments on 3D brain MRI data show that by selecting a good initial atlas set MAS with VoteNet significantly outperforms a number of other label fusion strategies as well as a direct DL segmentation approach. We also provide an experimental analysis of the upper performance bound achievable by our method. While unlikely achievable in practice, this bound suggests room for further performance improvements. Lastly, to address the runtime disadvantage of standard MAS, all our results make use of a fast DL registration approach.
Constrained Restless Bandits for Dynamic Scheduling in Cyber-Physical Systems
Authors: Kesav Kaza, Rahul Meshram, Varun Mehta, S. N. Merchant
Date: 2019-04-18 00:00:00
Tasks: Decision Making, Decision Making Under Uncertainty, Multi-Armed Bandits, Scheduling

This paper studies a class of constrained restless multi-armed bandits (CRMAB). The constraints are in the form of time varying set of actions (set of available arms). This variation can be either stochastic or semi-deterministic. Given a set of arms, a fixed number of them can be chosen to be played in each decision interval. The play of each arm yields a state dependent reward. The current states of arms are partially observable through binary feedback signals from arms that are played. The current availability of arms is fully observable. The objective is to maximize long term cumulative reward. The uncertainty about future availability of arms along with partial state information makes this objective challenging. Applications for CRMAB can be found in resource allocation in cyber-physical systems involving components with time varying availability. First, this optimization problem is analyzed using Whittle's index policy. To this end, a constrained restless single-armed bandit is studied. It is shown to admit a threshold-type optimal policy and is also indexable. An algorithm to compute Whittle's index is presented. An alternate solution method with lower complexity is also presented in the form of an online rollout policy. A detailed discussion on the complexity of both these schemes is also presented, which suggests that online rollout policy with short look ahead is simpler to implement than Whittle's index computation. Further, upper bounds on the value function are derived in order to estimate the degree of sub-optimality of various solutions. The simulation study compares the performance of Whittle's index, online rollout, myopic and modified Whittle's index policies.
RepGN:Object Detection with Relational Proposal Graph Network
Authors: Xingjian Du, Xuan Shi, Risheng Huang
Date: 2019-04-18 00:00:00
Tasks: General Classification, Graph Learning, Object, object-detection, Object Detection, Relation

Region based object detectors achieve the state-of-the-art performance, but
few consider to model the relation of proposals. In this paper, we explore the
idea of modeling the relationships among the proposals for object detection
from the graph learning perspective. Specifically, we present relational
proposal graph network (RepGN) which is defined on object proposals and the
semantic and spatial relation modeled as the edge. By integrating our RepGN
module into object detectors, the relation and context constraints will be
introduced to the feature extraction of regions and bounding boxes regression
and classification. Besides, we propose a novel graph-cut based pooling layer
for hierarchical coarsening of the graph, which empowers the RepGN module to
exploit the inter-regional correlation and scene description in a hierarchical
manner. We perform extensive experiments on COCO object detection dataset and
show promising results.
No Permanent Friends or Enemies: Tracking Relationships between Nations from News
Authors: Xiaochuang Han, Eunsol Choi, Chenhao Tan
Date: 2019-04-18 00:00:00
Proceeding: NAACL 2019 6
Tasks: Articles

Understanding the dynamics of international politics is important yet
challenging for civilians. In this work, we explore unsupervised neural models
to infer relations between nations from news articles. We extend existing
models by incorporating shallow linguistics information and propose a new
automatic evaluation metric that aligns relationship dynamics with manually
annotated key events. As understanding international relations requires
carefully analyzing complex relationships, we conduct in-person human
evaluations with three groups of participants. Overall, humans prefer the
outputs of our model and give insightful feedback that suggests future
directions for human-centered models. Furthermore, our model reveals
interesting regional differences in news coverage. For instance, with respect
to US-China relations, Singaporean media focus more on "strengthening" and
"purchasing", while US media focus more on "criticizing" and "denouncing".
Understanding Neural Networks via Feature Visualization: A survey
Authors: Anh Nguyen, Jason Yosinski, Jeff Clune
Date: 2019-04-18 00:00:00
Tasks: BIG-bench Machine Learning, Survey

A neuroscience method to understanding the brain is to find and study the
preferred stimuli that highly activate an individual cell or groups of cells.
Recent advances in machine learning enable a family of methods to synthesize
preferred stimuli that cause a neuron in an artificial or biological brain to
fire strongly. Those methods are known as Activation Maximization (AM) or
Feature Visualization via Optimization. In this chapter, we (1) review existing
AM techniques in the literature; (2) discuss a probabilistic interpretation for
AM; and (3) review the applications of AM in debugging and explaining networks.
Language Modeling through Long Term Memory Network
Authors: Anupiya Nugaliyadde, Kok Wai Wong, Ferdous Sohel, Hong Xie
Date: 2019-04-18 00:00:00
Tasks: Language Modeling, Language Modelling

Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), and
Memory Networks which contain memory are popularly used to learn patterns in
sequential data. Sequential data has long sequences that hold relationships.
RNN can handle long sequences but suffers from the vanishing and exploding
gradient problems. While LSTM and other memory networks address this problem,
they are not capable of handling long sequences (50 or more data points long
sequence patterns). Language modelling requiring learning from longer sequences
are affected by the need for more information in memory. This paper introduces
Long Term Memory network (LTM), which can tackle the exploding and vanishing
gradient problems and handles long sequences without forgetting. LTM is
designed to scale data in the memory and gives a higher weight to the input in
the sequence. LTM avoid overfitting by scaling the cell state after achieving
the optimal results. The LTM is tested on Penn treebank dataset, and Text8
dataset and LTM achieves test perplexities of 83 and 82 respectively. 650 LTM
cells achieved a test perplexity of 67 for Penn treebank, and 600 cells
achieved a test perplexity of 77 for Text8. LTM achieves state of the art
results by only using ten hidden LTM cells for both datasets.
Explaining Deep Classification of Time-Series Data with Learned Prototypes
Authors: Alan H. Gee, Diego Garcia-Olano, Joydeep Ghosh, David Paydarfar
Date: 2019-04-18 00:00:00
Tasks: Classification, Decision Making, General Classification, Time Series, Time Series Analysis

The emergence of deep learning networks raises a need for explainable AI so that users and domain experts can be confident applying them to high-risk decisions. In this paper, we leverage data from the latent space induced by deep learning models to learn stereotypical representations or "prototypes" during training to elucidate the algorithmic decision-making process. We study how leveraging prototypes effect classification decisions of two dimensional time-series data in a few different settings: (1) electrocardiogram (ECG) waveforms to detect clinical bradycardia, a slowing of heart rate, in preterm infants, (2) respiration waveforms to detect apnea of prematurity, and (3) audio waveforms to classify spoken digits. We improve upon existing models by optimizing for increased prototype diversity and robustness, visualize how these prototypes in the latent space are used by the model to distinguish classes, and show that prototypes are capable of learning features on two dimensional time-series data to produce explainable insights during classification tasks. We show that the prototypes are capable of learning real-world features - bradycardia in ECG, apnea in respiration, and articulation in speech - as well as features within sub-classes. Our novel work leverages learned prototypical framework on two dimensional time-series data to produce explainable insights during classification tasks.
Continual Learning for Sentence Representations Using Conceptors
Authors: Tianlin Liu, Lyle Ungar, João Sedoc
Date: 2019-04-18 00:00:00
Proceeding: NAACL 2019 6
Tasks: Continual Learning, Semantic Textual Similarity, Sentence

Distributed representations of sentences have become ubiquitous in natural
language processing tasks. In this paper, we consider a continual learning
scenario for sentence representations: Given a sequence of corpora, we aim to
optimize the sentence encoder with respect to the new corpus while maintaining
its accuracy on the old corpora. To address this problem, we propose to
initialize sentence encoders with the help of corpus-independent features, and
then sequentially update sentence encoders using Boolean operations of
conceptor matrices to learn corpus-dependent features. We evaluate our approach
on semantic textual similarity tasks and show that our proposed sentence
encoder can continually learn features from new corpora while retaining its
competence on previously encountered corpora.
Ensemble Convolutional Neural Networks for Mode Inference in Smartphone Travel Survey
Authors: Ali Yazdizadeh, Zachary Patterson, Bilal Farooq
Date: 2019-04-18 00:00:00

We develop ensemble Convolutional Neural Networks (CNNs) to classify the
transportation mode of trip data collected as part of a large-scale smartphone
travel survey in Montreal, Canada. Our proposed ensemble library is composed of
a series of CNN models with different hyper-parameter values and CNN
architectures. In our final model, we combine the output of CNN models using
"average voting", "majority voting" and "optimal weights" methods. Furthermore,
we exploit the ensemble library by deploying a Random Forest model as a
meta-learner. The ensemble method with random forest as meta-learner shows an
accuracy of 91.8% which surpasses the other three ensemble combination methods,
as well as other comparable models reported in the literature. The "majority
voting" and "optimal weights" combination methods result in prediction accuracy
rates around 89%, while "average voting" is able to achieve an accuracy of only
85%.
PLOTS: Procedure Learning from Observations using Subtask Structure
Authors: Tong Mu, Karan Goel, Emma Brunskill
Date: 2019-04-17 00:00:00
Tasks: Procedure Learning

In many cases an intelligent agent may want to learn how to mimic a single
observed demonstrated trajectory. In this work we consider how to perform such
procedural learning from observation, which could help to enable agents to
better use the enormous set of video data on observation sequences. Our
approach exploits the properties of this setting to incrementally build an open
loop action plan that can yield the desired subsequence, and can be used in
both Markov and partially observable Markov domains. In addition, procedures
commonly involve repeated extended temporal action subsequences. Our method
optimistically explores actions to leverage potential repeated structure in the
procedure. In comparing to some state-of-the-art approaches we find that our
explicit procedural learning from observation method is about 100 times faster
than policy-gradient based approaches that learn a stochastic policy and is
faster than model based approaches as well. We also find that performing
optimistic action selection yields substantial speed ups when latent dynamical
structure is present.
FLARe: Forecasting by Learning Anticipated Representations
Authors: Surya Teja Devarakonda, Joie Yeahuay Wu, Yi Ren Fung, Madalina Fiterau
Date: 2019-04-17 00:00:00

Computational models that forecast the progression of Alzheimer's disease at the patient level are extremely useful tools for identifying high risk cohorts for early intervention and treatment planning. The state-of-the-art work in this area proposes models that forecast by using latent representations extracted from the longitudinal data across multiple modalities, including volumetric information extracted from medical scans and demographic info. These models incorporate the time horizon, which is the amount of time between the last recorded visit and the future visit, by directly concatenating a representation of it to the data latent representation. In this paper, we present a model which generates a sequence of latent representations of the patient status across the time horizon, providing more informative modeling of the temporal relationships between the patient's history and future visits. Our proposed model outperforms the baseline in terms of forecasting accuracy and F1 score with the added benefit of robustly handling missing visits.
"Why did you do that?": Explaining black box models with Inductive Synthesis
Authors: Görkem Paçacı, David Johnson, Steve McKeever, Andreas Hamfelt
Date: 2019-04-17 00:00:00
Tasks: Program Synthesis

By their nature, the composition of black box models is opaque. This makes
the ability to generate explanations for the response to stimuli challenging.
The importance of explaining black box models has become increasingly important
given the prevalence of AI and ML systems and the need to build legal and
regulatory frameworks around them. Such explanations can also increase trust in
these uncertain systems. In our paper we present RICE, a method for generating
explanations of the behaviour of black box models by (1) probing a model to
extract model output examples using sensitivity analysis; (2) applying
CNPInduce, a method for inductive logic program synthesis, to generate logic
programs based on critical input-output pairs; and (3) interpreting the target
program as a human-readable explanation. We demonstrate the application of our
method by generating explanations of an artificial neural network trained to
follow simple traffic rules in a hypothetical self-driving car simulation. We
conclude with a discussion on the scalability and usability of our approach and
its potential applications to explanation-critical scenarios.
Class specific or shared? A cascaded dictionary learning framework for image classification
Authors: Yan-Jiang Wang, Shuai Shao, Rui Xu, Werifeng Liu, Bao-Di Liu
Date: 2019-04-17 00:00:00
Tasks: Dictionary Learning, General Classification, image-classification, Image Classification

Dictionary learning methods can be split into: i) class specific dictionary learning ii) class shared dictionary learning. The difference between the two categories is how to use discriminative information. With the first category, samples of different classes are mapped into different subspaces, which leads to some redundancy with the class specific base vectors. While for the second category, the samples in each specific class can not be described accurately. In this paper, we first propose a novel class shared dictionary learning method named label embedded dictionary learning (LEDL). It is the improvement based on LCKSVD, which is easier to find out the optimal solution. Then we propose a novel framework named cascaded dictionary learning framework (CDLF) to combine the specific dictionary learning with shared dictionary learning to describe the feature to boost the performance of classification sufficiently. Extensive experimental results on six benchmark datasets illustrate that our methods are capable of achieving superior performance compared to several state-of-art classification algorithms.
Predict Future Sales using Ensembled Random Forests
Authors: Yuwei Zhang, Xin Wu, Chenyang Gu, Yueqi Xie
Date: 2019-04-17 00:00:00
Tasks: Ensemble Learning, Feature Engineering

This is a method report for the Kaggle data competition 'Predict future
sales'. In this paper, we propose a rather simple approach to future sales
predicting based on feature engineering, Random Forest Regressor and ensemble
learning. Its performance turned out to exceed many of the conventional methods
and get final score 0.88186, representing root mean squared error. As of this
writing, our model ranked 5th on the leaderboard. (till 8.5.2018)