pwc-archive/papers-with-abstracts huggingface dataset:
Using Natural Language for Reward Shaping in Reinforcement Learning
Authors: Prasoon Goyal, Scott Niekum, Raymond J. Mooney
Date: 2019-03-05 00:00:00
Tasks: Atari Games, Montezuma's Revenge, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recent reinforcement learning (RL) approaches have shown strong performance in complex domains such as Atari games, but are often highly sample inefficient. A common approach to reduce interaction time with the environment is to use reward shaping, which involves carefully designing reward functions that provide the agent intermediate rewards for progress towards the goal. However, designing appropriate shaping rewards is known to be difficult as well as time-consuming. In this work, we address this problem by using natural language instructions to perform reward shaping. We propose the LanguagE-Action Reward Network (LEARN), a framework that maps free-form natural language instructions to intermediate rewards based on actions taken by the agent. These intermediate language-based rewards can seamlessly be integrated into any standard reinforcement learning algorithm. We experiment with Montezuma's Revenge from the Atari Learning Environment, a popular benchmark in RL. Our experiments on a diverse set of 15 tasks demonstrate that, for the same number of interactions with the environment, language-based rewards lead to successful completion of the task 60% more often on average, compared to learning without language.
Widely Linear Complex-valued Autoencoder: Dealing with Noncircularity in Generative-Discriminative Models
Authors: Zeyang Yu, Shengxi Li, Danilo Mandic
Date: 2019-03-05 00:00:00

We propose a new structure for the complex-valued autoencoder by introducing
additional degrees of freedom into its design through a widely linear (WL)
transform. The corresponding widely linear backpropagation algorithm is also
developed using the $\mathbb{CR}$ calculus, to unify the gradient calculation
of the cost function and the underlying WL model. More specifically, all the
existing complex-valued autoencoders employ the strictly linear transform,
which is optimal only when the complex-valued outputs of each network layer are
independent of the conjugate of the inputs. In addition, the widely linear
model which underpins our work allows us to consider all the second-order
statistics of inputs. This provides more freedom in the design and enhanced
optimization opportunities, as compared to the state-of-the-art. Furthermore,
we show that the most widely adopted cost function, i.e., the mean squared
error, is not best suited for the complex domain, as it is a real quantity with
a single degree of freedom, while both the phase and the amplitude information
need to be optimized. To resolve this issue, we design a new cost function,
which is capable of controlling the balance between the phase and the amplitude
contribution to the solution. The experimental results verify the superior
performance of the proposed autoencoder together with the new cost function,
especially for the imaging scenarios where the phase preserves extensive
information on edges and shapes.
PROPS: Probabilistic personalization of black-box sequence models
Authors: Michael Thomas Wojnowicz, Xuan Zhao
Date: 2019-03-05 00:00:00
Tasks: Articles, Language Modeling, Language Modelling, Transfer Learning

We present PROPS, a lightweight transfer learning mechanism for sequential
data. PROPS learns probabilistic perturbations around the predictions of one or
more arbitrarily complex, pre-trained black box models (such as recurrent
neural networks). The technique pins the black-box prediction functions to
"source nodes" of a hidden Markov model (HMM), and uses the remaining nodes as
"perturbation nodes" for learning customized perturbations around those
predictions. In this paper, we describe the PROPS model, provide an algorithm
for online learning of its parameters, and demonstrate the consistency of this
estimation. We also explore the utility of PROPS in the context of personalized
language modeling. In particular, we construct a baseline language model by
training a LSTM on the entire Wikipedia corpus of 2.5 million articles (around
6.6 billion words), and then use PROPS to provide lightweight customization
into a personalized language model of President Donald J. Trump's tweeting. We
achieved good customization after only 2,000 additional words, and find that
the PROPS model, being fully probabilistic, provides insight into when
President Trump's speech departs from generic patterns in the Wikipedia corpus.
Python code (for both the PROPS training algorithm as well as experiment
reproducibility) is available at
https://github.com/cylance/perturbed-sequence-model.
Statistically-informed deep learning for gravitational wave parameter estimation
Authors: Hongyu Shen, E. A. Huerta, Eamonn O'Shea, Prayush Kumar, Zhizhen Zhao
Date: 2019-03-05 00:00:00
Tasks: Contrastive Learning, Deep Learning, GPU, parameter estimation

We introduce deep learning models to estimate the masses of the binary components of black hole mergers, $(m_1,m_2)$, and three astrophysical properties of the post-merger compact remnant, namely, the final spin, $a_f$, and the frequency and damping time of the ringdown oscillations of the fundamental $\ell=m=2$ bar mode, $(\omega_R, \omega_I)$. Our neural networks combine a modified $\texttt{WaveNet}$ architecture with contrastive learning and normalizing flow. We validate these models against a Gaussian conjugate prior family whose posterior distribution is described by a closed analytical expression. Upon confirming that our models produce statistically consistent results, we used them to estimate the astrophysical parameters $(m_1,m_2, a_f, \omega_R, \omega_I)$ of five binary black holes: $\texttt{GW150914}, \texttt{GW170104}, \texttt{GW170814}, \texttt{GW190521}$ and $\texttt{GW190630}$. We use $\texttt{PyCBC Inference}$ to directly compare traditional Bayesian methodologies for parameter estimation with our deep-learning-based posterior distributions. Our results show that our neural network models predict posterior distributions that encode physical correlations, and that our data-driven median results and 90$\%$ confidence intervals are similar to those produced with gravitational wave Bayesian analyses. This methodology requires a single V100 $\texttt{NVIDIA}$ GPU to produce median values and posterior distributions within two milliseconds for each event. This neural network, and a tutorial for its use, are available at the $\texttt{Data and Learning Hub for Science}$.
Implicit Regularization in Over-parameterized Neural Networks
Authors: Masayoshi Kubo, Ryotaro Banno, Hidetaka Manabe, Masataka Minoji
Date: 2019-03-05 00:00:00

Over-parameterized neural networks generalize well in practice without any
explicit regularization. Although it has not been proven yet, empirical
evidence suggests that implicit regularization plays a crucial role in deep
learning and prevents the network from overfitting. In this work, we introduce
the gradient gap deviation and the gradient deflection as statistical measures
corresponding to the network curvature and the Hessian matrix to analyze
variations of network derivatives with respect to input parameters, and
investigate how implicit regularization works in ReLU neural networks from both
theoretical and empirical perspectives. Our result reveals that the network
output between each pair of input samples is properly controlled by random
initialization and stochastic gradient descent to keep interpolating between
samples almost straight, which results in low complexity of over-parameterized
neural networks.
A Grounded Interaction Protocol for Explainable Artificial Intelligence
Authors: Prashan Madumal, Tim Miller, Liz Sonenberg, Frank Vetere
Date: 2019-03-05 00:00:00
Tasks: Explainable artificial intelligence, Explainable Artificial Intelligence (XAI)

Explainable Artificial Intelligence (XAI) systems need to include an
explanation model to communicate the internal decisions, behaviours and actions
to the interacting humans. Successful explanation involves both cognitive and
social processes. In this paper we focus on the challenge of meaningful
interaction between an explainer and an explainee and investigate the
structural aspects of an interactive explanation to propose an interaction
protocol. We follow a bottom-up approach to derive the model by analysing
transcripts of different explanation dialogue types with 398 explanation
dialogues. We use grounded theory to code and identify key components of an
explanation dialogue. We formalize the model using the agent dialogue framework
(ADF) as a new dialogue type and then evaluate it in a human-agent interaction
study with 101 dialogues from 14 participants. Our results show that the
proposed model can closely follow the explanation dialogues of human-agent
conversations.
Convergence of gradient descent-ascent analyzed as a Newtonian dynamical system with dissipation
Authors: H. Sebastian Seung
Date: 2019-03-05 00:00:00

A dynamical system is defined in terms of the gradient of a payoff function.
Dynamical variables are of two types, ascent and descent. The ascent variables
move in the direction of the gradient, while the descent variables move in the
opposite direction. Dynamical systems of this form or very similar forms have
been studied in diverse fields such as game theory, optimization, neural
networks, and population biology. Gradient descent-ascent is approximated as a
Newtonian dynamical system that conserves total energy, defined as the sum of
the kinetic energy and a potential energy that is proportional to the payoff
function. The error of the approximation is a residual force that violates
energy conservation. If the residual force is purely dissipative, then the
energy serves as a Lyapunov function, and convergence of bounded trajectories
to steady states is guaranteed. A previous convergence theorem due to Kose and
Uzawa required the payoff function to be convex in the descent variables, and
concave in the ascent variables. Here the assumption is relaxed, so that the
payoff function need only be globally `less convex' or `more concave' in the
ascent variables than in the descent variables. Such relative convexity
conditions allow the existence of multiple steady states, unlike the
convex-concave assumption. When combined with sufficient conditions that imply
the existence of a minimax equilibrium, boundedness of trajectories is also
assured.
Automatic Handgun Detection in X-ray Images using Bag of Words Model with Selective Search
Authors: David Castro Piñol, Enrique Juan Marañón Reyes
Date: 2019-03-04 00:00:00

Baggage inspection systems using X-ray screening are crucial for security.
Only 90% of threat objects are recognized from the X-ray system based in human
inspection. Manual detection requires high concentration due to the images
complexity and the challenges objects points of view. An algorithm based on Bag
of Visual Word (BoVW) with Selective Search is proposed in this paper for
handguns detection in single energy X-ray images from the public GDXray
database. This approach is an adaptation of BoVW for X-ray baggage images
context. In order to evaluate the proposed method the algorithm effectiveness
recognition was tested on all bounding boxes returned by selective search
algorithm in 200 images. The most relevant result is the precision and true
positive rate (PPV = 80%, TPR= 92%). This approach achieves good performance
for handgun recognition. In addition, it is the first time the Selective Search
localization algorithm was tested in baggage X-ray images and showed
possibilities with Bag of Visual Words.
Optimizing Software Effort Estimation Models Using Firefly Algorithm
Authors: Nazeeh Ghatasheh, Hossam Faris, Ibrahim Aljarah, Rizik M. H. Al-Sayyed
Date: 2019-01-08 00:00:00
Tasks: Metaheuristic Optimization

Software development effort estimation is considered a fundamental task for
software development life cycle as well as for managing project cost, time and
quality. Therefore, accurate estimation is a substantial factor in projects
success and reducing the risks. In recent years, software effort estimation has
received a considerable amount of attention from researchers and became a
challenge for software industry. In the last two decades, many researchers and
practitioners proposed statistical and machine learning-based models for
software effort estimation. In this work, Firefly Algorithm is proposed as a
metaheuristic optimization method for optimizing the parameters of three
COCOMO-based models. These models include the basic COCOMO model and other two
models proposed in the literature as extensions of the basic COCOMO model. The
developed estimation models are evaluated using different evaluation metrics.
Experimental results show high accuracy and significant error minimization of
Firefly Algorithm over other metaheuristic optimization algorithms including
Genetic Algorithms and Particle Swarm Optimization.
Quantifying error contributions of computational steps, algorithms and hyperparameter choices in image classification pipelines
Authors: Aritra Chowdhury, Malik Magdin-Ismail, Bulent Yener
Date: 2019-02-25 00:00:00
Tasks: Bayesian Optimization, BIG-bench Machine Learning, General Classification, image-classification, Image Classification

Data science relies on pipelines that are organized in the form of
interdependent computational steps. Each step consists of various candidate
algorithms that maybe used for performing a particular function. Each algorithm
consists of several hyperparameters. Algorithms and hyperparameters must be
optimized as a whole to produce the best performance. Typical machine learning
pipelines typically consist of complex algorithms in each of the steps. Not
only is the selection process combinatorial, but it is also important to
interpret and understand the pipelines. We propose a method to quantify the
importance of different layers in the pipeline, by computing an error
contribution relative to an agnostic choice of algorithms in that layer. We
demonstrate our methodology on image classification pipelines. The agnostic
methodology quantifies the error contributions from the computational steps,
algorithms and hyperparameters in the image classification pipeline. We show
that algorithm selection and hyper-parameter optimization methods can be used
to quantify the error contribution and that random search is able to quantify
the contribution more accurately than Bayesian optimization. This methodology
can be used by domain experts to understand machine learning and data analysis
pipelines in terms of their individual components, which can help in
prioritizing different components of the pipeline.
Exploring Deep Spiking Neural Networks for Automated Driving Applications
Authors: Sambit Mohapatra, Heinrich Gotzig, Senthil Yogamani, Stefan Milz, Raoul Zollner
Date: 2019-01-11 00:00:00
Tasks: Depth Estimation, Moving Object Detection, object-detection, Object Detection, Semantic Segmentation, Visual Odometry

Neural networks have become the standard model for various computer vision
tasks in automated driving including semantic segmentation, moving object
detection, depth estimation, visual odometry, etc. The main flavors of neural
networks which are used commonly are convolutional (CNN) and recurrent (RNN).
In spite of rapid progress in embedded processors, power consumption and cost
is still a bottleneck. Spiking Neural Networks (SNNs) are gradually progressing
to achieve low-power event-driven hardware architecture which has a potential
for high efficiency. In this paper, we explore the role of deep spiking neural
networks (SNN) for automated driving applications. We provide an overview of
progress on SNN and argue how it can be a good fit for automated driving
applications.
PatchNet: A Tool for Deep Patch Classification
Authors: Thong Hoang, Julia Lawall, Richard J. Oentaryo, Yuan Tian, David Lo
Date: 2019-02-16 00:00:00
Tasks: Classification, Deep Learning, General Classification

This work proposes PatchNet, an automated tool based on hierarchical deep
learning for classifying patches by extracting features from commit messages
and code changes. PatchNet contains a deep hierarchical structure that mirrors
the hierarchical and sequential structure of a code change, differentiating it
from the existing deep learning models on source code. PatchNet provides
several options allowing users to select parameters for the training process.
The tool has been validated in the context of automatic identification of
stable-relevant patches in the Linux kernel and is potentially applicable to
automate other software engineering tasks that can be formulated as patch
classification problems. A video demonstrating PatchNet is available at
https://goo.gl/CZjG6X. The PatchNet implementation is available at
https://github.com/hvdthong/PatchNetTool.
A GA-based feature selection of the EEG signals by classification evaluation: Application in BCI systems
Authors: Samira Vafay Eslahi, Nader Jafarnia Dabanloo, Keivan Maghooli
Date: 2019-01-16 00:00:00
Tasks: Brain Computer Interface, EEG, Electroencephalogram (EEG), feature selection, General Classification

In electroencephalogram (EEG) signal processing, finding the appropriate
information from a dataset has been a big challenge for successful signal
classification. The feature selection methods make it possible to solve this
problem; however, the method selection is still under investigation to find out
which feature can perform the best to extract the most proper features of the
signal to improve the classification performance. In this study, we use the
genetic algorithm (GA), a heuristic searching algorithm, to find the optimum
combination of the feature extraction methods and the classifiers, in the
brain-computer interface (BCI) applications. A BCI system can be practical if
and only if it performs with high accuracy and high speed alongside each other.
In the proposed method, GA performs as a searching engine to find the best
combination of the features and classifications. The features used here are
Katz, Higuchi, Petrosian, Sevcik, and box-counting dimension (BCD) feature
extraction methods. These features are applied to the wavelet subbands and are
classified with four classifiers such as adaptive neuro-fuzzy inference system
(ANFIS), fuzzy k-nearest neighbors (FKNN), support vector machine (SVM) and
linear discriminant analysis (LDA). Due to the huge number of features, the GA
optimization is used to find the features with the optimum fitness value (FV).
Results reveal that Katz fractal feature estimation method with LDA
classification has the best FV. Consequently, due to the low computation time
of the first Daubechies wavelet transformation in comparison to the original
signal, the final selected methods contain the fractal features of the first
coefficient of the detail subbands.
DA-LSTM: A Long Short-Term Memory with Depth Adaptive to Non-uniform Information Flow in Sequential Data
Authors: Yifeng Zhang, Ka-Ho Chow, S. -H. Gary Chan
Date: 2019-01-18 00:00:00

Much sequential data exhibits highly non-uniform information distribution.
This cannot be correctly modeled by traditional Long Short-Term Memory (LSTM).
To address that, recent works have extended LSTM by adding more activations
between adjacent inputs. However, the approaches often use a fixed depth, which
is at the step of the most information content. This one-size-fits-all
worst-case approach is not satisfactory, because when little information is
distributed to some steps, shallow structures can achieve faster convergence
and consume less computation resource. In this paper, we develop a
Depth-Adaptive Long Short-Term Memory (DA-LSTM) architecture, which can
dynamically adjust the structure depending on information distribution without
prior knowledge. Experimental results on real-world datasets show that DA-LSTM
costs much less computation resource and substantially reduce convergence time
by $41.78\%$ and $46.01 \%$, compared with Stacked LSTM and Deep Transition
LSTM, respectively.
Direct Feedback Alignment with Sparse Connections for Local Learning
Authors: Brian Crafton, Abhinav Parihar, Evan Gebhardt, Arijit Raychowdhury
Date: 2019-01-30 00:00:00

Recent advances in deep neural networks (DNNs) owe their success to training algorithms that use backpropagation and gradient-descent. Backpropagation, while highly effective on von Neumann architectures, becomes inefficient when scaling to large networks. Commonly referred to as the weight transport problem, each neuron's dependence on the weights and errors located deeper in the network require exhaustive data movement which presents a key problem in enhancing the performance and energy-efficiency of machine-learning hardware. In this work, we propose a bio-plausible alternative to backpropagation drawing from advances in feedback alignment algorithms in which the error computation at a single synapse reduces to the product of three scalar values. Using a sparse feedback matrix, we show that a neuron needs only a fraction of the information previously used by the feedback alignment algorithms. Consequently, memory and compute can be partitioned and distributed whichever way produces the most efficient forward pass so long as a single error can be delivered to each neuron. Our results show orders of magnitude improvement in data movement and $2\times$ improvement in multiply-and-accumulate operations over backpropagation. Like previous work, we observe that any variant of feedback alignment suffers significant losses in classification accuracy on deep convolutional neural networks. By transferring trained convolutional layers and training the fully connected layers using direct feedback alignment, we demonstrate that direct feedback alignment can obtain results competitive with backpropagation. Furthermore, we observe that using an extremely sparse feedback matrix, rather than a dense one, results in a small accuracy drop while yielding hardware advantages. All the code and results are available under https://github.com/bcrafton/ssdfa.
Hypergraph and protein function prediction with gene expression data
Authors: Loc Tran
Date: 2012-12-03 00:00:00
Tasks: Prediction, Protein Function Prediction

Most network-based protein (or gene) function prediction methods are based on
the assumption that the labels of two adjacent proteins in the network are
likely to be the same. However, assuming the pairwise relationship between
proteins or genes is not complete, the information a group of genes that show
very similar patterns of expression and tend to have similar functions (i.e.
the functional modules) is missed. The natural way overcoming the information
loss of the above assumption is to represent the gene expression data as the
hypergraph. Thus, in this paper, the three un-normalized, random walk, and
symmetric normalized hypergraph Laplacian based semi-supervised learning
methods applied to hypergraph constructed from the gene expression data in
order to predict the functions of yeast proteins are introduced. Experiment
results show that the average accuracy performance measures of these three
hypergraph Laplacian based semi-supervised learning methods are the same.
However, their average accuracy performance measures of these three methods are
much greater than the average accuracy performance measures of un-normalized
graph Laplacian based semi-supervised learning method (i.e. the baseline method
of this paper) applied to gene co-expression network created from the gene
expression data.
RRAM based neuromorphic algorithms
Authors: Roshan Gopalakrishnan
Date: 2019-01-12 00:00:00

This submission is a report on RRAM based neuromorphic algorithms. This
report basically gives an overview of the algorithms implemented on
neuromorphic hardware with crossbar array of RRAM synapses. This report mainly
talks about the work on deep neural network to spiking neural network
conversion and its significance.