pwc-archive/papers-with-abstracts huggingface dataset:
Minimal Images in Deep Neural Networks: Fragile Object Recognition in Natural Images
Authors: Sanjana Srivastava, Guy Ben-Yosef, Xavier Boix
Date: 2019-02-08 00:00:00
Proceeding: ICLR 2019 5
Tasks: Object Recognition

The human ability to recognize objects is impaired when the object is not
shown in full. "Minimal images" are the smallest regions of an image that
remain recognizable for humans. Ullman et al. 2016 show that a slight
modification of the location and size of the visible region of the minimal
image produces a sharp drop in human recognition accuracy. In this paper, we
demonstrate that such drops in accuracy due to changes of the visible region
are a common phenomenon between humans and existing state-of-the-art deep
neural networks (DNNs), and are much more prominent in DNNs. We found many
cases where DNNs classified one region correctly and the other incorrectly,
though they only differed by one row or column of pixels, and were often bigger
than the average human minimal image size. We show that this phenomenon is
independent from previous works that have reported lack of invariance to minor
modifications in object location in DNNs. Our results thus reveal a new failure
mode of DNNs that also affects humans to a much lesser degree. They expose how
fragile DNN recognition ability is for natural images even without adversarial
patterns being introduced. Bringing the robustness of DNNs in natural images to
the human level remains an open challenge for the community.
Robust Streaming PCA
Authors: Daniel Bienstock, Minchan Jeong, Apurv Shukla, Se-Young Yun
Date: 2019-02-08 00:00:00

We consider streaming principal component analysis when the stochastic data-generating model is subject to perturbations. While existing models assume a fixed covariance, we adopt a robust perspective where the covariance matrix belongs to a temporal uncertainty set. Under this setting, we provide fundamental limits on convergence of any algorithm recovering principal components. We analyze the convergence of the noisy power method and Oja's algorithm, both studied for the stationary data generating model, and argue that the noisy power method is rate-optimal in our setting. Finally, we demonstrate the validity of our analysis through numerical experiments on synthetic and real-world dataset.
Tensor Variable Elimination for Plated Factor Graphs
Authors: Fritz Obermeyer, Eli Bingham, Martin Jankowiak, Justin Chiu, Neeraj Pradhan, Alexander Rush, Noah Goodman
Date: 2019-02-08 00:00:00
Tasks: Music Modeling, Probabilistic Programming, Sentiment Analysis, tensor algebra

A wide class of machine learning algorithms can be reduced to variable elimination on factor graphs. While factor graphs provide a unifying notation for these algorithms, they do not provide a compact way to express repeated structure when compared to plate diagrams for directed graphical models. To exploit efficient tensor algebra in graphs with plates of variables, we generalize undirected factor graphs to plated factor graphs and variable elimination to a tensor variable elimination algorithm that operates directly on plated factor graphs. Moreover, we generalize complexity bounds based on treewidth and characterize the class of plated factor graphs for which inference is tractable. As an application, we integrate tensor variable elimination into the Pyro probabilistic programming language to enable exact inference in discrete latent variable models with repeated structure. We validate our methods with experiments on both directed and undirected graphical models, including applications to polyphonic music modeling, animal movement modeling, and latent sentiment analysis.
Software-Defined FPGA Accelerator Design for Mobile Deep Learning Applications
Authors: Panagiotis G. Mousouliotis, Loukas P. Petrou
Date: 2019-02-08 00:00:00
Tasks: Deep Learning, High-Level Synthesis, object-detection, Object Detection, Object Recognition, Scene Understanding, Semantic Segmentation

Recently, the field of deep learning has received great attention by the
scientific community and it is used to provide improved solutions to many
computer vision problems. Convolutional neural networks (CNNs) have been
successfully used to attack problems such as object recognition, object
detection, semantic segmentation, and scene understanding. The rapid
development of deep learning goes hand by hand with the adaptation of GPUs for
accelerating its processes, such as network training and inference. Even though
FPGA design exists long before the use of GPUs for accelerating computations
and despite the fact that high-level synthesis (HLS) tools are getting more
attractive, the adaptation of FPGAs for deep learning research and application
development is poor due to the requirement of hardware design related
expertise. This work presents a workflow for deep learning mobile application
acceleration on small low-cost low-power FPGA devices using HLS tools. This
workflow eases the design of an improved version of the SqueezeJet accelerator
used for the speedup of mobile-friendly low-parameter ImageNet class CNNs, such
as the SqueezeNet v1.1 and the ZynqNet. Additionally, the workflow includes the
development of an HLS-driven analytical model which is used for performance
estimation of the accelerator. This model can be also used to direct the design
process and lead to future design improvements and optimizations.
Speaker diarisation using 2D self-attentive combination of embeddings
Authors: Guangzhi Sun, Chao Zhang, Phil Woodland
Date: 2019-02-08 00:00:00
Tasks: Diversity

Speaker diarisation systems often cluster audio segments using speaker
embeddings such as i-vectors and d-vectors. Since different types of embeddings
are often complementary, this paper proposes a generic framework to improve
performance by combining them into a single embedding, referred to as a
c-vector. This combination uses a 2-dimensional (2D) self-attentive structure,
which extends the standard self-attentive layer by averaging not only across
time but also across different types of embeddings. Two types of 2D
self-attentive structure in this paper are the simultaneous combination and the
consecutive combination, adopting a single and multiple self-attentive layers
respectively. The penalty term in the original self-attentive layer which is
jointly minimised with the objective function to encourage diversity of
annotation vectors is also modified to obtain not only different local peaks
but also the overall trends in the multiple annotation vectors. Experiments on
the AMI meeting corpus show that our modified penalty term improves the d-
vector relative speaker error rate (SER) by 6% and 21% for d-vector systems,
and a 10% further relative SER reduction can be obtained using the c-vector
from our best 2D self-attentive structure.
Controlled Forgetting: Targeted Stimulation and Dopaminergic Plasticity Modulation for Unsupervised Lifelong Learning in Spiking Neural Networks
Authors: Jason M. Allred, Kaushik Roy
Date: 2019-02-08 00:00:00
Tasks: Lifelong learning

Stochastic gradient descent requires that training samples be drawn from a uniformly random distribution of the data. For a deployed system that must learn online from an uncontrolled and unknown environment, the ordering of input samples often fails to meet this criterion, making lifelong learning a difficult challenge. We exploit the locality of the unsupervised Spike Timing Dependent Plasticity (STDP) learning rule to target local representations in a Spiking Neural Network (SNN) to adapt to novel information while protecting essential information in the remainder of the SNN from catastrophic forgetting. In our Controlled Forgetting Networks (CFNs), novel information triggers stimulated firing and heterogeneously modulated plasticity, inspired by biological dopamine signals, to cause rapid and isolated adaptation in the synapses of neurons associated with outlier information. This targeting controls the forgetting process in a way that reduces the degradation of accuracy for older tasks while learning new tasks. Our experimental results on the MNIST dataset validate the capability of CFNs to learn successfully over time from an unknown, changing environment, achieving 95.36% accuracy, which we believe is the best unsupervised accuracy ever achieved by a fixed-size, single-layer SNN on a completely disjoint MNIST dataset.
Partner Selection for the Emergence of Cooperation in Multi-Agent Systems Using Reinforcement Learning
Authors: Nicolas Anastassacos, Stephen Hailes, Mirco Musolesi
Date: 2019-02-08 00:00:00
Tasks: Multi-agent Reinforcement Learning, Reinforcement Learning

Social dilemmas have been widely studied to explain how humans are able to cooperate in society. Considerable effort has been invested in designing artificial agents for social dilemmas that incorporate explicit agent motivations that are chosen to favor coordinated or cooperative responses. The prevalence of this general approach points towards the importance of achieving an understanding of both an agent's internal design and external environment dynamics that facilitate cooperative behavior. In this paper, we investigate how partner selection can promote cooperative behavior between agents who are trained to maximize a purely selfish objective function. Our experiments reveal that agents trained with this dynamic learn a strategy that retaliates against defectors while promoting cooperation with other agents resulting in a prosocial society.
Scalable Nonparametric Sampling from Multimodal Posteriors with the Posterior Bootstrap
Authors: Edwin Fong, Simon Lyddon, Chris Holmes
Date: 2019-02-08 00:00:00
Tasks: Bayesian Inference, regression

Increasingly complex datasets pose a number of challenges for Bayesian inference. Conventional posterior sampling based on Markov chain Monte Carlo can be too computationally intensive, is serial in nature and mixes poorly between posterior modes. Further, all models are misspecified, which brings into question the validity of the conventional Bayesian update. We present a scalable Bayesian nonparametric learning routine that enables posterior sampling through the optimization of suitably randomized objective functions. A Dirichlet process prior on the unknown data distribution accounts for model misspecification, and admits an embarrassingly parallel posterior bootstrap algorithm that generates independent and exact samples from the nonparametric posterior distribution. Our method is particularly adept at sampling from multimodal posterior distributions via a random restart mechanism. We demonstrate our method on Gaussian mixture model and sparse logistic regression examples.
BINet: Multi-perspective Business Process Anomaly Classification
Authors: Timo Nolle, Stefan Luettgen, Alexander Seeliger, Max Mühlhäuser
Date: 2019-02-08 00:00:00
Tasks: Anomaly Classification, Anomaly Detection, Attribute, Classification, General Classification

In this paper, we introduce BINet, a neural network architecture for
real-time multi-perspective anomaly detection in business process event logs.
BINet is designed to handle both the control flow and the data perspective of a
business process. Additionally, we propose a set of heuristics for setting the
threshold of an anomaly detection algorithm automatically. We demonstrate that
BINet can be used to detect anomalies in event logs not only on a case level
but also on event attribute level. Finally, we demonstrate that a simple set of
rules can be used to utilize the output of BINet for anomaly classification. We
compare BINet to eight other state-of-the-art anomaly detection algorithms and
evaluate their performance on an elaborate data corpus of 29 synthetic and 15
real-life event logs. BINet outperforms all other methods both on the synthetic
as well as on the real-life datasets.
Discretization based Solutions for Secure Machine Learning against Adversarial Attacks
Authors: Priyadarshini Panda, Indranil Chakraborty, Kaushik Roy
Date: 2019-02-08 00:00:00
Tasks: Adversarial Robustness, BIG-bench Machine Learning

Adversarial examples are perturbed inputs that are designed (from a deep
learning network's (DLN) parameter gradients) to mislead the DLN during test
time. Intuitively, constraining the dimensionality of inputs or parameters of a
network reduces the 'space' in which adversarial examples exist. Guided by this
intuition, we demonstrate that discretization greatly improves the robustness
of DLNs against adversarial attacks. Specifically, discretizing the input space
(or allowed pixel levels from 256 values or 8-bit to 4 values or 2-bit)
extensively improves the adversarial robustness of DLNs for a substantial range
of perturbations for minimal loss in test accuracy. Furthermore, we find that
Binary Neural Networks (BNNs) and related variants are intrinsically more
robust than their full precision counterparts in adversarial scenarios.
Combining input discretization with BNNs furthers the robustness even waiving
the need for adversarial training for certain magnitude of perturbation values.
We evaluate the effect of discretization on MNIST, CIFAR10, CIFAR100 and
Imagenet datasets. Across all datasets, we observe maximal adversarial
resistance with 2-bit input discretization that incurs an adversarial accuracy
loss of just ~1-2% as compared to clean test accuracy.
Distributional reinforcement learning with linear function approximation
Authors: Marc G. Bellemare, Nicolas Le Roux, Pablo Samuel Castro, Subhodeep Moitra
Date: 2019-02-08 00:00:00
Tasks: Distributional Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Despite many algorithmic advances, our theoretical understanding of practical
distributional reinforcement learning methods remains limited. One exception is
Rowland et al. (2018)'s analysis of the C51 algorithm in terms of the Cram\'er
distance, but their results only apply to the tabular setting and ignore C51's
use of a softmax to produce normalized distributions. In this paper we adapt
the Cram\'er distance to deal with arbitrary vectors. From it we derive a new
distributional algorithm which is fully Cram\'er-based and can be combined to
linear function approximation, with formal guarantees in the context of policy
evaluation. In allowing the model's prediction to be any real vector, we lose
the probabilistic interpretation behind the method, but otherwise maintain the
appealing properties of distributional approaches. To the best of our
knowledge, ours is the first proof of convergence of a distributional algorithm
combined with function approximation. Perhaps surprisingly, our results provide
evidence that Cram\'er-based distributional methods may perform worse than
directly approximating the value function.
Novelty Search for Deep Reinforcement Learning Policy Network Weights by Action Sequence Edit Metric Distance
Authors: Ethan C. Jackson, Mark Daley
Date: 2019-02-08 00:00:00
Tasks: Deep Reinforcement Learning, Evolutionary Algorithms, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning (RL) problems often feature deceptive local optima,
and learning methods that optimize purely for reward signal often fail to learn
strategies for overcoming them. Deep neuroevolution and novelty search have
been proposed as effective alternatives to gradient-based methods for learning
RL policies directly from pixels. In this paper, we introduce and evaluate the
use of novelty search over agent action sequences by string edit metric
distance as a means for promoting innovation. We also introduce a method for
stagnation detection and population resampling inspired by recent developments
in the RL community that uses the same mechanisms as novelty search to promote
and develop innovative policies. Our methods extend a state-of-the-art method
for deep neuroevolution using a simple-yet-effective genetic algorithm (GA)
designed to efficiently learn deep RL policy network weights. Experiments using
four games from the Atari 2600 benchmark were conducted. Results provide
further evidence that GAs are competitive with gradient-based algorithms for
deep RL. Results also demonstrate that novelty search over action sequences is
an effective source of selection pressure that can be integrated into existing
evolutionary algorithms for deep RL.
Linear and Nonlinear Identification of Dryer System Using Artificial Intelligence and Neural Networks
Authors: Mostafa Darvishi
Date: 2018-11-16 00:00:00

As you read these words you are using a complex biological neural network.
You have a highly interconnected set of some neurons to facilitate your
reading, breathing, motion and thinking. Each of your biological neurons, a
rich assembly of tissue and chemistry, has the complexity, if not the speed, of
a microprocessor. Some of your neural structure was with you at birth. Other
parts have been established by experience.
Bounded Fuzzy Possibilistic Method
Authors: Hossein Yazdani
Date: 2019-02-08 00:00:00
Tasks: Anomaly Detection, Clustering, Management

This paper introduces Bounded Fuzzy Possibilistic Method (BFPM) by addressing
several issues that previous clustering/classification methods have not
considered. In fuzzy clustering, object's membership values should sum to 1.
Hence, any object may obtain full membership in at most one cluster.
Possibilistic clustering methods remove this restriction. However, BFPM differs
from previous fuzzy and possibilistic clustering approaches by allowing the
membership function to take larger values with respect to all clusters.
Furthermore, in BFPM, a data object can have full membership in multiple
clusters or even in all clusters. BFPM relaxes the boundary conditions
(restrictions) in membership assignment. The proposed methodology satisfies the
necessity of obtaining full memberships and overcomes the issues with
conventional methods on dealing with overlapping. Analysing the objects'
movements from their own cluster to another (mutation) is also proposed in this
paper. BFPM has been applied in different domains in geometry, set theory,
anomaly detection, risk management, diagnosis diseases, and other disciplines.
Validity and comparison indexes have been also used to evaluate the accuracy of
BFPM. BFPM has been evaluated in terms of accuracy, fuzzification constant
(different norms), objects' movement analysis, and covering diversity. The
promising results prove the importance of considering the proposed methodology
in learning methods to track the behaviour of data objects, in addition to
obtain accurate results.
Humor in Word Embeddings: Cockamamie Gobbledegook for Nincompoops
Authors: Limor Gultchin, Genevieve Patterson, Nancy Baym, Nathaniel Swinger, Adam Tauman Kalai
Date: 2019-02-08 00:00:00
Tasks: Clustering, Word Embeddings

While humor is often thought to be beyond the reach of Natural Language Processing, we show that several aspects of single-word humor correlate with simple linear directions in Word Embeddings. In particular: (a) the word vectors capture multiple aspects discussed in humor theories from various disciplines; (b) each individual's sense of humor can be represented by a vector, which can predict differences in people's senses of humor on new, unrated, words; and (c) upon clustering humor ratings of multiple demographic groups, different humor preferences emerge across the different groups. Humor ratings are taken from the work of Engelthaler and Hills (2017) as well as from an original crowdsourcing study of 120,000 words. Our dataset further includes annotations for the theoretically-motivated humor features we identify.
Using Background Knowledge to Rank Itemsets
Authors: Nikolaj Tatti, Michael Mampaey
Date: 2019-02-08 00:00:00

Assessing the quality of discovered results is an important open problem in
data mining. Such assessment is particularly vital when mining itemsets, since
commonly many of the discovered patterns can be easily explained by background
knowledge. The simplest approach to screen uninteresting patterns is to compare
the observed frequency against the independence model. Since the parameters for
the independence model are the column margins, we can view such screening as a
way of using the column margins as background knowledge.
  In this paper we study techniques for more flexible approaches for infusing
background knowledge. Namely, we show that we can efficiently use additional
knowledge such as row margins, lazarus counts, and bounds of ones. We
demonstrate that these statistics describe forms of data that occur in practice
and have been studied in data mining.
  To infuse the information efficiently we use a maximum entropy approach. In
its general setting, solving a maximum entropy model is infeasible, but we
demonstrate that for our setting it can be solved in polynomial time.
Experiments show that more sophisticated models fit the data better and that
using more information improves the frequency prediction of itemsets.
FocusNet: An attention-based Fully Convolutional Network for Medical Image Segmentation
Authors: Chaitanya Kaul, Suresh Manandhar, Nick Pears
Date: 2019-02-08 00:00:00
Tasks: Image Segmentation, Lesion Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation, Skin Cancer Segmentation

We propose a novel technique to incorporate attention within convolutional
neural networks using feature maps generated by a separate convolutional
autoencoder. Our attention architecture is well suited for incorporation with
deep convolutional networks. We evaluate our model on benchmark segmentation
datasets in skin cancer segmentation and lung lesion segmentation. Results show
highly competitive performance when compared with U-Net and it's residual
variant.
Addressing Overfitting on Pointcloud Classification using Atrous XCRF
Authors: Hasan Asyari Arief, Ulf Geir Indahl, Geir-Harald Strand, Håvard Tveite
Date: 2019-02-08 00:00:00
Tasks: Classification, General Classification

Advances in techniques for automated classification of pointcloud data
introduce great opportunities for many new and existing applications. However,
with a limited number of labeled points, automated classification by a machine
learning model is prone to overfitting and poor generalization. The present
paper addresses this problem by inducing controlled noise (on a trained model)
generated by invoking conditional random field similarity penalties using
nearby features. The method is called Atrous XCRF and works by forcing a
trained model to respect the similarity penalties provided by unlabeled data.
In a benchmark study carried out using the ISPRS 3D labeling dataset, our
technique achieves 84.97% in term of overall accuracy, and 71.05% in term of F1
score. The result is on par with the current best model for the benchmark
dataset and has the highest value in term of F1 score.
Affine Invariant Covariance Estimation for Heavy-Tailed Distributions
Authors: Dmitrii Ostrovskii, Alessandro Rudi
Date: 2019-02-08 00:00:00

In this work we provide an estimator for the covariance matrix of a heavy-tailed multivariate distributionWe prove that the proposed estimator $\widehat{\mathbf{S}}$ admits an \textit{affine-invariant} bound of the form \[(1-\varepsilon) \mathbf{S} \preccurlyeq \widehat{\mathbf{S}} \preccurlyeq (1+\varepsilon) \mathbf{S}\]in high probability, where $\mathbf{S}$ is the unknown covariance matrix, and $\preccurlyeq$ is the positive semidefinite order on symmetric matrices. The result only requires the existence of fourth-order moments, and allows for $\varepsilon = O(\sqrt{\kappa^4 d\log(d/\delta)/n})$ where $\kappa^4$ is a measure of kurtosis of the distribution, $d$ is the dimensionality of the space, $n$ is the sample size, and $1-\delta$ is the desired confidence level. More generally, we can allow for regularization with level $\lambda$, then $d$ gets replaced with the degrees of freedom number. Denoting $\text{cond}(\mathbf{S})$ the condition number of $\mathbf{S}$, the computational cost of the novel estimator is $O(d^2 n + d^3\log(\text{cond}(\mathbf{S})))$, which is comparable to the cost of the sample covariance estimator in the statistically interesing regime $n \ge d$. We consider applications of our estimator to eigenvalue estimation with relative error, and to ridge regression with heavy-tailed random design.