pwc-archive/papers-with-abstracts huggingface dataset:
Bayesian Allocation Model: Inference by Sequential Monte Carlo for Nonnegative Tensor Factorizations and Topic Models using Polya Urns
Authors: Ali Taylan Cemgil, Mehmet Burak Kurutmaz, Sinan Yildirim, Melih Barsbey, Umut Simsekli
Date: 2019-03-11 00:00:00
Tasks: Model Selection, Topic Models

We introduce a dynamic generative model, Bayesian allocation model (BAM),
which establishes explicit connections between nonnegative tensor factorization
(NTF), graphical models of discrete probability distributions and their
Bayesian extensions, and the topic models such as the latent Dirichlet
allocation. BAM is based on a Poisson process, whose events are marked by using
a Bayesian network, where the conditional probability tables of this network
are then integrated out analytically. We show that the resulting marginal
process turns out to be a Polya urn, an integer valued self-reinforcing
process. This urn processes, which we name a Polya-Bayes process, obey certain
conditional independence properties that provide further insight about the
nature of NTF. These insights also let us develop space efficient simulation
algorithms that respect the potential sparsity of data: we propose a class of
sequential importance sampling algorithms for computing NTF and approximating
their marginal likelihood, which would be useful for model selection. The
resulting methods can also be viewed as a model scoring method for topic models
and discrete Bayesian networks with hidden variables. The new algorithms have
favourable properties in the sparse data regime when contrasted with
variational algorithms that become more accurate when the total sum of the
elements of the observed tensor goes to infinity. We illustrate the performance
on several examples and numerically study the behaviour of the algorithms for
various data regimes.
Continual Learning via Neural Pruning
Authors: Siavash Golkar, Michael Kagan, Kyunghyun Cho
Date: 2019-03-11 00:00:00
Tasks: Continual Learning, Diagnostic, Lifelong learning

We introduce Continual Learning via Neural Pruning (CLNP), a new method aimed
at lifelong learning in fixed capacity models based on neuronal model
sparsification. In this method, subsequent tasks are trained using the inactive
neurons and filters of the sparsified network and cause zero deterioration to
the performance of previous tasks. In order to deal with the possible
compromise between model sparsity and performance, we formalize and incorporate
the concept of graceful forgetting: the idea that it is preferable to suffer a
small amount of forgetting in a controlled manner if it helps regain network
capacity and prevents uncontrolled loss of performance during the training of
future tasks. CLNP also provides simple continual learning diagnostic tools in
terms of the number of free neurons left for the training of future tasks as
well as the number of neurons that are being reused. In particular, we see in
experiments that CLNP verifies and automatically takes advantage of the fact
that the features of earlier layers are more transferable. We show empirically
that CLNP leads to significantly improved results over current weight
elasticity based methods.
The Past and the Present of the Color Checker Dataset Misuse
Authors: Nikola Banić, Karlo Koš{č}ević, Marko Subašić, Sven Lon{č}arić
Date: 2019-03-11 00:00:00
Tasks: Color Constancy

The pipelines of digital cameras contain a part for computational color
constancy, which aims to remove the influence of the illumination on the scene
colors. One of the best known and most widely used benchmark datasets for this
problem is the Color Checker dataset. However, due to the improper handling of
the black level in its images, this dataset has been widely misused and while
some recent publications tried to alleviate the problem, they nevertheless
erred and created additional wrong data. This paper gives a history of the
Color Checker dataset usage, it describes the origins and reasons for its
misuses, and it explains the old and new mistakes introduced in the most recent
publications that tried to handle the issue. This should, hopefully, help to
prevent similar future misuses.
Scaling up deep neural networks: a capacity allocation perspective
Authors: Jonathan Donier
Date: 2019-03-11 00:00:00

Following the recent work on capacity allocation, we formulate the conjecture
that the shattering problem in deep neural networks can only be avoided if the
capacity propagation through layers has a non-degenerate continuous limit when
the number of layers tends to infinity. This allows us to study a number of
commonly used architectures and determine which scaling relations should be
enforced in practice as the number of layers grows large. In particular, we
recover the conditions of Xavier initialization in the multi-channel case, and
we find that weights and biases should be scaled down as the inverse square
root of the number of layers for deep residual networks and as the inverse
square root of the desired memory length for recurrent networks.
Pragmatic inference and visual abstraction enable contextual flexibility during visual communication
Authors: Judith Fan, Robert Hawkins, Mike Wu, Noah Goodman
Date: 2019-03-11 00:00:00

Visual modes of communication are ubiquitous in modern life --- from maps to data plots to political cartoons. Here we investigate drawing, the most basic form of visual communication. Participants were paired in an online environment to play a drawing-based reference game. On each trial, both participants were shown the same four objects, but in different locations. The sketcher's goal was to draw one of these objects so that the viewer could select it from the array. On `close' trials, objects belonged to the same basic-level category, whereas on `far' trials objects belonged to different categories. We found that people exploited shared information to efficiently communicate about the target object: on far trials, sketchers achieved high recognition accuracy while applying fewer strokes, using less ink, and spending less time on their drawings than on close trials. We hypothesized that humans succeed in this task by recruiting two core faculties: visual abstraction, the ability to perceive the correspondence between an object and a drawing of it; and pragmatic inference, the ability to judge what information would help a viewer distinguish the target from distractors. To evaluate this hypothesis, we developed a computational model of the sketcher that embodied both faculties, instantiated as a deep convolutional neural network nested within a probabilistic program. We found that this model fit human data well and outperformed lesioned variants. Together, this work provides the first algorithmically explicit theory of how visual perception and social cognition jointly support contextual flexibility in visual communication.
Physics Enhanced Artificial Intelligence
Authors: Patrick O'Driscoll, Jaehoon Lee, Bo Fu
Date: 2019-03-11 00:00:00
Tasks: BIG-bench Machine Learning

We propose that intelligently combining models from the domains of Artificial
Intelligence or Machine Learning with Physical and Expert models will yield a
more "trustworthy" model than any one model from a single domain, given a
complex and narrow enough problem. Based on mean-variance portfolio theory and
bias-variance trade-off analysis, we prove combining models from various
domains produces a model that has lower risk, increasing user trust. We call
such combined models - physics enhanced artificial intelligence (PEAI), and
suggest use cases for PEAI.
Mean Field Analysis of Deep Neural Networks
Authors: Justin Sirignano, Konstantinos Spiliopoulos
Date: 2019-03-11 00:00:00
Tasks: valid

We analyze multi-layer neural networks in the asymptotic regime of simultaneously (A) large network sizes and (B) large numbers of stochastic gradient descent training iterations. We rigorously establish the limiting behavior of the multi-layer neural network output. The limit procedure is valid for any number of hidden layers and it naturally also describes the limiting behavior of the training loss. The ideas that we explore are to (a) take the limits of each hidden layer sequentially and (b) characterize the evolution of parameters in terms of their initialization. The limit satisfies a system of deterministic integro-differential equations. The proof uses methods from weak convergence and stochastic analysis. We show that, under suitable assumptions on the activation functions and the behavior for large times, the limit neural network recovers a global minimum (with zero loss for the objective function).
ETNLP: a visual-aided systematic approach to select pre-trained embeddings for a downstream task
Authors: Xuan-Son Vu, Thanh Vu, Son N. Tran, Lili Jiang
Date: 2019-03-11 00:00:00
Proceeding: RANLP 2019 9
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Word Embeddings

Given many recent advanced embedding models, selecting pre-trained word embedding (a.k.a., word representation) models best fit for a specific downstream task is non-trivial. In this paper, we propose a systematic approach, called ETNLP, for extracting, evaluating, and visualizing multiple sets of pre-trained word embeddings to determine which embeddings should be used in a downstream task. For extraction, we provide a method to extract subsets of the embeddings to be used in the downstream task. For evaluation, we analyse the quality of pre-trained embeddings using an input word analogy list. Finally, we visualize the word representations in the embedding space to explore the embedded words interactively. We demonstrate the effectiveness of the proposed approach on our pre-trained word embedding models in Vietnamese to select which models are suitable for a named entity recognition (NER) task. Specifically, we create a large Vietnamese word analogy list to evaluate and select the pre-trained embedding models for the task. We then utilize the selected embeddings for the NER task and achieve the new state-of-the-art results on the task benchmark dataset. We also apply the approach to another downstream task of privacy-guaranteed embedding selection, and show that it helps users quickly select the most suitable embeddings. In addition, we create an open-source system using the proposed systematic approach to facilitate similar studies on other NLP tasks. The source code and data are available at https://github.com/vietnlp/etnlp.
Augmenting expert detection of early coronary artery occlusion from 12 lead electrocardiograms using deep learning
Authors: Rob Brisk, Raymond R Bond. Dewar D Finlay, James McLaughlin, Alicja Piadlo, Stephen J Leslie, David E Gossman, Ian B A Menown, David J McEneaney
Date: 2019-03-11 00:00:00
Tasks: Sensitivity, Specificity

Early diagnosis of acute coronary artery occlusion based on electrocardiogram (ECG) findings is essential for prompt delivery of primary percutaneous coronary intervention. Current ST elevation (STE) criteria are specific but insensitive. Consequently, it is likely that many patients are missing out on potentially life-saving treatment. Experts combining non-specific ECG changes with STE detect ischaemia with higher sensitivity, but at the cost of specificity. We show that a deep learning model can detect ischaemia caused by acute coronary artery occlusion with a better balance of sensitivity and specificity than STE criteria, existing computerised analysers or expert cardiologists.
Diffusion $K$-means clustering on manifolds: provable exact recovery via semidefinite relaxations
Authors: Xiaohui Chen, Yun Yang
Date: 2019-03-11 00:00:00
Tasks: Clustering

We introduce the {\it diffusion $K$-means} clustering method on Riemannian submanifolds, which maximizes the within-cluster connectedness based on the diffusion distance. The diffusion $K$-means constructs a random walk on the similarity graph with vertices as data points randomly sampled on the manifolds and edges as similarities given by a kernel that captures the local geometry of manifolds. The diffusion $K$-means is a multi-scale clustering tool that is suitable for data with non-linear and non-Euclidean geometric features in mixed dimensions. Given the number of clusters, we propose a polynomial-time convex relaxation algorithm via the semidefinite programming (SDP) to solve the diffusion $K$-means. In addition, we also propose a nuclear norm regularized SDP that is adaptive to the number of clusters. In both cases, we show that exact recovery of the SDPs for diffusion $K$-means can be achieved under suitable between-cluster separability and within-cluster connectedness of the submanifolds, which together quantify the hardness of the manifold clustering problem. We further propose the {\it localized diffusion $K$-means} by using the local adaptive bandwidth estimated from the nearest neighbors. We show that exact recovery of the localized diffusion $K$-means is fully adaptive to the local probability density and geometric structures of the underlying submanifolds.
Building an Affordances Map with Interactive Perception
Authors: Leni K. Le Goff, Oussama Yaakoubi, Alexandre Coninx, Stephane Doncieux
Date: 2019-03-11 00:00:00
Tasks: General Classification, Scene Understanding

Robots need to understand their environment to perform their task. If it is
possible to pre-program a visual scene analysis process in closed environments,
robots operating in an open environment would benefit from the ability to learn
it through their interaction with their environment. This ability furthermore
opens the way to the acquisition of affordances maps in which the action
capabilities of the robot structure its visual scene understanding. We propose
an approach to build such affordances maps by relying on an interactive
perception approach and an online classification. In the proposed formalization
of affordances, actions and effects are related to visual features, not
objects, and they can be combined. We have tested the approach on three action
primitives and on a real PR2 robot.
Accuracy Booster: Performance Boosting using Feature Map Re-calibration
Authors: Pravendra Singh, Pratik Mazumder, Vinay P. Namboodiri
Date: 2019-03-11 00:00:00
Tasks: General Classification, object-detection, Object Detection

Convolution Neural Networks (CNN) have been extremely successful in solving intensive computer vision tasks. The convolutional filters used in CNNs have played a major role in this success, by extracting useful features from the inputs. Recently researchers have tried to boost the performance of CNNs by re-calibrating the feature maps produced by these filters, e.g., Squeeze-and-Excitation Networks (SENets). These approaches have achieved better performance by Exciting up the important channels or feature maps while diminishing the rest. However, in the process, architectural complexity has increased. We propose an architectural block that introduces much lower complexity than the existing methods of CNN performance boosting while performing significantly better than them. We carry out experiments on the CIFAR, ImageNet and MS-COCO datasets, and show that the proposed block can challenge the state-of-the-art results. Our method boosts the ResNet-50 architecture to perform comparably to the ResNet-152 architecture, which is a three times deeper network, on classification. We also show experimentally that our method is not limited to classification but also generalizes well to other tasks such as object detection.
Deep learning for molecular design - a review of the state of the art
Authors: Daniel C. Elton, Zois Boukouvalas, Mark D. Fuge, Peter W. Chung
Date: 2019-03-11 00:00:00
Tasks: Benchmarking, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In the space of only a few years, deep generative modeling has revolutionized how we think of artificial creativity, yielding autonomous systems which produce original images, music, and text. Inspired by these successes, researchers are now applying deep generative modeling techniques to the generation and optimization of molecules - in our review we found 45 papers on the subject published in the past two years. These works point to a future where such systems will be used to generate lead molecules, greatly reducing resources spent downstream synthesizing and characterizing bad leads in the lab. In this review we survey the increasingly complex landscape of models and representation schemes that have been proposed. The four classes of techniques we describe are recursive neural networks, autoencoders, generative adversarial networks, and reinforcement learning. After first discussing some of the mathematical fundamentals of each technique, we draw high level connections and comparisons with other techniques and expose the pros and cons of each. Several important high level themes emerge as a result of this work, including the shift away from the SMILES string representation of molecules towards more sophisticated representations such as graph grammars and 3D representations, the importance of reward function design, the need for better standards for benchmarking and testing, and the benefits of adversarial training and reinforcement learning over maximum likelihood based training.
SleepNet: Automated Sleep Analysis via Dense Convolutional Neural Network Using Physiological Time Series
Authors: Bahareh Pourbabaee, Matthew Howe-Patterson, Matthew Patterson, Frederic Benard
Date: 2019-03-11 00:00:00
Tasks: Multi-Task Learning, Time Series, Time Series Analysis

In this work, a dense recurrent convolutional neural network (DRCNN) was constructed to detect sleep disorders including arousal, apnea and hypopnea using Polysomnography (PSG) measurement channels provided in the 2018 Physionet challenge database. Our model structure is composed of multiple dense convolutional units (DCU) followed by a bidirectional long-short term memory (LSTM) layer followed by a softmax output layer. The sleep events including sleep stages, arousal regions and multiple types of apnea and hypopnea are manually annotated by experts which enables us to train our proposed network using a multi-task learning mechanism. Three binary cross-entropy loss functions corresponding to sleep/wake, target arousal and apnea-hypopnea/normal detection tasks are summed up to generate our overall network loss function that is optimized using the Adam method. Our model performance was evaluated using two metrics: the area under the precision-recall curve (AUPRC) and the area under the receiver operating characteristic curve (AUROC). To measure our model generalization, 4-fold cross-validation was also performed. For training, our model was applied to full night recording data. Finally, the average AUPRC and AUROC values associated with the arousal detection task were 0.505 and 0.922, respectively on our testing dataset. An ensemble of four models trained on different data folds improved the AUPRC and AUROC to 0.543 and 0.931, respectively. Our proposed algorithm achieved the first place in the official stage of the 2018 Physionet challenge for detecting sleep arousals with AUPRC of 0.54 on the blind testing dataset.
ADS-ME: Anomaly Detection System for Micro-expression Spotting
Authors: Dawood Al Chanti, Alice Caplier
Date: 2019-03-11 00:00:00
Tasks: Anomaly Detection, Micro-Expression Spotting

Micro-expressions (MEs) are infrequent and uncontrollable facial events that
can highlight emotional deception and appear in a high-stakes environment. This
paper propose an algorithm for spatiotemporal MEs spotting. Since MEs are
unusual events, we treat them as abnormal patterns that diverge from expected
Normal Facial Behaviour (NFBs) patterns. NFBs correspond to facial muscle
activations, eye blink/gaze events and mouth opening/closing movements that are
all facial deformation but not MEs. We propose a probabilistic model to
estimate the probability density function that models the spatiotemporal
distributions of NFBs patterns. To rank the outputs, we compute the negative
log-likelihood and we developed an adaptive thresholding technique to identify
MEs from NFBs. While working only with NFBs data, the main challenge is to
capture intrinsic spatiotemoral features, hence we design a recurrent
convolutional autoencoder for feature representation. Finally, we show that our
system is superior to previous works for MEs spotting.
Joint inference on structural and diffusion MRI for sequence-adaptive Bayesian segmentation of thalamic nuclei with probabilistic atlases
Authors: Juan Eugenio Iglesias, Koen van Leemput, Polina Golland, Anastasia Yendiki
Date: 2019-03-11 00:00:00
Tasks: Diffusion  MRI, Segmentation

Segmentation of structural and diffusion MRI (sMRI/dMRI) is usually performed
independently in neuroimaging pipelines. However, some brain structures (e.g.,
globus pallidus, thalamus and its nuclei) can be extracted more accurately by
fusing the two modalities. Following the framework of Bayesian segmentation
with probabilistic atlases and unsupervised appearance modeling, we present
here a novel algorithm to jointly segment multi-modal sMRI/dMRI data. We
propose a hierarchical likelihood term for the dMRI defined on the unit ball,
which combines the Beta and Dimroth-Scheidegger-Watson distributions to model
the data at each voxel. This term is integrated with a mixture of Gaussians for
the sMRI data, such that the resulting joint unsupervised likelihood enables
the analysis of multi-modal scans acquired with any type of MRI contrast,
b-values, or number of directions, which enables wide applicability. We also
propose an inference algorithm to estimate the maximum-a-posteriori model
parameters from input images, and to compute the most likely segmentation.
Using a recently published atlas derived from histology, we apply our method to
thalamic nuclei segmentation on two datasets: HCP (state of the art) and ADNI
(legacy) - producing lower sample sizes than Bayesian segmentation with sMRI
alone.
A Spiking Network for Inference of Relations Trained with Neuromorphic Backpropagation
Authors: Johannes C. Thiele, Olivier Bichler, Antoine Dupret, Sergio Solinas, Giacomo Indiveri
Date: 2019-03-11 00:00:00
Tasks: Sensor Fusion

The increasing need for intelligent sensors in a wide range of everyday
objects requires the existence of low power information processing systems
which can operate autonomously in their environment. In particular, merging and
processing the outputs of different sensors efficiently is a necessary
requirement for mobile agents with cognitive abilities. In this work, we
present a multi-layer spiking neural network for inference of relations between
stimuli patterns in dedicated neuromorphic systems. The system is trained with
a new version of the backpropagation algorithm adapted to on-chip learning in
neuromorphic hardware: Error gradients are encoded as spike signals which are
propagated through symmetric synapses, using the same integrate-and-fire
hardware infrastructure as used during forward propagation. We demonstrate the
strength of the approach on an arithmetic relation inference task and on visual
XOR on the MNIST dataset. Compared to previous, biologically-inspired
implementations of networks for learning and inference of relations, our
approach is able to achieve better performance with less neurons. Our
architecture is the first spiking neural network architecture with on-chip
learning capabilities, which is able to perform relational inference on complex
visual stimuli. These features make our system interesting for sensor fusion
applications and embedded learning in autonomous neuromorphic agents.