pwc-archive/papers-with-abstracts huggingface dataset:
Adversarial Message Passing For Graphical Models
Authors: Theofanis Karaletsos
Date: 2016-12-15 00:00:00
Tasks: Bayesian Inference, Probabilistic Programming

Bayesian inference on structured models typically relies on the ability to
infer posterior distributions of underlying hidden variables. However,
inference in implicit models or complex posterior distributions is hard. A
popular tool for learning implicit models are generative adversarial networks
(GANs) which learn parameters of generators by fooling discriminators.
Typically, GANs are considered to be models themselves and are not understood
in the context of inference. Current techniques rely on inefficient global
discrimination of joint distributions to perform learning, or only consider
discriminating a single output variable. We overcome these limitations by
treating GANs as a basis for likelihood-free inference in generative models and
generalize them to Bayesian posterior inference over factor graphs. We propose
local learning rules based on message passing minimizing a global divergence
criterion involving cooperating local adversaries used to sidestep explicit
likelihood evaluations. This allows us to compose models and yields a unified
inference and learning framework for adversarial learning. Our framework treats
model specification and inference separately and facilitates richly structured
models within the family of Directed Acyclic Graphs, including components such
as intractable likelihoods, non-differentiable models, simulators and generally
cumbersome models. A key result of our treatment is the insight that Bayesian
inference on structured models can be performed only with sampling and
discrimination when using nonparametric variational families, without access to
explicit distributions. As a side-result, we discuss the link to likelihood
maximization. These approaches hold promise to be useful in the toolbox of
probabilistic modelers and enrich the gamut of current probabilistic
programming applications.
Objective Micro-Facial Movement Detection Using FACS-Based Regions and Baseline Evaluation
Authors: Adrian K. Davison, Cliff Lansley, Choon Ching Ng, Kevin Tan, Moi Hoon Yap
Date: 2016-12-15 00:00:00
Tasks: Optical Flow Estimation

Micro-facial expressions are regarded as an important human behavioural event
that can highlight emotional deception. Spotting these movements is difficult
for humans and machines, however research into using computer vision to detect
subtle facial expressions is growing in popularity. This paper proposes an
individualised baseline micro-movement detection method using 3D Histogram of
Oriented Gradients (3D HOG) temporal difference method. We define a face
template consisting of 26 regions based on the Facial Action Coding System
(FACS). We extract the temporal features of each region using 3D HOG. Then, we
use Chi-square distance to find subtle facial motion in the local regions.
Finally, an automatic peak detector is used to detect micro-movements above the
newly proposed adaptive baseline threshold. The performance is validated on two
FACS coded datasets: SAMM and CASME II. This objective method focuses on the
movement of the 26 face regions. When comparing with the ground truth, the best
result was an AUC of 0.7512 and 0.7261 on SAMM and CASME II, respectively. The
results show that 3D HOG outperformed for micro-movement detection, compared to
state-of-the-art feature representations: Local Binary Patterns in Three
Orthogonal Planes and Histograms of Oriented Optical Flow.
Ontohub: A semantic repository for heterogeneous ontologies
Authors: Mihai Codescu, Eugen Kuksa, Oliver Kutz, Till Mossakowski, Fabian Neuhaus
Date: 2016-12-15 00:00:00

Ontohub is a repository engine for managing distributed heterogeneous
ontologies. The distributed nature enables communities to share and exchange
their contributions easily. The heterogeneous nature makes it possible to
integrate ontologies written in various ontology languages. Ontohub supports a
wide range of formal logical and ontology languages, as well as various
structuring and modularity constructs and inter-theory (concept) mappings,
building on the OMG-standardized DOL language. Ontohub repositories are
organised as Git repositories, thus inheriting all features of this popular
version control system. Moreover, Ontohub is the first repository engine
meeting a substantial amount of the requirements formulated in the context of
the Open Ontology Repository (OOR) initiative, including an API for federation
as well as support for logical inference and axiom selection.
Simultaneous Estimation of Noise Variance and Number of Peaks in Bayesian Spectral Deconvolution
Authors: Satoru Tokuda, Kenji Nagata, Masato Okada
Date: 2016-07-26 00:00:00
Tasks: Bayesian Inference, parameter estimation

The heuristic identification of peaks from noisy complex spectra often leads
to misunderstanding of the physical and chemical properties of matter. In this
paper, we propose a framework based on Bayesian inference, which enables us to
separate multipeak spectra into single peaks statistically and consists of two
steps. The first step is estimating both the noise variance and the number of
peaks as hyperparameters based on Bayes free energy, which generally is not
analytically tractable. The second step is fitting the parameters of each peak
function to the given spectrum by calculating the posterior density, which has
a problem of local minima and saddles since multipeak models are nonlinear and
hierarchical. Our framework enables the escape from local minima or saddles by
using the exchange Monte Carlo method and calculates Bayes free energy via the
multiple histogram method. We discuss a simulation demonstrating how efficient
our framework is and show that estimating both the noise variance and the
number of peaks prevents overfitting, overpenalizing, and misunderstanding the
precision of parameter estimation.
Identification of Cancer Patient Subgroups via Smoothed Shortest Path Graph Kernel
Authors: Ali Burak Ünal, Öznur Taştan
Date: 2016-12-13 00:00:00
Tasks: Clustering, Survival Analysis

Characterizing patient somatic mutations through next-generation sequencing
technologies opens up possibilities for refining cancer subtypes. However,
catalogues of mutations reveal that only a small fraction of the genes are
altered frequently in patients. On the other hand different genomic alterations
may perturb the same pathways. We propose a novel clustering procedure that
quantifies the similarities of patients from their mutational profile on
pathways via a novel graph kernel. We represent each KEGG pathway as an
undirected graph. For each patient the vertex labels are assigned based on her
altered genes. Smoothed shortest path graph kernel (smSPK) evaluates each pair
of patients by comparing their vertex labeled pathway graphs. Our clustering
procedure involves two steps: the smSPK kernel matrix derived for each pathway
are input to kernel k-means algorithm and each pathway is evaluated
individually. In the next step, only those pathways that are successful are
combined in to a single kernel input to kernel k-means to stratify patients.
Evaluating the procedure on simulated data showed that smSPK clusters patients
up to 88\% accuracy. Finally to identify ovarian cancer patient subgroups, we
apply our methodology to the cancer genome atlas ovarian data that involves 481
patients. The identified subgroups are evaluated through survival analysis.
Grouping patients into four clusters results with patients groups that are
significantly different in their survival times ($p$-value $\le 0.005$).
Graph-based semi-supervised learning for relational networks
Authors: Leto Peel
Date: 2016-12-15 00:00:00
Tasks: General Classification

We address the problem of semi-supervised learning in relational networks,
networks in which nodes are entities and links are the relationships or
interactions between them. Typically this problem is confounded with the
problem of graph-based semi-supervised learning (GSSL), because both problems
represent the data as a graph and predict the missing class labels of nodes.
However, not all graphs are created equally. In GSSL a graph is constructed,
often from independent data, based on similarity. As such, edges tend to
connect instances with the same class label. Relational networks, however, can
be more heterogeneous and edges do not always indicate similarity. For
instance, instead of links being more likely to connect nodes with the same
class label, they may occur more frequently between nodes with different class
labels (link-heterogeneity). Or nodes with the same class label do not
necessarily have the same type of connectivity across the whole network
(class-heterogeneity), e.g. in a network of sexual interactions we may observe
links between opposite genders in some parts of the graph and links between the
same genders in others. Performing classification in networks with different
types of heterogeneity is a hard problem that is made harder still when we do
not know a-priori the type or level of heterogeneity. Here we present two
scalable approaches for graph-based semi-supervised learning for the more
general case of relational networks. We demonstrate these approaches on
synthetic and real-world networks that display different link patterns within
and between classes. Compared to state-of-the-art approaches, ours give better
classification performance without prior knowledge of how classes interact. In
particular, our two-step label propagation algorithm gives consistently good
accuracy and runs on networks of over 1.6 million nodes and 30 million edges in
around 12 seconds.
TeKnowbase: Towards Construction of a Knowledge-base of Technical Concepts
Authors: Prajna Upadhyay, Tanuma Patra, Ashwini Purkar, Maya Ramanath
Date: 2016-12-15 00:00:00
Tasks: Classification, General Classification, Knowledge Base Construction

In this paper, we describe the construction of TeKnowbase, a knowledge-base
of technical concepts in computer science. Our main information sources are
technical websites such as Webopedia and Techtarget as well as Wikipedia and
online textbooks. We divide the knowledge-base construction problem into two
parts -- the acquisition of entities and the extraction of relationships among
these entities. Our knowledge-base consists of approximately 100,000 triples.
We conducted an evaluation on a sample of triples and report an accuracy of a
little over 90\%. We additionally conducted classification experiments on
StackOverflow data with features from TeKnowbase and achieved improved
classification accuracy.
Improving Neural Network Generalization by Combining Parallel Circuits with Dropout
Authors: Kien Tuong Phan, Tomas Henrique Maul, Tuong Thuy Vu, Lai Weng Kin
Date: 2016-12-15 00:00:00

In an attempt to solve the lengthy training times of neural networks, we
proposed Parallel Circuits (PCs), a biologically inspired architecture.
Previous work has shown that this approach fails to maintain generalization
performance in spite of achieving sharp speed gains. To address this issue, and
motivated by the way Dropout prevents node co-adaption, in this paper, we
suggest an improvement by extending Dropout to the PC architecture. The paper
provides multiple insights into this combination, including a variety of fusion
approaches. Experiments show promising results in which improved error rates
are achieved in most cases, whilst maintaining the speed advantage of the PC
approach.
Design of Image Matched Non-Separable Wavelet using Convolutional Neural Network
Authors: Naushad Ansari, Anubha Gupta, Rahul Duggal
Date: 2016-12-15 00:00:00
Tasks: Compressive Sensing, General Classification, image-classification, Image Classification

Image-matched nonseparable wavelets can find potential use in many
applications including image classification, segmen- tation, compressive
sensing, etc. This paper proposes a novel design methodology that utilizes
convolutional neural net- work (CNN) to design two-channel non-separable
wavelet matched to a given image. The design is proposed on quin- cunx lattice.
The loss function of the convolutional neural network is setup with total
squared error between the given input image to CNN and the reconstructed image
at the output of CNN, leading to perfect reconstruction at the end of train-
ing. Simulation results have been shown on some standard images.
Recurrent Image Captioner: Describing Images with Spatial-Invariant Transformation and Attention Filtering
Authors: Hao Liu, Yang Yang, Fumin Shen, Lixin Duan, Heng Tao Shen
Date: 2016-12-15 00:00:00
Tasks: Decoder, Image Captioning, Image Description, Variational Inference

Along with the prosperity of recurrent neural network in modelling sequential
data and the power of attention mechanism in automatically identify salient
information, image captioning, a.k.a., image description, has been remarkably
advanced in recent years. Nonetheless, most existing paradigms may suffer from
the deficiency of invariance to images with different scaling, rotation, etc.;
and effective integration of standalone attention to form a holistic end-to-end
system. In this paper, we propose a novel image captioning architecture, termed
Recurrent Image Captioner (\textbf{RIC}), which allows visual encoder and
language decoder to coherently cooperate in a recurrent manner. Specifically,
we first equip CNN-based visual encoder with a differentiable layer to enable
spatially invariant transformation of visual signals. Moreover, we deploy an
attention filter module (differentiable) between encoder and decoder to
dynamically determine salient visual parts. We also employ bidirectional LSTM
to preprocess sentences for generating better textual representations. Besides,
we propose to exploit variational inference to optimize the whole architecture.
Extensive experimental results on three benchmark datasets (i.e., Flickr8k,
Flickr30k and MS COCO) demonstrate the superiority of our proposed architecture
as compared to most of the state-of-the-art methods.
Stochastic Synapses Enable Efficient Brain-Inspired Learning Machines
Authors: Emre O. Neftci, Bruno U. Pedroni, Siddharth Joshi, Maruan Al-Shedivat, Gert Cauwenberghs
Date: 2015-11-14 00:00:00

Recent studies have shown that synaptic unreliability is a robust and
sufficient mechanism for inducing the stochasticity observed in cortex. Here,
we introduce Synaptic Sampling Machines, a class of neural network models that
uses synaptic stochasticity as a means to Monte Carlo sampling and unsupervised
learning. Similar to the original formulation of Boltzmann machines, these
models can be viewed as a stochastic counterpart of Hopfield networks, but
where stochasticity is induced by a random mask over the connections. Synaptic
stochasticity plays the dual role of an efficient mechanism for sampling, and a
regularizer during learning akin to DropConnect. A local synaptic plasticity
rule implementing an event-driven form of contrastive divergence enables the
learning of generative models in an on-line fashion. Synaptic sampling machines
perform equally well using discrete-timed artificial units (as in Hopfield
networks) or continuous-timed leaky integrate & fire neurons. The learned
representations are remarkably sparse and robust to reductions in bit precision
and synapse pruning: removal of more than 75% of the weakest connections
followed by cursory re-learning causes a negligible performance loss on
benchmark classification tasks. The spiking neuron-based synaptic sampling
machines outperform existing spike-based unsupervised learners, while
potentially offering substantial advantages in terms of power and complexity,
and are thus promising models for on-line learning in brain-inspired hardware.
Dynamical Kinds and their Discovery
Authors: Benjamin C. Jantzen
Date: 2016-12-15 00:00:00
Tasks: scientific discovery

We demonstrate the possibility of classifying causal systems into kinds that
share a common structure without first constructing an explicit dynamical model
or using prior knowledge of the system dynamics. The algorithmic ability to
determine whether arbitrary systems are governed by causal relations of the
same form offers significant practical applications in the development and
validation of dynamical models. It is also of theoretical interest as an
essential stage in the scientific inference of laws from empirical data. The
algorithm presented is based on the dynamical symmetry approach to dynamical
kinds. A dynamical symmetry with respect to time is an intervention on one or
more variables of a system that commutes with the time evolution of the system.
A dynamical kind is a class of systems sharing a set of dynamical symmetries.
The algorithm presented classifies deterministic, time-dependent causal systems
by directly comparing their exhibited symmetries. Using simulated, noisy data
from a variety of nonlinear systems, we show that this algorithm correctly
sorts systems into dynamical kinds. It is robust under significant sampling
error, is immune to violations of normality in sampling error, and fails
gracefully with increasing dynamical similarity. The algorithm we demonstrate
is the first to address this aspect of automated scientific discovery.
Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network
Authors: Anh Tuan Tran, Tal Hassner, Iacopo Masi, Gerard Medioni
Date: 2016-12-15 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Face Reconstruction, Face Recognition, Face Reconstruction, Face Verification

The 3D shapes of faces are well known to be discriminative. Yet despite this,
they are rarely used for face recognition and always under controlled viewing
conditions. We claim that this is a symptom of a serious but often overlooked
problem with existing methods for single view 3D face reconstruction: when
applied "in the wild", their 3D estimates are either unstable and change for
different photos of the same subject or they are over-regularized and generic.
In response, we describe a robust method for regressing discriminative 3D
morphable face models (3DMM). We use a convolutional neural network (CNN) to
regress 3DMM shape and texture parameters directly from an input photo. We
overcome the shortage of training data required for this purpose by offering a
method for generating huge numbers of labeled examples. The 3D estimates
produced by our CNN surpass state of the art accuracy on the MICC data set.
Coupled with a 3D-3D face matching pipeline, we show the first competitive face
recognition results on the LFW, YTF and IJB-A benchmarks using 3D face shapes
as representations, rather than the opaque deep feature vectors used by other
modern systems.
Tinkering Under the Hood: Interactive Zero-Shot Learning with Net Surgery
Authors: Vivek Krishnan, Deva Ramanan
Date: 2016-12-15 00:00:00
Tasks: Novel Concepts, Zero-Shot Learning

We consider the task of visual net surgery, in which a CNN can be
reconfigured without extra data to recognize novel concepts that may be omitted
from the training set. While most prior work make use of linguistic cues for
such "zero-shot" learning, we do so by using a pictorial language
representation of the training set, implicitly learned by a CNN, to generalize
to new classes. To this end, we introduce a set of visualization techniques
that better reveal the activation patterns and relations between groups of CNN
filters. We next demonstrate that knowledge of pictorial languages can be used
to rewire certain CNN neurons into a part model, which we call a pictorial
language classifier. We demonstrate the robustness of simple PLCs by applying
them in a weakly supervised manner: labeling unlabeled concepts for visual
classes present in the training data. Specifically we show that a PLC built on
top of a CNN trained for ImageNet classification can localize humans in Graz-02
and determine the pose of birds in PASCAL-VOC without extra labeled data or
additional training. We then apply PLCs in an interactive zero-shot manner,
demonstrating that pictorial languages are expressive enough to detect a set of
visual classes in MS-COCO that never appear in the ImageNet training set.
Learning binary or real-valued time-series via spike-timing dependent plasticity
Authors: Takayuki Osogami
Date: 2016-12-15 00:00:00
Tasks: Time Series, Time Series Analysis

A dynamic Boltzmann machine (DyBM) has been proposed as a model of a spiking
neural network, and its learning rule of maximizing the log-likelihood of given
time-series has been shown to exhibit key properties of spike-timing dependent
plasticity (STDP), which had been postulated and experimentally confirmed in
the field of neuroscience as a learning rule that refines the Hebbian rule.
Here, we relax some of the constraints in the DyBM in a way that it becomes
more suitable for computation and learning. We show that learning the DyBM can
be considered as logistic regression for binary-valued time-series. We also
show how the DyBM can learn real-valued data in the form of a Gaussian DyBM and
discuss its relation to the vector autoregressive (VAR) model. The Gaussian
DyBM extends the VAR by using additional explanatory variables, which
correspond to the eligibility traces of the DyBM and capture long term
dependency of the time-series. Numerical experiments show that the Gaussian
DyBM significantly improves the predictive accuracy over VAR.
Projected Regression Methods for Inverting Fredholm Integrals: Formalism and Application to Analytical Continuation
Authors: Louis-Francois Arsenault, Richard Neuberg, Lauren A. Hannah, Andrew J. Millis
Date: 2016-12-15 00:00:00
Tasks: BIG-bench Machine Learning, regression

We present a machine learning approach to the inversion of Fredholm integrals
of the first kind. The approach provides a natural regularization in cases
where the inverse of the Fredholm kernel is ill-conditioned. It also provides
an efficient and stable treatment of constraints. The key observation is that
the stability of the forward problem permits the construction of a large
database of outputs for physically meaningful inputs. We apply machine learning
to this database to generate a regression function of controlled complexity,
which returns approximate solutions for previously unseen inputs; the
approximate solutions are then projected onto the subspace of functions
satisfying relevant constraints. We also derive and present uncertainty
estimates. We illustrate the approach by applying it to the analytical
continuation problem of quantum many-body physics, which involves
reconstructing the frequency dependence of physical excitation spectra from
data obtained at specific points in the complex frequency plane. Under standard
error metrics the method performs as well or better than the Maximum Entropy
method for low input noise and is substantially more robust to increased input
noise. We expect the methodology to be similarly effective for any problem
involving a formally ill-conditioned inversion, provided that the forward
problem can be efficiently solved.