pwc-archive/papers-with-abstracts huggingface dataset:
Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations
Authors: Diane Bouchacourt, Ryota Tomioka, Sebastian Nowozin
Date: 2017-05-24 00:00:00
Tasks: Disentanglement

We would like to learn a representation of the data which decomposes an
observation into factors of variation which we can independently control.
Specifically, we want to use minimal supervision to learn a latent
representation that reflects the semantics behind a specific grouping of the
data, where within a group the samples share a common factor of variation. For
example, consider a collection of face images grouped by identity. We wish to
anchor the semantics of the grouping into a relevant and disentangled
representation that we can easily exploit. However, existing deep probabilistic
models often assume that the observations are independent and identically
distributed. We present the Multi-Level Variational Autoencoder (ML-VAE), a new
deep probabilistic model for learning a disentangled representation of a set of
grouped observations. The ML-VAE separates the latent representation into
semantically meaningful parts by working both at the group level and the
observation level, while retaining efficient test-time inference. Quantitative
and qualitative evaluations show that the ML-VAE model (i) learns a
semantically meaningful disentanglement of grouped data, (ii) enables
manipulation of the latent representation, and (iii) generalises to unseen
groups.
Deep Investigation of Cross-Language Plagiarism Detection Methods
Authors: Jeremy Ferrero, Laurent Besacier, Didier Schwab, Frederic Agnes
Date: 2017-05-24 00:00:00
Proceeding: WS 2017 8

This paper is a deep investigation of cross-language plagiarism detection
methods on a new recently introduced open dataset, which contains parallel and
comparable collections of documents with multiple characteristics (different
genres, languages and sizes of texts). We investigate cross-language plagiarism
detection methods for 6 language pairs on 2 granularities of text units in
order to draw robust conclusions on the best methods while deeply analyzing
correlations across document styles and languages.
Power Systems Data Fusion based on Belief Propagation
Authors: Francesco Fusco, Seshu Tirupathi, Robert Gormally
Date: 2017-05-24 00:00:00
Tasks: State Estimation

The increasing complexity of the power grid, due to higher penetration of
distributed resources and the growing availability of interconnected,
distributed metering devices re- quires novel tools for providing a unified and
consistent view of the system. A computational framework for power systems data
fusion, based on probabilistic graphical models, capable of combining
heterogeneous data sources with classical state estimation nodes and other
customised computational nodes, is proposed. The framework allows flexible
extension of the notion of grid state beyond the view of flows and injection in
bus-branch models, and an efficient, naturally distributed inference algorithm
can be derived. An application of the data fusion model to the quantification
of distributed solar energy is proposed through numerical examples based on
semi-synthetic simulations of the standard IEEE 14-bus test case.
Boundary Crossing Probabilities for General Exponential Families
Authors: Odalric-Ambrym Maillard
Date: 2017-05-24 00:00:00
Tasks: Multi-Armed Bandits

We consider parametric exponential families of dimension $K$ on the real
line. We study a variant of \textit{boundary crossing probabilities} coming
from the multi-armed bandit literature, in the case when the real-valued
distributions form an exponential family of dimension $K$. Formally, our result
is a concentration inequality that bounds the probability that
$\mathcal{B}^\psi(\hat \theta_n,\theta^\star)\geq f(t/n)/n$, where
$\theta^\star$ is the parameter of an unknown target distribution, $\hat
\theta_n$ is the empirical parameter estimate built from $n$ observations,
$\psi$ is the log-partition function of the exponential family and
$\mathcal{B}^\psi$ is the corresponding Bregman divergence. From the
perspective of stochastic multi-armed bandits, we pay special attention to the
case when the boundary function $f$ is logarithmic, as it is enables to analyze
the regret of the state-of-the-art \KLUCB\ and \KLUCBp\ strategies, whose
analysis was left open in such generality. Indeed, previous results only hold
for the case when $K=1$, while we provide results for arbitrary finite
dimension $K$, thus considerably extending the existing results. Perhaps
surprisingly, we highlight that the proof techniques to achieve these strong
results already existed three decades ago in the work of T.L. Lai, and were
apparently forgotten in the bandit community. We provide a modern rewriting of
these beautiful techniques that we believe are useful beyond the application to
stochastic multi-armed bandits.
Loyalty in Online Communities
Authors: William L. Hamilton, Justine Zhang, Cristian Danescu-Niculescu-Mizil, Dan Jurafsky, Jure Leskovec
Date: 2017-03-09 00:00:00

Loyalty is an essential component of multi-community engagement. When users
have the choice to engage with a variety of different communities, they often
become loyal to just one, focusing on that community at the expense of others.
However, it is unclear how loyalty is manifested in user behavior, or whether
loyalty is encouraged by certain community characteristics.
  In this paper we operationalize loyalty as a user-community relation: users
loyal to a community consistently prefer it over all others; loyal communities
retain their loyal users over time. By exploring this relation using a large
dataset of discussion communities from Reddit, we reveal that loyalty is
manifested in remarkably consistent behaviors across a wide spectrum of
communities. Loyal users employ language that signals collective identity and
engage with more esoteric, less popular content, indicating they may play a
curational role in surfacing new material. Loyal communities have denser
user-user interaction networks and lower rates of triadic closure, suggesting
that community-level loyalty is associated with more cohesive interactions and
less fragmentation into subgroups. We exploit these general patterns to predict
future rates of loyalty. Our results show that a user's propensity to become
loyal is apparent from their first interactions with a community, suggesting
that some users are intrinsically loyal from the very beginning.
Learning Agents in Black-Scholes Financial Markets: Consensus Dynamics and Volatility Smiles
Authors: Tushar Vaidya, Carlos Murguia, Georgios Piliouras
Date: 2017-04-25 00:00:00

Black-Scholes (BS) is the standard mathematical model for option pricing in financial markets. Option prices are calculated using an analytical formula whose main inputs are strike (at which price to exercise) and volatility. The BS framework assumes that volatility remains constant across all strikes, however, in practice it varies. How do traders come to learn these parameters? We introduce natural models of learning agents, in which they update their beliefs about the true implied volatility based on the opinions of other traders. We prove convergence of these opinion dynamics using techniques from control theory and leader-follower models, thus providing a resolution between theory and market practices. We allow for two different models, one with feedback and one with an unknown leader.
An evolutionary strategy for DeltaE - E identification
Authors: Katarzyna Schmidt, Oskar Wyszynski
Date: 2017-05-23 00:00:00

In this article we present an automatic method for charge and mass
identification of charged nuclear fragments produced in heavy ion collisions at
intermediate energies. The algorithm combines a generative model of DeltaE - E
relation and a Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES). The
CMA-ES is a stochastic and derivative-free method employed to search parameter
space of the model by means of a fitness function. The article describes
details of the method along with results of an application on simulated labeled
data.
Adaptive Detrending to Accelerate Convolutional Gated Recurrent Unit Training for Contextual Video Recognition
Authors: Minju Jung, Haanvid Lee, Jun Tani
Date: 2017-05-24 00:00:00
Tasks: Video Recognition

Based on the progress of image recognition, video recognition has been
extensively studied recently. However, most of the existing methods are focused
on short-term but not long-term video recognition, called contextual video
recognition. To address contextual video recognition, we use convolutional
recurrent neural networks (ConvRNNs) having a rich spatio-temporal information
processing capability, but ConvRNNs requires extensive computation that slows
down training. In this paper, inspired by the normalization and detrending
methods, we propose adaptive detrending (AD) for temporal normalization in
order to accelerate the training of ConvRNNs, especially for convolutional
gated recurrent unit (ConvGRU). AD removes internal covariate shift within a
sequence of each neuron in recurrent neural networks (RNNs) by subtracting a
trend. In the experiments for contextual recognition on ConvGRU, the results
show that (1) ConvGRU clearly outperforms the feed-forward neural networks, (2)
AD consistently offers a significant training acceleration and generalization
improvement, and (3) AD is further improved by collaborating with the existing
normalization methods.
Bidirectional Beam Search: Forward-Backward Inference in Neural Sequence Models for Fill-in-the-Blank Image Captioning
Authors: Qing Sun, Stefan Lee, Dhruv Batra
Date: 2017-05-24 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Captioning, Sentence

We develop the first approximate inference algorithm for 1-Best (and M-Best)
decoding in bidirectional neural sequence models by extending Beam Search (BS)
to reason about both forward and backward time dependencies. Beam Search (BS)
is a widely used approximate inference algorithm for decoding sequences from
unidirectional neural sequence models. Interestingly, approximate inference in
bidirectional models remains an open problem, despite their significant
advantage in modeling information from both the past and future. To enable the
use of bidirectional models, we present Bidirectional Beam Search (BiBS), an
efficient algorithm for approximate bidirectional inference.To evaluate our
method and as an interesting problem in its own right, we introduce a novel
Fill-in-the-Blank Image Captioning task which requires reasoning about both
past and future sentence structure to reconstruct sensible image descriptions.
We use this task as well as the Visual Madlibs dataset to demonstrate the
effectiveness of our approach, consistently outperforming all baseline methods.
Learning Semantic Relatedness From Human Feedback Using Metric Learning
Authors: Thomas Niebler, Martin Becker, Christian Pölitz, Andreas Hotho
Date: 2017-05-21 00:00:00
Tasks: Metric Learning, Word Embeddings

Assessing the degree of semantic relatedness between words is an important
task with a variety of semantic applications, such as ontology learning for the
Semantic Web, semantic search or query expansion. To accomplish this in an
automated fashion, many relatedness measures have been proposed. However, most
of these metrics only encode information contained in the underlying corpus and
thus do not directly model human intuition. To solve this, we propose to
utilize a metric learning approach to improve existing semantic relatedness
measures by learning from additional information, such as explicit human
feedback. For this, we argue to use word embeddings instead of traditional
high-dimensional vector representations in order to leverage their semantic
density and to reduce computational cost. We rigorously test our approach on
several domains including tagging data as well as publicly available embeddings
based on Wikipedia texts and navigation. Human feedback about semantic
relatedness for learning and evaluation is extracted from publicly available
datasets such as MEN or WS-353. We find that our method can significantly
improve semantic relatedness measures by learning from additional information,
such as explicit human feedback. For tagging data, we are the first to generate
and study embeddings. Our results are of special interest for ontology and
recommendation engineers, but also for any other researchers and practitioners
of Semantic Web techniques.
Non-Stationary Spectral Kernels
Authors: Sami Remes, Markus Heinonen, Samuel Kaski
Date: 2017-05-24 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: regression, Time Series, Time Series Analysis

We propose non-stationary spectral kernels for Gaussian process regression.
We propose to model the spectral density of a non-stationary kernel function as
a mixture of input-dependent Gaussian process frequency density surfaces. We
solve the generalised Fourier transform with such a model, and present a family
of non-stationary and non-monotonic kernels that can learn input-dependent and
potentially long-range, non-monotonic covariances between inputs. We derive
efficient inference using model whitening and marginalized posterior, and show
with case studies that these kernels are necessary when modelling even rather
simple time series, image or geospatial data with non-stationary
characteristics.
Efficient Private ERM for Smooth Objectives
Authors: Jiaqi Zhang, Kai Zheng, Wenlong Mou, Li-Wei Wang
Date: 2017-03-29 00:00:00

In this paper, we consider efficient differentially private empirical risk
minimization from the viewpoint of optimization algorithms. For strongly convex
and smooth objectives, we prove that gradient descent with output perturbation
not only achieves nearly optimal utility, but also significantly improves the
running time of previous state-of-the-art private optimization algorithms, for
both $\epsilon$-DP and $(\epsilon, \delta)$-DP. For non-convex but smooth
objectives, we propose an RRPSGD (Random Round Private Stochastic Gradient
Descent) algorithm, which provably converges to a stationary point with privacy
guarantee. Besides the expected utility bounds, we also provide guarantees in
high probability form. Experiments demonstrate that our algorithm consistently
outperforms existing method in both utility and running time.
Compacting Neural Network Classifiers via Dropout Training
Authors: Yotaro Kubo, George Tucker, Simon Wiesler
Date: 2016-11-18 00:00:00
Tasks: speech-recognition, Speech Recognition

We introduce dropout compaction, a novel method for training feed-forward
neural networks which realizes the performance gains of training a large model
with dropout regularization, yet extracts a compact neural network for run-time
efficiency. In the proposed method, we introduce a sparsity-inducing prior on
the per unit dropout retention probability so that the optimizer can
effectively prune hidden units during training. By changing the prior
hyperparameters, we can control the size of the resulting network. We performed
a systematic comparison of dropout compaction and competing methods on several
real-world speech recognition tasks and found that dropout compaction achieved
comparable accuracy with fewer than 50% of the hidden units, translating to a
2.5x speedup in run-time.
An experimental study of graph-based semi-supervised classification with additional node information
Authors: Bertrand Lebichot, Marco Saerens
Date: 2017-05-24 00:00:00
Tasks: Classification, General Classification, Graph Embedding

The volume of data generated by internet and social networks is increasing
every day, and there is a clear need for efficient ways of extracting useful
information from them. As those data can take different forms, it is important
to use all the available data representations for prediction.
  In this paper, we focus our attention on supervised classification using both
regular plain, tabular, data and structural information coming from a network
structure. 14 techniques are investigated and compared in this study and can be
divided in three classes: the first one uses only the plain data to build a
classification model, the second uses only the graph structure and the last
uses both information sources. The relative performances in these three cases
are investigated. Furthermore, the effect of using a graph embedding and
well-known indicators in spatial statistics is also studied.
  Possible applications are automatic classification of web pages or other
linked documents, of people in a social network or of proteins in a biological
complex system, to name a few.
  Based on our comparison, we draw some general conclusions and advices to
tackle this particular classification task: some datasets can be better
explained by their graph structure (graph-driven), or by their feature set
(features-driven). The most efficient methods are discussed in both cases.
Inclusive Flavour Tagging Algorithm
Authors: Tatiana Likhomanenko, Denis Derkach, Alex Rogozhnikov
Date: 2017-05-24 00:00:00
Tasks: TAG

Identifying the flavour of neutral $B$ mesons production is one of the most
important components needed in the study of time-dependent $CP$ violation. The
harsh environment of the Large Hadron Collider makes it particularly hard to
succeed in this task. We present an inclusive flavour-tagging algorithm as an
upgrade of the algorithms currently used by the LHCb experiment. Specifically,
a probabilistic model which efficiently combines information from reconstructed
vertices and tracks using machine learning is proposed. The algorithm does not
use information about underlying physics process. It reduces the dependence on
the performance of lower level identification capacities and thus increases the
overall performance. The proposed inclusive flavour-tagging algorithm is
applicable to tag the flavour of $B$ mesons in any proton-proton experiment.
AdaGAN: Boosting Generative Models
Authors: Ilya Tolstikhin, Sylvain Gelly, Olivier Bousquet, Carl-Johann Simon-Gabriel, Bernhard Schölkopf
Date: 2017-01-09 00:00:00
Proceeding: NeurIPS 2017 12

Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) are an
effective method for training generative models of complex data such as natural
images. However, they are notoriously hard to train and can suffer from the
problem of missing modes where the model is not able to produce examples in
certain regions of the space. We propose an iterative procedure, called AdaGAN,
where at every step we add a new component into a mixture model by running a
GAN algorithm on a reweighted sample. This is inspired by boosting algorithms,
where many potentially weak individual predictors are greedily aggregated to
form a strong composite predictor. We prove that such an incremental procedure
leads to convergence to the true distribution in a finite number of steps if
each step is optimal, and convergence at an exponential rate otherwise. We also
illustrate experimentally that this procedure addresses the problem of missing
modes.
Stochastic Sequential Neural Networks with Structured Inference
Authors: Hao Liu, Haoli Bai, Lirong He, Zenglin Xu
Date: 2017-05-24 00:00:00
Tasks: Medical Diagnosis, Segmentation, State Space Models, Time Series, Time Series Analysis

Unsupervised structure learning in high-dimensional time series data has
attracted a lot of research interests. For example, segmenting and labelling
high dimensional time series can be helpful in behavior understanding and
medical diagnosis. Recent advances in generative sequential modeling have
suggested to combine recurrent neural networks with state space models (e.g.,
Hidden Markov Models). This combination can model not only the long term
dependency in sequential data, but also the uncertainty included in the hidden
states. Inheriting these advantages of stochastic neural sequential models, we
propose a structured and stochastic sequential neural network, which models
both the long-term dependencies via recurrent neural networks and the
uncertainty in the segmentation and labels via discrete random variables. For
accurate and efficient inference, we present a bi-directional inference network
by reparamterizing the categorical segmentation and labels with the recent
proposed Gumbel-Softmax approximation and resort to the Stochastic Gradient
Variational Bayes. We evaluate the proposed model in a number of tasks,
including speech modeling, automatic segmentation and labeling in behavior
understanding, and sequential multi-objects recognition. Experimental results
have demonstrated that our proposed model can achieve significant improvement
over the state-of-the-art methods.
Criticality as It Could Be: organizational invariance as self-organized criticality in embodied agents
Authors: Miguel Aguilera, Manuel G. Bedia
Date: 2017-04-18 00:00:00
Tasks: Acrobot, Reinforcement Learning

This paper outlines a methodological approach for designing adaptive agents
driving themselves near points of criticality. Using a synthetic approach we
construct a conceptual model that, instead of specifying mechanistic
requirements to generate criticality, exploits the maintenance of an
organizational structure capable of reproducing critical behavior. Our approach
exploits the well-known principle of universality, which classifies critical
phenomena inside a few universality classes of systems independently of their
specific mechanisms or topologies. In particular, we implement an artificial
embodied agent controlled by a neural network maintaining a correlation
structure randomly sampled from a lattice Ising model at a critical point. We
evaluate the agent in two classical reinforcement learning scenarios: the
Mountain Car benchmark and the Acrobot double pendulum, finding that in both
cases the neural controller reaches a point of criticality, which coincides
with a transition point between two regimes of the agent's behaviour,
maximizing the mutual information between neurons and sensorimotor patterns.
Finally, we discuss the possible applications of this synthetic approach to the
comprehension of deeper principles connected to the pervasive presence of
criticality in biological and cognitive systems.
Object Tracking based on Quantum Particle Swarm Optimization
Authors: Rajesh Misra, Kumar S. Ray
Date: 2017-05-24 00:00:00
Tasks: Image Retrieval, Object, Object Tracking, Visual Object Tracking

In Computer Vision domain, moving Object Tracking considered as one of the
toughest problem.As there so many factors associated like illumination of
light, noise, occlusion, sudden start and stop of moving object, shading which
makes tracking even harder problem not only for dynamic background but also for
static background.In this paper we present a new object tracking algorithm
based on Dominant points on tracked object using Quantum particle swarm
optimization (QPSO) which is a new different version of PSO based on Quantum
theory. The novelty in our approach is that it can be successfully applicable
in variable background as well as static background and application of quantum
PSO makes the algorithm runs lot faster where other basic PSO algorithm failed
to do so due to heavy computation.In our approach firstly dominants points of
tracked objects detected, then a group of particles form a swarm are
initialized randomly over the image search space and then start searching the
curvature connected between two consecutive dominant points until they satisfy
fitness criteria. Obviously it is a Multi-Swarm approach as there are multiple
dominant points, as they moves, the curvature moves and the curvature movement
is tracked by the swarm throughout the video and eventually when the swarm
reaches optimal solution , a bounding box drawn based on particles final
position.Experimental results demonstrate this proposed QPSO based method work
efficiently and effectively in visual object tracking in both dynamic and
static environments and run time shows that it runs closely 90% faster than
basic PSO.in our approach we also apply parallelism using MatLab Parfor command
to show how very less number of iteration and swarm size will enable us to
successfully track object.