pwc-archive/papers-with-abstracts huggingface dataset:
Computational Optimal Transport
Authors: Gabriel Peyré, Marco Cuturi
Date: 2018-03-01 00:00:00
Tasks: MORPH, Sand

Optimal transport (OT) theory can be informally described using the words of the French mathematician Gaspard Monge (1746-1818): A worker with a shovel in hand has to move a large pile of sand lying on a construction site. The goal of the worker is to erect with all that sand a target pile with a prescribed shape (for example, that of a giant sand castle). Naturally, the worker wishes to minimize her total effort, quantified for instance as the total distance or time spent carrying shovelfuls of sand. Mathematicians interested in OT cast that problem as that of comparing two probability distributions, two different piles of sand of the same volume. They consider all of the many possible ways to morph, transport or reshape the first pile into the second, and associate a "global" cost to every such transport, using the "local" consideration of how much it costs to move a grain of sand from one place to another. Recent years have witnessed the spread of OT in several fields, thanks to the emergence of approximate solvers that can scale to sizes and dimensions that are relevant to data sciences. Thanks to this newfound scalability, OT is being increasingly used to unlock various problems in imaging sciences (such as color or texture processing), computer vision and graphics (for shape manipulation) or machine learning (for regression, classification and density fitting). This short book reviews OT with a bias toward numerical methods and their applications in data sciences, and sheds lights on the theoretical properties of OT that make it particularly useful for some of these applications.
The Alpha-Beta-Symetric Divergence and their Positive Definite Kernel
Authors: Mactar Ndaw, Macoumba Ndour, Papa Ngom
Date: 2018-03-01 00:00:00
Tasks: General Classification, image-classification, Image Classification

In this article we study the field of Hilbertian metrics and positive definit
(pd) kernels on probability measures, they have a real interest in kernel
methods. Firstly we will make a study based on the Alpha-Beta-divergence to
have a Hilbercan metric by proposing an improvement of this divergence by
constructing it so that its is symmetrical the Alpha-Beta-Symmetric-divergence
(ABS-divergence) and also do some studies on these properties but also propose
the kernels associated with this divergence. Secondly we will do mumerical
studies incorporating all proposed metrics/kernels into support vector machine
(SVM). Finally we presented a algorithm for image classification by using our
divergence.
The 2017 DAVIS Challenge on Video Object Segmentation
Authors: Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbeláez, Alex Sorkine-Hornung, Luc van Gool
Date: 2017-04-03 00:00:00
Tasks: Object, Scene Classification, Segmentation, Semantic Segmentation, Video Object Segmentation, Video Segmentation, Video Semantic Segmentation

We present the 2017 DAVIS Challenge on Video Object Segmentation, a public
dataset, benchmark, and competition specifically designed for the task of video
object segmentation. Following the footsteps of other successful initiatives,
such as ILSVRC and PASCAL VOC, which established the avenue of research in the
fields of scene classification and semantic segmentation, the DAVIS Challenge
comprises a dataset, an evaluation methodology, and a public competition with a
dedicated workshop co-located with CVPR 2017. The DAVIS Challenge follows up on
the recent publication of DAVIS (Densely-Annotated VIdeo Segmentation), which
has fostered the development of several novel state-of-the-art video object
segmentation techniques. In this paper we describe the scope of the benchmark,
highlight the main characteristics of the dataset, define the evaluation
metrics of the competition, and present a detailed analysis of the results of
the participants to the challenge.
Toward an Efficient Multi-class Classification in an Open Universe
Authors: Wajdi Dhifli, Abdoulaye Baniré Diallo
Date: 2015-11-02 00:00:00
Tasks: Classification, General Classification, Multi-class Classification, open-set classification, Open Set Learning

Classification is a fundamental task in machine learning and data mining.
Existing classification methods are designed to classify unknown instances
within a set of previously known training classes. Such a classification takes
the form of a prediction within a closed-set of classes. However, a more
realistic scenario that fits real-world applications is to consider the
possibility of encountering instances that do not belong to any of the training
classes, $i.e.$, an open-set classification. In such situation, existing
closed-set classifiers will assign a training label to these instances
resulting in a misclassification. In this paper, we introduce Galaxy-X, a novel
multi-class classification approach for open-set recognition problems. For each
class of the training set, Galaxy-X creates a minimum bounding hyper-sphere
that encompasses the distribution of the class by enclosing all of its
instances. In such manner, our method is able to distinguish instances
resembling previously seen classes from those that are of unknown ones. To
adequately evaluate open-set classification, we introduce a novel evaluation
procedure. Experimental results on benchmark datasets show the efficiency of
our approach in classifying novel instances from known as well as unknown
classes.
Composable Planning with Attributes
Authors: Amy Zhang, Adam Lerer, Sainbayar Sukhbaatar, Rob Fergus, Arthur Szlam
Date: 2018-03-01 00:00:00
Proceeding: ICML 2018 7
Tasks: Attribute, Starcraft

The tasks that an agent will need to solve often are not known during
training. However, if the agent knows which properties of the environment are
important then, after learning how its actions affect those properties, it may
be able to use this knowledge to solve complex tasks without training
specifically for them. Towards this end, we consider a setup in which an
environment is augmented with a set of user defined attributes that
parameterize the features of interest. We propose a method that learns a policy
for transitioning between "nearby" sets of attributes, and maintains a graph of
possible transitions. Given a task at test time that can be expressed in terms
of a target set of attributes, and a current state, our model infers the
attributes of the current state and searches over paths through attribute space
to get a high level plan, and then uses its low level policy to execute the
plan. We show in 3D block stacking, grid-world games, and StarCraft that our
model is able to generalize to longer, more complex tasks at test time by
composing simpler learned policies.
Out-distribution training confers robustness to deep neural networks
Authors: Mahdieh Abbasi, Christian Gagné
Date: 2018-02-20 00:00:00

The easiness at which adversarial instances can be generated in deep neural
networks raises some fundamental questions on their functioning and concerns on
their use in critical systems. In this paper, we draw a connection between
over-generalization and adversaries: a possible cause of adversaries lies in
models designed to make decisions all over the input space, leading to
inappropriate high-confidence decisions in parts of the input space not
represented in the training set. We empirically show an augmented neural
network, which is not trained on any types of adversaries, can increase the
robustness by detecting black-box one-step adversaries, i.e. assimilated to
out-distribution samples, and making generation of white-box one-step
adversaries harder.
DeepProbe: Information Directed Sequence Understanding and Chatbot Design via Recurrent Neural Networks
Authors: Zi Yin, Keng-hao Chang, Ruofei Zhang
Date: 2017-07-18 00:00:00
Tasks: Chatbot, Recommendation Systems

Information extraction and user intention identification are central topics
in modern query understanding and recommendation systems. In this paper, we
propose DeepProbe, a generic information-directed interaction framework which
is built around an attention-based sequence to sequence (seq2seq) recurrent
neural network. DeepProbe can rephrase, evaluate, and even actively ask
questions, leveraging the generative ability and likelihood estimation made
possible by seq2seq models. DeepProbe makes decisions based on a derived
uncertainty (entropy) measure conditioned on user inputs, possibly with
multiple rounds of interactions. Three applications, namely a rewritter, a
relevance scorer and a chatbot for ad recommendation, were built around
DeepProbe, with the first two serving as precursory building blocks for the
third. We first use the seq2seq model in DeepProbe to rewrite a user query into
one of standard query form, which is submitted to an ordinary recommendation
system. Secondly, we evaluate DeepProbe's seq2seq model-based relevance
scoring. Finally, we build a chatbot prototype capable of making active user
interactions, which can ask questions that maximize information gain, allowing
for a more efficient user intention idenfication process. We evaluate first two
applications by 1) comparing with baselines by BLEU and AUC, and 2) human judge
evaluation. Both demonstrate significant improvements compared with current
state-of-the-art systems, proving their values as useful tools on their own,
and at the same time laying a good foundation for the ongoing chatbot
application.
Unifying DAGs and UGs
Authors: Jose M. Peña
Date: 2017-08-29 00:00:00

We introduce a new class of graphical models that generalizes
Lauritzen-Wermuth-Frydenberg chain graphs by relaxing the semi-directed
acyclity constraint so that only directed cycles are forbidden. Moreover, up to
two edges are allowed between any pair of nodes. Specifically, we present
local, pairwise and global Markov properties for the new graphical models and
prove their equivalence. We also present an equivalent factorization property.
Finally, we present a causal interpretation of the new models.
The Power Mean Laplacian for Multilayer Graph Clustering
Authors: Pedro Mercado, Antoine Gautier, Francesco Tudisco, Matthias Hein
Date: 2018-03-01 00:00:00
Tasks: Clustering, Graph Clustering, Stochastic Block Model

Multilayer graphs encode different kind of interactions between the same set
of entities. When one wants to cluster such a multilayer graph, the natural
question arises how one should merge the information different layers. We
introduce in this paper a one-parameter family of matrix power means for
merging the Laplacians from different layers and analyze it in expectation in
the stochastic block model. We show that this family allows to recover ground
truth clusters under different settings and verify this in real world data.
While computing the matrix power mean can be very expensive for large graphs,
we introduce a numerical scheme to efficiently compute its eigenvectors for the
case of large sparse graphs.
Adding Context to Concept Trees
Authors: Kieran Greer
Date: 2016-06-17 00:00:00

A Concept Tree is a structure for storing knowledge where the trees are stored in a database called a Concept Base. It sits between the highly distributed neural architectures and the distributed information systems, with the intention of bringing brain-like and computer systems closer together. Concept Trees can grow from the semi-structured sources when consistent sequences of concepts are presented. Each tree ideally represents a single cohesive concept and the trees can link with each other for navigation and semantic purposes. The trees are therefore also a type of semantic network and would benefit from having a consistent level of context for each node. A consistent build process is managed through a 'counting rule' and some other rules that can normalise the database structure. This restricted structure can then be complimented and enriched by the more dynamic context. It is also suggested to use the linking structure of the licas system [15] as a basis for the context links, where the mathematical model is extended further to define this. A number of tests have demonstrated the soundness of the architecture. Building the trees from text documents shows that the tree structure could be inherent in natural language. Then, two types of query language are described. Both of these can perform consistent query processes to return knowledge to the user and even enhance the query with new knowledge. This is supported even further with direct comparisons to a cognitive model, also being developed by the author.
LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images
Authors: Gregory Rogez, Philippe Weinzaepfel, Cordelia Schmid
Date: 2018-03-01 00:00:00
Tasks: 3D Human Pose Estimation, 3D Multi-Person Pose Estimation (absolute), 3D Pose Estimation, Pose Estimation

We propose an end-to-end architecture for joint 2D and 3D human pose
estimation in natural images. Key to our approach is the generation and scoring
of a number of pose proposals per image, which allows us to predict 2D and 3D
poses of multiple people simultaneously. Hence, our approach does not require
an approximate localization of the humans for initialization. Our
Localization-Classification-Regression architecture, named LCR-Net, contains 3
main components: 1) the pose proposal generator that suggests candidate poses
at different locations in the image; 2) a classifier that scores the different
pose proposals; and 3) a regressor that refines pose proposals both in 2D and
3D. All three stages share the convolutional feature layers and are trained
jointly. The final pose estimation is obtained by integrating over neighboring
pose hypotheses, which is shown to improve over a standard non maximum
suppression algorithm. Our method recovers full-body 2D and 3D poses,
hallucinating plausible body parts when the persons are partially occluded or
truncated by the image boundary. Our approach significantly outperforms the
state of the art in 3D pose estimation on Human3.6M, a controlled environment.
Moreover, it shows promising results on real images for both single and
multi-person subsets of the MPII 2D pose benchmark and demonstrates satisfying
3D pose results even for multi-person images.
Optimal localist and distributed coding of spatiotemporal spike patterns through STDP and coincidence detection
Authors: Timothée Masquelier, Saeed Reza Kheradpisheh
Date: 2018-03-01 00:00:00

Repeating spatiotemporal spike patterns exist and carry information. Here we
investigated how a single spiking neuron can optimally respond to one given
pattern (localist coding), or to either one of several patterns (distributed
coding, i.e. the neuron's response is ambiguous but the identity of the pattern
could be inferred from the response of multiple neurons), but not to random
inputs. To do so, we extended a theory developed in a previous paper
[Masquelier, 2017], which was limited to localist coding. More specifically, we
computed analytically the signal-to-noise ratio (SNR) of a
multi-pattern-detector neuron, using a threshold-free leaky integrate-and-fire
(LIF) neuron model with non-plastic unitary synapses and homogeneous Poisson
inputs. Surprisingly, when increasing the number of patterns, the SNR decreases
slowly, and remains acceptable for several tens of independent patterns.
  In addition, we investigated whether spike-timing-dependent plasticity (STDP)
could enable a neuron to reach the theoretical optimal SNR. To this aim, we
simulated a LIF equipped with STDP, and repeatedly exposed it to multiple input
spike patterns, embedded in equally dense Poisson spike trains. The LIF
progressively became selective to every repeating pattern with no supervision,
and stopped discharging during the Poisson spike trains. Furthermore, using
certain STDP parameters, the resulting pattern detectors were optimal. Tens of
independent patterns could be learned by a single neuron using a low adaptive
threshold, in contrast with previous studies, in which higher thresholds led to
localist coding only.
  Taken together these results suggest that coincidence detection and STDP are
powerful mechanisms, fully compatible with distributed coding. Yet we
acknowledge that our theory is limited to single neurons, and thus also applies
to feed-forward networks, but not to recurrent ones.
Inferring Missing Categorical Information in Noisy and Sparse Web Markup
Authors: Nicolas Tempelmeier, Elena Demidova, Stefan Dietze
Date: 2018-03-01 00:00:00
Tasks: Diversity

Embedded markup of Web pages has seen widespread adoption throughout the past
years driven by standards such as RDFa and Microdata and initiatives such as
schema.org, where recent studies show an adoption by 39% of all Web pages
already in 2016. While this constitutes an important information source for
tasks such as Web search, Web page classification or knowledge graph
augmentation, individual markup nodes are usually sparsely described and often
lack essential information. For instance, from 26 million nodes describing
events within the Common Crawl in 2016, 59% of nodes provide less than six
statements and only 257,000 nodes (0.96%) are typed with more specific event
subtypes. Nevertheless, given the scale and diversity of Web markup data, nodes
that provide missing information can be obtained from the Web in large
quantities, in particular for categorical properties. Such data constitutes
potential training data for inferring missing information to significantly
augment sparsely described nodes. In this work, we introduce a supervised
approach for inferring missing categorical properties in Web markup. Our
experiments, conducted on properties of events and movies, show a performance
of 79% and 83% F1 score correspondingly, significantly outperforming existing
baselines.
Inverse Reinforcement Learning via Nonparametric Spatio-Temporal Subgoal Modeling
Authors: Adrian Šošić, Elmar Rueckert, Jan Peters, Abdelhak M. Zoubir, Heinz Koeppl
Date: 2018-03-01 00:00:00
Tasks: Active Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Advances in the field of inverse reinforcement learning (IRL) have led to
sophisticated inference frameworks that relax the original modeling assumption
of observing an agent behavior that reflects only a single intention. Instead
of learning a global behavioral model, recent IRL methods divide the
demonstration data into parts, to account for the fact that different
trajectories may correspond to different intentions, e.g., because they were
generated by different domain experts. In this work, we go one step further:
using the intuitive concept of subgoals, we build upon the premise that even a
single trajectory can be explained more efficiently locally within a certain
context than globally, enabling a more compact representation of the observed
behavior. Based on this assumption, we build an implicit intentional model of
the agent's goals to forecast its behavior in unobserved situations. The result
is an integrated Bayesian prediction framework that significantly outperforms
existing IRL solutions and provides smooth policy estimates consistent with the
expert's plan. Most notably, our framework naturally handles situations where
the intentions of the agent change over time and classical IRL algorithms fail.
In addition, due to its probabilistic nature, the model can be
straightforwardly applied in active learning scenarios to guide the
demonstration process of the expert.
Knowledge Transfer with Jacobian Matching
Authors: Suraj Srinivas, Francois Fleuret
Date: 2018-03-01 00:00:00
Proceeding: ICML 2018 7
Tasks: Transfer Learning

Classical distillation methods transfer representations from a "teacher"
neural network to a "student" network by matching their output activations.
Recent methods also match the Jacobians, or the gradient of output activations
with the input. However, this involves making some ad hoc decisions, in
particular, the choice of the loss function.
  In this paper, we first establish an equivalence between Jacobian matching
and distillation with input noise, from which we derive appropriate loss
functions for Jacobian matching. We then rely on this analysis to apply
Jacobian matching to transfer learning by establishing equivalence of a recent
transfer learning procedure to distillation.
  We then show experimentally on standard image datasets that Jacobian-based
penalties improve distillation, robustness to noisy inputs, and transfer
learning.
MaskGAN: Better Text Generation via Filling in the______
Authors: William Fedus, Ian Goodfellow, Andrew M. Dai
Date: 2018-01-23 00:00:00
Tasks: Multivariate Time Series Imputation, Text Generation

Neural text generation models are often autoregressive language models or
seq2seq models. These models generate text by sampling words sequentially, with
each word conditioned on the previous word, and are state-of-the-art for
several machine translation and summarization benchmarks. These benchmarks are
often defined by validation perplexity even though this is not a direct measure
of the quality of the generated text. Additionally, these models are typically
trained via maxi- mum likelihood and teacher forcing. These methods are
well-suited to optimizing perplexity but can result in poor sample quality
since generating text requires conditioning on sequences of words that may have
never been observed at training time. We propose to improve sample quality
using Generative Adversarial Networks (GANs), which explicitly train the
generator to produce high quality samples and have shown a lot of success in
image generation. GANs were originally designed to output differentiable
values, so discrete language generation is challenging for them. We claim that
validation perplexity alone is not indicative of the quality of text generated
by a model. We introduce an actor-critic conditional GAN that fills in missing
text conditioned on the surrounding context. We show qualitatively and
quantitatively, evidence that this produces more realistic conditional and
unconditional text samples compared to a maximum likelihood trained model.
Stochastic Training of Graph Convolutional Networks with Variance Reduction
Authors: Jianfei Chen, Jun Zhu, Le Song
Date: 2017-10-29 00:00:00
Proceeding: ICML 2018 7

Graph convolutional networks (GCNs) are powerful deep neural networks for
graph-structured data. However, GCN computes the representation of a node
recursively from its neighbors, making the receptive field size grow
exponentially with the number of layers. Previous attempts on reducing the
receptive field size by subsampling neighbors do not have a convergence
guarantee, and their receptive field size per node is still in the order of
hundreds. In this paper, we develop control variate based algorithms which
allow sampling an arbitrarily small neighbor size. Furthermore, we prove new
theoretical guarantee for our algorithms to converge to a local optimum of GCN.
Empirical results show that our algorithms enjoy a similar convergence with the
exact algorithm using only two neighbors per node. The runtime of our
algorithms on a large Reddit dataset is only one seventh of previous neighbor
sampling algorithms.