pwc-archive/papers-with-abstracts huggingface dataset:
Bayesian Optimization for Probabilistic Programs
Authors: Tom Rainforth, Tuan Anh Le, Jan-Willem van de Meent, Michael A. Osborne, Frank Wood
Date: 2017-07-13 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Bayesian Optimization

We present the first general purpose framework for marginal maximum a
posteriori estimation of probabilistic program variables. By using a series of
code transformations, the evidence of any probabilistic program, and therefore
of any graphical model, can be optimized with respect to an arbitrary subset of
its sampled variables. To carry out this optimization, we develop the first
Bayesian optimization package to directly exploit the source code of its
target, leading to innovations in problem-independent hyperpriors, unbounded
optimization, and implicit constraint satisfaction; delivering significant
performance improvements over prominent existing packages. We present
applications of our method to a number of tasks including engineering design
and parameter optimization.
The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task
Authors: Roy Schwartz, Maarten Sap, Ioannis Konstas, Li Zilles, Yejin Choi, Noah A. Smith
Date: 2017-02-07 00:00:00
Proceeding: CONLL 2017 8
Tasks: Language Modeling, Language Modelling

A writer's style depends not just on personal traits but also on her intent
and mental state. In this paper, we show how variants of the same writing task
can lead to measurable differences in writing style. We present a case study
based on the story cloze task (Mostafazadeh et al., 2016a), where annotators
were assigned similar writing tasks with different constraints: (1) writing an
entire story, (2) adding a story ending for a given story context, and (3)
adding an incoherent ending to a story. We show that a simple linear classifier
informed by stylistic features is able to successfully distinguish among the
three cases, without even looking at the story context. In addition, combining
our stylistic features with language model predictions reaches state of the art
performance on the story cloze challenge. Our results demonstrate that
different task framings can dramatically affect the way people write.
Coalescent-based species tree estimation: a stochastic Farris transform
Authors: Gautam Dasarathy, Elchanan Mossel, Robert Nowak, Sebastien Roch
Date: 2017-07-13 00:00:00

The reconstruction of a species phylogeny from genomic data faces two
significant hurdles: 1) the trees describing the evolution of each individual
gene--i.e., the gene trees--may differ from the species phylogeny and 2) the
molecular sequences corresponding to each gene often provide limited
information about the gene trees themselves. In this paper we consider an
approach to species tree reconstruction that addresses both these hurdles.
Specifically, we propose an algorithm for phylogeny reconstruction under the
multispecies coalescent model with a standard model of site substitution. The
multispecies coalescent is commonly used to model gene tree discordance due to
incomplete lineage sorting, a well-studied population-genetic effect.
  In previous work, an information-theoretic trade-off was derived in this
context between the number of loci, $m$, needed for an accurate reconstruction
and the length of the locus sequences, $k$. It was shown that to reconstruct an
internal branch of length $f$, one needs $m$ to be of the order of $1/[f^{2}
\sqrt{k}]$. That previous result was obtained under the molecular clock
assumption, i.e., under the assumption that mutation rates (as well as
population sizes) are constant across the species phylogeny.
  Here we generalize this result beyond the restrictive molecular clock
assumption, and obtain a new reconstruction algorithm that has the same data
requirement (up to log factors). Our main contribution is a novel reduction to
the molecular clock case under the multispecies coalescent. As a corollary, we
also obtain a new identifiability result of independent interest: for any
species tree with $n \geq 3$ species, the rooted species tree can be identified
from the distribution of its unrooted weighted gene trees even in the absence
of a molecular clock.
Deep reinforcement learning from human preferences
Authors: Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, Dario Amodei
Date: 2017-06-12 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Atari Games, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

For sophisticated reinforcement learning (RL) systems to interact usefully with real-world environments, we need to communicate complex goals to these systems. In this work, we explore goals defined in terms of (non-expert) human preferences between pairs of trajectory segments. We show that this approach can effectively solve complex RL tasks without access to the reward function, including Atari games and simulated robot locomotion, while providing feedback on less than one percent of our agent's interactions with the environment. This reduces the cost of human oversight far enough that it can be practically applied to state-of-the-art RL systems. To demonstrate the flexibility of our approach, we show that we can successfully train complex novel behaviors with about an hour of human time. These behaviors and environments are considerably more complex than any that have been previously learned from human feedback.
Predicting Abandonment in Online Coding Tutorials
Authors: An Yan, Michael J. Lee, Andrew J. Ko
Date: 2017-07-13 00:00:00

Learners regularly abandon online coding tutorials when they get bored or
frustrated, but there are few techniques for anticipating this abandonment to
intervene. In this paper, we examine the feasibility of predicting abandonment
with machine-learned classifiers. Using interaction logs from an online
programming game, we extracted a collection of features that are potentially
related to learner abandonment and engagement, then developed classifiers for
each level. Across the first five levels of the game, our classifiers
successfully predicted 61% to 76% of learners who did not complete the next
level, achieving an average AUC of 0.68. In these classifiers, features
negatively associated with abandonment included account activation and
help-seeking behaviors, whereas features positively associated with abandonment
included features indicating difficulty and disengagement. These findings
highlight the feasibility of providing timely intervention to learners likely
to quit.
DeepSkeleton: Learning Multi-task Scale-associated Deep Side Outputs for Object Skeleton Extraction in Natural Images
Authors: Wei Shen, Kai Zhao, Yuan Jiang, Yan Wang, Xiang Bai, Alan Yuille
Date: 2016-09-13 00:00:00
Tasks: Multi-Task Learning, Object, object-detection, Object Detection, Semantic Segmentation

Object skeletons are useful for object representation and object detection.
They are complementary to the object contour, and provide extra information,
such as how object scale (thickness) varies among object parts. But object
skeleton extraction from natural images is very challenging, because it
requires the extractor to be able to capture both local and non-local image
context in order to determine the scale of each skeleton pixel. In this paper,
we present a novel fully convolutional network with multiple scale-associated
side outputs to address this problem. By observing the relationship between the
receptive field sizes of the different layers in the network and the skeleton
scales they can capture, we introduce two scale-associated side outputs to each
stage of the network. The network is trained by multi-task learning, where one
task is skeleton localization to classify whether a pixel is a skeleton pixel
or not, and the other is skeleton scale prediction to regress the scale of each
skeleton pixel. Supervision is imposed at different stages by guiding the
scale-associated side outputs toward the groundtruth skeletons at the
appropriate scales. The responses of the multiple scale-associated side outputs
are then fused in a scale-specific way to detect skeleton pixels using multiple
scales effectively. Our method achieves promising results on two skeleton
extraction datasets, and significantly outperforms other competitors.
Additionally, the usefulness of the obtained skeletons and scales (thickness)
are verified on two object detection applications: Foreground object
segmentation and object proposal detection.
A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers
Authors: Thomas E. Potok, Catherine Schuman, Steven R. Young, Robert M. Patton, Federico Spedalieri, Jeremy Liu, Ke-Thia Yao, Garrett Rose, Gangotree Chakma
Date: 2017-03-15 00:00:00
Tasks: Deep Learning, GPU

Current Deep Learning approaches have been very successful using
convolutional neural networks (CNN) trained on large graphical processing units
(GPU)-based computers. Three limitations of this approach are: 1) they are
based on a simple layered network topology, i.e., highly connected layers,
without intra-layer connections; 2) the networks are manually configured to
achieve optimal results, and 3) the implementation of neuron model is expensive
in both cost and power. In this paper, we evaluate deep learning models using
three different computing architectures to address these problems: quantum
computing to train complex topologies, high performance computing (HPC) to
automatically determine network topology, and neuromorphic computing for a
low-power hardware implementation. We use the MNIST dataset for our experiment,
due to input size limitations of current quantum computers. Our results show
the feasibility of using the three architectures in tandem to address the above
deep learning limitations. We show a quantum computer can find high quality
values of intra-layer connections weights, in a tractable time as the
complexity of the network increases; a high performance computer can find
optimal layer-based topologies; and a neuromorphic computer can represent the
complex topology and weights derived from the other architectures in low power
memristive hardware.
On (Anti)Conditional Independence in Dempster-Shafer Theory
Authors: Mieczysław A. Kłopotek
Date: 2017-07-13 00:00:00

This paper verifies a result of {Shenoy:94} concerning graphoidal structure
of Shenoy's notion of independence for Dempster-Shafer theory of belief
functions. Shenoy proved that his notion of independence has graphoidal
properties for positive normal valuations.
  The requirement of strict positive normal valuations as prerequisite for
application of graphoidal properties excludes a wide class of DS belief
functions. It excludes especially so-called probabilistic belief functions. It
is demonstrated that the requirement of positiveness of valuation may be
weakened in that it may be required that commonality function is non-zero for
singleton sets instead, and the graphoidal properties for independence of
belief function variables are then preserved. This means especially that
probabilistic belief functions with all singleton sets as focal points possess
graphoidal properties for independence.
Comparative Study of Inference Methods for Bayesian Nonnegative Matrix Factorisation
Authors: Thomas Brouwer, Jes Frellsen, Pietro Lió
Date: 2017-07-13 00:00:00
Tasks: Bayesian Inference, Missing Values, Model Selection

In this paper, we study the trade-offs of different inference approaches for
Bayesian matrix factorisation methods, which are commonly used for predicting
missing values, and for finding patterns in the data. In particular, we
consider Bayesian nonnegative variants of matrix factorisation and
tri-factorisation, and compare non-probabilistic inference, Gibbs sampling,
variational Bayesian inference, and a maximum-a-posteriori approach. The
variational approach is new for the Bayesian nonnegative models. We compare
their convergence, and robustness to noise and sparsity of the data, on both
synthetic and real-world datasets. Furthermore, we extend the models with the
Bayesian automatic relevance determination prior, allowing the models to
perform automatic model selection, and demonstrate its efficiency.
Superposition de calques monochromes d'opacités variables
Authors: Alexandre Bali
Date: 2017-07-13 00:00:00

For a monochrome layer $x$ of opacity $0\le o_x\le1 $ placed on another monochrome layer of opacity 1, the result given by the standard formula is $$\small\Pi\left({\bf C}_\varphi\right)=1+\sum_{n=1}^2\left(2-n-(-1)^no_{\chi(\varphi+1)}\right)\left(\chi(n+\varphi-1)-o_{\chi(n+\varphi-1)}\right),$$ the formula being of course explained in detail in this paper. We will eventually deduce a very simple theorem, generalize it and then see its validity with alternative formulas to this standard containing the same main properties here exposed.
Fast Restricted Causal Inference
Authors: Mieczysław A. Kłopotek
Date: 2017-07-13 00:00:00
Tasks: Causal Inference

Hidden variables are well known sources of disturbance when recovering belief
networks from data based only on measurable variables. Hence models assuming
existence of hidden variables are under development.
  This paper presents a new algorithm "accelerating" the known CI algorithm of
Spirtes, Glymour and Scheines {Spirtes:93}. We prove that this algorithm does
not produces (conditional) independencies not present in the data if
statistical independence test is reliable.
  This result is to be considered as non-trivial since e.g. the same claim
fails to be true for FCI algorithm, another "accelerator" of CI, developed in
{Spirtes:93}.
Cultivating DNN Diversity for Large Scale Video Labelling
Authors: Mikel Bober-Irizar, Sameed Husain, Eng-Jon Ong, Miroslaw Bober
Date: 2017-07-13 00:00:00
Tasks: Diversity, Video Understanding

We investigate factors controlling DNN diversity in the context of the Google
Cloud and YouTube-8M Video Understanding Challenge. While it is well-known that
ensemble methods improve prediction performance, and that combining accurate
but diverse predictors helps, there is little knowledge on how to best promote
& measure DNN diversity. We show that diversity can be cultivated by some
unexpected means, such as model over-fitting or dropout variations. We also
present details of our solution to the video understanding problem, which
ranked #7 in the Kaggle competition (competing as the Yeti team).
Lithium NLP: A System for Rich Information Extraction from Noisy User Generated Text on Social Media
Authors: Preeti Bhargava, Nemanja Spasojevic, Guoning Hu
Date: 2017-07-13 00:00:00
Proceeding: WS 2017 9

In this paper, we describe the Lithium Natural Language Processing (NLP)
system - a resource-constrained, high- throughput and language-agnostic system
for information extraction from noisy user generated text on social media.
Lithium NLP extracts a rich set of information including entities, topics,
hashtags and sentiment from text. We discuss several real world applications of
the system currently incorporated in Lithium products. We also compare our
system with existing commercial and academic NLP systems in terms of
performance, information extracted and languages supported. We show that
Lithium NLP is at par with and in some cases, outperforms state- of-the-art
commercial NLP systems.
Neural Networks for Information Retrieval
Authors: Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
Date: 2017-07-13 00:00:00
Tasks: Information Retrieval, Retrieval

Machine learning plays a role in many aspects of modern IR systems, and deep
learning is applied in all of them. The fast pace of modern-day research has
given rise to many different approaches for many different IR problems. The
amount of information available can be overwhelming both for junior students
and for experienced researchers looking for new research topics and directions.
Additionally, it is interesting to see what key insights into IR problems the
new technologies are able to give us. The aim of this full-day tutorial is to
give a clear overview of current tried-and-trusted neural methods in IR and how
they benefit IR research. It covers key architectures, as well as the most
promising future directions.
Improving Sparsity in Kernel Adaptive Filters Using a Unit-Norm Dictionary
Authors: Felipe Tobar
Date: 2017-07-13 00:00:00
Tasks: Time Series, Time Series Analysis

Kernel adaptive filters, a class of adaptive nonlinear time-series models,
are known by their ability to learn expressive autoregressive patterns from
sequential data. However, for trivial monotonic signals, they struggle to
perform accurate predictions and at the same time keep computational complexity
within desired boundaries. This is because new observations are incorporated to
the dictionary when they are far from what the algorithm has seen in the past.
We propose a novel approach to kernel adaptive filtering that compares new
observations against dictionary samples in terms of their unit-norm
(normalised) versions, meaning that new observations that look like previous
samples but have a different magnitude are not added to the dictionary. We
achieve this by proposing the unit-norm Gaussian kernel and define a
sparsification criterion for this novel kernel. This new methodology is
validated on two real-world datasets against standard KAF in terms of the
normalised mean square error and the dictionary size.
Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
Authors: Shayegan Omidshafiei, Jason Pazis, Christopher Amato, Jonathan P. How, John Vian
Date: 2017-03-17 00:00:00
Proceeding: ICML 2017 8
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Many real-world tasks involve multiple agents with partial observability and
limited communication. Learning is challenging in these settings due to local
viewpoints of agents, which perceive the world as non-stationary due to
concurrently-exploring teammates. Approaches that learn specialized policies
for individual tasks face problems when applied to the real world: not only do
agents have to learn and store distinct policies for each task, but in practice
identities of tasks are often non-observable, making these approaches
inapplicable. This paper formalizes and addresses the problem of multi-task
multi-agent reinforcement learning under partial observability. We introduce a
decentralized single-task learning approach that is robust to concurrent
interactions of teammates, and present an approach for distilling single-task
policies into a unified policy that performs well across multiple related
tasks, without explicit provision of task identity.
Parsing with Traces: An $O(n^4)$ Algorithm and a Structural Representation
Authors: Jonathan K. Kummerfeld, Dan Klein
Date: 2017-07-13 00:00:00
Tasks: Constituency Parsing, Missing Elements

General treebank analyses are graph structured, but parsers are typically
restricted to tree structures for efficiency and modeling reasons. We propose a
new representation and algorithm for a class of graph structures that is
flexible enough to cover almost all treebank structures, while still admitting
efficient learning and inference. In particular, we consider directed, acyclic,
one-endpoint-crossing graph structures, which cover most long-distance
dislocation, shared argumentation, and similar tree-violating linguistic
phenomena. We describe how to convert phrase structure parses, including
traces, to our new representation in a reversible manner. Our dynamic program
uniquely decomposes structures, is sound and complete, and covers 97.3% of the
Penn English Treebank. We also implement a proof-of-concept parser that
recovers a range of null elements and trace types.
Learning Features from Co-occurrences: A Theoretical Analysis
Authors: Yanpeng Li
Date: 2017-07-13 00:00:00
Proceeding: COLING 2018 8

Representing a word by its co-occurrences with other words in context is an
effective way to capture the meaning of the word. However, the theory behind
remains a challenge. In this work, taking the example of a word classification
task, we give a theoretical analysis of the approaches that represent a word X
by a function f(P(C|X)), where C is a context feature, P(C|X) is the
conditional probability estimated from a text corpus, and the function f maps
the co-occurrence measure to a prediction score. We investigate the impact of
context feature C and the function f. We also explain the reasons why using the
co-occurrences with multiple context features may be better than just using a
single one. In addition, some of the results shed light on the theory of
feature learning and machine learning in general.
Semantic Image Inpainting with Deep Generative Models
Authors: Raymond A. Yeh, Chen Chen, Teck Yian Lim, Alexander G. Schwing, Mark Hasegawa-Johnson, Minh N. Do
Date: 2016-07-26 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Inpainting

Semantic image inpainting is a challenging task where large missing regions
have to be filled based on the available visual data. Existing methods which
extract information from only a single image generally produce unsatisfactory
results due to the lack of high level context. In this paper, we propose a
novel method for semantic image inpainting, which generates the missing content
by conditioning on the available data. Given a trained generative model, we
search for the closest encoding of the corrupted image in the latent image
manifold using our context and prior losses. This encoding is then passed
through the generative model to infer the missing content. In our method,
inference is possible irrespective of how the missing content is structured,
while the state-of-the-art learning based method requires specific information
about the holes in the training phase. Experiments on three datasets show that
our method successfully predicts information in large missing regions and
achieves pixel-level photorealism, significantly outperforming the
state-of-the-art methods.
On Optimality Conditions for Auto-Encoder Signal Recovery
Authors: Devansh Arpit, Yingbo Zhou, Hung Q. Ngo, Nils Napp, Venu Govindaraju
Date: 2016-05-23 00:00:00
Proceeding: ICLR 2018 1
Tasks: compressed sensing

Auto-Encoders are unsupervised models that aim to learn patterns from
observed data by minimizing a reconstruction cost. The useful representations
learned are often found to be sparse and distributed. On the other hand,
compressed sensing and sparse coding assume a data generating process, where
the observed data is generated from some true latent signal source, and try to
recover the corresponding signal from measurements. Looking at auto-encoders
from this \textit{signal recovery perspective} enables us to have a more
coherent view of these techniques. In this paper, in particular, we show that
the \textit{true} hidden representation can be approximately recovered if the
weight matrices are highly incoherent with unit $ \ell^{2} $ row length and the
bias vectors takes the value (approximately) equal to the negative of the data
mean. The recovery also becomes more and more accurate as the sparsity in
hidden signals increases. Additionally, we empirically demonstrate that
auto-encoders are capable of recovering the data generating dictionary when
only data samples are given.