pwc-archive/papers-with-abstracts huggingface dataset:
Semidefinite Programs for Exact Recovery of a Hidden Community
Authors: Bruce Hajek, Yihong Wu, Jiaming Xu
Date: 2016-02-20 00:00:00
Tasks: Stochastic Block Model

We study a semidefinite programming (SDP) relaxation of the maximum
likelihood estimation for exactly recovering a hidden community of cardinality
$K$ from an $n \times n$ symmetric data matrix $A$, where for distinct indices
$i,j$, $A_{ij} \sim P$ if $i, j$ are both in the community and $A_{ij} \sim Q$
otherwise, for two known probability distributions $P$ and $Q$. We identify a
sufficient condition and a necessary condition for the success of SDP for the
general model. For both the Bernoulli case ($P={{\rm Bern}}(p)$ and $Q={{\rm
Bern}}(q)$ with $p>q$) and the Gaussian case ($P=\mathcal{N}(\mu,1)$ and
$Q=\mathcal{N}(0,1)$ with $\mu>0$), which correspond to the problem of planted
dense subgraph recovery and submatrix localization respectively, the general
results lead to the following findings: (1) If $K=\omega( n /\log n)$, SDP
attains the information-theoretic recovery limits with sharp constants; (2) If
$K=\Theta(n/\log n)$, SDP is order-wise optimal, but strictly suboptimal by a
constant factor; (3) If $K=o(n/\log n)$ and $K \to \infty$, SDP is order-wise
suboptimal. The same critical scaling for $K$ is found to hold, up to constant
factors, for the performance of SDP on the stochastic block model of $n$
vertices partitioned into multiple communities of equal size $K$. A key
ingredient in the proof of the necessary condition is a construction of a
primal feasible solution based on random perturbation of the true cluster
matrix.
Improving Neural Machine Translation Models with Monolingual Data
Authors: Rico Sennrich, Barry Haddow, Alexandra Birch
Date: 2015-11-20 00:00:00
Proceeding: ACL 2016 8
Tasks: Cross-Lingual Bitext Mining, Decoder, Language Modeling, Language Modelling, Machine Translation, NMT, Translation

Neural Machine Translation (NMT) has obtained state-of-the art performance
for several language pairs, while only using parallel data for training.
Target-side monolingual data plays an important role in boosting fluency for
phrase-based statistical machine translation, and we investigate the use of
monolingual data for NMT. In contrast to previous work, which combines NMT
models with separately trained language models, we note that encoder-decoder
NMT architectures already have the capacity to learn the same information as a
language model, and we explore strategies to train with monolingual data
without changing the neural network architecture. By pairing monolingual
training data with an automatic back-translation, we can treat it as additional
parallel training data, and we obtain substantial improvements on the WMT 15
task English<->German (+2.8-3.7 BLEU), and for the low-resourced IWSLT 14 task
Turkish->English (+2.1-3.4 BLEU), obtaining new state-of-the-art results. We
also show that fine-tuning on in-domain monolingual and parallel data gives
substantial improvements for the IWSLT 15 task English->German.
The belief noisy-or model applied to network reliability analysis
Authors: Kuang Zhou, Arnaud Martin, Quan Pan
Date: 2016-06-03 00:00:00

One difficulty faced in knowledge engineering for Bayesian Network (BN) is
the quan-tification step where the Conditional Probability Tables (CPTs) are
determined. The number of parameters included in CPTs increases exponentially
with the number of parent variables. The most common solution is the
application of the so-called canonical gates. The Noisy-OR (NOR) gate, which
takes advantage of the independence of causal interactions, provides a
logarithmic reduction of the number of parameters required to specify a CPT. In
this paper, an extension of NOR model based on the theory of belief functions,
named Belief Noisy-OR (BNOR), is proposed. BNOR is capable of dealing with both
aleatory and epistemic uncertainty of the network. Compared with NOR, more rich
information which is of great value for making decisions can be got when the
available knowledge is uncertain. Specially, when there is no epistemic
uncertainty, BNOR degrades into NOR. Additionally, different structures of BNOR
are presented in this paper in order to meet various needs of engineers. The
application of BNOR model on the reliability evaluation problem of networked
systems demonstrates its effectiveness.
Degrees of Freedom in Deep Neural Networks
Authors: Tianxiang Gao, Vladimir Jojic
Date: 2016-03-30 00:00:00
Tasks: General Classification, Multi-class Classification

In this paper, we explore degrees of freedom in deep sigmoidal neural
networks. We show that the degrees of freedom in these models is related to the
expected optimism, which is the expected difference between test error and
training error. We provide an efficient Monte-Carlo method to estimate the
degrees of freedom for multi-class classification methods. We show degrees of
freedom are lower than the parameter count in a simple XOR network. We extend
these results to neural nets trained on synthetic and real data, and
investigate impact of network's architecture and different regularization
choices. The degrees of freedom in deep networks are dramatically smaller than
the number of parameters, in some real datasets several orders of magnitude.
Further, we observe that for fixed number of parameters, deeper networks have
less degrees of freedom exhibiting a regularization-by-depth.
ECMdd: Evidential c-medoids clustering with multiple prototypes
Authors: Kuang Zhou, Arnaud Martin, Quan Pan, Zhun-Ga Liu
Date: 2016-06-03 00:00:00
Tasks: Clustering

In this work, a new prototype-based clustering method named Evidential
C-Medoids (ECMdd), which belongs to the family of medoid-based clustering for
proximity data, is proposed as an extension of Fuzzy C-Medoids (FCMdd) on the
theoretical framework of belief functions. In the application of FCMdd and
original ECMdd, a single medoid (prototype), which is supposed to belong to the
object set, is utilized to represent one class. For the sake of clarity, this
kind of ECMdd using a single medoid is denoted by sECMdd. In real clustering
applications, using only one pattern to capture or interpret a class may not
adequately model different types of group structure and hence limits the
clustering performance. In order to address this problem, a variation of ECMdd
using multiple weighted medoids, denoted by wECMdd, is presented. Unlike
sECMdd, in wECMdd objects in each cluster carry various weights describing
their degree of representativeness for that class. This mechanism enables each
class to be represented by more than one object. Experimental results in
synthetic and real data sets clearly demonstrate the superiority of sECMdd and
wECMdd. Moreover, the clustering results by wECMdd can provide richer
information for the inner structure of the detected classes with the help of
prototype weights.
Learning under Distributed Weak Supervision
Authors: Martin Rajchl, Matthew C. H. Lee, Franklin Schrans, Alice Davidson, Jonathan Passerat-Palmbach, Giacomo Tarroni, Amir Alansary, Ozan Oktay, Bernhard Kainz, Daniel Rueckert
Date: 2016-06-03 00:00:00
Tasks: Brain Segmentation, Medical Image Analysis

The availability of training data for supervision is a frequently encountered
bottleneck of medical image analysis methods. While typically established by a
clinical expert rater, the increase in acquired imaging data renders
traditional pixel-wise segmentations less feasible. In this paper, we examine
the use of a crowdsourcing platform for the distribution of super-pixel weak
annotation tasks and collect such annotations from a crowd of non-expert
raters. The crowd annotations are subsequently used for training a fully
convolutional neural network to address the problem of fetal brain segmentation
in T2-weighted MR images. Using this approach we report encouraging results
compared to highly targeted, fully supervised methods and potentially address a
frequent problem impeding image analysis research.
Beyond CCA: Moment Matching for Multi-View Models
Authors: Anastasia Podosinnikova, Francis Bach, Simon Lacoste-Julien
Date: 2016-02-29 00:00:00

We introduce three novel semi-parametric extensions of probabilistic
canonical correlation analysis with identifiability guarantees. We consider
moment matching techniques for estimation in these models. For that, by drawing
explicit links between the new models and a discrete version of independent
component analysis (DICA), we first extend the DICA cumulant tensors to the new
discrete version of CCA. By further using a close connection with independent
component analysis, we introduce generalized covariance matrices, which can
replace the cumulant tensors in the moment matching framework, and, therefore,
improve sample complexity and simplify derivations and algorithms
significantly. As the tensor power method or orthogonal joint diagonalization
are not applicable in the new setting, we use non-orthogonal joint
diagonalization techniques for matching the cumulants. We demonstrate
performance of the proposed models and estimation techniques on experiments
with both synthetic and real datasets.
Compositional Sentence Representation from Character within Large Context Text
Authors: Geonmin Kim, Hwaran Lee, Jisu Choi, Soo-Young Lee
Date: 2016-05-02 00:00:00
Tasks: Dialogue Act Classification, General Classification, Sentence

This paper describes a Hierarchical Composition Recurrent Network (HCRN)
consisting of a 3-level hierarchy of compositional models: character, word and
sentence. This model is designed to overcome two problems of representing a
sentence on the basis of a constituent word sequence. The first is a
data-sparsity problem in word embedding, and the other is a no usage of
inter-sentence dependency. In the HCRN, word representations are built from
characters, thus resolving the data-sparsity problem, and inter-sentence
dependency is embedded into sentence representation at the level of sentence
composition. We adopt a hierarchy-wise learning scheme in order to alleviate
the optimization difficulties of learning deep hierarchical recurrent network
in end-to-end fashion. The HCRN was quantitatively and qualitatively evaluated
on a dialogue act classification task. Especially, sentence representations
with an inter-sentence dependency are able to capture both implicit and
explicit semantics of sentence, significantly improving performance. In the
end, the HCRN achieved state-of-the-art performance with a test error rate of
22.7% for dialogue act classification on the SWBD-DAMSL database.
Resolving References to Objects in Photographs using the Words-As-Classifiers Model
Authors: David Schlangen, Sina Zarriess, Casey Kennington
Date: 2015-10-07 00:00:00
Proceeding: ACL 2016 8

A common use of language is to refer to visually present objects. Modelling
it in computers requires modelling the link between language and perception.
The "words as classifiers" model of grounded semantics views words as
classifiers of perceptual contexts, and composes the meaning of a phrase
through composition of the denotations of its component words. It was recently
shown to perform well in a game-playing scenario with a small number of object
types. We apply it to two large sets of real-world photographs that contain a
much larger variety of types and for which referring expressions are available.
Using a pre-trained convolutional neural network to extract image features, and
augmenting these with in-picture positional information, we show that the model
achieves performance competitive with the state of the art in a reference
resolution task (given expression, find bounding box of its referent), while,
as we argue, being conceptually simpler and more flexible.
Thompson Sampling is Asymptotically Optimal in General Environments
Authors: Jan Leike, Tor Lattimore, Laurent Orseau, Marcus Hutter
Date: 2016-02-25 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Thompson Sampling

We discuss a variant of Thompson sampling for nonparametric reinforcement
learning in a countable classes of general stochastic environments. These
environments can be non-Markov, non-ergodic, and partially observable. We show
that Thompson sampling learns the environment class in the sense that (1)
asymptotically its value converges to the optimal value in mean and (2) given a
recoverability assumption regret is sublinear.
Machine Learning for E-mail Spam Filtering: Review,Techniques and Trends
Authors: Alexy Bhowmick, Shyamanta M. Hazarika
Date: 2016-06-03 00:00:00
Tasks: BIG-bench Machine Learning

We present a comprehensive review of the most effective content-based e-mail
spam filtering techniques. We focus primarily on Machine Learning-based spam
filters and their variants, and report on a broad review ranging from surveying
the relevant ideas, efforts, effectiveness, and the current progress. The
initial exposition of the background examines the basics of e-mail spam
filtering, the evolving nature of spam, spammers playing cat-and-mouse with
e-mail service providers (ESPs), and the Machine Learning front in fighting
spam. We conclude by measuring the impact of Machine Learning-based filters and
explore the promising offshoots of latest developments.
Group Equivariant Convolutional Networks
Authors: Taco S. Cohen, Max Welling
Date: 2016-02-24 00:00:00
Tasks: Breast Tumour Classification, Colorectal Gland Segmentation:, Multi-tissue Nucleus Segmentation, Rotated MNIST

We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a
natural generalization of convolutional neural networks that reduces sample
complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of
layer that enjoys a substantially higher degree of weight sharing than regular
convolution layers. G-convolutions increase the expressive capacity of the
network without increasing the number of parameters. Group convolution layers
are easy to use and can be implemented with negligible computational overhead
for discrete groups generated by translations, reflections and rotations.
G-CNNs achieve state of the art results on CIFAR10 and rotated MNIST.
Estimating a sharp convergence bound for randomized ensembles
Authors: Miles E. Lopes
Date: 2013-03-04 00:00:00
Tasks: Binary Classification, Density Estimation, Prediction

When randomized ensembles such as bagging or random forests are used for
binary classification, the prediction error of the ensemble tends to decrease
and stabilize as the number of classifiers increases. However, the precise
relationship between prediction error and ensemble size is unknown in practice.
In the standard case when classifiers are aggregated by majority vote, the
present work offers a way to quantify this convergence in terms of "algorithmic
variance," i.e. the variance of prediction error due only to the randomized
training algorithm. Specifically, we study a theoretical upper bound on this
variance, and show that it is sharp --- in the sense that it is attained by a
specific family of randomized classifiers. Next, we address the problem of
estimating the unknown value of the bound, which leads to a unique twist on the
classical problem of non-parametric density estimation. In particular, we
develop an estimator for the bound and show that its MSE matches optimal
non-parametric rates under certain conditions. (Concurrent with this work, some
closely related results have also been considered in Cannings and Samworth
(2017) and Lopes (2019).)
Exploit Bounding Box Annotations for Multi-label Object Recognition
Authors: Hao Yang, Joey Tianyi Zhou, Yu Zhang, Bin-Bin Gao, Jianxin Wu, Jianfei Cai
Date: 2015-04-22 00:00:00
Proceeding: CVPR 2016 6
Tasks: Multi-Label Classification, Object, Object Recognition

Convolutional neural networks (CNNs) have shown great performance as general
feature representations for object recognition applications. However, for
multi-label images that contain multiple objects from different categories,
scales and locations, global CNN features are not optimal. In this paper, we
incorporate local information to enhance the feature discriminative power. In
particular, we first extract object proposals from each image. With each image
treated as a bag and object proposals extracted from it treated as instances,
we transform the multi-label recognition problem into a multi-class
multi-instance learning problem. Then, in addition to extracting the typical
CNN feature representation from each proposal, we propose to make use of
ground-truth bounding box annotations (strong labels) to add another level of
local information by using nearest-neighbor relationships of local regions to
form a multi-view pipeline. The proposed multi-view multi-instance framework
utilizes both weak and strong labels effectively, and more importantly it has
the generalization ability to even boost the performance of unseen categories
by partial strong labels from other categories. Our framework is extensively
compared with state-of-the-art hand-crafted feature based methods and CNN based
methods on two multi-label benchmark datasets. The experimental results
validate the discriminative power and the generalization ability of the
proposed framework. With strong labels, our framework is able to achieve
state-of-the-art results in both datasets.
Selecting the Best Player Formation for Corner-Kick Situations Based on Bayes' Estimation
Authors: Jordan Henrio, Thomas Henn, Tomoharu Nakashima, Hidehisa Akiyama
Date: 2016-06-03 00:00:00

In the domain of the Soccer simulation 2D league of the RoboCup project,
appropriate player positioning against a given opponent team is an important
factor of soccer team performance. This work proposes a model which decides the
strategy that should be applied regarding a particular opponent team. This task
can be realized by applying preliminary a learning phase where the model
determines the most effective strategies against clusters of opponent teams.
The model determines the best strategies by using sequential Bayes' estimators.
As a first trial of the system, the proposed model is used to determine the
association of player formations against opponent teams in the particular
situation of corner-kick. The implemented model shows satisfying abilities to
compare player formations that are similar to each other in terms of
performance and determines the right ranking even by running a decent number of
simulation games.
Optically lightweight tracking of objects around a corner
Authors: Jonathan Klein, Christoph Peters, Jaime Martín, Martin Laurenzis, Matthias B. Hullin
Date: 2016-06-03 00:00:00
Tasks: Translation

The observation of objects located in inaccessible regions is a recurring
challenge in a wide variety of important applications. Recent work has shown
that indirect diffuse light reflections can be used to reconstruct objects and
two-dimensional (2D) patterns around a corner. However, these prior methods
always require some specialized setup involving either ultrafast detectors or
narrowband light sources. Here we show that occluded objects can be tracked in
real time using a standard 2D camera and a laser pointer. Unlike previous
methods based on the backprojection approach, we formulate the problem in an
analysis-by-synthesis sense. By repeatedly simulating light transport through
the scene, we determine the set of object parameters that most closely fits the
measured intensity distribution. We experimentally demonstrate that this
approach is capable of following the translation of unknown objects, and
translation and orientation of a known object, in real time.
On Recognizing Transparent Objects in Domestic Environments Using Fusion of Multiple Sensor Modalities
Authors: Alexander Hagg, Frederik Hegger, Paul Plöger
Date: 2016-06-03 00:00:00
Tasks: Object, Object Recognition, Transparent objects

Current object recognition methods fail on object sets that include both
diffuse, reflective and transparent materials, although they are very common in
domestic scenarios. We show that a combination of cues from multiple sensor
modalities, including specular reflectance and unavailable depth information,
allows us to capture a larger subset of household objects by extending a state
of the art object recognition method. This leads to a significant increase in
robustness of recognition over a larger set of commonly used objects.
A Graph-Based Semi-Supervised k Nearest-Neighbor Method for Nonlinear Manifold Distributed Data Classification
Authors: Enmei Tu, Yaqian Zhang, Lin Zhu, Jie Yang, Nikola Kasabov
Date: 2016-06-03 00:00:00
Tasks: General Classification

$k$ Nearest Neighbors ($k$NN) is one of the most widely used supervised
learning algorithms to classify Gaussian distributed data, but it does not
achieve good results when it is applied to nonlinear manifold distributed data,
especially when a very limited amount of labeled samples are available. In this
paper, we propose a new graph-based $k$NN algorithm which can effectively
handle both Gaussian distributed data and nonlinear manifold distributed data.
To achieve this goal, we first propose a constrained Tired Random Walk (TRW) by
constructing an $R$-level nearest-neighbor strengthened tree over the graph,
and then compute a TRW matrix for similarity measurement purposes. After this,
the nearest neighbors are identified according to the TRW matrix and the class
label of a query point is determined by the sum of all the TRW weights of its
nearest neighbors. To deal with online situations, we also propose a new
algorithm to handle sequential samples based a local neighborhood
reconstruction. Comparison experiments are conducted on both synthetic data
sets and real-world data sets to demonstrate the validity of the proposed new
$k$NN algorithm and its improvements to other version of $k$NN algorithms.
Given the widespread appearance of manifold structures in real-world problems
and the popularity of the traditional $k$NN algorithm, the proposed manifold
version $k$NN shows promising potential for classifying manifold-distributed
data.
Question Answering over Knowledge Base with Neural Attention Combining Global Knowledge Information
Authors: Yuanzhe Zhang, Kang Liu, Shizhu He, Guoliang Ji, Zhanyi Liu, Hua Wu, Jun Zhao
Date: 2016-06-03 00:00:00
Tasks: Question Answering

With the rapid growth of knowledge bases (KBs) on the web, how to take full
advantage of them becomes increasingly important. Knowledge base-based question
answering (KB-QA) is one of the most promising approaches to access the
substantial knowledge. Meantime, as the neural network-based (NN-based) methods
develop, NN-based KB-QA has already achieved impressive results. However,
previous work did not put emphasis on question representation, and the question
is converted into a fixed vector regardless of its candidate answers. This
simple representation strategy is unable to express the proper information of
the question. Hence, we present a neural attention-based model to represent the
questions dynamically according to the different focuses of various candidate
answer aspects. In addition, we leverage the global knowledge inside the
underlying KB, aiming at integrating the rich KB information into the
representation of the answers. And it also alleviates the out of vocabulary
(OOV) problem, which helps the attention model to represent the question more
precisely. The experimental results on WEBQUESTIONS demonstrate the
effectiveness of the proposed approach.