pwc-archive/papers-with-abstracts huggingface dataset:
Toward Implicit Sample Noise Modeling: Deviation-driven Matrix Factorization
Authors: Guang-He Lee, Shao-Wen Yang, Shou-De Lin
Date: 2016-10-28 00:00:00

The objective function of a matrix factorization model usually aims to
minimize the average of a regression error contributed by each element.
However, given the existence of stochastic noises, the implicit deviations of
sample data from their true values are almost surely diverse, which makes each
data point not equally suitable for fitting a model. In this case, simply
averaging the cost among data in the objective function is not ideal.
Intuitively we would like to emphasize more on the reliable instances (i.e.,
those contain smaller noise) while training a model. Motivated by such
observation, we derive our formula from a theoretical framework for optimal
weighting under heteroscedastic noise distribution. Specifically, by modeling
and learning the deviation of data, we design a novel matrix factorization
model. Our model has two advantages. First, it jointly learns the deviation and
conducts dynamic reweighting of instances, allowing the model to converge to a
better solution. Second, during learning the deviated instances are assigned
lower weights, which leads to faster convergence since the model does not need
to overfit the noise. The experiments are conducted in clean recommendation and
noisy sensor datasets to test the effectiveness of the model in various
scenarios. The results show that our model outperforms the state-of-the-art
factorization and deep learning models in both accuracy and efficiency.
Hierarchical Clustering via Spreading Metrics
Authors: Aurko Roy, Sebastian Pokutta
Date: 2016-10-28 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Clustering, graph partitioning

We study the cost function for hierarchical clusterings introduced by
[arXiv:1510.05043] where hierarchies are treated as first-class objects rather
than deriving their cost from projections into flat clusters. It was also shown
in [arXiv:1510.05043] that a top-down algorithm returns a hierarchical
clustering of cost at most $O\left(\alpha_n \log n\right)$ times the cost of
the optimal hierarchical clustering, where $\alpha_n$ is the approximation
ratio of the Sparsest Cut subroutine used. Thus using the best known
approximation algorithm for Sparsest Cut due to Arora-Rao-Vazirani, the top
down algorithm returns a hierarchical clustering of cost at most
$O\left(\log^{3/2} n\right)$ times the cost of the optimal solution. We improve
this by giving an $O(\log{n})$-approximation algorithm for this problem. Our
main technical ingredients are a combinatorial characterization of ultrametrics
induced by this cost function, deriving an Integer Linear Programming (ILP)
formulation for this family of ultrametrics, and showing how to iteratively
round an LP relaxation of this formulation by using the idea of \emph{sphere
growing} which has been extensively used in the context of graph partitioning.
We also prove that our algorithm returns an $O(\log{n})$-approximate
hierarchical clustering for a generalization of this cost function also studied
in [arXiv:1510.05043]. Experiments show that the hierarchies found by using the
ILP formulation as well as our rounding algorithm often have better projections
into flat clusters than the standard linkage based algorithms. We also give
constant factor inapproximability results for this problem.
Byte-based Language Identification with Deep Convolutional Networks
Authors: Johannes Bjerva
Date: 2016-09-28 00:00:00
Proceeding: WS 2016 12
Tasks: Language Identification

We report on our system for the shared task on discriminating between similar
languages (DSL 2016). The system uses only byte representations in a deep
residual network (ResNet). The system, named ResIdent, is trained only on the
data released with the task (closed training). We obtain 84.88% accuracy on
subtask A, 68.80% accuracy on subtask B1, and 69.80% accuracy on subtask B2. A
large difference in accuracy on development data can be observed with
relatively minor changes in our network's architecture and hyperparameters. We
therefore expect fine-tuning of these parameters to yield higher accuracies.
Learnable Visual Markers
Authors: Oleg Grinchuk, Vadim Lebedev, Victor Lempitsky
Date: 2016-10-28 00:00:00
Proceeding: NeurIPS 2016 12

We propose a new approach to designing visual markers (analogous to QR-codes,
markers for augmented reality, and robotic fiducial tags) based on the advances
in deep generative networks. In our approach, the markers are obtained as color
images synthesized by a deep network from input bit strings, whereas another
deep network is trained to recover the bit strings back from the photos of
these markers. The two networks are trained simultaneously in a joint
backpropagation process that takes characteristic photometric and geometric
distortions associated with marker fabrication and marker scanning into
account. Additionally, a stylization loss based on statistics of activations in
a pretrained classification network can be inserted into the learning in order
to shift the marker appearance towards some texture prototype. In the
experiments, we demonstrate that the markers obtained using our approach are
capable of retaining bit strings that are long enough to be practical. The
ability to automatically adapt markers according to the usage scenario and the
desired capacity as well as the ability to combine information encoding with
artistic stylization are the unique properties of our approach. As a byproduct,
our approach provides an insight on the structure of patterns that are most
suitable for recognition by ConvNets and on their ability to distinguish
composite patterns.
Text Segmentation using Named Entity Recognition and Co-reference Resolution in English and Greek Texts
Authors: Pavlina Fragkou
Date: 2016-10-28 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Segmentation, Text Segmentation

In this paper we examine the benefit of performing named entity recognition
(NER) and co-reference resolution to an English and a Greek corpus used for
text segmentation. The aim here is to examine whether the combination of text
segmentation and information extraction can be beneficial for the
identification of the various topics that appear in a document. NER was
performed manually in the English corpus and was compared with the output
produced by publicly available annotation tools while, an already existing tool
was used for the Greek corpus. Produced annotations from both corpora were
manually corrected and enriched to cover four types of named entities.
Co-reference resolution i.e., substitution of every reference of the same
instance with the same named entity identifier was subsequently performed. The
evaluation, using five text segmentation algorithms for the English corpus and
four for the Greek corpus leads to the conclusion that, the benefit highly
depends on the segment's topic, the number of named entity instances appearing
in it, as well as the segment's length.
Sentiment Analysis of Twitter Data for Predicting Stock Market Movements
Authors: Venkata Sasank Pagolu, Kamal Nayan Reddy Challa, Ganapati Panda, Babita Majhi
Date: 2016-10-28 00:00:00
Tasks: Sentiment Analysis, Stock Market Prediction

Predicting stock market movements is a well-known problem of interest.
Now-a-days social media is perfectly representing the public sentiment and
opinion about current events. Especially, twitter has attracted a lot of
attention from researchers for studying the public sentiments. Stock market
prediction on the basis of public sentiments expressed on twitter has been an
intriguing field of research. Previous studies have concluded that the
aggregate public mood collected from twitter may well be correlated with Dow
Jones Industrial Average Index (DJIA). The thesis of this work is to observe
how well the changes in stock prices of a company, the rises and falls, are
correlated with the public opinions being expressed in tweets about that
company. Understanding author's opinion from a piece of text is the objective
of sentiment analysis. The present paper have employed two different textual
representations, Word2vec and N-gram, for analyzing the public sentiments in
tweets. In this paper, we have applied sentiment analysis and supervised
machine learning principles to the tweets extracted from twitter and analyze
the correlation between stock market movements of a company and sentiments in
tweets. In an elaborate way, positive news and tweets in social media about a
company would definitely encourage people to invest in the stocks of that
company and as a result the stock price of that company would increase. At the
end of the paper, it is shown that a strong correlation exists between the rise
and falls in stock prices with the public sentiments in tweets.
Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning
Authors: Gang Niu, Marthinus Christoffel du Plessis, Tomoya Sakai, Yao Ma, Masashi Sugiyama
Date: 2016-03-10 00:00:00
Proceeding: NeurIPS 2016 12

In PU learning, a binary classifier is trained from positive (P) and
unlabeled (U) data without negative (N) data. Although N data is missing, it
sometimes outperforms PN learning (i.e., ordinary supervised learning).
Hitherto, neither theoretical nor experimental analysis has been given to
explain this phenomenon. In this paper, we theoretically compare PU (and NU)
learning against PN learning based on the upper bounds on estimation errors. We
find simple conditions when PU and NU learning are likely to outperform PN
learning, and we prove that, in terms of the upper bounds, either PU or NU
learning (depending on the class-prior probability and the sizes of P and N
data) given infinite U data will improve on PN learning. Our theoretical
findings well agree with the experimental results on artificial and benchmark
data even when the experimental setup does not match the theoretical
assumptions exactly.
MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild
Authors: Grégory Rogez, Cordelia Schmid
Date: 2016-07-07 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: 3D Human Pose Estimation, 3D Pose Estimation, Data Augmentation, Pose Estimation

This paper addresses the problem of 3D human pose estimation in the wild. A
significant challenge is the lack of training data, i.e., 2D images of humans
annotated with 3D poses. Such data is necessary to train state-of-the-art CNN
architectures. Here, we propose a solution to generate a large set of
photorealistic synthetic images of humans with 3D pose annotations. We
introduce an image-based synthesis engine that artificially augments a dataset
of real images with 2D human pose annotations using 3D Motion Capture (MoCap)
data. Given a candidate 3D pose our algorithm selects for each joint an image
whose 2D pose locally matches the projected 3D pose. The selected images are
then combined to generate a new synthetic image by stitching local image
patches in a kinematically constrained manner. The resulting images are used to
train an end-to-end CNN for full-body 3D pose estimation. We cluster the
training data into a large number of pose classes and tackle pose estimation as
a K-way classification problem. Such an approach is viable only with large
training sets such as ours. Our method outperforms the state of the art in
terms of 3D pose estimation in controlled environments (Human3.6M) and shows
promising results for in-the-wild images (LSP). This demonstrates that CNNs
trained on artificial images generalize well to real images.
Discriminative Gaifman Models
Authors: Mathias Niepert
Date: 2016-10-28 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: BIG-bench Machine Learning, Link Prediction, Relational Reasoning

We present discriminative Gaifman models, a novel family of relational
machine learning models. Gaifman models learn feature representations bottom up
from representations of locally connected and bounded-size regions of knowledge
bases (KBs). Considering local and bounded-size neighborhoods of knowledge
bases renders logical inference and learning tractable, mitigates the problem
of overfitting, and facilitates weight sharing. Gaifman models sample
neighborhoods of knowledge bases so as to make the learned relational models
more robust to missing objects and relations which is a common situation in
open-world KBs. We present the core ideas of Gaifman models and apply them to
large-scale relational learning problems. We also discuss the ways in which
Gaifman models relate to some existing relational machine learning approaches.
DISCO Nets: DISsimilarity COefficient Networks
Authors: Diane Bouchacourt, M. Pawan Kumar, Sebastian Nowozin
Date: 2016-06-08 00:00:00

We present a new type of probabilistic model which we call DISsimilarity
COefficient Networks (DISCO Nets). DISCO Nets allow us to efficiently sample
from a posterior distribution parametrised by a neural network. During
training, DISCO Nets are learned by minimising the dissimilarity coefficient
between the true distribution and the estimated distribution. This allows us to
tailor the training to the loss related to the task at hand. We empirically
show that (i) by modeling uncertainty on the output value, DISCO Nets
outperform equivalent non-probabilistic predictive networks and (ii) DISCO Nets
accurately model the uncertainty of the output, outperforming existing
probabilistic models based on deep neural networks.
Interpretable Distribution Features with Maximum Testing Power
Authors: Wittawat Jitkrittum, Zoltan Szabo, Kacper Chwialkowski, Arthur Gretton
Date: 2016-05-22 00:00:00
Proceeding: NeurIPS 2016 12

Two semimetrics on probability distributions are proposed, given as the sum
of differences of expectations of analytic functions evaluated at spatial or
frequency locations (i.e, features). The features are chosen so as to maximize
the distinguishability of the distributions, by optimizing a lower bound on
test power for a statistical test using these features. The result is a
parsimonious and interpretable indication of how and where two distributions
differ locally. An empirical estimate of the test power criterion converges
with increasing sample size, ensuring the quality of the returned features. In
real-world benchmarks on high-dimensional text and image data, linear-time
tests using the proposed semimetrics achieve comparable performance to the
state-of-the-art quadratic-time maximum mean discrepancy test, while returning
human-interpretable features that explain the test results.
Towards a continuous modeling of natural language domains
Authors: Sebastian Ruder, Parsa Ghaffari, John G. Breslin
Date: 2016-10-28 00:00:00
Proceeding: WS 2016 11
Tasks: Domain Adaptation, Language Modeling, Language Modelling, Navigate, Representation Learning

Humans continuously adapt their style and language to a variety of domains.
However, a reliable definition of `domain' has eluded researchers thus far.
Additionally, the notion of discrete domains stands in contrast to the
multiplicity of heterogeneous domains that humans navigate, many of which
overlap. In order to better understand the change and variation of human
language, we draw on research in domain adaptation and extend the notion of
discrete domains to the continuous spectrum. We propose representation
learning-based models that can adapt to continuous domains and detail how these
can be used to investigate variation in language. To this end, we propose to
use dialogue modeling as a test bed due to its proximity to language modeling
and its social component.
Fuzzy Bayesian Learning
Authors: Indranil Pan, Dirk Bester
Date: 2016-10-28 00:00:00
Tasks: Bayesian Inference, General Classification, regression

In this paper we propose a novel approach for learning from data using rule
based fuzzy inference systems where the model parameters are estimated using
Bayesian inference and Markov Chain Monte Carlo (MCMC) techniques. We show the
applicability of the method for regression and classification tasks using
synthetic data-sets and also a real world example in the financial services
industry. Then we demonstrate how the method can be extended for knowledge
extraction to select the individual rules in a Bayesian way which best explains
the given data. Finally we discuss the advantages and pitfalls of using this
method over state-of-the-art techniques and highlight the specific class of
problems where this would be useful.
Single- and Multi-Task Architectures for Surgical Workflow Challenge at M2CAI 2016
Authors: Andru P. Twinanda, Didier Mutter, Jacques Marescaux, Michel de Mathelin, Nicolas Padoy
Date: 2016-10-27 00:00:00
Tasks: Surgical phase recognition

The surgical workflow challenge at M2CAI 2016 consists of identifying 8
surgical phases in cholecystectomy procedures. Here, we propose to use deep
architectures that are based on our previous work where we presented several
architectures to perform multiple recognition tasks on laparoscopic videos. In
this technical report, we present the phase recognition results using two
architectures: (1) a single-task architecture designed to perform solely the
surgical phase recognition task and (2) a multi-task architecture designed to
perform jointly phase recognition and tool presence detection. On top of these
architectures we propose to use two different approaches to enforce the
temporal constraints of the surgical workflow: (1) HMM-based and (2) LSTM-based
pipelines. The results show that the LSTM-based approach is able to outperform
the HMM-based approach and also to properly enforce the temporal constraints
into the recognition process.
Decentralized Clustering and Linking by Networked Agents
Authors: Sahar Khawatmi, Ali H. Sayed, Abdelhak M. Zoubir
Date: 2016-10-28 00:00:00
Tasks: Clustering

We consider the problem of decentralized clustering and estimation over
multi-task networks, where agents infer and track different models of interest.
The agents do not know beforehand which model is generating their own data.
They also do not know which agents in their neighborhood belong to the same
cluster. We propose a decentralized clustering algorithm aimed at identifying
and forming clusters of agents of similar objectives, and at guiding
cooperation to enhance the inference performance. One key feature of the
proposed technique is the integration of the learning and clustering tasks into
a single strategy. We analyze the performance of the procedure and show that
the error probabilities of types I and II decay exponentially to zero with the
step-size parameter. While links between agents following different objectives
are ignored in the clustering process, we nevertheless show how to exploit
these links to relay critical information across the network for enhanced
performance. Simulation results illustrate the performance of the proposed
method in comparison to other useful techniques.
$f$-Divergence Inequalities via Functional Domination
Authors: Igal Sason, Sergio Verdú
Date: 2016-10-28 00:00:00

This paper considers derivation of $f$-divergence inequalities via the
approach of functional domination. Bounds on an $f$-divergence based on one or
several other $f$-divergences are introduced, dealing with pairs of probability
measures defined on arbitrary alphabets. In addition, a variety of bounds are
shown to hold under boundedness assumptions on the relative information. The
journal paper, which includes more approaches for the derivation of
f-divergence inequalities and proofs, is available on the arXiv at
https://arxiv.org/abs/1508.00335, and it has been published in the IEEE Trans.
on Information Theory, vol. 62, no. 11, pp. 5973-6006, November 2016.
On the Latent Variable Interpretation in Sum-Product Networks
Authors: Robert Peharz, Robert Gens, Franz Pernkopf, Pedro Domingos
Date: 2016-01-22 00:00:00

One of the central themes in Sum-Product networks (SPNs) is the
interpretation of sum nodes as marginalized latent variables (LVs). This
interpretation yields an increased syntactic or semantic structure, allows the
application of the EM algorithm and to efficiently perform MPE inference. In
literature, the LV interpretation was justified by explicitly introducing the
indicator variables corresponding to the LVs' states. However, as pointed out
in this paper, this approach is in conflict with the completeness condition in
SPNs and does not fully specify the probabilistic model. We propose a remedy
for this problem by modifying the original approach for introducing the LVs,
which we call SPN augmentation. We discuss conditional independencies in
augmented SPNs, formally establish the probabilistic interpretation of the
sum-weights and give an interpretation of augmented SPNs as Bayesian networks.
Based on these results, we find a sound derivation of the EM algorithm for
SPNs. Furthermore, the Viterbi-style algorithm for MPE proposed in literature
was never proven to be correct. We show that this is indeed a correct
algorithm, when applied to selective SPNs, and in particular when applied to
augmented SPNs. Our theoretical results are confirmed in experiments on
synthetic data and 103 real-world datasets.
Convergence guarantees for kernel-based quadrature rules in misspecified settings
Authors: Motonobu Kanagawa, Bharath K. Sriperumbudur, Kenji Fukumizu
Date: 2016-05-24 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Numerical Integration

Kernel-based quadrature rules are becoming important in machine learning and
statistics, as they achieve super-$\sqrt{n}$ convergence rates in numerical
integration, and thus provide alternatives to Monte Carlo integration in
challenging settings where integrands are expensive to evaluate or where
integrands are high dimensional. These rules are based on the assumption that
the integrand has a certain degree of smoothness, which is expressed as that
the integrand belongs to a certain reproducing kernel Hilbert space (RKHS).
However, this assumption can be violated in practice (e.g., when the integrand
is a black box function), and no general theory has been established for the
convergence of kernel quadratures in such misspecified settings. Our
contribution is in proving that kernel quadratures can be consistent even when
the integrand does not belong to the assumed RKHS, i.e., when the integrand is
less smooth than assumed. Specifically, we derive convergence rates that depend
on the (unknown) lesser smoothness of the integrand, where the degree of
smoothness is expressed via powers of RKHSs or via Sobolev spaces.
Recent advances in content based video copy detection
Authors: Sanket Shinde, Girija Chiddarwar
Date: 2016-10-28 00:00:00
Tasks: Copy Detection

With the immense number of videos being uploaded to the video sharing sites,
issue of copyright infringement arises with uploading of illicit copies or
transformed versions of original video. Thus safeguarding copyright of digital
media has become matter of concern. To address this concern, it is obliged to
have a video copy detection system which is sufficiently robust to detect these
transformed videos with ability to pinpoint location of copied segments. This
paper outlines recent advancement in content based video copy detection, mainly
focusing on different visual features employed by video copy detection systems.
Finally we evaluate performance of existing video copy detection systems.