pwc-archive/papers-with-abstracts huggingface dataset:
Why comparing survival curves between two subgroups may be misleading
Authors: Damjan Krstajic
Date: 2016-11-04 00:00:00
Tasks: Binary Classification, Diagnostic, Vocal Bursts Valence Prediction

We analyse an issue when comparing survival curves between two subgroups. We show that there is a direct relationship between estimates of subgroups' survival at a time point and positive and negative predictive values in the binary classification settings. Our findings present a case where current methods of comparing survival curves between subgroups may be misleading. We think that this ought to be taken into account during the validation of prognostic diagnostic tests that predict two prognostic subgroups for a given disease or treatment, when the validation data set consists of censored data.
Exact Inference Techniques for the Analysis of Bayesian Attack Graphs
Authors: Luis Muñoz-González, Daniele Sgandurra, Martín Barrère, Emil Lupu
Date: 2015-10-08 00:00:00

Attack graphs are a powerful tool for security risk assessment by analysing
network vulnerabilities and the paths attackers can use to compromise network
resources. The uncertainty about the attacker's behaviour makes Bayesian
networks suitable to model attack graphs to perform static and dynamic
analysis. Previous approaches have focused on the formalization of attack
graphs into a Bayesian model rather than proposing mechanisms for their
analysis. In this paper we propose to use efficient algorithms to make exact
inference in Bayesian attack graphs, enabling the static and dynamic network
risk assessments. To support the validity of our approach we have performed an
extensive experimental evaluation on synthetic Bayesian attack graphs with
different topologies, showing the computational advantages in terms of time and
memory use of the proposed techniques when compared to existing approaches.
Learning heat diffusion graphs
Authors: Dorina Thanou, Xiaowen Dong, Daniel Kressner, Pascal Frossard
Date: 2016-11-04 00:00:00
Tasks: Graph Learning

Effective information analysis generally boils down to properly identifying
the structure or geometry of the data, which is often represented by a graph.
In some applications, this structure may be partly determined by design
constraints or pre-determined sensing arrangements, like in road transportation
networks for example. In general though, the data structure is not readily
available and becomes pretty difficult to define. In particular, the global
smoothness assumptions, that most of the existing works adopt, are often too
general and unable to properly capture localized properties of data. In this
paper, we go beyond this classical data model and rather propose to represent
information as a sparse combination of localized functions that live on a data
structure represented by a graph. Based on this model, we focus on the problem
of inferring the connectivity that best explains the data samples at different
vertices of a graph that is a priori unknown. We concentrate on the case where
the observed data is actually the sum of heat diffusion processes, which is a
quite common model for data on networks or other irregular structures. We cast
a new graph learning problem and solve it with an efficient nonconvex
optimization algorithm. Experiments on both synthetic and real world data
finally illustrate the benefits of the proposed graph learning framework and
confirm that the data structure can be efficiently learned from data
observations only. We believe that our algorithm will help solving key
questions in diverse application domains such as social and biological network
analysis where it is crucial to unveil proper geometry for data understanding
and inference.
Exploiting the Structure: Stochastic Gradient Methods Using Raw Clusters
Authors: Zeyuan Allen-Zhu, Yang Yuan, Karthik Sridharan
Date: 2016-02-05 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: BIG-bench Machine Learning, Clustering

The amount of data available in the world is growing faster than our ability
to deal with it. However, if we take advantage of the internal
\emph{structure}, data may become much smaller for machine learning purposes.
In this paper we focus on one of the fundamental machine learning tasks,
empirical risk minimization (ERM), and provide faster algorithms with the help
from the clustering structure of the data.
  We introduce a simple notion of raw clustering that can be efficiently
computed from the data, and propose two algorithms based on clustering
information. Our accelerated algorithm ClusterACDM is built on a novel Haar
transformation applied to the dual space of the ERM problem, and our
variance-reduction based algorithm ClusterSVRG introduces a new gradient
estimator using clustering. Our algorithms outperform their classical
counterparts ACDM and SVRG respectively.
Ways of Conditioning Generative Adversarial Networks
Authors: Hanock Kwak, Byoung-Tak Zhang
Date: 2016-11-04 00:00:00

The GANs are generative models whose random samples realistically reflect
natural images. It also can generate samples with specific attributes by
concatenating a condition vector into the input, yet research on this field is
not well studied. We propose novel methods of conditioning generative
adversarial networks (GANs) that achieve state-of-the-art results on MNIST and
CIFAR-10. We mainly introduce two models: an information retrieving model that
extracts conditional information from the samples, and a spatial bilinear
pooling model that forms bilinear features derived from the spatial cross
product of an image and a condition vector. These methods significantly enhance
log-likelihood of test data under the conditional distributions compared to the
methods of concatenation.
Semi-supervised deep learning by metric embedding
Authors: Elad Hoffer, Nir Ailon
Date: 2016-11-04 00:00:00
Tasks: Deep Learning, General Classification

Deep networks are successfully used as classification models yielding
state-of-the-art results when trained on a large number of labeled samples.
These models, however, are usually much less suited for semi-supervised
problems because of their tendency to overfit easily when trained on small
amounts of data. In this work we will explore a new training objective that is
targeting a semi-supervised regime with only a small subset of labeled data.
This criterion is based on a deep metric embedding over distance relations
within the set of labeled samples, together with constraints over the
embeddings of the unlabeled set. The final learned representations are
discriminative in euclidean space, and hence can be used with subsequent
nearest-neighbor classification using the labeled samples.
Learning to Rank Scientific Documents from the Crowd
Authors: Jesse M Lingeman, Hong Yu
Date: 2016-11-04 00:00:00
Tasks: Articles, Document Ranking, Learning-To-Rank, text similarity

Finding related published articles is an important task in any science, but
with the explosion of new work in the biomedical domain it has become
especially challenging. Most existing methodologies use text similarity metrics
to identify whether two articles are related or not. However biomedical
knowledge discovery is hypothesis-driven. The most related articles may not be
ones with the highest text similarities. In this study, we first develop an
innovative crowd-sourcing approach to build an expert-annotated
document-ranking corpus. Using this corpus as the gold standard, we then
evaluate the approaches of using text similarity to rank the relatedness of
articles. Finally, we develop and evaluate a new supervised model to
automatically rank related scientific articles. Our results show that authors'
ranking differ significantly from rankings by text-similarity-based models. By
training a learning-to-rank model on a subset of the annotated corpus, we found
the best supervised learning-to-rank model (SVM-Rank) significantly surpassed
state-of-the-art baseline systems.
An Automatic Machine Translation Evaluation Metric Based on Dependency Parsing Model
Authors: Hui Yu, Xiaofeng Wu, Wenbin Jiang, Qun Liu, ShouXun Lin
Date: 2015-08-09 00:00:00
Tasks: Dependency Parsing, Machine Translation, Sentence, Translation

Most of the syntax-based metrics obtain the similarity by comparing the
sub-structures extracted from the trees of hypothesis and reference. These
sub-structures are defined by human and can't express all the information in
the trees because of the limited length of sub-structures. In addition, the
overlapped parts between these sub-structures are computed repeatedly. To avoid
these problems, we propose a novel automatic evaluation metric based on
dependency parsing model, with no need to define sub-structures by human.
First, we train a dependency parsing model by the reference dependency tree.
Then we generate the hypothesis dependency tree and the corresponding
probability by the dependency parsing model. The quality of the hypothesis can
be judged by this probability. In order to obtain the lexicon similarity, we
also introduce the unigram F-score to the new metric. Experiment results show
that the new metric gets the state-of-the-art performance on system level, and
is comparable with METEOR on sentence level.
Improve the Evaluation of Fluency Using Entropy for Machine Translation Evaluation Metrics
Authors: Hui Yu, Xiaofeng Wu, Wenbin Jiang, Qun Liu, ShouXun Lin
Date: 2015-08-10 00:00:00
Tasks: Machine Translation, Sentence, Translation

The widely-used automatic evaluation metrics cannot adequately reflect the
fluency of the translations. The n-gram-based metrics, like BLEU, limit the
maximum length of matched fragments to n and cannot catch the matched fragments
longer than n, so they can only reflect the fluency indirectly. METEOR, which
is not limited by n-gram, uses the number of matched chunks but it does not
consider the length of each chunk. In this paper, we propose an entropy-based
method, which can sufficiently reflect the fluency of translations through the
distribution of matched words. This method can easily combine with the
widely-used automatic evaluation metrics to improve the evaluation of fluency.
Experiments show that the correlations of BLEU and METEOR are improved on
sentence level after combining with the entropy-based method on WMT 2010 and
WMT 2012.
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
Authors: Tal Linzen, Emmanuel Dupoux, Yoav Goldberg
Date: 2016-11-04 00:00:00
Proceeding: TACL 2016 1
Tasks: Language Modeling, Language Modelling

The success of long short-term memory (LSTM) neural networks in language
processing is typically attributed to their ability to capture long-distance
statistical regularities. Linguistic regularities are often sensitive to
syntactic structure; can such dependencies be captured by LSTMs, which do not
have explicit structural representations? We begin addressing this question
using number agreement in English subject-verb dependencies. We probe the
architecture's grammatical competence both using training objectives with an
explicit grammatical target (number prediction, grammaticality judgments) and
using language models. In the strongly supervised settings, the LSTM achieved
very high overall accuracy (less than 1% errors), but errors increased when
sequential and structural information conflicted. The frequency of such errors
rose sharply in the language-modeling setting. We conclude that LSTMs can
capture a non-trivial amount of grammatical structure given targeted
supervision, but stronger architectures may be required to further reduce
errors; furthermore, the language modeling signal is insufficient for capturing
syntax-sensitive dependencies, and should be supplemented with more direct
supervision if such dependencies need to be captured.
A Perspective on Deep Imaging
Authors: Ge Wang
Date: 2016-09-10 00:00:00
Tasks: BIG-bench Machine Learning, Image Reconstruction

The combination of tomographic imaging and deep learning, or machine learning
in general, promises to empower not only image analysis but also image
reconstruction. The latter aspect is considered in this perspective article
with an emphasis on medical imaging to develop a new generation of image
reconstruction theories and techniques. This direction might lead to
intelligent utilization of domain knowledge from big data, innovative
approaches for image reconstruction, and superior performance in clinical and
preclinical applications. To realize the full impact of machine learning on
medical imaging, major challenges must be addressed.
Kernel-based Tests for Joint Independence
Authors: Niklas Pfister, Peter Bühlmann, Bernhard Schölkopf, Jonas Peters
Date: 2016-03-01 00:00:00
Tasks: Causal Discovery

We investigate the problem of testing whether $d$ random variables, which may
or may not be continuous, are jointly (or mutually) independent. Our method
builds on ideas of the two variable Hilbert-Schmidt independence criterion
(HSIC) but allows for an arbitrary number of variables. We embed the
$d$-dimensional joint distribution and the product of the marginals into a
reproducing kernel Hilbert space and define the $d$-variable Hilbert-Schmidt
independence criterion (dHSIC) as the squared distance between the embeddings.
In the population case, the value of dHSIC is zero if and only if the $d$
variables are jointly independent, as long as the kernel is characteristic.
Based on an empirical estimate of dHSIC, we define three different
non-parametric hypothesis tests: a permutation test, a bootstrap test and a
test based on a Gamma approximation. We prove that the permutation test
achieves the significance level and that the bootstrap test achieves pointwise
asymptotic significance level as well as pointwise asymptotic consistency
(i.e., it is able to detect any type of fixed dependence in the large sample
limit). The Gamma approximation does not come with these guarantees; however,
it is computationally very fast and for small $d$, it performs well in
practice. Finally, we apply the test to a problem in causal discovery.
Regularized Pel-Recursive Motion Estimation Using Generalized Cross-Validation and Spatial Adaptation
Authors: Vania V. Estrela, Luis A. Rivera, Paulo C. Beggio, Ricardo T. Lopes
Date: 2016-11-04 00:00:00
Tasks: Motion Estimation, Optical Flow Estimation

The computation of 2-D optical flow by means of regularized pel-recursive
algorithms raises a host of issues, which include the treatment of outliers,
motion discontinuities and occlusion among other problems. We propose a new
approach which allows us to deal with these issues within a common framework.
Our approach is based on the use of a technique called Generalized
Cross-Validation to estimate the best regularization scheme for a given pixel.
In our model, the regularization parameter is a matrix whose entries can
account for diverse sources of error. The estimation of the motion vectors
takes into consideration local properties of the image following a spatially
adaptive approach where each moving pixel is supposed to have its own
regularization matrix. Preliminary experiments indicate that this approach
provides robust estimates of the optical flow.
Fast Eigenspace Approximation using Random Signals
Authors: Johan Paratte, Lionel Martin
Date: 2016-11-03 00:00:00
Tasks: Clustering

We focus in this work on the estimation of the first $k$ eigenvectors of any
graph Laplacian using filtering of Gaussian random signals. We prove that we
only need $k$ such signals to be able to exactly recover as many of the
smallest eigenvectors, regardless of the number of nodes in the graph. In
addition, we address key issues in implementing the theoretical concepts in
practice using accurate approximated methods. We also propose fast algorithms
both for eigenspace approximation and for the determination of the $k$th
smallest eigenvalue $\lambda_k$. The latter proves to be extremely efficient
under the assumption of locally uniform distribution of the eigenvalue over the
spectrum. Finally, we present experiments which show the validity of our method
in practice and compare it to state-of-the-art methods for clustering and
visualization both on synthetic small-scale datasets and larger real-world
problems of millions of nodes. We show that our method allows a better scaling
with the number of nodes than all previous methods while achieving an almost
perfect reconstruction of the eigenspace formed by the first $k$ eigenvectors.
A Communication-Efficient Parallel Algorithm for Decision Tree
Authors: Qi Meng, Guolin Ke, Taifeng Wang, Wei Chen, Qiwei Ye, Zhi-Ming Ma, Tie-Yan Liu
Date: 2016-11-04 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: 2k, Attribute

Decision tree (and its extensions such as Gradient Boosting Decision Trees
and Random Forest) is a widely used machine learning algorithm, due to its
practical effectiveness and model interpretability. With the emergence of big
data, there is an increasing need to parallelize the training process of
decision tree. However, most existing attempts along this line suffer from high
communication costs. In this paper, we propose a new algorithm, called
\emph{Parallel Voting Decision Tree (PV-Tree)}, to tackle this challenge. After
partitioning the training data onto a number of (e.g., $M$) machines, this
algorithm performs both local voting and global voting in each iteration. For
local voting, the top-$k$ attributes are selected from each machine according
to its local data. Then, globally top-$2k$ attributes are determined by a
majority voting among these local candidates. Finally, the full-grained
histograms of the globally top-$2k$ attributes are collected from local
machines in order to identify the best (most informative) attribute and its
split point. PV-Tree can achieve a very low communication cost (independent of
the total number of attributes) and thus can scale out very well. Furthermore,
theoretical analysis shows that this algorithm can learn a near optimal
decision tree, since it can find the best attribute with a large probability.
Our experiments on real-world datasets show that PV-Tree significantly
outperforms the existing parallel decision tree algorithms in the trade-off
between accuracy and efficiency.
Semantic Noise Modeling for Better Representation Learning
Authors: Hyo-Eun Kim, Sangheum Hwang, Kyunghyun Cho
Date: 2016-11-04 00:00:00
Tasks: Representation Learning

Latent representation learned from multi-layered neural networks via
hierarchical feature abstraction enables recent success of deep learning. Under
the deep learning framework, generalization performance highly depends on the
learned latent representation which is obtained from an appropriate training
scenario with a task-specific objective on a designed network model. In this
work, we propose a novel latent space modeling method to learn better latent
representation. We designed a neural network model based on the assumption that
good base representation can be attained by maximizing the total correlation
between the input, latent, and output variables. From the base model, we
introduce a semantic noise modeling method which enables class-conditional
perturbation on latent space to enhance the representational power of learned
latent feature. During training, latent vector representation can be
stochastically perturbed by a modeled class-conditional additive noise while
maintaining its original semantic feature. It implicitly brings the effect of
semantic augmentation on the latent space. The proposed model can be easily
learned by back-propagation with common gradient-based optimization algorithms.
Experimental results show that the proposed method helps to achieve performance
benefits against various previous approaches. We also provide the empirical
analyses for the proposed class-conditional perturbation process including
t-SNE visualization.
Bayesian Non-parametric model to Target Gamification Notifications Using Big Data
Authors: Meisam Hejazi Nia, Brian Ratchford
Date: 2016-11-04 00:00:00

I suggest an approach that helps the online marketers to target their
Gamification elements to users by modifying the order of the list of tasks that
they send to users. It is more realistic and flexible as it allows the model to
learn more parameters when the online marketers collect more data. The
targeting approach is scalable and quick, and it can be used over streaming
data.
Generalized Topic Modeling
Authors: Avrim Blum, Nika Haghtalab
Date: 2016-11-04 00:00:00
Tasks: Topic Models

Recently there has been significant activity in developing algorithms with
provable guarantees for topic modeling. In standard topic models, a topic (such
as sports, business, or politics) is viewed as a probability distribution $\vec
a_i$ over words, and a document is generated by first selecting a mixture $\vec
w$ over topics, and then generating words i.i.d. from the associated mixture
$A{\vec w}$. Given a large collection of such documents, the goal is to recover
the topic vectors and then to correctly classify new documents according to
their topic mixture.
  In this work we consider a broad generalization of this framework in which
words are no longer assumed to be drawn i.i.d. and instead a topic is a complex
distribution over sequences of paragraphs. Since one could not hope to even
represent such a distribution in general (even if paragraphs are given using
some natural feature representation), we aim instead to directly learn a
document classifier. That is, we aim to learn a predictor that given a new
document, accurately predicts its topic mixture, without learning the
distributions explicitly. We present several natural conditions under which one
can do this efficiently and discuss issues such as noise tolerance and sample
complexity in this model. More generally, our model can be viewed as a
generalization of the multi-view or co-training setting in machine learning.
Long-term causal effects via behavioral game theory
Authors: Panagiotis, Toulis, David C. Parkes
Date: 2015-01-10 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Causal Inference

Planned experiments are the gold standard in reliably comparing the causal
effect of switching from a baseline policy to a new policy. One critical
shortcoming of classical experimental methods, however, is that they typically
do not take into account the dynamic nature of response to policy changes. For
instance, in an experiment where we seek to understand the effects of a new ad
pricing policy on auction revenue, agents may adapt their bidding in response
to the experimental pricing changes. Thus, causal effects of the new pricing
policy after such adaptation period, the {\em long-term causal effects}, are
not captured by the classical methodology even though they clearly are more
indicative of the value of the new policy. Here, we formalize a framework to
define and estimate long-term causal effects of policy changes in multiagent
economies. Central to our approach is behavioral game theory, which we leverage
to formulate the ignorability assumptions that are necessary for causal
inference. Under such assumptions we estimate long-term causal effects through
a latent space approach, where a behavioral model of how agents act conditional
on their latent behaviors is combined with a temporal model of how behaviors
evolve over time.