pwc-archive/papers-with-abstracts huggingface dataset:
Improved Style Transfer by Respecting Inter-layer Correlations
Authors: Mao-Chuang Yeh, Shuai Tang
Date: 2018-01-05 00:00:00
Tasks: Style Transfer, Texture Synthesis

A popular series of style transfer methods apply a style to a content image
by controlling mean and covariance of values in early layers of a feature
stack. This is insufficient for transferring styles that have strong structure
across spatial scales like, e.g., textures where dots lie on long curves. This
paper demonstrates that controlling inter-layer correlations yields visible
improvements in style transfer methods. We achieve this control by computing
cross-layer, rather than within-layer, gram matrices. We find that (a)
cross-layer gram matrices are sufficient to control within-layer statistics.
Inter-layer correlations improves style transfer and texture synthesis. The
paper shows numerous examples on "hard" real style transfer problems (e.g. long
scale and hierarchical patterns); (b) a fast approximate style transfer method
can control cross-layer gram matrices; (c) we demonstrate that multiplicative,
rather than additive style and content loss, results in very good style
transfer. Multiplicative loss produces a visible emphasis on boundaries, and
means that one hyper-parameter can be eliminated.
Gaussian Process bandits with adaptive discretization
Authors: Shubhanshu Shekhar, Tara Javidi
Date: 2017-12-05 00:00:00
Tasks: Multi-Armed Bandits

In this paper, the problem of maximizing a black-box function $f:\mathcal{X}
\to \mathbb{R}$ is studied in the Bayesian framework with a Gaussian Process
(GP) prior. In particular, a new algorithm for this problem is proposed, and
high probability bounds on its simple and cumulative regret are established.
The query point selection rule in most existing methods involves an exhaustive
search over an increasingly fine sequence of uniform discretizations of
$\mathcal{X}$. The proposed algorithm, in contrast, adaptively refines
$\mathcal{X}$ which leads to a lower computational complexity, particularly
when $\mathcal{X}$ is a subset of a high dimensional Euclidean space. In
addition to the computational gains, sufficient conditions are identified under
which the regret bounds of the new algorithm improve upon the known results.
Finally an extension of the algorithm to the case of contextual bandits is
proposed, and high probability bounds on the contextual regret are presented.
GeoSeq2Seq: Information Geometric Sequence-to-Sequence Networks
Authors: Alessandro Bay, Biswa Sengupta
Date: 2017-10-25 00:00:00
Proceeding: ICLR 2018 1
Tasks: Image Captioning, Translation

The Fisher information metric is an important foundation of information
geometry, wherein it allows us to approximate the local geometry of a
probability distribution. Recurrent neural networks such as the
Sequence-to-Sequence (Seq2Seq) networks that have lately been used to yield
state-of-the-art performance on speech translation or image captioning have so
far ignored the geometry of the latent embedding, that they iteratively learn.
We propose the information geometric Seq2Seq (GeoSeq2Seq) network which
abridges the gap between deep recurrent neural networks and information
geometry. Specifically, the latent embedding offered by a recurrent network is
encoded as a Fisher kernel of a parametric Gaussian Mixture Model, a formalism
common in computer vision. We utilise such a network to predict the shortest
routes between two nodes of a graph by learning the adjacency matrix using the
GeoSeq2Seq formalism; our results show that for such a problem the
probabilistic representation of the latent embedding supersedes the
non-probabilistic embedding by 10-15\%.
Strongly Hierarchical Factorization Machines and ANOVA Kernel Regression
Authors: Ruocheng Guo, Hamidreza Alvari, Paulo Shakarian
Date: 2017-12-25 00:00:00
Tasks: parameter estimation, regression, Sparse Learning

High-order parametric models that include terms for feature interactions are
applied to various data mining tasks, where ground truth depends on
interactions of features. However, with sparse data, the high- dimensional
parameters for feature interactions often face three issues: expensive
computation, difficulty in parameter estimation and lack of structure. Previous
work has proposed approaches which can partially re- solve the three issues. In
particular, models with factorized parameters (e.g. Factorization Machines) and
sparse learning algorithms (e.g. FTRL-Proximal) can tackle the first two issues
but fail to address the third. Regarding to unstructured parameters,
constraints or complicated regularization terms are applied such that
hierarchical structures can be imposed. However, these methods make the
optimization problem more challenging. In this work, we propose Strongly
Hierarchical Factorization Machines and ANOVA kernel regression where all the
three issues can be addressed without making the optimization problem more
difficult. Experimental results show the proposed models significantly
outperform the state-of-the-art in two data mining tasks: cold-start user
response time prediction and stock volatility prediction.
Beyond the Hazard Rate: More Perturbation Algorithms for Adversarial Multi-armed Bandits
Authors: Zifan Li, Ambuj Tewari
Date: 2017-02-17 00:00:00
Tasks: Multi-Armed Bandits

Recent work on follow the perturbed leader (FTPL) algorithms for the
adversarial multi-armed bandit problem has highlighted the role of the hazard
rate of the distribution generating the perturbations. Assuming that the hazard
rate is bounded, it is possible to provide regret analyses for a variety of
FTPL algorithms for the multi-armed bandit problem. This paper pushes the
inquiry into regret bounds for FTPL algorithms beyond the bounded hazard rate
condition. There are good reasons to do so: natural distributions such as the
uniform and Gaussian violate the condition. We give regret bounds for both
bounded support and unbounded support distributions without assuming the hazard
rate condition. We also disprove a conjecture that the Gaussian distribution
cannot lead to a low-regret algorithm. In fact, it turns out that it leads to
near optimal regret, up to logarithmic factors. A key ingredient in our
approach is the introduction of a new notion called the generalized hazard
rate.
Knowledge-based Word Sense Disambiguation using Topic Models
Authors: Devendra Singh Chaplot, Ruslan Salakhutdinov
Date: 2018-01-05 00:00:00
Tasks: Sentence, Topic Models, Word Sense Disambiguation

Word Sense Disambiguation is an open problem in Natural Language Processing
which is particularly challenging and useful in the unsupervised setting where
all the words in any given text need to be disambiguated without using any
labeled data. Typically WSD systems use the sentence or a small window of words
around the target word as the context for disambiguation because their
computational complexity scales exponentially with the size of the context. In
this paper, we leverage the formalism of topic model to design a WSD system
that scales linearly with the number of words in the context. As a result, our
system is able to utilize the whole document as the context for a word to be
disambiguated. The proposed method is a variant of Latent Dirichlet Allocation
in which the topic proportions for a document are replaced by synset
proportions. We further utilize the information in the WordNet by assigning a
non-uniform prior to synset distribution over words and a logistic-normal prior
for document distribution over synsets. We evaluate the proposed method on
Senseval-2, Senseval-3, SemEval-2007, SemEval-2013 and SemEval-2015 English
All-Word WSD datasets and show that it outperforms the state-of-the-art
unsupervised knowledge-based WSD system by a significant margin.
Clustering with Outlier Removal
Authors: Hongfu Liu, Jun Li, Yue Wu, Yun Fu
Date: 2018-01-05 00:00:00
Tasks: Clustering, Outlier Detection

Cluster analysis and outlier detection are strongly coupled tasks in data mining area. Cluster structure can be easily destroyed by few outliers; on the contrary, outliers are defined by the concept of cluster, which are recognized as the points belonging to none of the clusters. Unfortunately, most existing studies do not notice the coupled relationship between these two task and handle them separately. In light of this, we consider the joint cluster analysis and outlier detection problem, and propose the Clustering with Outlier Removal (COR) algorithm. Generally speaking, the original space is transformed into the binary space via generating basic partitions in order to define clusters. Then an objective function based Holoentropy is designed to enhance the compactness of each cluster with a few outliers removed. With further analyses on the objective function, only partial of the problem can be handled by K-means optimization. To provide an integrated solution, an auxiliary binary matrix is nontrivally introduced so that COR completely and efficiently solves the challenging problem via a unified K-means-- with theoretical supports. Extensive experimental results on numerous data sets in various domains demonstrate the effectiveness and efficiency of COR significantly over state-of-the-art methods in terms of cluster validity and outlier detection. Some key factors in COR are further analyzed for practical use. Finally, an application on flight trajectory is provided to demonstrate the effectiveness of COR in the real-world scenario.
Near Optimal Coded Data Shuffling for Distributed Learning
Authors: Mohamed A. Attia, Ravi Tandon
Date: 2018-01-05 00:00:00

Data shuffling between distributed cluster of nodes is one of the critical
steps in implementing large-scale learning algorithms. Randomly shuffling the
data-set among a cluster of workers allows different nodes to obtain fresh data
assignments at each learning epoch. This process has been shown to provide
improvements in the learning process. However, the statistical benefits of
distributed data shuffling come at the cost of extra communication overhead
from the master node to worker nodes, and can act as one of the major
bottlenecks in the overall time for computation. There has been significant
recent interest in devising approaches to minimize this communication overhead.
One approach is to provision for extra storage at the computing nodes. The
other emerging approach is to leverage coded communication to minimize the
overall communication overhead.
  The focus of this work is to understand the fundamental trade-off between the
amount of storage and the communication overhead for distributed data
shuffling. In this work, we first present an information theoretic formulation
for the data shuffling problem, accounting for the underlying problem
parameters (number of workers, $K$, number of data points, $N$, and the
available storage, $S$ per node). We then present an information theoretic
lower bound on the communication overhead for data shuffling as a function of
these parameters. We next present a novel coded communication scheme and show
that the resulting communication overhead of the proposed scheme is within a
multiplicative factor of at most $\frac{K}{K-1}$ from the information-theoretic
lower bound. Furthermore, we present the aligned coded shuffling scheme for
some storage values, which achieves the optimal storage vs communication
trade-off for $K<5$, and further reduces the maximum multiplicative gap down to
$\frac{K-\frac{1}{3}}{K-1}$, for $K\geq 5$.
Learning Implicit Brain MRI Manifolds with Deep Learning
Authors: Camilo Bermudez, Andrew J. Plassard, Larry T. Davis, Allen T. Newton, Susan M. Resnick, Bennett A. Landman
Date: 2018-01-05 00:00:00
Tasks: Deep Learning, Denoising, Generative Adversarial Network, Image Denoising, Image Generation

An important task in image processing and neuroimaging is to extract
quantitative information from the acquired images in order to make observations
about the presence of disease or markers of development in populations. Having
a lowdimensional manifold of an image allows for easier statistical comparisons
between groups and the synthesis of group representatives. Previous studies
have sought to identify the best mapping of brain MRI to a low-dimensional
manifold, but have been limited by assumptions of explicit similarity measures.
In this work, we use deep learning techniques to investigate implicit manifolds
of normal brains and generate new, high-quality images. We explore implicit
manifolds by addressing the problems of image synthesis and image denoising as
important tools in manifold learning. First, we propose the unsupervised
synthesis of T1-weighted brain MRI using a Generative Adversarial Network (GAN)
by learning from 528 examples of 2D axial slices of brain MRI. Synthesized
images were first shown to be unique by performing a crosscorrelation with the
training set. Real and synthesized images were then assessed in a blinded
manner by two imaging experts providing an image quality score of 1-5. The
quality score of the synthetic image showed substantial overlap with that of
the real images. Moreover, we use an autoencoder with skip connections for
image denoising, showing that the proposed method results in higher PSNR than
FSL SUSAN after denoising. This work shows the power of artificial networks to
synthesize realistic imaging data, which can be used to improve image
processing techniques and provide a quantitative framework to structural
changes in the brain.
Curriculum Learning of Visual Attribute Clusters for Multi-Task Classification
Authors: Nikolaos Sarafianos, Theodore Giannakopoulos, Christophoros Nikou, Ioannis A. Kakadiaris
Date: 2017-09-19 00:00:00
Tasks: Attribute, Classification, Clustering, General Classification, Image Description

Visual attributes, from simple objects (e.g., backpacks, hats) to
soft-biometrics (e.g., gender, height, clothing) have proven to be a powerful
representational approach for many applications such as image description and
human identification. In this paper, we introduce a novel method to combine the
advantages of both multi-task and curriculum learning in a visual attribute
classification framework. Individual tasks are grouped after performing
hierarchical clustering based on their correlation. The clusters of tasks are
learned in a curriculum learning setup by transferring knowledge between
clusters. The learning process within each cluster is performed in a multi-task
classification setup. By leveraging the acquired knowledge, we speed-up the
process and improve performance. We demonstrate the effectiveness of our method
via ablation studies and a detailed analysis of the covariates, on a variety of
publicly available datasets of humans standing with their full-body visible.
Extensive experimentation has proven that the proposed approach boosts the
performance by 4% to 10%.
Shielding Google's language toxicity model against adversarial attacks
Authors: Nestor Rodriguez, Sergio Rojas-Galeano
Date: 2018-01-05 00:00:00
Tasks: BIG-bench Machine Learning, model, Negation

Lack of moderation in online communities enables participants to incur in
personal aggression, harassment or cyberbullying, issues that have been
accentuated by extremist radicalisation in the contemporary post-truth politics
scenario. This kind of hostility is usually expressed by means of toxic
language, profanity or abusive statements. Recently Google has developed a
machine-learning-based toxicity model in an attempt to assess the hostility of
a comment; unfortunately, it has been suggested that said model can be deceived
by adversarial attacks that manipulate the text sequence of the comment. In
this paper we firstly characterise such adversarial attacks as using
obfuscation and polarity transformations. The former deceives by corrupting
toxic trigger content with typographic edits, whereas the latter deceives by
grammatical negation of the toxic content. Then, we propose a two--stage
approach to counter--attack these anomalies, bulding upon a recently proposed
text deobfuscation method and the toxicity scoring model. Lastly, we conducted
an experiment with approximately 24000 distorted comments, showing how in this
way it is feasible to restore toxicity of the adversarial variants, while
incurring roughly on a twofold increase in processing time. Even though novel
adversary challenges would keep coming up derived from the versatile nature of
written language, we anticipate that techniques combining machine learning and
text pattern recognition methods, each one targeting different layers of
linguistic features, would be needed to achieve robust detection of toxic
language, thus fostering aggression--free digital interaction.
Towards Understanding and Answering Multi-Sentence Recommendation Questions on Tourism
Authors: Danish Contractor, Barun Patra, Mausam Singla, Parag Singla
Date: 2018-01-05 00:00:00
Tasks: Negation, Sentence

We introduce the first system towards the novel task of answering complex
multisentence recommendation questions in the tourism domain. Our solution uses
a pipeline of two modules: question understanding and answering. For question
understanding, we define an SQL-like query language that captures the semantic
intent of a question; it supports operators like subset, negation, preference
and similarity, which are often found in recommendation questions. We train and
compare traditional CRFs as well as bidirectional LSTM-based models for
converting a question to its semantic representation. We extend these models to
a semisupervised setting with partially labeled sequences gathered through
crowdsourcing. We find that our best model performs semi-supervised training of
BiDiLSTM+CRF with hand-designed features and CCM(Chang et al., 2007)
constraints. Finally, in an end to end QA system, our answering component
converts our question representation into queries fired on underlying knowledge
sources. Our experiments on two different answer corpora demonstrate that our
system can significantly outperform baselines with up to 20 pt higher accuracy
and 17 pt higher recall.
A study of the classification of low-dimensional data with supervised manifold learning
Authors: Elif Vural, Christine Guillemot
Date: 2015-07-21 00:00:00
Tasks: Dimensionality Reduction, General Classification, Generalization Bounds

Supervised manifold learning methods learn data representations by preserving
the geometric structure of data while enhancing the separation between data
samples from different classes. In this work, we propose a theoretical study of
supervised manifold learning for classification. We consider nonlinear
dimensionality reduction algorithms that yield linearly separable embeddings of
training data and present generalization bounds for this type of algorithms. A
necessary condition for satisfactory generalization performance is that the
embedding allow the construction of a sufficiently regular interpolation
function in relation with the separation margin of the embedding. We show that
for supervised embeddings satisfying this condition, the classification error
decays at an exponential rate with the number of training samples. Finally, we
examine the separability of supervised nonlinear embeddings that aim to
preserve the low-dimensional geometric structure of data based on graph
representations. The proposed analysis is supported by experiments on several
real data sets.
Adversarial Examples: Attacks and Defenses for Deep Learning
Authors: Xiaoyong Yuan, Pan He, Qile Zhu, Xiaolin Li
Date: 2017-12-19 00:00:00
Tasks: Deep Learning

With rapid progress and significant successes in a wide spectrum of
applications, deep learning is being applied in many safety-critical
environments. However, deep neural networks have been recently found vulnerable
to well-designed input samples, called adversarial examples. Adversarial
examples are imperceptible to human but can easily fool deep neural networks in
the testing/deploying stage. The vulnerability to adversarial examples becomes
one of the major risks for applying deep neural networks in safety-critical
environments. Therefore, attacks and defenses on adversarial examples draw
great attention. In this paper, we review recent findings on adversarial
examples for deep neural networks, summarize the methods for generating
adversarial examples, and propose a taxonomy of these methods. Under the
taxonomy, applications for adversarial examples are investigated. We further
elaborate on countermeasures for adversarial examples and explore the
challenges and the potential solutions.
Superrational types
Authors: Fernando Tohmé, Ignacio Viglizzo
Date: 2018-01-05 00:00:00

We present a formal analysis of Douglas Hofstadter's concept of
\emph{superrationality}. We start by defining superrationally justifiable
actions, and study them in symmetric games. We then model the beliefs of the
players, in a way that leads them to different choices than the usual
assumption of rationality by restricting the range of conceivable choices.
These beliefs are captured in the formal notion of \emph{type} drawn from
epistemic game theory. The theory of coalgebras is used to frame type spaces
and to account for the existence of some of them. We find conditions that
guarantee superrational outcomes.
Deep Reinforcement Learning for List-wise Recommendations
Authors: Xiangyu Zhao, Liang Zhang, Long Xia, Zhuoye Ding, Dawei Yin, Jiliang Tang
Date: 2017-12-30 00:00:00
Tasks: Deep Reinforcement Learning, Recommendation Systems, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recommender systems play a crucial role in mitigating the problem of information overload by suggesting users' personalized items or services. The vast majority of traditional recommender systems consider the recommendation procedure as a static process and make recommendations following a fixed strategy. In this paper, we propose a novel recommender system with the capability of continuously improving its strategies during the interactions with users. We model the sequential interactions between users and a recommender system as a Markov Decision Process (MDP) and leverage Reinforcement Learning (RL) to automatically learn the optimal strategies via recommending trial-and-error items and receiving reinforcements of these items from users' feedbacks. In particular, we introduce an online user-agent interacting environment simulator, which can pre-train and evaluate model parameters offline before applying the model online. Moreover, we validate the importance of list-wise recommendations during the interactions between users and agent, and develop a novel approach to incorporate them into the proposed framework LIRD for list-wide recommendations. The experimental results based on a real-world e-commerce dataset demonstrate the effectiveness of the proposed framework.
Learning Feature Representations for Keyphrase Extraction
Authors: Corina Florescu, Wei Jin
Date: 2018-01-05 00:00:00
Tasks: BIG-bench Machine Learning, Feature Engineering, Keyphrase Extraction

In supervised approaches for keyphrase extraction, a candidate phrase is
encoded with a set of hand-crafted features and machine learning algorithms are
trained to discriminate keyphrases from non-keyphrases. Although the
manually-designed features have shown to work well in practice, feature
engineering is a difficult process that requires expert knowledge and normally
does not generalize well. In this paper, we present SurfKE, a feature learning
framework that exploits the text itself to automatically discover patterns that
keyphrases exhibit. Our model represents the document as a graph and
automatically learns feature representation of phrases. The proposed model
obtains remarkable improvements in performance over strong baselines.