pwc-archive/papers-with-abstracts huggingface dataset:
Parallelized Tensor Train Learning of Polynomial Classifiers
Authors: Zhongming Chen, Kim Batselier, Johan A. K. Suykens, Ngai Wong
Date: 2016-12-20 00:00:00
Tasks: General Classification

In pattern classification, polynomial classifiers are well-studied methods as
they are capable of generating complex decision surfaces. Unfortunately, the
use of multivariate polynomials is limited to kernels as in support vector
machines, because polynomials quickly become impractical for high-dimensional
problems. In this paper, we effectively overcome the curse of dimensionality by
employing the tensor train format to represent a polynomial classifier. Based
on the structure of tensor trains, two learning algorithms are proposed which
involve solving different optimization problems of low computational
complexity. Furthermore, we show how both regularization to prevent overfitting
and parallelization, which enables the use of large training sets, are
incorporated into these methods. Both the efficiency and efficacy of our
tensor-based polynomial classifier are then demonstrated on the two popular
datasets USPS and MNIST.
Authorship Analysis of Xenophon's Cyropaedia
Authors: Anjalie Field
Date: 2017-11-06 00:00:00
Tasks: Authorship Attribution

In the past several decades, many authorship attribution studies have used
computational methods to determine the authors of disputed texts. Disputed
authorship is a common problem in Classics, since little information about
ancient documents has survived the centuries. Many scholars have questioned the
authenticity of the final chapter of Xenophon's Cyropaedia, a 4th century B.C.
historical text. In this study, we use N-grams frequency vectors with a cosine
similarity function and word frequency vectors with Naive Bayes Classifiers
(NBC) and Support Vector Machines (SVM) to analyze the authorship of the
Cyropaedia. Although the N-gram analysis shows that the epilogue of the
Cyropaedia differs slightly from the rest of the work, comparing the analysis
of Xenophon with analyses of Aristotle and Plato suggests that this difference
is not significant. Both NBC and SVM analyses of word frequencies show that the
final chapter of the Cyropaedia is closely related to the other chapters of the
Cyropaedia. Therefore, this analysis suggests that the disputed chapter was
written by Xenophon. This information can help scholars better understand the
Cyropaedia and also demonstrates the usefulness of applying modern authorship
analysis techniques to classical literature.
Private Learning on Networks: Part II
Authors: Shripad Gade, Nitin H. Vaidya
Date: 2017-03-27 00:00:00
Tasks: Distributed Optimization

This paper considers a distributed multi-agent optimization problem, with the
global objective consisting of the sum of local objective functions of the
agents. The agents solve the optimization problem using local computation and
communication between adjacent agents in the network. We present two randomized
iterative algorithms for distributed optimization. To improve privacy, our
algorithms add "structured" randomization to the information exchanged between
the agents. We prove deterministic correctness (in every execution) of the
proposed algorithms despite the information being perturbed by noise with
non-zero mean. We prove that a special case of a proposed algorithm (called
function sharing) preserves privacy of individual polynomial objective
functions under a suitable connectivity condition on the network topology.
Matching neural paths: transfer from recognition to correspondence search
Authors: Nikolay Savinov, Lubor Ladicky, Marc Pollefeys
Date: 2017-05-19 00:00:00
Proceeding: NeurIPS 2017 12

Many machine learning tasks require finding per-part correspondences between
objects. In this work we focus on low-level correspondences - a highly
ambiguous matching problem. We propose to use a hierarchical semantic
representation of the objects, coming from a convolutional neural network, to
solve this ambiguity. Training it for low-level correspondence prediction
directly might not be an option in some domains where the ground-truth
correspondences are hard to obtain. We show how transfer from recognition can
be used to avoid such training. Our idea is to mark parts as "matching" if
their features are close to each other at all the levels of convolutional
feature hierarchy (neural paths). Although the overall number of such paths is
exponential in the number of layers, we propose a polynomial algorithm for
aggregating all of them in a single backward pass. The empirical validation is
done on the task of stereo correspondence and demonstrates that we achieve
competitive results among the methods which do not use labeled target domain
data.
Robustly Learning a Gaussian: Getting Optimal Error, Efficiently
Authors: Ilias Diakonikolas, Gautam Kamath, Daniel M. Kane, Jerry Li, Ankur Moitra, Alistair Stewart
Date: 2017-04-12 00:00:00

We study the fundamental problem of learning the parameters of a
high-dimensional Gaussian in the presence of noise -- where an
$\varepsilon$-fraction of our samples were chosen by an adversary. We give
robust estimators that achieve estimation error $O(\varepsilon)$ in the total
variation distance, which is optimal up to a universal constant that is
independent of the dimension.
  In the case where just the mean is unknown, our robustness guarantee is
optimal up to a factor of $\sqrt{2}$ and the running time is polynomial in $d$
and $1/\epsilon$. When both the mean and covariance are unknown, the running
time is polynomial in $d$ and quasipolynomial in $1/\varepsilon$. Moreover all
of our algorithms require only a polynomial number of samples. Our work shows
that the same sorts of error guarantees that were established over fifty years
ago in the one-dimensional setting can also be achieved by efficient algorithms
in high-dimensional settings.
A Deep Reinforcement Learning Chatbot
Authors: Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin, Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke, Sai Rajeshwar, Alexandre de Brebisson, Jose M. R. Sotelo, Dendi Suhubdy, Vincent Michalski, Alexandre Nguyen, Joelle Pineau, Yoshua Bengio
Date: 2017-09-07 00:00:00
Tasks: Chatbot, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Retrieval, Text Generation

We present MILABOT: a deep reinforcement learning chatbot developed by the
Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize
competition. MILABOT is capable of conversing with humans on popular small talk
topics through both speech and text. The system consists of an ensemble of
natural language generation and retrieval models, including template-based
models, bag-of-words models, sequence-to-sequence neural network and latent
variable neural network models. By applying reinforcement learning to
crowdsourced data and real-world user interactions, the system has been trained
to select an appropriate response from the models in its ensemble. The system
has been evaluated through A/B testing with real-world users, where it
performed significantly better than many competing systems. Due to its machine
learning architecture, the system is likely to improve with additional data.
Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
Authors: Matthias Hein, Maksym Andriushchenko
Date: 2017-05-23 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: General Classification

Recent work has shown that state-of-the-art classifiers are quite brittle, in
the sense that a small adversarial change of an originally with high confidence
correctly classified input leads to a wrong classification again with high
confidence. This raises concerns that such classifiers are vulnerable to
attacks and calls into question their usage in safety-critical systems. We show
in this paper for the first time formal guarantees on the robustness of a
classifier by giving instance-specific lower bounds on the norm of the input
manipulation required to change the classifier decision. Based on this analysis
we propose the Cross-Lipschitz regularization functional. We show that using
this form of regularization in kernel methods resp. neural networks improves
the robustness of the classifier without any loss in prediction performance.
Conditional Gradient Method for Stochastic Submodular Maximization: Closing the Gap
Authors: Aryan Mokhtari, Hamed Hassani, Amin Karbasi
Date: 2017-11-05 00:00:00

In this paper, we study the problem of \textit{constrained} and
\textit{stochastic} continuous submodular maximization. Even though the
objective function is not concave (nor convex) and is defined in terms of an
expectation, we develop a variant of the conditional gradient method, called
\alg, which achieves a \textit{tight} approximation guarantee. More precisely,
for a monotone and continuous DR-submodular function and subject to a
\textit{general} convex body constraint, we prove that \alg achieves a
$[(1-1/e)\text{OPT} -\eps]$ guarantee (in expectation) with
$\mathcal{O}{(1/\eps^3)}$ stochastic gradient computations. This guarantee
matches the known hardness results and closes the gap between deterministic and
stochastic continuous submodular maximization. By using stochastic continuous
optimization as an interface, we also provide the first $(1-1/e)$ tight
approximation guarantee for maximizing a \textit{monotone but stochastic}
submodular \textit{set} function subject to a general matroid constraint.
Spatial Pyramid Context-Aware Moving Object Detection and Tracking for Full Motion Video and Wide Aerial Motion Imagery
Authors: Mahdieh Poostchi
Date: 2017-11-05 00:00:00
Tasks: 3D Face Reconstruction, Face Reconstruction, GPU, Moving Object Detection, Object, object-detection, Object Detection

A robust and fast automatic moving object detection and tracking system is
essential to characterize target object and extract spatial and temporal
information for different functionalities including video surveillance systems,
urban traffic monitoring and navigation, robotic. In this dissertation, I
present a collaborative Spatial Pyramid Context-aware moving object detection
and Tracking system. The proposed visual tracker is composed of one master
tracker that usually relies on visual object features and two auxiliary
trackers based on object temporal motion information that will be called
dynamically to assist master tracker. SPCT utilizes image spatial context at
different level to make the video tracking system resistant to occlusion,
background noise and improve target localization accuracy and robustness. We
chose a pre-selected seven-channel complementary features including RGB color,
intensity and spatial pyramid of HoG to encode object color, shape and spatial
layout information. We exploit integral histogram as building block to meet the
demands of real-time performance. A novel fast algorithm is presented to
accurately evaluate spatially weighted local histograms in constant time
complexity using an extension of the integral histogram method. Different
techniques are explored to efficiently compute integral histogram on GPU
architecture and applied for fast spatio-temporal median computations and 3D
face reconstruction texturing. We proposed a multi-component framework based on
semantic fusion of motion information with projected building footprint map to
significantly reduce the false alarm rate in urban scenes with many tall
structures. The experiments on extensive VOTC2016 benchmark dataset and aerial
video confirm that combining complementary tracking cues in an intelligent
fusion framework enables persistent tracking for Full Motion Video and Wide
Aerial Motion Imagery.
Performance Comparison of Algorithms for Movie Rating Estimation
Authors: Alper Kose, Can Kanbak, Noyan Evirgen
Date: 2017-11-05 00:00:00
Tasks: Collaborative Filtering

In this paper, our goal is to compare performances of three different
algorithms to predict the ratings that will be given to movies by potential
users where we are given a user-movie rating matrix based on the past
observations. To this end, we evaluate User-Based Collaborative Filtering,
Iterative Matrix Factorization and Yehuda Koren's Integrated model using
neighborhood and factorization where we use root mean square error (RMSE) as
the performance evaluation metric. In short, we do not observe significant
differences between performances, especially when the complexity increase is
considered. We can conclude that Iterative Matrix Factorization performs fairly
well despite its simplicity.
A Detail Based Method for Linear Full Reference Image Quality Prediction
Authors: Elio D. Di Claudio, Giovanni Jacovitti
Date: 2017-09-10 00:00:00
Tasks: Image Quality Assessment

In this paper, a novel Full Reference method is proposed for image quality
assessment, using the combination of two separate metrics to measure the
perceptually distinct impact of detail losses and of spurious details. To this
purpose, the gradient of the impaired image is locally decomposed as a
predicted version of the original gradient, plus a gradient residual. It is
assumed that the detail attenuation identifies the detail loss, whereas the
gradient residuals describe the spurious details. It turns out that the
perceptual impact of detail losses is roughly linear with the loss of the
positional Fisher information, while the perceptual impact of the spurious
details is roughly proportional to a logarithmic measure of the signal to
residual ratio. The affine combination of these two metrics forms a new index
strongly correlated with the empirical Differential Mean Opinion Score (DMOS)
for a significant class of image impairments, as verified for three independent
popular databases. The method allowed alignment and merging of DMOS data coming
from these different databases to a common DMOS scale by affine
transformations. Unexpectedly, the DMOS scale setting is possible by the
analysis of a single image affected by additive noise.
Strategies for Conceptual Change in Convolutional Neural Networks
Authors: Maarten Grachten, Carlos Eduardo Cancino Chacón
Date: 2017-11-05 00:00:00

A remarkable feature of human beings is their capacity for creative behaviour, referring to their ability to react to problems in ways that are novel, surprising, and useful. Transformational creativity is a form of creativity where the creative behaviour is induced by a transformation of the actor's conceptual space, that is, the representational system with which the actor interprets its environment. In this report, we focus on ways of adapting systems of learned representations as they switch from performing one task to performing another. We describe an experimental comparison of multiple strategies for adaptation of learned features, and evaluate how effectively each of these strategies realizes the adaptation, in terms of the amount of training, and in terms of their ability to cope with restricted availability of training data. We show, among other things, that across handwritten digits, natural images, and classical music, adaptive strategies are systematically more effective than a baseline method that starts learning from scratch.
The Effect of Communication on Noncooperative Multiplayer Multi-Armed Bandit Problems
Authors: Noyan Evirgen, Alper Kose
Date: 2017-11-05 00:00:00
Tasks: Thompson Sampling

We consider decentralized stochastic multi-armed bandit problem with multiple
players in the case of different communication probabilities between players.
Each player makes a decision of pulling an arm without cooperation while aiming
to maximize his or her reward but informs his or her neighbors in the end of
every turn about the arm he or she pulled and the reward he or she got.
Neighbors of players are determined according to an Erdos-Renyi graph with
which is reproduced in the beginning of every turn. We consider i.i.d. rewards
generated by a Bernoulli distribution and assume that players are unaware about
the arms' probability distributions and their mean values. In case of a
collision, we assume that only one of the players who is randomly chosen gets
the reward where the others get zero reward. We study the effects of
connectivity, the degree of communication between players, on the cumulative
regret using well-known algorithms UCB1, epsilon-Greedy and Thompson Sampling.
Learning to Infer Graphics Programs from Hand-Drawn Images
Authors: Kevin Ellis, Daniel Ritchie, Armando Solar-Lezama, Joshua B. Tenenbaum
Date: 2017-07-30 00:00:00
Proceeding: ICLR 2018 1
Tasks: Program Synthesis

We introduce a model that learns to convert simple hand drawings into
graphics programs written in a subset of \LaTeX. The model combines techniques
from deep learning and program synthesis. We learn a convolutional neural
network that proposes plausible drawing primitives that explain an image. These
drawing primitives are like a trace of the set of primitive commands issued by
a graphics program. We learn a model that uses program synthesis techniques to
recover a graphics program from that trace. These programs have constructs like
variable bindings, iterative loops, or simple kinds of conditionals. With a
graphics program in hand, we can correct errors made by the deep network,
measure similarity between drawings by use of similar high-level geometric
structures, and extrapolate drawings. Taken together these results are a step
towards agents that induce useful, human-readable programs from perceptual
input.
Triple Generative Adversarial Nets
Authors: Chongxuan Li, Kun Xu, Jun Zhu, Bo Zhang
Date: 2017-03-07 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Image Generation

Generative Adversarial Nets (GANs) have shown promise in image generation and
semi-supervised learning (SSL). However, existing GANs in SSL have two
problems: (1) the generator and the discriminator (i.e. the classifier) may not
be optimal at the same time; and (2) the generator cannot control the semantics
of the generated samples. The problems essentially arise from the two-player
formulation, where a single discriminator shares incompatible roles of
identifying fake samples and predicting labels and it only estimates the data
without considering the labels. To address the problems, we present triple
generative adversarial net (Triple-GAN), which consists of three players---a
generator, a discriminator and a classifier. The generator and the classifier
characterize the conditional distributions between images and labels, and the
discriminator solely focuses on identifying fake image-label pairs. We design
compatible utilities to ensure that the distributions characterized by the
classifier and the generator both converge to the data distribution. Our
results on various datasets demonstrate that Triple-GAN as a unified model can
simultaneously (1) achieve the state-of-the-art classification results among
deep generative models, and (2) disentangle the classes and styles of the input
and transfer smoothly in the data space via interpolation in the latent space
class-conditionally.
Few-Shot Adversarial Domain Adaptation
Authors: Saeid Motiian, Quinn Jones, Seyed Mehdi Iranmanesh, Gianfranco Doretto
Date: 2017-11-05 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Domain Adaptation, Few-Shot Learning, Handwritten Digit Recognition, Object Recognition

This work provides a framework for addressing the problem of supervised
domain adaptation with deep models. The main idea is to exploit adversarial
learning to learn an embedded subspace that simultaneously maximizes the
confusion between two domains while semantically aligning their embedding. The
supervised setting becomes attractive especially when there are only a few
target data samples that need to be labeled. In this few-shot learning
scenario, alignment and separation of semantic probability distributions is
difficult because of the lack of data. We found that by carefully designing a
training scheme whereby the typical binary adversarial discriminator is
augmented to distinguish between four different classes, it is possible to
effectively address the supervised adaptation problem. In addition, the
approach has a high speed of adaptation, i.e. it requires an extremely low
number of labeled target training samples, even one per category can be
effective. We then extensively compare this approach to the state of the art in
domain adaptation in two experiments: one using datasets for handwritten digit
recognition, and one using datasets for visual object recognition.
Breaking the Nonsmooth Barrier: A Scalable Parallel Method for Composite Optimization
Authors: Fabian Pedregosa, Rémi Leblond, Simon Lacoste-Julien
Date: 2017-07-20 00:00:00
Proceeding: NeurIPS 2017 12

Due to their simplicity and excellent performance, parallel asynchronous
variants of stochastic gradient descent have become popular methods to solve a
wide range of large-scale optimization problems on multi-core architectures.
Yet, despite their practical success, support for nonsmooth objectives is still
lacking, making them unsuitable for many problems of interest in machine
learning, such as the Lasso, group Lasso or empirical risk minimization with
convex constraints.
  In this work, we propose and analyze ProxASAGA, a fully asynchronous sparse
method inspired by SAGA, a variance reduced incremental gradient algorithm. The
proposed method is easy to implement and significantly outperforms the state of
the art on several nonsmooth, large-scale problems. We prove that our method
achieves a theoretical linear speedup with respect to the sequential version
under assumptions on the sparsity of gradients and block-separability of the
proximal term. Empirical benchmarks on a multi-core architecture illustrate
practical speedups of up to 12x on a 20-core machine.
Gradient Descent Can Take Exponential Time to Escape Saddle Points
Authors: Simon S. Du, Chi Jin, Jason D. Lee, Michael. I. Jordan, Barnabas Poczos, Aarti Singh
Date: 2017-05-29 00:00:00
Proceeding: NeurIPS 2017 12

Although gradient descent (GD) almost always escapes saddle points
asymptotically [Lee et al., 2016], this paper shows that even with fairly
natural random initialization schemes and non-pathological functions, GD can be
significantly slowed down by saddle points, taking exponential time to escape.
On the other hand, gradient descent with perturbations [Ge et al., 2015, Jin et
al., 2017] is not slowed down by saddle points - it can find an approximate
local minimizer in polynomial time. This result implies that GD is inherently
slower than perturbed GD, and justifies the importance of adding perturbations
for efficient non-convex optimization. While our focus is theoretical, we also
present experiments that illustrate our theoretical findings.
Hypothesis Transfer Learning via Transformation Functions
Authors: Simon Shaolei Du, Jayanth Koushik, Aarti Singh, Barnabas Poczos
Date: 2016-12-03 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Transfer Learning

We consider the Hypothesis Transfer Learning (HTL) problem where one
incorporates a hypothesis trained on the source domain into the learning
procedure of the target domain. Existing theoretical analysis either only
studies specific algorithms or only presents upper bounds on the generalization
error but not on the excess risk. In this paper, we propose a unified
algorithm-dependent framework for HTL through a novel notion of transformation
function, which characterizes the relation between the source and the target
domains. We conduct a general risk analysis of this framework and in
particular, we show for the first time, if two domains are related, HTL enjoys
faster convergence rates of excess risks for Kernel Smoothing and Kernel Ridge
Regression than those of the classical non-transfer learning settings.
Experiments on real world data demonstrate the effectiveness of our framework.