pwc-archive/papers-with-abstracts huggingface dataset:
Same-different problems strain convolutional neural networks
Authors: Matthew Ricci, Junkyung Kim, Thomas Serre
Date: 2018-02-09 00:00:00
Tasks: Memorization, Visual Reasoning

The robust and efficient recognition of visual relations in images is a
hallmark of biological vision. We argue that, despite recent progress in visual
recognition, modern machine vision algorithms are severely limited in their
ability to learn visual relations. Through controlled experiments, we
demonstrate that visual-relation problems strain convolutional neural networks
(CNNs). The networks eventually break altogether when rote memorization becomes
impossible, as when intra-class variability exceeds network capacity. Motivated
by the comparable success of biological vision, we argue that feedback
mechanisms including attention and perceptual grouping may be the key
computational components underlying abstract visual reasoning.\
Exact Camera Location Recovery by Least Unsquared Deviations
Authors: Gilad Lerman, Yunpeng Shi, Teng Zhang
Date: 2017-09-27 00:00:00

We establish exact recovery for the Least Unsquared Deviations (LUD)
algorithm of Ozyesil and Singer. More precisely, we show that for sufficiently
many cameras with given corrupted pairwise directions, where both camera
locations and pairwise directions are generated by a special probabilistic
model, the LUD algorithm exactly recovers the camera locations with high
probability. A similar exact recovery guarantee was established for the
ShapeFit algorithm by Hand, Lee and Voroninski, but with typically less
corruption.
Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimization
Authors: Blake Woodworth, Jialei Wang, Adam Smith, Brendan Mcmahan, Nathan Srebro
Date: 2018-05-25 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Stochastic Optimization

We suggest a general oracle-based framework that captures different parallel
stochastic optimization settings described by a dependency graph, and derive
generic lower bounds in terms of this graph. We then use the framework and
derive lower bounds for several specific parallel optimization settings,
including delayed updates and parallel processing with intermittent
communication. We highlight gaps between lower and upper bounds on the oracle
complexity, and cases where the "natural" algorithms are not known to be
optimal.
Generating Thematic Chinese Poetry using Conditional Variational Autoencoders with Hybrid Decoders
Authors: Xiaopeng Yang, Xiaowen Lin, Shunda Suo, Ming Li
Date: 2017-11-21 00:00:00
Tasks: Decoder, Rhythm

Computer poetry generation is our first step towards computer writing. Writing must have a theme. The current approaches of using sequence-to-sequence models with attention often produce non-thematic poems. We present a novel conditional variational autoencoder with a hybrid decoder adding the deconvolutional neural networks to the general recurrent neural networks to fully learn topic information via latent variables. This approach significantly improves the relevance of the generated poems by representing each line of the poem not only in a context-sensitive manner but also in a holistic way that is highly related to the given keyword and the learned topic. A proposed augmented word2vec model further improves the rhythm and symmetry. Tests show that the generated poems by our approach are mostly satisfying with regulated rules and consistent themes, and 73.42% of them receive an Overall score no less than 3 (the highest score is 5).
Multiview Learning of Weighted Majority Vote by Bregman Divergence Minimization
Authors: Anil Goyal, Emilie Morvant, Massih-Reza Amini
Date: 2018-05-25 00:00:00
Tasks: Document Classification, Multilingual text classification, Multiview Learning, Text Classification

We tackle the issue of classifier combinations when observations have
multiple views. Our method jointly learns view-specific weighted majority vote
classifiers (i.e. for each view) over a set of base voters, and a second
weighted majority vote classifier over the set of these view-specific weighted
majority vote classifiers. We show that the empirical risk minimization of the
final majority vote given a multiview training set can be cast as the
minimization of Bregman divergences. This allows us to derive a parallel-update
optimization algorithm for learning our multiview model. We empirically study
our algorithm with a particular focus on the impact of the training set size on
the multiview learning results. The experiments show that our approach is able
to overcome the lack of labeled information.
A Quasi-Bayesian Perspective to Online Clustering
Authors: Le Li, Benjamin Guedj, Sébastien Loustau
Date: 2016-02-01 00:00:00
Tasks: Clustering, Online Clustering

When faced with high frequency streams of data, clustering raises theoretical
and algorithmic pitfalls. We introduce a new and adaptive online clustering
algorithm relying on a quasi-Bayesian approach, with a dynamic (i.e.,
time-dependent) estimation of the (unknown and changing) number of clusters. We
prove that our approach is supported by minimax regret bounds. We also provide
an RJMCMC-flavored implementation (called PACBO, see
https://cran.r-project.org/web/packages/PACBO/index.html) for which we give a
convergence guarantee. Finally, numerical experiments illustrate the potential
of our procedure.
COREclust: a new package for a robust and scalable analysis of complex data
Authors: Camille Champion, Anne-Claire Brunet, Jean-Michel Loubes, Laurent Risser
Date: 2018-05-25 00:00:00
Tasks: Clustering, Graph Clustering

In this paper, we present a new R package COREclust dedicated to the
detection of representative variables in high dimensional spaces with a
potentially limited number of observations. Variable sets detection is based on
an original graph clustering strategy denoted CORE-clustering algorithm that
detects CORE-clusters, i.e. variable sets having a user defined size range and
in which each variable is very similar to at least another variable.
Representative variables are then robustely estimate as the CORE-cluster
centers. This strategy is entirely coded in C++ and wrapped by R using the Rcpp
package. A particular effort has been dedicated to keep its algorithmic cost
reasonable so that it can be used on large datasets. After motivating our work,
we will explain the CORE-clustering algorithm as well as a greedy extension of
this algorithm. We will then present how to use it and results obtained on
synthetic and real data.
Psychophysics, Gestalts and Games
Authors: José Lezama, Samy Blusseau, Jean-Michel Morel, Gregory Randall, Rafael Grompone von Gioi
Date: 2018-05-25 00:00:00
Tasks: Human Detection

Many psychophysical studies are dedicated to the evaluation of the human
gestalt detection on dot or Gabor patterns, and to model its dependence on the
pattern and background parameters. Nevertheless, even for these constrained
percepts, psychophysics have not yet reached the challenging prediction stage,
where human detection would be quantitatively predicted by a (generic) model.
On the other hand, Computer Vision has attempted at defining automatic
detection thresholds. This chapter sketches a procedure to confront these two
methodologies inspired in gestaltism. Using a computational quantitative
version of the non-accidentalness principle, we raise the possibility that the
psychophysical and the (older) gestaltist setups, both applicable on dot or
Gabor patterns, find a useful complement in a Turing test. In our perceptual
Turing test, human performance is compared by the scientist to the detection
result given by a computer. This confrontation permits to revive the abandoned
method of gestaltic games. We sketch the elaboration of such a game, where the
subjects of the experiment are confronted to an alignment detection algorithm,
and are invited to draw examples that will fool it. We show that in that way a
more precise definition of the alignment gestalt and of its computational
formulation seems to emerge. Detection algorithms might also be relevant to
more classic psychophysical setups, where they can again play the role of a
Turing test. To a visual experiment where subjects were invited to detect
alignments in Gabor patterns, we associated a single function measuring the
alignment detectability in the form of a number of false alarms (NFA). The
first results indicate that the values of the NFA, as a function of all
simulation parameters, are highly correlated to the human detection. This fact,
that we intend to support by further experiments , might end up confirming that
human alignment detection is the result of a single mechanism.
On the Estimation of Entropy in the FastICA Algorithm
Authors: Elena Issoglio, Paul Smith, Jochen Voss
Date: 2018-05-25 00:00:00
Tasks: Computational Efficiency, Dimensionality Reduction

The fastICA method is a popular dimension reduction technique used to reveal patterns in data. Here we show both theoretically and in practice that the approximations used in fastICA can result in patterns not being successfully recognised. We demonstrate this problem using a two-dimensional example where a clear structure is immediately visible to the naked eye, but where the projection chosen by fastICA fails to reveal this structure. This implies that care is needed when applying fastICA. We discuss how the problem arises and how it is intrinsically connected to the approximations that form the basis of the computational efficiency of fastICA.
Multimodal Sentiment Analysis To Explore the Structure of Emotions
Authors: Anthony Hu, Seth Flaxman
Date: 2018-05-25 00:00:00
Proceeding: ICLR 2018 1
Tasks: Multimodal Sentiment Analysis, Sentence, Sentiment Analysis

We propose a novel approach to multimodal sentiment analysis using deep
neural networks combining visual analysis and natural language processing. Our
goal is different than the standard sentiment analysis goal of predicting
whether a sentence expresses positive or negative sentiment; instead, we aim to
infer the latent emotional state of the user. Thus, we focus on predicting the
emotion word tags attached by users to their Tumblr posts, treating these as
"self-reported emotions." We demonstrate that our multimodal model combining
both text and image features outperforms separate models based solely on either
images or text. Our model's results are interpretable, automatically yielding
sensible word lists associated with emotions. We explore the structure of
emotions implied by our model and compare it to what has been posited in the
psychology literature, and validate our model on a set of images that have been
used in psychology studies. Finally, our work also provides a useful tool for
the growing academic study of images - both photographs and memes - on social
networks.
Adversarial examples from computational constraints
Authors: Sébastien Bubeck, Eric Price, Ilya Razenshteyn
Date: 2018-05-25 00:00:00
Tasks: Binary Classification, Classification, General Classification

Why are classifiers in high dimension vulnerable to "adversarial"
perturbations? We show that it is likely not due to information theoretic
limitations, but rather it could be due to computational constraints.
  First we prove that, for a broad set of classification tasks, the mere
existence of a robust classifier implies that it can be found by a possibly
exponential-time algorithm with relatively few training examples. Then we give
a particular classification task where learning a robust classifier is
computationally intractable. More precisely we construct a binary
classification task in high dimensional space which is (i) information
theoretically easy to learn robustly for large perturbations, (ii) efficiently
learnable (non-robustly) by a simple linear separator, (iii) yet is not
efficiently robustly learnable, even for small perturbations, by any algorithm
in the statistical query (SQ) model. This example gives an exponential
separation between classical learning and robust learning in the statistical
query model. It suggests that adversarial examples may be an unavoidable
byproduct of computational limitations of learning algorithms.
Qunatification of Metabolites in MR Spectroscopic Imaging using Machine Learning
Authors: Dhritiman Das, Eduardo Coello, Rolf F Schulte, Bjoern H. Menze
Date: 2018-05-25 00:00:00
Tasks: BIG-bench Machine Learning

Magnetic Resonance Spectroscopic Imaging (MRSI) is a clinical imaging
modality for measuring tissue metabolite levels in-vivo. An accurate estimation
of spectral parameters allows for better assessment of spectral quality and
metabolite concentration levels. The current gold standard quantification
method is the LCModel - a commercial fitting tool. However, this fails for
spectra having poor signal-to-noise ratio (SNR) or a large number of artifacts.
This paper introduces a framework based on random forest regression for
accurate estimation of the output parameters of a model based analysis of MR
spectroscopy data. The goal of our proposed framework is to learn the spectral
features from a training set comprising of different variations of both
simulated and in-vivo brain spectra and then use this learning for the
subsequent metabolite quantification. Experiments involve training and testing
on simulated and in-vivo human brain spectra. We estimate parameters such as
concentration of metabolites and compare our results with that from the
LCModel.
Maximizing acquisition functions for Bayesian optimization
Authors: James T. Wilson, Frank Hutter, Marc Peter Deisenroth
Date: 2018-05-25 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Bayesian Optimization, global-optimization

Bayesian optimization is a sample-efficient approach to global optimization
that relies on theoretically motivated value heuristics (acquisition functions)
to guide its search process. Fully maximizing acquisition functions produces
the Bayes' decision rule, but this ideal is difficult to achieve since these
functions are frequently non-trivial to optimize. This statement is especially
true when evaluating queries in parallel, where acquisition functions are
routinely non-convex, high-dimensional, and intractable. We first show that
acquisition functions estimated via Monte Carlo integration are consistently
amenable to gradient-based optimization. Subsequently, we identify a common
family of acquisition functions, including EI and UCB, whose properties not
only facilitate but justify use of greedy approaches for their maximization.
Personalized Influence Estimation Technique
Authors: Kumarjit Pathak, Jitin Kapila, Aasheesh Barvey
Date: 2018-05-25 00:00:00
Tasks: Fault Detection, Fraud Detection

Customer Satisfaction is the most important factors in the industry
irrespective of domain. Key Driver Analysis is a common practice in data
science to help the business to evaluate the same. Understanding key features,
which influence the outcome or dependent feature, is highly important in
statistical model building. This helps to eliminate not so important factors
from the model to minimize noise coming from the features, which does not
contribute significantly enough to explain the behavior of the dependent
feature, which we want to predict. Personalized Influence Estimation is a
technique introduced in this paper, which can estimate key factor influence for
individual observations, which contribute most for each observations behavior
pattern based on the dependent class or estimate. Observations can come from
multiple business problem i.e. customers related to satisfaction study,
customer related to Fraud Detection, network devices for Fault detection etc.
It is highly important to understand the cause of issue at each observation
level to take appropriate Individualized action at customer level or device
level etc. This technique is based on joint behavior of the feature dimension
for the specific observation, and relative importance of the feature to
estimate impact. The technique mentioned in this paper is aimed to help
organizations to understand each respondents or observations individual key
contributing factor of Influence. Result of the experiment is really
encouraging and able to justify key reasons for churn for majority of the
sample appropriately
Overcoming the vanishing gradient problem in plain recurrent networks
Authors: Yuhuang Hu, Adrian Huber, Jithendar Anumula, Shih-Chii Liu
Date: 2018-01-18 00:00:00
Proceeding: ICLR 2018 1
Tasks: Permuted-MNIST, Question Answering

Plain recurrent networks greatly suffer from the vanishing gradient problem while Gated Neural Networks (GNNs) such as Long-short Term Memory (LSTM) and Gated Recurrent Unit (GRU) deliver promising results in many sequence learning tasks through sophisticated network designs. This paper shows how we can address this problem in a plain recurrent network by analyzing the gating mechanisms in GNNs. We propose a novel network called the Recurrent Identity Network (RIN) which allows a plain recurrent network to overcome the vanishing gradient problem while training very deep models without the use of gates. We compare this model with IRNNs and LSTMs on multiple sequence modeling benchmarks. The RINs demonstrate competitive performance and converge faster in all tasks. Notably, small RIN models produce 12%--67% higher accuracy on the Sequential and Permuted MNIST datasets and reach state-of-the-art performance on the bAbI question answering dataset.
Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces
Authors: Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, Maël Primet, Joseph Dureau
Date: 2018-05-25 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), BIG-bench Machine Learning, Natural Language Understanding, Speech Recognition, Spoken Language Understanding

This paper presents the machine learning architecture of the Snips Voice
Platform, a software solution to perform Spoken Language Understanding on
microprocessors typical of IoT devices. The embedded inference is fast and
accurate while enforcing privacy by design, as no personal user data is ever
collected. Focusing on Automatic Speech Recognition and Natural Language
Understanding, we detail our approach to training high-performance Machine
Learning models that are small enough to run in real-time on small devices.
Additionally, we describe a data generation procedure that provides sufficient,
high-quality training data without compromising user privacy.
Recursive Neural Network Based Preordering for English-to-Japanese Machine Translation
Authors: Yuki Kawara, Chenhui Chu, Yuki Arase
Date: 2018-05-25 00:00:00
Proceeding: ACL 2018 7
Tasks: Machine Translation, Translation

The word order between source and target languages significantly influences
the translation quality in machine translation. Preordering can effectively
address this problem. Previous preordering methods require a manual feature
design, making language dependent design costly. In this paper, we propose a
preordering method with a recursive neural network that learns features from
raw inputs. Experiments show that the proposed method achieves comparable gain
in translation quality to the state-of-the-art method but without a manual
feature design.
A Generative Model for Inverse Design of Metamaterials
Authors: Zhaocheng Liu, Dayu Zhu, Sean P. Rodrigues, Kyu-Tae Lee, Wenshan Cai
Date: 2018-05-25 00:00:00

The advent of two-dimensional metamaterials in recent years has ushered in a
revolutionary means to manipulate the behavior of light on the nanoscale. The
effective parameters of these architected materials render unprecedented
control over the optical properties of light, thereby eliciting previously
unattainable applications in flat lenses, holographic imaging, and emission
control among others. The design of such structures, to date, has relied on the
expertise of an optical scientist to guide a progression of electromagnetic
simulations that iteratively solve Maxwell's equations until a locally
optimized solution can be attained. In this work, we identify a solution to
circumvent this intuition-guided design by means of a deep learning
architecture. When fed an input set of optical spectra, the constructed
generative network assimilates a candidate pattern from a user-defined dataset
of geometric structures in order to match the input spectra. The generated
metamaterial patterns demonstrate high fidelity, yielding equivalent optical
spectra at an average accuracy of about 0.9. This approach reveals an
opportunity to expedite the discovery and design of metasurfaces for tailored
optical responses in a systematic, inverse-design manner.
Pyramid Attention Network for Semantic Segmentation
Authors: Hanchao Li, Pengfei Xiong, Jie An, Lingxue Wang
Date: 2018-05-25 00:00:00
Tasks: Decoder, Segmentation, Semantic Segmentation, Thermal Image Segmentation

A Pyramid Attention Network(PAN) is proposed to exploit the impact of global
contextual information in semantic segmentation. Different from most existing
works, we combine attention mechanism and spatial pyramid to extract precise
dense features for pixel labeling instead of complicated dilated convolution
and artificially designed decoder networks. Specifically, we introduce a
Feature Pyramid Attention module to perform spatial pyramid attention structure
on high-level output and combining global pooling to learn a better feature
representation, and a Global Attention Upsample module on each decoder layer to
provide global context as a guidance of low-level features to select category
localization details. The proposed approach achieves state-of-the-art
performance on PASCAL VOC 2012 and Cityscapes benchmarks with a new record of
mIoU accuracy 84.0% on PASCAL VOC 2012, while training without COCO dataset.
A Univariate Bound of Area Under ROC
Authors: Siwei Lyu, Yiming Ying
Date: 2018-04-16 00:00:00
Tasks: Binary Classification

Area under ROC (AUC) is an important metric for binary classification and
bipartite ranking problems. However, it is difficult to directly optimizing AUC
as a learning objective, so most existing algorithms are based on optimizing a
surrogate loss to AUC. One significant drawback of these surrogate losses is
that they require pairwise comparisons among training data, which leads to slow
running time and increasing local storage for online learning. In this work, we
describe a new surrogate loss based on a reformulation of the AUC risk, which
does not require pairwise comparison but rankings of the predictions. We
further show that the ranking operation can be avoided, and the learning
objective obtained based on this surrogate enjoys linear complexity in time and
storage. We perform experiments to demonstrate the effectiveness of the online
and batch algorithms for AUC optimization based on the proposed surrogate loss.