pwc-archive/papers-with-abstracts huggingface dataset:
Plan, Attend, Generate: Character-level Neural Machine Translation with Planning in the Decoder
Authors: Caglar Gulcehre, Francis Dutil, Adam Trischler, Yoshua Bengio
Date: 2017-06-13 00:00:00
Tasks: Decoder, Machine Translation, Translation

We investigate the integration of a planning mechanism into an
encoder-decoder architecture with an explicit alignment for character-level
machine translation. We develop a model that plans ahead when it computes
alignments between the source and target sequences, constructing a matrix of
proposed future alignments and a commitment vector that governs whether to
follow or recompute the plan. This mechanism is inspired by the strategic
attentive reader and writer (STRAW) model. Our proposed model is end-to-end
trainable with fully differentiable operations. We show that it outperforms a
strong baseline on three character-level decoder neural machine translation on
WMT'15 corpus. Our analysis demonstrates that our model can compute
qualitatively intuitive alignments and achieves superior performance with fewer
parameters.
Effects of Additional Data on Bayesian Clustering
Authors: Keisuke Yamazaki
Date: 2016-07-13 00:00:00
Tasks: Clustering, Transfer Learning

Hierarchical probabilistic models, such as mixture models, are used for
cluster analysis. These models have two types of variables: observable and
latent. In cluster analysis, the latent variable is estimated, and it is
expected that additional information will improve the accuracy of the
estimation of the latent variable. Many proposed learning methods are able to
use additional data; these include semi-supervised learning and transfer
learning. However, from a statistical point of view, a complex probabilistic
model that encompasses both the initial and additional data might be less
accurate due to having a higher-dimensional parameter. The present paper
presents a theoretical analysis of the accuracy of such a model and clarifies
which factor has the greatest effect on its accuracy, the advantages of
obtaining additional data, and the disadvantages of increasing the complexity.
Toward Goal-Driven Neural Network Models for the Rodent Whisker-Trigeminal System
Authors: Chengxu Zhuang, Jonas Kubilius, Mitra Hartmann, Daniel Yamins
Date: 2017-06-23 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Decision Making

In large part, rodents see the world through their whiskers, a powerful
tactile sense enabled by a series of brain areas that form the
whisker-trigeminal system. Raw sensory data arrives in the form of mechanical
input to the exquisitely sensitive, actively-controllable whisker array, and is
processed through a sequence of neural circuits, eventually arriving in
cortical regions that communicate with decision-making and memory areas.
Although a long history of experimental studies has characterized many aspects
of these processing stages, the computational operations of the
whisker-trigeminal system remain largely unknown. In the present work, we take
a goal-driven deep neural network (DNN) approach to modeling these
computations. First, we construct a biophysically-realistic model of the rat
whisker array. We then generate a large dataset of whisker sweeps across a wide
variety of 3D objects in highly-varying poses, angles, and speeds. Next, we
train DNNs from several distinct architectural families to solve a shape
recognition task in this dataset. Each architectural family represents a
structurally-distinct hypothesis for processing in the whisker-trigeminal
system, corresponding to different ways in which spatial and temporal
information can be integrated. We find that most networks perform poorly on the
challenging shape recognition task, but that specific architectures from
several families can achieve reasonable performance levels. Finally, we show
that Representational Dissimilarity Matrices (RDMs), a tool for comparing
population codes between neural systems, can separate these higher-performing
networks with data of a type that could plausibly be collected in a
neurophysiological or imaging experiment. Our results are a proof-of-concept
that goal-driven DNN networks of the whisker-trigeminal system are potentially
within reach.
Efficient Approximate Solutions to Mutual Information Based Global Feature Selection
Authors: Hemanth Venkateswara, Prasanth Lade, Binbin Lin, Jieping Ye, Sethuraman Panchanathan
Date: 2017-06-23 00:00:00
Tasks: feature selection

Mutual Information (MI) is often used for feature selection when developing
classifier models. Estimating the MI for a subset of features is often
intractable. We demonstrate, that under the assumptions of conditional
independence, MI between a subset of features can be expressed as the
Conditional Mutual Information (CMI) between pairs of features. But selecting
features with the highest CMI turns out to be a hard combinatorial problem. In
this work, we have applied two unique global methods, Truncated Power Method
(TPower) and Low Rank Bilinear Approximation (LowRank), to solve the feature
selection problem. These algorithms provide very good approximations to the
NP-hard CMI based feature selection problem. We experimentally demonstrate the
effectiveness of these procedures across multiple datasets and compare them
with existing MI based global and iterative feature selection procedures.
Multiresolution Match Kernels for Gesture Video Classification
Authors: Hemanth Venkateswara, Vineeth N. Balasubramanian, Prasanth Lade, Sethuraman Panchanathan
Date: 2017-06-23 00:00:00
Tasks: Classification, General Classification, Video Classification

The emergence of depth imaging technologies like the Microsoft Kinect has
renewed interest in computational methods for gesture classification based on
videos. For several years now, researchers have used the Bag-of-Features (BoF)
as a primary method for generation of feature vectors from video data for
recognition of gestures. However, the BoF method is a coarse representation of
the information in a video, which often leads to poor similarity measures
between videos. Besides, when features extracted from different spatio-temporal
locations in the video are pooled to create histogram vectors in the BoF
method, there is an intrinsic loss of their original locations in space and
time. In this paper, we propose a new Multiresolution Match Kernel (MMK) for
video classification, which can be considered as a generalization of the BoF
method. We apply this procedure to hand gesture classification based on RGB-D
videos of the American Sign Language(ASL) hand gestures and our results show
promise and usefulness of this new method.
On Mixed Memberships and Symmetric Nonnegative Matrix Factorizations
Authors: Xueyu Mao, Purnamrita Sarkar, Deepayan Chakrabarti
Date: 2016-07-01 00:00:00
Proceeding: ICML 2017 8

The problem of finding overlapping communities in networks has gained much
attention recently. Optimization-based approaches use non-negative matrix
factorization (NMF) or variants, but the global optimum cannot be provably
attained in general. Model-based approaches, such as the popular
mixed-membership stochastic blockmodel or MMSB (Airoldi et al., 2008), use
parameters for each node to specify the overlapping communities, but standard
inference techniques cannot guarantee consistency. We link the two approaches,
by (a) establishing sufficient conditions for the symmetric NMF optimization to
have a unique solution under MMSB, and (b) proposing a computationally
efficient algorithm called GeoNMF that is provably optimal and hence consistent
for a broad parameter regime. We demonstrate its accuracy on both simulated and
real-world datasets.
Model Selection with Nonlinear Embedding for Unsupervised Domain Adaptation
Authors: Hemanth Venkateswara, Shayok Chakraborty, Troy McDaniel, Sethuraman Panchanathan
Date: 2017-06-23 00:00:00
Tasks: Domain Adaptation, General Classification, Model Selection, Unsupervised Domain Adaptation

Domain adaptation deals with adapting classifiers trained on data from a
source distribution, to work effectively on data from a target distribution. In
this paper, we introduce the Nonlinear Embedding Transform (NET) for
unsupervised domain adaptation. The NET reduces cross-domain disparity through
nonlinear domain alignment. It also embeds the domain-aligned data such that
similar data points are clustered together. This results in enhanced
classification. To determine the parameters in the NET model (and in other
unsupervised domain adaptation models), we introduce a validation procedure by
sampling source data points that are similar in distribution to the target
data. We test the NET and the validation procedure using popular image datasets
and compare the classification results across competitive procedures for
unsupervised domain adaptation.
Coupled Support Vector Machines for Supervised Domain Adaptation
Authors: Hemanth Venkateswara, Prasanth Lade, Jieping Ye, Sethuraman Panchanathan
Date: 2017-06-22 00:00:00
Tasks: Domain Adaptation

Popular domain adaptation (DA) techniques learn a classifier for the target
domain by sampling relevant data points from the source and combining it with
the target data. We present a Support Vector Machine (SVM) based supervised DA
technique, where the similarity between source and target domains is modeled as
the similarity between their SVM decision boundaries. We couple the source and
target SVMs and reduce the model to a standard single SVM. We test the
Coupled-SVM on multiple datasets and compare our results with other popular SVM
based DA approaches.
Nonlinear Embedding Transform for Unsupervised Domain Adaptation
Authors: Hemanth Venkateswara, Shayok Chakraborty, Sethuraman Panchanathan
Date: 2017-06-22 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

The problem of domain adaptation (DA) deals with adapting classifier models
trained on one data distribution to different data distributions. In this
paper, we introduce the Nonlinear Embedding Transform (NET) for unsupervised DA
by combining domain alignment along with similarity-based embedding. We also
introduce a validation procedure to estimate the model parameters for the NET
algorithm using the source data. Comprehensive evaluations on multiple vision
datasets demonstrate that the NET algorithm outperforms existing competitive
procedures for unsupervised DA.
Deep Hashing Network for Unsupervised Domain Adaptation
Authors: Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, Sethuraman Panchanathan
Date: 2017-06-22 00:00:00
Proceeding: CVPR 2017 7
Tasks: Deep Hashing, Domain Adaptation, Retrieval, Transfer Learning, Unsupervised Domain Adaptation

In recent years, deep neural networks have emerged as a dominant machine
learning tool for a wide variety of application domains. However, training a
deep neural network requires a large amount of labeled data, which is an
expensive process in terms of time, labor and human expertise. Domain
adaptation or transfer learning algorithms address this challenge by leveraging
labeled data in a different, but related source domain, to develop a model for
the target domain. Further, the explosive growth of digital data has posed a
fundamental challenge concerning its storage and retrieval. Due to its storage
and retrieval efficiency, recent years have witnessed a wide application of
hashing in a variety of computer vision applications. In this paper, we first
introduce a new dataset, Office-Home, to evaluate domain adaptation algorithms.
The dataset contains images of a variety of everyday objects from multiple
domains. We then propose a novel deep learning framework that can exploit
labeled source data and unlabeled target data to learn informative hash codes,
to accurately classify unseen target data. To the best of our knowledge, this
is the first research effort to exploit the feature learning capabilities of
deep neural networks to learn representative hash codes to address the domain
adaptation problem. Our extensive empirical studies on multiple transfer tasks
corroborate the usefulness of the framework in learning efficient hash codes
which outperform existing competitive baselines for unsupervised domain
adaptation.
Neural Machine Translation with Gumbel-Greedy Decoding
Authors: Jiatao Gu, Daniel Jiwoong Im, Victor O. K. Li
Date: 2017-06-22 00:00:00
Tasks: Heuristic Search, Machine Translation, Translation

Previous neural machine translation models used some heuristic search
algorithms (e.g., beam search) in order to avoid solving the maximum a
posteriori problem over translation sentences at test time. In this paper, we
propose the Gumbel-Greedy Decoding which trains a generative network to predict
translation under a trained model. We solve such a problem using the
Gumbel-Softmax reparameterization, which makes our generative network
differentiable and trainable through standard stochastic gradient methods. We
empirically demonstrate that our proposed model is effective for generating
sequences of discrete words.
Clustering with Noisy Queries
Authors: Arya Mazumdar, Barna Saha
Date: 2017-06-22 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Clustering, Entity Resolution, Stochastic Block Model

In this paper, we initiate a rigorous theoretical study of clustering with
noisy queries (or a faulty oracle). Given a set of $n$ elements, our goal is to
recover the true clustering by asking minimum number of pairwise queries to an
oracle. Oracle can answer queries of the form : "do elements $u$ and $v$ belong
to the same cluster?" -- the queries can be asked interactively (adaptive
queries), or non-adaptively up-front, but its answer can be erroneous with
probability $p$. In this paper, we provide the first information theoretic
lower bound on the number of queries for clustering with noisy oracle in both
situations. We design novel algorithms that closely match this query complexity
lower bound, even when the number of clusters is unknown. Moreover, we design
computationally efficient algorithms both for the adaptive and non-adaptive
settings. The problem captures/generalizes multiple application scenarios. It
is directly motivated by the growing body of work that use crowdsourcing for
{\em entity resolution}, a fundamental and challenging data mining task aimed
to identify all records in a database referring to the same entity. Here crowd
represents the noisy oracle, and the number of queries directly relates to the
cost of crowdsourcing. Another application comes from the problem of {\em sign
edge prediction} in social network, where social interactions can be both
positive and negative, and one must identify the sign of all pair-wise
interactions by querying a few pairs. Furthermore, clustering with noisy oracle
is intimately connected to correlation clustering, leading to improvement
therein. Finally, it introduces a new direction of study in the popular {\em
stochastic block model} where one has an incomplete stochastic block model
matrix to recover the clusters.
Fractal dimension analysis for automatic morphological galaxy classification
Authors: Jorge de la Calleja, Elsa M. de la Calleja, Hugo Jair Escalante
Date: 2017-06-22 00:00:00
Tasks: Classification, General Classification

In this report we present experimental results using
\emph{Haussdorf-Besicovich} fractal dimension for performing morphological
galaxy classification. The fractal dimension is a topological, structural and
spatial property that give us information about the space were an object lives.
We have calculated the fractal dimension value of the main types of galaxies:
ellipticals, spirals and irregulars; and we use it as a feature for classifying
them. Also, we have performed an image analysis process in order to standardize
the galaxy images, and we have used principal component analysis to obtain the
main attributes in the images. Galaxy classification was performed using
machine learning algorithms: C4.5, k-nearest neighbors, random forest and
support vector machines. Preliminary experimental results using 10-fold
cross-validation show that fractal dimension helps to improve classification,
with over 88 per cent accuracy for elliptical galaxies, 100 per cent accuracy
for spiral galaxies and over 40 per cent for irregular galaxies.
Inter-Session Modeling for Session-Based Recommendation
Authors: Massimiliano Ruocco, Ole Steinar Lillestøl Skrede, Helge Langseth
Date: 2017-06-22 00:00:00
Tasks: Recommendation Systems, Session-Based Recommendations

In recent years, research has been done on applying Recurrent Neural Networks
(RNNs) as recommender systems. Results have been promising, especially in the
session-based setting where RNNs have been shown to outperform state-of-the-art
models. In many of these experiments, the RNN could potentially improve the
recommendations by utilizing information about the user's past sessions, in
addition to its own interactions in the current session. A problem for
session-based recommendation, is how to produce accurate recommendations at the
start of a session, before the system has learned much about the user's current
interests. We propose a novel approach that extends a RNN recommender to be
able to process the user's recent sessions, in order to improve
recommendations. This is done by using a second RNN to learn from recent
sessions, and predict the user's interest in the current session. By feeding
this information to the original RNN, it is able to improve its
recommendations. Our experiments on two different datasets show that the
proposed approach can significantly improve recommendations throughout the
sessions, compared to a single RNN working only on the current session. The
proposed model especially improves recommendations at the start of sessions,
and is therefore able to deal with the cold start problem within sessions.
Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets
Authors: Maciej Wielgosz, Andrzej Skoczeń, Matej Mertik
Date: 2016-11-18 00:00:00
Tasks: Time Series, Time Series Analysis

The superconducting LHC magnets are coupled with an electronic monitoring
system which records and analyses voltage time series reflecting their
performance. A currently used system is based on a range of preprogrammed
triggers which launches protection procedures when a misbehavior of the magnets
is detected. All the procedures used in the protection equipment were designed
and implemented according to known working scenarios of the system and are
updated and monitored by human operators.
  This paper proposes a novel approach to monitoring and fault protection of
the Large Hadron Collider (LHC) superconducting magnets which employs
state-of-the-art Deep Learning algorithms. Consequently, the authors of the
paper decided to examine the performance of LSTM recurrent neural networks for
modeling of voltage time series of the magnets. In order to address this
challenging task different network architectures and hyper-parameters were used
to achieve the best possible performance of the solution. The regression
results were measured in terms of RMSE for different number of future steps and
history length taken into account for the prediction. The best result of
RMSE=0.00104 was obtained for a network of 128 LSTM cells within the internal
layer and 16 steps history buffer.
Horseshoe Regularization for Feature Subset Selection
Authors: Anindya Bhadra, Jyotishka Datta, Nicholas G. Polson, Brandon Willard
Date: 2017-02-23 00:00:00
Tasks: compressed sensing, Uncertainty Quantification

Feature subset selection arises in many high-dimensional applications of
statistics, such as compressed sensing and genomics. The $\ell_0$ penalty is
ideal for this task, the caveat being it requires the NP-hard combinatorial
evaluation of all models. A recent area of considerable interest is to develop
efficient algorithms to fit models with a non-convex $\ell_\gamma$ penalty for
$\gamma\in (0,1)$, which results in sparser models than the convex $\ell_1$ or
lasso penalty, but is harder to fit. We propose an alternative, termed the
horseshoe regularization penalty for feature subset selection, and demonstrate
its theoretical and computational advantages. The distinguishing feature from
existing non-convex optimization approaches is a full probabilistic
representation of the penalty as the negative of the logarithm of a suitable
prior, which in turn enables efficient expectation-maximization and local
linear approximation algorithms for optimization and MCMC for uncertainty
quantification. In synthetic and real data, the resulting algorithms provide
better statistical performance, and the computation requires a fraction of time
of state-of-the-art non-convex solvers.
Revised Note on Learning Algorithms for Quadratic Assignment with Graph Neural Networks
Authors: Alex Nowak, Soledad Villar, Afonso S. Bandeira, Joan Bruna
Date: 2017-06-22 00:00:00

Inverse problems correspond to a certain type of optimization problems
formulated over appropriate input distributions. Recently, there has been a
growing interest in understanding the computational hardness of these
optimization problems, not only in the worst case, but in an average-complexity
sense under this same input distribution.
  In this revised note, we are interested in studying another aspect of
hardness, related to the ability to learn how to solve a problem by simply
observing a collection of previously solved instances. These 'planted
solutions' are used to supervise the training of an appropriate predictive
model that parametrizes a broad class of algorithms, with the hope that the
resulting model will provide good accuracy-complexity tradeoffs in the average
sense.
  We illustrate this setup on the Quadratic Assignment Problem, a fundamental
problem in Network Science. We observe that data-driven models based on Graph
Neural Networks offer intriguingly good performance, even in regimes where
standard relaxation based techniques appear to suffer.
Deep Transfer Learning: A new deep learning glitch classification method for advanced LIGO
Authors: Daniel George, Hongyu Shen, E. A. Huerta
Date: 2017-06-22 00:00:00
Tasks: Clustering, General Classification, Object Recognition, Sensitivity, Time Series Analysis, Transfer Learning

The exquisite sensitivity of the advanced LIGO detectors has enabled the
detection of multiple gravitational wave signals. The sophisticated design of
these detectors mitigates the effect of most types of noise. However, advanced
LIGO data streams are contaminated by numerous artifacts known as glitches:
non-Gaussian noise transients with complex morphologies. Given their high rate
of occurrence, glitches can lead to false coincident detections, obscure and
even mimic gravitational wave signals. Therefore, successfully characterizing
and removing glitches from advanced LIGO data is of utmost importance. Here, we
present the first application of Deep Transfer Learning for glitch
classification, showing that knowledge from deep learning algorithms trained
for real-world object recognition can be transferred for classifying glitches
in time-series based on their spectrogram images. Using the Gravity Spy
dataset, containing hand-labeled, multi-duration spectrograms obtained from
real LIGO data, we demonstrate that this method enables optimal use of very
deep convolutional neural networks for classification given small training
datasets, significantly reduces the time for training the networks, and
achieves state-of-the-art accuracy above 98.8%, with perfect precision-recall
on 8 out of 22 classes. Furthermore, new types of glitches can be classified
accurately given few labeled examples with this technique. Once trained via
transfer learning, we show that the convolutional neural networks can be
truncated and used as excellent feature extractors for unsupervised clustering
methods to identify new classes based on their morphology, without any labeled
examples. Therefore, this provides a new framework for dynamic glitch
classification for gravitational wave detectors, which are expected to
encounter new types of noise as they undergo gradual improvements to attain
design sensitivity.