pwc-archive/papers-with-abstracts huggingface dataset:
LightRNN: Memory and Computation-Efficient Recurrent Neural Networks
Authors: Xiang Li, Tao Qin, Jian Yang, Tie-Yan Liu
Date: 2016-10-31 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: GPU, Language Modeling, Language Modelling, Machine Translation

Recurrent neural networks (RNNs) have achieved state-of-the-art performances
in many natural language processing tasks, such as language modeling and
machine translation. However, when the vocabulary is large, the RNN model will
become very big (e.g., possibly beyond the memory capacity of a GPU device) and
its training will become very inefficient. In this work, we propose a novel
technique to tackle this challenge. The key idea is to use 2-Component (2C)
shared embedding for word representations. We allocate every word in the
vocabulary into a table, each row of which is associated with a vector, and
each column associated with another vector. Depending on its position in the
table, a word is jointly represented by two components: a row vector and a
column vector. Since the words in the same row share the row vector and the
words in the same column share the column vector, we only need $2 \sqrt{|V|}$
vectors to represent a vocabulary of $|V|$ unique words, which are far less
than the $|V|$ vectors required by existing approaches. Based on the
2-Component shared embedding, we design a new RNN algorithm and evaluate it
using the language modeling task on several benchmark datasets. The results
show that our algorithm significantly reduces the model size and speeds up the
training process, without sacrifice of accuracy (it achieves similar, if not
better, perplexity as compared to state-of-the-art language models).
Remarkably, on the One-Billion-Word benchmark Dataset, our algorithm achieves
comparable perplexity to previous language models, whilst reducing the model
size by a factor of 40-100, and speeding up the training process by a factor of
2. We name our proposed algorithm \emph{LightRNN} to reflect its very small
model size and very high training speed.
Temporal Matrix Completion with Locally Linear Latent Factors for Medical Applications
Authors: Frodo Kin Sun Chan, Andy J. Ma, Pong C. Yuen, Terry Cheuk-Fung Yip, Yee-Kit Tse, Vincent Wai-Sun Wong, Grace Lai-Hung Wong
Date: 2016-10-31 00:00:00
Tasks: Imputation, Matrix Completion, valid

Regular medical records are useful for medical practitioners to analyze and
monitor patient health status especially for those with chronic disease, but
such records are usually incomplete due to unpunctuality and absence of
patients. In order to resolve the missing data problem over time, tensor-based
model is suggested for missing data imputation in recent papers because this
approach makes use of low rank tensor assumption for highly correlated data.
However, when the time intervals between records are long, the data correlation
is not high along temporal direction and such assumption is not valid. To
address this problem, we propose to decompose a matrix with missing data into
its latent factors. Then, the locally linear constraint is imposed on these
factors for matrix completion in this paper. By using a publicly available
dataset and two medical datasets collected from hospital, experimental results
show that the proposed algorithm achieves the best performance by comparing
with the existing methods.
A Survey of Brain Inspired Technologies for Engineering
Authors: Jarryd Son, Amit Kumar Mishra
Date: 2016-10-31 00:00:00
Tasks: Survey

Cognitive engineering is a multi-disciplinary field and hence it is difficult
to find a review article consolidating the leading developments in the field.
The in-credible pace at which technology is advancing pushes the boundaries of
what is achievable in cognitive engineering. There are also differing
approaches to cognitive engineering brought about from the multi-disciplinary
nature of the field and the vastness of possible applications. Thus research
communities require more frequent reviews to keep up to date with the latest
trends. In this paper we shall dis-cuss some of the approaches to cognitive
engineering holistically to clarify the reasoning behind the different
approaches and to highlight their strengths and weaknesses. We shall then show
how developments from seemingly disjointed views could be integrated to achieve
the same goal of creating cognitive machines. By reviewing the major
contributions in the different fields and showing the potential for a combined
approach, this work intends to assist the research community in devising more
unified methods and techniques for developing cognitive machines.
Application Specific Instrumentation (ASIN): A Bio-inspired Paradigm to Instrumentation using recognition before detection
Authors: Amit Kumar Mishra
Date: 2016-10-31 00:00:00

In this paper we present a new scheme for instrumentation, which has been
inspired by the way small mammals sense their environment. We call this scheme
Application Specific Instrumentation (ASIN). A conventional instrumentation
system focuses on gathering as much information about the scene as possible.
This, usually, is a generic system whose data can be used by another system to
take a specific action. ASIN fuses these two steps into one. The major merit of
the proposed scheme is that it uses low resolution sensors and much less
computational overhead to give good performance for a highly specialised
application
Analysis of Nonstationary Time Series Using Locally Coupled Gaussian Processes
Authors: Luca Ambrogioni, Eric Maris
Date: 2016-10-31 00:00:00
Tasks: Gaussian Processes, Time Series, Time Series Analysis

The analysis of nonstationary time series is of great importance in many
scientific fields such as physics and neuroscience. In recent years, Gaussian
process regression has attracted substantial attention as a robust and powerful
method for analyzing time series. In this paper, we introduce a new framework
for analyzing nonstationary time series using locally stationary Gaussian
process analysis with parameters that are coupled through a hidden Markov
model. The main advantage of this framework is that arbitrary complex
nonstationary covariance functions can be obtained by combining simpler
stationary building blocks whose hidden parameters can be estimated in
closed-form. We demonstrate the flexibility of the method by analyzing two
examples of synthetic nonstationary signals: oscillations with time varying
frequency and time series with two dynamical states. Finally, we report an
example application on real magnetoencephalographic measurements of brain
activity.
Breaking the Curse of Dimensionality with Convex Neural Networks
Authors: Francis Bach
Date: 2014-12-30 00:00:00
Tasks: Variable Selection

We consider neural networks with a single hidden layer and non-decreasing
homogeneous activa-tion functions like the rectified linear units. By letting
the number of hidden units grow unbounded and using classical non-Euclidean
regularization tools on the output weights, we provide a detailed theoretical
analysis of their generalization performance, with a study of both the
approximation and the estimation errors. We show in particular that they are
adaptive to unknown underlying linear structures, such as the dependence on the
projection of the input variables onto a low-dimensional subspace. Moreover,
when using sparsity-inducing norms on the input weights, we show that
high-dimensional non-linear variable selection may be achieved, without any
strong assumption regarding the data and with a total number of variables
potentially exponential in the number of ob-servations. In addition, we provide
a simple geometric interpretation to the non-convex problem of addition of a
new unit, which is the core potentially hard computational element in the
framework of learning from continuously many basis functions. We provide simple
conditions for convex relaxations to achieve the same generalization error
bounds, even when constant-factor approxi-mations cannot be found (e.g.,
because it is NP-hard such as for the zero-homogeneous activation function). We
were not able to find strong enough convex relaxations and leave open the
existence or non-existence of polynomial-time algorithms.
Robust Gait Recognition by Integrating Inertial and RGBD Sensors
Authors: Qin Zou, Lihao Ni, Qian Wang, Qingquan Li, Song Wang
Date: 2016-10-31 00:00:00
Tasks: Gait Recognition, Person Identification

Gait has been considered as a promising and unique biometric for person
identification. Traditionally, gait data are collected using either color
sensors, such as a CCD camera, depth sensors, such as a Microsoft Kinect, or
inertial sensors, such as an accelerometer. However, a single type of sensors
may only capture part of the dynamic gait features and make the gait
recognition sensitive to complex covariate conditions, leading to fragile
gait-based person identification systems. In this paper, we propose to combine
all three types of sensors for gait data collection and gait recognition, which
can be used for important identification applications, such as identity
recognition to access a restricted building or area. We propose two new
algorithms, namely EigenGait and TrajGait, to extract gait features from the
inertial data and the RGBD (color and depth) data, respectively. Specifically,
EigenGait extracts general gait dynamics from the accelerometer readings in the
eigenspace and TrajGait extracts more detailed sub-dynamics by analyzing 3D
dense trajectories. Finally, both extracted features are fed into a supervised
classifier for gait recognition and person identification. Experiments on 50
subjects, with comparisons to several other state-of-the-art gait-recognition
approaches, show that the proposed approach can achieve higher recognition
accuracy and robustness.
Universal Correspondence Network
Authors: Christopher B. Choy, JunYoung Gwak, Silvio Savarese, Manmohan Chandraker
Date: 2016-06-11 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Metric Learning, Semantic Similarity, Semantic Textual Similarity

We present a deep learning framework for accurate visual correspondences and
demonstrate its effectiveness for both geometric and semantic matching,
spanning across rigid motions to intra-class shape or appearance variations. In
contrast to previous CNN-based approaches that optimize a surrogate patch
similarity objective, we use deep metric learning to directly learn a feature
space that preserves either geometric or semantic similarity. Our fully
convolutional architecture, along with a novel correspondence contrastive loss
allows faster training by effective reuse of computations, accurate gradient
computation through the use of thousands of examples per image pair and faster
testing with $O(n)$ feed forward passes for $n$ keypoints, instead of $O(n^2)$
for typical patch similarity methods. We propose a convolutional spatial
transformer to mimic patch normalization in traditional features like SIFT,
which is shown to dramatically boost accuracy for semantic correspondences
across intra-class shape variations. Extensive experiments on KITTI, PASCAL,
and CUB-2011 datasets demonstrate the significant advantages of our features
over prior works that use either hand-constructed or learned features.
Experiments with POS Tagging Code-mixed Indian Social Media Text
Authors: Prakash B. Pimpale, Raj Nath Patel
Date: 2016-10-31 00:00:00
Tasks: Part-Of-Speech Tagging, POS, POS Tagging, TAG

This paper presents Centre for Development of Advanced Computing Mumbai's
(CDACM) submission to the NLP Tools Contest on Part-Of-Speech (POS) Tagging For
Code-mixed Indian Social Media Text (POSCMISMT) 2015 (collocated with ICON
2015). We submitted results for Hindi (hi), Bengali (bn), and Telugu (te)
languages mixed with English (en). In this paper, we have described our
approaches to the POS tagging techniques, we exploited for this task. Machine
learning has been used to POS tag the mixed language text. For POS tagging,
distributed representations of words in vector space (word2vec) for feature
extraction and Log-linear models have been tried. We report our work on all
three languages hi, bn, and te mixed with en.
Flexible Models for Microclustering with Application to Entity Resolution
Authors: Giacomo Zanella, Brenda Betancourt, Hanna Wallach, Jeffrey Miller, Abbas Zaidi, Rebecca C. Steorts
Date: 2016-10-31 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Clustering, Entity Resolution

Most generative models for clustering implicitly assume that the number of
data points in each cluster grows linearly with the total number of data
points. Finite mixture models, Dirichlet process mixture models, and
Pitman--Yor process mixture models make this assumption, as do all other
infinitely exchangeable clustering models. However, for some applications, this
assumption is inappropriate. For example, when performing entity resolution,
the size of each cluster should be unrelated to the size of the data set, and
each cluster should contain a negligible fraction of the total number of data
points. These applications require models that yield clusters whose sizes grow
sublinearly with the size of the data set. We address this requirement by
defining the microclustering property and introducing a new class of models
that can exhibit this property. We compare models within this class to two
commonly used clustering models using four entity-resolution data sets.
DPPred: An Effective Prediction Framework with Concise Discriminative Patterns
Authors: Jingbo Shang, Meng Jiang, Wenzhu Tong, Jinfeng Xiao, Jian Peng, Jiawei Han
Date: 2016-10-31 00:00:00
Tasks: Prediction

In the literature, two series of models have been proposed to address
prediction problems including classification and regression. Simple models,
such as generalized linear models, have ordinary performance but strong
interpretability on a set of simple features. The other series, including
tree-based models, organize numerical, categorical and high dimensional
features into a comprehensive structure with rich interpretable information in
the data.
  In this paper, we propose a novel Discriminative Pattern-based Prediction
framework (DPPred) to accomplish the prediction tasks by taking their
advantages of both effectiveness and interpretability. Specifically, DPPred
adopts the concise discriminative patterns that are on the prefix paths from
the root to leaf nodes in the tree-based models. DPPred selects a limited
number of the useful discriminative patterns by searching for the most
effective pattern combination to fit generalized linear models. Extensive
experiments show that in many scenarios, DPPred provides competitive accuracy
with the state-of-the-art as well as the valuable interpretability for
developers and experts. In particular, taking a clinical application dataset as
a case study, our DPPred outperforms the baselines by using only 40 concise
discriminative patterns out of a potentially exponentially large set of
patterns.
Deep Recurrent Regression for Facial Landmark Detection
Authors: Hanjiang Lai, Shengtao Xiao, Yan Pan, Zhen Cui, Jiashi Feng, Chunyan Xu, Jian Yin, Shuicheng Yan
Date: 2015-10-30 00:00:00
Tasks: Facial Landmark Detection, regression

We propose a novel end-to-end deep architecture for face landmark detection,
based on a deep convolutional and deconvolutional network followed by carefully
designed recurrent network structures. The pipeline of this architecture
consists of three parts. Through the first part, we encode an input face image
to resolution-preserved deconvolutional feature maps via a deep network with
stacked convolutional and deconvolutional layers. Then, in the second part, we
estimate the initial coordinates of the facial key points by an additional
convolutional layer on top of these deconvolutional feature maps. In the last
part, by using the deconvolutional feature maps and the initial facial key
points as input, we refine the coordinates of the facial key points by a
recurrent network that consists of multiple Long-Short Term Memory (LSTM)
components. Extensive evaluations on several benchmark datasets show that the
proposed deep architecture has superior performance against the
state-of-the-art methods.
Meta-Path Guided Embedding for Similarity Search in Large-Scale Heterogeneous Information Networks
Authors: Jingbo Shang, Meng Qu, Jialu Liu, Lance M. Kaplan, Jiawei Han, Jian Peng
Date: 2016-10-31 00:00:00

Most real-world data can be modeled as heterogeneous information networks
(HINs) consisting of vertices of multiple types and their relationships. Search
for similar vertices of the same type in large HINs, such as bibliographic
networks and business-review networks, is a fundamental problem with broad
applications. Although similarity search in HINs has been studied previously,
most existing approaches neither explore rich semantic information embedded in
the network structures nor take user's preference as a guidance.
  In this paper, we re-examine similarity search in HINs and propose a novel
embedding-based framework. It models vertices as low-dimensional vectors to
explore network structure-embedded similarity. To accommodate user preferences
at defining similarity semantics, our proposed framework, ESim, accepts
user-defined meta-paths as guidance to learn vertex vectors in a user-preferred
embedding space. Moreover, an efficient and parallel sampling-based
optimization algorithm has been developed to learn embeddings in large-scale
HINs. Extensive experiments on real-world large-scale HINs demonstrate a
significant improvement on the effectiveness of ESim over several
state-of-the-art algorithms as well as its scalability.
A New Distance Measure for Non-Identical Data with Application to Image Classification
Authors: Muthukaruppan Swaminathan, Pankaj Kumar Yadav, Obdulio Piloto, Tobias Sjöblom, Ian Cheong
Date: 2016-10-31 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

Distance measures are part and parcel of many computer vision algorithms. The
underlying assumption in all existing distance measures is that feature
elements are independent and identically distributed. However, in real-world
settings, data generally originate from heterogeneous sources even if they do
possess a common data-generating mechanism. Since these sources are not
identically distributed by necessity, the assumption of identical distribution
is inappropriate. Here, we use statistical analysis to show that feature
elements of local image descriptors are indeed non-identically distributed. To
test the effect of omitting the unified distribution assumption, we created a
new distance measure called the Poisson-Binomial Radius (PBR). PBR is a
bin-to-bin distance which accounts for the dispersion of bin-to-bin
information. PBR's performance was evaluated on twelve benchmark data sets
covering six different classification and recognition applications: texture,
material, leaf, scene, ear biometrics and category-level image classification.
Results from these experiments demonstrate that PBR outperforms
state-of-the-art distance measures for most of the data sets and achieves
comparable performance on the rest, suggesting that accounting for different
distributions in distance measures can improve performance in classification
and recognition tasks.
Recurrent Fully Convolutional Networks for Video Segmentation
Authors: Sepehr Valipour, Mennatullah Siam, Martin Jagersand, Nilanjan Ray
Date: 2016-06-01 00:00:00
Tasks: Change Detection, Image Segmentation, Segmentation, Semantic Segmentation, Video Segmentation, Video Semantic Segmentation

Image segmentation is an important step in most visual tasks. While
convolutional neural networks have shown to perform well on single image
segmentation, to our knowledge, no study has been been done on leveraging
recurrent gated architectures for video segmentation. Accordingly, we propose a
novel method for online segmentation of video sequences that incorporates
temporal data. The network is built from fully convolutional element and
recurrent unit that works on a sliding window over the temporal data. We also
introduce a novel convolutional gated recurrent unit that preserves the spatial
information and reduces the parameters learned. Our method has the advantage
that it can work in an online fashion instead of operating over the whole input
batch of video frames. The network is tested on the change detection dataset,
and proved to have 5.5\% improvement in F-measure over a plain fully
convolutional network for per frame segmentation. It was also shown to have
improvement of 1.4\% for the F-measure compared to our baseline network that we
call FCN 12s.
Active Learning from Imperfect Labelers
Authors: Songbai Yan, Kamalika Chaudhuri, Tara Javidi
Date: 2016-10-30 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Active Learning

We study active learning where the labeler can not only return incorrect
labels but also abstain from labeling. We consider different noise and
abstention conditions of the labeler. We propose an algorithm which utilizes
abstention responses, and analyze its statistical consistency and query
complexity under fairly natural assumptions on the noise and abstention rate of
the labeler. This algorithm is adaptive in a sense that it can automatically
request less queries with a more informed or less noisy labeler. We couple our
algorithm with lower bounds to show that under some technical conditions, it
achieves nearly optimal query complexity.
The Multi-fidelity Multi-armed Bandit
Authors: Kirthevasan Kandasamy, Gautam Dasarathy, Jeff Schneider, Barnabás Póczos
Date: 2016-10-30 00:00:00
Proceeding: NeurIPS 2016 12

We study a variant of the classical stochastic $K$-armed bandit where
observing the outcome of each arm is expensive, but cheap approximations to
this outcome are available. For example, in online advertising the performance
of an ad can be approximated by displaying it for shorter time periods or to
narrower audiences. We formalise this task as a multi-fidelity bandit, where,
at each time step, the forecaster may choose to play an arm at any one of $M$
fidelities. The highest fidelity (desired outcome) expends cost
$\lambda^{(m)}$. The $m^{\text{th}}$ fidelity (an approximation) expends
$\lambda^{(m)} < \lambda^{(M)}$ and returns a biased estimate of the highest
fidelity. We develop MF-UCB, a novel upper confidence bound procedure for this
setting and prove that it naturally adapts to the sequence of available
approximations and costs thus attaining better regret than naive strategies
which ignore the approximations. For instance, in the above online advertising
example, MF-UCB would use the lower fidelities to quickly eliminate suboptimal
ads and reserve the larger expensive experiments on a small set of promising
candidates. We complement this result with a lower bound and show that MF-UCB
is nearly optimal under certain conditions.