pwc-archive/papers-with-abstracts huggingface dataset:
Distributed Adaptive LMF Algorithm for Sparse Parameter Estimation in Gaussian Mixture Noise
Authors: Mojtaba Hajiabadi
Date: 2015-12-08 00:00:00
Tasks: parameter estimation

A distributed adaptive algorithm for estimation of sparse unknown parameters
in the presence of nonGaussian noise is proposed in this paper based on
normalized least mean fourth (NLMF) criterion. At the first step, local
adaptive NLMF algorithm is modified by zero norm in order to speed up the
convergence rate and also to reduce the steady state error power in sparse
conditions. Then, the proposed algorithm is extended for distributed scenario
in which more improvement in estimation performance is achieved due to
cooperation of local adaptive filters. Simulation results show the superiority
of the proposed algorithm in comparison with conventional NLMF algorithms.
Selective Sequential Model Selection
Authors: William Fithian, Jonathan Taylor, Robert Tibshirani, Ryan Tibshirani
Date: 2015-12-08 00:00:00
Tasks: model, Model Selection, regression

Many model selection algorithms produce a path of fits specifying a sequence
of increasingly complex models. Given such a sequence and the data used to
produce them, we consider the problem of choosing the least complex model that
is not falsified by the data. Extending the selected-model tests of Fithian et
al. (2014), we construct p-values for each step in the path which account for
the adaptive selection of the model path using the data. In the case of linear
regression, we propose two specific tests, the max-t test for forward stepwise
regression (generalizing a proposal of Buja and Brown (2014)), and the
next-entry test for the lasso. These tests improve on the power of the
saturated-model test of Tibshirani et al. (2014), sometimes dramatically. In
addition, our framework extends beyond linear regression to a much more general
class of parametric and nonparametric model selection problems.
  To select a model, we can feed our single-step p-values as inputs into
sequential stopping rules such as those proposed by G'Sell et al. (2013) and Li
and Barber (2015), achieving control of the familywise error rate or false
discovery rate (FDR) as desired. The FDR-controlling rules require the null
p-values to be independent of each other and of the non-null p-values, a
condition not satisfied by the saturated-model p-values of Tibshirani et al.
(2014). We derive intuitive and general sufficient conditions for independence,
and show that our proposed constructions yield independent p-values.
Deep Learning for Single and Multi-Session i-Vector Speaker Recognition
Authors: Omid Ghahabi, Javier Hernando
Date: 2015-12-08 00:00:00
Tasks: Speaker Recognition, speech-recognition, Speech Recognition

The promising performance of Deep Learning (DL) in speech recognition has
motivated the use of DL in other speech technology applications such as speaker
recognition. Given i-vectors as inputs, the authors proposed an impostor
selection algorithm and a universal model adaptation process in a hybrid system
based on Deep Belief Networks (DBN) and Deep Neural Networks (DNN) to
discriminatively model each target speaker. In order to have more insight into
the behavior of DL techniques in both single and multi-session speaker
enrollment tasks, some experiments have been carried out in this paper in both
scenarios. Additionally, the parameters of the global model, referred to as
universal DBN (UDBN), are normalized before adaptation. UDBN normalization
facilitates training DNNs specifically with more than one hidden layer.
Experiments are performed on the NIST SRE 2006 corpus. It is shown that the
proposed impostor selection algorithm and UDBN adaptation process enhance the
performance of conventional DNNs 8-20 % and 16-20 % in terms of EER for the
single and multi-session tasks, respectively. In both scenarios, the proposed
architectures outperform the baseline systems obtaining up to 17 % reduction in
EER.
Gibbs-type Indian buffet processes
Authors: Creighton Heaukulani, Daniel M. Roy
Date: 2015-12-08 00:00:00
Tasks: Vocal Bursts Type Prediction

We investigate a class of feature allocation models that generalize the Indian buffet process and are parameterized by Gibbs-type random measures. Two existing classes are contained as special cases: the original two-parameter Indian buffet process, corresponding to the Dirichlet process, and the stable (or three-parameter) Indian buffet process, corresponding to the Pitman--Yor process. Asymptotic behavior of the Gibbs-type partitions, such as power laws holding for the number of latent clusters, translates into analogous characteristics for this class of Gibbs-type feature allocation models. Despite containing several different distinct subclasses, the properties of Gibbs-type partitions allow us to develop a black-box procedure for posterior inference within any subclass of models. Through numerical experiments, we compare and contrast a few of these subclasses and highlight the utility of varying power-law behaviors in the latent features.
Alternating direction method of multipliers for penalized zero-variance discriminant analysis
Authors: Brendan P. W. Ames, Mingyi Hong
Date: 2014-01-21 00:00:00
Tasks: feature selection, General Classification, Time Series, Time Series Analysis, Time Series Classification

We consider the task of classification in the high dimensional setting where
the number of features of the given data is significantly greater than the
number of observations. To accomplish this task, we propose a heuristic, called
sparse zero-variance discriminant analysis (SZVD), for simultaneously
performing linear discriminant analysis and feature selection on high
dimensional data. This method combines classical zero-variance discriminant
analysis, where discriminant vectors are identified in the null space of the
sample within-class covariance matrix, with penalization applied to induce
sparse structures in the resulting vectors. To approximately solve the
resulting nonconvex problem, we develop a simple algorithm based on the
alternating direction method of multipliers. Further, we show that this
algorithm is applicable to a larger class of penalized generalized eigenvalue
problems, including a particular relaxation of the sparse principal component
analysis problem. Finally, we establish theoretical guarantees for convergence
of our algorithm to stationary points of the original nonconvex problem, and
empirically demonstrate the effectiveness of our heuristic for classifying
simulated data and data drawn from applications in time-series classification.
Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
Authors: Grégoire Montavon, Sebastian Bach, Alexander Binder, Wojciech Samek, Klaus-Robert Müller
Date: 2015-12-08 00:00:00
Tasks: Action Recognition, Classification, General Classification, image-classification, Image Classification, Temporal Action Localization

Nonlinear methods such as Deep Neural Networks (DNNs) are the gold standard
for various challenging machine learning problems, e.g., image classification,
natural language processing or human action recognition. Although these methods
perform impressively well, they have a significant disadvantage, the lack of
transparency, limiting the interpretability of the solution and thus the scope
of application in practice. Especially DNNs act as black boxes due to their
multilayer nonlinear structure. In this paper we introduce a novel methodology
for interpreting generic multilayer neural networks by decomposing the network
classification decision into contributions of its input elements. Although our
focus is on image classification, the method is applicable to a broad set of
input data, learning tasks and network architectures. Our method is based on
deep Taylor decomposition and efficiently utilizes the structure of the network
by backpropagating the explanations from the output to the input layer. We
evaluate the proposed method empirically on the MNIST and ILSVRC data sets.
Convolutional Neural Networks over Tree Structures for Programming Language Processing
Authors: Lili Mou, Ge Li, Lu Zhang, Tao Wang, Zhi Jin
Date: 2014-09-18 00:00:00
Tasks: Sentence

Programming language processing (similar to natural language processing) is a
hot research topic in the field of software engineering; it has also aroused
growing interest in the artificial intelligence community. However, different
from a natural language sentence, a program contains rich, explicit, and
complicated structural information. Hence, traditional NLP models may be
inappropriate for programs. In this paper, we propose a novel tree-based
convolutional neural network (TBCNN) for programming language processing, in
which a convolution kernel is designed over programs' abstract syntax trees to
capture structural information. TBCNN is a generic architecture for programming
language processing; our experiments show its effectiveness in two different
program analysis tasks: classifying programs according to functionality, and
detecting code snippets of certain patterns. TBCNN outperforms baseline
methods, including several neural models for NLP.
A Novel Paradigm for Calculating Ramsey Number via Artificial Bee Colony Algorithm
Authors: Wei-Hao Mao, Fei Gao, Yi-Jin Dong, Wen-Ming Li
Date: 2015-12-05 00:00:00

The Ramsey number is of vital importance in Ramsey's theorem. This paper
proposed a novel methodology for constructing Ramsey graphs about R(3,10),
which uses Artificial Bee Colony optimization(ABC) to raise the lower bound of
Ramsey number R(3,10). The r(3,10)-graph contains two limitations, that is,
neither complete graphs of order 3 nor independent sets of order 10. To resolve
these limitations, a special mathematical model is put in the paradigm to
convert the problems into discrete optimization whose smaller minimizers are
correspondent to bigger lower bound as approximation of inf R(3,10). To
demonstrate the potential of the proposed method, simulations are done to to
minimize the amount of these two types of graphs. For the first time, four
r(3,9,39) graphs with best approximation for inf R(3,10) are reported in
simulations to support the current lower bound for R(3,10). The experiments'
results show that the proposed paradigm for Ramsey number's calculation driven
by ABC is a successful method with the advantages of high precision and
robustness.
Online Gradient Descent in Function Space
Authors: Changbo Zhu, Huan Xu
Date: 2015-12-08 00:00:00
Tasks: BIG-bench Machine Learning

In many problems in machine learning and operations research, we need to
optimize a function whose input is a random variable or a probability density
function, i.e. to solve optimization problems in an infinite dimensional space.
On the other hand, online learning has the advantage of dealing with streaming
examples, and better model a changing environ- ment. In this paper, we extend
the celebrated online gradient descent algorithm to Hilbert spaces (function
spaces), and analyze the convergence guarantee of the algorithm. Finally, we
demonstrate that our algorithms can be useful in several important problems.
Online Crowdsourcing
Authors: Changbo Zhu, Huan Xu, Shuicheng Yan
Date: 2015-12-08 00:00:00

With the success of modern internet based platform, such as Amazon Mechanical
Turk, it is now normal to collect a large number of hand labeled samples from
non-experts. The Dawid- Skene algorithm, which is based on Expectation-
Maximization update, has been widely used for inferring the true labels from
noisy crowdsourced labels. However, Dawid-Skene scheme requires all the data to
perform each EM iteration, and can be infeasible for streaming data or large
scale data. In this paper, we provide an online version of Dawid- Skene
algorithm that only requires one data frame for each iteration. Further, we
prove that under mild conditions, the online Dawid-Skene scheme with projection
converges to a stationary point of the marginal log-likelihood of the observed
data. Our experiments demonstrate that the online Dawid- Skene scheme achieves
state of the art performance comparing with other methods based on the Dawid-
Skene scheme.
Sentiment of Emojis
Authors: Petra Kralj Novak, Jasmina Smailović, Borut Sluban, Igor Mozetič
Date: 2015-09-25 00:00:00
Tasks: Sentiment Analysis

There is a new generation of emoticons, called emojis, that is increasingly
being used in mobile communications and social media. In the past two years,
over ten billion emojis were used on Twitter. Emojis are Unicode graphic
symbols, used as a shorthand to express concepts and ideas. In contrast to the
small number of well-known emoticons that carry clear emotional contents, there
are hundreds of emojis. But what are their emotional contents? We provide the
first emoji sentiment lexicon, called the Emoji Sentiment Ranking, and draw a
sentiment map of the 751 most frequently used emojis. The sentiment of the
emojis is computed from the sentiment of the tweets in which they occur. We
engaged 83 human annotators to label over 1.6 million tweets in 13 European
languages by the sentiment polarity (negative, neutral, or positive). About 4%
of the annotated tweets contain emojis. The sentiment analysis of the emojis
allows us to draw several interesting conclusions. It turns out that most of
the emojis are positive, especially the most popular ones. The sentiment
distribution of the tweets with and without emojis is significantly different.
The inter-annotator agreement on the tweets with emojis is higher. Emojis tend
to occur at the end of the tweets, and their sentiment polarity increases with
the distance. We observe no significant differences in the emoji rankings
between the 13 languages and the Emoji Sentiment Ranking. Consequently, we
propose our Emoji Sentiment Ranking as a European language-independent resource
for automated sentiment analysis. Finally, the paper provides a formalization
of sentiment and a novel visualization in the form of a sentiment bar.
Towards the Application of Linear Programming Methods For Multi-Camera Pose Estimation
Authors: Masoud Aghamohamadian-Sharbaf, Ahmadreza Heravi, Hamid-Reza Pourreza
Date: 2015-12-08 00:00:00
Tasks: Camera Pose Estimation, Pose Estimation

We presented a separation based optimization algorithm which, rather than
optimization the entire variables altogether, This would allow us to employ: 1)
a class of nonlinear functions with three variables and 2) a convex quadratic
multivariable polynomial, for minimization of reprojection error. Neglecting
the inversion required to minimize the nonlinear functions, in this paper we
demonstrate how separation allows eradication of matrix inversion.
Is Hamming distance the only way for matching binary image feature descriptors?
Authors: Erkan Bostanci
Date: 2015-12-08 00:00:00

Brute force matching of binary image feature descriptors is conventionally
performed using the Hamming distance. This paper assesses the use of
alternative metrics in order to see whether they can produce feature
correspondences that yield more accurate homography matrices. Two statistical
tests, namely ANOVA (Analysis of Variance) and McNemar's test were employed for
evaluation. Results show that Jackard-Needham and Dice metrics can display
better performance for some descriptors. Yet, these performance differences
were not found to be statistically significant.
Learning to Point and Count
Authors: Jie Shao, Dequan Wang, xiangyang xue, Zheng Zhang
Date: 2015-12-08 00:00:00
Tasks: General Classification

This paper proposes the problem of point-and-count as a test case to break
the what-and-where deadlock. Different from the traditional detection problem,
the goal is to discover key salient points as a way to localize and count the
number of objects simultaneously. We propose two alternatives, one that counts
first and then point, and another that works the other way around.
Fundamentally, they pivot around whether we solve "what" or "where" first. We
evaluate their performance on dataset that contains multiple instances of the
same class, demonstrating the potentials and their synergies. The experiences
derive a few important insights that explains why this is a much harder problem
than classification, including strong data bias and the inability to deal with
object scales robustly in state-of-art convolutional neural networks.
Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression
Authors: Takuya Narihira, Michael Maire, Stella X. Yu
Date: 2015-12-08 00:00:00
Proceeding: ICCV 2015 12
Tasks: Intrinsic Image Decomposition, regression

We introduce a new approach to intrinsic image decomposition, the task of
decomposing a single image into albedo and shading components. Our strategy,
which we term direct intrinsics, is to learn a convolutional neural network
(CNN) that directly predicts output albedo and shading channels from an input
RGB image patch. Direct intrinsics is a departure from classical techniques for
intrinsic image decomposition, which typically rely on physically-motivated
priors and graph-based inference algorithms.
  The large-scale synthetic ground-truth of the MPI Sintel dataset plays a key
role in training direct intrinsics. We demonstrate results on both the
synthetic images of Sintel and the real images of the classic MIT intrinsic
image dataset. On Sintel, direct intrinsics, using only RGB input, outperforms
all prior work, including methods that rely on RGB+Depth input. Direct
intrinsics also generalizes across modalities; it produces quite reasonable
decompositions on the real images of the MIT dataset. Our results indicate that
the marriage of CNNs with synthetic training data may be a powerful new
technique for tackling classic problems in computer vision.
Nonparametric Reduced-Rank Regression for Multi-SNP, Multi-Trait Association Mapping
Authors: Ashlee Valente, Geoffrey Ginsburg, Barbara E. Engelhardt
Date: 2015-12-08 00:00:00
Tasks: regression

Genome-wide association studies have proven to be essential for understanding
the genetic basis of disease. However, many complex traits---personality
traits, facial features, disease subtyping---are inherently high-dimensional,
impeding simple approaches to association mapping. We developed a nonparametric
Bayesian reduced rank regression model for multi-SNP, multi-trait association
mapping that does not require the rank of the linear subspace to be specified.
We show in simulations and real data that our model shares strength over SNPs
and over correlated traits, improving statistical power to identify genetic
associations with an interpretable, SNP-supervised low-dimensional linear
projection of the high-dimensional phenotype. On the HapMap phase 3 gene
expression QTL study data, we identify pleiotropic expression QTLs that
classical univariate tests are underpowered to find and that two step
approaches cannot recover. Our Python software, BERRRI, is publicly available
at GitHub: https://github.com/ashlee1031/BERRRI.
Optimal strategies for the control of autonomous vehicles in data assimilation
Authors: Damon McDougall, Richard Moore
Date: 2015-12-07 00:00:00
Tasks: Autonomous Vehicles

We propose a method to compute optimal control paths for autonomous vehicles
deployed for the purpose of inferring a velocity field. In addition to being
advected by the flow, the vehicles are able to effect a fixed relative speed
with arbitrary control over direction. It is this direction that is used as the
basis for the locally optimal control algorithm presented here, with objective
formed from the variance trace of the expected posterior distribution. We
present results for linear flows near hyperbolic fixed points.
On-the-Job Learning with Bayesian Decision Theory
Authors: Keenon Werling, Arun Chaganty, Percy Liang, Chris Manning
Date: 2015-06-10 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: General Classification, image-classification, Image Classification, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Sentiment Analysis, Sentiment Classification

Our goal is to deploy a high-accuracy system starting with zero training
examples. We consider an "on-the-job" setting, where as inputs arrive, we use
real-time crowdsourcing to resolve uncertainty where needed and output our
prediction when confident. As the model improves over time, the reliance on
crowdsourcing queries decreases. We cast our setting as a stochastic game based
on Bayesian decision theory, which allows us to balance latency, cost, and
accuracy objectives in a principled way. Computing the optimal policy is
intractable, so we develop an approximation based on Monte Carlo Tree Search.
We tested our approach on three datasets---named-entity recognition, sentiment
classification, and image classification. On the NER task we obtained more than
an order of magnitude reduction in cost compared to full human annotation,
while boosting performance relative to the expert provided labels. We also
achieve a 8% F1 improvement over having a single human label the whole set, and
a 28% F1 improvement over online learning.
Learning FRAME Models Using CNN Filters
Authors: Yang Lu, Song-Chun Zhu, Ying Nian Wu
Date: 2015-09-28 00:00:00

The convolutional neural network (ConvNet or CNN) has proven to be very
successful in many tasks such as those in computer vision. In this conceptual
paper, we study the generative perspective of the discriminative CNN. In
particular, we propose to learn the generative FRAME (Filters, Random field,
And Maximum Entropy) model using the highly expressive filters pre-learned by
the CNN at the convolutional layers. We show that the learning algorithm can
generate realistic and rich object and texture patterns in natural scenes. We
explain that each learned model corresponds to a new CNN unit at a layer above
the layer of filters employed by the model. We further show that it is possible
to learn a new layer of CNN units using a generative CNN model, which is a
product of experts model, and the learning algorithm admits an EM
interpretation with binary latent variables.
The Teaching Dimension of Linear Learners
Authors: Ji Liu, Xiaojin Zhu
Date: 2015-12-07 00:00:00
Tasks: regression

Teaching dimension is a learning theoretic quantity that specifies the
minimum training set size to teach a target model to a learner. Previous
studies on teaching dimension focused on version-space learners which maintain
all hypotheses consistent with the training data, and cannot be applied to
modern machine learners which select a specific hypothesis via optimization.
This paper presents the first known teaching dimension for ridge regression,
support vector machines, and logistic regression. We also exhibit optimal
training sets that match these teaching dimensions. Our approach generalizes to
other linear learners.