pwc-archive/papers-with-abstracts huggingface dataset:
Learning Meta-Embeddings by Using Ensembles of Embedding Sets
Authors: Wenpeng Yin, Hinrich Schütze
Date: 2015-08-18 00:00:00
Tasks: Part-Of-Speech Tagging, Word Embeddings, Word Similarity

Word embeddings -- distributed representations of words -- in deep learning
are beneficial for many tasks in natural language processing (NLP). However,
different embedding sets vary greatly in quality and characteristics of the
captured semantics. Instead of relying on a more advanced algorithm for
embedding learning, this paper proposes an ensemble approach of combining
different public embedding sets with the aim of learning meta-embeddings.
Experiments on word similarity and analogy tasks and on part-of-speech tagging
show better performance of meta-embeddings compared to individual embedding
sets. One advantage of meta-embeddings is the increased vocabulary coverage. We
will release our meta-embeddings publicly.
Sharp Computational-Statistical Phase Transitions via Oracle Computational Model
Authors: Zhaoran Wang, Quanquan Gu, Han Liu
Date: 2015-12-30 00:00:00
Tasks: Two-sample testing

We study the fundamental tradeoffs between computational tractability and
statistical accuracy for a general family of hypothesis testing problems with
combinatorial structures. Based upon an oracle model of computation, which
captures the interactions between algorithms and data, we establish a general
lower bound that explicitly connects the minimum testing risk under
computational budget constraints with the intrinsic probabilistic and
combinatorial structures of statistical problems. This lower bound mirrors the
classical statistical lower bound by Le Cam (1986) and allows us to quantify
the optimal statistical performance achievable given limited computational
budgets in a systematic fashion. Under this unified framework, we sharply
characterize the statistical-computational phase transition for two testing
problems, namely, normal mean detection and sparse principal component
detection. For normal mean detection, we consider two combinatorial structures,
namely, sparse set and perfect matching. For these problems we identify
significant gaps between the optimal statistical accuracy that is achievable
under computational tractability constraints and the classical statistical
lower bounds. Compared with existing works on computational lower bounds for
statistical problems, which consider general polynomial-time algorithms on
Turing machines, and rely on computational hardness hypotheses on problems like
planted clique detection, we focus on the oracle computational model, which
covers a broad range of popular algorithms, and do not rely on unproven
hypotheses. Moreover, our result provides an intuitive and concrete
interpretation for the intrinsic computational intractability of
high-dimensional statistical problems. One byproduct of our result is a lower
bound for a strict generalization of the matrix permanent problem, which is of
independent interest.
Learning with a Wasserstein Loss
Authors: Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya-Polo, Tomaso Poggio
Date: 2015-06-17 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Multi-Label Learning, TAG

Learning to predict multi-label outputs is challenging, but in many problems
there is a natural metric on the outputs that can be used to improve
predictions. In this paper we develop a loss function for multi-label learning,
based on the Wasserstein distance. The Wasserstein distance provides a natural
notion of dissimilarity for probability measures. Although optimizing with
respect to the exact Wasserstein distance is costly, recent work has described
a regularized approximation that is efficiently computed. We describe an
efficient learning algorithm based on this regularization, as well as a novel
extension of the Wasserstein distance from probability measures to unnormalized
measures. We also describe a statistical learning bound for the loss. The
Wasserstein loss can encourage smoothness of the predictions with respect to a
chosen metric on the output space. We demonstrate this property on a real-data
tag prediction problem, using the Yahoo Flickr Creative Commons dataset,
outperforming a baseline that doesn't use the metric.
Joint limiting laws for high-dimensional independence tests
Authors: Danning Li, Lingzhou Xue
Date: 2015-12-30 00:00:00
Tasks: Form, Vocal Bursts Intensity Prediction

Testing independence is of significant interest in many important areas of
large-scale inference. Using extreme-value form statistics to test against
sparse alternatives and using quadratic form statistics to test against dense
alternatives are two important testing procedures for high-dimensional
independence. However, quadratic form statistics suffer from low power against
sparse alternatives, and extreme-value form statistics suffer from low power
against dense alternatives with small disturbances and may have size
distortions due to its slow convergence. For real-world applications, it is
important to derive powerful testing procedures against more general
alternatives. Based on intermediate limiting distributions, we derive
(model-free) joint limiting laws of extreme-value form and quadratic form
statistics, and surprisingly, we prove that they are asymptotically
independent. Given such asymptotic independencies, we propose (model-free)
testing procedures to boost the power against general alternatives and also
retain the correct asymptotic size. Under the high-dimensional setting, we
derive the closed-form limiting null distributions, and obtain their explicit
rates of uniform convergence. We prove their consistent statistical powers
against general alternatives. We demonstrate the performance of our proposed
test statistics in simulation studies. Our work provides very helpful insights
to high-dimensional independence tests, and fills an important gap.
Combined statistical and model based texture features for improved image classification
Authors: Omar Al-Kadi
Date: 2015-12-29 00:00:00
Tasks: General Classification, image-classification, Image Classification, Texture Classification

This paper aims to improve the accuracy of texture classification based on
extracting texture features using five different texture methods and
classifying the patterns using a naive Bayesian classifier. Three
statistical-based and two model-based methods are used to extract texture
features from eight different texture images, then their accuracy is ranked
after using each method individually and in pairs. The accuracy improved up to
97.01% when model based -Gaussian Markov random field (GMRF) and fractional
Brownian motion (fBm) - were used together for classification as compared to
the highest achieved using each of the five different methods alone; and proved
to be better in classifying as compared to statistical methods. Also, using
GMRF with statistical based methods, such as Gray level co-occurrence (GLCM)
and run-length (RLM) matrices, improved the overall accuracy to 96.94% and
96.55%; respectively.
Combining Fuzzy Cognitive Maps and Discrete Random Variables
Authors: Piotr Szwed
Date: 2015-12-29 00:00:00

In this paper we propose an extension to the Fuzzy Cognitive Maps (FCMs) that
aims at aggregating a number of reasoning tasks into a one parallel run. The
described approach consists in replacing real-valued activation levels of
concepts (and further influence weights) by random variables. Such extension,
followed by the implemented software tool, allows for determining ranges
reached by concept activation levels, sensitivity analysis as well as
statistical analysis of multiple reasoning results. We replace multiplication
and addition operators appearing in the FCM state equation by appropriate
convolutions applicable for discrete random variables. To make the model
computationally feasible, it is further augmented with aggregation operations
for discrete random variables. We discuss four implemented aggregators, as well
as we report results of preliminary tests.
Matrix Completion Under Monotonic Single Index Models
Authors: Ravi Ganti, Laura Balzano, Rebecca Willett
Date: 2015-12-29 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Matrix Completion

Most recent results in matrix completion assume that the matrix under
consideration is low-rank or that the columns are in a union of low-rank
subspaces. In real-world settings, however, the linear structure underlying
these models is distorted by a (typically unknown) nonlinear transformation.
This paper addresses the challenge of matrix completion in the face of such
nonlinearities. Given a few observations of a matrix that are obtained by
applying a Lipschitz, monotonic function to a low rank matrix, our task is to
estimate the remaining unobserved entries. We propose a novel matrix completion
method that alternates between low-rank matrix estimation and monotonic
function estimation to estimate the missing matrix elements. Mean squared error
bounds provide insight into how well the matrix can be estimated based on the
size, rank of the matrix and properties of the nonlinear transformation.
Empirical results on synthetic and real-world datasets demonstrate the
competitiveness of the proposed approach.
On the Foundations of the Brussels Operational-Realistic Approach to Cognition
Authors: Diederik Aerts, Massimiliano Sassoli de Bianchi, Sandro Sozzo
Date: 2015-12-29 00:00:00
Tasks: Decision Making

The scientific community is becoming more and more interested in the research
that applies the mathematical formalism of quantum theory to model human
decision-making. In this paper, we provide the theoretical foundations of the
quantum approach to cognition that we developed in Brussels. These foundations
rest on the results of two decade studies on the axiomatic and
operational-realistic approaches to the foundations of quantum physics. The
deep analogies between the foundations of physics and cognition lead us to
investigate the validity of quantum theory as a general and unitary framework
for cognitive processes, and the empirical success of the Hilbert space models
derived by such investigation provides a strong theoretical confirmation of
this validity. However, two situations in the cognitive realm, 'question order
effects' and 'response replicability', indicate that even the Hilbert space
framework could be insufficient to reproduce the collected data. This does not
mean that the mentioned operational-realistic approach would be incorrect, but
simply that a larger class of measurements would be in force in human
cognition, so that an extended quantum formalism may be needed to deal with all
of them. As we will explain, the recently derived 'extended Bloch
representation' of quantum theory (and the associated 'general
tension-reduction' model) precisely provides such extended formalism, while
remaining within the same unitary interpretative framework.
Error Bounds for Compressed Sensing Algorithms With Group Sparsity: A Unified Approach
Authors: M. Eren Ahsen, M. Vidyasagar
Date: 2015-12-29 00:00:00
Tasks: compressed sensing

In compressed sensing, in order to recover a sparse or nearly sparse vector
from possibly noisy measurements, the most popular approach is $\ell_1$-norm
minimization. Upper bounds for the $\ell_2$- norm of the error between the true
and estimated vectors are given in [1] and reviewed in [2], while bounds for
the $\ell_1$-norm are given in [3]. When the unknown vector is not
conventionally sparse but is "group sparse" instead, a variety of alternatives
to the $\ell_1$-norm have been proposed in the literature, including the group
LASSO, sparse group LASSO, and group LASSO with tree structured overlapping
groups. However, no error bounds are available for any of these modified
objective functions. In the present paper, a unified approach is presented for
deriving upper bounds on the error between the true vector and its
approximation, based on the notion of decomposable and $\gamma$-decomposable
norms. The bounds presented cover all of the norms mentioned above, and also
provide a guideline for choosing norms in future to accommodate alternate forms
of sparsity.
Robust Scene Text Recognition Using Sparse Coding based Features
Authors: Da-Han Wang, Hanzi Wang, Dong Zhang, Jonathan Li, David Zhang
Date: 2015-12-29 00:00:00
Tasks: Scene Text Recognition

In this paper, we propose an effective scene text recognition method using
sparse coding based features, called Histograms of Sparse Codes (HSC) features.
For character detection, we use the HSC features instead of using the
Histograms of Oriented Gradients (HOG) features. The HSC features are extracted
by computing sparse codes with dictionaries that are learned from data using
K-SVD, and aggregating per-pixel sparse codes to form local histograms. For
word recognition, we integrate multiple cues including character detection
scores and geometric contexts in an objective function. The final recognition
results are obtained by searching for the words which correspond to the maximum
value of the objective function. The parameters in the objective function are
learned using the Minimum Classification Error (MCE) training method.
Experiments on several challenging datasets demonstrate that the proposed
HSC-based scene text recognition method outperforms HOG-based methods
significantly and outperforms most state-of-the-art methods.
A framework for robust object multi-detection with a vote aggregation and a cascade filtering
Authors: Grzegorz Kurzejamski, Jacek Zawistowski, Grzegorz Sarwas
Date: 2015-12-29 00:00:00
Tasks: Object, object-detection, Object Detection

This paper presents a framework designed for the multi-object detection
purposes and adjusted for the application of product search on the market
shelves. The framework uses a single feedback loop and a pattern resizing
mechanism to demonstrate the top effectiveness of the state-of-the-art local
features. A high detection rate with a low false detection chance can be
achieved with use of only one pattern per object and no manual parameters
adjustments. The method incorporates well known local features and a basic
matching process to create a reliable voting space. Further steps comprise of
metric transformations, graphical vote space representation, two-phase vote
aggregation process and a cascade of verifying filters.
Triplet Spike Time Dependent Plasticity: A floating-gate Implementation
Authors: Roshan Gopalakrishnan, Arindam Basu
Date: 2015-12-03 00:00:00
Tasks: Triplet

Synapse plays an important role of learning in a neural network; the learning
rules which modify the synaptic strength based on the timing difference between
the pre- and post-synaptic spike occurrence is termed as Spike Time Dependent
Plasticity (STDP). The most commonly used rule posits weight change based on
time difference between one pre- and one post spike and is hence termed doublet
STDP (DSTDP). However, D-STDP could not reproduce results of many biological
experiments; a triplet STDP (T-STDP) that considers triplets of spikes as the
fundamental unit has been proposed recently to explain these observations. This
paper describes the compact implementation of a synapse using single
floating-gate (FG) transistor that can store a weight in a nonvolatile manner
and demonstrate the triplet STDP (T-STDP) learning rule by modifying drain
voltages according to triplets of spikes. We describe a mathematical procedure
to obtain control voltages for the FG device for T-STDP and also show
measurement results from a FG synapse fabricated in TSMC 0.35um CMOS process to
support the theory. Possible VLSI implementation of drain voltage waveform
generator circuits are also presented with simulation results.
Tight Bounds for Approximate Carathéodory and Beyond
Authors: Vahab Mirrokni, Renato Paes Leme, Adrian Vladu, Sam Chiu-wai Wong
Date: 2015-12-29 00:00:00
Proceeding: ICML 2017 8

We give a deterministic nearly-linear time algorithm for approximating any
point inside a convex polytope with a sparse convex combination of the
polytope's vertices. Our result provides a constructive proof for the
Approximate Carath\'{e}odory Problem, which states that any point inside a
polytope contained in the $\ell_p$ ball of radius $D$ can be approximated to
within $\epsilon$ in $\ell_p$ norm by a convex combination of only $O\left(D^2
p/\epsilon^2\right)$ vertices of the polytope for $p \geq 2$. We also show that
this bound is tight, using an argument based on anti-concentration for the
binomial distribution.
  Along the way of establishing the upper bound, we develop a technique for
minimizing norms over convex sets with complicated geometry; this is achieved
by running Mirror Descent on a dual convex function obtained via Sion's
Theorem.
  As simple extensions of our method, we then provide new algorithms for
submodular function minimization and SVM training. For submodular function
minimization we obtain a simplification and (provable) speed-up over Wolfe's
algorithm, the method commonly found to be the fastest in practice. For SVM
training, we obtain $O(1/\epsilon^2)$ convergence for arbitrary kernels; each
iteration only requires matrix-vector operations involving the kernel matrix,
so we overcome the obstacle of having to explicitly store the kernel or compute
its Cholesky factorization.
Optimal Selective Attention in Reactive Agents
Authors: Roy Fox, Naftali Tishby
Date: 2015-12-29 00:00:00
Tasks: Diversity

In POMDPs, information about the hidden state, delivered through
observations, is both valuable to the agent, allowing it to base its actions on
better informed internal states, and a "curse", exploding the size and
diversity of the internal state space. One attempt to deal with this is to
focus on reactive policies, that only base their actions on the most recent
observation. However, even reactive policies can be demanding on resources, and
agents need to pay selective attention to only some of the information
available to them in observations. In this report we present the
minimum-information principle for selective attention in reactive agents. We
further motivate this approach by reducing the general problem of optimal
control in POMDPs, to reactive control with complex observations. Lastly, we
explore a newly discovered phenomenon of this optimization process - period
doubling bifurcations. This necessitates periodic policies, and raises many
more questions regarding stability, periodicity and chaos in optimal control.
Structured Pruning of Deep Convolutional Neural Networks
Authors: Sajid Anwar, Kyuyeon Hwang, Wonyong Sung
Date: 2015-12-29 00:00:00
Tasks: Network Pruning

Real time application of deep learning algorithms is often hindered by high
computational complexity and frequent memory accesses. Network pruning is a
promising technique to solve this problem. However, pruning usually results in
irregular network connections that not only demand extra representation efforts
but also do not fit well on parallel computation. We introduce structured
sparsity at various scales for convolutional neural networks, which are channel
wise, kernel wise and intra kernel strided sparsity. This structured sparsity
is very advantageous for direct computational resource savings on embedded
computers, parallel computing environments and hardware based systems. To
decide the importance of network connections and paths, the proposed method
uses a particle filtering approach. The importance weight of each particle is
assigned by computing the misclassification rate with corresponding
connectivity pattern. The pruned network is re-trained to compensate for the
losses due to pruning. While implementing convolutions as matrix products, we
particularly show that intra kernel strided sparsity with a simple constraint
can significantly reduce the size of kernel and feature map matrices. The
pruned network is finally fixed point optimized with reduced word length
precision. This results in significant reduction in the total storage size
providing advantages for on-chip memory based implementations of deep neural
networks.
Analyzing Walter Skeat's Forty-Five Parallel Extracts of William Langland's Piers Plowman
Authors: Roger Bilisoly
Date: 2015-12-29 00:00:00
Tasks: Clustering

Walter Skeat published his critical edition of William Langland's 14th
century alliterative poem, Piers Plowman, in 1886. In preparation for this he
located forty-five manuscripts, and to compare dialects, he published excerpts
from each of these. This paper does three statistical analyses using these
excerpts, each of which mimics a task he did in writing his critical edition.
First, he combined multiple versions of a poetic line to create a best line,
which is compared to the mean string that is computed by a generalization of
the arithmetic mean that uses edit distance. Second, he claims that a certain
subset of manuscripts varies little. This is quantified by computing a string
variance, which is closely related to the above generalization of the mean.
Third, he claims that the manuscripts fall into three groups, which is a
clustering problem that is addressed by using edit distance. The overall goal
is to develop methodology that would be of use to a literary critic.
Conditional probability generation methods for high reliability effects-based decision making
Authors: Wolfgang Garn, Panos Louvieris
Date: 2015-12-28 00:00:00
Tasks: Decision Making, parameter estimation, Vocal Bursts Intensity Prediction

Decision making is often based on Bayesian networks. The building blocks for
Bayesian networks are its conditional probability tables (CPTs). These tables
are obtained by parameter estimation methods, or they are elicited from subject
matter experts (SME). Some of these knowledge representations are insufficient
approximations. Using knowledge fusion of cause and effect observations lead to
better predictive decisions. We propose three new methods to generate CPTs,
which even work when only soft evidence is provided. The first two are novel
ways of mapping conditional expectations to the probability space. The third is
a column extraction method, which obtains CPTs from nonlinear functions such as
the multinomial logistic regression. Case studies on military effects and burnt
forest desertification have demonstrated that so derived CPTs have highly
reliable predictive power, including superiority over the CPTs obtained from
SMEs. In this context, new quality measures for determining the goodness of a
CPT and for comparing CPTs with each other have been introduced. The predictive
power and enhanced reliability of decision making based on the novel CPT
generation methods presented in this paper have been confirmed and validated
within the context of the case studies.