pwc-archive/papers-with-abstracts huggingface dataset:
STICK: Spike Time Interval Computational Kernel, A Framework for General Purpose Computation using Neurons, Precise Timing, Delays, and Synchrony
Authors: Xavier Lagorce, Ryad Benosman
Date: 2015-07-22 00:00:00
Tasks: Diversity

There has been significant research over the past two decades in developing
new platforms for spiking neural computation. Current neural computers are
primarily developed to mimick biology. They use neural networks which can be
trained to perform specific tasks to mainly solve pattern recognition problems.
These machines can do more than simulate biology, they allow us to re-think our
current paradigm of computation. The ultimate goal is to develop brain inspired
general purpose computation architectures that can breach the current
bottleneck introduced by the Von Neumann architecture. This work proposes a new
framework for such a machine. We show that the use of neuron like units with
precise timing representation, synaptic diversity, and temporal delays allows
us to set a complete, scalable compact computation framework. The presented
framework provides both linear and non linear operations, allowing us to
represent and solve any function. We show usability in solving real use cases
from simple differential equations to sets of non-linear differential equations
leading to chaotic attractors.
Bayesian Time-of-Flight for Realtime Shape, Illumination and Albedo
Authors: Amit Adam, Christoph Dann, Omer Yair, Shai Mazor, Sebastian Nowozin
Date: 2015-07-22 00:00:00

We propose a computational model for shape, illumination and albedo inference
in a pulsed time-of-flight (TOF) camera. In contrast to TOF cameras based on
phase modulation, our camera enables general exposure profiles. This results in
added flexibility and requires novel computational approaches.
  To address this challenge we propose a generative probabilistic model that
accurately relates latent imaging conditions to observed camera responses.
While principled, realtime inference in the model turns out to be infeasible,
and we propose to employ efficient non-parametric regression trees to
approximate the model outputs. As a result we are able to provide, for each
pixel, at video frame rate, estimates and uncertainty for depth, effective
albedo, and ambient light intensity. These results we present are
state-of-the-art in depth imaging.
  The flexibility of our approach allows us to easily enrich our generative
model. We demonstrate that by extending the original single-path model to a
two-path model, capable of describing some multipath effects. The new model is
seamlessly integrated in the system at no additional computational cost.
  Our work also addresses the important question of optimal exposure design in
pulsed TOF systems. Finally, for benchmark purposes and to obtain realistic
empirical priors of multipath and insights into this phenomena, we propose a
physically accurate simulation of multipath phenomena.
Data-free parameter pruning for Deep Neural Networks
Authors: Suraj Srinivas, R. Venkatesh Babu
Date: 2015-07-22 00:00:00

Deep Neural nets (NNs) with millions of parameters are at the heart of many
state-of-the-art computer vision systems today. However, recent works have
shown that much smaller models can achieve similar levels of performance. In
this work, we address the problem of pruning parameters in a trained NN model.
Instead of removing individual weights one at a time as done in previous works,
we remove one neuron at a time. We show how similar neurons are redundant, and
propose a systematic way to remove them. Our experiments in pruning the densely
connected layers show that we can remove upto 85\% of the total parameters in
an MNIST-trained network, and about 35\% for AlexNet without significantly
affecting performance. Our method can be applied on top of most networks with a
fully connected layer to give a smaller network.
Compression of Fully-Connected Layer in Neural Network by Kronecker Product
Authors: Shuchang Zhou, Jia-Nan Wu
Date: 2015-07-21 00:00:00

In this paper we propose and study a technique to reduce the number of
parameters and computation time in fully-connected layers of neural networks
using Kronecker product, at a mild cost of the prediction quality. The
technique proceeds by replacing Fully-Connected layers with so-called Kronecker
Fully-Connected layers, where the weight matrices of the FC layers are
approximated by linear combinations of multiple Kronecker products of smaller
matrices. In particular, given a model trained on SVHN dataset, we are able to
construct a new KFC model with 73\% reduction in total number of parameters,
while the error only rises mildly. In contrast, using low-rank method can only
achieve 35\% reduction in total number of parameters given similar quality
degradation allowance. If we only compare the KFC layer with its counterpart
fully-connected layer, the reduction in the number of parameters exceeds 99\%.
The amount of computation is also reduced as we replace matrix product of the
large matrices in FC layers with matrix products of a few smaller matrices in
KFC layers. Further experiments on MNIST, SVHN and some Chinese Character
recognition models also demonstrate effectiveness of our technique.
Banzhaf Random Forests
Authors: Jianyuan Sun, Guoqiang Zhong, Junyu Dong, Yajuan Cai
Date: 2015-07-22 00:00:00

Random forests are a type of ensemble method which makes predictions by
combining the results of several independent trees. However, the theory of
random forests has long been outpaced by their application. In this paper, we
propose a novel random forests algorithm based on cooperative game theory.
Banzhaf power index is employed to evaluate the power of each feature by
traversing possible feature coalitions. Unlike the previously used information
gain rate of information theory, which simply chooses the most informative
feature, the Banzhaf power index can be considered as a metric of the
importance of each feature on the dependency among a group of features. More
importantly, we have proved the consistency of the proposed algorithm, named
Banzhaf random forests (BRF). This theoretical analysis takes a step towards
narrowing the gap between the theory and practice of random forests for
classification problems. Experiments on several UCI benchmark data sets show
that BRF is competitive with state-of-the-art classifiers and dramatically
outperforms previous consistent random forests. Particularly, it is much more
efficient than previous consistent random forests.
Taming Primary Key Violations to Query Large Inconsistent Data
Authors: Marco Manna, Francesco Ricca, Giorgio Terracina
Date: 2015-07-22 00:00:00

Consistent query answering over a database that violates primary key
constraints is a classical hard problem in database research that has been
traditionally dealt with logic programming. However, the applicability of
existing logic-based solutions is restricted to data sets of moderate size.
This paper presents a novel decomposition and pruning strategy that reduces, in
polynomial time, the problem of computing the consistent answer to a
conjunctive query over a database subject to primary key constraints to a
collection of smaller problems of the same sort that can be solved
independently. The new strategy is naturally modeled and implemented using
Answer Set Programming (ASP). An experiment run on benchmarks from the database
world prove the effectiveness and efficiency of our ASP-based approach also on
large data sets. To appear in Theory and Practice of Logic Programming (TPLP),
Proceedings of ICLP 2015.
Fast, simple and accurate handwritten digit classification by training shallow neural network classifiers with the 'extreme learning machine' algorithm
Authors: Mark D. McDonnell, Migel D. Tissera, Tony Vladusich, André van Schaik, Jonathan Tapson
Date: 2014-12-29 00:00:00
Tasks: General Classification, speech-recognition, Speech Recognition

Recent advances in training deep (multi-layer) architectures have inspired a
renaissance in neural network use. For example, deep convolutional networks are
becoming the default option for difficult tasks on large datasets, such as
image and speech recognition. However, here we show that error rates below 1%
on the MNIST handwritten digit benchmark can be replicated with shallow
non-convolutional neural networks. This is achieved by training such networks
using the 'Extreme Learning Machine' (ELM) approach, which also enables a very
rapid training time (~10 minutes). Adding distortions, as is common practise
for MNIST, reduces error rates even further. Our methods are also shown to be
capable of achieving less than 5.5% error rates on the NORB image database. To
achieve these results, we introduce several enhancements to the standard ELM
algorithm, which individually and in combination can significantly improve
performance. The main innovation is to ensure each hidden-unit operates only on
a randomly sized and positioned patch of each image. This form of random
`receptive field' sampling of the input ensures the input weight matrix is
sparse, with about 90% of weights equal to zero. Furthermore, combining our
methods with a small number of iterations of a single-batch backpropagation
method can significantly reduce the number of hidden-units required to achieve
a particular performance. Our close to state-of-the-art results for MNIST and
NORB suggest that the ease of use and accuracy of the ELM algorithm for
designing a single-hidden-layer neural network classifier should cause it to be
given greater consideration either as a standalone method for simpler problems,
or as the final classification stage in deep neural networks applied to more
difficult problems.
MixEst: An Estimation Toolbox for Mixture Models
Authors: Reshad Hosseini, Mohamadreza Mash'al
Date: 2015-07-22 00:00:00
Tasks: Clustering, Density Estimation, Riemannian optimization

Mixture models are powerful statistical models used in many applications
ranging from density estimation to clustering and classification. When dealing
with mixture models, there are many issues that the experimenter should be
aware of and needs to solve. The MixEst toolbox is a powerful and user-friendly
package for MATLAB that implements several state-of-the-art approaches to
address these problems. Additionally, MixEst gives the possibility of using
manifold optimization for fitting the density model, a feature specific to this
toolbox. MixEst simplifies using and integration of mixture models in
statistical models and applications. For developing mixture models of new
densities, the user just needs to provide a few functions for that statistical
distribution and the toolbox takes care of all the issues regarding mixture
models. MixEst is available at visionlab.ut.ac.ir/mixest and is fully
documented and is licensed under GPL.
Adapting Stochastic Search For Real-time Dynamic Weighted Constraint Satisfaction
Authors: Gregory Hasseler
Date: 2015-07-22 00:00:00

This work presents two new algorithms for performing constraint satisfaction.
The first algorithm presented, DMaxWalkSat, is a constraint solver specialized
for solving dynamic, weighted constraint satisfaction problems. The second
algorithm, RDMaxWalkSat, is a derivative of DMaxWalkSat that has been modified
into an anytime algorithm, and hence support realtime constraint satisfaction.
DMaxWalkSat is shown to offer performance advantages in terms of solution
quality and runtime over its parent constraint solver, MaxWalkSat. RDMaxWalkSat
is shown to support anytime operation. The introduction of these algorithms
brings another tool to the areas of computer science that naturally represent
problems as constraint satisfaction problems, an example of which is the robust
coherence algorithm.
A Neural Conversational Model
Authors: Oriol Vinyals, Quoc Le
Date: 2015-06-19 00:00:00
Tasks: Common Sense Reasoning, model, Natural Language Understanding, Sentence

Conversational modeling is an important task in natural language
understanding and machine intelligence. Although previous approaches exist,
they are often restricted to specific domains (e.g., booking an airline ticket)
and require hand-crafted rules. In this paper, we present a simple approach for
this task which uses the recently proposed sequence to sequence framework. Our
model converses by predicting the next sentence given the previous sentence or
sentences in a conversation. The strength of our model is that it can be
trained end-to-end and thus requires much fewer hand-crafted rules. We find
that this straightforward model can generate simple conversations given a large
conversational training dataset. Our preliminary results suggest that, despite
optimizing the wrong objective function, the model is able to converse well. It
is able extract knowledge from both a domain specific dataset, and from a
large, noisy, and general domain dataset of movie subtitles. On a
domain-specific IT helpdesk dataset, the model can find a solution to a
technical problem via conversations. On a noisy open-domain movie transcript
dataset, the model can perform simple forms of common sense reasoning. As
expected, we also find that the lack of consistency is a common failure mode of
our model.
Dependent Types for Pragmatics
Authors: Darryl McAdams, Jonathan Sterling
Date: 2014-10-17 00:00:00

This paper proposes the use of dependent types for pragmatic phenomena such
as pronoun binding and presupposition resolution as a type-theoretic
alternative to formalisms such as Discourse Representation Theory and Dynamic
Semantics.
Elastic Net Procedure for Partially Linear Models
Authors: Chunhong Li, Dengxiang Huang, Hongshuai Dai, Xinxing Wei
Date: 2015-07-22 00:00:00
Tasks: Variable Selection

Variable selection plays an important role in the high-dimensional data
analysis. However the high-dimensional data often induces the strongly
correlated variables problem. In this paper, we propose Elastic Net procedure
for partially linear models and prove the group effect of its estimate. By a
simulation study, we show that the strongly correlated variables problem can be
better handled by the Elastic Net procedure than Lasso, ALasso and Ridge. Based
on an empirical analysis, we can get that the Elastic Net procedure is
particularly useful when the number of predictors $p$ is much bigger than the
sample size $n$.
The challenges of SVM optimization using Adaboost on a phoneme recognition problem
Authors: Rimah Amami, Dorra Ben Ayed, Noureddine Ellouze
Date: 2015-07-22 00:00:00
Tasks: Phoneme Recognition

The use of digital technology is growing at a very fast pace which led to the
emergence of systems based on the cognitive infocommunications. The expansion
of this sector impose the use of combining methods in order to ensure the
robustness in cognitive systems.
Incorporating Belief Function in SVM for Phoneme Recognition
Authors: Rimah Amami, Dorra Ben Ayed, Nouerddine Ellouze
Date: 2015-07-22 00:00:00
Tasks: Phoneme Recognition

The Support Vector Machine (SVM) method has been widely used in numerous
classification tasks. The main idea of this algorithm is based on the principle
of the margin maximization to find an hyperplane which separates the data into
two different classes.In this paper, SVM is applied to phoneme recognition
task. However, in many real-world problems, each phoneme in the data set for
recognition problems may differ in the degree of significance due to noise,
inaccuracies, or abnormal characteristics; All those problems can lead to the
inaccuracies in the prediction phase. Unfortunately, the standard formulation
of SVM does not take into account all those problems and, in particular, the
variation in the speech input. This paper presents a new formulation of SVM
(B-SVM) that attributes to each phoneme a confidence degree computed based on
its geometric position in the space. Then, this degree is used in order to
strengthen the class membership of the tested phoneme. Hence, we introduce a
reformulation of the standard SVM that incorporates the degree of belief.
Experimental performance on TIMIT database shows the effectiveness of the
proposed method B-SVM on a phoneme recognition problem.
Robust speech recognition using consensus function based on multi-layer networks
Authors: Rimah Amami, Ghaith Manita, Abir Smiti
Date: 2015-07-22 00:00:00
Tasks: Clustering, Clustering Ensemble, General Classification, Robust Speech Recognition, speech-recognition, Speech Recognition

The clustering ensembles mingle numerous partitions of a specified data into
a single clustering solution. Clustering ensemble has emerged as a potent
approach for ameliorating both the forcefulness and the stability of
unsupervised classification results. One of the major problems in clustering
ensembles is to find the best consensus function. Finding final partition from
different clustering results requires skillfulness and robustness of the
classification algorithm. In addition, the major problem with the consensus
function is its sensitivity to the used data sets quality. This limitation is
due to the existence of noisy, silence or redundant data. This paper proposes a
novel consensus function of cluster ensembles based on Multilayer networks
technique and a maintenance database method. This maintenance database approach
is used in order to handle any given noisy speech and, thus, to guarantee the
quality of databases. This can generates good results and efficient data
partitions. To show its effectiveness, we support our strategy with empirical
evaluation using distorted speech from Aurora speech databases.
An Empirical Comparison of SVM and Some Supervised Learning Algorithms for Vowel recognition
Authors: Rimah Amami, Dorra Ben Ayed, Noureddine Ellouze
Date: 2015-07-22 00:00:00

In this article, we conduct a study on the performance of some supervised
learning algorithms for vowel recognition. This study aims to compare the
accuracy of each algorithm. Thus, we present an empirical comparison between
five supervised learning classifiers and two combined classifiers: SVM, KNN,
Naive Bayes, Quadratic Bayes Normal (QDC) and Nearst Mean. Those algorithms
were tested for vowel recognition using TIMIT Corpus and Mel-frequency cepstral
coefficients (MFCCs).
Practical Selection of SVM Supervised Parameters with Different Feature Representations for Vowel Recognition
Authors: Rimah Amami, Dorra Ben Ayed, Noureddine Ellouze
Date: 2015-07-22 00:00:00
Tasks: General Classification

It is known that the classification performance of Support Vector Machine
(SVM) can be conveniently affected by the different parameters of the kernel
tricks and the regularization parameter, C. Thus, in this article, we propose a
study in order to find the suitable kernel with which SVM may achieve good
generalization performance as well as the parameters to use. We need to analyze
the behavior of the SVM classifier when these parameters take very small or
very large values. The study is conducted for a multi-class vowel recognition
using the TIMIT corpus. Furthermore, for the experiments, we used different
feature representations such as MFCC and PLP. Finally, a comparative study was
done to point out the impact of the choice of the parameters, kernel trick and
feature representations on the performance of the SVM classifier
The Population Posterior and Bayesian Inference on Streams
Authors: James McInerney, Rajesh Ranganath, David M. Blei
Date: 2015-07-19 00:00:00
Tasks: Bayesian Inference

Many modern data analysis problems involve inferences from streaming data.
However, streaming data is not easily amenable to the standard probabilistic
modeling approaches, which assume that we condition on finite data. We develop
population variational Bayes, a new approach for using Bayesian modeling to
analyze streams of data. It approximates a new type of distribution, the
population posterior, which combines the notion of a population distribution of
the data with Bayesian inference in a probabilistic model. We study our method
with latent Dirichlet allocation and Dirichlet process mixtures on several
large-scale data sets.
On the Worst-Case Approximability of Sparse PCA
Authors: Siu On Chan, Dimitris Papailiopoulos, Aviad Rubinstein
Date: 2015-07-21 00:00:00

It is well known that Sparse PCA (Sparse Principal Component Analysis) is
NP-hard to solve exactly on worst-case instances. What is the complexity of
solving Sparse PCA approximately? Our contributions include: 1) a simple and
efficient algorithm that achieves an $n^{-1/3}$-approximation; 2) NP-hardness
of approximation to within $(1-\varepsilon)$, for some small constant
$\varepsilon > 0$; 3) SSE-hardness of approximation to within any constant
factor; and 4) an $\exp\exp\left(\Omega\left(\sqrt{\log \log n}\right)\right)$
("quasi-quasi-polynomial") gap for the standard semidefinite program.
Generalized Totalizer Encoding for Pseudo-Boolean Constraints
Authors: Saurabh Joshi, Ruben Martins, Vasco Manquinho
Date: 2015-07-21 00:00:00
Tasks: ARC

Pseudo-Boolean constraints, also known as 0-1 Integer Linear Constraints, are
used to model many real-world problems. A common approach to solve these
constraints is to encode them into a SAT formula. The runtime of the SAT solver
on such formula is sensitive to the manner in which the given pseudo-Boolean
constraints are encoded. In this paper, we propose generalized Totalizer
encoding (GTE), which is an arc-consistency preserving extension of the
Totalizer encoding to pseudo-Boolean constraints. Unlike some other encodings,
the number of auxiliary variables required for GTE does not depend on the
magnitudes of the coefficients. Instead, it depends on the number of distinct
combinations of these coefficients. We show the superiority of GTE with respect
to other encodings when large pseudo-Boolean constraints have low number of
distinct coefficients. Our experimental results also show that GTE remains
competitive even when the pseudo-Boolean constraints do not have this
characteristic.
Decision Maker based on Atomic Switches
Authors: Song-Ju Kim, Tohru Tsuruoka, Tsuyoshi Hasegawa, Masakazu Aono
Date: 2015-07-21 00:00:00
Tasks: Decision Making

We propose a simple model for an atomic switch-based decision maker (ASDM),
and show that, as long as its total volume of precipitated Ag atoms is
conserved when coupled with suitable operations, an atomic switch system
provides a sophisticated "decision-making" capability that is known to be one
of the most important intellectual abilities in human beings. We considered the
multi-armed bandit problem (MAB); the problem of finding, as accurately and
quickly as possible, the most profitable option from a set of options that
gives stochastic rewards. These decisions are made as dictated by each volume
of precipitated Ag atoms, which is moved in a manner similar to the
fluctuations of a rigid body in a tug-of-war game. The "tug-of-war (TOW)
dynamics" of the ASDM exhibits higher efficiency than conventional MAB solvers.
We show analytical calculations that validate the statistical reasons for the
ASDM dynamics to produce such high performance, despite its simplicity. These
results imply that various physical systems, in which some conservation law
holds, can be used to implement efficient "decision-making objects." Efficient
MAB solvers are useful for many practical applications, because MAB abstracts a
variety of decision-making problems in real- world situations where an
efficient trial-and-error is required. The proposed scheme will introduce a new
physics-based analog computing paradigm, which will include such things as
"intelligent nano devices" and "intelligent information networks" based on
self-detection and self-judgment.