pwc-archive/papers-with-abstracts huggingface dataset:
AltecOnDB: A Large-Vocabulary Arabic Online Handwriting Recognition Database
Authors: Ibrahim Abdelaziz, Sherif Abdou
Date: 2014-12-24 00:00:00
Tasks: Handwriting Recognition, Sentence

Arabic is a semitic language characterized by a complex and rich morphology.
The exceptional degree of ambiguity in the writing system, the rich morphology,
and the highly complex word formation process of roots and patterns all
contribute to making computational approaches to Arabic very challenging. As a
result, a practical handwriting recognition system should support large
vocabulary to provide a high coverage and use the context information for
disambiguation. Several research efforts have been devoted for building online
Arabic handwriting recognition systems. Most of these methods are either using
their small private test data sets or a standard database with limited lexicon
and coverage. A large scale handwriting database is an essential resource that
can advance the research of online handwriting recognition. Currently, there is
no online Arabic handwriting database with large lexicon, high coverage, large
number of writers and training/testing data.
  In this paper, we introduce AltecOnDB, a large scale online Arabic
handwriting database. AltecOnDB has 98% coverage of all the possible PAWS of
the Arabic language. The collected samples are complete sentences that include
digits and punctuation marks. The collected data is available on sentence, word
and character levels, hence, high-level linguistic models can be used for
performance improvements. Data is collected from more than 1000 writers with
different backgrounds, genders and ages. Annotation and verification tools are
developed to facilitate the annotation and verification phases. We built an
elementary recognition system to test our database and show the existing
difficulties when handling a large vocabulary and dealing with large amounts of
styles variations in the collected data.
Particle Metropolis adjusted Langevin algorithms for state space models
Authors: Chris Nemeth, Paul Fearnhead
Date: 2014-02-04 00:00:00
Tasks: State Space Models

Particle MCMC is a class of algorithms that can be used to analyse
state-space models. They use MCMC moves to update the parameters of the models,
and particle filters to propose values for the path of the state-space model.
Currently the default is to use random walk Metropolis to update the parameter
values. We show that it is possible to use information from the output of the
particle filter to obtain better proposal distributions for the parameters. In
particular it is possible to obtain estimates of the gradient of the log
posterior from each run of the particle filter, and use these estimates within
a Langevin-type proposal. We propose using the recent computationally efficient
approach of Nemeth et al. (2013) for obtaining such estimates. We show
empirically that for a variety of state-space models this proposal is more
efficient than the standard random walk Metropolis proposal in terms of:
reducing autocorrelation of the posterior samples, reducing the burn-in time of
the MCMC sampler and increasing the squared jump distance between posterior
samples.
The Computational Theory of Intelligence: Data Aggregation
Authors: Daniel Kovach
Date: 2014-12-24 00:00:00

In this paper, we will expound upon the concepts proffered in [1], where we
proposed an information theoretic approach to intelligence in the computational
sense. We will examine data and meme aggregation, and study the effect of
limited resources on the resulting meme amplitudes.
The Computational Theory of Intelligence: Information Entropy
Authors: Daniel Kovach
Date: 2014-12-24 00:00:00
Tasks: Clustering

This paper presents an information theoretic approach to the concept of
intelligence in the computational sense. We introduce a probabilistic framework
from which computational intelligence is shown to be an entropy minimizing
process at the local level. Using this new scheme, we develop a simple data
driven clustering example and discuss its applications.
Crypto-Nets: Neural Networks over Encrypted Data
Authors: Pengtao Xie, Misha Bilenko, Tom Finley, Ran Gilad-Bachrach, Kristin Lauter, Michael Naehrig
Date: 2014-12-18 00:00:00
Tasks: Prediction

The problem we address is the following: how can a user employ a predictive
model that is held by a third party, without compromising private information.
For example, a hospital may wish to use a cloud service to predict the
readmission risk of a patient. However, due to regulations, the patient's
medical files cannot be revealed. The goal is to make an inference using the
model, without jeopardizing the accuracy of the prediction or the privacy of
the data.
  To achieve high accuracy, we use neural networks, which have been shown to
outperform other learning models for many tasks. To achieve the privacy
requirements, we use homomorphic encryption in the following protocol: the data
owner encrypts the data and sends the ciphertexts to the third party to obtain
a prediction from a trained model. The model operates on these ciphertexts and
sends back the encrypted prediction. In this protocol, not only the data
remains private, even the values predicted are available only to the data
owner.
  Using homomorphic encryption and modifications to the activation functions
and training algorithms of neural networks, we show that it is protocol is
possible and may be feasible. This method paves the way to build a secure
cloud-based neural network prediction services without invading users' privacy.
Differential Privacy and Machine Learning: a Survey and Review
Authors: Zhanglong Ji, Zachary C. Lipton, Charles Elkan
Date: 2014-12-24 00:00:00
Tasks: BIG-bench Machine Learning, Privacy Preserving, Survey

The objective of machine learning is to extract useful information from data,
while privacy is preserved by concealing information. Thus it seems hard to
reconcile these competing interests. However, they frequently must be balanced
when mining sensitive data. For example, medical research represents an
important application where it is necessary both to extract useful information
and protect patient privacy. One way to resolve the conflict is to extract
general characteristics of whole populations without disclosing the private
information of individuals.
  In this paper, we consider differential privacy, one of the most popular and
powerful definitions of privacy. We explore the interplay between machine
learning and differential privacy, namely privacy-preserving machine learning
algorithms and learning-based data release mechanisms. We also describe some
theoretical results that address what can be learned differentially privately
and upper bounds of loss functions for differentially private algorithms.
  Finally, we present some open questions, including how to incorporate public
data, how to deal with missing data in private datasets, and whether, as the
number of observed samples grows arbitrarily large, differentially private
machine learning algorithms can be achieved at no cost to utility as compared
to corresponding non-differentially private algorithms.
Matching Pursuit LASSO Part II: Applications and Sparse Recovery over Batch Signals
Authors: Mingkui Tan, Ivor W. Tsang, Li Wang
Date: 2013-02-20 00:00:00
Tasks: Compressive Sensing, Face Recognition

Matching Pursuit LASSIn Part I \cite{TanPMLPart1}, a Matching Pursuit LASSO
({MPL}) algorithm has been presented for solving large-scale sparse recovery
(SR) problems. In this paper, we present a subspace search to further improve
the performance of MPL, and then continue to address another major challenge of
SR -- batch SR with many signals, a consideration which is absent from most of
previous $\ell_1$-norm methods. As a result, a batch-mode {MPL} is developed to
vastly speed up sparse recovery of many signals simultaneously. Comprehensive
numerical experiments on compressive sensing and face recognition tasks
demonstrate the superior performance of MPL and BMPL over other methods
considered in this paper, in terms of sparse recovery ability and efficiency.
In particular, BMPL is up to 400 times faster than existing $\ell_1$-norm
methods considered to be state-of-the-art.O Part II: Applications and Sparse
Recovery over Batch Signals
A Time and Space Efficient Junction Tree Architecture
Authors: Stephen Pasteris
Date: 2013-07-31 00:00:00

The junction tree algorithm is a way of computing marginals of boolean
multivariate probability distributions that factorise over sets of random
variables. The junction tree algorithm first constructs a tree called a
junction tree who's vertices are sets of random variables. The algorithm then
performs a generalised version of belief propagation on the junction tree. The
Shafer-Shenoy and Hugin architectures are two ways to perform this belief
propagation that tradeoff time and space complexities in different ways: Hugin
propagation is at least as fast as Shafer-Shenoy propagation and in the cases
that we have large vertices of high degree is significantly faster. However,
this speed increase comes at the cost of an increased space complexity. This
paper first introduces a simple novel architecture, ARCH-1, which has the best
of both worlds: the speed of Hugin propagation and the low space requirements
of Shafer-Shenoy propagation. A more complicated novel architecture, ARCH-2, is
then introduced which has, up to a factor only linear in the maximum
cardinality of any vertex, time and space complexities at least as good as
ARCH-1 and in the cases that we have large vertices of high degree is
significantly faster than ARCH-1.
Model Selection in High-Dimensional Misspecified Models
Authors: Pallavi Basu, Yang Feng, Jinchi Lv
Date: 2014-12-23 00:00:00
Tasks: model, Model Selection, Vocal Bursts Intensity Prediction

Model selection is indispensable to high-dimensional sparse modeling in
selecting the best set of covariates among a sequence of candidate models. Most
existing work assumes implicitly that the model is correctly specified or of
fixed dimensions. Yet model misspecification and high dimensionality are common
in real applications. In this paper, we investigate two classical
Kullback-Leibler divergence and Bayesian principles of model selection in the
setting of high-dimensional misspecified models. Asymptotic expansions of these
principles reveal that the effect of model misspecification is crucial and
should be taken into account, leading to the generalized AIC and generalized
BIC in high dimensions. With a natural choice of prior probabilities, we
suggest the generalized BIC with prior probability which involves a logarithmic
factor of the dimensionality in penalizing model complexity. We further
establish the consistency of the covariance contrast matrix estimator in a
general setting. Our results and new method are supported by numerical studies.
Learning Temporal Dependencies in Data Using a DBN-BLSTM
Authors: Kratarth Goel, Raunaq Vohra
Date: 2014-12-18 00:00:00
Tasks: Music Generation

Since the advent of deep learning, it has been used to solve various problems
using many different architectures. The application of such deep architectures
to auditory data is also not uncommon. However, these architectures do not
always adequately consider the temporal dependencies in data. We thus propose a
new generic architecture called the Deep Belief Network - Bidirectional Long
Short-Term Memory (DBN-BLSTM) network that models sequences by keeping track of
the temporal information while enabling deep representations in the data. We
demonstrate this new architecture by applying it to the task of music
generation and obtain state-of-the-art results.
Quantifying Natural and Artificial Intelligence in Robots and Natural Systems with an Algorithmic Behavioural Test
Authors: Hector Zenil
Date: 2014-12-20 00:00:00
Tasks: Artificial Life

One of the most important aims of the fields of robotics, artificial
intelligence and artificial life is the design and construction of systems and
machines as versatile and as reliable as living organisms at performing high
level human-like tasks. But how are we to evaluate artificial systems if we are
not certain how to measure these capacities in living systems, let alone how to
define life or intelligence? Here I survey a concrete metric towards measuring
abstract properties of natural and artificial systems, such as the ability to
react to the environment and to control one's own behaviour.
Facial Expressions recognition Based on Principal Component Analysis (PCA)
Authors: Abdelmajid Hassan Mansour, Gafar Zen Alabdeen Salh, Ali Shaif Alhalemi
Date: 2014-12-23 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER)

The facial expression recognition is an ocular task that can be performed
without human discomfort, is really a speedily growing on the computer research
field. There are many applications and programs uses facial expression to
evaluate human character, judgment, feelings, and viewpoint. The process of
recognizing facial expression is a hard task due to the several circumstances
such as facial occlusions, face shape, illumination, face colors, and etc. This
paper present a PCA methodology to distinguish expressions of faces under
different circumstances and identifying it. Relies on Eigen faces technique
using standard Data base images. So as to overcome the problem of difficulty to
computers to identify the features and expressions of persons.
Global registration of multiple point clouds using semidefinite programming
Authors: Kunal. N. Chaudhury, Yuehaw Khoo, Amit Singer
Date: 2013-06-21 00:00:00

Consider $N$ points in $\mathbb{R}^d$ and $M$ local coordinate systems that
are related through unknown rigid transforms. For each point we are given
(possibly noisy) measurements of its local coordinates in some of the
coordinate systems. Alternatively, for each coordinate system, we observe the
coordinates of a subset of the points. The problem of estimating the global
coordinates of the $N$ points (up to a rigid transform) from such measurements
comes up in distributed approaches to molecular conformation and sensor network
localization, and also in computer vision and graphics.
  The least-squares formulation of this problem, though non-convex, has a well
known closed-form solution when $M=2$ (based on the singular value
decomposition). However, no closed form solution is known for $M\geq 3$.
  In this paper, we demonstrate how the least-squares formulation can be
relaxed into a convex program, namely a semidefinite program (SDP). By setting
up connections between the uniqueness of this SDP and results from rigidity
theory, we prove conditions for exact and stable recovery for the SDP
relaxation. In particular, we prove that the SDP relaxation can guarantee
recovery under more adversarial conditions compared to earlier proposed
spectral relaxations, and derive error bounds for the registration error
incurred by the SDP relaxation.
  We also present results of numerical experiments on simulated data to confirm
the theoretical findings. We empirically demonstrate that (a) unlike the
spectral relaxation, the relaxation gap is mostly zero for the semidefinite
program (i.e., we are able to solve the original non-convex least-squares
problem) up to a certain noise threshold, and (b) the semidefinite program
performs significantly better than spectral and manifold-optimization methods,
particularly at large noise levels.
Fusing Color and Texture Cues to Categorize the Fruit Diseases from Images
Authors: Shiv Ram Dubey, Anand Singh Jalal
Date: 2014-12-23 00:00:00
Tasks: Clustering, General Classification

The economic and production losses in agricultural industry worldwide are due
to the presence of diseases in the several kinds of fruits. In this paper, a
method for the classification of fruit diseases is proposed and experimentally
validated. The image processing based proposed approach is composed of the
following main steps; in the first step K-Means clustering technique is used
for the defect segmentation, in the second step color and textural cues are
extracted and fused from the segmented image, and finally images are classified
into one of the classes by using a Multi-class Support Vector Machine. We have
considered diseases of apple as a test case and evaluated our approach for
three types of apple diseases namely apple scab, apple blotch and apple rot and
normal apples without diseases. Our experimentation points out that the
proposed fusion scheme can significantly support accurate detection and
automatic classification of fruit diseases.
Text to Multi-level MindMaps: A Novel Method for Hierarchical Visual Abstraction of Natural Language Text
Authors: Mohamed Elhoseiny, Ahmed Elgammal
Date: 2014-08-01 00:00:00

MindMapping is a well-known technique used in note taking, which encourages
learning and studying. MindMapping has been manually adopted to help present
knowledge and concepts in a visual form. Unfortunately, there is no reliable
automated approach to generate MindMaps from Natural Language text. This work
firstly introduces MindMap Multilevel Visualization concept which is to jointly
visualize and summarize textual information. The visualization is achieved
pictorially across multiple levels using semantic information (i.e. ontology),
while the summarization is achieved by the information in the highest levels as
they represent abstract information in the text. This work also presents the
first automated approach that takes a text input and generates a MindMap
visualization out of it. The approach could visualize text documents in
multilevel MindMaps, in which a high-level MindMap node could be expanded into
child MindMaps. \ignore{ As far as we know, this is the first work that view
MindMapping as a new approach to jointly summarize and visualize textual
information.} The proposed method involves understanding of the input text and
converting it into intermediate Detailed Meaning Representation (DMR). The DMR
is then visualized with two modes; Single level or Multiple levels, which is
convenient for larger text. The generated MindMaps from both approaches were
evaluated based on Human Subject experiments performed on Amazon Mechanical
Turk with various parameter settings.
Improved Distributed Principal Component Analysis
Authors: Maria-Florina Balcan, Vandana Kanchanapally, YIngyu Liang, David Woodruff
Date: 2014-08-25 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: Clustering, Computational Efficiency, Distributed Computing

We study the distributed computing setting in which there are multiple
servers, each holding a set of points, who wish to compute functions on the
union of their point sets. A key task in this setting is Principal Component
Analysis (PCA), in which the servers would like to compute a low dimensional
subspace capturing as much of the variance of the union of their point sets as
possible. Given a procedure for approximate PCA, one can use it to
approximately solve $\ell_2$-error fitting problems such as $k$-means
clustering and subspace clustering. The essential properties of an approximate
distributed PCA algorithm are its communication cost and computational
efficiency for a given desired accuracy in downstream applications. We give new
algorithms and analyses for distributed PCA which lead to improved
communication and computational costs for $k$-means clustering and related
problems. Our empirical study on real world data shows a speedup of orders of
magnitude, preserving communication with only a negligible degradation in
solution quality. Some of these techniques we develop, such as a general
transformation from a constant success probability subspace embedding to a high
success probability subspace embedding with a dimension and sparsity
independent of the success probability, may be of independent interest.
An $\{l_1,l_2,l_{\infty}\}$-Regularization Approach to High-Dimensional Errors-in-variables Models
Authors: Alexandre Belloni, Mathieu Rosenbaum, Alexandre B. Tsybakov
Date: 2014-12-22 00:00:00
Tasks: regression

Several new estimation methods have been recently proposed for the linear
regression model with observation error in the design. Different assumptions on
the data generating process have motivated different estimators and analysis.
In particular, the literature considered (1) observation errors in the design
uniformly bounded by some $\bar \delta$, and (2) zero mean independent
observation errors. Under the first assumption, the rates of convergence of the
proposed estimators depend explicitly on $\bar \delta$, while the second
assumption has been applied when an estimator for the second moment of the
observational error is available. This work proposes and studies two new
estimators which, compared to other procedures for regression models with
errors in the design, exploit an additional $l_{\infty}$-norm regularization.
The first estimator is applicable when both (1) and (2) hold but does not
require an estimator for the second moment of the observational error. The
second estimator is applicable under (2) and requires an estimator for the
second moment of the observation error. Importantly, we impose no assumption on
the accuracy of this pilot estimator, in contrast to the previously known
procedures. As the recent proposals, we allow the number of covariates to be
much larger than the sample size. We establish the rates of convergence of the
estimators and compare them with the bounds obtained for related estimators in
the literature. These comparisons show interesting insights on the interplay of
the assumptions and the achievable rates of convergence.
Online Distributed Optimization on Dynamic Networks
Authors: Saghar Hosseini, Airlie Chapman, Mehran Mesbahi
Date: 2014-12-22 00:00:00
Tasks: Distributed Optimization

This paper presents a distributed optimization scheme over a network of
agents in the presence of cost uncertainties and over switching communication
topologies. Inspired by recent advances in distributed convex optimization, we
propose a distributed algorithm based on a dual sub-gradient averaging. The
objective of this algorithm is to minimize a cost function cooperatively.
Furthermore, the algorithm changes the weights on the communication links in
the network to adapt to varying reliability of neighboring agents. A
convergence rate analysis as a function of the underlying network topology is
then presented, followed by simulation results for representative classes of
sensor networks.
Non-stationary Stochastic Optimization
Authors: O. Besbes, Y. Gur, A. Zeevi
Date: 2013-07-20 00:00:00
Tasks: Stochastic Optimization

We consider a non-stationary variant of a sequential stochastic optimization
problem, in which the underlying cost functions may change along the horizon.
We propose a measure, termed variation budget, that controls the extent of said
change, and study how restrictions on this budget impact achievable
performance. We identify sharp conditions under which it is possible to achieve
long-run-average optimality and more refined performance measures such as rate
optimality that fully characterize the complexity of such problems. In doing
so, we also establish a strong connection between two rather disparate strands
of literature: adversarial online convex optimization; and the more traditional
stochastic approximation paradigm (couched in a non-stationary setting). This
connection is the key to deriving well performing policies in the latter, by
leveraging structure of optimal policies in the former. Finally, tight bounds
on the minimax regret allow us to quantify the "price of non-stationarity,"
which mathematically captures the added complexity embedded in a temporally
changing environment versus a stationary one.