pwc-archive/papers-with-abstracts huggingface dataset:
Distinction between features extracted using deep belief networks
Authors: Mohammad Pezeshki, Sajjad Gholami, Ahmad Nickabadi
Date: 2013-12-20 00:00:00
Tasks: BIG-bench Machine Learning, Face Recognition

Data representation is an important pre-processing step in many machine
learning algorithms. There are a number of methods used for this task such as
Deep Belief Networks (DBNs) and Discrete Fourier Transforms (DFTs). Since some
of the features extracted using automated feature extraction methods may not
always be related to a specific machine learning task, in this paper we propose
two methods in order to make a distinction between extracted features based on
their relevancy to the task. We applied these two methods to a Deep Belief
Network trained for a face recognition task.
Deep Belief Networks for Image Denoising
Authors: Mohammad Ali Keyvanrad, Mohammad Pezeshki, Mohammad Ali Homayounpour
Date: 2013-12-20 00:00:00
Tasks: Denoising, Image Denoising

Deep Belief Networks which are hierarchical generative models are effective
tools for feature representation and extraction. Furthermore, DBNs can be used
in numerous aspects of Machine Learning such as image denoising. In this paper,
we propose a novel method for image denoising which relies on the DBNs' ability
in feature representation. This work is based upon learning of the noise
behavior. Generally, features which are extracted using DBNs are presented as
the values of the last layer nodes. We train a DBN a way that the network
totally distinguishes between nodes presenting noise and nodes presenting image
content in the last later of DBN, i.e. the nodes in the last layer of trained
DBN are divided into two distinct groups of nodes. After detecting the nodes
which are presenting the noise, we are able to make the noise nodes inactive
and reconstruct a noiseless image. In section 4 we explore the results of
applying this method on the MNIST dataset of handwritten digits which is
corrupted with additive white Gaussian noise (AWGN). A reduction of 65.9% in
average mean square error (MSE) was achieved when the proposed method was used
for the reconstruction of the noisy images.
An empirical analysis of dropout in piecewise linear networks
Authors: David Warde-Farley, Ian J. Goodfellow, Aaron Courville, Yoshua Bengio
Date: 2013-12-21 00:00:00

The recently introduced dropout training criterion for neural networks has
been the subject of much attention due to its simplicity and remarkable
effectiveness as a regularizer, as well as its interpretation as a training
procedure for an exponentially large ensemble of networks that share
parameters. In this work we empirically investigate several questions related
to the efficacy of dropout, specifically as it concerns networks employing the
popular rectified linear activation function. We investigate the quality of the
test time weight-scaling inference procedure by evaluating the geometric
average exactly in small models, as well as compare the performance of the
geometric mean to the arithmetic mean more commonly employed by ensemble
techniques. We explore the effect of tied weights on the ensemble
interpretation by training ensembles of masked networks without tied weights.
Finally, we investigate an alternative criterion based on a biased estimator of
the maximum likelihood ensemble gradient.
Reducing the Computational Cost in Multi-objective Evolutionary Algorithms by Filtering Worthless Individuals
Authors: Zahra Pourbahman, Ali Hamzeh
Date: 2014-01-02 00:00:00
Tasks: Evolutionary Algorithms

The large number of exact fitness function evaluations makes evolutionary
algorithms to have computational cost. In some real-world problems, reducing
number of these evaluations is much more valuable even by increasing
computational complexity and spending more time. To fulfill this target, we
introduce an effective factor, in spite of applied factor in Adaptive Fuzzy
Fitness Granulation with Non-dominated Sorting Genetic Algorithm-II, to filter
out worthless individuals more precisely. Our proposed approach is compared
with respect to Adaptive Fuzzy Fitness Granulation with Non-dominated Sorting
Genetic Algorithm-II, using the Hyper volume and the Inverted Generational
Distance performance measures. The proposed method is applied to 1 traditional
and 1 state-of-the-art benchmarks with considering 3 different dimensions. From
an average performance view, the results indicate that although decreasing the
number of fitness evaluations leads to have performance reduction but it is not
tangible compared to what we gain.
Hybrid Approach to Face Recognition System using Principle component and Independent component with score based fusion process
Authors: Trupti M. Kodinariya
Date: 2014-01-02 00:00:00
Tasks: Face Recognition, General Classification

Hybrid approach has a special status among Face Recognition Systems as they combine different recognition approaches in an either serial or parallel to overcome the shortcomings of individual methods. This paper explores the area of Hybrid Face Recognition using score based strategy as a combiner/fusion process. In proposed approach, the recognition system operates in two modes: training and classification. Training mode involves normalization of the face images (training set), extracting appropriate features using Principle Component Analysis (PCA) and Independent Component Analysis (ICA). The extracted features are then trained in parallel using Back-propagation neural networks (BPNNs) to partition the feature space in to different face classes. In classification mode, the trained PCA BPNN and ICA BPNN are fed with new face image(s). The score based strategy which works as a combiner is applied to the results of both PCA BPNN and ICA BPNN to classify given new face image(s) according to face classes obtained during the training mode. The proposed approach has been tested on ORL and other face databases; the experimented results show that the proposed system has higher accuracy than face recognition systems using single feature extractor.
Relaxations for inference in restricted Boltzmann machines
Authors: Sida I. Wang, Roy Frostig, Percy Liang, Christopher D. Manning
Date: 2013-12-21 00:00:00

We propose a relaxation-based approximate inference algorithm that samples
near-MAP configurations of a binary pairwise Markov random field. We experiment
on MAP inference tasks in several restricted Boltzmann machines. We also use
our underlying sampler to estimate the log-partition function of restricted
Boltzmann machines and compare against other sampling-based methods.
Generalization Bounds for Representative Domain Adaptation
Authors: Chao Zhang, Lei Zhang, Wei Fan, Jieping Ye
Date: 2014-01-02 00:00:00
Tasks: Domain Adaptation, Generalization Bounds, Vocal Bursts Type Prediction

In this paper, we propose a novel framework to analyze the theoretical
properties of the learning process for a representative type of domain
adaptation, which combines data from multiple sources and one target (or
briefly called representative domain adaptation). In particular, we use the
integral probability metric to measure the difference between the distributions
of two domains and meanwhile compare it with the H-divergence and the
discrepancy distance. We develop the Hoeffding-type, the Bennett-type and the
McDiarmid-type deviation inequalities for multiple domains respectively, and
then present the symmetrization inequality for representative domain
adaptation. Next, we use the derived inequalities to obtain the Hoeffding-type
and the Bennett-type generalization bounds respectively, both of which are
based on the uniform entropy number. Moreover, we present the generalization
bounds based on the Rademacher complexity. Finally, we analyze the asymptotic
convergence and the rate of convergence of the learning process for
representative domain adaptation. We discuss the factors that affect the
asymptotic behavior of the learning process and the numerical experiments
support our theoretical findings as well. Meanwhile, we give a comparison with
the existing results of domain adaptation and the classical results under the
same-distribution assumption.
Convex optimization on Banach Spaces
Authors: R. A. DeVore, V. N. Temlyakov
Date: 2014-01-01 00:00:00

Greedy algorithms which use only function evaluations are applied to convex
optimization in a general Banach space $X$. Along with algorithms that use
exact evaluations, algorithms with approximate evaluations are treated. A
priori upper bounds for the convergence rate of the proposed algorithms are
given. These bounds depend on the smoothness of the objective function and the
sparsity or compressibility (with respect to a given dictionary) of a point in
$X$ where the minimum is attained.
Design of a GIS-based Assistant Software Agent for the Incident Commander to Coordinate Emergency Response Operations
Authors: Reza Nourjou, Michinori Hatayama, Stephen F. Smith, Atabak Sadeghi, Pedro Szekely
Date: 2014-01-01 00:00:00

Problem: This paper addresses the design of an intelligent software system
for the IC (incident commander) of a team in order to coordinate actions of
agents (field units or robots) in the domain of emergency/crisis response
operations. Objective: This paper proposes GICoordinator. It is a GIS-based
assistant software agent that assists and collaborates with the human planner
in strategic planning and macro tasks assignment for centralized multi-agent
coordination. Method: Our approach to design GICoordinator was to: analyze the
problem, design a complete data model, design an architecture of GICoordinator,
specify required capabilities of human and system in coordination problem
solving, specify development tools, and deploy. Result: The result was an
architecture/design of GICoordinator that contains system requirements.
Findings: GICoordinator efficiently integrates geoinformatics with artifice
intelligent techniques in order to provide a spatial intelligent coordinator
system for an IC to efficiently coordinate and control agents by making
macro/strategic decisions. Results define a framework for future works to
develop this system.
Design of an Agent for Answering Back in Smart Phones
Authors: Sandeep Venkatesh, Meera V Patil, Nanditha Swamy
Date: 2013-06-25 00:00:00

The objective of the paper is to design an agent which provides efficient
response to the caller when a call goes unanswered in smartphones. The agent
provides responses through text messages, email etc stating the most likely
reason as to why the callee is unable to answer a call. Responses are composed
taking into consideration the importance of the present call and the situation
the callee is in at the moment like driving, sleeping, at work etc. The agent
makes decisons in the compostion of response messages based on the patterns it
has come across in the learning environment. Initially the user helps the agent
to compose response messages. The agent associates this message to the percept
it recieves with respect to the environment the callee is in. The user may
thereafter either choose to make to response system automatic or choose to
recieve suggestions from the agent for responses messages and confirm what is
to be sent to the caller.
Direct Learning of Sparse Changes in Markov Networks by Density Ratio Estimation
Authors: Song Liu, John A. Quinn, Michael U. Gutmann, Taiji Suzuki, Masashi Sugiyama
Date: 2013-04-25 00:00:00
Tasks: Density Ratio Estimation

We propose a new method for detecting changes in Markov network structure
between two sets of samples. Instead of naively fitting two Markov network
models separately to the two data sets and figuring out their difference, we
\emph{directly} learn the network structure change by estimating the ratio of
Markov network models. This density-ratio formulation naturally allows us to
introduce sparsity in the network structure change, which highly contributes to
enhancing interpretability. Furthermore, computation of the normalization term,
which is a critical bottleneck of the naive approach, can be remarkably
mitigated. We also give the dual formulation of the optimization problem, which
further reduces the computation cost for large-scale Markov networks. Through
experiments, we demonstrate the usefulness of our method.
Modeling Attractiveness and Multiple Clicks in Sponsored Search Results
Authors: Dinesh Govindaraj, Tao Wang, S. V. N. Vishwanathan
Date: 2014-01-01 00:00:00

Click models are an important tool for leveraging user feedback, and are used
by commercial search engines for surfacing relevant search results. However,
existing click models are lacking in two aspects. First, they do not share
information across search results when computing attractiveness. Second, they
assume that users interact with the search results sequentially. Based on our
analysis of the click logs of a commercial search engine, we observe that the
sequential scan assumption does not always hold, especially for sponsored
search results. To overcome the above two limitations, we propose a new click
model. Our key insight is that sharing information across search results helps
in identifying important words or key-phrases which can then be used to
accurately compute attractiveness of a search result. Furthermore, we argue
that the click probability of a position as well as its attractiveness changes
during a user session and depends on the user's past click experience. Our
model seamlessly incorporates the effect of externalities (quality of other
search results displayed in response to a user query), user fatigue, as well as
pre and post-click relevance of a sponsored search result. We propose an
efficient one-pass inference scheme and empirically evaluate the performance of
our model via extensive experiments using the click logs of a large commercial
search engine.
A Review: Expert System for Diagnosis of Myocardial Infarction
Authors: S. J Gath, R. V Kulkarni
Date: 2014-01-01 00:00:00
Tasks: Management

A computer Program Capable of performing at a human-expert level in a narrow
problem domain area is called an expert system. Management of uncertainty is an
intrinsically important issue in the design of expert systems because much of
the information in the knowledge base of a typical expert system is imprecise,
incomplete or not totally reliable. In this paper, the author present s the
review of past work that has been carried out by various researchers based on
development of expert systems for the diagnosis of cardiac disease
Consistent Bounded-Asynchronous Parameter Servers for Distributed ML
Authors: Jinliang Wei, Wei Dai, Abhimanu Kumar, Xun Zheng, Qirong Ho, Eric P. Xing
Date: 2013-12-30 00:00:00

In distributed ML applications, shared parameters are usually replicated
among computing nodes to minimize network overhead. Therefore, proper
consistency model must be carefully chosen to ensure algorithm's correctness
and provide high throughput. Existing consistency models used in
general-purpose databases and modern distributed ML systems are either too
loose to guarantee correctness of the ML algorithms or too strict and thus fail
to fully exploit the computing power of the underlying distributed system.
  Many ML algorithms fall into the category of \emph{iterative convergent
algorithms} which start from a randomly chosen initial point and converge to
optima by repeating iteratively a set of procedures. We've found that many such
algorithms are to a bounded amount of inconsistency and still converge
correctly. This property allows distributed ML to relax strict consistency
models to improve system performance while theoretically guarantees algorithmic
correctness. In this paper, we present several relaxed consistency models for
asynchronous parallel computation and theoretically prove their algorithmic
correctness. The proposed consistency models are implemented in a distributed
parameter server and evaluated in the context of a popular ML application:
topic modeling.
Sparse Recovery with Very Sparse Compressed Counting
Authors: Ping Li, Cun-Hui Zhang, Tong Zhang
Date: 2013-12-31 00:00:00
Tasks: compressed sensing

Compressed sensing (sparse signal recovery) often encounters nonnegative data
(e.g., images). Recently we developed the methodology of using (dense)
Compressed Counting for recovering nonnegative K-sparse signals. In this paper,
we adopt very sparse Compressed Counting for nonnegative signal recovery. Our
design matrix is sampled from a maximally-skewed p-stable distribution (0<p<1),
and we sparsify the design matrix so that on average (1-g)-fraction of the
entries become zero. The idea is related to very sparse stable random
projections (Li et al 2006 and Li 2007), the prior work for estimating summary
statistics of the data.
  In our theoretical analysis, we show that, when p->0, it suffices to use M=
K/(1-exp(-gK) log N measurements, so that all coordinates can be recovered in
one scan of the coordinates. If g = 1 (i.e., dense design), then M = K log N.
If g= 1/K or 2/K (i.e., very sparse design), then M = 1.58K log N or M = 1.16K
log N. This means the design matrix can be indeed very sparse at only a minor
inflation of the sample complexity.
  Interestingly, as p->1, the required number of measurements is essentially M
= 2.7K log N, provided g= 1/K. It turns out that this result is a general
worst-case bound.
Query-focused Multi-document Summarization: Combining a Novel Topic Model with Graph-based Semi-supervised Learning
Authors: Jiwei Li, Sujian Li
Date: 2012-12-10 00:00:00
Tasks: Document Summarization, Multi-Document Summarization, Sentence

Graph-based semi-supervised learning has proven to be an effective approach
for query-focused multi-document summarization. The problem of previous
semi-supervised learning is that sentences are ranked without considering the
higher level information beyond sentence level. Researches on general
summarization illustrated that the addition of topic level can effectively
improve the summary quality. Inspired by previous researches, we propose a
two-layer (i.e. sentence layer and topic layer) graph-based semi-supervised
learning approach. At the same time, we propose a novel topic model which makes
full use of the dependence between sentences and words. Experimental results on
DUC and TAC data sets demonstrate the effectiveness of our proposed approach.
Decision Making under Uncertainty: A Quasimetric Approach
Authors: Steve N'Guyen, Clément Moulin-Frier, Jacques Droulez
Date: 2013-12-31 00:00:00
Tasks: Decision Making, Decision Making Under Uncertainty

We propose a new approach for solving a class of discrete decision making
problems under uncertainty with positive cost. This issue concerns multiple and
diverse fields such as engineering, economics, artificial intelligence,
cognitive science and many others. Basically, an agent has to choose a single
or series of actions from a set of options, without knowing for sure their
consequences. Schematically, two main approaches have been followed: either the
agent learns which option is the correct one to choose in a given situation by
trial and error, or the agent already has some knowledge on the possible
consequences of his decisions; this knowledge being generally expressed as a
conditional probability distribution. In the latter case, several optimal or
suboptimal methods have been proposed to exploit this uncertain knowledge in
various contexts. In this work, we propose following a different approach,
based on the geometric intuition of distance. More precisely, we define a goal
independent quasimetric structure on the state space, taking into account both
cost function and transition probability. We then compare precision and
computation time with classical approaches.
Gaussian Process Kernels for Pattern Discovery and Extrapolation
Authors: Andrew Gordon Wilson, Ryan Prescott Adams
Date: 2013-02-18 00:00:00
Tasks: Gaussian Processes

Gaussian processes are rich distributions over functions, which provide a
Bayesian nonparametric approach to smoothing and interpolation. We introduce
simple closed form kernels that can be used with Gaussian processes to discover
patterns and enable extrapolation. These kernels are derived by modelling a
spectral density -- the Fourier transform of a kernel -- with a Gaussian
mixture. The proposed kernels support a broad class of stationary covariances,
but Gaussian process inference remains simple and analytic. We demonstrate the
proposed kernels by discovering patterns and performing long range
extrapolation on synthetic examples, as well as atmospheric CO2 trends and
airline passenger data. We also show that we can reconstruct standard
covariances within our framework.
Medical Image Fusion: A survey of the state of the art
Authors: A. P. James, B. V. Dasarathy
Date: 2013-12-31 00:00:00

Medical image fusion is the process of registering and combining multiple
images from single or multiple imaging modalities to improve the imaging
quality and reduce randomness and redundancy in order to increase the clinical
applicability of medical images for diagnosis and assessment of medical
problems. Multi-modal medical image fusion algorithms and devices have shown
notable achievements in improving clinical accuracy of decisions based on
medical images. This review article provides a factual listing of methods and
summarizes the broad scientific challenges faced in the field of medical image
fusion. We characterize the medical image fusion research based on (1) the
widely used image fusion methods, (2) imaging modalities, and (3) imaging of
organs that are under study. This review concludes that even though there
exists several open ended technological and scientific challenges, the fusion
of medical images has proved to be useful for advancing the clinical
reliability of using medical imaging for medical diagnostics and analysis, and
is a scientific discipline that has the potential to significantly grow in the
coming years.
Speeding-Up Convergence via Sequential Subspace Optimization: Current State and Future Directions
Authors: Michael Zibulevsky
Date: 2013-12-31 00:00:00
Tasks: Compressive Sensing, Denoising, Image Denoising, Stochastic Optimization

This is an overview paper written in style of research proposal. In recent
years we introduced a general framework for large-scale unconstrained
optimization -- Sequential Subspace Optimization (SESOP) and demonstrated its
usefulness for sparsity-based signal/image denoising, deconvolution,
compressive sensing, computed tomography, diffraction imaging, support vector
machines. We explored its combination with Parallel Coordinate Descent and
Separable Surrogate Function methods, obtaining state of the art results in
above-mentioned areas. There are several methods, that are faster than plain
SESOP under specific conditions: Trust region Newton method - for problems with
easily invertible Hessian matrix; Truncated Newton method - when fast
multiplication by Hessian is available; Stochastic optimization methods - for
problems with large stochastic-type data; Multigrid methods - for problems with
nested multilevel structure. Each of these methods can be further improved by
merge with SESOP. One can also accelerate Augmented Lagrangian method for
constrained optimization problems and Alternating Direction Method of
Multipliers for problems with separable objective function and non-separable
constraints.