pwc-archive/papers-with-abstracts huggingface dataset:
Accelerating Hessian-free optimization for deep neural networks by implicit preconditioning and sampling
Authors: Tara N. Sainath, Lior Horesh, Brian Kingsbury, Aleksandr Y. Aravkin, Bhuvana Ramabhadran
Date: 2013-09-05 00:00:00

Hessian-free training has become a popular parallel second or- der
optimization technique for Deep Neural Network training. This study aims at
speeding up Hessian-free training, both by means of decreasing the amount of
data used for training, as well as through reduction of the number of Krylov
subspace solver iterations used for implicit estimation of the Hessian. In this
paper, we develop an L-BFGS based preconditioning scheme that avoids the need
to access the Hessian explicitly. Since L-BFGS cannot be regarded as a
fixed-point iteration, we further propose the employment of flexible Krylov
subspace solvers that retain the desired theoretical convergence guarantees of
their conventional counterparts. Second, we propose a new sampling algorithm,
which geometrically increases the amount of data utilized for gradient and
Krylov subspace iteration calculations. On a 50-hr English Broadcast News task,
we find that these methodologies provide roughly a 1.5x speed-up, whereas, on a
300-hr Switchboard task, these techniques provide over a 2.3x speedup, with no
loss in WER. These results suggest that even further speed-up is expected, as
problems scale and complexity grows.
Improvements to deep convolutional neural networks for LVCSR
Authors: Tara N. Sainath, Brian Kingsbury, Abdel-rahman Mohamed, George E. Dahl, George Saon, Hagen Soltau, Tomas Beran, Aleksandr Y. Aravkin, Bhuvana Ramabhadran
Date: 2013-09-05 00:00:00
Tasks: Speech Recognition

Deep Convolutional Neural Networks (CNNs) are more powerful than Deep Neural
Networks (DNN), as they are able to better reduce spectral variation in the
input signal. This has also been confirmed experimentally, with CNNs showing
improvements in word error rate (WER) between 4-12% relative compared to DNNs
across a variety of LVCSR tasks. In this paper, we describe different methods
to further improve CNN performance. First, we conduct a deep analysis comparing
limited weight sharing and full weight sharing with state-of-the-art features.
Second, we apply various pooling strategies that have shown improvements in
computer vision to an LVCSR speech task. Third, we introduce a method to
effectively incorporate speaker adaptation, namely fMLLR, into log-mel
features. Fourth, we introduce an effective strategy to use dropout during
Hessian-free sequence training. We find that with these improvements,
particularly with fMLLR and dropout, we are able to achieve an additional 2-3%
relative improvement in WER on a 50-hour Broadcast News task over our previous
best CNN baseline. On a larger 400-hour BN task, we find an additional 4-5%
relative improvement over our previous best CNN baseline.
Improving circuit miniaturization and its efficiency using Rough Set Theory
Authors: Sarvesh SS Rawat, Dheeraj Dilip Mor, Anugrah Kumar, Sanjiban Shekar Roy, Rohit Kumar
Date: 2013-12-10 00:00:00

High-speed, accuracy, meticulousness and quick response are notion of the
vital necessities for modern digital world. An efficient electronic circuit
unswervingly affects the maneuver of the whole system. Different tools are
required to unravel different types of engineering tribulations. Improving the
efficiency, accuracy and low power consumption in an electronic circuit is
always been a bottle neck problem. So the need of circuit miniaturization is
always there. It saves a lot of time and power that is wasted in switching of
gates, the wiring-crises is reduced, cross-sectional area of chip is reduced,
the number of transistors that can implemented in chip is multiplied many
folds. Therefore to trounce with this problem we have proposed an Artificial
intelligence (AI) based approach that make use of Rough Set Theory for its
implementation. Theory of rough set has been proposed by Z Pawlak in the year
1982. Rough set theory is a new mathematical tool which deals with uncertainty
and vagueness. Decisions can be generated using rough set theory by reducing
the unwanted and superfluous data. We have condensed the number of gates
without upsetting the productivity of the given circuit. This paper proposes an
approach with the help of rough set theory which basically lessens the number
of gates in the circuit, based on decision rules.
Phishing Detection by determining reliability factor using rough set theory
Authors: Anugrah Kumar, Sanjiban Shekar Roy, Sarvesh SS Rawat, Sanklan Saxena
Date: 2013-12-10 00:00:00
Tasks: valid

Phishing is a common online weapon, used against users, by Phishers for
acquiring a confidential information through deception. Since the inception of
internet, nearly everything, ranging from money transaction to sharing
information, is done online in most parts of the world. This has also given
rise to malicious activities such as Phishing. Detecting Phishing is an
intricate process due to complexity, ambiguity and copious amount of
possibilities of factors responsible for phishing . Rough sets can be a
powerful tool, when working on such kind of Applications containing vague or
imprecise data. This paper proposes an approach towards Phishing Detection
Using Rough Set Theory. The Thirteen basic factors, directly responsible
towards Phishing, are grouped into four Strata. Reliability Factor is
determined on the basis of the outcome of these strata, using Rough Set Theory
. Reliability Factor determines the possibility of a suspected site to be Valid
or Fake. Using Rough set Theory most and the least influential factors towards
Phishing are also determined.
Multi-Task Classification Hypothesis Space with Improved Generalization Bounds
Authors: Cong Li, Michael Georgiopoulos, Georgios C. Anagnostopoulos
Date: 2013-12-09 00:00:00
Tasks: Classification, General Classification, Generalization Bounds, Multi-Task Learning

This paper presents a RKHS, in general, of vector-valued functions intended
to be used as hypothesis space for multi-task classification. It extends
similar hypothesis spaces that have previously considered in the literature.
Assuming this space, an improved Empirical Rademacher Complexity-based
generalization bound is derived. The analysis is itself extended to an MKL
setting. The connection between the proposed hypothesis space and a Group-Lasso
type regularizer is discussed. Finally, experimental results, with some
SVM-based Multi-Task Learning problems, underline the quality of the derived
bounds and validate the paper's analysis.
Object Recognition System Design in Computer Vision: a Universal Approach
Authors: Andrew Gleibman
Date: 2013-10-27 00:00:00
Tasks: General Classification, object-detection, Object Detection, Object Recognition

The first contribution of this paper is architecture of a multipurpose
system, which delegates a range of object detection tasks to a classifier,
applied in special grid positions of the tested image. The second contribution
is Gray Level-Radius Co-occurrence Matrix, which describes local image texture
and topology and, unlike common second order statistics methods, is robust to
image resolution. The third contribution is a parametrically controlled
automatic synthesis of unlimited number of numerical features for
classification. The fourth contribution is a method of optimizing parameters C
and gamma in LibSVM-based classifier, which is 20-100 times faster than the
commonly applied method. The work is essentially experimental, with
demonstration of various methods for definition of objects of interest in
images and video sequences.
An Application of Answer Set Programming to the Field of Second Language Acquisition
Authors: Daniela Inclezan
Date: 2013-12-09 00:00:00
Tasks: Language Acquisition

This paper explores the contributions of Answer Set Programming (ASP) to the
study of an established theory from the field of Second Language Acquisition:
Input Processing. The theory describes default strategies that learners of a
second language use in extracting meaning out of a text, based on their
knowledge of the second language and their background knowledge about the
world. We formalized this theory in ASP, and as a result we were able to
determine opportunities for refining its natural language description, as well
as directions for future theory development. We applied our model to automating
the prediction of how learners of English would interpret sentences containing
the passive voice. We present a system, PIas, that uses these predictions to
assist language instructors in designing teaching materials. To appear in
Theory and Practice of Logic Programming (TPLP).
On the Performance of Filters for Reduction of Speckle Noise in SAR Images off the Coast of the Gulf of Guinea
Authors: Griffith S. Klogo, Akpeko Gasonoo, Isaac K. E. Ampomah
Date: 2013-12-09 00:00:00

Synthetic Aperture Radar (SAR) imagery to monitor oil spills are some methods
that have been proposed for the West African sub-region. With the increase in
the number of oil exploration companies in Ghana (and her neighbors) and the
rise in the coastal activities in the sub-region, there is the need for proper
monitoring of the environmental impact of these socio-economic activities on
the environment. Detection and near real-time information about oil spills are
fundamental in reducing oil spill environmental impact. SAR images are prone to
some noise, which is predominantly speckle noise around the coastal areas. This
paper evaluates the performance of the mean and median filters used in the
preprocessing filtering to reduce speckle noise in SAR images for most image
processing algorithms.
A Unified Markov Chain Approach to Analysing Randomised Search Heuristics
Authors: Jun He, Feidun He, Xin Yao
Date: 2013-12-09 00:00:00

The convergence, convergence rate and expected hitting time play fundamental
roles in the analysis of randomised search heuristics. This paper presents a
unified Markov chain approach to studying them. Using the approach, the
sufficient and necessary conditions of convergence in distribution are
established. Then the average convergence rate is introduced to randomised
search heuristics and its lower and upper bounds are derived. Finally, novel
average drift analysis and backward drift analysis are proposed for bounding
the expected hitting time. A computational study is also conducted to
investigate the convergence, convergence rate and expected hitting time. The
theoretical study belongs to a prior and general study while the computational
study belongs to a posterior and case study.
A preliminary survey on optimized multiobjective metaheuristic methods for data clustering using evolutionary approaches
Authors: Ramachandra Rao Kurada, Dr. K Karteeka Pavan, Dr. AV Dattareya Rao
Date: 2013-12-09 00:00:00
Tasks: Clustering, Multiobjective Optimization, Survey

The present survey provides the state-of-the-art of research, copiously
devoted to Evolutionary Approach (EAs) for clustering exemplified with a
diversity of evolutionary computations. The Survey provides a nomenclature that
highlights some aspects that are very important in the context of evolutionary
data clustering. The paper missions the clustering trade-offs branched out with
wide-ranging Multi Objective Evolutionary Approaches (MOEAs) methods. Finally,
this study addresses the potential challenges of MOEA design and data
clustering, along with conclusions and recommendations for novice and
researchers by positioning most promising paths of future research. MOEAs have
substantial success across a variety of MOP applications, from pedagogical
multifunction optimization to real-world engineering design. The survey paper
noticeably organizes the developments witnessed in the past three decades for
EAs based metaheuristics to solve multiobjective optimization problems (MOP)
and to derive significant progression in ruling high quality elucidations in a
single run. Data clustering is an exigent task, whose intricacy is caused by a
lack of unique and precise definition of a cluster. The discrete optimization
problem uses the cluster space to derive a solution for Multiobjective data
clustering. Discovery of a majority or all of the clusters (of illogical
shapes) present in the data is a long-standing goal of unsupervised predictive
learning problems or exploratory pattern analysis.
Event-Driven Contrastive Divergence for Spiking Neuromorphic Systems
Authors: Emre Neftci, Srinjoy Das, Bruno Pedroni, Kenneth Kreutz-Delgado, Gert Cauwenberghs
Date: 2013-11-05 00:00:00
Tasks: Dimensionality Reduction

Restricted Boltzmann Machines (RBMs) and Deep Belief Networks have been
demonstrated to perform efficiently in a variety of applications, such as
dimensionality reduction, feature learning, and classification. Their
implementation on neuromorphic hardware platforms emulating large-scale
networks of spiking neurons can have significant advantages from the
perspectives of scalability, power dissipation and real-time interfacing with
the environment. However the traditional RBM architecture and the commonly used
training algorithm known as Contrastive Divergence (CD) are based on discrete
updates and exact arithmetics which do not directly map onto a dynamical neural
substrate. Here, we present an event-driven variation of CD to train a RBM
constructed with Integrate & Fire (I&F) neurons, that is constrained by the
limitations of existing and near future neuromorphic hardware platforms. Our
strategy is based on neural sampling, which allows us to synthesize a spiking
neural network that samples from a target Boltzmann distribution. The recurrent
activity of the network replaces the discrete steps of the CD algorithm, while
Spike Time Dependent Plasticity (STDP) carries out the weight updates in an
online, asynchronous fashion. We demonstrate our approach by training an RBM
composed of leaky I&F neurons with STDP synapses to learn a generative model of
the MNIST hand-written digit dataset, and by testing it in recognition,
generation and cue integration tasks. Our results contribute to a machine
learning-driven approach for synthesizing networks of spiking neurons capable
of carrying out practical, high-level functionality.
Bayesian Structural Inference for Hidden Processes
Authors: Christopher C. Strelioff, James P. Crutchfield
Date: 2013-09-05 00:00:00

We introduce a Bayesian approach to discovering patterns in structurally
complex processes. The proposed method of Bayesian Structural Inference (BSI)
relies on a set of candidate unifilar HMM (uHMM) topologies for inference of
process structure from a data series. We employ a recently developed exact
enumeration of topological epsilon-machines. (A sequel then removes the
topological restriction.) This subset of the uHMM topologies has the added
benefit that inferred models are guaranteed to be epsilon-machines,
irrespective of estimated transition probabilities. Properties of
epsilon-machines and uHMMs allow for the derivation of analytic expressions for
estimating transition probabilities, inferring start states, and comparing the
posterior probability of candidate model topologies, despite process internal
structure being only indirectly present in data. We demonstrate BSI's
effectiveness in estimating a process's randomness, as reflected by the Shannon
entropy rate, and its structure, as quantified by the statistical complexity.
We also compare using the posterior distribution over candidate models and the
single, maximum a posteriori model for point estimation and show that the
former more accurately reflects uncertainty in estimated values. We apply BSI
to in-class examples of finite- and infinite-order Markov processes, as well to
an out-of-class, infinite-state hidden process.
State Transition Algorithm
Authors: Xiaojun Zhou, Chunhua Yang, Weihua Gui
Date: 2012-05-30 00:00:00
Tasks: Translation

In terms of the concepts of state and state transition, a new heuristic
random search algorithm named state transition algorithm is proposed. For
continuous function optimization problems, four special transformation
operators called rotation, translation, expansion and axesion are designed.
Adjusting measures of the transformations are mainly studied to keep the
balance of exploration and exploitation. Convergence analysis is also discussed
about the algorithm based on random search theory. In the meanwhile, to
strengthen the search ability in high dimensional space, communication strategy
is introduced into the basic algorithm and intermittent exchange is presented
to prevent premature convergence. Finally, experiments are carried out for the
algorithms. With 10 common benchmark unconstrained continuous functions used to
test the performance, the results show that state transition algorithms are
promising algorithms due to their good global search capability and convergence
property when compared with some popular algorithms.
Scalable Object Detection using Deep Neural Networks
Authors: Dumitru Erhan, Christian Szegedy, Alexander Toshev, Dragomir Anguelov
Date: 2013-12-08 00:00:00
Proceeding: CVPR 2014 6
Tasks: Object, object-detection, Object Detection, Object Recognition

Deep convolutional neural networks have recently achieved state-of-the-art
performance on a number of image recognition benchmarks, including the ImageNet
Large-Scale Visual Recognition Challenge (ILSVRC-2012). The winning model on
the localization sub-task was a network that predicts a single bounding box and
a confidence score for each object category in the image. Such a model captures
the whole-image context around the objects but cannot handle multiple instances
of the same object in the image without naively replicating the number of
outputs for each instance. In this work, we propose a saliency-inspired neural
network model for detection, which predicts a set of class-agnostic bounding
boxes along with a single score for each box, corresponding to its likelihood
of containing any object of interest. The model naturally handles a variable
number of instances for each class and allows for cross-class generalization at
the highest levels of the network. We are able to obtain competitive
recognition performance on VOC2007 and ILSVRC2012, while using only the top few
predicted locations in each image and a small number of neural network
evaluations.
CLIC: A Framework for Distributed, On-Demand, Human-Machine Cognitive Systems
Authors: N. Mavridis, S. Konstantopoulos, I. Vetsikas, I. Heldal, P. Karampiperis, G. Mathiason, S. Thill, K. Stathis, V. Karkaletsis
Date: 2013-12-08 00:00:00

Traditional Artificial Cognitive Systems (for example, intelligent robots)
share a number of limitations. First, they are usually made up only of machine
components; humans are only playing the role of user or supervisor. And yet,
there are tasks in which the current state of the art of AI has much worse
performance or is more expensive than humans: thus, it would be highly
beneficial to have a systematic way of creating systems with both human and
machine components, possibly with remote non-expert humans providing
short-duration real-time services. Second, their components are often dedicated
to only one system, and underutilized for a big part of their lifetime. Third,
there is no inherent support for robust operation, and if a new better
component becomes available, one cannot easily replace the old component.
Fourth, they are viewed as a resource to be developed and owned, not as a
utility. Thus, we are presenting CLIC: a framework for constructing cognitive
systems that overcome the above limitations. The architecture of CLIC provides
specific mechanisms for creating and operating cognitive systems that fulfill a
set of desiderata: First, that are distributed yet situated, interacting with
the physical world though sensing and actuation services, and that are also
combining human as well as machine services. Second, that are made up of
components that are time-shared and re-usable. Third, that provide increased
robustness through self-repair. Fourth, that are constructed and reconstructed
on the fly, with components that dynamically enter and exit the system during
operation, on the basis of availability, pricing, and need. Importantly, fifth,
the cognitive systems created and operated by CLIC do not need to be owned and
can be provided on demand, as a utility; thus transforming human-machine
situated intelligence to a service, and opening up many interesting
opportunities.
Nonparametric Estimation of Multi-View Latent Variable Models
Authors: Le Song, Animashree Anandkumar, Bo Dai, Bo Xie
Date: 2013-11-13 00:00:00

Spectral methods have greatly advanced the estimation of latent variable
models, generating a sequence of novel and efficient algorithms with strong
theoretical guarantees. However, current spectral algorithms are largely
restricted to mixtures of discrete or Gaussian distributions. In this paper, we
propose a kernel method for learning multi-view latent variable models,
allowing each mixture component to be nonparametric. The key idea of the method
is to embed the joint distribution of a multi-view latent variable into a
reproducing kernel Hilbert space, and then the latent parameters are recovered
using a robust tensor power method. We establish that the sample complexity for
the proposed method is quadratic in the number of latent components and is a
low order polynomial in the other relevant parameters. Thus, our non-parametric
tensor approach to learning latent variable models enjoys good sample and
computational efficiencies. Moreover, the non-parametric tensor power method
compares favorably to EM algorithm and other existing spectral algorithms in
our experiments.
Sequential Monte Carlo Inference of Mixed Membership Stochastic Blockmodels for Dynamic Social Networks
Authors: Tomoki Kobayashi, Koji Eguchi
Date: 2013-12-07 00:00:00
Tasks: Bayesian Inference

Many kinds of data can be represented as a network or graph. It is crucial to
infer the latent structure underlying such a network and to predict unobserved
links in the network. Mixed Membership Stochastic Blockmodel (MMSB) is a
promising model for network data. Latent variables and unknown parameters in
MMSB have been estimated through Bayesian inference with the entire network;
however, it is important to estimate them online for evolving networks. In this
paper, we first develop online inference methods for MMSB through sequential
Monte Carlo methods, also known as particle filters. We then extend them for
time-evolving networks, taking into account the temporal dependency of the
network structure. We demonstrate through experiments that the time-dependent
particle filter outperformed several baselines in terms of prediction
performance in an online condition.
End-to-end Phoneme Sequence Recognition using Convolutional Neural Networks
Authors: Dimitri Palaz, Ronan Collobert, Mathew Magimai. -Doss
Date: 2013-12-07 00:00:00
Tasks: Phoneme Recognition

Most phoneme recognition state-of-the-art systems rely on a classical neural
network classifiers, fed with highly tuned features, such as MFCC or PLP
features. Recent advances in ``deep learning'' approaches questioned such
systems, but while some attempts were made with simpler features such as
spectrograms, state-of-the-art systems still rely on MFCCs. This might be
viewed as a kind of failure from deep learning approaches, which are often
claimed to have the ability to train with raw signals, alleviating the need of
hand-crafted features. In this paper, we investigate a convolutional neural
network approach for raw speech signals. While convolutional architectures got
tremendous success in computer vision or text processing, they seem to have
been let down in the past recent years in the speech processing field. We show
that it is possible to learn an end-to-end phoneme sequence classifier system
directly from raw signal, with similar performance on the TIMIT and WSJ
datasets than existing systems based on MFCC, questioning the need of complex
hand-crafted features on large datasets.