pwc-archive/papers-with-abstracts huggingface dataset:
Efficient Evolutionary Algorithm for Single-Objective Bilevel Optimization
Authors: Ankur Sinha, Pekka Malo, Kalyanmoy Deb
Date: 2013-03-15 00:00:00
Tasks: Bilevel Optimization

Bilevel optimization problems are a class of challenging optimization
problems, which contain two levels of optimization tasks. In these problems,
the optimal solutions to the lower level problem become possible feasible
candidates to the upper level problem. Such a requirement makes the
optimization problem difficult to solve, and has kept the researchers busy
towards devising methodologies, which can efficiently handle the problem.
Despite the efforts, there hardly exists any effective methodology, which is
capable of handling a complex bilevel problem. In this paper, we introduce
bilevel evolutionary algorithm based on quadratic approximations (BLEAQ) of
optimal lower level variables with respect to the upper level variables. The
approach is capable of handling bilevel problems with different kinds of
complexities in relatively smaller number of function evaluations. Ideas from
classical optimization have been hybridized with evolutionary methods to
generate an efficient optimization algorithm for generic bilevel problems. The
efficacy of the algorithm has been shown on two sets of test problems. The
first set is a recently proposed SMD test set, which contains problems with
controllable complexities, and the second set contains standard test problems
collected from the literature. The proposed method has been evaluated against
two benchmarks, and the performance gain is observed to be significant.
Towards Minimax Online Learning with Unknown Time Horizon
Authors: Haipeng Luo, Robert E. Schapire
Date: 2013-07-31 00:00:00

We consider online learning when the time horizon is unknown. We apply a
minimax analysis, beginning with the fixed horizon case, and then moving on to
two unknown-horizon settings, one that assumes the horizon is chosen randomly
according to some known distribution, and the other which allows the adversary
full control over the horizon. For the random horizon setting with restricted
losses, we derive a fully optimal minimax algorithm. And for the adversarial
horizon setting, we prove a nontrivial lower bound which shows that the
adversary obtains strictly more power than when the horizon is fixed and known.
Based on the minimax solution of the random horizon setting, we then propose a
new adaptive algorithm which "pretends" that the horizon is drawn from a
distribution from a special family, but no matter how the actual horizon is
chosen, the worst-case regret is of the optimal rate. Furthermore, our
algorithm can be combined and applied in many ways, for instance, to online
convex optimization, follow the perturbed leader, exponential weights algorithm
and first order bounds. Experiments show that our algorithm outperforms many
other existing algorithms in an online linear optimization setting.
Cross-lingual Pseudo-Projected Expectation Regularization for Weakly Supervised Learning
Authors: Mengqiu Wang, Christopher D. Manning
Date: 2013-10-06 00:00:00
Tasks: NER, Weakly-supervised Learning

We consider a multilingual weakly supervised learning scenario where
knowledge from annotated corpora in a resource-rich language is transferred via
bitext to guide the learning in other languages. Past approaches project labels
across bitext and use them as features or gold labels for training. We propose
a new method that projects model expectations rather than labels, which
facilities transfer of model uncertainty across language boundaries. We encode
expectations as constraints and train a discriminative CRF model using
Generalized Expectation Criteria (Mann and McCallum, 2010). Evaluated on
standard Chinese-English and German-English NER datasets, our method
demonstrates F1 scores of 64% and 60% when no labeled data is used. Attaining
the same accuracy with supervised CRFs requires 12k and 1.5k labeled sentences.
Furthermore, when combined with labeled examples, our method yields significant
improvements over state-of-the-art supervised methods, achieving best reported
numbers to date on Chinese OntoNotes and German CoNLL-03 datasets.
Evolution of the Modern Phase of Written Bangla: A Statistical Study
Authors: Paheli Bhattacharya, Arnab Bhattacharya
Date: 2013-10-06 00:00:00
Tasks: Sentence

Active languages such as Bangla (or Bengali) evolve over time due to a
variety of social, cultural, economic, and political issues. In this paper, we
analyze the change in the written form of the modern phase of Bangla
quantitatively in terms of character-level, syllable-level, morpheme-level and
word-level features. We collect three different types of corpora---classical,
newspapers and blogs---and test whether the differences in their features are
statistically significant. Results suggest that there are significant changes
in the length of a word when measured in terms of characters, but there is not
much difference in usage of different characters, syllables and morphemes in a
word or of different words in a sentence. To the best of our knowledge, this is
the first work on Bangla of this kind.
Copula Mixed-Membership Stochastic Blockmodel for Intra-Subgroup Correlations
Authors: Xuhui Fan, Longbing Cao, Richard Yi Da Xu
Date: 2013-06-12 00:00:00
Tasks: Link Prediction

The \emph{Mixed-Membership Stochastic Blockmodel (MMSB)} is a popular
framework for modeling social network relationships. It can fully exploit each
individual node's participation (or membership) in a social structure. Despite
its powerful representations, this model makes an assumption that the
distributions of relational membership indicators between two nodes are
independent. Under many social network settings, however, it is possible that
certain known subgroups of people may have high or low correlations in terms of
their membership categories towards each other, and such prior information
should be incorporated into the model. To this end, we introduce a \emph{Copula
Mixed-Membership Stochastic Blockmodel (cMMSB)} where an individual Copula
function is employed to jointly model the membership pairs of those nodes
within the subgroup of interest. The model enables the use of various Copula
functions to suit the scenario, while maintaining the membership's marginal
distribution, as needed, for modeling membership indicators with other nodes
outside of the subgroup of interest. We describe the proposed model and its
inference algorithm in detail for both the finite and infinite cases. In the
experiment section, we compare our algorithms with other popular models in
terms of link prediction, using both synthetic and real world data.
Learning Hidden Structures with Relational Models by Adequately Involving Rich Information in A Network
Authors: Xuhui Fan, Richard Yi Da Xu, Longbing Cao, Yin Song
Date: 2013-10-06 00:00:00

Effectively modelling hidden structures in a network is very practical but
theoretically challenging. Existing relational models only involve very limited
information, namely the binary directional link data, embedded in a network to
learn hidden networking structures. There is other rich and meaningful
information (e.g., various attributes of entities and more granular information
than binary elements such as "like" or "dislike") missed, which play a critical
role in forming and understanding relations in a network. In this work, we
propose an informative relational model (InfRM) framework to adequately involve
rich information and its granularity in a network, including metadata
information about each entity and various forms of link data. Firstly, an
effective metadata information incorporation method is employed on the prior
information from relational models MMSB and LFRM. This is to encourage the
entities with similar metadata information to have similar hidden structures.
Secondly, we propose various solutions to cater for alternative forms of link
data. Substantial efforts have been made towards modelling appropriateness and
efficiency, for example, using conjugate priors. We evaluate our framework and
its inference algorithms in different datasets, which shows the generality and
effectiveness of our models in capturing implicit structures in networks.
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Authors: Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell
Date: 2013-10-06 00:00:00
Tasks: Clustering, Domain Adaptation, Object Recognition, Scene Recognition, Transfer Learning

We evaluate whether features extracted from the activation of a deep
convolutional network trained in a fully supervised fashion on a large, fixed
set of object recognition tasks can be re-purposed to novel generic tasks. Our
generic tasks may differ significantly from the originally trained tasks and
there may be insufficient labeled or unlabeled data to conventionally train or
adapt a deep architecture to the new tasks. We investigate and visualize the
semantic clustering of deep convolutional features with respect to a variety of
such tasks, including scene recognition, domain adaptation, and fine-grained
recognition challenges. We compare the efficacy of relying on various network
levels to define a fixed feature, and report novel results that significantly
outperform the state-of-the-art on several important vision challenges. We are
releasing DeCAF, an open-source implementation of these deep convolutional
activation features, along with all associated network parameters to enable
vision researchers to be able to conduct experimentation with deep
representations across a range of visual concept learning paradigms.
Contraction Principle based Robust Iterative Algorithms for Machine Learning
Authors: Rangeet Mitra, Amit Kumar Mishra
Date: 2013-10-05 00:00:00
Tasks: BIG-bench Machine Learning

Iterative algorithms are ubiquitous in the field of data mining. Widely known
examples of such algorithms are the least mean square algorithm,
backpropagation algorithm of neural networks. Our contribution in this paper is
an improvement upon this iterative algorithms in terms of their respective
performance metrics and robustness. This improvement is achieved by a new
scaling factor which is multiplied to the error term. Our analysis shows that
in essence, we are minimizing the corresponding LASSO cost function, which is
the reason of its increased robustness. We also give closed form expressions
for the number of iterations for convergence and the MSE floor of the original
cost function for a minimum targeted value of the L1 norm. As a concluding
theme based on the stochastic subgradient algorithm, we give a comparison
between the well known Dantzig selector and our algorithm based on contraction
principle. By these simulations we attempt to show the optimality of our
approach for any widely used parent iterative optimization problem.
High dimensional Sparse Gaussian Graphical Mixture Model
Authors: Anani Lotsi, Ernst Wit
Date: 2013-08-15 00:00:00
Tasks: model, parameter estimation, Variable Selection, Vocal Bursts Intensity Prediction

This paper considers the problem of networks reconstruction from
heterogeneous data using a Gaussian Graphical Mixture Model (GGMM). It is well
known that parameter estimation in this context is challenging due to large
numbers of variables coupled with the degeneracy of the likelihood. We propose
as a solution a penalized maximum likelihood technique by imposing an $l_{1}$
penalty on the precision matrix. Our approach shrinks the parameters thereby
resulting in better identifiability and variable selection. We use the
Expectation Maximization (EM) algorithm which involves the graphical LASSO to
estimate the mixing coefficients and the precision matrices. We show that under
certain regularity conditions the Penalized Maximum Likelihood (PML) estimates
are consistent. We demonstrate the performance of the PML estimator through
simulations and we show the utility of our method for high dimensional data
analysis in a genomic application.
Local Feature or Mel Frequency Cepstral Coefficients - Which One is Better for MLN-Based Bangla Speech Recognition?
Authors: Foyzul Hassan, Mohammed Rokibul Alam Kotwal, Md. Mostafizur Rahman, Mohammad Nasiruddin, Md. Abdul Latif, Mohammad Nurul Huda
Date: 2013-10-05 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

This paper discusses the dominancy of local features (LFs), as input to the
multilayer neural network (MLN), extracted from a Bangla input speech over mel
frequency cepstral coefficients (MFCCs). Here, LF-based method comprises three
stages: (i) LF extraction from input speech, (ii) phoneme probabilities
extraction using MLN from LF and (iii) the hidden Markov model (HMM) based
classifier to obtain more accurate phoneme strings. In the experiments on
Bangla speech corpus prepared by us, it is observed that the LFbased automatic
speech recognition (ASR) system provides higher phoneme correct rate than the
MFCC-based system. Moreover, the proposed system requires fewer mixture
components in the HMMs.
A State of the Art of Word Sense Induction: A Way Towards Word Sense Disambiguation for Under-Resourced Languages
Authors: Mohammad Nasiruddin
Date: 2013-10-05 00:00:00
Tasks: Natural Language Understanding, Sentence, Word Sense Disambiguation, Word Sense Induction

Word Sense Disambiguation (WSD), the process of automatically identifying the
meaning of a polysemous word in a sentence, is a fundamental task in Natural
Language Processing (NLP). Progress in this approach to WSD opens up many
promising developments in the field of NLP and its applications. Indeed,
improvement over current performance levels could allow us to take a first step
towards natural language understanding. Due to the lack of lexical resources it
is sometimes difficult to perform WSD for under-resourced languages. This paper
is an investigation on how to initiate research in WSD for under-resourced
languages by applying Word Sense Induction (WSI) and suggests some interesting
topics to focus on.
Narrowing the Gap: Random Forests In Theory and In Practice
Authors: Misha Denil, David Matheson, Nando de Freitas
Date: 2013-10-04 00:00:00
Tasks: regression

Despite widespread interest and practical use, the theoretical properties of
random forests are still not well understood. In this paper we contribute to
this understanding in two ways. We present a new theoretically tractable
variant of random regression forests and prove that our algorithm is
consistent. We also provide an empirical evaluation, comparing our algorithm
and other theoretically tractable random forest models to the random forest
algorithm used in practice. Our experiments provide insight into the relative
importance of different simplifications that theoreticians have made to obtain
tractable models for analysis.
Multiple Kernel Sparse Representations for Supervised and Unsupervised Learning
Authors: Jayaraman J. Thiagarajan, Karthikeyan Natesan Ramamurthy, Andreas Spanias
Date: 2013-03-03 00:00:00
Tasks: Clustering, Dictionary Learning, Graph Embedding, Image Clustering, Object Recognition

In complex visual recognition tasks it is typical to adopt multiple
descriptors, that describe different aspects of the images, for obtaining an
improved recognition performance. Descriptors that have diverse forms can be
fused into a unified feature space in a principled manner using kernel methods.
Sparse models that generalize well to the test data can be learned in the
unified kernel space, and appropriate constraints can be incorporated for
application in supervised and unsupervised learning. In this paper, we propose
to perform sparse coding and dictionary learning in the multiple kernel space,
where the weights of the ensemble kernel are tuned based on graph-embedding
principles such that class discrimination is maximized. In our proposed
algorithm, dictionaries are inferred using multiple levels of 1-D subspace
clustering in the kernel space, and the sparse codes are obtained using a
simple levelwise pursuit scheme. Empirical results for object recognition and
image clustering show that our algorithm outperforms existing sparse coding
based approaches, and compares favorably to other state-of-the-art methods.
Sequential Monte Carlo Bandits
Authors: Michael Cherkassky, Luke Bornn
Date: 2013-10-04 00:00:00
Tasks: Multi-Armed Bandits

In this paper we propose a flexible and efficient framework for handling
multi-armed bandits, combining sequential Monte Carlo algorithms with
hierarchical Bayesian modeling techniques. The framework naturally encompasses
restless bandits, contextual bandits, and other bandit variants under a single
inferential model. Despite the model's generality, we propose efficient Monte
Carlo algorithms to make inference scalable, based on recent developments in
sequential Monte Carlo methods. Through two simulation studies, the framework
is shown to outperform other empirical methods, while also naturally scaling to
more complex problems for which existing approaches can not cope. Additionally,
we successfully apply our framework to online video-based advertising
recommendation, and show its increased efficacy as compared to current state of
the art bandit algorithms.
The Relevance of Proofs of the Rationality of Probability Theory to Automated Reasoning and Cognitive Models
Authors: Ernest Davis
Date: 2013-10-04 00:00:00

A number of well-known theorems, such as Cox's theorem and de Finetti's
theorem. prove that any model of reasoning with uncertain information that
satisfies specified conditions of "rationality" must satisfy the axioms of
probability theory. I argue here that these theorems do not in themselves
demonstrate that probabilistic models are in fact suitable for any specific
task in automated reasoning or plausible for cognitive models. First, the
theorems only establish that there exists some probabilistic model; they do not
establish that there exists a useful probabilistic model, i.e. one with a
tractably small number of numerical parameters and a large number of
independence assumptions. Second, there are in general many different
probabilistic models for a given situation, many of which may be far more
irrational, in the usual sense of the term, than a model that violates the
axioms of probability theory. I illustrate this second point with an extended
examples of two tasks of induction, of a similar structure, where the
reasonable probabilistic models are very different.
A Novel Progressive Image Scanning and Reconstruction Scheme based on Compressed Sensing and Linear Prediction
Authors: Giulio Coluccia, Enrico Magli
Date: 2013-10-04 00:00:00
Tasks: compressed sensing

Compressed sensing (CS) is an innovative technique allowing to represent
signals through a small number of their linear projections. In this paper we
address the application of CS to the scenario of progressive acquisition of 2D
visual signals in a line-by-line fashion. This is an important setting which
encompasses diverse systems such as flatbed scanners and remote sensing
imagers. The use of CS in such setting raises the problem of reconstructing a
very high number of samples, as are contained in an image, from their linear
projections. Conventional reconstruction algorithms, whose complexity is cubic
in the number of samples, are computationally intractable. In this paper we
develop an iterative reconstruction algorithm that reconstructs an image by
iteratively estimating a row, and correlating adjacent rows by means of linear
prediction. We develop suitable predictors and test the proposed algorithm in
the context of flatbed scanners and remote sensing imaging systems. We show
that this approach can significantly improve the results of separate
reconstruction of each row, providing very good reconstruction quality with
reasonable complexity.
Reading Stockholm Riots 2013 in social media by text-mining
Authors: Andrzej Jarynowski, Amir Rostami
Date: 2013-10-04 00:00:00
Tasks: Sentiment Analysis

The riots in Stockholm in May 2013 were an event that reverberated in the
world media for its dimension of violence that had spread through the Swedish
capital. In this study we have investigated the role of social media in
creating media phenomena via text mining and natural language processing. We
have focused on two channels of communication for our analysis: Twitter and
Poloniainfo.se (Forum of Polish community in Sweden). Our preliminary results
show some hot topics driving discussion related mostly to Swedish Police and
Swedish Politics by counting word usage. Typical features for media
intervention are presented. We have built networks of most popular phrases,
clustered by categories (geography, media institution, etc.). Sentiment
analysis shows negative connotation with Police. The aim of this preliminary
exploratory quantitative study was to generate questions and hypotheses, which
we could carefully follow by deeper more qualitative methods.
The Novel Approach of Adaptive Twin Probability for Genetic Algorithm
Authors: Anagha P. Khedkar, Shaila Subbaraman
Date: 2013-10-04 00:00:00

The performance of GA is measured and analyzed in terms of its performance
parameters against variations in its genetic operators and associated
parameters. Since last four decades huge numbers of researchers have been
working on the performance of GA and its enhancement. This earlier research
work on analyzing the performance of GA enforces the need to further
investigate the exploration and exploitation characteristics and observe its
impact on the behavior and overall performance of GA. This paper introduces the
novel approach of adaptive twin probability associated with the advanced twin
operator that enhances the performance of GA. The design of the advanced twin
operator is extrapolated from the twin offspring birth due to single ovulation
in natural genetic systems as mentioned in the earlier works. The twin
probability of this operator is adaptively varied based on the fitness of best
individual thereby relieving the GA user from statically defining its value.
This novel approach of adaptive twin probability is experimented and tested on
the standard benchmark optimization test functions. The experimental results
show the increased accuracy in terms of the best individual and reduced
convergence time.
Spatially Scalable Compressed Image Sensing with Hybrid Transform and Inter-layer Prediction Model
Authors: Diego Valsesia, Enrico Magli
Date: 2013-10-04 00:00:00
Tasks: compressed sensing

Compressive imaging is an emerging application of compressed sensing, devoted
to acquisition, encoding and reconstruction of images using random projections
as measurements. In this paper we propose a novel method to provide a scalable
encoding of an image acquired by means of compressed sensing techniques. Two
bit-streams are generated to provide two distinct quality levels: a
low-resolution base layer and full-resolution enhancement layer. In the
proposed method we exploit a fast preview of the image at the encoder in order
to perform inter-layer prediction and encode the prediction residuals only. The
proposed method successfully provides resolution and quality scalability with
modest complexity and it provides gains in the quality of the reconstructed
images with respect to separate encoding of the quality layers. Remarkably, we
also show that the scheme can also provide significant gains with respect to a
direct, non-scalable system, thus accomplishing two features at once:
scalability and improved reconstruction performance.