pwc-archive/papers-with-abstracts huggingface dataset:
An Efficient Implementation for WalkSAT
Authors: Sixue Liu
Date: 2015-10-25 00:00:00

Stochastic local search (SLS) algorithms have exhibited great effectiveness
in finding models of random instances of the Boolean satisfiability problem
(SAT). As one of the most widely known and used SLS algorithm, WalkSAT plays a
key role in the evolutions of SLS for SAT, and also hold state-of-the-art
performance on random instances. This work proposes a novel implementation for
WalkSAT which decreases the redundant calculations leading to a dramatically
speeding up, thus dominates the latest version of WalkSAT including its
advanced variants.
Q-Networks for Binary Vector Actions
Authors: Naoto Yoshida
Date: 2015-12-04 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper reinforcement learning with binary vector actions was
investigated. We suggest an effective architecture of the neural networks for
approximating an action-value function with binary vector actions. The proposed
architecture approximates the action-value function by a linear function with
respect to the action vector, but is still non-linear with respect to the state
input. We show that this approximation method enables the efficient calculation
of greedy action selection and softmax action selection. Using this
architecture, we suggest an online algorithm based on Q-learning. The empirical
results in the grid world and the blocker task suggest that our approximation
architecture would be effective for the RL problems with large discrete action
sets.
Asynchronous Distributed Semi-Stochastic Gradient Optimization
Authors: Ruiliang Zhang, Shuai Zheng, James T. Kwok
Date: 2015-08-07 00:00:00
Tasks: Cloud Computing

With the recent proliferation of large-scale learning problems,there have
been a lot of interest on distributed machine learning algorithms, particularly
those that are based on stochastic gradient descent (SGD) and its variants.
However, existing algorithms either suffer from slow convergence due to the
inherent variance of stochastic gradients, or have a fast linear convergence
rate but at the expense of poorer solution quality. In this paper, we combine
their merits by proposing a fast distributed asynchronous SGD-based algorithm
with variance reduction. A constant learning rate can be used, and it is also
guaranteed to converge linearly to the optimal solution. Experiments on the
Google Cloud Computing Platform demonstrate that the proposed algorithm
outperforms state-of-the-art distributed asynchronous algorithms in terms of
both wall clock time and solution quality.
Toward a Taxonomy and Computational Models of Abnormalities in Images
Authors: Babak Saleh, Ahmed Elgammal, Jacob Feldman, Ali Farhadi
Date: 2015-12-04 00:00:00

The human visual system can spot an abnormal image, and reason about what
makes it strange. This task has not received enough attention in computer
vision. In this paper we study various types of atypicalities in images in a
more comprehensive way than has been done before. We propose a new dataset of
abnormal images showing a wide range of atypicalities. We design human subject
experiments to discover a coarse taxonomy of the reasons for abnormality. Our
experiments reveal three major categories of abnormality: object-centric,
scene-centric, and contextual. Based on this taxonomy, we propose a
comprehensive computational model that can predict all different types of
abnormality in images and outperform prior arts in abnormality recognition.
An Online Unsupervised Structural Plasticity Algorithm for Spiking Neural Networks
Authors: Subhrajit Roy, Arindam Basu
Date: 2015-12-04 00:00:00
Tasks: General Classification, Specificity

In this article, we propose a novel Winner-Take-All (WTA) architecture
employing neurons with nonlinear dendrites and an online unsupervised
structural plasticity rule for training it. Further, to aid hardware
implementations, our network employs only binary synapses. The proposed
learning rule is inspired by spike time dependent plasticity (STDP) but differs
for each dendrite based on its activation level. It trains the WTA network
through formation and elimination of connections between inputs and synapses.
To demonstrate the performance of the proposed network and learning rule, we
employ it to solve two, four and six class classification of random Poisson
spike time inputs. The results indicate that by proper tuning of the inhibitory
time constant of the WTA, a trade-off between specificity and sensitivity of
the network can be achieved. We use the inhibitory time constant to set the
number of subpatterns per pattern we want to detect. We show that while the
percentage of successful trials are 92%, 88% and 82% for two, four and six
class classification when no pattern subdivisions are made, it increases to
100% when each pattern is subdivided into 5 or 10 subpatterns. However, the
former scenario of no pattern subdivision is more jitter resilient than the
later ones.
Adjusting for Chance Clustering Comparison Measures
Authors: Simone Romano, Nguyen Xuan Vinh, James Bailey, Karin Verspoor
Date: 2015-12-03 00:00:00
Tasks: Clustering

Adjusted for chance measures are widely used to compare
partitions/clusterings of the same data set. In particular, the Adjusted Rand
Index (ARI) based on pair-counting, and the Adjusted Mutual Information (AMI)
based on Shannon information theory are very popular in the clustering
community. Nonetheless it is an open problem as to what are the best
application scenarios for each measure and guidelines in the literature for
their usage are sparse, with the result that users often resort to using both.
Generalized Information Theoretic (IT) measures based on the Tsallis entropy
have been shown to link pair-counting and Shannon IT measures. In this paper,
we aim to bridge the gap between adjustment of measures based on pair-counting
and measures based on information theory. We solve the key technical challenge
of analytically computing the expected value and variance of generalized IT
measures. This allows us to propose adjustments of generalized IT measures,
which reduce to well known adjusted clustering comparison measures as special
cases. Using the theory of generalized IT measures, we are able to propose the
following guidelines for using ARI and AMI as external validation indices: ARI
should be used when the reference clustering has large equal sized clusters;
AMI should be used when the reference clustering is unbalanced and there exist
small clusters.
Predicting the top and bottom ranks of billboard songs using Machine Learning
Authors: Vivek Datla, Abhinav Vishnu
Date: 2015-12-03 00:00:00
Tasks: BIG-bench Machine Learning

The music industry is a $130 billion industry. Predicting whether a song
catches the pulse of the audience impacts the industry. In this paper we
analyze language inside the lyrics of the songs using several computational
linguistic algorithms and predict whether a song would make to the top or
bottom of the billboard rankings based on the language features. We trained and
tested an SVM classifier with a radial kernel function on the linguistic
features. Results indicate that we can classify whether a song belongs to top
and bottom of the billboard charts with a precision of 0.76.
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Authors: Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, Zheng Zhang
Date: 2015-12-03 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Dimensionality Reduction, General Classification, GPU

MXNet is a multi-language machine learning (ML) library to ease the
development of ML algorithms, especially for deep neural networks. Embedded in
the host language, it blends declarative symbolic expression with imperative
tensor computation. It offers auto differentiation to derive gradients. MXNet
is computation and memory efficient and runs on various heterogeneous systems,
ranging from mobile devices to distributed GPU clusters.
  This paper describes both the API design and the system implementation of
MXNet, and explains how embedding of both symbolic expression and tensor
operation is handled in a unified fashion. Our preliminary experiments reveal
promising results on large scale deep neural network applications using
multiple GPU machines.
CrossCat: A Fully Bayesian Nonparametric Method for Analyzing Heterogeneous, High Dimensional Data
Authors: Vikash Mansinghka, Patrick Shafto, Eric Jonas, Cap Petschulat, Max Gasner, Joshua B. Tenenbaum
Date: 2015-12-03 00:00:00
Tasks: Bayesian Inference, Common Sense Reasoning, Vocal Bursts Intensity Prediction

There is a widespread need for statistical methods that can analyze
high-dimensional datasets with- out imposing restrictive or opaque modeling
assumptions. This paper describes a domain-general data analysis method called
CrossCat. CrossCat infers multiple non-overlapping views of the data, each
consisting of a subset of the variables, and uses a separate nonparametric
mixture to model each view. CrossCat is based on approximately Bayesian
inference in a hierarchical, nonparamet- ric model for data tables. This model
consists of a Dirichlet process mixture over the columns of a data table in
which each mixture component is itself an independent Dirichlet process mixture
over the rows; the inner mixture components are simple parametric models whose
form depends on the types of data in the table. CrossCat combines strengths of
mixture modeling and Bayesian net- work structure learning. Like mixture
modeling, CrossCat can model a broad class of distributions by positing latent
variables, and produces representations that can be efficiently conditioned and
sampled from for prediction. Like Bayesian networks, CrossCat represents the
dependencies and independencies between variables, and thus remains accurate
when there are multiple statistical signals. Inference is done via a scalable
Gibbs sampling scheme; this paper shows that it works well in practice. This
paper also includes empirical results on heterogeneous tabular data of up to 10
million cells, such as hospital cost and quality measures, voting records,
unemployment rates, gene expression measurements, and images of handwritten
digits. CrossCat infers structure that is consistent with accepted findings and
common-sense knowledge in multiple domains and yields predictive accuracy
competitive with generative, discriminative, and model-free alternatives.
Mind the duality gap: safer rules for the Lasso
Authors: Olivier Fercoq, Alexandre Gramfort, Joseph Salmon
Date: 2015-05-13 00:00:00

Screening rules allow to early discard irrelevant variables from the
optimization in Lasso problems, or its derivatives, making solvers faster. In
this paper, we propose new versions of the so-called $\textit{safe rules}$ for
the Lasso. Based on duality gap considerations, our new rules create safe test
regions whose diameters converge to zero, provided that one relies on a
converging solver. This property helps screening out more variables, for a
wider range of regularization parameter values. In addition to faster
convergence, we prove that we correctly identify the active sets (supports) of
the solutions in finite time. While our proposed strategy can cope with any
solver, its performance is demonstrated using a coordinate descent algorithm
particularly adapted to machine learning use cases. Significant computing time
reductions are obtained with respect to previous safe rules.
Confusing Deep Convolution Networks by Relabelling
Authors: Leigh Robinson, Benjamin Graham
Date: 2015-10-23 00:00:00

Deep convolutional neural networks have become the gold standard for image
recognition tasks, demonstrating many current state-of-the-art results and even
achieving near-human level performance on some tasks. Despite this fact it has
been shown that their strong generalisation qualities can be fooled to
misclassify previously correctly classified natural images and give erroneous
high confidence classifications to nonsense synthetic images. In this paper we
extend that work, by presenting a straightforward way to perturb an image in
such a way as to cause it to acquire any other label from within the dataset
while leaving this perturbed image visually indistinguishable from the
original.
The Human Kernel
Authors: Andrew Gordon Wilson, Christoph Dann, Christopher G. Lucas, Eric P. Xing
Date: 2015-10-26 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Gaussian Processes

Bayesian nonparametric models, such as Gaussian processes, provide a
compelling framework for automatic statistical modelling: these models have a
high degree of flexibility, and automatically calibrated complexity. However,
automating human expertise remains elusive; for example, Gaussian processes
with standard kernels struggle on function extrapolation problems that are
trivial for human learners. In this paper, we create function extrapolation
problems and acquire human responses, and then design a kernel learning
framework to reverse engineer the inductive biases of human learners across a
set of behavioral experiments. We use the learned kernels to gain psychological
insights and to extrapolate in human-like ways that go beyond traditional
stationary and polynomial kernels. Finally, we investigate Occam's razor in
human and Gaussian process based function learning.
Building Memory with Concept Learning Capabilities from Large-scale Knowledge Base
Authors: Jiaxin Shi, Jun Zhu
Date: 2015-12-03 00:00:00

We present a new perspective on neural knowledge base (KB) embeddings, from
which we build a framework that can model symbolic knowledge in the KB together
with its learning process. We show that this framework well regularizes
previous neural KB embedding model for superior performance in reasoning tasks,
while having the capabilities of dealing with unseen entities, that is, to
learn their embeddings from natural language descriptions, which is very like
human's behavior of learning semantic concepts.
Towards Principled Unsupervised Learning
Authors: Ilya Sutskever, Rafal Jozefowicz, Karol Gregor, Danilo Rezende, Tim Lillicrap, Oriol Vinyals
Date: 2015-11-19 00:00:00
Tasks: Domain Adaptation

General unsupervised learning is a long-standing conceptual problem in
machine learning. Supervised learning is successful because it can be solved by
the minimization of the training error cost function. Unsupervised learning is
not as successful, because the unsupervised objective may be unrelated to the
supervised task of interest. For an example, density modelling and
reconstruction have often been used for unsupervised learning, but they did not
produced the sought-after performance gains, because they have no knowledge of
the supervised tasks.
  In this paper, we present an unsupervised cost function which we name the
Output Distribution Matching (ODM) cost, which measures a divergence between
the distribution of predictions and distributions of labels. The ODM cost is
appealing because it is consistent with the supervised cost in the following
sense: a perfect supervised classifier is also perfect according to the ODM
cost. Therefore, by aggressively optimizing the ODM cost, we are almost
guaranteed to improve our supervised performance whenever the space of possible
predictions is exponentially large.
  We demonstrate that the ODM cost works well on number of small and
semi-artificial datasets using no (or almost no) labelled training cases.
Finally, we show that the ODM cost can be used for one-shot domain adaptation,
which allows the model to classify inputs that differ from the input
distribution in significant ways without the need for prior exposure to the new
domain.
A Generative Model of Words and Relationships from Multiple Sources
Authors: Stephanie L. Hyland, Theofanis Karaletsos, Gunnar Rätsch
Date: 2015-10-01 00:00:00
Tasks: Link Prediction

Neural language models are a powerful tool to embed words into semantic
vector spaces. However, learning such models generally relies on the
availability of abundant and diverse training examples. In highly specialised
domains this requirement may not be met due to difficulties in obtaining a
large corpus, or the limited range of expression in average use. Such domains
may encode prior knowledge about entities in a knowledge base or ontology. We
propose a generative model which integrates evidence from diverse data sources,
enabling the sharing of semantic information. We achieve this by generalising
the concept of co-occurrence from distributional semantics to include other
relationships between entities or words, which we model as affine
transformations on the embedding space. We demonstrate the effectiveness of
this approach by outperforming recent models on a link prediction task and
demonstrating its ability to profit from partially or fully unobserved data
training labels. We further demonstrate the usefulness of learning from
different data sources with overlapping vocabularies.
Occlusion-Aware Human Pose Estimation with Mixtures of Sub-Trees
Authors: Ibrahim Radwan, Abhinav Dhall, Roland Goecke
Date: 2015-12-03 00:00:00
Tasks: Pose Estimation

In this paper, we study the problem of learning a model for human pose
estimation as mixtures of compositional sub-trees in two layers of prediction.
This involves estimating the pose of a sub-tree followed by identifying the
relationships between sub-trees that are used to handle occlusions between
different parts. The mixtures of the sub-trees are learnt utilising both
geometric and appearance distances. The Chow-Liu (CL) algorithm is recursively
applied to determine the inter-relations between the nodes and to build the
structure of the sub-trees. These structures are used to learn the latent
parameters of the sub-trees and the inference is done using a standard belief
propagation technique. The proposed method handles occlusions during the
inference process by identifying overlapping regions between different
sub-trees and introducing a penalty term for overlapping parts. Experiments are
performed on three different datasets: the Leeds Sports, Image Parse and UIUC
People datasets. The results show the robustness of the proposed method to
occlusions over the state-of-the-art approaches.
Approaches for Sentiment Analysis on Twitter: A State-of-Art study
Authors: Harsh Thakkar, Dhiren Patel
Date: 2015-12-03 00:00:00
Tasks: Opinion Mining, Retrieval, Sentiment Analysis

Microbloging is an extremely prevalent broadcast medium amidst the Internet
fraternity these days. People share their opinions and sentiments about variety
of subjects like products, news, institutions, etc., every day on microbloging
websites. Sentiment analysis plays a key role in prediction systems, opinion
mining systems, etc. Twitter, one of the microbloging platforms allows a limit
of 140 characters to its users. This restriction stimulates users to be very
concise about their opinion and twitter an ocean of sentiments to analyze.
Twitter also provides developer friendly streaming API for data retrieval
purpose allowing the analyst to search real time tweets from various users. In
this paper, we discuss the state-of-art of the works which are focused on
Twitter, the online social network platform, for sentiment analysis. We survey
various lexical, machine learning and hybrid approaches for sentiment analysis
on Twitter.
Querying with Łukasiewicz logic
Authors: Stefano Aguzzoli, Pietro Codara, Tommaso Flaminio, Brunella Gerla, Diego Valota
Date: 2015-12-03 00:00:00

In this paper we present, by way of case studies, a proof of concept, based
on a prototype working on a automotive data set, aimed at showing the potential
usefulness of using formulas of {\L}ukasiewicz propositional logic to query
databases in a fuzzy way. Our approach distinguishes itself for its stress on
the purely linguistic, contraposed with numeric, formulations of queries. Our
queries are expressed in the pure language of logic, and when we use (integer)
numbers, these stand for shortenings of formulas on the syntactic level, and
serve as linguistic hedges on the semantic one. Our case-study queries aim
first at showing that each numeric-threshold fuzzy query is simulated by a
{\L}ukasiewicz formula. Then they focus on the expressing power of
{\L}ukasiewicz logic which easily allows for updating queries by clauses and
for modifying them through a potentially infinite variety of linguistic hedges
implemented with a uniform syntactic mechanism. Finally we shall hint how,
already at propositional level, {\L}ukasiewicz natural semantics enjoys a
degree of reflection, allowing to write syntactically simple queries that
semantically work as meta-queries weighing the contribution of simpler ones.
Simulations for Validation of Vision Systems
Authors: V. S. R. Veeravasarapu, Rudra Narayan Hota, Constantin Rothkopf, Ramesh Visvanathan
Date: 2015-12-03 00:00:00
Tasks: Change Detection, Open-Ended Question Answering

As the computer vision matures into a systems science and engineering
discipline, there is a trend in leveraging latest advances in computer graphics
simulations for performance evaluation, learning, and inference. However, there
is an open question on the utility of graphics simulations for vision with
apparently contradicting views in the literature. In this paper, we place the
results from the recent literature in the context of performance
characterization methodology outlined in the 90's and note that insights
derived from simulations can be qualitative or quantitative depending on the
degree of fidelity of models used in simulation and the nature of the question
posed by the experimenter. We describe a simulation platform that incorporates
latest graphics advances and use it for systematic performance characterization
and trade-off analysis for vision system design. We verify the utility of the
platform in a case study of validating a generative model inspired vision
hypothesis, Rank-Order consistency model, in the contexts of global and local
illumination changes, and bad weather, and high-frequency noise. Our approach
establishes the link between alternative viewpoints, involving models with
physics based semantics and signal and perturbation semantics and confirms
insights in literature on robust change detection.
Good parts first - a new algorithm for approximate search in lexica and string databases
Authors: Stefan Gerdjikov, Stoyan Mihov, Petar Mitankin, Klaus U. Schulz
Date: 2013-01-04 00:00:00

We present a new efficient method for approximate search in electronic
lexica. Given an input string (the pattern) and a similarity threshold, the
algorithm retrieves all entries of the lexicon that are sufficiently similar to
the pattern. Search is organized in subsearches that always start with an exact
partial match where a substring of the input pattern is aligned with a
substring of a lexicon word. Afterwards this partial match is extended stepwise
to larger substrings. For aligning further parts of the pattern with
corresponding parts of lexicon entries, more errors are tolerated at each
subsequent step. For supporting this alignment order, which may start at any
part of the pattern, the lexicon is represented as a structure that enables
immediate access to any substring of a lexicon word and permits the extension
of such substrings in both directions. Experimental evaluations of the
approximate search procedure are given that show significant efficiency
improvements compared to existing techniques. Since the technique can be used
for large error bounds it offers interesting possibilities for approximate
search in special collections of "long" strings, such as phrases, sentences, or
book ti