pwc-archive/papers-with-abstracts huggingface dataset:
Max-Entropy Feed-Forward Clustering Neural Network
Authors: Han Xiao, Xiaoyan Zhu
Date: 2015-06-11 00:00:00
Tasks: Clustering

The outputs of non-linear feed-forward neural network are positive, which
could be treated as probability when they are normalized to one. If we take
Entropy-Based Principle into consideration, the outputs for each sample could
be represented as the distribution of this sample for different clusters.
Entropy-Based Principle is the principle with which we could estimate the
unknown distribution under some limited conditions. As this paper defines two
processes in Feed-Forward Neural Network, our limited condition is the
abstracted features of samples which are worked out in the abstraction process.
And the final outputs are the probability distribution for different clusters
in the clustering process. As Entropy-Based Principle is considered into the
feed-forward neural network, a clustering method is born. We have conducted
some experiments on six open UCI datasets, comparing with a few baselines and
applied purity as the measurement . The results illustrate that our method
outperforms all the other baselines that are most popular clustering methods.
Distributed Recurrent Neural Forward Models with Synaptic Adaptation for Complex Behaviors of Walking Robots
Authors: Sakyasingha Dasgupta, Dennis Goldschmidt, Florentin Wörgötter, Poramate Manoonpong
Date: 2015-06-11 00:00:00

Walking animals, like stick insects, cockroaches or ants, demonstrate a
fascinating range of locomotive abilities and complex behaviors. The locomotive
behaviors can consist of a variety of walking patterns along with adaptation
that allow the animals to deal with changes in environmental conditions, like
uneven terrains, gaps, obstacles etc. Biological study has revealed that such
complex behaviors are a result of a combination of biome- chanics and neural
mechanism thus representing the true nature of embodied interactions. While the
biomechanics helps maintain flexibility and sustain a variety of movements, the
neural mechanisms generate movements while making appropriate predictions
crucial for achieving adaptation. Such predictions or planning ahead can be
achieved by way of in- ternal models that are grounded in the overall behavior
of the animal. Inspired by these findings, we present here, an artificial
bio-inspired walking system which effectively com- bines biomechanics (in terms
of the body and leg structures) with the underlying neural mechanisms. The
neural mechanisms consist of 1) central pattern generator based control for
generating basic rhythmic patterns and coordinated movements, 2) distributed
(at each leg) recurrent neural network based adaptive forward models with
efference copies as internal models for sensory predictions and instantaneous
state estimations, and 3) searching and elevation control for adapting the
movement of an individual leg to deal with different environmental conditions.
Using simulations we show that this bio-inspired approach with adaptive
internal models allows the walking robot to perform complex loco- motive
behaviors as observed in insects, including walking on undulated terrains,
crossing large gaps as well as climbing over high obstacles...
Modeling Order in Neural Word Embeddings at Scale
Authors: Andrew Trask, David Gilmore, Matthew Russell
Date: 2015-06-08 00:00:00
Tasks: Language Modeling, Language Modelling, Word Embeddings

Natural Language Processing (NLP) systems commonly leverage bag-of-words
co-occurrence techniques to capture semantic and syntactic word relationships.
The resulting word-level distributed representations often ignore morphological
information, though character-level embeddings have proven valuable to NLP
tasks. We propose a new neural language model incorporating both word order and
character order in its embedding. The model produces several vector spaces with
meaningful substructure, as evidenced by its performance of 85.8% on a recent
word-analogy task, exceeding best published syntactic word-analogy scores by a
58% error margin. Furthermore, the model includes several parallel training
methods, most notably allowing a skip-gram network with 160 billion parameters
to be trained overnight on 3 multi-core CPUs, 14x larger than the previous
largest neural network.
BoWFire: Detection of Fire in Still Images by Integrating Pixel Color and Texture Analysis
Authors: Daniel Y. T. Chino, Letricia P. S. Avalhais, Jose F. Rodrigues Jr., Agma J. M. Traina
Date: 2015-06-10 00:00:00
Tasks: Decision Making, Fire Detection, General Classification, Management, Texture Classification

Emergency events involving fire are potentially harmful, demanding a fast and
precise decision making. The use of crowdsourcing image and videos on crisis
management systems can aid in these situations by providing more information
than verbal/textual descriptions. Due to the usual high volume of data,
automatic solutions need to discard non-relevant content without losing
relevant information. There are several methods for fire detection on video
using color-based models. However, they are not adequate for still image
processing, because they can suffer on high false-positive results. These
methods also suffer from parameters with little physical meaning, which makes
fine tuning a difficult task. In this context, we propose a novel fire
detection method for still images that uses classification based on color
features combined with texture classification on superpixel regions. Our method
uses a reduced number of parameters if compared to previous works, easing the
process of fine tuning the method. Results show the effectiveness of our method
of reducing false-positives while its precision remains compatible with the
state-of-the-art methods.
Bayesian Poisson Tensor Factorization for Inferring Multilateral Relations from Sparse Dyadic Event Counts
Authors: Aaron Schein, John Paisley, David M. Blei, Hanna Wallach
Date: 2015-06-10 00:00:00
Tasks: Form

We present a Bayesian tensor factorization model for inferring latent group
structures from dynamic pairwise interaction patterns. For decades, political
scientists have collected and analyzed records of the form "country $i$ took
action $a$ toward country $j$ at time $t$"---known as dyadic events---in order
to form and test theories of international relations. We represent these event
data as a tensor of counts and develop Bayesian Poisson tensor factorization to
infer a low-dimensional, interpretable representation of their salient
patterns. We demonstrate that our model's predictive performance is better than
that of standard non-negative tensor factorization methods. We also provide a
comparison of our variational updates to their maximum likelihood counterparts.
In doing so, we identify a better way to form point estimates of the latent
factors than that typically used in Bayesian Poisson matrix factorization.
Finally, we showcase our model as an exploratory analysis tool for political
scientists. We show that the inferred latent factor matrices capture
interpretable multilateral relations that both conform to and inform our
knowledge of international affairs.
Truthful Linear Regression
Authors: Rachel Cummings, Stratis Ioannidis, Katrina Ligett
Date: 2015-06-10 00:00:00
Tasks: regression

We consider the problem of fitting a linear model to data held by individuals
who are concerned about their privacy. Incentivizing most players to truthfully
report their data to the analyst constrains our design to mechanisms that
provide a privacy guarantee to the participants; we use differential privacy to
model individuals' privacy losses. This immediately poses a problem, as
differentially private computation of a linear model necessarily produces a
biased estimation, and existing approaches to design mechanisms to elicit data
from privacy-sensitive individuals do not generalize well to biased estimators.
We overcome this challenge through an appropriate design of the computation and
payment scheme.
Spectral Detection in the Censored Block Model
Authors: Alaa Saade, Florent Krzakala, Marc Lelarge, Lenka Zdeborová
Date: 2015-01-31 00:00:00
Tasks: Clustering, Community Detection, model

We consider the problem of partially recovering hidden binary variables from
the observation of (few) censored edge weights, a problem with applications in
community detection, correlation clustering and synchronization. We describe
two spectral algorithms for this task based on the non-backtracking and the
Bethe Hessian operators. These algorithms are shown to be asymptotically
optimal for the partial recovery problem, in that they detect the hidden
assignment as soon as it is information theoretically possible to do so.
Fast Online Clustering with Randomized Skeleton Sets
Authors: Krzysztof Choromanski, Sanjiv Kumar, Xiaofeng Liu
Date: 2015-06-10 00:00:00
Tasks: Clustering, Nonparametric Clustering, Online Clustering

We present a new fast online clustering algorithm that reliably recovers
arbitrary-shaped data clusters in high throughout data streams. Unlike the
existing state-of-the-art online clustering methods based on k-means or
k-medoid, it does not make any restrictive generative assumptions. In addition,
in contrast to existing nonparametric clustering techniques such as DBScan or
DenStream, it gives provable theoretical guarantees. To achieve fast
clustering, we propose to represent each cluster by a skeleton set which is
updated continuously as new data is seen. A skeleton set consists of weighted
samples from the data where weights encode local densities. The size of each
skeleton set is adapted according to the cluster geometry. The proposed
technique automatically detects the number of clusters and is robust to
outliers. The algorithm works for the infinite data stream where more than one
pass over the data is not feasible. We provide theoretical guarantees on the
quality of the clustering and also demonstrate its advantage over the existing
state-of-the-art on several datasets.
piCholesky: Polynomial Interpolation of Multiple Cholesky Factors for Efficient Approximate Cross-Validation
Authors: Da Kuang, Alex Gittens, Raffay Hamid
Date: 2014-04-02 00:00:00

The dominant cost in solving least-square problems using Newton's method is
often that of factorizing the Hessian matrix over multiple values of the
regularization parameter ($\lambda$). We propose an efficient way to
interpolate the Cholesky factors of the Hessian matrix computed over a small
set of $\lambda$ values. This approximation enables us to optimally minimize
the hold-out error while incurring only a fraction of the cost compared to
exact cross-validation. We provide a formal error bound for our approximation
scheme and present solutions to a set of key implementation challenges that
allow our approach to maximally exploit the compute power of modern
architectures. We present a thorough empirical analysis over multiple datasets
to show the effectiveness of our approach.
Optimal Rates of Convergence for Noisy Sparse Phase Retrieval via Thresholded Wirtinger Flow
Authors: T. Tony Cai, Xiao-Dong Li, Zongming Ma
Date: 2015-06-10 00:00:00
Tasks: Retrieval

This paper considers the noisy sparse phase retrieval problem: recovering a
sparse signal $x \in \mathbb{R}^p$ from noisy quadratic measurements $y_j =
(a_j' x )^2 + \epsilon_j$, $j=1, \ldots, m$, with independent sub-exponential
noise $\epsilon_j$. The goals are to understand the effect of the sparsity of
$x$ on the estimation precision and to construct a computationally feasible
estimator to achieve the optimal rates. Inspired by the Wirtinger Flow [12]
proposed for noiseless and non-sparse phase retrieval, a novel thresholded
gradient descent algorithm is proposed and it is shown to adaptively achieve
the minimax optimal rates of convergence over a wide range of sparsity levels
when the $a_j$'s are independent standard Gaussian random vectors, provided
that the sample size is sufficiently large compared to the sparsity of $x$.
Accelerated Stochastic Gradient Descent for Minimizing Finite Sums
Authors: Atsushi Nitanda
Date: 2015-06-09 00:00:00

We propose an optimization method for minimizing the finite sums of smooth
convex functions. Our method incorporates an accelerated gradient descent (AGD)
and a stochastic variance reduction gradient (SVRG) in a mini-batch setting.
Unlike SVRG, our method can be directly applied to non-strongly and strongly
convex problems. We show that our method achieves a lower overall complexity
than the recently proposed methods that supports non-strongly convex problems.
Moreover, this method has a fast rate of convergence for strongly convex
problems. Our experiments show the effectiveness of our method.
Genetic Algorithms for multimodal optimization: a review
Authors: Noe Casas
Date: 2015-06-10 00:00:00
Tasks: Diversity

In this article we provide a comprehensive review of the different
evolutionary algorithm techniques used to address multimodal optimization
problems, classifying them according to the nature of their approach. On the
one hand there are algorithms that address the issue of the early convergence
to a local optimum by differentiating the individuals of the population into
groups and limiting their interaction, hence having each group evolve with a
high degree of independence. On the other hand other approaches are based on
directly addressing the lack of genetic diversity of the population by
introducing elements into the evolutionary dynamics that promote new niches of
the genotypical space to be explored. Finally, we study multi-objective
optimization genetic algorithms, that handle the situations where multiple
criteria have to be satisfied with no penalty for any of them. Very rich
literature has arised over the years on these topics, and we aim at offering an
overview of the most important techniques of each branch of the field.
Amoeba Techniques for Shape and Texture Analysis
Authors: Martin Welk
Date: 2014-11-12 00:00:00
Tasks: Texture Classification

Morphological amoebas are image-adaptive structuring elements for
morphological and other local image filters introduced by Lerallut et al. Their
construction is based on combining spatial distance with contrast information
into an image-dependent metric. Amoeba filters show interesting parallels to
image filtering methods based on partial differential equations (PDEs), which
can be confirmed by asymptotic equivalence results. In computing amoebas, graph
structures are generated that hold information about local image texture. This
paper reviews and summarises the work of the author and his coauthors on
morphological amoebas, particularly their relations to PDE filters and texture
analysis. It presents some extensions and points out directions for future
investigation on the subject.
Optical Flow on Evolving Sphere-Like Surfaces
Authors: Lukas F. Lang, Otmar Scherzer
Date: 2015-06-10 00:00:00
Tasks: Optical Flow Estimation

In this work we consider optical flow on evolving Riemannian 2-manifolds
which can be parametrised from the 2-sphere. Our main motivation is to estimate
cell motion in time-lapse volumetric microscopy images depicting fluorescently
labelled cells of a live zebrafish embryo. We exploit the fact that the
recorded cells float on the surface of the embryo and allow for the extraction
of an image sequence together with a sphere-like surface. We solve the
resulting variational problem by means of a Galerkin method based on vector
spherical harmonics and present numerical results computed from the
aforementioned microscopy data.
Detecting Clusters of Anomalies on Low-Dimensional Feature Subsets with Application to Network Traffic Flow Data
Authors: Zhicong Qiu, David J. Miller, George Kesidis
Date: 2015-06-10 00:00:00
Tasks: Anomaly Detection, Group Anomaly Detection, Intrusion Detection, Network Intrusion Detection

In a variety of applications, one desires to detect groups of anomalous data
samples, with a group potentially manifesting its atypicality (relative to a
reference model) on a low-dimensional subset of the full measured set of
features. Samples may only be weakly atypical individually, whereas they may be
strongly atypical when considered jointly. What makes this group anomaly
detection problem quite challenging is that it is a priori unknown which subset
of features jointly manifests a particular group of anomalies. Moreover, it is
unknown how many anomalous groups are present in a given data batch. In this
work, we develop a group anomaly detection (GAD) scheme to identify the subset
of samples and subset of features that jointly specify an anomalous cluster. We
apply our approach to network intrusion detection to detect BotNet and
peer-to-peer flow clusters. Unlike previous studies, our approach captures and
exploits statistical dependencies that may exist between the measured features.
Experiments on real world network traffic data demonstrate the advantage of our
proposed system, and highlight the importance of exploiting feature dependency
structure, compared to the feature (or test) independence assumption made in
previous studies.
Wide baseline stereo matching with convex bounded-distortion constraints
Authors: Meirav Galun, Tal Amir, Tal Hassner, Ronen Basri, Yaron Lipman
Date: 2015-06-10 00:00:00
Tasks: Stereo Matching, Stereo Matching Hand

Finding correspondences in wide baseline setups is a challenging problem.
Existing approaches have focused largely on developing better feature
descriptors for correspondence and on accurate recovery of epipolar line
constraints. This paper focuses on the challenging problem of finding
correspondences once approximate epipolar constraints are given. We introduce a
novel method that integrates a deformation model. Specifically, we formulate
the problem as finding the largest number of corresponding points related by a
bounded distortion map that obeys the given epipolar constraints. We show that,
while the set of bounded distortion maps is not convex, the subset of maps that
obey the epipolar line constraints is convex, allowing us to introduce an
efficient algorithm for matching. We further utilize a robust cost function for
matching and employ majorization-minimization for its optimization. Our
experiments indicate that our method finds significantly more accurate maps
than existing approaches.
First-order regret bounds for combinatorial semi-bandits
Authors: Gergely Neu
Date: 2015-02-23 00:00:00
Tasks: Combinatorial Optimization

We consider the problem of online combinatorial optimization under
semi-bandit feedback, where a learner has to repeatedly pick actions from a
combinatorial decision set in order to minimize the total losses associated
with its decisions. After making each decision, the learner observes the losses
associated with its action, but not other losses. For this problem, there are
several learning algorithms that guarantee that the learner's expected regret
grows as $\widetilde{O}(\sqrt{T})$ with the number of rounds $T$. In this
paper, we propose an algorithm that improves this scaling to
$\widetilde{O}(\sqrt{{L_T^*}})$, where $L_T^*$ is the total loss of the best
action. Our algorithm is among the first to achieve such guarantees in a
partial-feedback scheme, and the first one to do so in a combinatorial setting.
Memory and information processing in neuromorphic systems
Authors: Giacomo Indiveri, Shih-Chii Liu
Date: 2015-06-10 00:00:00

A striking difference between brain-inspired neuromorphic processors and
current von Neumann processors architectures is the way in which memory and
processing is organized. As Information and Communication Technologies continue
to address the need for increased computational power through the increase of
cores within a digital processor, neuromorphic engineers and scientists can
complement this need by building processor architectures where memory is
distributed with the processing. In this paper we present a survey of
brain-inspired processor architectures that support models of cortical networks
and deep neural networks. These architectures range from serial clocked
implementations of multi-neuron systems to massively parallel asynchronous ones
and from purely digital systems to mixed analog/digital systems which implement
more biological-like models of neurons and synapses together with a suite of
adaptation and learning mechanisms analogous to the ones found in biological
nervous systems. We describe the advantages of the different approaches being
pursued and present the challenges that need to be addressed for building
artificial neural processing systems that can display the richness of behaviors
seen in biological systems.
Combining Temporal Information and Topic Modeling for Cross-Document Event Ordering
Authors: Borja Navarro-Colorado, Estela Saquete
Date: 2015-06-10 00:00:00
Tasks: Articles, coreference-resolution, Coreference Resolution, Event Coreference Resolution, Relation Extraction, Temporal Relation Extraction

Building unified timelines from a collection of written news articles
requires cross-document event coreference resolution and temporal relation
extraction. In this paper we present an approach event coreference resolution
according to: a) similar temporal information, and b) similar semantic
arguments. Temporal information is detected using an automatic temporal
information system (TIPSem), while semantic information is represented by means
of LDA Topic Modeling. The evaluation of our approach shows that it obtains the
highest Micro-average F-score results in the SemEval2015 Task 4: TimeLine:
Cross-Document Event Ordering (25.36\% for TrackB, 23.15\% for SubtrackB), with
an improvement of up to 6\% in comparison to the other systems. However, our
experiment also showed some draw-backs in the Topic Modeling approach that
degrades performance of the system.
A Scale Mixture Perspective of Multiplicative Noise in Neural Networks
Authors: Eric Nalisnick, Anima Anandkumar, Padhraic Smyth
Date: 2015-06-10 00:00:00
Tasks: Model Compression

Corrupting the input and hidden layers of deep neural networks (DNNs) with
multiplicative noise, often drawn from the Bernoulli distribution (or
'dropout'), provides regularization that has significantly contributed to deep
learning's success. However, understanding how multiplicative corruptions
prevent overfitting has been difficult due to the complexity of a DNN's
functional form. In this paper, we show that when a Gaussian prior is placed on
a DNN's weights, applying multiplicative noise induces a Gaussian scale
mixture, which can be reparameterized to circumvent the problematic likelihood
function. Analysis can then proceed by using a type-II maximum likelihood
procedure to derive a closed-form expression revealing how regularization
evolves as a function of the network's weights. Results show that
multiplicative noise forces weights to become either sparse or invariant to
rescaling. We find our analysis has implications for model compression as it
naturally reveals a weight pruning rule that starkly contrasts with the
commonly used signal-to-noise ratio (SNR). While the SNR prunes weights with
large variances, seeing them as noisy, our approach recognizes their robustness
and retains them. We empirically demonstrate our approach has a strong
advantage over the SNR heuristic and is competitive to retraining with soft
targets produced from a teacher model.
ICDAR 2015 Text Reading in the Wild Competition
Authors: Xinyu Zhou, Shuchang Zhou, Cong Yao, Zhimin Cao, Qi Yin
Date: 2015-06-10 00:00:00
Tasks: Text Detection

Recently, text detection and recognition in natural scenes are becoming
increasing popular in the computer vision community as well as the document
analysis community. However, majority of the existing ideas, algorithms and
systems are specifically designed for English. This technical report presents
the final results of the ICDAR 2015 Text Reading in the Wild (TRW 2015)
competition, which aims at establishing a benchmark for assessing detection and
recognition algorithms devised for both Chinese and English scripts and
providing a playground for researchers from the community. In this article, we
describe in detail the dataset, tasks, evaluation protocols and participants of
this competition, and report the performance of the participating methods.
Moreover, promising directions for future research are discussed.