pwc-archive/papers-with-abstracts huggingface dataset:
A Factorized Model for Transitive Verbs in Compositional Distributional Semantics
Authors: Lilach Edelstein, Roi Reichart
Date: 2016-09-25 00:00:00
Tasks: Object

We present a factorized compositional distributional semantics model for the
representation of transitive verb constructions. Our model first produces
(subject, verb) and (verb, object) vector representations based on the
similarity of the nouns in the construction to each of the nouns in the
vocabulary and the tendency of these nouns to take the subject and object roles
of the verb. These vectors are then combined into a final (subject,verb,object)
representation through simple vector operations. On two established tasks for
the transitive verb construction our model outperforms recent previous work.
Accurate and Efficient Hyperbolic Tangent Activation Function on FPGA using the DCT Interpolation Filter
Authors: Ahmed M. Abdelsalam, J. M. Pierre Langlois, F. Cheriet
Date: 2016-09-25 00:00:00

Implementing an accurate and fast activation function with low cost is a
crucial aspect to the implementation of Deep Neural Networks (DNNs) on FPGAs.
We propose a high-accuracy approximation approach for the hyperbolic tangent
activation function of artificial neurons in DNNs. It is based on the Discrete
Cosine Transform Interpolation Filter (DCTIF). The proposed architecture
combines simple arithmetic operations on stored samples of the hyperbolic
tangent function and on input data. The proposed DCTIF implementation achieves
two orders of magnitude greater precision than previous work while using the
same or fewer computational resources. Various combinations of DCTIF parameters
can be chosen to tradeoff the accuracy and complexity of the hyperbolic tangent
function. In one case, the proposed architecture approximates the hyperbolic
tangent activation function with 10E-5 maximum error while requiring only 1.52
Kbits memory and 57 LUTs of a Virtex-7 FPGA. We also discuss how the activation
function accuracy affects the performance of DNNs in terms of their training
and testing accuracies. We show that a high accuracy approximation can be
necessary in order to maintain the same DNN training and testing performances
realized by the exact function.
Generalized Kalman Smoothing: Modeling and Algorithms
Authors: A. Y. Aravkin, J. V. Burke, L. Ljung, A. Lozano, G. Pillonetto
Date: 2016-09-20 00:00:00
Tasks: BIG-bench Machine Learning, Survey

State-space smoothing has found many applications in science and engineering.
Under linear and Gaussian assumptions, smoothed estimates can be obtained using
efficient recursions, for example Rauch-Tung-Striebel and Mayne-Fraser
algorithms. Such schemes are equivalent to linear algebraic techniques that
minimize a convex quadratic objective function with structure induced by the
dynamic model.
  These classical formulations fall short in many important circumstances. For
instance, smoothers obtained using quadratic penalties can fail when outliers
are present in the data, and cannot track impulsive inputs and abrupt state
changes. Motivated by these shortcomings, generalized Kalman smoothing
formulations have been proposed in the last few years, replacing quadratic
models with more suitable, often nonsmooth, convex functions. In contrast to
classical models, these general estimators require use of iterated algorithms,
and these have received increased attention from control, signal processing,
machine learning, and optimization communities.
  In this survey we show that the optimization viewpoint provides the control
and signal processing community great freedom in the development of novel
modeling and inference frameworks for dynamical systems. We discuss general
statistical models for dynamic systems, making full use of nonsmooth convex
penalties and constraints, and providing links to important models in signal
processing and machine learning. We also survey optimization techniques for
these formulations, paying close attention to dynamic problem structure.
Modeling concepts and algorithms are illustrated with numerical examples.
Orthogonal parallel MCMC methods for sampling and optimization
Authors: L. Martino, V. Elvira, D. Luengo, J. Corander, F. Louzada
Date: 2015-07-30 00:00:00
Tasks: Bayesian Inference

Monte Carlo (MC) methods are widely used for Bayesian inference and
optimization in statistics, signal processing and machine learning. A
well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms.
In order to foster better exploration of the state space, specially in
high-dimensional applications, several schemes employing multiple parallel MCMC
chains have been recently introduced. In this work, we describe a novel
parallel interacting MCMC scheme, called {\it orthogonal MCMC} (O-MCMC), where
a set of "vertical" parallel MCMC chains share information using some
"horizontal" MCMC techniques working on the entire population of current
states. More specifically, the vertical chains are led by random-walk
proposals, whereas the horizontal MCMC techniques employ independent proposals,
thus allowing an efficient combination of global exploration and local
approximation. The interaction is contained in these horizontal iterations.
Within the analysis of different implementations of O-MCMC, novel schemes in
order to reduce the overall computational cost of parallel multiple try
Metropolis (MTM) chains are also presented. Furthermore, a modified version of
O-MCMC for optimization is provided by considering parallel simulated annealing
(SA) algorithms. Numerical results show the advantages of the proposed sampling
scheme in terms of efficiency in the estimation, as well as robustness in terms
of independence with respect to initial values and the choice of the
parameters.
The RNN-ELM Classifier
Authors: Athanasios Vlontzos
Date: 2016-09-25 00:00:00
Tasks: General Classification

In this paper we examine learning methods combining the Random Neural
Network, a biologically inspired neural network and the Extreme Learning
Machine that achieve state of the art classification performance while
requiring much shorter training time. The Random Neural Network is a integrate
and fire computational model of a neural network whose mathematical structure
permits the efficient analysis of large ensembles of neurons. An activation
function is derived from the RNN and used in an Extreme Learning Machine. We
compare the performance of this combination against the ELM with various
activation functions, we reduce the input dimensionality via PCA and compare
its performance vs. autoencoder based versions of the RNN-ELM.
Sooner than Expected: Hitting the Wall of Complexity in Evolution
Authors: Thomas Schmickl, Payam Zahadat, Heiko Hamann
Date: 2016-09-25 00:00:00
Tasks: Artificial Life, Stochastic Optimization

In evolutionary robotics an encoding of the control software, which maps
sensor data (input) to motor control values (output), is shaped by stochastic
optimization methods to complete a predefined task. This approach is assumed to
be beneficial compared to standard methods of controller design in those cases
where no a-priori model is available that could help to optimize performance.
Also for robots that have to operate in unpredictable environments, an
evolutionary robotics approach is favorable. We demonstrate here that such a
model-free approach is not a free lunch, as already simple tasks can represent
unsolvable barriers for fully open-ended uninformed evolutionary computation
techniques. We propose here the 'Wankelmut' task as an objective for an
evolutionary approach that starts from scratch without pre-shaped controller
software or any other informed approach that would force the behavior to be
evolved in a desired way. Our focal claim is that 'Wankelmut' represents the
simplest set of problems that makes plain-vanilla evolutionary computation
fail. We demonstrate this by a series of simple standard evolutionary
approaches using different fitness functions and standard artificial neural
networks as well as continuous-time recurrent neural networks. All our tested
approaches failed. We claim that any other evolutionary approach will also fail
that does per-se not favor or enforce modularity and does not freeze or protect
already evolved functionalities. Thus we propose a hard-to-pass benchmark and
make a strong statement for self-complexifying and generative approaches in
evolutionary computation. We anticipate that defining such a 'simplest task to
fail' is a valuable benchmark for promoting future development in the field of
artificial intelligence, evolutionary robotics and artificial life.
Joint Learning of Siamese CNNs and Temporally Constrained Metrics for Tracklet Association
Authors: Bing Wang, Li Wang, Bing Shuai, Zhen Zuo, Ting Liu, Kap Luk Chan, Gang Wang
Date: 2016-05-15 00:00:00
Tasks: Multi-Object Tracking, Multi-Task Learning, Object Tracking

In this paper, we study the challenging problem of multi-object tracking in a
complex scene captured by a single camera. Different from the existing tracklet
association-based tracking methods, we propose a novel and efficient way to
obtain discriminative appearance-based tracklet affinity models. Our proposed
method jointly learns the convolutional neural networks (CNNs) and temporally
constrained metrics. In our method, a Siamese convolutional neural network
(CNN) is first pre-trained on the auxiliary data. Then the Siamese CNN and
temporally constrained metrics are jointly learned online to construct the
appearance-based tracklet affinity models. The proposed method can jointly
learn the hierarchical deep features and temporally constrained segment-wise
metrics under a unified framework. For reliable association between tracklets,
a novel loss function incorporating temporally constrained multi-task learning
mechanism is proposed. By employing the proposed method, tracklet association
can be accomplished even in challenging situations. Moreover, a new dataset
with 40 fully annotated sequences is created to facilitate the tracking
evaluation. Experimental results on five public datasets and the new
large-scale dataset show that our method outperforms several state-of-the-art
approaches in multi-object tracking.
Generative Visual Manipulation on the Natural Image Manifold
Authors: Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, Alexei A. Efros
Date: 2016-09-12 00:00:00
Tasks: Image Manipulation

Realistic image manipulation is challenging because it requires modifying the
image appearance in a user-controlled way, while preserving the realism of the
result. Unless the user has considerable artistic skill, it is easy to "fall
off" the manifold of natural images while editing. In this paper, we propose to
learn the natural image manifold directly from data using a generative
adversarial neural network. We then define a class of image editing operations,
and constrain their output to lie on that learned manifold at all times. The
model automatically adjusts the output keeping all edits as realistic as
possible. All our manipulations are expressed in terms of constrained
optimization and are applied in near-real time. We evaluate our algorithm on
the task of realistic photo manipulation of shape and color. The presented
method can further be used for changing one image to look like the other, as
well as generating novel imagery from scratch based on user's scribbles.
Learning by Stimulation Avoidance: A Principle to Control Spiking Neural Networks Dynamics
Authors: Lana Sinapayen, Atsushi Masumori, Takashi Ikegami
Date: 2016-09-25 00:00:00

Learning based on networks of real neurons, and by extension biologically
inspired models of neural networks, has yet to find general learning rules
leading to widespread applications. In this paper, we argue for the existence
of a principle allowing to steer the dynamics of a biologically inspired neural
network. Using carefully timed external stimulation, the network can be driven
towards a desired dynamical state. We term this principle "Learning by
Stimulation Avoidance" (LSA). We demonstrate through simulation that the
minimal sufficient conditions leading to LSA in artificial networks are also
sufficient to reproduce learning results similar to those obtained in
biological neurons by Shahaf and Marom [1]. We examine the mechanism's basic
dynamics in a reduced network, and demonstrate how it scales up to a network of
100 neurons. We show that LSA has a higher explanatory power than existing
hypotheses about the response of biological neural networks to external
simulation, and can be used as a learning rule for an embodied application:
learning of wall avoidance by a simulated robot. The surge in popularity of
artificial neural networks is mostly directed to disembodied models of neurons
with biologically irrelevant dynamics: to the authors' knowledge, this is the
first work demonstrating sensory-motor learning with random spiking networks
through pure Hebbian learning.
Large-Scale Machine Translation between Arabic and Hebrew: Available Corpora and Initial Results
Authors: Yonatan Belinkov, James Glass
Date: 2016-09-25 00:00:00
Tasks: Machine Translation, Translation

Machine translation between Arabic and Hebrew has so far been limited by a
lack of parallel corpora, despite the political and cultural importance of this
language pair. Previous work relied on manually-crafted grammars or pivoting
via English, both of which are unsatisfactory for building a scalable and
accurate MT system. In this work, we compare standard phrase-based and neural
systems on Arabic-Hebrew translation. We experiment with tokenization by
external tools and sub-word modeling by character-level neural models, and show
that both methods lead to improved translation performance, with a small
advantage to the neural models.
The distribution of information content in English sentences
Authors: Shuiyuan Yu, Jin Cong, Junying Liang, Haitao Liu
Date: 2016-09-24 00:00:00
Tasks: Position, Sentence

Sentence is a basic linguistic unit, however, little is known about how
information content is distributed across different positions of a sentence.
Based on authentic language data of English, the present study calculated the
entropy and other entropy-related statistics for different sentence positions.
The statistics indicate a three-step staircase-shaped distribution pattern,
with entropy in the initial position lower than the medial positions (positions
other than the initial and final), the medial positions lower than the final
position and the medial positions showing no significant difference. The
results suggest that: (1) the hypotheses of Constant Entropy Rate and Uniform
Information Density do not hold for the sentence-medial positions; (2) the
context of a word in a sentence should not be simply defined as all the words
preceding it in the same sentence; and (3) the contextual information content
in a sentence does not accumulate incrementally but follows a pattern of "the
whole is greater than the sum of parts".
Existence of Hierarchies and Human's Pursuit of Top Hierarchy Lead to Power Law
Authors: Shuiyuan Yu, Junying Liang, Haitao Liu
Date: 2016-09-24 00:00:00

The power law is ubiquitous in natural and social phenomena, and is
considered as a universal relationship between the frequency and its rank for
diverse social systems. However, a general model is still lacking to interpret
why these seemingly unrelated systems share great similarity. Through a
detailed analysis of natural language texts and simulation experiments based on
the proposed 'Hierarchical Selection Model', we found that the existence of
hierarchies and human's pursuit of top hierarchy lead to the power law.
Further, the power law is a statistical and emergent performance of
hierarchies, and it is the universality of hierarchies that contributes to the
ubiquity of the power law.
Nonparametric Bayesian Storyline Detection from Microtexts
Authors: Vinodh Krishnan, Jacob Eisenstein
Date: 2016-01-18 00:00:00
Proceeding: WS 2016 11
Tasks: Clustering, Retrieval

News events and social media are composed of evolving storylines, which
capture public attention for a limited period of time. Identifying storylines
requires integrating temporal and linguistic information, and prior work takes
a largely heuristic approach. We present a novel online non-parametric Bayesian
framework for storyline detection, using the distance-dependent Chinese
Restaurant Process (dd-CRP). To ensure efficient linear-time inference, we
employ a fixed-lag Gibbs sampling procedure, which is novel for the dd-CRP. We
evaluate on the TREC Twitter Timeline Generation (TTG), obtaining encouraging
results: despite using a weak baseline retrieval model, the dd-CRP story
clustering method is competitive with the best entries in the 2014 TTG task.
Max-Norm Optimization for Robust Matrix Recovery
Authors: Ethan X. Fang, Han Liu, Kim-Chuan Toh, Wen-Xin Zhou
Date: 2016-09-24 00:00:00
Tasks: Matrix Completion

This paper studies the matrix completion problem under arbitrary sampling
schemes. We propose a new estimator incorporating both max-norm and
nuclear-norm regularization, based on which we can conduct efficient low-rank
matrix recovery using a random subset of entries observed with additive noise
under general non-uniform and unknown sampling distributions. This method
significantly relaxes the uniform sampling assumption imposed for the widely
used nuclear-norm penalized approach, and makes low-rank matrix recovery
feasible in more practical settings. Theoretically, we prove that the proposed
estimator achieves fast rates of convergence under different settings.
Computationally, we propose an alternating direction method of multipliers
algorithm to efficiently compute the estimator, which bridges a gap between
theory and practice of machine learning methods with max-norm regularization.
Further, we provide thorough numerical studies to evaluate the proposed method
using both simulated and real datasets.
Optimizing the Learning Order of Chinese Characters Using a Novel Topological Sort Algorithm
Authors: James C. Loach, Jinzhao Wang
Date: 2016-02-28 00:00:00
Tasks: Scheduling

We present a novel algorithm for optimizing the order in which Chinese
characters are learned, one that incorporates the benefits of learning them in
order of usage frequency and in order of their hierarchal structural
relationships. We show that our work outperforms previously published orders
and algorithms. Our algorithm is applicable to any scheduling task where nodes
have intrinsic differences in importance and must be visited in topological
order.
Rationale-Augmented Convolutional Neural Networks for Text Classification
Authors: Ye Zhang, Iain Marshall, Byron C. Wallace
Date: 2016-05-14 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Classification, General Classification, Sentence, text-classification, Text Classification

We present a new Convolutional Neural Network (CNN) model for text
classification that jointly exploits labels on documents and their component
sentences. Specifically, we consider scenarios in which annotators explicitly
mark sentences (or snippets) that support their overall document
categorization, i.e., they provide rationales. Our model exploits such
supervision via a hierarchical approach in which each document is represented
by a linear combination of the vector representations of its component
sentences. We propose a sentence-level convolutional model that estimates the
probability that a given sentence is a rationale, and we then scale the
contribution of each sentence to the aggregate document representation in
proportion to these estimates. Experiments on five classification datasets that
have document labels and associated rationales demonstrate that our approach
consistently outperforms strong baselines. Moreover, our model naturally
provides explanations for its predictions.
Perceptual uniform descriptor and Ranking on manifold: A bridge between image representation and ranking for image retrieval
Authors: Shenglan Liu, Jun Wu, Lin Feng, Yang Liu, Hong Qiao, Wenbo Luo Muxin Sun, Wei Wang
Date: 2016-09-24 00:00:00
Tasks: Image Retrieval, Retrieval

Incompatibility of image descriptor and ranking is always neglected in image
retrieval. In this paper, manifold learning and Gestalt psychology theory are
involved to solve the incompatibility problem. A new holistic descriptor called
Perceptual Uniform Descriptor (PUD) based on Gestalt psychology is proposed,
which combines color and gradient direction to imitate the human visual
uniformity. PUD features in the same class images distributes on one manifold
in most cases because PUD improves the visual uniformity of the traditional
descriptors. Thus, we use manifold ranking and PUD to realize image retrieval.
Experiments were carried out on five benchmark data sets, and the proposed
method can greatly improve the accuracy of image retrieval. Our experimental
results in the Ukbench and Corel-1K datasets demonstrated that N-S score
reached to 3.58 (HSV 3.4) and mAP to 81.77% (ODBTC 77.9%) respectively by
utilizing PUD which has only 280 dimension. The results are higher than other
holistic image descriptors (even some local ones) and state-of-the-arts
retrieval methods.
Three Tiers Neighborhood Graph and Multi-graph Fusion Ranking for Multi-feature Image Retrieval: A Manifold Aspect
Authors: Shenglan Liu, Muxin Sun, Lin Feng, Yang Liu, Jun Wu
Date: 2016-09-24 00:00:00
Tasks: Image Retrieval, Re-Ranking, Retrieval

Single feature is inefficient to describe content of an image, which is a
shortcoming in traditional image retrieval task. We know that one image can be
described by different features. Multi-feature fusion ranking can be utilized
to improve the ranking list of query. In this paper, we first analyze graph
structure and multi-feature fusion re-ranking from manifold aspect. Then, Three
Tiers Neighborhood Graph (TTNG) is constructed to re-rank the original ranking
list by single feature and to enhance precision of single feature. Furthermore,
we propose Multi-graph Fusion Ranking (MFR) for multi-feature ranking, which
considers the correlation of all images in multiple neighborhood graphs.
Evaluations are conducted on UK-bench, Corel-1K, Corel-10K and Cifar-10
benchmark datasets. The experimental results show that our TTNG and MFR
outperform than other state-of-the-art methods. For example, we achieve
competitive results N-S score 3.91 and precision 65.00% on UK-bench and
Corel-10K datasets respectively.
DimensionApp : android app to estimate object dimensions
Authors: Suriya Singh, Vijay Kumar
Date: 2016-09-24 00:00:00
Tasks: Line Detection, Object

In this project, we develop an android app that uses on computer vision
techniques to estimate an object dimension present in field of view. The app
while having compact size, is accurate upto +/- 5 mm and robust towards touch
inputs. We use single-view metrology to compute accurate measurement. Unlike
previous approaches, our technique does not rely on line detection and can be
generalize to any object shape easily.
An Investigation of Recurrent Neural Architectures for Drug Name Recognition
Authors: Raghavendra Chalapathy, Ehsan Zare Borzeshi, Massimo Piccardi
Date: 2016-09-24 00:00:00
Proceeding: WS 2016 11
Tasks: Pharmacovigilance

Drug name recognition (DNR) is an essential step in the Pharmacovigilance
(PV) pipeline. DNR aims to find drug name mentions in unstructured biomedical
texts and classify them into predefined categories. State-of-the-art DNR
approaches heavily rely on hand crafted features and domain specific resources
which are difficult to collect and tune. For this reason, this paper
investigates the effectiveness of contemporary recurrent neural architectures -
the Elman and Jordan networks and the bidirectional LSTM with CRF decoding - at
performing DNR straight from the text. The experimental results achieved on the
authoritative SemEval-2013 Task 9.1 benchmarks show that the bidirectional
LSTM-CRF ranks closely to highly-dedicated, hand-crafted systems.