pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Understand Phrases by Embedding the Dictionary
Authors: Felix Hill, Kyunghyun Cho, Anna Korhonen, Yoshua Bengio
Date: 2015-04-02 00:00:00
Proceeding: TACL 2016 1
Tasks: General Knowledge

Distributional models that learn rich semantic word representations are a
success story of recent NLP research. However, developing models that learn
useful representations of phrases and sentences has proved far harder. We
propose using the definitions found in everyday dictionaries as a means of
bridging this gap between lexical and phrasal semantics. Neural language
embedding models can be effectively trained to map dictionary definitions
(phrases) to (lexical) representations of the words defined by those
definitions. We present two applications of these architectures: "reverse
dictionaries" that return the name of a concept given a definition or
description and general-knowledge crossword question answerers. On both tasks,
neural language embedding models trained on definitions from a handful of
freely-available lexical resources perform as well or better than existing
commercial systems that rely on significant task-specific engineering. The
results highlight the effectiveness of both neural embedding architectures and
definition-based training for developing models that understand phrases and
sentences.
Multi-velocity neural networks for gesture recognition in videos
Authors: Otkrist Gupta, Dan Raviv, Ramesh Raskar
Date: 2016-03-22 00:00:00
Tasks: Action Recognition, General Classification, Gesture Recognition, Temporal Action Localization

We present a new action recognition deep neural network which adaptively
learns the best action velocities in addition to the classification. While deep
neural networks have reached maturity for image understanding tasks, we are
still exploring network topologies and features to handle the richer
environment of video clips. Here, we tackle the problem of multiple velocities
in action recognition, and provide state-of-the-art results for gesture
recognition, on known and new collected datasets. We further provide the
training steps for our semi-supervised network, suited to learn from huge
unlabeled datasets with only a fraction of labeled examples.
Object Recognition and Identification Using ESM Data
Authors: E. Taghavi, D. Song, R. Tharmarasa, T. Kirubarajan, Anne-Claire Boury-Brisset, Bhashyam Balaji
Date: 2016-03-22 00:00:00
Tasks: Object, Object Recognition

Recognition and identification of unknown targets is a crucial task in
surveillance and security systems. Electronic Support Measures (ESM) are one of
the most effective sensors for identification, especially for maritime and
air--to--ground applications. In typical surveillance systems multiple ESM
sensors are usually deployed along with kinematic sensors like radar. Different
ESM sensors may produce different types of reports ready to be sent to the
fusion center. The focus of this paper is to develop a new architecture for
target recognition and identification when non--homogeneous ESM and possibly
kinematic reports are received at the fusion center. The new fusion
architecture is evaluated using simulations to show the benefit of utilizing
different ESM reports such as attributes and signal level ESM data.
Patterns of Scalable Bayesian Inference
Authors: Elaine Angelino, Matthew James Johnson, Ryan P. Adams
Date: 2016-02-16 00:00:00
Tasks: Bayesian Inference

Datasets are growing not just in size but in complexity, creating a demand
for rich models and quantification of uncertainty. Bayesian methods are an
excellent fit for this demand, but scaling Bayesian inference is a challenge.
In response to this challenge, there has been considerable recent work based on
varying assumptions about model structure, underlying computational resources,
and the importance of asymptotic correctness. As a result, there is a zoo of
ideas with few clear overarching principles.
  In this paper, we seek to identify unifying principles, patterns, and
intuitions for scaling Bayesian inference. We review existing work on utilizing
modern computing resources with both MCMC and variational approximation
techniques. From this taxonomy of ideas, we characterize the general principles
that have proven successful for designing scalable inference procedures and
comment on the path forward.
Adaptive Parameter Selection in Evolutionary Algorithms by Reinforcement Learning with Dynamic Discretization of Parameter Range
Authors: Arkady Rost, Irina Petrova, Arina Buzdalova
Date: 2016-03-22 00:00:00
Tasks: Evolutionary Algorithms, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Online parameter controllers for evolutionary algorithms adjust values of
parameters during the run of an evolutionary algorithm. Recently a new
efficient parameter controller based on reinforcement learning was proposed by
Karafotias et al. In this method ranges of parameters are discretized into
several intervals before the run. However, performing adaptive discretization
during the run may increase efficiency of an evolutionary algorithm. Aleti et
al. proposed another efficient controller with adaptive discretization.
  In the present paper we propose a parameter controller based on reinforcement
learning with adaptive discretization. The proposed controller is compared with
the existing parameter adjusting methods on several test problems using
different configurations of an evolutionary algorithm. For the test problems,
we consider four continuous functions, namely the sphere function, the
Rosenbrock function, the Levi function and the Rastrigin function. Results show
that the new controller outperforms the other controllers on most of the
considered test problems.
Multi-domain machine translation enhancements by parallel data extraction from comparable corpora
Authors: Krzysztof Wołk, Emilia Rejmund, Krzysztof Marasek
Date: 2016-03-22 00:00:00
Tasks: Machine Translation, Translation

Parallel texts are a relatively rare language resource, however, they
constitute a very useful research material with a wide range of applications.
This study presents and analyses new methodologies we developed for obtaining
such data from previously built comparable corpora. The methodologies are
automatic and unsupervised which makes them good for large scale research. The
task is highly practical as non-parallel multilingual data occur much more
frequently than parallel corpora and accessing them is easy, although parallel
sentences are a considerably more useful resource. In this study, we propose a
method of automatic web crawling in order to build topic-aligned comparable
corpora, e.g. based on the Wikipedia or Euronews.com. We also developed new
methods of obtaining parallel sentences from comparable data and proposed
methods of filtration of corpora capable of selecting inconsistent or only
partially equivalent translations. Our methods are easily scalable to other
languages. Evaluation of the quality of the created corpora was performed by
analysing the impact of their use on statistical machine translation systems.
Experiments were presented on the basis of the Polish-English language pair for
texts from different domains, i.e. lectures, phrasebooks, film dialogues,
European Parliament proceedings and texts contained medicines leaflets. We also
tested a second method of creating parallel corpora based on data from
comparable corpora which allows for automatically expanding the existing corpus
of sentences about a given domain on the basis of analogies found between them.
It does not require, therefore, having past parallel resources in order to
train a classifier.
Doubly Random Parallel Stochastic Methods for Large Scale Learning
Authors: Aryan Mokhtari, Alec Koppel, Alejandro Ribeiro
Date: 2016-03-22 00:00:00

We consider learning problems over training sets in which both, the number of
training examples and the dimension of the feature vectors, are large. To solve
these problems we propose the random parallel stochastic algorithm (RAPSA). We
call the algorithm random parallel because it utilizes multiple processors to
operate in a randomly chosen subset of blocks of the feature vector. We call
the algorithm parallel stochastic because processors choose elements of the
training set randomly and independently. Algorithms that are parallel in either
of these dimensions exist, but RAPSA is the first attempt at a methodology that
is parallel in both, the selection of blocks and the selection of elements of
the training set. In RAPSA, processors utilize the randomly chosen functions to
compute the stochastic gradient component associated with a randomly chosen
block. The technical contribution of this paper is to show that this minimally
coordinated algorithm converges to the optimal classifier when the training
objective is convex. In particular, we show that: (i) When using decreasing
stepsizes, RAPSA converges almost surely over the random choice of blocks and
functions. (ii) When using constant stepsizes, convergence is to a neighborhood
of optimality with a rate that is linear in expectation. RAPSA is numerically
evaluated on the MNIST digit recognition problem.
Energy-Efficient ConvNets Through Approximate Computing
Authors: Bert Moons, Bert de Brabandere, Luc van Gool, Marian Verhelst
Date: 2016-03-22 00:00:00
Tasks: Classification, General Classification

Recently ConvNets or convolutional neural networks (CNN) have come up as
state-of-the-art classification and detection algorithms, achieving near-human
performance in visual detection. However, ConvNet algorithms are typically very
computation and memory intensive. In order to be able to embed ConvNet-based
classification into wearable platforms and embedded systems such as smartphones
or ubiquitous electronics for the internet-of-things, their energy consumption
should be reduced drastically. This paper proposes methods based on approximate
computing to reduce energy consumption in state-of-the-art ConvNet
accelerators. By combining techniques both at the system- and circuit level, we
can gain energy in the systems arithmetic: up to 30x without losing
classification accuracy and more than 100x at 99% classification accuracy,
compared to the commonly used 16-bit fixed point number format.
Convolution in Convolution for Network in Network
Authors: Yanwei Pang, Manli Sun, Xiaoheng Jiang, Xuelong. Li
Date: 2016-03-22 00:00:00

Network in Netwrok (NiN) is an effective instance and an important extension
of Convolutional Neural Network (CNN) consisting of alternating convolutional
layers and pooling layers. Instead of using a linear filter for convolution,
NiN utilizes shallow MultiLayer Perceptron (MLP), a nonlinear function, to
replace the linear filter. Because of the powerfulness of MLP and $ 1\times 1 $
convolutions in spatial domain, NiN has stronger ability of feature
representation and hence results in better recognition rate. However, MLP
itself consists of fully connected layers which give rise to a large number of
parameters. In this paper, we propose to replace dense shallow MLP with sparse
shallow MLP. One or more layers of the sparse shallow MLP are sparely connected
in the channel dimension or channel-spatial domain. The proposed method is
implemented by applying unshared convolution across the channel dimension and
applying shared convolution across the spatial dimension in some computational
layers. The proposed method is called CiC. Experimental results on the CIFAR10
dataset, augmented CIFAR10 dataset, and CIFAR100 dataset demonstrate the
effectiveness of the proposed CiC method.
Recognition from Hand Cameras
Authors: Cheng-Sheng Chan, Shou-Zhong Chen, Pei-Xuan Xie, Chiung-Chih Chang, Min Sun
Date: 2015-12-07 00:00:00

We revisit the study of a wrist-mounted camera system (referred to as
HandCam) for recognizing activities of hands. HandCam has two unique properties
as compared to egocentric systems (referred to as HeadCam): (1) it avoids the
need to detect hands; (2) it more consistently observes the activities of
hands. By taking advantage of these properties, we propose a
deep-learning-based method to recognize hand states (free v.s. active hands,
hand gestures, object categories), and discover object categories. Moreover, we
propose a novel two-streams deep network to further take advantage of both
HandCam and HeadCam. We have collected a new synchronized HandCam and HeadCam
dataset with 20 videos captured in three scenes for hand states recognition.
Experiments show that our HandCam system consistently outperforms a
deep-learning-based HeadCam method (with estimated manipulation regions) and a
dense-trajectory-based HeadCam method in all tasks. We also show that HandCam
videos captured by different users can be easily aligned to improve free v.s.
active recognition accuracy (3.3% improvement) in across-scenes use case.
Moreover, we observe that finetuning Convolutional Neural Network consistently
improves accuracy. Finally, our novel two-streams deep network combining
HandCam and HeadCam features achieves the best performance in four out of five
tasks. With more data, we believe a joint HandCam and HeadCam system can
robustly log hand states in daily life.
Image Super-Resolution Based on Sparsity Prior via Smoothed $l_0$ Norm
Authors: Mohammad Rostami, Zhou Wang
Date: 2016-03-22 00:00:00
Tasks: Combinatorial Optimization, compressed sensing, Image Super-Resolution, SSIM, Super-Resolution

In this paper we aim to tackle the problem of reconstructing a
high-resolution image from a single low-resolution input image, known as single
image super-resolution. In the literature, sparse representation has been used
to address this problem, where it is assumed that both low-resolution and
high-resolution images share the same sparse representation over a pair of
coupled jointly trained dictionaries. This assumption enables us to use the
compressed sensing theory to find the jointly sparse representation via the
low-resolution image and then use it to recover the high-resolution image.
However, sparse representation of a signal over a known dictionary is an
ill-posed, combinatorial optimization problem. Here we propose an algorithm
that adopts the smoothed $l_0$-norm (SL0) approach to find the jointly sparse
representation. Improved quality of the reconstructed image is obtained for
most images in terms of both peak signal-to-noise-ratio (PSNR) and structural
similarity (SSIM) measures.
Stitching Stabilizer: Two-frame-stitching Video Stabilization for Embedded Systems
Authors: Masaki Satoh
Date: 2016-03-22 00:00:00
Tasks: Video Stabilization, Vocal Bursts Valence Prediction

In conventional electronic video stabilization, the stabilized frame is
obtained by cropping the input frame to cancel camera shake. While a small
cropping size results in strong stabilization, it does not provide us
satisfactory results from the viewpoint of image quality, because it narrows
the angle of view. By fusing several frames, we can effectively expand the area
of input frames, and achieve strong stabilization even with a large cropping
size. Several methods for doing so have been studied. However, their
computational costs are too high for embedded systems such as smartphones.
  We propose a simple, yet surprisingly effective algorithm, called the
stitching stabilizer. It stitches only two frames together with a minimal
computational cost. It can achieve real-time processes in embedded systems, for
Full HD and 30 FPS videos. To clearly show the effect, we apply it to
hyperlapse. Using several clips, we show it produces more strongly stabilized
and natural results than the existing solutions from Microsoft and Instagram.
Learning Executable Semantic Parsers for Natural Language Understanding
Authors: Percy Liang
Date: 2016-03-22 00:00:00
Tasks: Natural Language Understanding, Question Answering, Semantic Parsing

For building question answering systems and natural language interfaces,
semantic parsing has emerged as an important and powerful paradigm. Semantic
parsers map natural language into logical forms, the classic representation for
many important linguistic phenomena. The modern twist is that we are interested
in learning semantic parsers from data, which introduces a new layer of
statistical and computational issues. This article lays out the components of a
statistical semantic parser, highlighting the key challenges. We will see that
semantic parsing is a rich fusion of the logical and the statistical world, and
that this fusion will play an integral role in the future of natural language
understanding systems.
Implementation of a FPGA-Based Feature Detection and Networking System for Real-time Traffic Monitoring
Authors: Jieshi Chen, Benjamin Carrion Schafer, Ivan Wang-Hei Ho
Date: 2016-03-22 00:00:00

With the growing demand of real-time traffic monitoring nowadays,
software-based image processing can hardly meet the real-time data processing
requirement due to the serial data processing nature. In this paper, the
implementation of a hardware-based feature detection and networking system
prototype for real-time traffic monitoring as well as data transmission is
presented. The hardware architecture of the proposed system is mainly composed
of three parts: data collection, feature detection, and data transmission.
Overall, the presented prototype can tolerate a high data rate of about 60
frames per second. By integrating the feature detection and data transmission
functions, the presented system can be further developed for various VANET
application scenarios to improve road safety and traffic efficiency. For
example, detection of vehicles that violate traffic rules, parking enforcement,
etc.
Information Processing by Nonlinear Phase Dynamics in Locally Connected Arrays
Authors: Richard A. Kiehl
Date: 2016-03-22 00:00:00

Research toward powerful information processing systems that circumvent the
interconnect bottleneck by exploiting the nonlinear evolution of multiple phase
dynamics in locally connected arrays is discussed. We focus on a scheme in
which logic states are defined by the electrical phase of a dynamic process and
information processing is realized through interactions between the elements in
the array. Simulation results are given for networks comprised of neuron-like
integrate-and-fire elements, which could potentially be implemented by
ultra-small tunnel junctions, molecules and other types of nanoscale elements.
This approach could lead to powerful information processing systems due to
massive parallelism in simple, highly scalable nano-architectures. The rational
for this approach, its advantages, simulation results, critical issues, and
future research directions are discussed.
Input Aggregated Network for Face Video Representation
Authors: Zhen Dong, Su Jia, Chi Zhang, Mingtao Pei
Date: 2016-03-22 00:00:00

Recently, deep neural network has shown promising performance in face image
recognition. The inputs of most networks are face images, and there is hardly
any work reported in literature on network with face videos as input. To
sufficiently discover the useful information contained in face videos, we
present a novel network architecture called input aggregated network which is
able to learn fixed-length representations for variable-length face videos. To
accomplish this goal, an aggregation unit is designed to model a face video
with various frames as a point on a Riemannian manifold, and the mapping unit
aims at mapping the point into high-dimensional space where face videos
belonging to the same subject are close-by and others are distant. These two
units together with the frame representation unit build an end-to-end learning
system which can learn representations of face videos for the specific tasks.
Experiments on two public face video datasets demonstrate the effectiveness of
the proposed network.
Information Theoretic-Learning Auto-Encoder
Authors: Eder Santana, Matthew Emigh, Jose C. Principe
Date: 2016-03-22 00:00:00

We propose Information Theoretic-Learning (ITL) divergence measures for
variational regularization of neural networks. We also explore ITL-regularized
autoencoders as an alternative to variational autoencoding bayes, adversarial
autoencoders and generative adversarial networks for randomly generating sample
data without explicitly defining a partition function. This paper also
formalizes, generative moment matching networks under the ITL framework.
Morphological Inflection Generation Using Character Sequence to Sequence Learning
Authors: Manaal Faruqui, Yulia Tsvetkov, Graham Neubig, Chris Dyer
Date: 2015-12-18 00:00:00
Proceeding: NAACL 2016 6
Tasks: Decoder, LEMMA, Morphological Inflection

Morphological inflection generation is the task of generating the inflected
form of a given lemma corresponding to a particular linguistic transformation.
We model the problem of inflection generation as a character sequence to
sequence learning problem and present a variant of the neural encoder-decoder
model for solving it. Our model is language independent and can be trained in
both supervised and semi-supervised settings. We evaluate our system on seven
datasets of morphologically rich languages and achieve either better or
comparable results to existing state-of-the-art models of inflection
generation.
Towards Automatic Wild Animal Monitoring: Identification of Animal Species in Camera-trap Images using Very Deep Convolutional Neural Networks
Authors: Alexander Gomez, Augusto Salazar, Francisco Vargas
Date: 2016-03-20 00:00:00

Non intrusive monitoring of animals in the wild is possible using camera
trapping framework, which uses cameras triggered by sensors to take a burst of
images of animals in their habitat. However camera trapping framework produces
a high volume of data (in the order on thousands or millions of images), which
must be analyzed by a human expert. In this work, a method for animal species
identification in the wild using very deep convolutional neural networks is
presented. Multiple versions of the Snapshot Serengeti dataset were used in
order to probe the ability of the method to cope with different challenges that
camera-trap images demand. The method reached 88.9% of accuracy in Top-1 and
98.1% in Top-5 in the evaluation set using a residual network topology. Also,
the results show that the proposed method outperforms previous approximations
and proves that recognition in camera-trap images can be automated.
Variational Autoencoders for Feature Detection of Magnetic Resonance Imaging Data
Authors: R. Devon Hjelm, Sergey M. Plis, Vince C. Calhoun
Date: 2016-03-21 00:00:00
Tasks: blind source separation, Dimensionality Reduction

Independent component analysis (ICA), as an approach to the blind
source-separation (BSS) problem, has become the de-facto standard in many
medical imaging settings. Despite successes and a large ongoing research
effort, the limitation of ICA to square linear transformations have not been
overcome, so that general INFOMAX is still far from being realized. As an
alternative, we present feature analysis in medical imaging as a problem solved
by Helmholtz machines, which include dimensionality reduction and
reconstruction of the raw data under the same objective, and which recently
have overcome major difficulties in inference and learning with deep and
nonlinear configurations. We demonstrate one approach to training Helmholtz
machines, variational auto-encoders (VAE), as a viable approach toward feature
extraction with magnetic resonance imaging (MRI) data.