pwc-archive/papers-with-abstracts huggingface dataset:
A Comparative Study of Object Trackers for Infrared Flying Bird Tracking
Authors: Ying Huang, Hong Zheng, Haibin Ling, Erik Blasch, Hao Yang
Date: 2016-01-18 00:00:00

Bird strikes present a huge risk for aircraft, especially since traditional
airport bird surveillance is mainly dependent on inefficient human observation.
Computer vision based technology has been proposed to automatically detect
birds, determine bird flying trajectories, and predict aircraft takeoff delays.
However, the characteristics of bird flight using imagery and the performance
of existing methods applied to flying bird task are not well known. Therefore,
we perform infrared flying bird tracking experiments using 12 state-of-the-art
algorithms on a real BIRDSITE-IR dataset to obtain useful clues and recommend
feature analysis. We also develop a Struck-scale method to demonstrate the
effectiveness of multiple scale sampling adaption in handling the object of
flying bird with varying shape and scale. The general analysis can be used to
develop specialized bird tracking methods for airport safety, wildness and
urban bird population studies.
Information Extraction Under Privacy Constraints
Authors: Shahab Asoodeh, Mario Diaz, Fady Alajaji, Tamás Linder
Date: 2015-11-07 00:00:00
Tasks: Quantization

A privacy-constrained information extraction problem is considered where for
a pair of correlated discrete random variables $(X,Y)$ governed by a given
joint distribution, an agent observes $Y$ and wants to convey to a potentially
public user as much information about $Y$ as possible without compromising the
amount of information revealed about $X$. To this end, the so-called {\em
rate-privacy function} is introduced to quantify the maximal amount of
information (measured in terms of mutual information) that can be extracted
from $Y$ under a privacy constraint between $X$ and the extracted information,
where privacy is measured using either mutual information or maximal
correlation. Properties of the rate-privacy function are analyzed and
information-theoretic and estimation-theoretic interpretations of it are
presented for both the mutual information and maximal correlation privacy
measures. It is also shown that the rate-privacy function admits a closed-form
expression for a large family of joint distributions of $(X,Y)$. Finally, the
rate-privacy function under the mutual information privacy measure is
considered for the case where $(X,Y)$ has a joint probability density function
by studying the problem where the extracted information is a uniform
quantization of $Y$ corrupted by additive Gaussian noise. The asymptotic
behavior of the rate-privacy function is studied as the quantization resolution
grows without bound and it is observed that not all of the properties of the
rate-privacy function carry over from the discrete to the continuous case.
A Novel Regularized Principal Graph Learning Framework on Explicit Graph Representation
Authors: Qi Mao, Li Wang, Ivor W. Tsang, Yijun Sun
Date: 2015-12-09 00:00:00
Tasks: Graph Embedding, Graph Learning

Many scientific datasets are of high dimension, and the analysis usually
requires visual manipulation by retaining the most important structures of
data. Principal curve is a widely used approach for this purpose. However, many
existing methods work only for data with structures that are not
self-intersected, which is quite restrictive for real applications. A few
methods can overcome the above problem, but they either require complicated
human-made rules for a specific task with lack of convergence guarantee and
adaption flexibility to different tasks, or cannot obtain explicit structures
of data. To address these issues, we develop a new regularized principal graph
learning framework that captures the local information of the underlying graph
structure based on reversed graph embedding. As showcases, models that can
learn a spanning tree or a weighted undirected $\ell_1$ graph are proposed, and
a new learning algorithm is developed that learns a set of principal points and
a graph structure from data, simultaneously. The new algorithm is simple with
guaranteed convergence. We then extend the proposed framework to deal with
large-scale data. Experimental results on various synthetic and six real world
datasets show that the proposed method compares favorably with baselines and
can uncover the underlying structure correctly.
Building a Learning Database for the Neural Network Retrieval of Sea Surface Salinity from SMOS Brightness Temperatures
Authors: Adel Ammar, Sylvie Labroue, Estelle Obligis, Michel Crépon, Sylvie Thiria
Date: 2016-01-17 00:00:00
Tasks: Retrieval

This article deals with an important aspect of the neural network retrieval
of sea surface salinity (SSS) from SMOS brightness temperatures (TBs). The
neural network retrieval method is an empirical approach that offers the
possibility of being independent from any theoretical emissivity model, during
the in-flight phase. A Previous study [1] has proven that this approach is
applicable to all pixels on ocean, by designing a set of neural networks with
different inputs. The present study focuses on the choice of the learning
database and demonstrates that a judicious distribution of the geophysical
parameters allows to markedly reduce the systematic regional biases of the
retrieved SSS, which are due to the high noise on the TBs. An equalization of
the distribution of the geophysical parameters, followed by a new technique for
boosting the learning process, makes the regional biases almost disappear for
latitudes between 40{\deg}S and 40{\deg}N, while the global standard deviation
remains between 0.6 psu (at the center of the of the swath) and 1 psu (at the
edges).
Face-space Action Recognition by Face-Object Interactions
Authors: Amir Rosenfeld, Shimon Ullman
Date: 2016-01-17 00:00:00
Tasks: Action Recognition, Action Recognition In Still Images, Object, Object Recognition, Pose Estimation, Temporal Action Localization

Action recognition in still images has seen major improvement in recent years
due to advances in human pose estimation, object recognition and stronger
feature representations. However, there are still many cases in which
performance remains far from that of humans. In this paper, we approach the
problem by learning explicitly, and then integrating three components of
transitive actions: (1) the human body part relevant to the action (2) the
object being acted upon and (3) the specific form of interaction between the
person and the object. The process uses class-specific features and relations
not used in the past for action recognition and which use inherently two cycles
in the process unlike most standard approaches. We focus on face-related
actions (FRA), a subset of actions that includes several currently challenging
categories. We present an average relative improvement of 52% over state-of-the
art. We also make a new benchmark publicly available.
On-line Bayesian System Identification
Authors: Diego Romeres, Giulia Prando, Gianluigi Pillonetto, Alessandro Chiuso
Date: 2016-01-17 00:00:00

We consider an on-line system identification setting, in which new data
become available at given time steps. In order to meet real-time estimation
requirements, we propose a tailored Bayesian system identification procedure,
in which the hyper-parameters are still updated through Marginal Likelihood
maximization, but after only one iteration of a suitable iterative optimization
algorithm. Both gradient methods and the EM algorithm are considered for the
Marginal Likelihood optimization. We compare this "1-step" procedure with the
standard one, in which the optimization method is run until convergence to a
local minimum. The experiments we perform confirm the effectiveness of the
approach we propose.
Deep Recurrent Neural Networks for Sequential Phenotype Prediction in Genomics
Authors: Farhad Pouladi, Hojjat Salehinejad, Amir Mohammad Gilani
Date: 2015-11-09 00:00:00
Tasks: Imputation

In analyzing of modern biological data, we are often dealing with ill-posed
problems and missing data, mostly due to high dimensionality and
multicollinearity of the dataset. In this paper, we have proposed a system
based on matrix factorization (MF) and deep recurrent neural networks (DRNNs)
for genotype imputation and phenotype sequences prediction. In order to model
the long-term dependencies of phenotype data, the new Recurrent Linear Units
(ReLU) learning strategy is utilized for the first time. The proposed model is
implemented for parallel processing on central processing units (CPUs) and
graphic processing units (GPUs). Performance of the proposed model is compared
with other training algorithms for learning long-term dependencies as well as
the sparse partial least square (SPLS) method on a set of genotype and
phenotype data with 604 samples, 1980 single-nucleotide polymorphisms (SNPs),
and two traits. The results demonstrate performance of the ReLU training
algorithm in learning long-term dependencies in RNNs.
Unsupervised Visual Representation Learning by Context Prediction
Authors: Carl Doersch, Abhinav Gupta, Alexei A. Efros
Date: 2015-05-19 00:00:00
Proceeding: ICCV 2015 12
Tasks: Prediction, Representation Learning

This work explores the use of spatial context as a source of free and
plentiful supervisory signal for training a rich visual representation. Given
only a large, unlabeled image collection, we extract random pairs of patches
from each image and train a convolutional neural net to predict the position of
the second patch relative to the first. We argue that doing well on this task
requires the model to learn to recognize objects and their parts. We
demonstrate that the feature representation learned using this within-image
context indeed captures visual similarity across images. For example, this
representation allows us to perform unsupervised visual discovery of objects
like cats, people, and even birds from the Pascal VOC 2011 detection dataset.
Furthermore, we show that the learned ConvNet can be used in the R-CNN
framework and provides a significant boost over a randomly-initialized ConvNet,
resulting in state-of-the-art performance among algorithms which use only
Pascal-provided training set annotations.
Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
Authors: Uri Shaham, Yutaro Yamada, Sahand Negahban
Date: 2015-11-17 00:00:00

We propose a general framework for increasing local stability of Artificial
Neural Nets (ANNs) using Robust Optimization (RO). We achieve this through an
alternating minimization-maximization procedure, in which the loss of the
network is minimized over perturbed examples that are generated at each
parameter update. We show that adversarial training of ANNs is in fact
robustification of the network optimization, and that our proposed framework
generalizes previous approaches for increasing local stability of ANNs.
Experimental results reveal that our approach increases the robustness of the
network to existing adversarial examples, while making it harder to generate
new ones. Furthermore, our algorithm improves the accuracy of the network also
on the original test data.
Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware
Authors: Peter U. Diehl, Guido Zarrella, Andrew Cassidy, Bruno U. Pedroni, Emre Neftci
Date: 2016-01-16 00:00:00
Tasks: General Classification, Representation Learning, Temporal Sequences

In recent years the field of neuromorphic low-power systems that consume
orders of magnitude less power gained significant momentum. However, their
wider use is still hindered by the lack of algorithms that can harness the
strengths of such architectures. While neuromorphic adaptations of
representation learning algorithms are now emerging, efficient processing of
temporal sequences or variable length-inputs remain difficult. Recurrent neural
networks (RNN) are widely used in machine learning to solve a variety of
sequence learning tasks. In this work we present a train-and-constrain
methodology that enables the mapping of machine learned (Elman) RNNs on a
substrate of spiking neurons, while being compatible with the capabilities of
current and near-future neuromorphic systems. This "train-and-constrain" method
consists of first training RNNs using backpropagation through time, then
discretizing the weights and finally converting them to spiking RNNs by
matching the responses of artificial neurons with those of the spiking neurons.
We demonstrate our approach by mapping a natural language processing task
(question classification), where we demonstrate the entire mapping process of
the recurrent layer of the network on IBM's Neurosynaptic System "TrueNorth", a
spike-based digital neuromorphic hardware architecture. TrueNorth imposes
specific constraints on connectivity, neural and synaptic parameters. To
satisfy these constraints, it was necessary to discretize the synaptic weights
and neural activities to 16 levels, and to limit fan-in to 64 inputs. We find
that short synaptic delays are sufficient to implement the dynamical (temporal)
aspect of the RNN in the question classification task. The hardware-constrained
model achieved 74% accuracy in question classification while using less than
0.025% of the cores on one TrueNorth chip, resulting in an estimated power
consumption of ~17 uW.
TrueHappiness: Neuromorphic Emotion Recognition on TrueNorth
Authors: Peter U. Diehl, Bruno U. Pedroni, Andrew Cassidy, Paul Merolla, Emre Neftci, Guido Zarrella
Date: 2016-01-16 00:00:00
Tasks: Emotion Recognition, Sentiment Analysis

We present an approach to constructing a neuromorphic device that responds to
language input by producing neuron spikes in proportion to the strength of the
appropriate positive or negative emotional response. Specifically, we perform a
fine-grained sentiment analysis task with implementations on two different
systems: one using conventional spiking neural network (SNN) simulators and the
other one using IBM's Neurosynaptic System TrueNorth. Input words are projected
into a high-dimensional semantic space and processed through a fully-connected
neural network (FCNN) containing rectified linear units trained via
backpropagation. After training, this FCNN is converted to a SNN by
substituting the ReLUs with integrate-and-fire neurons. We show that there is
practically no performance loss due to conversion to a spiking network on a
sentiment analysis test set, i.e. correlations between predictions and human
annotations differ by less than 0.02 comparing the original DNN and its spiking
equivalent. Additionally, we show that the SNN generated with this technique
can be mapped to existing neuromorphic hardware -- in our case, the TrueNorth
chip. Mapping to the chip involves 4-bit synaptic weight discretization and
adjustment of the neuron thresholds. The resulting end-to-end system can take a
user input, i.e. a word in a vocabulary of over 300,000 words, and estimate its
sentiment on TrueNorth with a power consumption of approximately 50 uW.
Regularized Orthogonal Tensor Decompositions for Multi-Relational Learning
Authors: Fanhua Shang, James Cheng, Hong Cheng
Date: 2015-12-26 00:00:00
Tasks: Relational Reasoning

Multi-relational learning has received lots of attention from researchers in
various research communities. Most existing methods either suffer from
superlinear per-iteration cost, or are sensitive to the given ranks. To address
both issues, we propose a scalable core tensor trace norm Regularized
Orthogonal Iteration Decomposition (ROID) method for full or incomplete tensor
analytics, which can be generalized as a graph Laplacian regularized version by
using auxiliary information or a sparse higher-order orthogonal iteration
(SHOOI) version. We first induce the equivalence relation of the Schatten
p-norm (0<p<\infty) of a low multi-linear rank tensor and its core tensor. Then
we achieve a much smaller matrix trace norm minimization problem. Finally, we
develop two efficient augmented Lagrange multiplier algorithms to solve our
problems with convergence guarantees. Extensive experiments using both real and
synthetic datasets, even though with only a few observations, verified both the
efficiency and effectiveness of our methods.
Regret bounds for Narendra-Shapiro bandit algorithms
Authors: Sébastien Gadat, Fabien Panloup, Sofiane Saadane
Date: 2015-02-17 00:00:00

Narendra-Shapiro (NS) algorithms are bandit-type algorithms that have been
introduced in the sixties (with a view to applications in Psychology or
learning automata), whose convergence has been intensively studied in the
stochastic algorithm literature. In this paper, we adress the following
question: are the Narendra-Shapiro (NS) bandit algorithms competitive from a
\textit{regret} point of view? In our main result, we show that some
competitive bounds can be obtained for such algorithms in their penalized
version (introduced in \cite{Lamberton_Pages}). More precisely, up to an
over-penalization modification, the pseudo-regret $\bar{R}_n$ related to the
penalized two-armed bandit algorithm is uniformly bounded by $C \sqrt{n}$
(where $C$ is made explicit in the paper). \noindent We also generalize
existing convergence and rates of convergence results to the multi-armed case
of the over-penalized bandit algorithm, including the convergence toward the
invariant measure of a Piecewise Deterministic Markov Process (PDMP) after a
suitable renormalization. Finally, ergodic properties of this PDMP are given in
the multi-armed case.
Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval
Authors: Hamid Palangi, Li Deng, Yelong Shen, Jianfeng Gao, Xiaodong He, Jianshu Chen, Xinying Song, Rabab Ward
Date: 2015-02-24 00:00:00
Tasks: Information Retrieval, Retrieval, Sentence, Sentence Embedding, Sentence-Embedding

This paper develops a model that addresses sentence embedding, a hot topic in
current natural language processing research, using recurrent neural networks
with Long Short-Term Memory (LSTM) cells. Due to its ability to capture long
term memory, the LSTM-RNN accumulates increasingly richer information as it
goes through the sentence, and when it reaches the last word, the hidden layer
of the network provides a semantic representation of the whole sentence. In
this paper, the LSTM-RNN is trained in a weakly supervised manner on user
click-through data logged by a commercial web search engine. Visualization and
analysis are performed to understand how the embedding process works. The model
is found to automatically attenuate the unimportant words and detects the
salient keywords in the sentence. Furthermore, these detected keywords are
found to automatically activate different cells of the LSTM-RNN, where words
belonging to a similar topic activate the same cell. As a semantic
representation of the sentence, the embedding vector can be used in many
different applications. These automatic keyword detection and topic allocation
abilities enabled by the LSTM-RNN allow the network to perform document
retrieval, a difficult language processing task, where the similarity between
the query and documents can be measured by the distance between their
corresponding sentence embedding vectors computed by the LSTM-RNN. On a web
search task, the LSTM-RNN embedding is shown to significantly outperform
several existing state of the art methods. We emphasize that the proposed model
generates sentence embedding vectors that are specially useful for web document
retrieval tasks. A comparison with a well known general sentence embedding
method, the Paragraph Vector, is performed. The results show that the proposed
method in this paper significantly outperforms it for web document retrieval
task.
Engineering Safety in Machine Learning
Authors: Kush R. Varshney
Date: 2016-01-16 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning algorithms are increasingly influencing our decisions and
interacting with us in all parts of our daily lives. Therefore, just like for
power plants, highways, and myriad other engineered sociotechnical systems, we
must consider the safety of systems involving machine learning. In this paper,
we first discuss the definition of safety in terms of risk, epistemic
uncertainty, and the harm incurred by unwanted outcomes. Then we examine
dimensions, such as the choice of cost function and the appropriateness of
minimizing the empirical average training cost, along which certain real-world
applications may not be completely amenable to the foundational principle of
modern statistical machine learning: empirical risk minimization. In
particular, we note an emerging dichotomy of applications: ones in which safety
is important and risk minimization is not the complete story (we name these
Type A applications), and ones in which safety is not so critical and risk
minimization is sufficient (we name these Type B applications). Finally, we
discuss how four different strategies for achieving safety in engineering
(inherently safe design, safety reserves, safe fail, and procedural safeguards)
can be mapped to the machine learning context through interpretability and
causality of predictive models, objectives beyond expected prediction accuracy,
human involvement for labeling difficult or rare examples, and user experience
design of software.
Learning with a Strong Adversary
Authors: Ruitong Huang, Bing Xu, Dale Schuurmans, Csaba Szepesvari
Date: 2015-11-10 00:00:00
Tasks: General Classification

The robustness of neural networks to intended perturbations has recently
attracted significant attention. In this paper, we propose a new method,
\emph{learning with a strong adversary}, that learns robust classifiers from
supervised data. The proposed method takes finding adversarial examples as an
intermediate step. A new and simple way of finding adversarial examples is
presented and experimentally shown to be efficient. Experimental results
demonstrate that resulting learning method greatly improves the robustness of
the classification models produced.
Learning the Semantics of Structured Data Sources
Authors: Mohsen Taheriyan, Craig A. Knoblock, Pedro Szekely, Jose Luis Ambite
Date: 2016-01-16 00:00:00
Tasks: Knowledge Graphs

Information sources such as relational databases, spreadsheets, XML, JSON,
and Web APIs contain a tremendous amount of structured data that can be
leveraged to build and augment knowledge graphs. However, they rarely provide a
semantic model to describe their contents. Semantic models of data sources
represent the implicit meaning of the data by specifying the concepts and the
relationships within the data. Such models are the key ingredients to
automatically publish the data into knowledge graphs. Manually modeling the
semantics of data sources requires significant effort and expertise, and
although desirable, building these models automatically is a challenging
problem. Most of the related work focuses on semantic annotation of the data
fields (source attributes). However, constructing a semantic model that
explicitly describes the relationships between the attributes in addition to
their semantic types is critical.
  We present a novel approach that exploits the knowledge from a domain
ontology and the semantic models of previously modeled sources to automatically
learn a rich semantic model for a new source. This model represents the
semantics of the new source in terms of the concepts and relationships defined
by the domain ontology. Given some sample data from the new source, we leverage
the knowledge in the domain ontology and the known semantic models to construct
a weighted graph that represents the space of plausible semantic models for the
new source. Then, we compute the top k candidate semantic models and suggest to
the user a ranked list of the semantic models for the new source. The approach
takes into account user corrections to learn more accurate semantic models on
future data sources. Our evaluation shows that our method generates expressive
semantic models for data sources and services with minimal user input. ...