pwc-archive/papers-with-abstracts huggingface dataset:
Independent Motion Detection with Event-driven Cameras
Authors: Valentina Vasco, Arren Glover, Elias Mueggler, Davide Scaramuzza, Lorenzo Natale, Chiara Bartolozzi
Date: 2017-06-27 00:00:00
Tasks: Motion Detection, Visual Tracking

Unlike standard cameras that send intensity images at a constant frame rate,
event-driven cameras asynchronously report pixel-level brightness changes,
offering low latency and high temporal resolution (both in the order of
micro-seconds). As such, they have great potential for fast and low power
vision algorithms for robots. Visual tracking, for example, is easily achieved
even for very fast stimuli, as only moving objects cause brightness changes.
However, cameras mounted on a moving robot are typically non-stationary and the
same tracking problem becomes confounded by background clutter events due to
the robot ego-motion. In this paper, we propose a method for segmenting the
motion of an independently moving object for event-driven cameras. Our method
detects and tracks corners in the event stream and learns the statistics of
their motion as a function of the robot's joint velocities when no
independently moving objects are present. During robot operation, independently
moving objects are identified by discrepancies between the predicted corner
velocities from ego-motion and the measured corner velocities. We validate the
algorithm on data collected from the neuromorphic iCub robot. We achieve a
precision of ~ 90 % and show that the method is robust to changes in speed of
both the head and the target.
Provable Inductive Robust PCA via Iterative Hard Thresholding
Authors: U. N. Niranjan, Arun Rajkumar, Theja Tulabandhula
Date: 2017-04-02 00:00:00

The robust PCA problem, wherein, given an input data matrix that is the
superposition of a low-rank matrix and a sparse matrix, we aim to separate out
the low-rank and sparse components, is a well-studied problem in machine
learning. One natural question that arises is that, as in the inductive
setting, if features are provided as input as well, can we hope to do better?
Answering this in the affirmative, the main goal of this paper is to study the
robust PCA problem while incorporating feature information. In contrast to
previous works in which recovery guarantees are based on the convex relaxation
of the problem, we propose a simple iterative algorithm based on
hard-thresholding of appropriate residuals. Under weaker assumptions than
previous works, we prove the global convergence of our iterative procedure;
moreover, it admits a much faster convergence rate and lesser computational
complexity per iteration. In practice, through systematic synthetic and real
data simulations, we confirm our theoretical findings regarding improvements
obtained by using feature information.
Recurrent Highway Networks
Authors: Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, Jürgen Schmidhuber
Date: 2016-07-12 00:00:00
Proceeding: ICML 2017 8
Tasks: Language Modeling, Language Modelling

Many sequential processing tasks require complex nonlinear transition
functions from one step to the next. However, recurrent neural networks with
'deep' transition functions remain difficult to train, even when using Long
Short-Term Memory (LSTM) networks. We introduce a novel theoretical analysis of
recurrent networks based on Gersgorin's circle theorem that illuminates several
modeling and optimization issues and improves our understanding of the LSTM
cell. Based on this analysis we propose Recurrent Highway Networks, which
extend the LSTM architecture to allow step-to-step transition depths larger
than one. Several language modeling experiments demonstrate that the proposed
architecture results in powerful and efficient models. On the Penn Treebank
corpus, solely increasing the transition depth from 1 to 10 improves word-level
perplexity from 90.6 to 65.4 using the same number of parameters. On the larger
Wikipedia datasets for character prediction (text8 and enwik8), RHNs outperform
all previous results and achieve an entropy of 1.27 bits per character.
CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages
Authors: Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sandra Kübler, David Yarowsky, Jason Eisner, Mans Hulden
Date: 2017-06-27 00:00:00
Proceeding: CONLL 2017 8
Tasks: Data Augmentation, Inductive Bias, LEMMA, Task 2

The CoNLL-SIGMORPHON 2017 shared task on supervised morphological generation
required systems to be trained and tested in each of 52 typologically diverse
languages. In sub-task 1, submitted systems were asked to predict a specific
inflected form of a given lemma. In sub-task 2, systems were given a lemma and
some of its specific inflected forms, and asked to complete the inflectional
paradigm by predicting all of the remaining inflected forms. Both sub-tasks
included high, medium, and low-resource conditions. Sub-task 1 received 24
system submissions, while sub-task 2 received 3 system submissions. Following
the success of neural sequence-to-sequence models in the SIGMORPHON 2016 shared
task, all but one of the submissions included a neural component. The results
show that high performance can be achieved with small training datasets, so
long as models have appropriate inductive bias or make use of additional
unlabeled data or synthetic data. However, different biasing and data
augmentation resulted in disjoint sets of inflected forms being predicted
correctly, suggesting that there is room for future improvement.
Improving Slot Filling Performance with Attentive Neural Networks on Dependency Structures
Authors: Lifu Huang, Avirup Sil, Heng Ji, Radu Florian
Date: 2017-07-04 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Relation Extraction, Sentence, slot-filling, Slot Filling

Slot Filling (SF) aims to extract the values of certain types of attributes
(or slots, such as person:cities\_of\_residence) for a given entity from a
large collection of source documents. In this paper we propose an effective DNN
architecture for SF with the following new strategies: (1). Take a regularized
dependency graph instead of a raw sentence as input to DNN, to compress the
wide contexts between query and candidate filler; (2). Incorporate two
attention mechanisms: local attention learned from query and candidate filler,
and global attention learned from external knowledge bases, to guide the model
to better select indicative contexts to determine slot type. Experiments show
that this framework outperforms state-of-the-art on both relation extraction
(16\% absolute F-score gain) and slot filling validation for each individual
system (up to 8.5\% absolute F-score gain).
Projection Free Rank-Drop Steps
Authors: Edward Cheung, Yuying Li
Date: 2017-04-13 00:00:00

The Frank-Wolfe (FW) algorithm has been widely used in solving nuclear norm
constrained problems, since it does not require projections. However, FW often
yields high rank intermediate iterates, which can be very expensive in time and
space costs for large problems. To address this issue, we propose a rank-drop
method for nuclear norm constrained problems. The goal is to generate descent
steps that lead to rank decreases, maintaining low-rank solutions throughout
the algorithm. Moreover, the optimization problems are constrained to ensure
that the rank-drop step is also feasible and can be readily incorporated into a
projection-free minimization method, e.g., Frank-Wolfe. We demonstrate that by
incorporating rank-drop steps into the Frank-Wolfe algorithm, the rank of the
solution is greatly reduced compared to the original Frank-Wolfe or its common
variants.
Structured Black Box Variational Inference for Latent Time Series Models
Authors: Robert Bamler, Stephan Mandt
Date: 2017-07-04 00:00:00
Tasks: Collaborative Filtering, Time Series, Time Series Analysis, Topic Models, Variational Inference

Continuous latent time series models are prevalent in Bayesian modeling;
examples include the Kalman filter, dynamic collaborative filtering, or dynamic
topic models. These models often benefit from structured, non mean field
variational approximations that capture correlations between time steps. Black
box variational inference with reparameterization gradients (BBVI) allows us to
explore a rich new class of Bayesian non-conjugate latent time series models;
however, a naive application of BBVI to such structured variational models
would scale quadratically in the number of time steps. We describe a BBVI
algorithm analogous to the forward-backward algorithm which instead scales
linearly in time. It allows us to efficiently sample from the variational
distribution and estimate the gradients of the ELBO. Finally, we show results
on the recently proposed dynamic word embedding model, which was trained using
our method.
Toward Computation and Memory Efficient Neural Network Acoustic Models with Binary Weights and Activations
Authors: Liang Lu
Date: 2017-06-28 00:00:00
Tasks: Efficient Neural Network, speech-recognition, Speech Recognition

Neural network acoustic models have significantly advanced state of the art
speech recognition over the past few years. However, they are usually
computationally expensive due to the large number of matrix-vector
multiplications and nonlinearity operations. Neural network models also require
significant amounts of memory for inference because of the large model size.
For these two reasons, it is challenging to deploy neural network based speech
recognizers on resource-constrained platforms such as embedded devices. This
paper investigates the use of binary weights and activations for computation
and memory efficient neural network acoustic models. Compared to real-valued
weight matrices, binary weights require much fewer bits for storage, thereby
cutting down the memory footprint. Furthermore, with binary weights or
activations, the matrix-vector multiplications are turned into addition and
subtraction operations, which are computationally much faster and more energy
efficient for hardware platforms. In this paper, we study the applications of
binary weights and activations for neural network acoustic modeling, reporting
encouraging results on the WSJ and AMI corpora.
Zero-Shot Transfer Learning for Event Extraction
Authors: Lifu Huang, Heng Ji, Kyunghyun Cho, Clare R. Voss
Date: 2017-07-04 00:00:00
Proceeding: ACL 2018 7
Tasks: Event Extraction, Transfer Learning

Most previous event extraction studies have relied heavily on features
derived from annotated event mentions, thus cannot be applied to new event
types without annotation effort. In this work, we take a fresh look at event
extraction and model it as a grounding problem. We design a transferable neural
architecture, mapping event mentions and types jointly into a shared semantic
space using structural and compositional neural networks, where the type of
each event mention can be determined by the closest of all candidate types . By
leveraging (1)~available manual annotations for a small set of existing event
types and (2)~existing event ontologies, our framework applies to new event
types without requiring additional annotation. Experiments on both existing
event types (e.g., ACE, ERE) and new event types (e.g., FrameNet) demonstrate
the effectiveness of our approach. \textit{Without any manual annotations} for
23 new event types, our zero-shot framework achieved performance comparable to
a state-of-the-art supervised model which is trained from the annotations of
500 event mentions.
High-Dimensional Materials and Process Optimization using Data-driven Experimental Design with Well-Calibrated Uncertainty Estimates
Authors: Julia Ling, Max Hutchinson, Erin Antono, Sean Paradiso, Bryce Meredig
Date: 2017-04-21 00:00:00
Tasks: Experimental Design

The optimization of composition and processing to obtain materials that
exhibit desirable characteristics has historically relied on a combination of
scientist intuition, trial and error, and luck. We propose a methodology that
can accelerate this process by fitting data-driven models to experimental data
as it is collected to suggest which experiment should be performed next. This
methodology can guide the scientist to test the most promising candidates
earlier, and can supplement scientific intuition and knowledge with data-driven
insights. A key strength of the proposed framework is that it scales to
high-dimensional parameter spaces, as are typical in materials discovery
applications. Importantly, the data-driven models incorporate uncertainty
analysis, so that new experiments are proposed based on a combination of
exploring high-uncertainty candidates and exploiting high-performing regions of
parameter space. Over four materials science test cases, our methodology led to
the optimal candidate being found with three times fewer required measurements
than random guessing on average.
Robust Optimization for Non-Convex Objectives
Authors: Robert Chen, Brendan Lucier, Yaron Singer, Vasilis Syrgkanis
Date: 2017-07-04 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Bayesian Optimization, General Classification

We consider robust optimization problems, where the goal is to optimize in
the worst case over a class of objective functions. We develop a reduction from
robust improper optimization to Bayesian optimization: given an oracle that
returns $\alpha$-approximate solutions for distributions over objectives, we
compute a distribution over solutions that is $\alpha$-approximate in the worst
case. We show that de-randomizing this solution is NP-hard in general, but can
be done for a broad class of statistical learning tasks. We apply our results
to robust neural network training and submodular optimization. We evaluate our
approach experimentally on corrupted character classification, and robust
influence maximization in networks.
How Noisy Data Affects Geometric Semantic Genetic Programming
Authors: Luis F. Miranda, Luiz Otavio V. B. Oliveira, Joao Francisco B. S. Martins, Gisele L. Pappa
Date: 2017-07-04 00:00:00

Noise is a consequence of acquiring and pre-processing data from the
environment, and shows fluctuations from different sources---e.g., from
sensors, signal processing technology or even human error. As a machine
learning technique, Genetic Programming (GP) is not immune to this problem,
which the field has frequently addressed. Recently, Geometric Semantic Genetic
Programming (GSGP), a semantic-aware branch of GP, has shown robustness and
high generalization capability. Researchers believe these characteristics may
be associated with a lower sensibility to noisy data. However, there is no
systematic study on this matter. This paper performs a deep analysis of the
GSGP performance over the presence of noise. Using 15 synthetic datasets where
noise can be controlled, we added different ratios of noise to the data and
compared the results obtained with those of a canonical GP. The results show
that, as we increase the percentage of noisy instances, the generalization
performance degradation is more pronounced in GSGP than GP. However, in
general, GSGP is more robust to noise than GP in the presence of up to 10% of
noise, and presents no statistical difference for values higher than that in
the test bed.
Kernel Scaling for Manifold Learning and Classification
Authors: Ofir Lindenbaum, Moshe Salhov, Arie Yeredor, Amir Averbuch
Date: 2017-07-04 00:00:00
Tasks: Classification, Clustering, Dimensionality Reduction, General Classification

Kernel methods play a critical role in many machine learning algorithms. They are useful in manifold learning, classification, clustering and other data analysis tasks. Setting the kernel's scale parameter, also referred to as the kernel's bandwidth, highly affects the performance of the task in hand. We propose to set a scale parameter that is tailored to one of two types of tasks: classification and manifold learning. For manifold learning, we seek a scale which is best at capturing the manifold's intrinsic dimension. For classification, we propose three methods for estimating the scale, which optimize the classification results in different senses. The proposed frameworks are simulated on artificial and on real datasets. The results show a high correlation between optimal classification rates and the estimated scales. Finally, we demonstrate the approach on a seismic event classification task.
Automatic Generation of Natural Language Explanations
Authors: Felipe Costa, Sixun Ouyang, Peter Dolog, Aonghus Lawlor
Date: 2017-07-04 00:00:00
Tasks: Negation, Recommendation Systems

An important task for recommender system is to generate explanations
according to a user's preferences. Most of the current methods for explainable
recommendations use structured sentences to provide descriptions along with the
recommendations they produce. However, those methods have neglected the
review-oriented way of writing a text, even though it is known that these
reviews have a strong influence over user's decision.
  In this paper, we propose a method for the automatic generation of natural
language explanations, for predicting how a user would write about an item,
based on user ratings from different items' features. We design a
character-level recurrent neural network (RNN) model, which generates an item's
review explanations using long-short term memories (LSTM). The model generates
text reviews given a combination of the review and ratings score that express
opinions about different factors or aspects of an item. Our network is trained
on a sub-sample from the large real-world dataset BeerAdvocate. Our empirical
evaluation using natural language processing metrics shows the generated text's
quality is close to a real user written review, identifying negation,
misspellings, and domain specific vocabulary.
Identifying Significant Predictive Bias in Classifiers
Authors: Zhe Zhang, Daniel B. Neill
Date: 2016-11-24 00:00:00

We present a novel subset scan method to detect if a probabilistic binary
classifier has statistically significant bias -- over or under predicting the
risk -- for some subgroup, and identify the characteristics of this subgroup.
This form of model checking and goodness-of-fit test provides a way to
interpretably detect the presence of classifier bias or regions of poor
classifier fit. This allows consideration of not just subgroups of a priori
interest or small dimensions, but the space of all possible subgroups of
features. To address the difficulty of considering these exponentially many
possible subgroups, we use subset scan and parametric bootstrap-based methods.
Extending this method, we can penalize the complexity of the detected subgroup
and also identify subgroups with high classification errors. We demonstrate
these methods and find interesting results on the COMPAS crime recidivism and
credit delinquency data.
An empirical study on the effectiveness of images in Multimodal Neural Machine Translation
Authors: Jean-Benoit Delbrouck, Stéphane Dupont
Date: 2017-07-04 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Decoder, Machine Translation, NMT, Sentence, Translation

In state-of-the-art Neural Machine Translation (NMT), an attention mechanism
is used during decoding to enhance the translation. At every step, the decoder
uses this mechanism to focus on different parts of the source sentence to
gather the most useful information before outputting its target word. Recently,
the effectiveness of the attention mechanism has also been explored for
multimodal tasks, where it becomes possible to focus both on sentence parts and
image regions that they describe. In this paper, we compare several attention
mechanism on the multimodal translation task (English, image to German) and
evaluate the ability of the model to make use of images to improve translation.
We surpass state-of-the-art scores on the Multi30k data set, we nevertheless
identify and report different misbehavior of the machine while translating.
Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning
Authors: Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, Susanne Saminger-Platz
Date: 2017-02-28 00:00:00
Tasks: Domain Adaptation, Object Recognition, Representation Learning, Sentiment Analysis

The learning of domain-invariant representations in the context of domain
adaptation with neural networks is considered. We propose a new regularization
method that minimizes the discrepancy between domain-specific latent feature
representations directly in the hidden activation space. Although some standard
distribution matching approaches exist that can be interpreted as the matching
of weighted sums of moments, e.g. Maximum Mean Discrepancy (MMD), an explicit
order-wise matching of higher order moments has not been considered before. We
propose to match the higher order central moments of probability distributions
by means of order-wise moment differences. Our model does not require
computationally expensive distance and kernel matrix computations. We utilize
the equivalent representation of probability distributions by moment sequences
to define a new distance function, called Central Moment Discrepancy (CMD). We
prove that CMD is a metric on the set of probability distributions on a compact
interval. We further prove that convergence of probability distributions on
compact intervals w.r.t. the new metric implies convergence in distribution of
the respective random variables. We test our approach on two different
benchmark data sets for object recognition (Office) and sentiment analysis of
product reviews (Amazon reviews). CMD achieves a new state-of-the-art
performance on most domain adaptation tasks of Office and outperforms networks
trained with MMD, Variational Fair Autoencoders and Domain Adversarial Neural
Networks on Amazon reviews. In addition, a post-hoc parameter sensitivity
analysis shows that the new approach is stable w.r.t. parameter changes in a
certain interval. The source code of the experiments is publicly available.
Anomaly Detection and Modeling in 802.11 Wireless Networks
Authors: Anisa Allahdadi, Ricardo Morla
Date: 2017-07-04 00:00:00
Tasks: Anomaly Detection, Management

IEEE 802.11 Wireless Networks are getting more and more popular at university
campuses, enterprises, shopping centers, airports and in so many other public
places, providing Internet access to a large crowd openly and quickly. The
wireless users are also getting more dependent on WiFi technology and therefore
demanding more reliability and higher performance for this vital technology.
However, due to unstable radio conditions, faulty equipment, and dynamic user
behavior among other reasons, there are always unpredictable performance
problems in a wireless covered area. Detection and prediction of such problems
is of great significance to network managers if they are to alleviate the
connectivity issues of the mobile users and provide a higher quality wireless
service. This paper aims to improve the management of the 802.11 wireless
networks by characterizing and modeling wireless usage patterns in a set of
anomalous scenarios that can occur in such networks. We apply time-invariant
(Gaussian Mixture Models) and time-variant (Hidden Markov Models) modeling
approaches to a dataset generated from a large production network and describe
how we use these models for anomaly detection. We then generate several common
anomalies on a Testbed network and evaluate the proposed anomaly detection
methodologies in a controlled environment. The experimental results of the
Testbed show that HMM outperforms GMM and yields a higher anomaly detection
ratio and a lower false alarm rate.