pwc-archive/papers-with-abstracts huggingface dataset:
HDRFusion: HDR SLAM using a low-cost auto-exposure RGB-D sensor
Authors: Shuda Li, Ankur Handa, Yang Zhang, Andrew Calway
Date: 2016-04-04 00:00:00

We describe a new method for comparing frame appearance in a frame-to-model
3-D mapping and tracking system using an low dynamic range (LDR) RGB-D camera
which is robust to brightness changes caused by auto exposure. It is based on a
normalised radiance measure which is invariant to exposure changes and not only
robustifies the tracking under changing lighting conditions, but also enables
the following exposure compensation perform accurately to allow online building
of high dynamic range (HDR) maps. The latter facilitates the frame-to-model
tracking to minimise drift as well as better capturing light variation within
the scene. Results from experiments with synthetic and real data demonstrate
that the method provides both improved tracking and maps with far greater
dynamic range of luminosity.
Comparing Convolutional Neural Networks to Traditional Models for Slot Filling
Authors: Heike Adel, Benjamin Roth, Hinrich Schütze
Date: 2016-03-16 00:00:00
Proceeding: NAACL 2016 6
Tasks: Classification, General Classification, Relation, Relation Classification, Sentence, slot-filling, Slot Filling

We address relation classification in the context of slot filling, the task
of finding and evaluating fillers like "Steve Jobs" for the slot X in "X
founded Apple". We propose a convolutional neural network which splits the
input sentence into three parts according to the relation arguments and compare
it to state-of-the-art and traditional approaches of relation classification.
Finally, we combine different methods and show that the combination is better
than individual approaches. We also analyze the effect of genre differences on
performance.
Automatic Knowledge Base Evolution by Learning Instances
Authors: Sundong Kim
Date: 2016-04-04 00:00:00

Knowledge base is the way to store structured and unstructured data
throughout the web. Since the size of the web is increasing rapidly, there are
huge needs to structure the knowledge in a fully automated way. However
fully-automated knowledge-base evolution on the Semantic Web is a major
challenges, although there are many ontology evolution techniques available.
Therefore learning ontology automatically can contribute to the semantic web
society significantly. In this paper, we propose full-automated ontology
learning algorithm to generate refined knowledge base from incomplete knowledge
base and rdf-triples. Our algorithm is data-driven approach which is based on
the property of each instance. Ontology class is being elaborated by
generalizing frequent property of its instances. By using that developed class
information, each instance can find its most relatively matching class. By
repeating these two steps, we achieve fully-automated ontology evolution from
incomplete basic knowledge base.
Recurrent Neural Networks for Polyphonic Sound Event Detection in Real Life Recordings
Authors: Giambattista Parascandolo, Heikki Huttunen, Tuomas Virtanen
Date: 2016-04-04 00:00:00
Tasks: Data Augmentation, Event Detection, Sound Event Detection

In this paper we present an approach to polyphonic sound event detection in
real life recordings based on bi-directional long short term memory (BLSTM)
recurrent neural networks (RNNs). A single multilabel BLSTM RNN is trained to
map acoustic features of a mixture signal consisting of sounds from multiple
classes, to binary activity indicators of each event class. Our method is
tested on a large database of real-life recordings, with 61 classes (e.g.
music, car, speech) from 10 different everyday contexts. The proposed method
outperforms previous approaches by a large margin, and the results are further
improved using data augmentation techniques. Overall, our system reports an
average F1-score of 65.5% on 1 second blocks and 64.7% on single frames, a
relative improvement over previous state-of-the-art approach of 6.8% and 15.1%
respectively.
Lipschitz Continuity of Mahalanobis Distances and Bilinear Forms
Authors: Valentina Zantedeschi, Rémi Emonet, Marc Sebban
Date: 2016-04-04 00:00:00
Tasks: BIG-bench Machine Learning

Many theoretical results in the machine learning domain stand only for
functions that are Lipschitz continuous. Lipschitz continuity is a strong form
of continuity that linearly bounds the variations of a function. In this paper,
we derive tight Lipschitz constants for two families of metrics: Mahalanobis
distances and bounded-space bilinear forms. To our knowledge, this is the first
time the Mahalanobis distance is formally proved to be Lipschitz continuous and
that such tight Lipschitz constants are derived.
Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers
Authors: Alexander Binder, Grégoire Montavon, Sebastian Bach, Klaus-Robert Müller, Wojciech Samek
Date: 2016-04-04 00:00:00

Layer-wise relevance propagation is a framework which allows to decompose the
prediction of a deep neural network computed over a sample, e.g. an image, down
to relevance scores for the single input dimensions of the sample such as
subpixels of an image. While this approach can be applied directly to
generalized linear mappings, product type non-linearities are not covered. This
paper proposes an approach to extend layer-wise relevance propagation to neural
networks with local renormalization layers, which is a very common product-type
non-linearity in convolutional neural networks. We evaluate the proposed method
for local renormalization layers on the CIFAR-10, Imagenet and MIT Places
datasets.
Deep Reinforcement Learning in Large Discrete Action Spaces
Authors: Gabriel Dulac-Arnold, Richard Evans, Hado van Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, Theophane Weber, Thomas Degris, Ben Coppin
Date: 2015-12-24 00:00:00
Tasks: Deep Reinforcement Learning, Recommendation Systems, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Being able to reason in an environment with a large number of discrete
actions is essential to bringing reinforcement learning to a larger class of
problems. Recommender systems, industrial plants and language models are only
some of the many real-world tasks involving large numbers of discrete actions
for which current methods are difficult or even often impossible to apply. An
ability to generalize over the set of actions as well as sub-linear complexity
relative to the size of the set are both necessary to handle such tasks.
Current approaches are not able to provide both of these, which motivates the
work in this paper. Our proposed approach leverages prior information about the
actions to embed them in a continuous space upon which it can generalize.
Additionally, approximate nearest-neighbor methods allow for logarithmic-time
lookup complexity relative to the number of actions, which is necessary for
time-wise tractable training. This combined approach allows reinforcement
learning methods to be applied to large-scale learning problems previously
intractable with current methods. We demonstrate our algorithm's abilities on a
series of tasks having up to one million actions.
Conformant Planning as a Case Study of Incremental QBF Solving
Authors: Uwe Egly, Martin Kronegger, Florian Lonsing, Andreas Pfandler
Date: 2014-05-28 00:00:00

We consider planning with uncertainty in the initial state as a case study of
incremental quantified Boolean formula (QBF) solving. We report on experiments
with a workflow to incrementally encode a planning instance into a sequence of
QBFs. To solve this sequence of incrementally constructed QBFs, we use our
general-purpose incremental QBF solver DepQBF. Since the generated QBFs have
many clauses and variables in common, our approach avoids redundancy both in
the encoding phase and in the solving phase. Experimental results show that
incremental QBF solving outperforms non-incremental QBF solving. Our results
are the first empirical study of incremental QBF solving in the context of
planning and motivate its use in other application domains.
Extending DLR with Labelled Tuples, Projections, Functional Dependencies and Objectification (full version)
Authors: Alessandro Artale, Enrico Franconi
Date: 2016-04-04 00:00:00
Tasks: Attribute

We introduce an extension of the n-ary description logic DLR to deal with
attribute-labelled tuples (generalising the positional notation), with
arbitrary projections of relations (inclusion dependencies), generic functional
dependencies and with global and local objectification (reifying relations or
their projections). We show how a simple syntactic condition on the appearance
of projections and functional dependencies in a knowledge base makes the
language decidable without increasing the computational complexity of the basic
DLR language.
Topic Model Based Multi-Label Classification from the Crowd
Authors: Divya Padmanabhan, Satyanath Bhat, Shirish Shevade, Y. Narahari
Date: 2016-04-04 00:00:00
Tasks: Classification, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Multi-label classification is a common supervised machine learning problem
where each instance is associated with multiple classes. The key challenge in
this problem is learning the correlations between the classes. An additional
challenge arises when the labels of the training instances are provided by
noisy, heterogeneous crowdworkers with unknown qualities. We first assume
labels from a perfect source and propose a novel topic model where the present
as well as the absent classes generate the latent topics and hence the words.
We non-trivially extend our topic model to the scenario where the labels are
provided by noisy crowdworkers. Extensive experimentation on real world
datasets reveals the superior performance of the proposed model. The proposed
model learns the qualities of the annotators as well, even with minimal
training data.
Controlling Explanatory Heatmap Resolution and Semantics via Decomposition Depth
Authors: Sebastian Bach, Alexander Binder, Klaus-Robert Müller, Wojciech Samek
Date: 2016-03-21 00:00:00
Tasks: Prediction

We present an application of the Layer-wise Relevance Propagation (LRP)
algorithm to state of the art deep convolutional neural networks and Fisher
Vector classifiers to compare the image perception and prediction strategies of
both classifiers with the use of visualized heatmaps. Layer-wise Relevance
Propagation (LRP) is a method to compute scores for individual components of an
input image, denoting their contribution to the prediction of the classifier
for one particular test point. We demonstrate the impact of different choices
of decomposition cut-off points during the LRP-process, controlling the
resolution and semantics of the heatmap on test images from the PASCAL VOC 2007
test data set.
The CMA Evolution Strategy: A Tutorial
Authors: Nikolaus Hansen
Date: 2016-04-04 00:00:00

This tutorial introduces the CMA Evolution Strategy (ES), where CMA stands for Covariance Matrix Adaptation. The CMA-ES is a stochastic, or randomized, method for real-parameter (continuous domain) optimization of non-linear, non-convex functions. We try to motivate and derive the algorithm from intuitive concepts and from requirements of non-linear, non-convex search in continuous domain.
Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks
Authors: Matthew Francis-Landau, Greg Durrett, Dan Klein
Date: 2016-04-04 00:00:00
Proceeding: NAACL 2016 6
Tasks: Entity Linking, Semantic correspondence, Semantic Similarity, Semantic Textual Similarity

A key challenge in entity linking is making effective use of contextual
information to disambiguate mentions that might refer to different entities in
different contexts. We present a model that uses convolutional neural networks
to capture semantic correspondence between a mention's context and a proposed
target entity. These convolutional networks operate at multiple granularities
to exploit various kinds of topic information, and their rich parameterization
gives them the capacity to learn which n-grams characterize different topics.
We combine these networks with a sparse linear model to achieve
state-of-the-art performance on multiple entity linking datasets, outperforming
the prior systems of Durrett and Klein (2014) and Nguyen et al. (2014).
Waterdrop Stereo
Authors: Shaodi You, Robby T. Tan, Rei Kawakami, Yasuhiro Mukaigawa, Katsushi Ikeuchi
Date: 2016-04-04 00:00:00
Tasks: Depth Estimation

This paper introduces depth estimation from water drops. The key idea is that
a single water drop adhered to window glass is totally transparent and convex,
and thus optically acts like a fisheye lens. If we have more than one water
drop in a single image, then through each of them we can see the environment
with different view points, similar to stereo. To realize this idea, we need to
rectify every water drop imagery to make radially distorted planar surfaces
look flat. For this rectification, we consider two physical properties of water
drops: (1) A static water drop has constant volume, and its geometric convex
shape is determined by the balance between the tension force and gravity. This
implies that the 3D geometric shape can be obtained by minimizing the overall
potential energy, which is the sum of the tension energy and the gravitational
potential energy. (2) The imagery inside a water-drop is determined by the
water-drop 3D shape and total reflection at the boundary. This total reflection
generates a dark band commonly observed in any adherent water drops. Hence,
once the 3D shape of water drops are recovered, we can rectify the water drop
images through backward raytracing. Subsequently, we can compute depth using
stereo. In addition to depth estimation, we can also apply image refocusing.
Experiments on real images and a quantitative evaluation show the effectiveness
of our proposed method. To our best knowledge, never before have adherent water
drops been used to estimate depth.
Text Understanding from Scratch
Authors: Xiang Zhang, Yann Lecun
Date: 2015-02-05 00:00:00
Tasks: General Classification, Sentiment Analysis

This article demontrates that we can apply deep learning to text
understanding from character-level inputs all the way up to abstract text
concepts, using temporal convolutional networks (ConvNets). We apply ConvNets
to various large-scale datasets, including ontology classification, sentiment
analysis, and text categorization. We show that temporal ConvNets can achieve
astonishing performance without the knowledge of words, phrases, sentences and
any other syntactic or semantic structures with regards to a human language.
Evidence shows that our models can work for both English and Chinese.
Character-level Convolutional Networks for Text Classification
Authors: Xiang Zhang, Junbo Zhao, Yann Lecun
Date: 2015-09-04 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: General Classification, Sentiment Analysis, Text Classification

This article offers an empirical exploration on the use of character-level
convolutional networks (ConvNets) for text classification. We constructed
several large-scale datasets to show that character-level convolutional
networks could achieve state-of-the-art or competitive results. Comparisons are
offered against traditional models such as bag of words, n-grams and their
TFIDF variants, and deep learning models such as word-based ConvNets and
recurrent neural networks.
Semi-supervised Question Retrieval with Gated Convolutions
Authors: Tao Lei, Hrishikesh Joshi, Regina Barzilay, Tommi Jaakkola, Katerina Tymoshenko, Alessandro Moschitti, Lluis Marquez
Date: 2015-12-17 00:00:00
Proceeding: NAACL 2016 6
Tasks: Decoder, Question Answering, Retrieval

Question answering forums are rapidly growing in size with no effective
automated ability to refer to and reuse answers already available for previous
posted questions. In this paper, we develop a methodology for finding
semantically related questions. The task is difficult since 1) key pieces of
information are often buried in extraneous details in the question body and 2)
available annotations on similar questions are scarce and fragmented. We design
a recurrent and convolutional model (gated convolution) to effectively map
questions to their semantic representations. The models are pre-trained within
an encoder-decoder framework (from body to title) on the basis of the entire
raw corpus, and fine-tuned discriminatively from limited annotations. Our
evaluation demonstrates that our model yields substantial gains over a standard
IR baseline and various neural network architectures (including CNNs, LSTMs and
GRUs).
Top-down Tree Long Short-Term Memory Networks
Authors: Xingxing Zhang, Liang Lu, Mirella Lapata
Date: 2015-10-31 00:00:00
Proceeding: NAACL 2016 6
Tasks: Dependency Parsing, Reranking, Sentence, Sentence Completion

Long Short-Term Memory (LSTM) networks, a type of recurrent neural network
with a more complex computational unit, have been successfully applied to a
variety of sequence modeling tasks. In this paper we develop Tree Long
Short-Term Memory (TreeLSTM), a neural network model based on LSTM, which is
designed to predict a tree rather than a linear sequence. TreeLSTM defines the
probability of a sentence by estimating the generation probability of its
dependency tree. At each time step, a node is generated based on the
representation of the generated sub-tree. We further enhance the modeling power
of TreeLSTM by explicitly representing the correlations between left and right
dependents. Application of our model to the MSR sentence completion challenge
achieves results beyond the current state of the art. We also report results on
dependency parsing reranking achieving competitive performance.
ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
Authors: Kan Chen, Jiang Wang, Liang-Chieh Chen, Haoyuan Gao, Wei Xu, Ram Nevatia
Date: 2015-11-18 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

We propose a novel attention based deep learning architecture for visual
question answering task (VQA). Given an image and an image related natural
language question, VQA generates the natural language answer for the question.
Generating the correct answers requires the model's attention to focus on the
regions corresponding to the question, because different questions inquire
about the attributes of different image regions. We introduce an attention
based configurable convolutional neural network (ABC-CNN) to learn such
question-guided attention. ABC-CNN determines an attention map for an
image-question pair by convolving the image feature map with configurable
convolutional kernels derived from the question's semantics. We evaluate the
ABC-CNN architecture on three benchmark VQA datasets: Toronto COCO-QA, DAQUAR,
and VQA dataset. ABC-CNN model achieves significant improvements over
state-of-the-art methods on these datasets. The question-guided attention
generated by ABC-CNN is also shown to reflect the regions that are highly
relevant to the questions.
Noisy Activation Functions
Authors: Caglar Gulcehre, Marcin Moczulski, Misha Denil, Yoshua Bengio
Date: 2016-03-01 00:00:00

Common nonlinear activation functions used in neural networks can cause
training difficulties due to the saturation behavior of the activation
function, which may hide dependencies that are not visible to vanilla-SGD
(using first order gradients only). Gating mechanisms that use softly
saturating activation functions to emulate the discrete switching of digital
logic circuits are good examples of this. We propose to exploit the injection
of appropriate noise so that the gradients may flow easily, even if the
noiseless application of the activation function would yield zero gradient.
Large noise will dominate the noise-free gradient and allow stochastic gradient
descent toexplore more. By adding noise only to the problematic parts of the
activation function, we allow the optimization procedure to explore the
boundary between the degenerate (saturating) and the well-behaved parts of the
activation function. We also establish connections to simulated annealing, when
the amount of noise is annealed down, making it easier to optimize hard
objective functions. We find experimentally that replacing such saturating
activation functions by noisy variants helps training in many contexts,
yielding state-of-the-art or competitive results on different datasets and
task, especially when training seems to be the most difficult, e.g., when
curriculum learning is necessary to obtain good results.
The Offset Tree for Learning with Partial Labels
Authors: Alina Beygelzimer, John Langford
Date: 2008-12-21 00:00:00
Tasks: Binary Classification, Classification, General Classification

We present an algorithm, called the Offset Tree, for learning to make
decisions in situations where the payoff of only one choice is observed, rather
than all choices. The algorithm reduces this setting to binary classification,
allowing one to reuse of any existing, fully supervised binary classification
algorithm in this partial information setting. We show that the Offset Tree is
an optimal reduction to binary classification. In particular, it has regret at
most $(k-1)$ times the regret of the binary classifier it uses (where $k$ is
the number of choices), and no reduction to binary classification can do
better. This reduction is also computationally optimal, both at training and
test time, requiring just $O(\log_2 k)$ work to train on an example or make a
prediction.
  Experiments with the Offset Tree show that it generally performs better than
several alternative approaches.
Multi-Bias Non-linear Activation in Deep Neural Networks
Authors: Hongyang Li, Wanli Ouyang, Xiaogang Wang
Date: 2016-04-03 00:00:00

As a widely used non-linear activation, Rectified Linear Unit (ReLU)
separates noise and signal in a feature map by learning a threshold or bias.
However, we argue that the classification of noise and signal not only depends
on the magnitude of responses, but also the context of how the feature
responses would be used to detect more abstract patterns in higher layers. In
order to output multiple response maps with magnitude in different ranges for a
particular visual pattern, existing networks employing ReLU and its variants
have to learn a large number of redundant filters. In this paper, we propose a
multi-bias non-linear activation (MBA) layer to explore the information hidden
in the magnitudes of responses. It is placed after the convolution layer to
decouple the responses to a convolution kernel into multiple maps by
multi-thresholding magnitudes, thus generating more patterns in the feature
space at a low computational cost. It provides great flexibility of selecting
responses to different visual patterns in different magnitude ranges to form
rich representations in higher layers. Such a simple and yet effective scheme
achieves the state-of-the-art performance on several benchmarks.