pwc-archive/papers-with-abstracts huggingface dataset:
Deep Supervised Hashing with Triplet Labels
Authors: Xiaofang Wang, Yi Shi, Kris M. Kitani
Date: 2016-12-12 00:00:00
Tasks: Deep Hashing, Image Retrieval, Retrieval, Triplet

Hashing is one of the most popular and powerful approximate nearest neighbor
search techniques for large-scale image retrieval. Most traditional hashing
methods first represent images as off-the-shelf visual features and then
produce hashing codes in a separate stage. However, off-the-shelf visual
features may not be optimally compatible with the hash code learning procedure,
which may result in sub-optimal hash codes. Recently, deep hashing methods have
been proposed to simultaneously learn image features and hash codes using deep
neural networks and have shown superior performance over traditional hashing
methods. Most deep hashing methods are given supervised information in the form
of pairwise labels or triplet labels. The current state-of-the-art deep hashing
method DPSH~\cite{li2015feature}, which is based on pairwise labels, performs
image feature learning and hash code learning simultaneously by maximizing the
likelihood of pairwise similarities. Inspired by DPSH~\cite{li2015feature}, we
propose a triplet label based deep hashing method which aims to maximize the
likelihood of the given triplet labels. Experimental results show that our
method outperforms all the baselines on CIFAR-10 and NUS-WIDE datasets,
including the state-of-the-art method DPSH~\cite{li2015feature} and all the
previous triplet label based deep hashing methods.
Inverse Compositional Spatial Transformer Networks
Authors: Chen-Hsuan Lin, Simon Lucey
Date: 2016-12-12 00:00:00
Proceeding: CVPR 2017 7
Tasks: Classification, General Classification

In this paper, we establish a theoretical connection between the classical
Lucas & Kanade (LK) algorithm and the emerging topic of Spatial Transformer
Networks (STNs). STNs are of interest to the vision and learning communities
due to their natural ability to combine alignment and classification within the
same theoretical framework. Inspired by the Inverse Compositional (IC) variant
of the LK algorithm, we present Inverse Compositional Spatial Transformer
Networks (IC-STNs). We demonstrate that IC-STNs can achieve better performance
than conventional STNs with less model capacity; in particular, we show
superior performance in pure image alignment tasks as well as joint
alignment/classification problems on real-world problems.
Generalizable Features From Unsupervised Learning
Authors: Mehdi Mirza, Aaron Courville, Yoshua Bengio
Date: 2016-12-12 00:00:00
Tasks: Physical Intuition

Humans learn a predictive model of the world and use this model to reason
about future events and the consequences of actions. In contrast to most
machine predictors, we exhibit an impressive ability to generalize to unseen
scenarios and reason intelligently in these settings. One important aspect of
this ability is physical intuition(Lake et al., 2016). In this work, we explore
the potential of unsupervised learning to find features that promote better
generalization to settings outside the supervised training distribution. Our
task is predicting the stability of towers of square blocks. We demonstrate
that an unsupervised model, trained to predict future frames of a video
sequence of stable and unstable block configurations, can yield features that
support extrapolating stability prediction to blocks configurations outside the
training set distribution
A Unit Selection Methodology for Music Generation Using Deep Neural Networks
Authors: Mason Bretan, Gil Weinberg, Larry Heck
Date: 2016-12-12 00:00:00
Tasks: Music Generation

Several methods exist for a computer to generate music based on data
including Markov chains, recurrent neural networks, recombinancy, and grammars.
We explore the use of unit selection and concatenation as a means of generating
music using a procedure based on ranking, where, we consider a unit to be a
variable length number of measures of music. We first examine whether a unit
selection method, that is restricted to a finite size unit library, can be
sufficient for encompassing a wide spectrum of music. We do this by developing
a deep autoencoder that encodes a musical input and reconstructs the input by
selecting from the library. We then describe a generative model that combines a
deep structured semantic model (DSSM) with an LSTM to predict the next unit,
where units consist of four, two, and one measures of music. We evaluate the
generative model using objective metrics including mean rank and accuracy and
with a subjective listening test in which expert musicians are asked to
complete a forced-choiced ranking task. We compare our model to a note-level
generative baseline that consists of a stacked LSTM trained to predict forward
by one note.
Online Reinforcement Learning for Real-Time Exploration in Continuous State and Action Markov Decision Processes
Authors: Ludovic Hofer, Hugo Gimbert
Date: 2016-12-12 00:00:00
Tasks: regression, Reinforcement Learning, Reinforcement Learning (RL)

This paper presents a new method to learn online policies in continuous
state, continuous action, model-free Markov decision processes, with two
properties that are crucial for practical applications. First, the policies are
implementable with a very low computational cost: once the policy is computed,
the action corresponding to a given state is obtained in logarithmic time with
respect to the number of samples used. Second, our method is versatile: it does
not rely on any a priori knowledge of the structure of optimal policies. We
build upon the Fitted Q-iteration algorithm which represents the $Q$-value as
the average of several regression trees. Our algorithm, the Fitted Policy
Forest algorithm (FPF), computes a regression forest representing the Q-value
and transforms it into a single tree representing the policy, while keeping
control on the size of the policy using resampling and leaf merging. We
introduce an adaptation of Multi-Resolution Exploration (MRE) which is
particularly suited to FPF. We assess the performance of FPF on three classical
benchmarks for reinforcement learning: the "Inverted Pendulum", the "Double
Integrator" and "Car on the Hill" and show that FPF equals or outperforms other
algorithms, although these algorithms rely on the use of particular
representations of the policies, especially chosen in order to fit each of the
three problems. Finally, we exhibit that the combination of FPF and MRE allows
to find nearly optimal solutions in problems where $\epsilon$-greedy approaches
would fail.
Context-aware Sentiment Word Identification: sentiword2vec
Authors: Yushi Yao, Guangjian Li
Date: 2016-12-12 00:00:00
Tasks: Language Modeling, Language Modelling, Sentiment Analysis

Traditional sentiment analysis often uses sentiment dictionary to extract
sentiment information in text and classify documents. However, emerging
informal words and phrases in user generated content call for analysis aware to
the context. Usually, they have special meanings in a particular context.
Because of its great performance in representing inter-word relation, we use
sentiment word vectors to identify the special words. Based on the distributed
language model word2vec, in this paper we represent a novel method about
sentiment representation of word under particular context, to be detailed, to
identify the words with abnormal sentiment polarity in long answers. Result
shows the improved model shows better performance in representing the words
with special meaning, while keep doing well in representing special idiomatic
pattern. Finally, we will discuss the meaning of vectors representing in the
field of sentiment, which may be different from general object-based
conditions.
From narrative descriptions to MedDRA: automagically encoding adverse drug reactions
Authors: Carlo Combi, Margherita Zorzi, Gabriele Pozzani, Ugo Moretti
Date: 2016-12-12 00:00:00
Tasks: Pharmacovigilance

The collection of narrative spontaneous reports is an irreplaceable source
for the prompt detection of suspected adverse drug reactions (ADRs): qualified
domain experts manually revise a huge amount of narrative descriptions and then
encode texts according to MedDRA standard terminology. The manual annotation of
narrative documents with medical terminology is a subtle and expensive task,
since the number of reports is growing up day-by-day. MagiCoder, a Natural
Language Processing algorithm, is proposed for the automatic encoding of
free-text descriptions into MedDRA terms. MagiCoder procedure is efficient in
terms of computational complexity (in particular, it is linear in the size of
the narrative input and the terminology). We tested it on a large dataset of
about 4500 manually revised reports, by performing an automated comparison
between human and MagiCoder revisions. For the current base version of
MagiCoder, we measured: on short descriptions, an average recall of $86\%$ and
an average precision of $88\%$; on medium-long descriptions (up to 255
characters), an average recall of $64\%$ and an average precision of $63\%$.
From a practical point of view, MagiCoder reduces the time required for
encoding ADR reports. Pharmacologists have simply to review and validate the
MagiCoder terms proposed by the application, instead of choosing the right
terms among the 70K low level terms of MedDRA. Such improvement in the
efficiency of pharmacologists' work has a relevant impact also on the quality
of the subsequent data analysis. We developed MagiCoder for the Italian
pharmacovigilance language. However, our proposal is based on a general
approach, not depending on the considered language nor the term dictionary.
Stochastic Quasi-Newton Langevin Monte Carlo
Authors: Umut Şimşekli, Roland Badeau, A. Taylan Cemgil, Gaël Richard
Date: 2016-02-10 00:00:00
Tasks: Second-order methods

Recently, Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have
been proposed for scaling up Monte Carlo computations to large data problems.
Whilst these approaches have proven useful in many applications, vanilla
SG-MCMC might suffer from poor mixing rates when random variables exhibit
strong couplings under the target densities or big scale differences. In this
study, we propose a novel SG-MCMC method that takes the local geometry into
account by using ideas from Quasi-Newton optimization methods. These second
order methods directly approximate the inverse Hessian by using a limited
history of samples and their gradients. Our method uses dense approximations of
the inverse Hessian while keeping the time and memory complexities linear with
the dimension of the problem. We provide a formal theoretical analysis where we
show that the proposed method is asymptotically unbiased and consistent with
the posterior expectations. We illustrate the effectiveness of the approach on
both synthetic and real datasets. Our experiments on two challenging
applications show that our method achieves fast convergence rates similar to
Riemannian approaches while at the same time having low computational
requirements similar to diagonal preconditioning approaches.
Subspace Learning from Extremely Compressed Measurements
Authors: Akshay Krishnamurthy, Martin Azizyan, Aarti Singh
Date: 2014-04-03 00:00:00

We consider learning the principal subspace of a large set of vectors from an
extremely small number of compressive measurements of each vector. Our
theoretical results show that even a constant number of measurements per column
suffices to approximate the principal subspace to arbitrary precision, provided
that the number of vectors is large. This result is achieved by a simple
algorithm that computes the eigenvectors of an estimate of the covariance
matrix. The main insight is to exploit an averaging effect that arises from
applying a different random projection to each vector. We provide a number of
simulations confirming our theoretical results.
Empirical Evaluation of A New Approach to Simplifying Long Short-term Memory (LSTM)
Authors: Yuzhen Lu
Date: 2016-12-12 00:00:00

The standard LSTM, although it succeeds in the modeling long-range
dependences, suffers from a highly complex structure that can be simplified
through modifications to its gate units. This paper was to perform an empirical
comparison between the standard LSTM and three new simplified variants that
were obtained by eliminating input signal, bias and hidden unit signal from
individual gates, on the tasks of modeling two sequence datasets. The
experiments show that the three variants, with reduced parameters, can achieve
comparable performance with the standard LSTM. Due attention should be paid to
turning the learning rate to achieve high accuracies
Segmentation of large images based on super-pixels and community detection in graphs
Authors: Oscar A. C. Linares, Glenda Michele Botelho, Francisco Aparecido Rodrigues, João Batista Neto
Date: 2016-12-12 00:00:00
Tasks: Community Detection, Image Segmentation, Medical Diagnosis, Segmentation, Semantic Segmentation

Image segmentation has many applications which range from machine learning to
medical diagnosis. In this paper, we propose a framework for the segmentation
of images based on super-pixels and algorithms for community identification in
graphs. The super-pixel pre-segmentation step reduces the number of nodes in
the graph, rendering the method the ability to process large images. Moreover,
community detection algorithms provide more accurate segmentation than
traditional approaches, such as those based on spectral graph partition. We
also compare our method with two algorithms: a) the graph-based approach by
Felzenszwalb and Huttenlocher and b) the contour-based method by Arbelaez.
Results have shown that our method provides more precise segmentation and is
faster than both of them.
Poincaré inequalities on intervals -- application to sensitivity analysis
Authors: Olivier Roustant, Franck Barthe, Bertrand Iooss
Date: 2016-12-12 00:00:00
Tasks: Sensitivity

The development of global sensitivity analysis of numerical model outputs has
recently raised new issues on 1-dimensional Poincar\'e inequalities. Typically
two kind of sensitivity indices are linked by a Poincar\'e type inequality,
which provide upper bounds of the most interpretable index by using the other
one, cheaper to compute. This allows performing a low-cost screening of
unessential variables. The efficiency of this screening then highly depends on
the accuracy of the upper bounds in Poincar\'e inequalities. The novelty in the
questions concern the wide range of probability distributions involved, which
are often truncated on intervals. After providing an overview of the existing
knowledge and techniques, we add some theory about Poincar\'e constants on
intervals, with improvements for symmetric intervals. Then we exploit the
spectral interpretation for computing exact value of Poincar\'e constants of
any admissible distribution on a given interval. We give semi-analytical
results for some frequent distributions (truncated exponential, triangular,
truncated normal), and present a numerical method in the general case. Finally,
an application is made to a hydrological problem, showing the benefits of the
new results in Poincar\'e inequalities to sensitivity analysis.
Analysis and Optimization of Loss Functions for Multiclass, Top-k, and Multilabel Classification
Authors: Maksim Lapin, Matthias Hein, Bernt Schiele
Date: 2016-12-12 00:00:00
Tasks: General Classification, image-classification, Image Classification

Top-k error is currently a popular performance measure on large scale image
classification benchmarks such as ImageNet and Places. Despite its wide
acceptance, our understanding of this metric is limited as most of the previous
research is focused on its special case, the top-1 error. In this work, we
explore two directions that shed more light on the top-k error. First, we
provide an in-depth analysis of established and recently proposed single-label
multiclass methods along with a detailed account of efficient optimization
algorithms for them. Our results indicate that the softmax loss and the smooth
multiclass SVM are surprisingly competitive in top-k error uniformly across all
k, which can be explained by our analysis of multiclass top-k calibration.
Further improvements for a specific k are possible with a number of proposed
top-k loss functions. Second, we use the top-k methods to explore the
transition from multiclass to multilabel learning. In particular, we find that
it is possible to obtain effective multilabel classifiers on Pascal VOC using a
single label per image for training, while the gap between multiclass and
multilabel methods on MS COCO is more significant. Finally, our contribution of
efficient algorithms for training with the considered top-k and multilabel loss
functions is of independent interest.
Unraveling reported dreams with text analytics
Authors: Iris Hendrickx, Louis Onrust, Florian Kunneman, Ali Hürriyetoğlu, Antal Van den Bosch, Wessel Stoop
Date: 2016-12-12 00:00:00
Tasks: text-classification, Text Classification

We investigate what distinguishes reported dreams from other personal
narratives. The continuity hypothesis, stemming from psychological dream
analysis work, states that most dreams refer to a person's daily life and
personal concerns, similar to other personal narratives such as diary entries.
Differences between the two texts may reveal the linguistic markers of dream
text, which could be the basis for new dream analysis work and for the
automatic detection of dream descriptions. We used three text analytics
methods: text classification, topic modeling, and text coherence analysis, and
applied these methods to a balanced set of texts representing dreams, diary
entries, and other personal stories. We observed that dream texts could be
distinguished from other personal narratives nearly perfectly, mostly based on
the presence of uncertainty markers and descriptions of scenes. Important
markers for non-dream narratives are specific time expressions and
conversational expressions. Dream texts also exhibit a lower discourse
coherence than other personal narratives.
Mastering 2048 with Delayed Temporal Coherence Learning, Multi-Stage Weight Promotion, Redundant Encoding and Carousel Shaping
Authors: Wojciech Jaśkowski
Date: 2016-04-18 00:00:00
Tasks: CPU, Playing the Game of 2048, Reinforcement Learning

2048 is an engaging single-player, nondeterministic video puzzle game, which,
thanks to the simple rules and hard-to-master gameplay, has gained massive
popularity in recent years. As 2048 can be conveniently embedded into the
discrete-state Markov decision processes framework, we treat it as a testbed
for evaluating existing and new methods in reinforcement learning. With the aim
to develop a strong 2048 playing program, we employ temporal difference
learning with systematic n-tuple networks. We show that this basic method can
be significantly improved with temporal coherence learning, multi-stage
function approximator with weight promotion, carousel shaping, and redundant
encoding. In addition, we demonstrate how to take advantage of the
characteristics of the n-tuple network, to improve the algorithmic
effectiveness of the learning process by i) delaying the (decayed) update and
applying lock-free optimistic parallelism to effortlessly make advantage of
multiple CPU cores. This way, we were able to develop the best known 2048
playing program to date, which confirms the effectiveness of the introduced
methods for discrete-state Markov decision problems.
FastText.zip: Compressing text classification models
Authors: Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, Tomas Mikolov
Date: 2016-12-12 00:00:00
Tasks: General Classification, Quantization, Text Classification, Word Embeddings

We consider the problem of producing compact architectures for text
classification, such that the full model fits in a limited amount of memory.
After considering different solutions inspired by the hashing literature, we
propose a method built upon product quantization to store word embeddings.
While the original technique leads to a loss in accuracy, we adapt this method
to circumvent quantization artefacts. Our experiments carried out on several
benchmarks show that our approach typically requires two orders of magnitude
less memory than fastText while being only slightly inferior with respect to
accuracy. As a result, it outperforms the state of the art by a good margin in
terms of the compromise between memory usage and accuracy.
Video Description using Bidirectional Recurrent Neural Networks
Authors: Álvaro Peris, Marc Bolaños, Petia Radeva, Francisco Casacuberta
Date: 2016-04-12 00:00:00
Tasks: Decoder, Text Generation, Translation, Video Captioning, Video Description

Although traditionally used in the machine translation field, the
encoder-decoder framework has been recently applied for the generation of video
and image descriptions. The combination of Convolutional and Recurrent Neural
Networks in these models has proven to outperform the previous state of the
art, obtaining more accurate video descriptions. In this work we propose
pushing further this model by introducing two contributions into the encoding
stage. First, producing richer image representations by combining object and
location information from Convolutional Neural Networks and second, introducing
Bidirectional Recurrent Neural Networks for capturing both forward and backward
temporal relationships in the input frames.
A Binary Convolutional Encoder-decoder Network for Real-time Natural Scene Text Processing
Authors: Zichuan Liu, Yixing Li, Fengbo Ren, Hao Yu
Date: 2016-12-12 00:00:00
Tasks: Decoder, GPU

In this paper, we develop a binary convolutional encoder-decoder network
(B-CEDNet) for natural scene text processing (NSTP). It converts a text image
to a class-distinguished salience map that reveals the categorical, spatial and
morphological information of characters. The existing solutions are either
memory consuming or run-time consuming that cannot be applied to real-time
applications on resource-constrained devices such as advanced driver assistance
systems. The developed network can process multiple regions containing
characters by one-off forward operation, and is trained to have binary weights
and binary feature maps, which lead to both remarkable inference run-time
speedup and memory usage reduction. By training with over 200, 000 synthesis
scene text images (size of $32\times128$), it can achieve $90\%$ and $91\%$
pixel-wise accuracy on ICDAR-03 and ICDAR-13 datasets. It only consumes $4.59\
ms$ inference run-time realized on GPU with a small network size of 2.14 MB,
which is up to $8\times$ faster and $96\%$ smaller than it full-precision
version.
VIBIKNet: Visual Bidirectional Kernelized Network for Visual Question Answering
Authors: Marc Bolaños, Álvaro Peris, Francisco Casacuberta, Petia Radeva
Date: 2016-12-12 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

In this paper, we address the problem of visual question answering by
proposing a novel model, called VIBIKNet. Our model is based on integrating
Kernelized Convolutional Neural Networks and Long-Short Term Memory units to
generate an answer given a question about an image. We prove that VIBIKNet is
an optimal trade-off between accuracy and computational load, in terms of
memory and time consumption. We validate our method on the VQA challenge
dataset and compare it to the top performing methods in order to illustrate its
performance and speed.