pwc-archive/papers-with-abstracts huggingface dataset:
Softmax GAN
Authors: Min Lin
Date: 2017-04-20 00:00:00
Tasks: Generative Adversarial Network

Softmax GAN is a novel variant of Generative Adversarial Network (GAN). The key idea of Softmax GAN is to replace the classification loss in the original GAN with a softmax cross-entropy loss in the sample space of one single batch. In the adversarial learning of $N$ real training samples and $M$ generated samples, the target of discriminator training is to distribute all the probability mass to the real samples, each with probability $\frac{1}{M}$, and distribute zero probability to generated data. In the generator training phase, the target is to assign equal probability to all data points in the batch, each with probability $\frac{1}{M+N}$. While the original GAN is closely related to Noise Contrastive Estimation (NCE), we show that Softmax GAN is the Importance Sampling version of GAN. We futher demonstrate with experiments that this simple change stabilizes GAN training.
Learning a Unified Control Policy for Safe Falling
Authors: Visak CV Kumar, Sehoon Ha, C. Karen Liu
Date: 2017-03-08 00:00:00
Tasks: continuous-control, Continuous Control

Being able to fall safely is a necessary motor skill for humanoids performing
highly dynamic tasks, such as running and jumping. We propose a new method to
learn a policy that minimizes the maximal impulse during the fall. The
optimization solves for both a discrete contact planning problem and a
continuous optimal control problem. Once trained, the policy can compute the
optimal next contacting body part (e.g. left foot, right foot, or hands),
contact location and timing, and the required joint actuation. We represent the
policy as a mixture of actor-critic neural network, which consists of n control
policies and the corresponding value functions. Each pair of actor-critic is
associated with one of the n possible contacting body parts. During execution,
the policy corresponding to the highest value function will be executed while
the associated body part will be the next contact with the ground. With this
mixture of actor-critic architecture, the discrete contact sequence planning is
solved through the selection of the best critics while the continuous control
problem is solved by the optimization of actors. We show that our policy can
achieve comparable, sometimes even higher, rewards than a recursive search of
the action space using dynamic programming, while enjoying 50 to 400 times of
speed gain during online execution.
Exploring epoch-dependent stochastic residual networks
Authors: Fabio Carrara, Andrea Esuli, Fabrizio Falchi, Alejandro Moreo Fernández
Date: 2017-04-20 00:00:00
Tasks: Management

The recently proposed stochastic residual networks selectively activate or
bypass the layers during training, based on independent stochastic choices,
each of which following a probability distribution that is fixed in advance. In
this paper we present a first exploration on the use of an epoch-dependent
distribution, starting with a higher probability of bypassing deeper layers and
then activating them more frequently as training progresses. Preliminary
results are mixed, yet they show some potential of adding an epoch-dependent
management of distributions, worth of further investigation.
Designing labeled graph classifiers by exploiting the Rényi entropy of the dissimilarity representation
Authors: Lorenzo Livi
Date: 2014-08-22 00:00:00
Tasks: Benchmarking, Clustering, General Classification

Representing patterns as labeled graphs is becoming increasingly common in
the broad field of computational intelligence. Accordingly, a wide repertoire
of pattern recognition tools, such as classifiers and knowledge discovery
procedures, are nowadays available and tested for various datasets of labeled
graphs. However, the design of effective learning procedures operating in the
space of labeled graphs is still a challenging problem, especially from the
computational complexity viewpoint. In this paper, we present a major
improvement of a general-purpose classifier for graphs, which is conceived on
an interplay between dissimilarity representation, clustering,
information-theoretic techniques, and evolutionary optimization algorithms. The
improvement focuses on a specific key subroutine devised to compress the input
data. We prove different theorems which are fundamental to the setting of the
parameters controlling such a compression operation. We demonstrate the
effectiveness of the resulting classifier by benchmarking the developed
variants on well-known datasets of labeled graphs, considering as distinct
performance indicators the classification accuracy, computing time, and
parsimony in terms of structural complexity of the synthesized classification
models. The results show state-of-the-art standards in terms of test set
accuracy and a considerable speed-up for what concerns the computing time.
BB_twtr at SemEval-2017 Task 4: Twitter Sentiment Analysis with CNNs and LSTMs
Authors: Mathieu Cliche
Date: 2017-04-20 00:00:00
Proceeding: SEMEVAL 2017 8
Tasks: Sentiment Analysis, Twitter Sentiment Analysis, Word Embeddings

In this paper we describe our attempt at producing a state-of-the-art Twitter
sentiment classifier using Convolutional Neural Networks (CNNs) and Long Short
Term Memory (LSTMs) networks. Our system leverages a large amount of unlabeled
data to pre-train word embeddings. We then use a subset of the unlabeled data
to fine tune the embeddings using distant supervision. The final CNNs and LSTMs
are trained on the SemEval-2017 Twitter dataset where the embeddings are fined
tuned again. To boost performances we ensemble several CNNs and LSTMs together.
Our approach achieved first rank on all of the five English subtasks amongst 40
teams.
Non-Redundant Spectral Dimensionality Reduction
Authors: Yochai Blau, Tomer Michaeli
Date: 2016-12-11 00:00:00
Tasks: Dimensionality Reduction

Spectral dimensionality reduction algorithms are widely used in numerous
domains, including for recognition, segmentation, tracking and visualization.
However, despite their popularity, these algorithms suffer from a major
limitation known as the "repeated Eigen-directions" phenomenon. That is, many
of the embedding coordinates they produce typically capture the same direction
along the data manifold. This leads to redundant and inefficient
representations that do not reveal the true intrinsic dimensionality of the
data. In this paper, we propose a general method for avoiding redundancy in
spectral algorithms. Our approach relies on replacing the orthogonality
constraints underlying those methods by unpredictability constraints.
Specifically, we require that each embedding coordinate be unpredictable (in
the statistical sense) from all previous ones. We prove that these constraints
necessarily prevent redundancy, and provide a simple technique to incorporate
them into existing methods. As we illustrate on challenging high-dimensional
scenarios, our approach produces significantly more informative and compact
representations, which improve visualization and classification tasks.
Using Mise-En-Scène Visual Features based on MPEG-7 and Deep Learning for Movie Recommendation
Authors: Yashar Deldjoo, Massimo Quadrana, Mehdi Elahi, Paolo Cremonesi
Date: 2017-04-20 00:00:00
Tasks: 4k, Movie Recommendation, Recommendation Systems, TAG

Item features play an important role in movie recommender systems, where
recommendations can be generated by using explicit or implicit preferences of
users on traditional features (attributes) such as tag, genre, and cast.
Typically, movie features are human-generated, either editorially (e.g., genre
and cast) or by leveraging the wisdom of the crowd (e.g., tag), and as such,
they are prone to noise and are expensive to collect. Moreover, these features
are often rare or absent for new items, making it difficult or even impossible
to provide good quality recommendations.
  In this paper, we show that user's preferences on movies can be better
described in terms of the mise-en-sc\`ene features, i.e., the visual aspects of
a movie that characterize design, aesthetics and style (e.g., colors,
textures). We use both MPEG-7 visual descriptors and Deep Learning hidden
layers as example of mise-en-sc\`ene features that can visually describe
movies. Interestingly, mise-en-sc\`ene features can be computed automatically
from video files or even from trailers, offering more flexibility in handling
new items, avoiding the need for costly and error-prone human-based tagging,
and providing good scalability.
  We have conducted a set of experiments on a large catalogue of 4K movies.
Results show that recommendations based on mise-en-sc\`ene features
consistently provide the best performance with respect to richer sets of more
traditional features, such as genre and tag.
The Dependent Doors Problem: An Investigation into Sequential Decisions without Feedback
Authors: Amos Korman, Yoav Rodeh
Date: 2017-04-20 00:00:00

We introduce the dependent doors problem as an abstraction for situations in
which one must perform a sequence of possibly dependent decisions, without
receiving feedback information on the effectiveness of previously made actions.
Informally, the problem considers a set of $d$ doors that are initially closed,
and the aim is to open all of them as fast as possible. To open a door, the
algorithm knocks on it and it might open or not according to some probability
distribution. This distribution may depend on which other doors are currently
open, as well as on which other doors were open during each of the previous
knocks on that door. The algorithm aims to minimize the expected time until all
doors open. Crucially, it must act at any time without knowing whether or which
other doors have already opened. In this work, we focus on scenarios where
dependencies between doors are both positively correlated and acyclic.The
fundamental distribution of a door describes the probability it opens in the
best of conditions (with respect to other doors being open or closed). We show
that if in two configurations of $d$ doors corresponding doors share the same
fundamental distribution, then these configurations have the same optimal
running time up to a universal constant, no matter what are the dependencies
between doors and what are the distributions. We also identify algorithms that
are optimal up to a universal constant factor. For the case in which all doors
share the same fundamental distribution we additionally provide a simpler
algorithm, and a formula to calculate its running time. We furthermore analyse
the price of lacking feedback for several configurations governed by standard
fundamental distributions. In particular, we show that the price is logarithmic
in $d$ for memoryless doors, but can potentially grow to be linear in $d$ for
other distributions.We then turn our attention to investigate precise bounds.
Even for the case of two doors, identifying the optimal sequence is an
intriguing combinatorial question. Here, we study the case of two cascading
memoryless doors. That is, the first door opens on each knock independently
with probability $p\_1$. The second door can only open if the first door is
open, in which case it will open on each knock independently with probability
$p\_2$. We solve this problem almost completely by identifying algorithms that
are optimal up to an additive term of 1.
Knowledge Fusion via Embeddings from Text, Knowledge Graphs, and Images
Authors: Steffen Thoma, Achim Rettinger, Fabian Both
Date: 2017-04-20 00:00:00
Tasks: Knowledge Graphs

We present a baseline approach for cross-modal knowledge fusion. Different
basic fusion methods are evaluated on existing embedding approaches to show the
potential of joining knowledge about certain concepts across modalities in a
fused concept representation.
End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network
Authors: Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Marius Staring, Ivana Išgum
Date: 2017-04-20 00:00:00
Tasks: Image Registration

In this work we propose a deep learning network for deformable image
registration (DIRNet). The DIRNet consists of a convolutional neural network
(ConvNet) regressor, a spatial transformer, and a resampler. The ConvNet
analyzes a pair of fixed and moving images and outputs parameters for the
spatial transformer, which generates the displacement vector field that enables
the resampler to warp the moving image to the fixed image. The DIRNet is
trained end-to-end by unsupervised optimization of a similarity metric between
input image pairs. A trained DIRNet can be applied to perform registration on
unseen image pairs in one pass, thus non-iteratively. Evaluation was performed
with registration of images of handwritten digits (MNIST) and cardiac cine MR
scans (Sunnybrook Cardiac Data). The results demonstrate that registration with
DIRNet is as accurate as a conventional deformable image registration method
with substantially shorter execution times.
Every Untrue Label is Untrue in its Own Way: Controlling Error Type with the Log Bilinear Loss
Authors: Yehezkel S. Resheff, Amit Mandelbaum, Daphna Weinshall
Date: 2017-04-20 00:00:00
Tasks: Multi-class Classification

Deep learning has become the method of choice in many application domains of
machine learning in recent years, especially for multi-class classification
tasks. The most common loss function used in this context is the cross-entropy
loss, which reduces to the log loss in the typical case when there is a single
correct response label. While this loss is insensitive to the identity of the
assigned class in the case of misclassification, in practice it is often the
case that some errors may be more detrimental than others. Here we present the
bilinear-loss (and related log-bilinear-loss) which differentially penalizes
the different wrong assignments of the model. We thoroughly test this method
using standard models and benchmark image datasets. As one application, we show
the ability of this method to better contain error within the correct
super-class, in the hierarchically labeled CIFAR100 dataset, without affecting
the overall performance of the classifier.
A Framework for Parallel and Distributed Training of Neural Networks
Authors: Simone Scardapane, Paolo Di Lorenzo
Date: 2016-10-24 00:00:00

The aim of this paper is to develop a general framework for training neural
networks (NNs) in a distributed environment, where training data is partitioned
over a set of agents that communicate with each other through a sparse,
possibly time-varying, connectivity pattern. In such distributed scenario, the
training problem can be formulated as the (regularized) optimization of a
non-convex social cost function, given by the sum of local (non-convex) costs,
where each agent contributes with a single error term defined with respect to
its local dataset. To devise a flexible and efficient solution, we customize a
recently proposed framework for non-convex optimization over networks, which
hinges on a (primal) convexification-decomposition technique to handle
non-convexity, and a dynamic consensus procedure to diffuse information among
the agents. Several typical choices for the training criterion (e.g., squared
loss, cross entropy, etc.) and regularization (e.g., $\ell_2$ norm, sparsity
inducing penalties, etc.) are included in the framework and explored along the
paper. Convergence to a stationary solution of the social non-convex problem is
guaranteed under mild assumptions. Additionally, we show a principled way
allowing each agent to exploit a possible multi-core architecture (e.g., a
local cloud) in order to parallelize its local optimization step, resulting in
strategies that are both distributed (across the agents) and parallel (inside
each agent) in nature. A comprehensive set of experimental results validate the
proposed approach.
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
Authors: Richard Zhang, Phillip Isola, Alexei A. Efros
Date: 2016-11-29 00:00:00
Proceeding: CVPR 2017 7
Tasks: Representation Learning, Self-Supervised Image Classification, Transfer Learning

We propose split-brain autoencoders, a straightforward modification of the
traditional autoencoder architecture, for unsupervised representation learning.
The method adds a split to the network, resulting in two disjoint sub-networks.
Each sub-network is trained to perform a difficult task -- predicting one
subset of the data channels from another. Together, the sub-networks extract
features from the entire input signal. By forcing the network to solve
cross-channel prediction tasks, we induce a representation within the network
which transfers well to other, unseen tasks. This method achieves
state-of-the-art performance on several large-scale transfer learning
benchmarks.
Understanding the Mechanisms of Deep Transfer Learning for Medical Images
Authors: Hariharan Ravishankar, Prasad Sudhakar, Rahul Venkataramani, Sheshadri Thiruvenkadam, Pavan Annangi, Narayanan Babu, Vivek Vaidya
Date: 2017-04-20 00:00:00
Tasks: image-classification, Image Classification, Transfer Learning

The ability to automatically learn task specific feature representations has
led to a huge success of deep learning methods. When large training data is
scarce, such as in medical imaging problems, transfer learning has been very
effective. In this paper, we systematically investigate the process of
transferring a Convolutional Neural Network, trained on ImageNet images to
perform image classification, to kidney detection problem in ultrasound images.
We study how the detection performance depends on the extent of transfer. We
show that a transferred and tuned CNN can outperform a state-of-the-art feature
engineered pipeline and a hybridization of these two techniques achieves 20\%
higher performance. We also investigate how the evolution of intermediate
response images from our network. Finally, we compare these responses to
state-of-the-art image processing filters in order to gain greater insight into
how transfer learning is able to effectively manage widely varying imaging
regimes.
End-to-end representation learning for Correlation Filter based tracking
Authors: Jack Valmadre, Luca Bertinetto, João F. Henriques, Andrea Vedaldi, Philip H. S. Torr
Date: 2017-04-20 00:00:00
Proceeding: CVPR 2017 7
Tasks: Object Tracking, Representation Learning

The Correlation Filter is an algorithm that trains a linear template to
discriminate between images and their translations. It is well suited to object
tracking because its formulation in the Fourier domain provides a fast
solution, enabling the detector to be re-trained once per frame. Previous works
that use the Correlation Filter, however, have adopted features that were
either manually designed or trained for a different task. This work is the
first to overcome this limitation by interpreting the Correlation Filter
learner, which has a closed-form solution, as a differentiable layer in a deep
neural network. This enables learning deep features that are tightly coupled to
the Correlation Filter. Experiments illustrate that our method has the
important practical benefit of allowing lightweight architectures to achieve
state-of-the-art performance at high framerates.
Predicting Cognitive Decline with Deep Learning of Brain Metabolism and Amyloid Imaging
Authors: Hongyoon Choi, Kyong Hwan Jin
Date: 2017-04-20 00:00:00

For effective treatment of Alzheimer disease (AD), it is important to
identify subjects who are most likely to exhibit rapid cognitive decline.
Herein, we developed a novel framework based on a deep convolutional neural
network which can predict future cognitive decline in mild cognitive impairment
(MCI) patients using flurodeoxyglucose and florbetapir positron emission
tomography (PET). The architecture of the network only relies on baseline PET
studies of AD and normal subjects as the training dataset. Feature extraction
and complicated image preprocessing including nonlinear warping are unnecessary
for our approach. Accuracy of prediction (84.2%) for conversion to AD in MCI
patients outperformed conventional feature-based quantification approaches. ROC
analyses revealed that performance of CNN-based approach was significantly
higher than that of the conventional quantification methods (p < 0.05). Output
scores of the network were strongly correlated with the longitudinal change in
cognitive measurements. These results show the feasibility of deep learning as
a tool for predicting disease outcome using brain images.
Performance Limits of Stochastic Sub-Gradient Learning, Part II: Multi-Agent Case
Authors: Bicheng Ying, Ali H. Sayed
Date: 2017-04-20 00:00:00
Tasks: Stochastic Optimization

The analysis in Part I revealed interesting properties for subgradient
learning algorithms in the context of stochastic optimization when gradient
noise is present. These algorithms are used when the risk functions are
non-smooth and involve non-differentiable components. They have been long
recognized as being slow converging methods. However, it was revealed in Part I
that the rate of convergence becomes linear for stochastic optimization
problems, with the error iterate converging at an exponential rate $\alpha^i$
to within an $O(\mu)-$neighborhood of the optimizer, for some $\alpha \in
(0,1)$ and small step-size $\mu$. The conclusion was established under weaker
assumptions than the prior literature and, moreover, several important problems
(such as LASSO, SVM, and Total Variation) were shown to satisfy these weaker
assumptions automatically (but not the previously used conditions from the
literature). These results revealed that sub-gradient learning methods have
more favorable behavior than originally thought when used to enable continuous
adaptation and learning. The results of Part I were exclusive to single-agent
adaptation. The purpose of the current Part II is to examine the implications
of these discoveries when a collection of networked agents employs subgradient
learning as their cooperative mechanism. The analysis will show that, despite
the coupled dynamics that arises in a networked scenario, the agents are still
able to attain linear convergence in the stochastic case; they are also able to
reach agreement within $O(\mu)$ of the optimizer.
Deep Occlusion Reasoning for Multi-Camera Multi-Target Detection
Authors: Pierre Baqué, François Fleuret, Pascal Fua
Date: 2017-04-19 00:00:00
Proceeding: ICCV 2017 10
Tasks: Multiview Detection

People detection in single 2D images has improved greatly in recent years.
However, comparatively little of this progress has percolated into multi-camera
multi-people tracking algorithms, whose performance still degrades severely
when scenes become very crowded. In this work, we introduce a new architecture
that combines Convolutional Neural Nets and Conditional Random Fields to
explicitly model those ambiguities. One of its key ingredients are high-order
CRF terms that model potential occlusions and give our approach its robustness
even when many people are present. Our model is trained end-to-end and we show
that it outperforms several state-of-art algorithms on challenging scenes.