pwc-archive/papers-with-abstracts huggingface dataset:
Discriminative k-shot learning using probabilistic models
Authors: Matthias Bauer, Mateo Rojas-Carulla, Jakub Bartłomiej Świątkowski, Bernhard Schölkopf, Richard E. Turner
Date: 2017-06-01 00:00:00
Proceeding: ICLR 2018 1
Tasks: Few-Shot Image Classification, General Classification, image-classification, Image Classification

This paper introduces a probabilistic framework for k-shot image
classification. The goal is to generalise from an initial large-scale
classification task to a separate task comprising new classes and small numbers
of examples. The new approach not only leverages the feature-based
representation learned by a neural network from the initial task
(representational transfer), but also information about the classes (concept
transfer). The concept information is encapsulated in a probabilistic model for
the final layer weights of the neural network which acts as a prior for
probabilistic k-shot learning. We show that even a simple probabilistic model
achieves state-of-the-art on a standard k-shot learning dataset by a large
margin. Moreover, it is able to accurately model uncertainty, leading to well
calibrated classifiers, and is easily extensible and flexible, unlike many
recent approaches to k-shot learning.
Precision Learning: Towards Use of Known Operators in Neural Networks
Authors: Andreas Maier, Frank Schebesch, Christopher Syben, Tobias Würfl, Stefan Steidl, Jang-Hwan Choi, Rebecca Fahrig
Date: 2017-12-01 00:00:00
Tasks: SSIM

In this paper, we consider the use of prior knowledge within neural networks.
In particular, we investigate the effect of a known transform within the
mapping from input data space to the output domain. We demonstrate that use of
known transforms is able to change maximal error bounds.
  In order to explore the effect further, we consider the problem of X-ray
material decomposition as an example to incorporate additional prior knowledge.
We demonstrate that inclusion of a non-linear function known from the physical
properties of the system is able to reduce prediction errors therewith
improving prediction quality from SSIM values of 0.54 to 0.88.
  This approach is applicable to a wide set of applications in physics and
signal processing that provide prior knowledge on such transforms. Also maximal
error estimation and network understanding could be facilitated within the
context of precision learning.
Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks
Authors: Shankar Krishnan, Ying Xiao, Rif A. Saurous
Date: 2017-12-08 00:00:00
Proceeding: ICLR 2018 1
Tasks: Stochastic Optimization

Progress in deep learning is slowed by the days or weeks it takes to train
large models. The natural solution of using more hardware is limited by
diminishing returns, and leads to inefficient use of additional resources. In
this paper, we present a large batch, stochastic optimization algorithm that is
both faster than widely used algorithms for fixed amounts of computation, and
also scales up substantially better as more computational resources become
available. Our algorithm implicitly computes the inverse Hessian of each
mini-batch to produce descent directions; we do so without either an explicit
approximation to the Hessian or Hessian-vector products. We demonstrate the
effectiveness of our algorithm by successfully training large ImageNet models
(Inception-V3, Resnet-50, Resnet-101 and Inception-Resnet-V2) with mini-batch
sizes of up to 32000 with no loss in validation error relative to current
baselines, and no increase in the total number of steps. At smaller mini-batch
sizes, our optimizer improves the validation error in these models by 0.8-0.9%.
Alternatively, we can trade off this accuracy to reduce the number of training
steps needed by roughly 10-30%. Our work is practical and easily usable by
others -- only one hyperparameter (learning rate) needs tuning, and
furthermore, the algorithm is as computationally cheap as the commonly used
Adam optimizer.
Fast Low-Rank Matrix Estimation without the Condition Number
Authors: Mohammadreza Soltani, Chinmay Hegde
Date: 2017-12-08 00:00:00

In this paper, we study the general problem of optimizing a convex function
$F(L)$ over the set of $p \times p$ matrices, subject to rank constraints on
$L$. However, existing first-order methods for solving such problems either are
too slow to converge, or require multiple invocations of singular value
decompositions. On the other hand, factorization-based non-convex algorithms,
while being much faster, require stringent assumptions on the \emph{condition
number} of the optimum. In this paper, we provide a novel algorithmic framework
that achieves the best of both worlds: asymptotically as fast as factorization
methods, while requiring no dependency on the condition number.
  We instantiate our general framework for three important matrix estimation
problems that impact several practical applications; (i) a \emph{nonlinear}
variant of affine rank minimization, (ii) logistic PCA, and (iii) precision
matrix estimation in probabilistic graphical model learning. We then derive
explicit bounds on the sample complexity as well as the running time of our
approach, and show that it achieves the best possible bounds for both cases. We
also provide an extensive range of experimental results, and demonstrate that
our algorithm provides a very attractive tradeoff between estimation accuracy
and running time.
Nintendo Super Smash Bros. Melee: An "Untouchable" Agent
Authors: Ben Parr, Deepak Dilipkumar, Yu-An Liu
Date: 2017-12-08 00:00:00

Nintendo's Super Smash Bros. Melee fighting game can be emulated on modern
hardware allowing us to inspect internal memory states, such as character
positions. We created an AI that avoids being hit by training using these
internal memory states and outputting controller button presses. After training
on a month's worth of Melee matches, our best agent learned to avoid the
toughest AI built into the game for a full minute 74.6% of the time.
Transformational Sparse Coding
Authors: Dimitrios C. Gklezakos, Rajesh P. N. Rao
Date: 2017-12-08 00:00:00
Tasks: Object, Object Recognition

A fundamental problem faced by object recognition systems is that objects and
their features can appear in different locations, scales and orientations.
Current deep learning methods attempt to achieve invariance to local
translations via pooling, discarding the locations of features in the process.
Other approaches explicitly learn transformed versions of the same feature,
leading to representations that quickly explode in size. Instead of discarding
the rich and useful information about feature transformations to achieve
invariance, we argue that models should learn object features conjointly with
their transformations to achieve equivariance. We propose a new model of
unsupervised learning based on sparse coding that can learn object features
jointly with their affine transformations directly from images. Results based
on learning from natural images indicate that our approach matches the
reconstruction quality of traditional sparse coding but with significantly
fewer degrees of freedom while simultaneously learning transformations from
data. These results open the door to scaling up unsupervised learning to allow
deep feature+transformation learning in a manner consistent with the
ventral+dorsal stream architecture of the primate visual cortex.
Social Emotion Mining Techniques for Facebook Posts Reaction Prediction
Authors: Florian Krebs, Bruno Lubascher, Tobias Moers, Pieter Schaap, Gerasimos Spanakis
Date: 2017-12-08 00:00:00
Tasks: Word Embeddings

As of February 2016 Facebook allows users to express their experienced
emotions about a post by using five so-called `reactions'. This research paper
proposes and evaluates alternative methods for predicting these reactions to
user posts on public pages of firms/companies (like supermarket chains). For
this purpose, we collected posts (and their reactions) from Facebook pages of
large supermarket chains and constructed a dataset which is available for other
researches. In order to predict the distribution of reactions of a new post,
neural network architectures (convolutional and recurrent neural networks) were
tested using pretrained word embeddings. Results of the neural networks were
improved by introducing a bootstrapping approach for sentiment and emotion
mining on the comments for each post. The final model (a combination of neural
network and a baseline emotion miner) is able to predict the reaction
distribution on Facebook posts with a mean squared error (or misclassification
rate) of 0.135.
Long Text Generation via Adversarial Training with Leaked Information
Authors: Jiaxian Guo, Sidi Lu, Han Cai, Wei-Nan Zhang, Yong Yu, Jun Wang
Date: 2017-09-24 00:00:00
Tasks: Reinforcement Learning, Sentence, Text Generation

Automatically generating coherent and semantically meaningful text has many
applications in machine translation, dialogue systems, image captioning, etc.
Recently, by combining with policy gradient, Generative Adversarial Nets (GAN)
that use a discriminative model to guide the training of the generative model
as a reinforcement learning policy has shown promising results in text
generation. However, the scalar guiding signal is only available after the
entire text has been generated and lacks intermediate information about text
structure during the generative process. As such, it limits its success when
the length of the generated text samples is long (more than 20 words). In this
paper, we propose a new framework, called LeakGAN, to address the problem for
long text generation. We allow the discriminative net to leak its own
high-level extracted features to the generative net to further help the
guidance. The generator incorporates such informative signals into all
generation steps through an additional Manager module, which takes the
extracted features of current generated words and outputs a latent vector to
guide the Worker module for next-word generation. Our extensive experiments on
synthetic data and various real-world tasks with Turing test demonstrate that
LeakGAN is highly effective in long text generation and also improves the
performance in short text generation scenarios. More importantly, without any
supervision, LeakGAN would be able to implicitly learn sentence structures only
through the interaction between Manager and Worker.
Basic Thresholding Classification
Authors: Mehmet Altan Toksöz
Date: 2017-12-08 00:00:00
Tasks: Classification, Face Identification, General Classification, image-classification, Image Classification, parameter estimation

In this thesis, we propose a light-weight sparsity-based algorithm, basic
thresholding classifier (BTC), for classification applications (such as face
identification, hyper-spectral image classification, etc.) which is capable of
identifying test samples extremely rapidly and performing high classification
accuracy. Originally BTC is a linear classifier which works based on the
assumption that the samples of the classes of a given dataset are linearly
separable. However, in practice those samples may not be linearly separable. In
this context, we also propose another algorithm namely kernel basic
thresholding classifier (KBTC) which is a non-linear kernel version of the BTC
algorithm. KBTC can achieve promising results especially when the given samples
are linearly non-separable. For both proposals, we introduce sufficient
identification conditions (SICs) under which BTC and KBTC can identify any test
sample in the range space of a given dictionary. By using SICs, we develop
parameter estimation procedures which do not require any cross validation. Both
BTC and KBTC algorithms provide efficient classifier fusion schemes in which
individual classifier outputs are combined to produce better classification
results. For instance, for the application of face identification, this is done
by combining the residuals having different random projectors. For spatial
applications such as hyper-spectral image classification, the fusion is carried
out by incorporating the spatial information, in which the output residual maps
are filtered using a smoothing filter. Numerical results on publicly available
face and hyper-spectral datasets show that our proposal outperforms well-known
support vector machines (SVM)-based techniques, multinomial logistic regression
(MLR)-based methods, and sparsity-based approaches like $l_1$-minimization and
simultaneous orthogonal matching pursuit (SOMP).
Improving Negative Sampling for Word Representation using Self-embedded Features
Authors: Long Chen, Fajie Yuan, Joemon M. Jose, Wei-Nan Zhang
Date: 2017-10-26 00:00:00

Although the word-popularity based negative sampler has shown superb
performance in the skip-gram model, the theoretical motivation behind
oversampling popular (non-observed) words as negative samples is still not well
understood. In this paper, we start from an investigation of the gradient
vanishing issue in the skipgram model without a proper negative sampler. By
performing an insightful analysis from the stochastic gradient descent (SGD)
learning perspective, we demonstrate that, both theoretically and intuitively,
negative samples with larger inner product scores are more informative than
those with lower scores for the SGD learner in terms of both convergence rate
and accuracy. Understanding this, we propose an alternative sampling algorithm
that dynamically selects informative negative samples during each SGD update.
More importantly, the proposed sampler accounts for multi-dimensional
self-embedded features during the sampling process, which essentially makes it
more effective than the original popularity-based (one-dimensional) sampler.
Empirical experiments further verify our observations, and show that our
fine-grained samplers gain significant improvement over the existing ones
without increasing computational complexity.
Flower Categorization using Deep Convolutional Neural Networks
Authors: Ayesha Gurnani, Viraj Mavani, Vandit Gajjar, Yash Khandhediya
Date: 2017-08-12 00:00:00
Tasks: Classification, General Classification, Image Segmentation, Semantic Segmentation

We have developed a deep learning network for classification of different
flowers. For this, we have used Visual Geometry Group's 102 category flower
dataset having 8189 images of 102 different flowers from University of Oxford.
The method is basically divided into two parts; Image segmentation and
classification. We have compared the performance of two different Convolutional
Neural Network architectures GoogLeNet and AlexNet for classification purpose.
By keeping the hyper parameters same for both architectures, we have found that
the top 1 and top 5 accuracies of GoogLeNet are 47.15% and 69.17% respectively
whereas the top 1 and top 5 accuracies of AlexNet are 43.39% and 68.68%
respectively. These results are extremely good when compared to random
classification accuracy of 0.98%. This method for classification of flowers can
be implemented in real time applications and can be used to help botanists for
their research as well as camping enthusiasts.
Prosocial learning agents solve generalized Stag Hunts better than selfish ones
Authors: Alexander Peysakhovich, Adam Lerer
Date: 2017-09-08 00:00:00
Tasks: Deep Reinforcement Learning, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning has become an important paradigm for constructing
agents that can enter complex multi-agent situations and improve their policies
through experience. One commonly used technique is reactive training - applying
standard RL methods while treating other agents as a part of the learner's
environment. It is known that in general-sum games reactive training can lead
groups of agents to converge to inefficient outcomes. We focus on one such
class of environments: Stag Hunt games. Here agents either choose a risky
cooperative policy (which leads to high payoffs if both choose it but low
payoffs to an agent who attempts it alone) or a safe one (which leads to a safe
payoff no matter what). We ask how we can change the learning rule of a single
agent to improve its outcomes in Stag Hunts that include other reactive
learners. We extend existing work on reward-shaping in multi-agent
reinforcement learning and show that that making a single agent prosocial, that
is, making them care about the rewards of their partners can increase the
probability that groups converge to good outcomes. Thus, even if we control a
single agent in a group making that agent prosocial can increase our agent's
long-run payoff. We show experimentally that this result carries over to a
variety of more complex environments with Stag Hunt-like dynamics including
ones where agents must learn from raw input pixels.
Characterizing the hyper-parameter space of LSTM language models for mixed context applications
Authors: Victor Akinwande, Sekou L. Remy
Date: 2017-12-08 00:00:00
Tasks: Language Modeling, Language Modelling, Sensitivity

Applying state of the art deep learning models to novel real world datasets
gives a practical evaluation of the generalizability of these models. Of
importance in this process is how sensitive the hyper parameters of such models
are to novel datasets as this would affect the reproducibility of a model. We
present work to characterize the hyper parameter space of an LSTM for language
modeling on a code-mixed corpus. We observe that the evaluated model shows
minimal sensitivity to our novel dataset bar a few hyper parameters.
Optimized Structured Sparse Sensing Matrices for Compressive Sensing
Authors: Tao Hong, Xiao Li, Zhihui Zhu, Qiuwei Li
Date: 2017-09-19 00:00:00
Tasks: Compressive Sensing, Image Compression

We consider designing a robust structured sparse sensing matrix consisting of
a sparse matrix with a few non-zero entries per row and a dense base matrix for
capturing signals efficiently We design the robust structured sparse sensing
matrix through minimizing the distance between the Gram matrix of the
equivalent dictionary and the target Gram of matrix holding small mutual
coherence. Moreover, a regularization is added to enforce the robustness of the
optimized structured sparse sensing matrix to the sparse representation error
(SRE) of signals of interests. An alternating minimization algorithm with
global sequence convergence is proposed for solving the corresponding
optimization problem. Numerical experiments on synthetic data and natural
images show that the obtained structured sensing matrix results in a higher
signal reconstruction than a random dense sensing matrix.
Class Rectification Hard Mining for Imbalanced Deep Learning
Authors: Qi Dong, Shaogang Gong, Xiatian Zhu
Date: 2017-12-08 00:00:00
Proceeding: ICCV 2017 10
Tasks: Attribute, Deep Learning

Recognising detailed facial or clothing attributes in images of people is a
challenging task for computer vision, especially when the training data are
both in very large scale and extremely imbalanced among different attribute
classes. To address this problem, we formulate a novel scheme for batch
incremental hard sample mining of minority attribute classes from imbalanced
large scale training data. We develop an end-to-end deep learning framework
capable of avoiding the dominant effect of majority classes by discovering
sparsely sampled boundaries of minority classes. This is made possible by
introducing a Class Rectification Loss (CRL) regularising algorithm. We
demonstrate the advantages and scalability of CRL over existing
state-of-the-art attribute recognition and imbalanced data learning models on
two large scale imbalanced benchmark datasets, the CelebA facial attribute
dataset and the X-Domain clothing attribute dataset.
Minimal Solvers for Monocular Rolling Shutter Compensation under Ackermann Motion
Authors: Pulak Purkait, Christopher Zach
Date: 2017-12-08 00:00:00
Tasks: Motion Compensation

Modern automotive vehicles are often equipped with a budget commercial
rolling shutter camera. These devices often produce distorted images due to the
inter-row delay of the camera while capturing the image. Recent methods for
monocular rolling shutter motion compensation utilize blur kernel and the
straightness property of line segments. However, these methods are limited to
handling rotational motion and also are not fast enough to operate in real
time. In this paper, we propose a minimal solver for the rolling shutter motion
compensation which assumes known vertical direction of the camera. Thanks to
the Ackermann motion model of vehicles which consists of only two motion
parameters, and two parameters for the simplified depth assumption that lead to
a 4-line algorithm. The proposed minimal solver estimates the rolling shutter
camera motion efficiently and accurately. The extensive experiments on real and
simulated datasets demonstrate the benefits of our approach in terms of
qualitative and quantitative results.
The Birth of Collective Memories: Analyzing Emerging Entities in Text Streams
Authors: David Graus, Daan Odijk, Maarten de Rijke
Date: 2017-01-15 00:00:00

We study how collective memories are formed online. We do so by tracking
entities that emerge in public discourse, that is, in online text streams such
as social media and news streams, before they are incorporated into Wikipedia,
which, we argue, can be viewed as an online place for collective memory. By
tracking how entities emerge in public discourse, i.e., the temporal patterns
between their first mention in online text streams and subsequent incorporation
into collective memory, we gain insights into how the collective remembrance
process happens online. Specifically, we analyze nearly 80,000 entities as they
emerge in online text streams before they are incorporated into Wikipedia. The
online text streams we use for our analysis comprise of social media and news
streams, and span over 579 million documents in a timespan of 18 months. We
discover two main emergence patterns: entities that emerge in a "bursty"
fashion, i.e., that appear in public discourse without a precedent, blast into
activity and transition into collective memory. Other entities display a
"delayed" pattern, where they appear in public discourse, experience a period
of inactivity, and then resurface before transitioning into our cultural
collective memory.
Combining Deep Universal Features, Semantic Attributes, and Hierarchical Classification for Zero-Shot Learning
Authors: Jared Markowitz, Aurora C. Schmidt, Philippe M. Burlina, I-Jeng Wang
Date: 2017-12-08 00:00:00
Tasks: Attribute, General Classification, Zero-Shot Learning

We address zero-shot (ZS) learning, building upon prior work in hierarchical
classification by combining it with approaches based on semantic attribute
estimation. For both non-novel and novel image classes we compare multiple
formulations of the problem, starting with deep universal features in each
case. We investigate the effect of using different posterior probabilities as
inputs to the hierarchical classifier, comparing the performances of posteriors
derived from distances to SVM classifier boundaries with those of posteriors
based on semantic attribute estimation. Using a dataset consisting of 150
object classes from the ImageNet ILSVRC2012 data set, we find that the
hierarchical classification method that maximizes expected reward for non-novel
classes differs from the method that maximizes expected reward for novel
classes. We also show that using input posteriors based on semantic attributes
improves the expected reward for novel classes.