pwc-archive/papers-with-abstracts huggingface dataset:
Classification Driven Dynamic Image Enhancement
Authors: Vivek Sharma, Ali Diba, Davy Neven, Michael S. Brown, Luc van Gool, Rainer Stiefelhagen
Date: 2017-10-20 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Image Enhancement, Texture Classification

Convolutional neural networks rely on image texture and structure to serve as
discriminative features to classify the image content. Image enhancement
techniques can be used as preprocessing steps to help improve the overall image
quality and in turn improve the overall effectiveness of a CNN. Existing image
enhancement methods, however, are designed to improve the perceptual quality of
an image for a human observer. In this paper, we are interested in learning
CNNs that can emulate image enhancement and restoration, but with the overall
goal to improve image classification and not necessarily human perception. To
this end, we present a unified CNN architecture that uses a range of
enhancement filters that can enhance image-specific details via end-to-end
dynamic filter learning. We demonstrate the effectiveness of this strategy on
four challenging benchmark datasets for fine-grained, object, scene, and
texture classification: CUB-200-2011, PASCAL-VOC2007, MIT-Indoor, and DTD.
Experiments using our proposed enhancement show promising results on all the
datasets. In addition, our approach is capable of improving the performance of
all generic CNN architectures.
Artificial Intelligence and Robotics
Authors: Javier Andreu Perez, Fani Deligianni, Daniele Ravi, Guang-Zhong Yang
Date: 2018-03-28 00:00:00
Tasks: Management

The recent successes of AI have captured the wildest imagination of both the
scientific communities and the general public. Robotics and AI amplify human
potentials, increase productivity and are moving from simple reasoning towards
human-like cognitive abilities. Current AI technologies are used in a set area
of applications, ranging from healthcare, manufacturing, transport, energy, to
financial services, banking, advertising, management consulting and government
agencies. The global AI market is around 260 billion USD in 2016 and it is
estimated to exceed 3 trillion by 2024. To understand the impact of AI, it is
important to draw lessons from it's past successes and failures and this white
paper provides a comprehensive explanation of the evolution of AI, its current
status and future directions.
Learning to Become an Expert: Deep Networks Applied To Super-Resolution Microscopy
Authors: Louis-Émile Robitaille, Audrey Durand, Marc-André Gardner, Christian Gagné, Paul De Koninck, Flavie Lavoie-Cardinal
Date: 2018-03-28 00:00:00
Tasks: Super-Resolution

With super-resolution optical microscopy, it is now possible to observe
molecular interactions in living cells. The obtained images have a very high
spatial precision but their overall quality can vary a lot depending on the
structure of interest and the imaging parameters. Moreover, evaluating this
quality is often difficult for non-expert users. In this work, we tackle the
problem of learning the quality function of super- resolution images from
scores provided by experts. More specifically, we are proposing a system based
on a deep neural network that can provide a quantitative quality measure of a
STED image of neuronal structures given as input. We conduct a user study in
order to evaluate the quality of the predictions of the neural network against
those of a human expert. Results show the potential while highlighting some of
the limits of the proposed approach.
Knowledge Completion for Generics using Guided Tensor Factorization
Authors: Hanie Sedghi, Ashish Sabharwal
Date: 2016-12-12 00:00:00
Proceeding: TACL 2018 1
Tasks: Active Learning, General Knowledge, Question Answering

Given a knowledge base or KB containing (noisy) facts about common nouns or
generics, such as "all trees produce oxygen" or "some animals live in forests",
we consider the problem of inferring additional such facts at a precision
similar to that of the starting KB. Such KBs capture general knowledge about
the world, and are crucial for various applications such as question answering.
Different from commonly studied named entity KBs such as Freebase, generics KBs
involve quantification, have more complex underlying regularities, tend to be
more incomplete, and violate the commonly used locally closed world assumption
(LCWA). We show that existing KB completion methods struggle with this new
task, and present the first approach that is successful. Our results
demonstrate that external information, such as relation schemas and entity
taxonomies, if used appropriately, can be a surprisingly powerful tool in this
setting. First, our simple yet effective knowledge guided tensor factorization
approach achieves state-of-the-art results on two generics KBs (80% precise)
for science, doubling their size at 74%-86% precision. Second, our novel
taxonomy guided, submodular, active learning method for collecting annotations
about rare entities (e.g., oriole, a bird) is 6x more effective at inferring
further new facts about them than multiple active learning baselines.
Modeling Customer Engagement from Partial Observations
Authors: Jelena Stojanovic, Djordje Gligorijevic, Zoran Obradovic
Date: 2018-03-28 00:00:00
Tasks: Missing Values, Representation Learning

It is of high interest for a company to identify customers expected to bring
the largest profit in the upcoming period. Knowing as much as possible about
each customer is crucial for such predictions. However, their demographic data,
preferences, and other information that might be useful for building loyalty
programs is often missing. Additionally, modeling relations among different
customers as a network can be beneficial for predictions at an individual
level, as similar customers tend to have similar purchasing patterns. We
address this problem by proposing a robust framework for structured regression
on deficient data in evolving networks with a supervised representation
learning based on neural features embedding. The new method is compared to
several unstructured and structured alternatives for predicting customer
behavior (e.g. purchasing frequency and customer ticket) on user networks
generated from customer databases of two companies from different industries.
The obtained results show $4\%$ to $130\%$ improvement in accuracy over
alternatives when all customer information is known. Additionally, the
robustness of our method is demonstrated when up to $80\%$ of demographic
information was missing where it was up to several folds more accurate as
compared to alternatives that are either ignoring cases with missing values or
learn their feature representation in an unsupervised manner.
TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild
Authors: Matthias Müller, Adel Bibi, Silvio Giancola, Salman Al-Subaihi, Bernard Ghanem
Date: 2018-03-28 00:00:00
Proceeding: ECCV 2018 9
Tasks: Object, object-detection, Object Detection, Object Tracking

Despite the numerous developments in object tracking, further development of
current tracking algorithms is limited by small and mostly saturated datasets.
As a matter of fact, data-hungry trackers based on deep-learning currently rely
on object detection datasets due to the scarcity of dedicated large-scale
tracking datasets. In this work, we present TrackingNet, the first large-scale
dataset and benchmark for object tracking in the wild. We provide more than 30K
videos with more than 14 million dense bounding box annotations. Our dataset
covers a wide selection of object classes in broad and diverse context. By
releasing such a large-scale dataset, we expect deep trackers to further
improve and generalize. In addition, we introduce a new benchmark composed of
500 novel videos, modeled with a distribution similar to our training dataset.
By sequestering the annotation of the test set and providing an online
evaluation server, we provide a fair benchmark for future development of object
trackers. Deep trackers fine-tuned on a fraction of our dataset improve their
performance by up to 1.6% on OTB100 and up to 1.7% on TrackingNet Test. We
provide an extensive benchmark on TrackingNet by evaluating more than 20
trackers. Our results suggest that object tracking in the wild is far from
being solved.
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods
Authors: Nicolas Loizou, Peter Richtárik
Date: 2017-12-27 00:00:00
Tasks: Stochastic Optimization

In this paper we study several classes of stochastic optimization algorithms
enriched with heavy ball momentum. Among the methods studied are: stochastic
gradient descent, stochastic Newton, stochastic proximal point and stochastic
dual subspace ascent. This is the first time momentum variants of several of
these methods are studied. We choose to perform our analysis in a setting in
which all of the above methods are equivalent. We prove global nonassymptotic
linear convergence rates for all methods and various measures of success,
including primal function values, primal iterates (in L2 sense), and dual
function values. We also show that the primal iterates converge at an
accelerated linear rate in the L1 sense. This is the first time a linear rate
is shown for the stochastic heavy ball method (i.e., stochastic gradient
descent method with momentum). Under somewhat weaker conditions, we establish a
sublinear convergence rate for Cesaro averages of primal iterates. Moreover, we
propose a novel concept, which we call stochastic momentum, aimed at decreasing
the cost of performing the momentum step. We prove linear convergence of
several stochastic methods with stochastic momentum, and show that in some
sparse data regimes and for sufficiently small momentum parameters, these
methods enjoy better overall complexity than methods with deterministic
momentum. Finally, we perform extensive numerical testing on artificial and
real datasets, including data coming from average consensus problems.
Stochastic IMT (insulator-metal-transition) neurons: An interplay of thermal and threshold noise at bifurcation
Authors: Abhinav Parihar, Matthew Jerry, Suman Datta, Arijit Raychowdhury
Date: 2017-08-16 00:00:00

Artificial neural networks can harness stochasticity in multiple ways to
enable a vast class of computationally powerful models. Electronic
implementation of such stochastic networks is currently limited to addition of
algorithmic noise to digital machines which is inherently inefficient; albeit
recent efforts to harness physical noise in devices for stochasticity have
shown promise. To succeed in fabricating electronic neuromorphic networks we
need experimental evidence of devices with measurable and controllable
stochasticity which is complemented with the development of reliable
statistical models of such observed stochasticity. Current research literature
has sparse evidence of the former and a complete lack of the latter. This
motivates the current article where we demonstrate a stochastic neuron using an
insulator-metal-transition (IMT) device, based on electrically induced
phase-transition, in series with a tunable resistance. We show that an IMT
neuron has dynamics similar to a piecewise linear FitzHugh-Nagumo (FHN) neuron
and incorporates all characteristics of a spiking neuron in the device
phenomena. We experimentally demonstrate spontaneous stochastic spiking along
with electrically controllable firing probabilities using Vanadium Dioxide
(VO$_2$) based IMT neurons which show a sigmoid-like transfer function. The
stochastic spiking is explained by two noise sources - thermal noise and
threshold fluctuations, which act as precursors of bifurcation. As such, the
IMT neuron is modeled as an Ornstein-Uhlenbeck (OU) process with a fluctuating
boundary resulting in transfer curves that closely match experiments. As one of
the first comprehensive studies of a stochastic neuron hardware and its
statistical properties, this article would enable efficient implementation of a
large class of neuro-mimetic networks and algorithms.
Learning Word Vectors for 157 Languages
Authors: Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, Tomas Mikolov
Date: 2018-02-19 00:00:00
Proceeding: LREC 2018 5
Tasks: Only Connect Walls Dataset Task 1 (Grouping)

Distributed word representations, or word vectors, have recently been applied
to many tasks in natural language processing, leading to state-of-the-art
performance. A key ingredient to the successful application of these
representations is to train them on very large corpora, and use these
pre-trained models in downstream tasks. In this paper, we describe how we
trained such high quality word representations for 157 languages. We used two
sources of data to train these models: the free online encyclopedia Wikipedia
and data from the common crawl project. We also introduce three new word
analogy datasets to evaluate these word vectors, for French, Hindi and Polish.
Finally, we evaluate our pre-trained word vectors on 10 languages for which
evaluation datasets exists, showing very strong performance compared to
previous models.
Unsupervised Predictive Memory in a Goal-Directed Agent
Authors: Greg Wayne, Chia-Chun Hung, David Amos, Mehdi Mirza, Arun Ahuja, Agnieszka Grabska-Barwinska, Jack Rae, Piotr Mirowski, Joel Z. Leibo, Adam Santoro, Mevlana Gemici, Malcolm Reynolds, Tim Harley, Josh Abramson, Shakir Mohamed, Danilo Rezende, David Saxton, Adam Cain, Chloe Hillier, David Silver, Koray Kavukcuoglu, Matt Botvinick, Demis Hassabis, Timothy Lillicrap
Date: 2018-03-28 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

Animals execute goal-directed behaviours despite the limited range and scope
of their sensors. To cope, they explore environments and store memories
maintaining estimates of important information that is not presently available.
Recently, progress has been made with artificial intelligence (AI) agents that
learn to perform tasks from sensory input, even at a human level, by merging
reinforcement learning (RL) algorithms with deep neural networks, and the
excitement surrounding these results has led to the pursuit of related ideas as
explanations of non-human animal learning. However, we demonstrate that
contemporary RL algorithms struggle to solve simple tasks when enough
information is concealed from the sensors of the agent, a property called
"partial observability". An obvious requirement for handling partially observed
tasks is access to extensive memory, but we show memory is not enough; it is
critical that the right information be stored in the right format. We develop a
model, the Memory, RL, and Inference Network (MERLIN), in which memory
formation is guided by a process of predictive modeling. MERLIN facilitates the
solution of tasks in 3D virtual reality environments for which partial
observability is severe and memories must be maintained over long durations.
Our model demonstrates a single learning agent architecture that can solve
canonical behavioural tasks in psychology and neurobiology without strong
simplifying assumptions about the dimensionality of sensory input or the
duration of experiences.
Human Interaction with Recommendation Systems
Authors: Sven Schmit, Carlos Riquelme
Date: 2017-03-01 00:00:00
Tasks: Recommendation Systems

Many recommendation algorithms rely on user data to generate recommendations.
However, these recommendations also affect the data obtained from future users.
This work aims to understand the effects of this dynamic interaction. We
propose a simple model where users with heterogeneous preferences arrive over
time. Based on this model, we prove that naive estimators, i.e. those which
ignore this feedback loop, are not consistent. We show that consistent
estimators are efficient in the presence of myopic agents. Our results are
validated using extensive simulations.
Adversarial Network Compression
Authors: Vasileios Belagiannis, Azade Farshad, Fabio Galasso
Date: 2018-03-28 00:00:00
Tasks: Neural Network Compression, Transfer Learning

Neural network compression has recently received much attention due to the
computational requirements of modern deep models. In this work, our objective
is to transfer knowledge from a deep and accurate model to a smaller one. Our
contributions are threefold: (i) we propose an adversarial network compression
approach to train the small student network to mimic the large teacher, without
the need for labels during training; (ii) we introduce a regularization scheme
to prevent a trivially-strong discriminator without reducing the network
capacity and (iii) our approach generalizes on different teacher-student
models.
  In an extensive evaluation on five standard datasets, we show that our
student has small accuracy drop, achieves better performance than other
knowledge transfer approaches and it surpasses the performance of the same
network trained with labels. In addition, we demonstrate state-of-the-art
results compared to other compression strategies.
Pseudo-marginal Bayesian inference for supervised Gaussian process latent variable models
Authors: Charles Gadd, Sara Wade, Akeel Shah, Dimitris Grammatopoulos
Date: 2018-03-28 00:00:00
Tasks: Bayesian Inference, Uncertainty Quantification, Variational Inference

We introduce a Bayesian framework for inference with a supervised version of
the Gaussian process latent variable model. The framework overcomes the high
correlations between latent variables and hyperparameters by using an unbiased
pseudo estimate for the marginal likelihood that approximately integrates over
the latent variables. This is used to construct a Markov Chain to explore the
posterior of the hyperparameters. We demonstrate the procedure on simulated and
real examples, showing its ability to capture uncertainty and multimodality of
the hyperparameters and improved uncertainty quantification in predictions when
compared with variational inference.
Pattern Analysis with Layered Self-Organizing Maps
Authors: David Friedlander
Date: 2018-03-23 00:00:00

This paper defines a new learning architecture, Layered Self-Organizing Maps
(LSOMs), that uses the SOM and supervised-SOM learning algorithms. The
architecture is validated with the MNIST database of hand-written digit images.
LSOMs are similar to convolutional neural nets (covnets) in the way they sample
data, but different in the way they represent features and learn. LSOMs analyze
(or generate) image patches with maps of exemplars determined by the SOM
learning algorithm rather than feature maps from filter-banks learned via
backprop.
  LSOMs provide an alternative to features derived from covnets. Multi-layer
LSOMs are trained bottom-up, without the use of backprop and therefore may be
of interest as a model of the visual cortex. The results show organization at
multiple levels. The algorithm appears to be resource efficient in learning,
classifying and generating images. Although LSOMs can be used for
classification, their validation accuracy for these exploratory runs was well
below the state of the art. The goal of this article is to define the
architecture and display the structures resulting from its application to the
MNIST images.
Disentangling 3D Pose in A Dendritic CNN for Unconstrained 2D Face Alignment
Authors: Amit Kumar, Rama Chellappa
Date: 2018-02-19 00:00:00
Proceeding: CVPR 2018 6
Tasks: Face Alignment, General Classification, regression

Heatmap regression has been used for landmark localization for quite a while
now. Most of the methods use a very deep stack of bottleneck modules for
heatmap classification stage, followed by heatmap regression to extract the
keypoints. In this paper, we present a single dendritic CNN, termed as Pose
Conditioned Dendritic Convolution Neural Network (PCD-CNN), where a
classification network is followed by a second and modular classification
network, trained in an end to end fashion to obtain accurate landmark points.
Following a Bayesian formulation, we disentangle the 3D pose of a face image
explicitly by conditioning the landmark estimation on pose, making it different
from multi-tasking approaches. Extensive experimentation shows that
conditioning on pose reduces the localization error by making it agnostic to
face pose. The proposed model can be extended to yield variable number of
landmark points and hence broadening its applicability to other datasets.
Instead of increasing depth or width of the network, we train the CNN
efficiently with Mask-Softmax Loss and hard sample mining to achieve upto
$15\%$ reduction in error compared to state-of-the-art methods for extreme and
medium pose face images from challenging datasets including AFLW, AFW, COFW and
IBUG.
What is the Machine Learning?
Authors: Spencer Chang, Timothy Cohen, Bryan Ostdiek
Date: 2017-09-28 00:00:00
Tasks: BIG-bench Machine Learning, Physical Intuition, Sensitivity

Applications of machine learning tools to problems of physical interest are
often criticized for producing sensitivity at the expense of transparency. To
address this concern, we explore a data planing procedure for identifying
combinations of variables -- aided by physical intuition -- that can
discriminate signal from background. Weights are introduced to smooth away the
features in a given variable(s). New networks are then trained on this modified
data. Observed decreases in sensitivity diagnose the variable's discriminating
power. Planing also allows the investigation of the linear versus non-linear
nature of the boundaries between signal and background. We demonstrate the
efficacy of this approach using a toy example, followed by an application to an
idealized heavy resonance scenario at the Large Hadron Collider. By unpacking
the information being utilized by these algorithms, this method puts in context
what it means for a machine to learn.
(Machine) Learning to Do More with Less
Authors: Timothy Cohen, Marat Freytsis, Bryan Ostdiek
Date: 2017-06-28 00:00:00
Tasks: BIG-bench Machine Learning

Determining the best method for training a machine learning algorithm is
critical to maximizing its ability to classify data. In this paper, we compare
the standard "fully supervised" approach (that relies on knowledge of
event-by-event truth-level labels) with a recent proposal that instead utilizes
class ratios as the only discriminating information provided during training.
This so-called "weakly supervised" technique has access to less information
than the fully supervised method and yet is still able to yield impressive
discriminating power. In addition, weak supervision seems particularly well
suited to particle physics since quantum mechanics is incompatible with the
notion of mapping an individual event onto any single Feynman diagram. We
examine the technique in detail -- both analytically and numerically -- with a
focus on the robustness to issues of mischaracterizing the training samples.
Weakly supervised networks turn out to be remarkably insensitive to systematic
mismodeling. Furthermore, we demonstrate that the event level outputs for
weakly versus fully supervised networks are probing different kinematics, even
though the numerical quality metrics are essentially identical. This implies
that it should be possible to improve the overall classification ability by
combining the output from the two types of networks. For concreteness, we apply
this technology to a signature of beyond the Standard Model physics to
demonstrate that all these impressive features continue to hold in a scenario
of relevance to the LHC.