pwc-archive/papers-with-abstracts huggingface dataset:
Relaxed Quantization for Discretized Neural Networks
Authors: Christos Louizos, Matthias Reisser, Tijmen Blankevoort, Efstratios Gavves, Max Welling
Date: 2018-10-03 00:00:00
Proceeding: ICLR 2019 5
Tasks: General Classification, Quantization

Neural network quantization has become an important research area due to its
great impact on deployment of large models on resource constrained devices. In
order to train networks that can be effectively discretized without loss of
performance, we introduce a differentiable quantization procedure.
Differentiability can be achieved by transforming continuous distributions over
the weights and activations of the network to categorical distributions over
the quantization grid. These are subsequently relaxed to continuous surrogates
that can allow for efficient gradient-based optimization. We further show that
stochastic rounding can be seen as a special case of the proposed approach and
that under this formulation the quantization grid itself can also be optimized
with gradient descent. We experimentally validate the performance of our method
on MNIST, CIFAR 10 and Imagenet classification.
Combining Natural Gradient with Hessian Free Methods for Sequence Training
Authors: Adnan Haider, P. C. Woodland
Date: 2018-10-03 00:00:00
Tasks: speech-recognition, Speech Recognition

This paper presents a new optimisation approach to train Deep Neural Networks
(DNNs) with discriminative sequence criteria. At each iteration, the method
combines information from the Natural Gradient (NG) direction with local
curvature information of the error surface that enables better paths on the
parameter manifold to be traversed. The method is derived using an alternative
derivation of Taylor's theorem using the concepts of manifolds, tangent vectors
and directional derivatives from the perspective of Information Geometry. The
efficacy of the method is shown within a Hessian Free (HF) style optimisation
framework to sequence train both standard fully-connected DNNs and Time Delay
Neural Networks as speech recognition acoustic models. It is shown that for the
same number of updates the proposed approach achieves larger reductions in the
word error rate (WER) than both NG and HF, and also leads to a lower WER than
standard stochastic gradient descent. The paper also addresses the issue of
over-fitting due to mismatch between training criterion and Word Error Rate
(WER) that primarily arises during sequence training of ReLU-DNN models.
Learning agent's spatial configuration from sensorimotor invariants
Authors: Alban Laflaquière, J. Kevin O'Regan, Sylvain Argentieri, Bruno Gas, Alexander V. Terekhov
Date: 2018-10-03 00:00:00

The design of robotic systems is largely dictated by our purely human
intuition about how we perceive the world. This intuition has been proven
incorrect with regard to a number of critical issues, such as visual change
blindness. In order to develop truly autonomous robots, we must step away from
this intuition and let robotic agents develop their own way of perceiving. The
robot should start from scratch and gradually develop perceptual notions, under
no prior assumptions, exclusively by looking into its sensorimotor experience
and identifying repetitive patterns and invariants. One of the most fundamental
perceptual notions, space, cannot be an exception to this requirement. In this
paper we look into the prerequisites for the emergence of simplified spatial
notions on the basis of a robot's sensorimotor flow. We show that the notion of
space as environment-independent cannot be deduced solely from exteroceptive
information, which is highly variable and is mainly determined by the contents
of the environment. The environment-independent definition of space can be
approached by looking into the functions that link the motor commands to
changes in exteroceptive inputs. In a sufficiently rich environment, the
kernels of these functions correspond uniquely to the spatial configuration of
the agent's exteroceptors. We simulate a redundant robotic arm with a retina
installed at its end-point and show how this agent can learn the configuration
space of its retina. The resulting manifold has the topology of the Cartesian
product of a plane and a circle, and corresponds to the planar position and
orientation of the retina.
Grounding the Experience of a Visual Field through Sensorimotor Contingencies
Authors: Alban Laflaquière
Date: 2018-10-03 00:00:00

Artificial perception is traditionally handled by hand-designing task
specific algorithms. However, a truly autonomous robot should develop
perceptive abilities on its own, by interacting with its environment, and
adapting to new situations. The sensorimotor contingencies theory proposes to
ground the development of those perceptive abilities in the way the agent can
actively transform its sensory inputs. We propose a sensorimotor approach,
inspired by this theory, in which the agent explores the world and discovers
its properties by capturing the sensorimotor regularities they induce. This
work presents an application of this approach to the discovery of a so-called
visual field as the set of regularities that a visual sensor imposes on a naive
agent's experience. A formalism is proposed to describe how those regularities
can be captured in a sensorimotor predictive model. Finally, the approach is
evaluated on a simulated system coarsely inspired from the human retina.
Grounding Perception: A Developmental Approach to Sensorimotor Contingencies
Authors: Alban Laflaquière, Nikolas Hemion, Michaël Garcia Ortiz, Jean-Christophe Baillie
Date: 2018-10-03 00:00:00

Sensorimotor contingency theory offers a promising account of the nature of
perception, a topic rarely addressed in the robotics community. We propose a
developmental framework to address the problem of the autonomous acquisition of
sensorimotor contingencies by a naive robot. While exploring the world, the
robot internally encodes contingencies as predictive models that capture the
structure they imply in its sensorimotor experience. Three preliminary
applications are presented to illustrate our approach to the acquisition of
perceptive abilities: discovering the environment, discovering objects, and
discovering a visual field.
Machine Learning Suites for Online Toxicity Detection
Authors: David Noever
Date: 2018-10-03 00:00:00
Tasks: BIG-bench Machine Learning

To identify and classify toxic online commentary, the modern tools of data
science transform raw text into key features from which either thresholding or
learning algorithms can make predictions for monitoring offensive
conversations. We systematically evaluate 62 classifiers representing 19 major
algorithmic families against features extracted from the Jigsaw dataset of
Wikipedia comments. We compare the classifiers based on statistically
significant differences in accuracy and relative execution time. Among these
classifiers for identifying toxic comments, tree-based algorithms provide the
most transparently explainable rules and rank-order the predictive contribution
of each feature. Among 28 features of syntax, sentiment, emotion and outlier
word dictionaries, a simple bad word list proves most predictive of offensive
commentary.
Set Aggregation Network as a Trainable Pooling Layer
Authors: Łukasz Maziarka, Marek Śmieja, Aleksandra Nowak, Jacek Tabor, Łukasz Struski, Przemysław Spurek
Date: 2018-10-03 00:00:00

Global pooling, such as max- or sum-pooling, is one of the key ingredients in deep neural networks used for processing images, texts, graphs and other types of structured data. Based on the recent DeepSets architecture proposed by Zaheer et al. (NIPS 2017), we introduce a Set Aggregation Network (SAN) as an alternative global pooling layer. In contrast to typical pooling operators, SAN allows to embed a given set of features to a vector representation of arbitrary size. We show that by adjusting the size of embedding, SAN is capable of preserving the whole information from the input. In experiments, we demonstrate that replacing global pooling layer by SAN leads to the improvement of classification accuracy. Moreover, it is less prone to overfitting and can be used as a regularizer.
A Non-linear Approach to Space Dimension Perception by a Naive Agent
Authors: Alban Laflaquière, Sylvain Argentieri, Olivia Breysse, Stéphane Genet, Bruno Gas
Date: 2018-10-03 00:00:00

Developmental Robotics offers a new approach to numerous AI features that are
often taken as granted. Traditionally, perception is supposed to be an inherent
capacity of the agent. Moreover, it largely relies on models built by the
system's designer. A new approach is to consider perception as an
experimentally acquired ability that is learned exclusively through the
analysis of the agent's sensorimotor flow. Previous works, based on
H.Poincar\'e's intuitions and the sensorimotor contingencies theory, allow a
simulated agent to extract the dimension of geometrical space in which it is
immersed without any a priori knowledge. Those results are limited to
infinitesimal movement's amplitude of the system. In this paper, a non-linear
dimension estimation method is proposed to push back this limitation.
Learning an internal representation of the end-effector configuration space
Authors: Alban Laflaquière, Alexander V. Terekhov, Bruno Gas, J. Kevin O'Regan
Date: 2018-10-03 00:00:00
Tasks: BIG-bench Machine Learning, Position

Current machine learning techniques proposed to automatically discover a
robot kinematics usually rely on a priori information about the robot's
structure, sensors properties or end-effector position. This paper proposes a
method to estimate a certain aspect of the forward kinematics model with no
such information. An internal representation of the end-effector configuration
is generated from unstructured proprioceptive and exteroceptive data flow under
very limited assumptions. A mapping from the proprioceptive space to this
representational space can then be used to control the robot.
Robust identification of thermal models for in-production High-Performance-Computing clusters with machine learning-based data selection
Authors: Federico Pittino, Roberto Diversi, Luca Benini, Andrea Bartolini
Date: 2018-10-03 00:00:00
Tasks: Management, Quantization

Power and thermal management are critical components of
High-Performance-Computing (HPC) systems, due to their high power density and
large total power consumption. The assessment of thermal dissipation by means
of compact models directly from the thermal response of the final device
enables more robust and precise thermal control strategies as well as automated
diagnosis. However, when dealing with large scale systems "in production", the
accuracy of learned thermal models depends on the dynamics of the power
excitation, which depends also on the executed workload, and measurement
nonidealities, such as quantization. In this paper we show that, using an
advanced system identification algorithm, we are able to generate very accurate
thermal models (average error lower than our sensors quantization step of
1{\deg}C) for a large scale HPC system on real workloads for very long time
periods. However, we also show that: 1) not all real workloads allow for the
identification of a good model; 2) starting from the theory of system
identification it is very difficult to evaluate if a trace of data leads to a
good estimated model. We then propose and validate a set of techniques based on
machine learning and deep learning algorithms for the choice of data traces to
be used for model identification. We also show that deep learning techniques
are absolutely necessary to correctly choose such traces up to 96% of the
times.
Agnostic Sample Compression Schemes for Regression
Authors: Idan Attias, Steve Hanneke, Aryeh Kontorovich, Menachem Sadigurschi
Date: 2018-10-03 00:00:00
Tasks: Open-Ended Question Answering, regression

We obtain the first positive results for bounded sample compression in the agnostic regression setting with the $\ell_p$ loss, where $p\in [1,\infty]$. We construct a generic approximate sample compression scheme for real-valued function classes exhibiting exponential size in the fat-shattering dimension but independent of the sample size. Notably, for linear regression, an approximate compression of size linear in the dimension is constructed. Moreover, for $\ell_1$ and $\ell_\infty$ losses, we can even exhibit an efficient exact sample compression scheme of size linear in the dimension. We further show that for every other $\ell_p$ loss, $p\in (1,\infty)$, there does not exist an exact agnostic compression scheme of bounded size. This refines and generalizes a negative result of David, Moran, and Yehudayoff for the $\ell_2$ loss. We close by posing general open questions: for agnostic regression with $\ell_1$ loss, does every function class admits an exact compression scheme of size equal to its pseudo-dimension? For the $\ell_2$ loss, does every function class admit an approximate compression scheme of polynomial size in the fat-shattering dimension? These questions generalize Warmuth's classic sample compression conjecture for realizable-case classification.
A Comprehensive Approach for Learning-based Fully-Automated Inter-slice Motion Correction for Short-Axis Cine Cardiac MR Image Stacks
Authors: Giacomo Tarroni, Ozan Oktay, Matthew Sinclair, Wenjia Bai, Andreas Schuh, Hideaki Suzuki, Antonio de Marvao, Declan O'Regan, Stuart Cook, Daniel Rueckert
Date: 2018-10-03 00:00:00
Tasks: Motion Compensation

In the clinical routine, short axis (SA) cine cardiac MR (CMR) image stacks
are acquired during multiple subsequent breath-holds. If the patient cannot
consistently hold the breath at the same position, the acquired image stack
will be affected by inter-slice respiratory motion and will not correctly
represent the cardiac volume, introducing potential errors in the following
analyses and visualisations. We propose an approach to automatically correct
inter-slice respiratory motion in SA CMR image stacks. Our approach makes use
of probabilistic segmentation maps (PSMs) of the left ventricular (LV) cavity
generated with decision forests. PSMs are generated for each slice of the SA
stack and rigidly registered in-plane to a target PSM. If long axis (LA) images
are available, PSMs are generated for them and combined to create the target
PSM; if not, the target PSM is produced from the same stack using a 3D model
trained from motion-free stacks. The proposed approach was tested on a dataset
of SA stacks acquired from 24 healthy subjects (for which anatomical 3D cardiac
images were also available as reference) and compared to two techniques which
use LA intensity images and LA segmentations as targets, respectively. The
results show the accuracy and robustness of the proposed approach in motion
compensation.
Inhibited Softmax for Uncertainty Estimation in Neural Networks
Authors: Marcin Możejko, Mateusz Susik, Rafał Karczewski
Date: 2018-10-03 00:00:00
Tasks: Out-of-Distribution Detection, Sentiment Analysis

We present a new method for uncertainty estimation and out-of-distribution
detection in neural networks with softmax output. We extend softmax layer with
an additional constant input. The corresponding additional output is able to
represent the uncertainty of the network. The proposed method requires neither
additional parameters nor multiple forward passes nor input preprocessing nor
out-of-distribution datasets. We show that our method performs comparably to
more computationally expensive methods and outperforms baselines on our
experiments from image recognition and sentiment analysis domains.
GINN: Geometric Illustration of Neural Networks
Authors: Luke N. Darlow, Amos J. Storkey
Date: 2018-10-02 00:00:00

This informal technical report details the geometric illustration of decision
boundaries for ReLU units in a three layer fully connected neural network. The
network is designed and trained to predict pixel intensity from an (x, y) input
location. The Geometric Illustration of Neural Networks (GINN) tool was built
to visualise and track the points at which ReLU units switch from being active
to off (or vice versa) as the network undergoes training. Several phenomenon
were observed and are discussed herein. This technical report is a supporting
document to the blog post with online demos and is available at
http://www.bayeswatch.com/2018/09/17/GINN/.
Contextual Multi-Armed Bandits for Causal Marketing
Authors: Neela Sawant, Chitti Babu Namballa, Narayanan Sadagopan, Houssam Nassif
Date: 2018-10-02 00:00:00
Tasks: Causal Inference, counterfactual, Marketing, Multi-Armed Bandits, Thompson Sampling

This work explores the idea of a causal contextual multi-armed bandit
approach to automated marketing, where we estimate and optimize the causal
(incremental) effects. Focusing on causal effect leads to better return on
investment (ROI) by targeting only the persuadable customers who wouldn't have
taken the action organically. Our approach draws on strengths of causal
inference, uplift modeling, and multi-armed bandits. It optimizes on causal
treatment effects rather than pure outcome, and incorporates counterfactual
generation within data collection. Following uplift modeling results, we
optimize over the incremental business metric. Multi-armed bandit methods allow
us to scale to multiple treatments and to perform off-policy policy evaluation
on logged data. The Thompson sampling strategy in particular enables
exploration of treatments on similar customer contexts and materialization of
counterfactual outcomes. Preliminary offline experiments on a retail Fashion
marketing dataset show merits of our proposal.
Representation Flow for Action Recognition
Authors: AJ Piergiovanni, Michael S. Ryoo
Date: 2018-10-02 00:00:00
Proceeding: CVPR 2019 6
Tasks: Action Classification, Action Recognition, Action Recognition In Videos, Activity Recognition, Activity Recognition In Videos, Optical Flow Estimation, Temporal Action Localization, Video Classification, Video Understanding

In this paper, we propose a convolutional layer inspired by optical flow algorithms to learn motion representations. Our representation flow layer is a fully-differentiable layer designed to capture the `flow' of any representation channel within a convolutional neural network for action recognition. Its parameters for iterative flow optimization are learned in an end-to-end fashion together with the other CNN model parameters, maximizing the action recognition performance. Furthermore, we newly introduce the concept of learning `flow of flow' representations by stacking multiple representation flow layers. We conducted extensive experimental evaluations, confirming its advantages over previous recognition models using traditional optical flows in both computational speed and performance. Code/models available here: https://piergiaj.github.io/rep-flow-site/
Scientific image rendering for space scenes with the SurRender software
Authors: Roland Brochard, Jérémy Lebreton, Cyril Robin, Keyvan Kanani, Grégory Jonniaux, Aurore Masson, Noela Despré, Ahmad Berjaoui
Date: 2018-10-02 00:00:00

Spacecraft autonomy can be enhanced by vision-based navigation (VBN)
techniques. Applications range from manoeuvers around Solar System objects and
landing on planetary surfaces, to in-orbit servicing or space debris removal.
The development and validation of VBN algorithms relies on the availability of
physically accurate relevant images. Yet archival data from past missions can
rarely serve this purpose and acquiring new data is often costly. The SurRender
software is an image simulator that addresses the challenges of realistic image
rendering, with high representativeness for space scenes. Images are rendered
by raytracing, which implements the physical principles of geometrical light
propagation, in physical units. A macroscopic instrument model and scene
objects reflectance functions are used. SurRender is specially optimized for
space scenes, with huge distances between objects and scenes up to Solar System
size. Raytracing conveniently tackles some important effects for VBN
algorithms: image quality, eclipses, secondary illumination, subpixel limb
imaging, etc. A simulation is easily setup (in MATLAB, Python, and more) by
specifying the position of the bodies (camera, Sun, planets, satellites) over
time, 3D shapes and material surface properties. SurRender comes with its own
modelling tool enabling to go beyond existing models for shapes, materials and
sensors (projection, temporal sampling, electronics, etc.). It is natively
designed to simulate different kinds of sensors (visible, LIDAR, etc.). Tools
are available for manipulating huge datasets to store albedo maps and digital
elevation models, or for procedural (fractal) texturing that generates
high-quality images for a large range of observing distances (from millions of
km to touchdown). We illustrate SurRender performances with a selection of case
studies, placing particular emphasis on a 900-km Moon flyby simulation.
Rough set based lattice structure for knowledge representation in medical expert systems: low back pain management case study
Authors: Debarpita Santra, Swapan Kumar Basu, Jyotsna Kumar Mandal, Subrata Goswami
Date: 2018-10-02 00:00:00
Tasks: Computational Efficiency, Management

The aim of medical knowledge representation is to capture the detailed domain
knowledge in a clinically efficient manner and to offer a reliable resolution
with the acquired knowledge. The knowledge base to be used by a medical expert
system should allow incremental growth with inclusion of updated knowledge over
the time. As knowledge are gathered from a variety of knowledge sources by
different knowledge engineers, the problem of redundancy is an important
concern here due to increased processing time of knowledge and occupancy of
large computational storage to accommodate all the gathered knowledge. Also
there may exist many inconsistent knowledge in the knowledge base. In this
paper, we have proposed a rough set based lattice structure for knowledge
representation in medical expert systems which overcomes the problem of
redundancy and inconsistency in knowledge and offers computational efficiency
with respect to both time and space. We have also generated an optimal set of
decision rules that would be used directly by the inference engine. The
reliability of each rule has been measured using a new metric called
credibility factor, and the certainty and coverage factors of a decision rule
have been re-defined. With a set of decisions rules arranged in descending
order according to their reliability measures, the medical expert system will
consider the highly reliable and certain rules at first, then it would search
for the possible and uncertain rules at later stage, if recommended by
physicians. The proposed knowledge representation technique has been
illustrated using an example from the domain of low back pain. The proposed
scheme ensures completeness, consistency, integrity, non-redundancy, and ease
of access.