pwc-archive/papers-with-abstracts huggingface dataset:
Neural Conditional Gradients
Authors: Patrick Schramowski, Christian Bauckhage, Kristian Kersting
Date: 2018-03-12 00:00:00

The move from hand-designed to learned optimizers in machine learning has
been quite successful for gradient-based and -free optimizers. When facing a
constrained problem, however, maintaining feasibility typically requires a
projection step, which might be computationally expensive and not
differentiable. We show how the design of projection-free convex optimization
algorithms can be cast as a learning problem based on Frank-Wolfe Networks:
recurrent networks implementing the Frank-Wolfe algorithm aka. conditional
gradients. This allows them to learn to exploit structure when, e.g.,
optimizing over rank-1 matrices. Our LSTM-learned optimizers outperform
hand-designed as well learned but unconstrained ones. We demonstrate this for
training support vector machines and softmax classifiers.
Wasserstein Auto-Encoders
Authors: Ilya Tolstikhin, Olivier Bousquet, Sylvain Gelly, Bernhard Schoelkopf
Date: 2017-11-05 00:00:00
Proceeding: ICLR 2018 1
Tasks: Decoder

We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Auto-Encoder (VAE). This regularizer encourages the encoded training distribution to match the prior. We compare our algorithm with several other techniques and show that it is a generalization of adversarial auto-encoders (AAE). Our experiments show that WAE shares many of the properties of VAEs (stable training, encoder-decoder architecture, nice latent manifold structure) while generating samples of better quality, as measured by the FID score.
Layered TPOT: Speeding up Tree-based Pipeline Optimization
Authors: Pieter Gijsbers, Joaquin Vanschoren, Randal S. Olson
Date: 2018-01-18 00:00:00
Tasks: Automated Feature Engineering, AutoML, BIG-bench Machine Learning, Hyperparameter Optimization

With the demand for machine learning increasing, so does the demand for tools
which make it easier to use. Automated machine learning (AutoML) tools have
been developed to address this need, such as the Tree-Based Pipeline
Optimization Tool (TPOT) which uses genetic programming to build optimal
pipelines. We introduce Layered TPOT, a modification to TPOT which aims to
create pipelines equally good as the original, but in significantly less time.
This approach evaluates candidate pipelines on increasingly large subsets of
the data according to their fitness, using a modified evolutionary algorithm to
allow for separate competition between pipelines trained on different sample
sizes. Empirical evaluation shows that, on sufficiently large datasets, Layered
TPOT indeed finds better models faster.
EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
Authors: Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, Kostas Daniilidis
Date: 2018-02-19 00:00:00
Tasks: Optical Flow Estimation

Event-based cameras have shown great promise in a variety of situations where
frame based cameras suffer, such as high speed motions and high dynamic range
scenes. However, developing algorithms for event measurements requires a new
class of hand crafted algorithms. Deep learning has shown great success in
providing model free solutions to many problems in the vision community, but
existing networks have been developed with frame based images in mind, and
there does not exist the wealth of labeled data for events as there does for
images for supervised training. To these points, we present EV-FlowNet, a novel
self-supervised deep learning pipeline for optical flow estimation for event
based cameras. In particular, we introduce an image based representation of a
given event stream, which is fed into a self-supervised neural network as the
sole input. The corresponding grayscale images captured from the same camera at
the same time as the events are then used as a supervisory signal to provide a
loss function at training time, given the estimated flow from the network. We
show that the resulting network is able to accurately predict optical flow from
events only in a variety of different scenes, with performance competitive to
image based networks. This method not only allows for accurate estimation of
dense optical flow, but also provides a framework for the transfer of other
self-supervised methods to the event-based domain.
In-depth Assessment of an Interactive Graph-based Approach for the Segmentation for Pancreatic Metastasis in Ultrasound Acquisitions of the Liver with two Specialists in Internal Medicine
Authors: Jan Egger, Xiaojun Chen, Lucas Bettac, Mark Hänle, Tilmann Gräter, Wolfram Zoller, Dieter Schmalstieg, Alexander Hann
Date: 2018-03-12 00:00:00
Tasks: Segmentation

The manual outlining of hepatic metastasis in (US) ultrasound acquisitions
from patients suffering from pancreatic cancer is common practice. However,
such pure manual measurements are often very time consuming, and the results
repeatedly differ between the raters. In this contribution, we study the
in-depth assessment of an interactive graph-based approach for the segmentation
for pancreatic metastasis in US images of the liver with two specialists in
Internal Medicine. Thereby, evaluating the approach with over one hundred
different acquisitions of metastases. The two physicians or the algorithm had
never assessed the acquisitions before the evaluation. In summary, the
physicians first performed a pure manual outlining followed by an algorithmic
segmentation over one month later. As a result, the experts satisfied in up to
ninety percent of algorithmic segmentation results. Furthermore, the
algorithmic segmentation was much faster than manual outlining and achieved a
median Dice Similarity Coefficient (DSC) of over eighty percent. Ultimately,
the algorithm enables a fast and accurate segmentation of liver metastasis in
clinical US images, which can support the manual outlining in daily practice.
From Monocular SLAM to Autonomous Drone Exploration
Authors: Lukas von Stumberg, Vladyslav Usenko, Jakob Engel, Jörg Stückler, Daniel Cremers
Date: 2016-09-26 00:00:00
Tasks: Autonomous Navigation

Micro aerial vehicles (MAVs) are strongly limited in their payload and power
capacity. In order to implement autonomous navigation, algorithms are therefore
desirable that use sensory equipment that is as small, low-weight, and
low-power consuming as possible. In this paper, we propose a method for
autonomous MAV navigation and exploration using a low-cost consumer-grade
quadrocopter equipped with a monocular camera. Our vision-based navigation
system builds on LSD-SLAM which estimates the MAV trajectory and a semi-dense
reconstruction of the environment in real-time. Since LSD-SLAM only determines
depth at high gradient pixels, texture-less areas are not directly observed so
that previous exploration methods that assume dense map information cannot
directly be applied. We propose an obstacle mapping and exploration approach
that takes the properties of our semi-dense monocular SLAM system into account.
In experiments, we demonstrate our vision-based autonomous navigation and
exploration system with a Parrot Bebop MAV.
Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network
Authors: Charis Lanaras, José Bioucas-Dias, Silvano Galliani, Emmanuel Baltsavias, Konrad Schindler
Date: 2018-03-12 00:00:00
Tasks: Super-Resolution

The Sentinel-2 satellite mission delivers multi-spectral imagery with 13
spectral bands, acquired at three different spatial resolutions. The aim of
this research is to super-resolve the lower-resolution (20 m and 60 m Ground
Sampling Distance - GSD) bands to 10 m GSD, so as to obtain a complete data
cube at the maximal sensor resolution. We employ a state-of-the-art
convolutional neural network (CNN) to perform end-to-end upsampling, which is
trained with data at lower resolution, i.e., from 40->20 m, respectively
360->60 m GSD. In this way, one has access to a virtually infinite amount of
training data, by downsampling real Sentinel-2 images. We use data sampled
globally over a wide range of geographical locations, to obtain a network that
generalises across different climate zones and land-cover types, and can
super-resolve arbitrary Sentinel-2 images without the need of retraining. In
quantitative evaluations (at lower scale, where ground truth is available), our
network, which we call DSen2, outperforms the best competing approach by almost
50% in RMSE, while better preserving the spectral characteristics. It also
delivers visually convincing results at the full 10 m GSD. The code is
available at https://github.com/lanha/DSen2
Neural network feedback controller for inertial platform
Authors: Yan Anisimov, Alexandr Lysov, Dmitry Katsai
Date: 2018-03-07 00:00:00

The paper describes an algorithm for the synthesis of neural networks to
control gyro stabilizer. The neural network performs the role of observer for
state vector. The role of an observer in a feedback of gyro stabilizer is
illustrated. Paper detail a problem specific features stage of classics
algorithm: choosing of network architecture, learning of neural network and
verification of result feedback control. In the article presented optimal
configuration of the neural network like a memory depth, the number of layers
and neuron in these layers and activation functions in layers. Using the
information of dynamic system for improving learning of neural network is
provided. A scheme creation of an optimal training sample is provided.
FeTa: A DCA Pruning Algorithm with Generalization Error Guarantees
Authors: Konstantinos Pitas, Mike Davies, Pierre Vandergheynst
Date: 2018-03-12 00:00:00
Tasks: General Classification

Recent DNN pruning algorithms have succeeded in reducing the number of
parameters in fully connected layers, often with little or no drop in
classification accuracy. However, most of the existing pruning schemes either
have to be applied during training or require a costly retraining procedure
after pruning to regain classification accuracy. We start by proposing a cheap
pruning algorithm for fully connected DNN layers based on difference of convex
functions (DC) optimisation, that requires little or no retraining. We then
provide a theoretical analysis for the growth in the Generalization Error (GE)
of a DNN for the case of bounded perturbations to the hidden layers, of which
weight pruning is a special case. Our pruning method is orders of magnitude
faster than competing approaches, while our theoretical analysis sheds light to
previously observed problems in DNN pruning. Experiments on commnon feedforward
neural networks validate our results.
The Contextual Loss for Image Transformation with Non-Aligned Data
Authors: Roey Mechrez, Itamar Talmi, Lihi Zelnik-Manor
Date: 2018-03-06 00:00:00
Proceeding: ECCV 2018 9

Feed-forward CNNs trained for image transformation problems rely on loss
functions that measure the similarity between the generated image and a target
image. Most of the common loss functions assume that these images are spatially
aligned and compare pixels at corresponding locations. However, for many tasks,
aligned training pairs of images will not be available. We present an
alternative loss function that does not require alignment, thus providing an
effective and simple solution for a new space of problems. Our loss is based on
both context and semantics -- it compares regions with similar semantic
meaning, while considering the context of the entire image. Hence, for example,
when transferring the style of one face to another, it will translate
eyes-to-eyes and mouth-to-mouth. Our code can be found at
https://www.github.com/roimehrez/contextualLoss
Variational Inference for Gaussian Process with Panel Count Data
Authors: Hongyi Ding, Young Lee, Issei Sato, Masashi Sugiyama
Date: 2018-03-12 00:00:00
Tasks: Variational Inference

We present the first framework for Gaussian-process-modulated Poisson
processes when the temporal data appear in the form of panel counts. Panel
count data frequently arise when experimental subjects are observed only at
discrete time points and only the numbers of occurrences of the events between
subsequent observation times are available. The exact occurrence timestamps of
the events are unknown. The method of conducting the efficient variational
inference is presented, based on the assumption of a Gaussian-process-modulated
intensity function. We derive a tractable lower bound to alleviate the problems
of the intractable evidence lower bound inherent in the variational inference
framework. Our algorithm outperforms classical methods on both synthetic and
three real panel count sets.
Omnidirectional CNN for Visual Place Recognition and Navigation
Authors: Tsun-Hsuan Wang, Hung-Jui Huang, Juan-Ting Lin, Chan-Wei Hu, Kuo-Hao Zeng, Min Sun
Date: 2018-03-12 00:00:00
Tasks: Navigate, Visual Place Recognition

$ $Visual place recognition is challenging, especially when only a few place
exemplars are given. To mitigate the challenge, we consider place recognition
method using omnidirectional cameras and propose a novel Omnidirectional
Convolutional Neural Network (O-CNN) to handle severe camera pose variation.
Given a visual input, the task of the O-CNN is not to retrieve the matched
place exemplar, but to retrieve the closest place exemplar and estimate the
relative distance between the input and the closest place. With the ability to
estimate relative distance, a heuristic policy is proposed to navigate a robot
to the retrieved closest place. Note that the network is designed to take
advantage of the omnidirectional view by incorporating circular padding and
rotation invariance. To train a powerful O-CNN, we build a virtual world for
training on a large scale. We also propose a continuous lifted structured
feature embedding loss to learn the concept of distance efficiently. Finally,
our experimental results confirm that our method achieves state-of-the-art
accuracy and speed with both the virtual world and real-world datasets.
Leveraging Crowdsourcing Data For Deep Active Learning - An Application: Learning Intents in Alexa
Authors: Jie Yang, Thomas Drake, Andreas Damianou, Yoelle Maarek
Date: 2018-03-12 00:00:00
Tasks: Active Learning, Deep Learning, intent-classification, Intent Classification

This paper presents a generic Bayesian framework that enables any deep
learning model to actively learn from targeted crowds. Our framework inherits
from recent advances in Bayesian deep learning, and extends existing work by
considering the targeted crowdsourcing approach, where multiple annotators with
unknown expertise contribute an uncontrolled amount (often limited) of
annotations. Our framework leverages the low-rank structure in annotations to
learn individual annotator expertise, which then helps to infer the true labels
from noisy and sparse annotations. It provides a unified Bayesian model to
simultaneously infer the true labels and train the deep learning model in order
to reach an optimal learning efficacy. Finally, our framework exploits the
uncertainty of the deep learning model during prediction as well as the
annotators' estimated expertise to minimize the number of required annotations
and annotators for optimally training the deep learning model.
  We evaluate the effectiveness of our framework for intent classification in
Alexa (Amazon's personal assistant), using both synthetic and real-world
datasets. Experiments show that our framework can accurately learn annotator
expertise, infer true labels, and effectively reduce the amount of annotations
in model training as compared to state-of-the-art approaches. We further
discuss the potential of our proposed framework in bridging machine learning
and crowdsourcing towards improved human-in-the-loop systems.
Deep Face Feature for Face Alignment
Authors: Boyi Jiang, Juyong Zhang, Bailin Deng, Yudong Guo, Ligang Liu
Date: 2017-08-09 00:00:00
Tasks: Face Alignment, Inverse Rendering, Robust Face Alignment

In this paper, we present a deep learning based image feature extraction
method designed specifically for face images. To train the feature extraction
model, we construct a large scale photo-realistic face image dataset with
ground-truth correspondence between multi-view face images, which are
synthesized from real photographs via an inverse rendering procedure. The deep
face feature (DFF) is trained using correspondence between face images rendered
from different views. Using the trained DFF model, we can extract a feature
vector for each pixel of a face image, which distinguishes different facial
regions and is shown to be more effective than general-purpose feature
descriptors for face-related tasks such as matching and alignment. Based on the
DFF, we develop a robust face alignment method, which iteratively updates
landmarks, pose and 3D shape. Extensive experiments demonstrate that our method
can achieve state-of-the-art results for face alignment under highly
unconstrained face images.
High Throughput Synchronous Distributed Stochastic Gradient Descent
Authors: Michael Teng, Frank Wood
Date: 2018-03-12 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We introduce a new, high-throughput, synchronous, distributed, data-parallel,
stochastic-gradient-descent learning algorithm. This algorithm uses amortized
inference in a compute-cluster-specific, deep, generative, dynamical model to
perform joint posterior predictive inference of the mini-batch gradient
computation times of all worker-nodes in a parallel computing cluster. We show
that a synchronous parameter server can, by utilizing such a model, choose an
optimal cutoff time beyond which mini-batch gradient messages from slow workers
are ignored that maximizes overall mini-batch gradient computations per second.
In keeping with earlier findings we observe that, under realistic conditions,
eagerly discarding the mini-batch gradient computations of stragglers not only
increases throughput but actually increases the overall rate of convergence as
a function of wall-clock time by virtue of eliminating idleness. The principal
novel contribution and finding of this work goes beyond this by demonstrating
that using the predicted run-times from a generative model of cluster worker
performance to dynamically adjust the cutoff improves substantially over the
static-cutoff prior art, leading to, among other things, significantly reduced
deep neural net training times on large computer clusters.
Semiparametric Contextual Bandits
Authors: Akshay Krishnamurthy, Zhiwei Steven Wu, Vasilis Syrgkanis
Date: 2018-03-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Multi-Armed Bandits

This paper studies semiparametric contextual bandits, a generalization of the
linear stochastic bandit problem where the reward for an action is modeled as a
linear function of known action features confounded by an non-linear
action-independent term. We design new algorithms that achieve
$\tilde{O}(d\sqrt{T})$ regret over $T$ rounds, when the linear function is
$d$-dimensional, which matches the best known bounds for the simpler
unconfounded case and improves on a recent result of Greenewald et al. (2017).
Via an empirical evaluation, we show that our algorithms outperform prior
approaches when there are non-linear confounding effects on the rewards.
Technically, our algorithms use a new reward estimator inspired by
doubly-robust approaches and our proofs require new concentration inequalities
for self-normalized martingales.
Automated detection and segmentation of non-mass enhancing breast tumors with dynamic contrast-enhanced magnetic resonance imaging
Authors: Ignacio Alvarez Illan, Javier Ramirez, Juan M. Gorriz, Maria Adele Marino, Daly Avendaño, Thomas Helbich, Pascal Baltzer, Katja Pinker, Anke Meyer-Baese
Date: 2018-03-12 00:00:00
Tasks: Diagnostic

Non-mass enhancing lesions (NME) constitute a diagnostic challenge in dynamic
contrast enhanced magnetic resonance imaging (DCE-MRI) of the breast. Computer
Aided Diagnosis (CAD) systems provide physicians with advanced tools for
analysis, assessment and evaluation that have a significant impact on the
diagnostic performance. Here, we propose a new approach to address the
challenge of NME detection and segmentation, taking advantage of independent
component analysis (ICA) to extract data-driven dynamic lesion
characterizations. A set of independent sources was obtained from DCE-MRI
dataset of breast patients, and the dynamic behavior of the different tissues
was described by multiple dynamic curves, together with a set of eigenimages
describing the scores for each voxel. A new test image is projected onto the
independent source space using the unmixing matrix, and each voxel is
classified by a support vector machine (SVM) that has already been trained with
manually delineated data. A solution to the high false positive rate problem is
proposed by controlling the SVM hyperplane location, outperforming previously
published approaches.
Multi-kernel Regression For Graph Signal Processing
Authors: Arun Venkitaraman, Saikat Chatterjee, Peter Händel
Date: 2018-03-12 00:00:00
Tasks: regression

We develop a multi-kernel based regression method for graph signal processing
where the target signal is assumed to be smooth over a graph. In multi-kernel
regression, an effective kernel function is expressed as a linear combination
of many basis kernel functions. We estimate the linear weights to learn the
effective kernel function by appropriate regularization based on graph
smoothness. We show that the resulting optimization problem is shown to be
convex and pro- pose an accelerated projected gradient descent based solution.
Simulation results using real-world graph signals show efficiency of the
multi-kernel based approach over a standard kernel based approach.
Extreme Learning Machine for Graph Signal Processing
Authors: Arun Venkitaraman, Saikat Chatterjee, Peter Händel
Date: 2018-03-12 00:00:00
Tasks: regression

In this article, we improve extreme learning machines for regression tasks
using a graph signal processing based regularization. We assume that the target
signal for prediction or regression is a graph signal. With this assumption, we
use the regularization to enforce that the output of an extreme learning
machine is smooth over a given graph. Simulation results with real data confirm
that such regularization helps significantly when the available training data
is limited in size and corrupted by noise.
Noise2Noise: Learning Image Restoration without Clean Data
Authors: Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, Timo Aila
Date: 2018-03-12 00:00:00
Proceeding: ICML 2018 7
Tasks: BIG-bench Machine Learning, Denoising, Image Restoration, Salt-And-Pepper Noise Removal

We apply basic statistical reasoning to signal reconstruction by machine
learning -- learning to map corrupted observations to clean signals -- with a
simple and powerful conclusion: it is possible to learn to restore images by
only looking at corrupted examples, at performance at and sometimes exceeding
training using clean data, without explicit image priors or likelihood models
of the corruption. In practice, we show that a single model learns photographic
noise removal, denoising synthetic Monte Carlo images, and reconstruction of
undersampled MRI scans -- all corrupted by different processes -- based on
noisy data only.