pwc-archive/papers-with-abstracts huggingface dataset:
Sim2Real View Invariant Visual Servoing by Recurrent Control
Authors: Fereshteh Sadeghi, Alexander Toshev, Eric Jang, Sergey Levine
Date: 2017-12-20 00:00:00
Tasks: Reinforcement Learning

Humans are remarkably proficient at controlling their limbs and tools from a
wide range of viewpoints and angles, even in the presence of optical
distortions. In robotics, this ability is referred to as visual servoing:
moving a tool or end-point to a desired location using primarily visual
feedback. In this paper, we study how viewpoint-invariant visual servoing
skills can be learned automatically in a robotic manipulation scenario. To this
end, we train a deep recurrent controller that can automatically determine
which actions move the end-point of a robotic arm to a desired object. The
problem that must be solved by this controller is fundamentally ambiguous:
under severe variation in viewpoint, it may be impossible to determine the
actions in a single feedforward operation. Instead, our visual servoing system
must use its memory of past movements to understand how the actions affect the
robot motion from the current viewpoint, correcting mistakes and gradually
moving closer to the target. This ability is in stark contrast to most visual
servoing methods, which either assume known dynamics or require a calibration
phase. We show how we can learn this recurrent controller using simulated data
and a reinforcement learning objective. We then describe how the resulting
model can be transferred to a real-world robot by disentangling perception from
control and only adapting the visual layers. The adapted model can servo to
previously unseen objects from novel viewpoints on a real-world Kuka IIWA
robotic arm. For supplementary videos, see:
https://fsadeghi.github.io/Sim2RealViewInvariantServo
Image Segmentation to Distinguish Between Overlapping Human Chromosomes
Authors: R. Lily Hu, Jeremy Karnowski, Ross Fadely, Jean-Patrick Pommier
Date: 2017-12-20 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

In medicine, visualizing chromosomes is important for medical diagnostics,
drug development, and biomedical research. Unfortunately, chromosomes often
overlap and it is necessary to identify and distinguish between the overlapping
chromosomes. A segmentation solution that is fast and automated will enable
scaling of cost effective medicine and biomedical research. We apply neural
network-based image segmentation to the problem of distinguishing between
partially overlapping DNA chromosomes. A convolutional neural network is
customized for this problem. The results achieved intersection over union (IOU)
scores of 94.7% for the overlapping region and 88-94% on the non-overlapping
chromosome regions.
Deep Learning with Lung Segmentation and Bone Shadow Exclusion Techniques for Chest X-Ray Analysis of Lung Cancer
Authors: Yu. Gordienko, Peng Gang, Jiang Hui, Wei Zeng, Yu. Kochura, O. Alienin, O. Rokovyi, S. Stirenko
Date: 2017-12-20 00:00:00
Tasks: Segmentation

The recent progress of computing, machine learning, and especially deep
learning, for image recognition brings a meaningful effect for automatic
detection of various diseases from chest X-ray images (CXRs). Here efficiency
of lung segmentation and bone shadow exclusion techniques is demonstrated for
analysis of 2D CXRs by deep learning approach to help radiologists identify
suspicious lesions and nodules in lung cancer patients. Training and validation
was performed on the original JSRT dataset (dataset #01), BSE-JSRT dataset,
i.e. the same JSRT dataset, but without clavicle and rib shadows (dataset #02),
original JSRT dataset after segmentation (dataset #03), and BSE-JSRT dataset
after segmentation (dataset #04). The results demonstrate the high efficiency
and usefulness of the considered pre-processing techniques in the simplified
configuration even. The pre-processed dataset without bones (dataset #02)
demonstrates the much better accuracy and loss results in comparison to the
other pre-processed datasets after lung segmentation (datasets #02 and #03).
Improving Generalization Performance by Switching from Adam to SGD
Authors: Nitish Shirish Keskar, Richard Socher
Date: 2017-12-20 00:00:00
Tasks: Language Modeling, Language Modelling

Despite superior training outcomes, adaptive optimization methods such as
Adam, Adagrad or RMSprop have been found to generalize poorly compared to
Stochastic gradient descent (SGD). These methods tend to perform well in the
initial portion of training but are outperformed by SGD at later stages of
training. We investigate a hybrid strategy that begins training with an
adaptive method and switches to SGD when appropriate. Concretely, we propose
SWATS, a simple strategy which switches from Adam to SGD when a triggering
condition is satisfied. The condition we propose relates to the projection of
Adam steps on the gradient subspace. By design, the monitoring process for this
condition adds very little overhead and does not increase the number of
hyperparameters in the optimizer. We report experiments on several standard
benchmarks such as: ResNet, SENet, DenseNet and PyramidNet for the CIFAR-10 and
CIFAR-100 data sets, ResNet on the tiny-ImageNet data set and language modeling
with recurrent networks on the PTB and WT2 data sets. The results show that our
strategy is capable of closing the generalization gap between SGD and Adam on a
majority of the tasks.
SRPGAN: Perceptual Generative Adversarial Network for Single Image Super Resolution
Authors: Bingzhe Wu, Haodong Duan, Zhichao Liu, Guangyu Sun
Date: 2017-12-16 00:00:00
Tasks: Generative Adversarial Network, Image Super-Resolution, SSIM, Super-Resolution

Single image super resolution (SISR) is to reconstruct a high resolution
image from a single low resolution image. The SISR task has been a very
attractive research topic over the last two decades. In recent years,
convolutional neural network (CNN) based models have achieved great performance
on SISR task. Despite the breakthroughs achieved by using CNN models, there are
still some problems remaining unsolved, such as how to recover high frequency
details of high resolution images. Previous CNN based models always use a pixel
wise loss, such as l2 loss. Although the high resolution images constructed by
these models have high peak signal-to-noise ratio (PSNR), they often tend to be
blurry and lack high-frequency details, especially at a large scaling factor.
In this paper, we build a super resolution perceptual generative adversarial
network (SRPGAN) framework for SISR tasks. In the framework, we propose a
robust perceptual loss based on the discriminator of the built SRPGAN model. We
use the Charbonnier loss function to build the content loss and combine it with
the proposed perceptual loss and the adversarial loss. Compared with other
state-of-the-art methods, our method has demonstrated great ability to
construct images with sharp edges and rich details. We also evaluate our method
on different benchmarks and compare it with previous CNN based methods. The
results show that our method can achieve much higher structural similarity
index (SSIM) scores on most of the benchmarks than the previous state-of-art
methods.
Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations
Authors: Boris Hanin
Date: 2017-08-09 00:00:00

This article concerns the expressive power of depth in neural nets with ReLU
activations and bounded width. We are particularly interested in the following
questions: what is the minimal width $w_{\text{min}}(d)$ so that ReLU nets of
width $w_{\text{min}}(d)$ (and arbitrary depth) can approximate any continuous
function on the unit cube $[0,1]^d$ aribitrarily well? For ReLU nets near this
minimal width, what can one say about the depth necessary to approximate a
given function? Our approach to this paper is based on the observation that,
due to the convexity of the ReLU activation, ReLU nets are particularly
well-suited for representing convex functions. In particular, we prove that
ReLU nets with width $d+1$ can approximate any continuous convex function of
$d$ variables arbitrarily well. These results then give quantitative depth
estimates for the rate of approximation of any continuous scalar function on
the $d$-dimensional cube $[0,1]^d$ by ReLU nets with width $d+3.$
Predicting readmission risk from doctors' notes
Authors: Erin Craig, Carlos Arias, David Gillman
Date: 2017-11-29 00:00:00
Tasks: Deep Learning

We develop a model using deep learning techniques and natural language
processing on unstructured text from medical records to predict hospital-wide
$30$-day unplanned readmission, with c-statistic $.70$. Our model is
constructed to allow physicians to interpret the significant features for
prediction.
Experimental Phase Estimation Enhanced By Machine Learning
Authors: Alessandro Lumino, Emanuele Polino, Adil S. Rab, Giorgio Milani, Nicolò Spagnolo, Nathan Wiebe, Fabio Sciarrino
Date: 2017-12-20 00:00:00
Tasks: BIG-bench Machine Learning

Phase estimation protocols provide a fundamental benchmark for the field of
quantum metrology. The latter represents one of the most relevant applications
of quantum theory, potentially enabling the capability of measuring unknown
physical parameters with improved precision over classical strategies. Within
this context, most theoretical and experimental studies have focused on
determining the fundamental bounds and how to achieve them in the asymptotic
regime where a large number of resources is employed. However, in most
applications it is necessary to achieve optimal precisions by performing only a
limited number of measurements. To this end, machine learning techniques can be
applied as a powerful optimization tool. Here, we implement experimentally
single-photon adaptive phase estimation protocols enhanced by machine learning,
showing the capability of reaching optimal precision after a small number of
trials. In particular, we introduce a new approach for Bayesian estimation that
exhibit best performances for very low number of photons N. Furthermore, we
study the resilience to noise of the tested methods, showing that the optimized
Bayesian approach is very robust in the presence of imperfections. Application
of this methodology can be envisaged in the more general multiparameter case,
that represents a paradigmatic scenario for several tasks including imaging or
Hamiltonian learning.
An Ensemble Model with Ranking for Social Dialogue
Authors: Ioannis Papaioannou, Amanda Cercas Curry, Jose L. Part, Igor Shalyminov, Xinnuo Xu, Yanchao Yu, Ondřej Dušek, Verena Rieser, Oliver Lemon
Date: 2017-12-20 00:00:00
Tasks: model

Open-domain social dialogue is one of the long-standing goals of Artificial
Intelligence. This year, the Amazon Alexa Prize challenge was announced for the
first time, where real customers get to rate systems developed by leading
universities worldwide. The aim of the challenge is to converse "coherently and
engagingly with humans on popular topics for 20 minutes". We describe our Alexa
Prize system (called 'Alana') consisting of an ensemble of bots, combining
rule-based and machine learning systems, and using a contextual ranking
mechanism to choose a system response. The ranker was trained on real user
feedback received during the competition, where we address the problem of how
to train on the noisy and sparse feedback obtained during the competition.
VAMPnets: Deep learning of molecular kinetics
Authors: Andreas Mardt, Luca Pasquali, Hao Wu, Frank Noé
Date: 2017-10-16 00:00:00
Tasks: Clustering, Deep Learning, Dimensionality Reduction

There is an increasing demand for computing the relevant structures,
equilibria and long-timescale kinetics of biomolecular processes, such as
protein-drug binding, from high-throughput molecular dynamics simulations.
Current methods employ transformation of simulated coordinates into structural
features, dimension reduction, clustering the dimension-reduced data, and
estimation of a Markov state model or related model of the interconversion
rates between molecular structures. This handcrafted approach demands a
substantial amount of modeling expertise, as poor decisions at any step will
lead to large modeling errors. Here we employ the variational approach for
Markov processes (VAMP) to develop a deep learning framework for molecular
kinetics using neural networks, dubbed VAMPnets. A VAMPnet encodes the entire
mapping from molecular coordinates to Markov states, thus combining the whole
data processing pipeline in a single end-to-end framework. Our method performs
equally or better than state-of-the art Markov modeling methods and provides
easily interpretable few-state kinetic models.
Discrete approximations of the affine Gaussian derivative model for visual receptive fields
Authors: Tony Lindeberg
Date: 2017-01-09 00:00:00

The affine Gaussian derivative model can in several respects be regarded as a
canonical model for receptive fields over a spatial image domain: (i) it can be
derived by necessity from scale-space axioms that reflect structural properties
of the world, (ii) it constitutes an excellent model for the receptive fields
of simple cells in the primary visual cortex and (iii) it is covariant under
affine image deformations, which enables more accurate modelling of image
measurements under the local image deformations caused by the perspective
mapping, compared to the more commonly used Gaussian derivative model based on
derivatives of the rotationally symmetric Gaussian kernel.
  This paper presents a theory for discretizing the affine Gaussian scale-space
concept underlying the affine Gaussian derivative model, so that scale-space
properties hold also for the discrete implementation.
  Two ways of discretizing spatial smoothing with affine Gaussian kernels are
presented: (i) by solving semi-discretized affine diffusion equation, which has
derived by necessity from the requirements of a semi-group structure over scale
as parameterized by a family of spatial covariance matrices and obeying
non-creation of new structures from any finer to any coarser scale in terms of
non-enhancement of local extrema and (ii) approximating these semi-discrete
affine receptive fields by parameterized families of 3x3-kernels as obtained
from an additional discretization along the scale direction. The latter
discrete approach can be optionally complemented by spatial subsampling at
coarser scales, leading to the notion of affine hybrid pyramids.
  Using these theoretical results, we outline hybrid architectures for discrete
approximations of affine covariant receptive field families, to be used as a
first processing layer for affine covariant and affine invariant visual
operations at higher levels.
Use of Deep Learning in Modern Recommendation System: A Summary of Recent Works
Authors: Ayush Singhal, Pradeep Sinha, Rakesh Pant
Date: 2017-12-20 00:00:00
Tasks: Deep Learning, Information Retrieval, Recommendation Systems, Retrieval, speech-recognition, Speech Recognition

With the exponential increase in the amount of digital information over the
internet, online shops, online music, video and image libraries, search engines
and recommendation system have become the most convenient ways to find relevant
information within a short time. In the recent times, deep learning's advances
have gained significant attention in the field of speech recognition, image
processing and natural language processing. Meanwhile, several recent studies
have shown the utility of deep learning in the area of recommendation systems
and information retrieval as well. In this short review, we cover the recent
advances made in the field of recommendation using various variants of deep
learning technology. We organize the review in three parts: Collaborative
system, Content based system and Hybrid system. The review also discusses the
contribution of deep learning integrated recommendation systems into several
application domains. The review concludes by discussion of the impact of deep
learning in recommendation system in various domain and whether deep learning
has shown any significant improvement over the conventional systems for
recommendation. Finally, we also provide future directions of research which
are possible based on the current state of use of deep learning in
recommendation systems.
Statistical Inference for the Population Landscape via Moment Adjusted Stochastic Gradients
Authors: Tengyuan Liang, Weijie Su
Date: 2017-12-20 00:00:00
Tasks: Stochastic Optimization, Uncertainty Quantification

Modern statistical inference tasks often require iterative optimization
methods to compute the solution. Convergence analysis from an optimization
viewpoint only informs us how well the solution is approximated numerically but
overlooks the sampling nature of the data. In contrast, recognizing the
randomness in the data, statisticians are keen to provide uncertainty
quantification, or confidence, for the solution obtained using iterative
optimization methods. This paper makes progress along this direction by
introducing the moment-adjusted stochastic gradient descents, a new stochastic
optimization method for statistical inference. We establish non-asymptotic
theory that characterizes the statistical distribution for certain iterative
methods with optimization guarantees. On the statistical front, the theory
allows for model mis-specification, with very mild conditions on the data. For
optimization, the theory is flexible for both convex and non-convex cases.
Remarkably, the moment-adjusting idea motivated from "error standardization" in
statistics achieves a similar effect as acceleration in first-order
optimization methods used to fit generalized linear models. We also demonstrate
this acceleration effect in the non-convex setting through numerical
experiments.
Ethical Questions in NLP Research: The (Mis)-Use of Forensic Linguistics
Authors: Anil Kumar Singh, Akhilesh Sudhakar
Date: 2017-12-20 00:00:00
Tasks: BIG-bench Machine Learning

Ideas from forensic linguistics are now being used frequently in Natural
Language Processing (NLP), using machine learning techniques. While the role of
forensic linguistics was more benign earlier, it is now being used for purposes
which are questionable. Certain methods from forensic linguistics are employed,
without considering their scientific limitations and ethical concerns. While we
take the specific case of forensic linguistics as an example of such trends in
NLP and machine learning, the issue is a larger one and present in many other
scientific and data-driven domains. We suggest that such trends indicate that
some of the applied sciences are exceeding their legal and scientific briefs.
We highlight how carelessly implemented practices are serving to short-circuit
the due processes of law as well breach ethical codes.
The Recycling Gibbs Sampler for Efficient Learning
Authors: Luca Martino, Victor Elvira, Gustau Camps-Valls
Date: 2016-11-21 00:00:00
Tasks: Bayesian Inference, Computational Efficiency, Gaussian Processes

Monte Carlo methods are essential tools for Bayesian inference. Gibbs
sampling is a well-known Markov chain Monte Carlo (MCMC) algorithm, extensively
used in signal processing, machine learning, and statistics, employed to draw
samples from complicated high-dimensional posterior distributions. The key
point for the successful application of the Gibbs sampler is the ability to
draw efficiently samples from the full-conditional probability density
functions. Since in the general case this is not possible, in order to speed up
the convergence of the chain, it is required to generate auxiliary samples
whose information is eventually disregarded. In this work, we show that these
auxiliary samples can be recycled within the Gibbs estimators, improving their
efficiency with no extra cost. This novel scheme arises naturally after
pointing out the relationship between the standard Gibbs sampler and the chain
rule used for sampling purposes. Numerical simulations involving simple and
real inference problems confirm the excellent performance of the proposed
scheme in terms of accuracy and computational efficiency. In particular we give
empirical evidence of performance in a toy example, inference of Gaussian
processes hyperparameters, and learning dependence graphs through regression.
Monocular Dense 3D Reconstruction of a Complex Dynamic Scene from Two Perspective Frames
Authors: Suryansh Kumar, Yuchao Dai, Hongdong Li
Date: 2017-08-15 00:00:00
Proceeding: ICCV 2017 10
Tasks: 3D Reconstruction, Dynamic Reconstruction, Segmentation, Semantic Segmentation, Superpixels

This paper proposes a new approach for monocular dense 3D reconstruction of a
complex dynamic scene from two perspective frames. By applying superpixel
over-segmentation to the image, we model a generically dynamic (hence
non-rigid) scene with a piecewise planar and rigid approximation. In this way,
we reduce the dynamic reconstruction problem to a "3D jigsaw puzzle" problem
which takes pieces from an unorganized "soup of superpixels". We show that our
method provides an effective solution to the inherent relative scale ambiguity
in structure-from-motion. Since our method does not assume a template prior, or
per-object segmentation, or knowledge about the rigidity of the dynamic scene,
it is applicable to a wide range of scenarios. Extensive experiments on both
synthetic and real monocular sequences demonstrate the superiority of our
method compared with the state-of-the-art methods.
Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
Authors: Takuhiro Kaneko, Hirokazu Kameoka
Date: 2017-11-30 00:00:00
Tasks: Voice Conversion

We propose a parallel-data-free voice-conversion (VC) method that can learn a
mapping from source to target speech without relying on parallel data. The
proposed method is general purpose, high quality, and parallel-data free and
works without any extra data, modules, or alignment procedure. It also avoids
over-smoothing, which occurs in many conventional statistical model-based VC
methods. Our method, called CycleGAN-VC, uses a cycle-consistent adversarial
network (CycleGAN) with gated convolutional neural networks (CNNs) and an
identity-mapping loss. A CycleGAN learns forward and inverse mappings
simultaneously using adversarial and cycle-consistency losses. This makes it
possible to find an optimal pseudo pair from unpaired data. Furthermore, the
adversarial loss contributes to reducing over-smoothing of the converted
feature sequence. We configure a CycleGAN with gated CNNs and train it with an
identity-mapping loss. This allows the mapping function to capture sequential
and hierarchical structures while preserving linguistic information. We
evaluated our method on a parallel-data-free VC task. An objective evaluation
showed that the converted feature sequence was near natural in terms of global
variance and modulation spectra. A subjective evaluation showed that the
quality of the converted speech was comparable to that obtained with a Gaussian
mixture model-based method under advantageous conditions with parallel and
twice the amount of data.