pwc-archive/papers-with-abstracts huggingface dataset:
StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks
Authors: Hirokazu Kameoka, Takuhiro Kaneko, Kou Tanaka, Nobukatsu Hojo
Date: 2018-06-06 00:00:00
Tasks: Attribute, Generative Adversarial Network, Voice Conversion

This paper proposes a method that allows non-parallel many-to-many voice
conversion (VC) by using a variant of a generative adversarial network (GAN)
called StarGAN. Our method, which we call StarGAN-VC, is noteworthy in that it
(1) requires no parallel utterances, transcriptions, or time alignment
procedures for speech generator training, (2) simultaneously learns
many-to-many mappings across different attribute domains using a single
generator network, (3) is able to generate converted speech signals quickly
enough to allow real-time implementations and (4) requires only several minutes
of training examples to generate reasonably realistic-sounding speech.
Subjective evaluation experiments on a non-parallel many-to-many speaker
identity conversion task revealed that the proposed method obtained higher
sound quality and speaker similarity than a state-of-the-art method based on
variational autoencoding GANs.
Pattern Search Multidimensional Scaling
Authors: Georgios Paraskevopoulos, Efthymios Tzinis, Emmanouil-Vasileios Vlatakis-Gkaragkounis, Alexandros Potamianos
Date: 2018-06-01 00:00:00

We present a novel view of nonlinear manifold learning using derivative-free optimization techniques. Specifically, we propose an extension of the classical multi-dimensional scaling (MDS) method, where instead of performing gradient descent, we sample and evaluate possible "moves" in a sphere of fixed radius for each point in the embedded space. A fixed-point convergence guarantee can be shown by formulating the proposed algorithm as an instance of General Pattern Search (GPS) framework. Evaluation on both clean and noisy synthetic datasets shows that pattern search MDS can accurately infer the intrinsic geometry of manifolds embedded in high-dimensional spaces. Additionally, experiments on real data, even under noisy conditions, demonstrate that the proposed pattern search MDS yields state-of-the-art results.
Learning Representations for Counterfactual Inference
Authors: Fredrik D. Johansson, Uri Shalit, David Sontag
Date: 2016-05-12 00:00:00
Tasks: Causal Inference, counterfactual, Counterfactual Inference, Domain Adaptation, Representation Learning

Observational studies are rising in importance due to the widespread
accumulation of data in fields such as healthcare, education, employment and
ecology. We consider the task of answering counterfactual questions such as,
"Would this patient have lower blood sugar had she received a different
medication?". We propose a new algorithmic framework for counterfactual
inference which brings together ideas from domain adaptation and representation
learning. In addition to a theoretical justification, we perform an empirical
comparison with previous approaches to causal inference from observational
data. Our deep learning algorithm significantly outperforms the previous
state-of-the-art.
Optimization Landscape and Expressivity of Deep CNNs
Authors: Quynh Nguyen, Matthias Hein
Date: 2017-10-30 00:00:00
Proceeding: ICML 2018 7

We analyze the loss landscape and expressiveness of practical deep
convolutional neural networks (CNNs) with shared weights and max pooling
layers. We show that such CNNs produce linearly independent features at a
"wide" layer which has more neurons than the number of training samples. This
condition holds e.g. for the VGG network. Furthermore, we provide for such wide
CNNs necessary and sufficient conditions for global minima with zero training
error. For the case where the wide layer is followed by a fully connected layer
we show that almost every critical point of the empirical loss is a global
minimum with zero training error. Our analysis suggests that both depth and
width are very important in deep learning. While depth brings more
representational power and allows the network to learn high level features,
width smoothes the optimization landscape of the loss function in the sense
that a sufficiently wide network has a well-behaved loss surface with almost no
bad local minima.
Bayesian Quadrature for Multiple Related Integrals
Authors: Xiaoyue Xi, François-Xavier Briol, Mark Girolami
Date: 2018-01-12 00:00:00
Proceeding: ICML 2018 7

Bayesian probabilistic numerical methods are a set of tools providing
posterior distributions on the output of numerical methods. The use of these
methods is usually motivated by the fact that they can represent our
uncertainty due to incomplete/finite information about the continuous
mathematical problem being approximated. In this paper, we demonstrate that
this paradigm can provide additional advantages, such as the possibility of
transferring information between several numerical methods. This allows users
to represent uncertainty in a more faithful manner and, as a by-product,
provide increased numerical efficiency. We propose the first such numerical
method by extending the well-known Bayesian quadrature algorithm to the case
where we are interested in computing the integral of several related functions.
We then prove convergence rates for the method in the well-specified and
misspecified cases, and demonstrate its efficiency in the context of
multi-fidelity models for complex engineering systems and a problem of global
illumination in computer graphics.
Neural Taylor Approximations: Convergence and Exploration in Rectifier Networks
Authors: David Balduzzi, Brian McWilliams, Tony Butler-Yeoman
Date: 2016-11-07 00:00:00
Proceeding: ICML 2017 8

Modern convolutional networks, incorporating rectifiers and max-pooling, are
neither smooth nor convex; standard guarantees therefore do not apply.
Nevertheless, methods from convex optimization such as gradient descent and
Adam are widely used as building blocks for deep learning algorithms. This
paper provides the first convergence guarantee applicable to modern convnets,
which furthermore matches a lower bound for convex nonsmooth functions. The key
technical tool is the neural Taylor approximation -- a straightforward
application of Taylor expansions to neural networks -- and the associated
Taylor loss. Experiments on a range of optimizers, layers, and tasks provide
evidence that the analysis accurately captures the dynamics of neural
optimization. The second half of the paper applies the Taylor approximation to
isolate the main difficulty in training rectifier nets -- that gradients are
shattered -- and investigates the hypothesis that, by exploring the space of
activation configurations more thoroughly, adaptive optimizers such as RMSProp
and Adam are able to converge to better solutions.
Adversarial Auto-encoders for Speech Based Emotion Recognition
Authors: Saurabh Sahu, Rahul Gupta, Ganesh Sivaraman, Wael Abd-Almageed, Carol Espy-Wilson
Date: 2018-06-06 00:00:00
Tasks: Emotion Recognition, Face Recognition

Recently, generative adversarial networks and adversarial autoencoders have
gained a lot of attention in machine learning community due to their
exceptional performance in tasks such as digit classification and face
recognition. They map the autoencoder's bottleneck layer output (termed as code
vectors) to different noise Probability Distribution Functions (PDFs), that can
be further regularized to cluster based on class information. In addition, they
also allow a generation of synthetic samples by sampling the code vectors from
the mapped PDFs. Inspired by these properties, we investigate the application
of adversarial autoencoders to the domain of emotion recognition. Specifically,
we conduct experiments on the following two aspects: (i) their ability to
encode high dimensional feature vector representations for emotional utterances
into a compressed space (with a minimal loss of emotion class discriminability
in the compressed space), and (ii) their ability to regenerate synthetic
samples in the original feature space, to be later used for purposes such as
training emotion recognition classifiers. We demonstrate the promise of
adversarial autoencoders with regards to these aspects on the Interactive
Emotional Dyadic Motion Capture (IEMOCAP) corpus and present our analysis.
Strongly-Typed Agents are Guaranteed to Interact Safely
Authors: David Balduzzi
Date: 2017-02-24 00:00:00
Proceeding: ICML 2017 8
Tasks: blind source separation, Common Sense Reasoning, Tensor Decomposition

As artificial agents proliferate, it is becoming increasingly important to
ensure that their interactions with one another are well-behaved. In this
paper, we formalize a common-sense notion of when algorithms are well-behaved:
an algorithm is safe if it does no harm. Motivated by recent progress in deep
learning, we focus on the specific case where agents update their actions
according to gradient descent. The paper shows that that gradient descent
converges to a Nash equilibrium in safe games. The main contribution is to
define strongly-typed agents and show they are guaranteed to interact safely,
thereby providing sufficient conditions to guarantee safe interactions. A
series of examples show that strong-typing generalizes certain key features of
convexity, is closely related to blind source separation, and introduces a new
perspective on classical multilinear games based on tensor decomposition.
Multi-chart Generative Surface Modeling
Authors: Heli Ben-Hamu, Haggai Maron, Itay Kezurer, Gal Avineri, Yaron Lipman
Date: 2018-06-06 00:00:00
Tasks: 3D Shape Generation, Translation

This paper introduces a 3D shape generative model based on deep neural
networks. A new image-like (i.e., tensor) data representation for genus-zero 3D
shapes is devised. It is based on the observation that complicated shapes can
be well represented by multiple parameterizations (charts), each focusing on a
different part of the shape. The new tensor data representation is used as
input to Generative Adversarial Networks for the task of 3D shape generation.
The 3D shape tensor representation is based on a multi-chart structure that
enjoys a shape covering property and scale-translation rigidity.
Scale-translation rigidity facilitates high quality 3D shape learning and
guarantees unique reconstruction. The multi-chart structure uses as input a
dataset of 3D shapes (with arbitrary connectivity) and a sparse correspondence
between them. The output of our algorithm is a generative model that learns the
shape distribution and is able to generate novel shapes, interpolate shapes,
and explore the generated shape space. The effectiveness of the method is
demonstrated for the task of anatomic shape generation including human body and
bone (teeth) shape generation.
A New Framework for Machine Intelligence: Concepts and Prototype
Authors: Abel Torres Montoya
Date: 2018-06-06 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning (ML) and artificial intelligence (AI) have become hot topics
in many information processing areas, from chatbots to scientific data
analysis. At the same time, there is uncertainty about the possibility of
extending predominant ML technologies to become general solutions with
continuous learning capabilities. Here, a simple, yet comprehensive,
theoretical framework for intelligent systems is presented. A combination of
Mirror Compositional Representations (MCR) and a Solution-Critic Loop (SCL) is
proposed as a generic approach for different types of problems. A prototype
implementation is presented for document comparison using English Wikipedia
corpus.
Efficient Differentiable Programming in a Functional Array-Processing Language
Authors: Amir Shaikhha, Andrew Fitzgibbon, Dimitrios Vytiniotis, Simon Peyton Jones, Christoph Koch
Date: 2018-06-06 00:00:00
Tasks: BIG-bench Machine Learning

We present a system for the automatic differentiation of a higher-order
functional array-processing language. The core functional language underlying
this system simultaneously supports both source-to-source automatic
differentiation and global optimizations such as loop transformations. Thanks
to this feature, we demonstrate how for some real-world machine learning and
computer vision benchmarks, the system outperforms the state-of-the-art
automatic differentiation tools.
Addendum to "HTN Acting: A Formalism and an Algorithm"
Authors: Lavindra de Silva
Date: 2018-06-06 00:00:00

Hierarchical Task Network (HTN) planning is a practical and efficient approach to planning when the 'standard operating procedures' for a domain are available. Like Belief-Desire-Intention (BDI) agent reasoning, HTN planning performs hierarchical and context-based refinement of goals into subgoals and basic actions. However, while HTN planners 'lookahead' over the consequences of choosing one refinement over another, BDI agents interleave refinement with acting. There has been renewed interest in making HTN planners behave more like BDI agent systems, e.g. to have a unified representation for acting and planning. However, past work on the subject has remained informal or implementation-focused. This paper is a formal account of 'HTN acting', which supports interleaved deliberation, acting, and failure recovery. We use the syntax of the most general HTN planning formalism and build on its core semantics, and we provide an algorithm which combines our new formalism with the processing of exogenous events. We also study the properties of HTN acting and its relation to HTN planning.
Microscopy Cell Segmentation via Adversarial Neural Networks
Authors: Assaf Arbelle, Tammy Riklin Raviv
Date: 2017-09-18 00:00:00
Tasks: Cell Segmentation, Segmentation

We present a novel method for cell segmentation in microscopy images which is
inspired by the Generative Adversarial Neural Network (GAN) approach. Our
framework is built on a pair of two competitive artificial neural networks,
with a unique architecture, termed Rib Cage, which are trained simultaneously
and together define a min-max game resulting in an accurate segmentation of a
given image. Our approach has two main strengths, similar to the GAN, the
method does not require a formulation of a loss function for the optimization
process. This allows training on a limited amount of annotated data in a weakly
supervised manner. Promising segmentation results on real fluorescent
microscopy data are presented. The code is freely available at:
https://github.com/arbellea/DeepCellSeg.git
TextRay: Mining Clinical Reports to Gain a Broad Understanding of Chest X-rays
Authors: Jonathan Laserson, Christine Dan Lantsman, Michal Cohen-Sfady, Itamar Tamir, Eli Goz, Chen Brestel, Shir Bar, Maya Atar, Eldad Elnekave
Date: 2018-06-06 00:00:00

The chest X-ray (CXR) is by far the most commonly performed radiological
examination for screening and diagnosis of many cardiac and pulmonary diseases.
There is an immense world-wide shortage of physicians capable of providing
rapid and accurate interpretation of this study. A radiologist-driven analysis
of over two million CXR reports generated an ontology including the 40 most
prevalent pathologies on CXR. By manually tagging a relatively small set of
sentences, we were able to construct a training set of 959k studies. A deep
learning model was trained to predict the findings given the patient frontal
and lateral scans. For 12 of the findings we compare the model performance
against a team of radiologists and show that in most cases the radiologists
agree on average more with the algorithm than with each other.
Attention Incorporate Network: A network can adapt various data size
Authors: Liangbo He, Hao Sun
Date: 2018-06-06 00:00:00
Proceeding: ICLR 2019 5

In traditional neural networks for image processing, the inputs of the neural
networks should be the same size such as 224*224*3. But how can we train the
neural net model with different input size? A common way to do is image
deformation which accompany a problem of information loss (e.g. image crop or
wrap). Sequence model(RNN, LSTM, etc.) can accept different size of input like
text and audio. But one disadvantage for sequence model is that the previous
information will become more fragmentary during the transfer in time step, it
will make the network hard to train especially for long sequential data. In
this paper we propose a new network structure called Attention Incorporate
Network(AIN). It solve the problem of different size of inputs including:
images, text, audio, and extract the key features of the inputs by attention
mechanism, pay different attention depends on the importance of the features
not rely on the data size. Experimentally, AIN achieve a higher accuracy,
better convergence comparing to the same size of other network structure
Bounding Bloat in Genetic Programming
Authors: Benjamin Doerr, Timo Kötzing, J. A. Gregor Lagodzinski, Johannes Lengler
Date: 2018-06-06 00:00:00

While many optimization problems work with a fixed number of decision
variables and thus a fixed-length representation of possible solutions, genetic
programming (GP) works on variable-length representations. A naturally
occurring problem is that of bloat (unnecessary growth of solutions) slowing
down optimization. Theoretical analyses could so far not bound bloat and
required explicit assumptions on the magnitude of bloat. In this paper we
analyze bloat in mutation-based genetic programming for the two test functions
ORDER and MAJORITY. We overcome previous assumptions on the magnitude of bloat
and give matching or close-to-matching upper and lower bounds for the expected
optimization time. In particular, we show that the (1+1) GP takes (i)
$\Theta(T_{init} + n \log n)$ iterations with bloat control on ORDER as well as
MAJORITY; and (ii) $O(T_{init} \log T_{init} + n (\log n)^3)$ and
$\Omega(T_{init} + n \log n)$ (and $\Omega(T_{init} \log T_{init})$ for $n=1$)
iterations without bloat control on MAJORITY.
The Shattered Gradients Problem: If resnets are the answer, then what is the question?
Authors: David Balduzzi, Marcus Frean, Lennox Leary, JP Lewis, Kurt Wan-Duo Ma, Brian McWilliams
Date: 2017-02-28 00:00:00
Proceeding: ICML 2017 8

A long-standing obstacle to progress in deep learning is the problem of
vanishing and exploding gradients. Although, the problem has largely been
overcome via carefully constructed initializations and batch normalization,
architectures incorporating skip-connections such as highway and resnets
perform much better than standard feedforward architectures despite well-chosen
initialization and batch normalization. In this paper, we identify the
shattered gradients problem. Specifically, we show that the correlation between
gradients in standard feedforward networks decays exponentially with depth
resulting in gradients that resemble white noise whereas, in contrast, the
gradients in architectures with skip-connections are far more resistant to
shattering, decaying sublinearly. Detailed empirical evidence is presented in
support of the analysis, on both fully-connected networks and convnets.
Finally, we present a new "looks linear" (LL) initialization that prevents
shattering, with preliminary experiments showing the new initialization allows
to train very deep networks without the addition of skip-connections.
Conditionally Independent Multiresolution Gaussian Processes
Authors: Jalil Taghia, Thomas B. Schön
Date: 2018-02-25 00:00:00
Tasks: Bayesian Inference, Gaussian Processes

The multiresolution Gaussian process (GP) has gained increasing attention as
a viable approach towards improving the quality of approximations in GPs that
scale well to large-scale data. Most of the current constructions assume full
independence across resolutions. This assumption simplifies the inference, but
it underestimates the uncertainties in transitioning from one resolution to
another. This in turn results in models which are prone to overfitting in the
sense of excessive sensitivity to the chosen resolution, and predictions which
are non-smooth at the boundaries. Our contribution is a new construction which
instead assumes conditional independence among GPs across resolutions. We show
that relaxing the full independence assumption enables robustness against
overfitting, and that it delivers predictions that are smooth at the
boundaries. Our new model is compared against current state of the art on 2
synthetic and 9 real-world datasets. In most cases, our new conditionally
independent construction performed favorably when compared against models based
on the full independence assumption. In particular, it exhibits little to no
signs of overfitting.
Properties of interaction networks, structure coefficients, and benefit-to-cost ratios
Authors: Hendrik Richter
Date: 2018-05-29 00:00:00

In structured populations the spatial arrangement of cooperators and
defectors on the interaction graph together with the structure of the graph
itself determines the game dynamics and particularly whether or not fixation of
cooperation (or defection) is favored. For a single cooperator (and a single
defector) and a network described by a regular graph the question of fixation
can be addressed by a single parameter, the structure coefficient. As this
quantity is generic for any regular graph, we may call it the generic structure
coefficient. For two and more cooperators (or several defectors) fixation
properties can also be assigned by structure coefficients. These structure
coefficients, however, depend on the arrangement of cooperators and defectors
which we may interpret as a configuration of the game. Moreover, the
coefficients are specific for a given interaction network modeled as regular
graph, which is why we may call them specific structure coefficients. In this
paper, we study how specific structure coefficients vary over interaction
graphs and link the distributions obtained over different graphs to spectral
properties of interaction networks. We also discuss implications for the
benefit-to-cost ratios of donation games.
Convolutional Sequence to Sequence Non-intrusive Load Monitoring
Authors: Kunjin Chen, Qin Wang, Ziyu He, Kunlong Chen, Jun Hu, Jinliang He
Date: 2018-06-06 00:00:00
Tasks: Non-Intrusive Load Monitoring

A convolutional sequence to sequence non-intrusive load monitoring model is
proposed in this paper. Gated linear unit convolutional layers are used to
extract information from the sequences of aggregate electricity consumption.
Residual blocks are also introduced to refine the output of the neural network.
The partially overlapped output sequences of the network are averaged to
produce the final output of the model. We apply the proposed model to the REDD
dataset and compare it with the convolutional sequence to point model in the
literature. Results show that the proposed model is able to give satisfactory
disaggregation performance for appliances with varied characteristics.
Deep Exploration via Randomized Value Functions
Authors: Ian Osband, Benjamin Van Roy, Daniel Russo, Zheng Wen
Date: 2017-03-22 00:00:00
Tasks: Efficient Exploration, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We study the use of randomized value functions to guide deep exploration in reinforcement learning. This offers an elegant means for synthesizing statistically and computationally efficient exploration with common practical approaches to value function learning. We present several reinforcement learning algorithms that leverage randomized value functions and demonstrate their efficacy through computational studies. We also prove a regret bound that establishes statistical efficiency with a tabular representation.