pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Domain Adaptation using Regularized Hyper-graph Matching
Authors: Debasmit Das, C. S. George Lee
Date: 2018-05-22 00:00:00
Tasks: Domain Adaptation, Graph Matching, image-classification, Image Classification, Object Recognition, Unsupervised Domain Adaptation

Domain adaptation (DA) addresses the real-world image classification problem
of discrepancy between training (source) and testing (target) data
distributions. We propose an unsupervised DA method that considers the presence
of only unlabelled data in the target domain. Our approach centers on finding
matches between samples of the source and target domains. The matches are
obtained by treating the source and target domains as hyper-graphs and carrying
out a class-regularized hyper-graph matching using first-, second- and
third-order similarities between the graphs. We have also developed a
computationally efficient algorithm by initially selecting a subset of the
samples to construct a graph and then developing a customized optimization
routine for graph-matching based on Conditional Gradient and Alternating
Direction Multiplier Method. This allows the proposed method to be used widely.
We also performed a set of experiments on standard object recognition datasets
to validate the effectiveness of our framework over state-of-the-art
approaches.
Learning Time/Memory-Efficient Deep Architectures with Budgeted Super Networks
Authors: Tom Veniat, Ludovic Denoyer
Date: 2017-05-31 00:00:00
Proceeding: CVPR 2018 6

We propose to focus on the problem of discovering neural network
architectures efficient in terms of both prediction quality and cost. For
instance, our approach is able to solve the following tasks: learn a neural
network able to predict well in less than 100 milliseconds or learn an
efficient model that fits in a 50 Mb memory. Our contribution is a novel family
of models called Budgeted Super Networks (BSN). They are learned using gradient
descent techniques applied on a budgeted learning objective function which
integrates a maximum authorized cost, while making no assumption on the nature
of this cost. We present a set of experiments on computer vision problems and
analyze the ability of our technique to deal with three different costs: the
computation cost, the memory consumption cost and a distributed computation
cost. We particularly show that our model can discover neural network
architectures that have a better accuracy than the ResNet and Convolutional
Neural Fabrics architectures on CIFAR-10 and CIFAR-100, at a lower cost.
Functional Decision Theory: A New Theory of Instrumental Rationality
Authors: Eliezer Yudkowsky, Nate Soares
Date: 2017-10-13 00:00:00
Tasks: Decision Making

This paper describes and motivates a new decision theory known as functional
decision theory (FDT), as distinct from causal decision theory and evidential
decision theory. Functional decision theorists hold that the normative
principle for action is to treat one's decision as the output of a fixed
mathematical function that answers the question, "Which output of this very
function would yield the best outcome?" Adhering to this principle delivers a
number of benefits, including the ability to maximize wealth in an array of
traditional decision-theoretic and game-theoretic problems where CDT and EDT
perform poorly. Using one simple and coherent decision rule, functional
decision theorists (for example) achieve more utility than CDT on Newcomb's
problem, more utility than EDT on the smoking lesion problem, and more utility
than both in Parfit's hitchhiker problem. In this paper, we define FDT, explore
its prescriptions in a number of different decision problems, compare it to CDT
and EDT, and give philosophical justifications for FDT as a normative theory of
decision-making.
Divide and Conquer Networks
Authors: Alex Nowak-Vila, David Folqué, Joan Bruna
Date: 2016-11-08 00:00:00
Proceeding: ICLR 2018 1
Tasks: Clustering, Inductive Bias

We consider the learning of algorithmic tasks by mere observation of
input-output pairs. Rather than studying this as a black-box discrete
regression problem with no assumption whatsoever on the input-output mapping,
we concentrate on tasks that are amenable to the principle of divide and
conquer, and study what are its implications in terms of learning. This
principle creates a powerful inductive bias that we leverage with neural
architectures that are defined recursively and dynamically, by learning two
scale-invariant atomic operations: how to split a given input into smaller
sets, and how to merge two partially solved tasks into a larger partial
solution. Our model can be trained in weakly supervised environments, namely by
just observing input-output pairs, and in even weaker environments, using a
non-differentiable reward signal. Moreover, thanks to the dynamic aspect of our
architecture, we can incorporate the computational complexity as a
regularization term that can be optimized by backpropagation. We demonstrate
the flexibility and efficiency of the Divide-and-Conquer Network on several
combinatorial and geometric tasks: convex hull, clustering, knapsack and
euclidean TSP. Thanks to the dynamic programming nature of our model, we show
significant improvements in terms of generalization error and computational
complexity.
Rate-Optimal Denoising with Deep Neural Networks
Authors: Reinhard Heckel, Wen Huang, Paul Hand, Vladislav Voroninski
Date: 2018-05-22 00:00:00
Proceeding: ICLR 2019 5
Tasks: Denoising, Image Denoising

Deep neural networks provide state-of-the-art performance for image
denoising, where the goal is to recover a near noise-free image from a noisy
observation. The underlying principle is that neural networks trained on large
datasets have empirically been shown to be able to generate natural images well
from a low-dimensional latent representation of the image. Given such a
generator network, a noisy image can be denoised by i) finding the closest
image in the range of the generator or by ii) passing it through an
encoder-generator architecture (known as an autoencoder). However, there is
little theory to justify this success, let alone to predict the denoising
performance as a function of the network parameters. In this paper we consider
the problem of denoising an image from additive Gaussian noise using the two
generator based approaches. In both cases, we assume the image is well
described by a deep neural network with ReLU activations functions, mapping a
$k$-dimensional code to an $n$-dimensional image. In the case of the
autoencoder, we show that the feedforward network reduces noise energy by a
factor of $O(k/n)$. In the case of optimizing over the range of a generative
model, we state and analyze a simple gradient algorithm that minimizes a
non-convex loss function, and provably reduces noise energy by a factor of
$O(k/n)$. We also demonstrate in numerical experiments that this denoising
performance is, indeed, achieved by generative priors learned from data.
Optimal Transport for Multi-source Domain Adaptation under Target Shift
Authors: Ievgen Redko, Nicolas Courty, Rémi Flamary, Devis Tuia
Date: 2018-03-13 00:00:00
Tasks: Domain Adaptation, Image Segmentation, Semantic Segmentation

In this paper, we propose to tackle the problem of reducing discrepancies
between multiple domains referred to as multi-source domain adaptation and
consider it under the target shift assumption: in all domains we aim to solve a
classification problem with the same output classes, but with labels'
proportions differing across them. This problem, generally ignored in the vast
majority papers on domain adaptation papers, is nevertheless critical in
real-world applications, and we theoretically show its impact on the adaptation
success. To address this issue, we design a method based on optimal transport,
a theory that has been successfully used to tackle adaptation problems in
machine learning. Our method performs multi-source adaptation and target shift
correction simultaneously by learning the class probabilities of the unlabeled
target sample and the coupling allowing to align two (or more) probability
distributions. Experiments on both synthetic and real-world data related to
satellite image segmentation task show the superiority of the proposed method
over the state-of-the-art.
Mean Actor Critic
Authors: Cameron Allen, Kavosh Asadi, Melrose Roderick, Abdel-rahman Mohamed, George Konidaris, Michael Littman
Date: 2017-09-01 00:00:00
Tasks: Atari Games, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a new algorithm, Mean Actor-Critic (MAC), for discrete-action
continuous-state reinforcement learning. MAC is a policy gradient algorithm
that uses the agent's explicit representation of all action values to estimate
the gradient of the policy, rather than using only the actions that were
actually executed. We prove that this approach reduces variance in the policy
gradient estimate relative to traditional actor-critic methods. We show
empirical results on two control domains and on six Atari games, where MAC is
competitive with state-of-the-art policy search algorithms.
Counterfactual Mean Embeddings
Authors: Krikamol Muandet, Motonobu Kanagawa, Sorawit Saengkyongam, Sanparith Marukatat
Date: 2018-05-22 00:00:00
Tasks: Causal Inference, counterfactual, Counterfactual Inference, Econometrics, Medical Diagnosis, Off-policy evaluation, Recommendation Systems

Counterfactual inference has become a ubiquitous tool in online advertisement, recommendation systems, medical diagnosis, and econometrics. Accurate modeling of outcome distributions associated with different interventions -- known as counterfactual distributions -- is crucial for the success of these applications. In this work, we propose to model counterfactual distributions using a novel Hilbert space representation called counterfactual mean embedding (CME). The CME embeds the associated counterfactual distribution into a reproducing kernel Hilbert space (RKHS) endowed with a positive definite kernel, which allows us to perform causal inference over the entire landscape of the counterfactual distribution. Based on this representation, we propose a distributional treatment effect (DTE) that can quantify the causal effect over entire outcome distributions. Our approach is nonparametric as the CME can be estimated under the unconfoundedness assumption from observational data without requiring any parametric assumption about the underlying distributions. We also establish a rate of convergence of the proposed estimator which depends on the smoothness of the conditional mean and the Radon-Nikodym derivative of the underlying marginal distributions. Furthermore, our framework allows for more complex outcomes such as images, sequences, and graphs. Our experimental results on synthetic data and off-policy evaluation tasks demonstrate the advantages of the proposed estimator.
Distribution Matching Losses Can Hallucinate Features in Medical Image Translation
Authors: Joseph Paul Cohen, Margaux Luck, Sina Honari
Date: 2018-05-22 00:00:00
Tasks: Image Generation, Translation

This paper discusses how distribution matching losses, such as those used in
CycleGAN, when used to synthesize medical images can lead to mis-diagnosis of
medical conditions. It seems appealing to use these new image synthesis methods
for translating images from a source to a target domain because they can
produce high quality images and some even do not require paired data. However,
the basis of how these image translation models work is through matching the
translation output to the distribution of the target domain. This can cause an
issue when the data provided in the target domain has an over or under
representation of some classes (e.g. healthy or sick). When the output of an
algorithm is a transformed image there are uncertainties whether all known and
unknown class labels have been preserved or changed. Therefore, we recommend
that these translated images should not be used for direct interpretation (e.g.
by doctors) because they may lead to misdiagnosis of patients based on
hallucinated image features by an algorithm that matches a distribution.
However there are many recent papers that seem as though this is the goal.
Clustering - What Both Theoreticians and Practitioners are Doing Wrong
Authors: Shai Ben-David
Date: 2018-05-22 00:00:00
Tasks: Clustering, Model Selection

Unsupervised learning is widely recognized as one of the most important
challenges facing machine learning nowa- days. However, in spite of hundreds of
papers on the topic being published every year, current theoretical
understanding and practical implementations of such tasks, in particular of
clustering, is very rudimentary. This note focuses on clustering. I claim that
the most signif- icant challenge for clustering is model selection. In contrast
with other common computational tasks, for clustering, dif- ferent algorithms
often yield drastically different outcomes. Therefore, the choice of a
clustering algorithm, and their pa- rameters (like the number of clusters) may
play a crucial role in the usefulness of an output clustering solution.
However, currently there exists no methodical guidance for clustering
tool-selection for a given clustering task. Practitioners pick the algorithms
they use without awareness to the implications of their choices and the vast
majority of theory of clustering papers focus on providing savings to the
resources needed to solve optimization problems that arise from picking some
concrete clustering objective. Saving that pale in com- parison to the costs of
mismatch between those objectives and the intended use of clustering results. I
argue the severity of this problem and describe some recent proposals aiming to
address this crucial lacuna.
Nonparametric Density Estimation under Adversarial Losses
Authors: Shashank Singh, Ananya Uppal, Boyue Li, Chun-Liang Li, Manzil Zaheer, Barnabás Póczos
Date: 2018-05-22 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Density Estimation

We study minimax convergence rates of nonparametric density estimation under
a large class of loss functions called "adversarial losses", which, besides
classical $\mathcal{L}^p$ losses, includes maximum mean discrepancy (MMD),
Wasserstein distance, and total variation distance. These losses are closely
related to the losses encoded by discriminator networks in generative
adversarial networks (GANs). In a general framework, we study how the choice of
loss and the assumed smoothness of the underlying density together determine
the minimax rate. We also discuss implications for training GANs based on deep
ReLU networks, and more general connections to learning implicit generative
models in a minimax statistical sense.
Local Differential Privacy for Evolving Data
Authors: Matthew Joseph, Aaron Roth, Jonathan Ullman, Bo Waggoner
Date: 2018-02-20 00:00:00
Proceeding: NeurIPS 2018 12

There are now several large scale deployments of differential privacy used to
collect statistical information about users. However, these deployments
periodically recollect the data and recompute the statistics using algorithms
designed for a single use. As a result, these systems do not provide meaningful
privacy guarantees over long time scales. Moreover, existing techniques to
mitigate this effect do not apply in the "local model" of differential privacy
that these systems use.
  In this paper, we introduce a new technique for local differential privacy
that makes it possible to maintain up-to-date statistics over time, with
privacy guarantees that degrade only in the number of changes in the underlying
distribution rather than the number of collection periods. We use our technique
for tracking a changing statistic in the setting where users are partitioned
into an unknown collection of groups, and at every time period each user draws
a single bit from a common (but changing) group-specific distribution. We also
provide an application to frequency and heavy-hitter estimation.
Characteristic and Universal Tensor Product Kernels
Authors: Zoltan Szabo, Bharath K. Sriperumbudur
Date: 2017-08-28 00:00:00

Maximum mean discrepancy (MMD), also called energy distance or N-distance in
statistics and Hilbert-Schmidt independence criterion (HSIC), specifically
distance covariance in statistics, are among the most popular and successful
approaches to quantify the difference and independence of random variables,
respectively. Thanks to their kernel-based foundations, MMD and HSIC are
applicable on a wide variety of domains. Despite their tremendous success,
quite little is known about when HSIC characterizes independence and when MMD
with tensor product kernel can discriminate probability distributions. In this
paper, we answer these questions by studying various notions of characteristic
property of the tensor product kernel.
Rapid seismic domain transfer: Seismic velocity inversion and modeling using deep generative neural networks
Authors: Lukas Mosser, Wouter Kimman, Jesper Dramsch, Steve Purves, Alfredo De la Fuente, Graham Ganssle
Date: 2018-05-22 00:00:00
Tasks: Generative Adversarial Network

Traditional physics-based approaches to infer sub-surface properties such as
full-waveform inversion or reflectivity inversion are time-consuming and
computationally expensive. We present a deep-learning technique that eliminates
the need for these computationally complex methods by posing the problem as one
of domain transfer. Our solution is based on a deep convolutional generative
adversarial network and dramatically reduces computation time. Training based
on two different types of synthetic data produced a neural network that
generates realistic velocity models when applied to a real dataset. The
system's ability to generalize means it is robust against the inherent
occurrence of velocity errors and artifacts in both training and test datasets.
Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation
Authors: Saeid Asgari Taghanaki, Yefeng Zheng, S. Kevin Zhou, Bogdan Georgescu, Puneet Sharma, Daguang Xu, Dorin Comaniciu, Ghassan Hamarneh
Date: 2018-05-08 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Organ Segmentation, Segmentation, Semantic Segmentation

Simultaneous segmentation of multiple organs from different medical imaging modalities is a crucial task as it can be utilized for computer-aided diagnosis, computer-assisted surgery, and therapy planning. Thanks to the recent advances in deep learning, several deep neural networks for medical image segmentation have been introduced successfully for this purpose. In this paper, we focus on learning a deep multi-organ segmentation network that labels voxels. In particular, we examine the critical choice of a loss function in order to handle the notorious imbalance problem that plagues both the input and output of a learning model. The input imbalance refers to the class-imbalance in the input training samples (i.e., small foreground objects embedded in an abundance of background voxels, as well as organs of varying sizes). The output imbalance refers to the imbalance between the false positives and false negatives of the inference model. In order to tackle both types of imbalance during training and inference, we introduce a new curriculum learning based loss function. Specifically, we leverage Dice similarity coefficient to deter model parameters from being held at bad local minima and at the same time gradually learn better model parameters by penalizing for false positives/negatives using a cross entropy term. We evaluated the proposed loss function on three datasets: whole body positron emission tomography (PET) scans with 5 target organs, magnetic resonance imaging (MRI) prostate scans, and ultrasound echocardigraphy images with a single target organ i.e., left ventricular. We show that a simple network architecture with the proposed integrative loss function can outperform state-of-the-art methods and results of the competing methods can be improved when our proposed loss is used.
Geometry-Based Data Generation
Authors: Ofir Lindenbaum, Jay S. Stanley III, Guy Wolf, Smita Krishnaswamy
Date: 2018-02-14 00:00:00

Many generative models attempt to replicate the density of their input data.
However, this approach is often undesirable, since data density is highly
affected by sampling biases, noise, and artifacts. We propose a method called
SUGAR (Synthesis Using Geometrically Aligned Random-walks) that uses a
diffusion process to learn a manifold geometry from the data. Then, it
generates new points evenly along the manifold by pulling randomly generated
points into its intrinsic structure using a diffusion kernel. SUGAR equalizes
the density along the manifold by selectively generating points in sparse areas
of the manifold. We demonstrate how the approach corrects sampling biases and
artifacts, while also revealing intrinsic patterns (e.g. progression) and
relations in the data. The method is applicable for correcting missing data,
finding hypothetical data points, and learning relationships between data
features.
Fast Neural Architecture Construction using EnvelopeNets
Authors: Purushotham Kamath, Abhishek Singh, Debo Dutta
Date: 2018-03-18 00:00:00
Tasks: Evolutionary Algorithms, GPU, image-classification, Image Classification, Neural Architecture Search, Reinforcement Learning

Fast Neural Architecture Construction (NAC) is a method to construct deep
network architectures by pruning and expansion of a base network. In recent
years, several automated search methods for neural network architectures have
been proposed using methods such as evolutionary algorithms and reinforcement
learning. These methods use a single scalar objective function (usually
accuracy) that is evaluated after a full training and evaluation cycle. In
contrast NAC directly compares the utility of different filters using
statistics derived from filter featuremaps reach a state where the utility of
different filters within a network can be compared and hence can be used to
construct networks. The training epochs needed for filters within a network to
reach this state is much less than the training epochs needed for the accuracy
of a network to stabilize. NAC exploits this finding to construct convolutional
neural nets (CNNs) with close to state of the art accuracy, in < 1 GPU day,
faster than most of the current neural architecture search methods. The
constructed networks show close to state of the art performance on the image
classification problem on well known datasets (CIFAR-10, ImageNet) and
consistently show better performance than hand constructed and randomly
generated networks of the same depth, operators and approximately the same
number of parameters.
Infinite-Task Learning with RKHSs
Authors: Romain Brault, Alex Lambert, Zoltán Szabó, Maxime Sangnier, Florence d'Alché-Buc
Date: 2018-05-22 00:00:00
Tasks: Multi-Task Learning, quantile regression

Machine learning has witnessed tremendous success in solving tasks depending
on a single hyperparameter. When considering simultaneously a finite number of
tasks, multi-task learning enables one to account for the similarities of the
tasks via appropriate regularizers. A step further consists of learning a
continuum of tasks for various loss functions. A promising approach, called
\emph{Parametric Task Learning}, has paved the way in the continuum setting for
affine models and piecewise-linear loss functions. In this work, we introduce a
novel approach called \emph{Infinite Task Learning} whose goal is to learn a
function whose output is a function over the hyperparameter space. We leverage
tools from operator-valued kernels and the associated vector-valued RKHSs that
provide an explicit control over the role of the hyperparameters, and also
allows us to consider new type of constraints. We provide generalization
guarantees to the suggested scheme and illustrate its efficiency in
cost-sensitive classification, quantile regression and density level set
estimation.