pwc-archive/papers-with-abstracts huggingface dataset:
Normalized Wasserstein Distance for Mixture Distributions with Applications in Adversarial Learning and Domain Adaptation
Authors: Yogesh Balaji, Rama Chellappa, Soheil Feizi
Date: 2019-02-01 00:00:00
Tasks: Clustering, Domain Adaptation

Understanding proper distance measures between distributions is at the core of several learning tasks such as generative models, domain adaptation, clustering, etc. In this work, we focus on mixture distributions that arise naturally in several application domains where the data contains different sub-populations. For mixture distributions, established distance measures such as the Wasserstein distance do not take into account imbalanced mixture proportions. Thus, even if two mixture distributions have identical mixture components but different mixture proportions, the Wasserstein distance between them will be large. This often leads to undesired results in distance-based learning methods for mixture distributions. In this paper, we resolve this issue by introducing the Normalized Wasserstein measure. The key idea is to introduce mixture proportions as optimization variables, effectively normalizing mixture proportions in the Wasserstein formulation. Using the proposed normalized Wasserstein measure leads to significant performance gains for mixture distributions with imbalanced mixture proportions compared to the vanilla Wasserstein distance. We demonstrate the effectiveness of the proposed measure in GANs, domain adaptation and adversarial clustering in several benchmark datasets.
Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation
Authors: Sahil Singla, Eric Wallace, Shi Feng, Soheil Feizi
Date: 2019-02-01 00:00:00
Tasks: Deep Learning, Feature Importance, General Classification

Current methods to interpret deep learning models by generating saliency maps generally rely on two key assumptions. First, they use first-order approximations of the loss function neglecting higher-order terms such as the loss curvatures. Second, they evaluate each feature's importance in isolation, ignoring their inter-dependencies. In this work, we study the effect of relaxing these two assumptions. First, by characterizing a closed-form formula for the Hessian matrix of a deep ReLU network, we prove that, for a classification problem with a large number of classes, if an input has a high confidence classification score, the inclusion of the Hessian term has small impacts in the final solution. We prove this result by showing that in this case the Hessian matrix is approximately of rank one and its leading eigenvector is almost parallel to the gradient of the loss function. Our empirical experiments on ImageNet samples are consistent with our theory. This result can have implications in other related problems such as adversarial examples as well. Second, we compute the importance of group-features in deep learning interpretation by introducing a sparsity regularization term. We use the $L_0-L_1$ relaxation technique along with the proximal gradient descent to have an efficient computation of group feature importance scores. Our empirical results indicate that considering group features can improve deep learning interpretation significantly.
Scalable Learning-Based Sampling Optimization for Compressive Dynamic MRI
Authors: Thomas Sanchez, Baran Gözcü, Ruud B. van Heeswijk, Armin Eftekhari, Efe Ilıcak, Tolga Çukur, Volkan Cevher
Date: 2019-02-01 00:00:00
Tasks: compressed sensing

Compressed sensing applied to magnetic resonance imaging (MRI) allows to reduce the scanning time by enabling images to be reconstructed from highly undersampled data. In this paper, we tackle the problem of designing a sampling mask for an arbitrary reconstruction method and a limited acquisition budget. Namely, we look for an optimal probability distribution from which a mask with a fixed cardinality is drawn. We demonstrate that this problem admits a compactly supported solution, which leads to a deterministic optimal sampling mask. We then propose a stochastic greedy algorithm that (i) provides an approximate solution to this problem, and (ii) resolves the scaling issues of [1,2]. We validate its performance on in vivo dynamic MRI with retrospective undersampling, showing that our method preserves the performance of [1,2] while reducing the computational burden by a factor close to 200.
Learnable Embedding Space for Efficient Neural Architecture Compression
Authors: Shengcao Cao, Xiaofang Wang, Kris M. Kitani
Date: 2019-02-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Bayesian Optimization, Neural Architecture Search, Reinforcement Learning

We propose a method to incrementally learn an embedding space over the domain
of network architectures, to enable the careful selection of architectures for
evaluation during compressed architecture search. Given a teacher network, we
search for a compressed network architecture by using Bayesian Optimization
(BO) with a kernel function defined over our proposed embedding space to select
architectures for evaluation. We demonstrate that our search algorithm can
significantly outperform various baseline methods, such as random search and
reinforcement learning (Ashok et al., 2018). The compressed architectures found
by our method are also better than the state-of-the-art manually-designed
compact architecture ShuffleNet (Zhang et al., 2018). We also demonstrate that
the learned embedding space can be transferred to new settings for architecture
search, such as a larger teacher network or a teacher network in a different
architecture family, without any training. Code is publicly available here:
https://github.com/Friedrich1006/ESNAC .
Dynamic fairness - Breaking vicious cycles in automatic decision making
Authors: Benjamin Paaßen, Astrid Bunge, Carolin Hainke, Leon Sindelar, Matthias Vogelsang
Date: 2019-02-01 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Fairness

In recent years, machine learning techniques have been increasingly applied
in sensitive decision making processes, raising fairness concerns. Past
research has shown that machine learning may reproduce and even exacerbate
human bias due to biased training data or flawed model assumptions, and thus
may lead to discriminatory actions. To counteract such biased models,
researchers have proposed multiple mathematical definitions of fairness
according to which classifiers can be optimized. However, it has also been
shown that the outcomes generated by some fairness notions may be
unsatisfactory.
  In this contribution, we add to this research by considering decision making
processes in time. We establish a theoretic model in which even perfectly
accurate classifiers which adhere to almost all common fairness definitions
lead to stable long-term inequalities due to vicious cycles. Only demographic
parity, which enforces equal rates of positive decisions across groups, avoids
these effects and establishes a virtuous cycle, which leads to perfectly
accurate and fair classification in the long term.
Intelligent architectures for robotics: The merging of cognition and emotion
Authors: Luiz Pessoa
Date: 2019-02-01 00:00:00

What is the place of emotion in intelligent robots? In the past two decades,
researchers have advocated for the inclusion of some emotion-related components
in the general information processing architecture of autonomous agents, say,
for better communication with humans, or to instill a sense of urgency to
action. The framework advanced here goes beyond these approaches and proposes
that emotion and motivation need to be integrated with all aspects of the
architecture. Thus, cognitive-emotional integration is a key design principle.
Emotion is not an "add on" that endows a robot with "feelings" (for instance,
reporting or expressing its internal state). It allows the significance of
percepts, plans, and actions to be an integral part of all its computations. It
is hypothesized that a sophisticated artificial intelligence cannot be built
from separate cognitive and emotional modules. A hypothetical test inspired by
the Turing test, called the Dolores test, is proposed to test this assertion.
Projection-Based 2.5D U-net Architecture for Fast Volumetric Segmentation
Authors: Christoph Angermann, Markus Haltmeier, Ruth Steiger, Sergiy Pereverzyev Jr, Elke Gizewski
Date: 2019-02-01 00:00:00
Tasks: Segmentation

Convolutional neural networks are state-of-the-art for various segmentation tasks. While for 2D images these networks are also computationally efficient, 3D convolutions have huge storage requirements and require long training time. To overcome this issue, we introduce a network structure for volumetric data without 3D convolutional layers. The main idea is to include maximum intensity projections from different directions to transform the volumetric data to a sequence of images, where each image contains information of the full data. We then apply 2D convolutions to these projection images and lift them again to volumetric data using a trainable reconstruction algorithm.The proposed network architecture has less storage requirements than network structures using 3D convolutions. For a tested binary segmentation task, it even shows better performance than the 3D U-net and can be trained much faster.
Tree-Sliced Variants of Wasserstein Distances
Authors: Tam Le, Makoto Yamada, Kenji Fukumizu, Marco Cuturi
Date: 2019-02-01 00:00:00
Proceeding: NeurIPS 2019 12

Optimal transport (\OT) theory defines a powerful set of tools to compare probability distributions. \OT~suffers however from a few drawbacks, computational and statistical, which have encouraged the proposal of several regularized variants of OT in the recent literature, one of the most notable being the \textit{sliced} formulation, which exploits the closed-form formula between univariate distributions by projecting high-dimensional measures onto random lines. We consider in this work a more general family of ground metrics, namely \textit{tree metrics}, which also yield fast closed-form computations and negative definite, and of which the sliced-Wasserstein distance is a particular case (the tree is a chain). We propose the tree-sliced Wasserstein distance, computed by averaging the Wasserstein distance between these measures using random tree metrics, built adaptively in either low or high-dimensional spaces. Exploiting the negative definiteness of that distance, we also propose a positive definite kernel, and test it against other baselines on a few benchmark tasks.
Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication
Authors: Anastasia Koloskova, Sebastian U. Stich, Martin Jaggi
Date: 2019-02-01 00:00:00
Tasks: Stochastic Optimization

We consider decentralized stochastic optimization with the objective function
(e.g. data samples for machine learning task) being distributed over $n$
machines that can only communicate to their neighbors on a fixed communication
graph. To reduce the communication bottleneck, the nodes compress (e.g.
quantize or sparsify) their model updates. We cover both unbiased and biased
compression operators with quality denoted by $\omega \leq 1$ ($\omega=1$
meaning no compression). We (i) propose a novel gossip-based stochastic
gradient descent algorithm, CHOCO-SGD, that converges at rate
$\mathcal{O}\left(1/(nT) + 1/(T \delta^2 \omega)^2\right)$ for strongly convex
objectives, where $T$ denotes the number of iterations and $\delta$ the
eigengap of the connectivity matrix. Despite compression quality and network
connectivity affecting the higher order terms, the first term in the rate,
$\mathcal{O}(1/(nT))$, is the same as for the centralized baseline with exact
communication. We (ii) present a novel gossip algorithm, CHOCO-GOSSIP, for the
average consensus problem that converges in time
$\mathcal{O}(1/(\delta^2\omega) \log (1/\epsilon))$ for accuracy $\epsilon >
0$. This is (up to our knowledge) the first gossip algorithm that supports
arbitrary compressed messages for $\omega > 0$ and still exhibits linear
convergence. We (iii) show in experiments that both of our algorithms do
outperform the respective state-of-the-art baselines and CHOCO-SGD can reduce
communication by at least two orders of magnitudes.
SensitiveNets: Learning Agnostic Representations with Application to Face Images
Authors: Aythami Morales, Julian Fierrez, Ruben Vera-Rodriguez, Ruben Tolosana
Date: 2019-02-01 00:00:00
Tasks: Face Recognition, Fairness, Privacy Preserving, Representation Learning

This work proposes a novel privacy-preserving neural network feature representation to suppress the sensitive information of a learned space while maintaining the utility of the data. The new international regulation for personal data protection forces data controllers to guarantee privacy and avoid discriminative hazards while managing sensitive data of users. In our approach, privacy and discrimination are related to each other. Instead of existing approaches aimed directly at fairness improvement, the proposed feature representation enforces the privacy of selected attributes. This way fairness is not the objective, but the result of a privacy-preserving learning method. This approach guarantees that sensitive information cannot be exploited by any agent who process the output of the model, ensuring both privacy and equality of opportunity. Our method is based on an adversarial regularizer that introduces a sensitive information removal function in the learning objective. The method is evaluated on three different primary tasks (identity, attractiveness, and smiling) and three publicly available benchmarks. In addition, we present a new face annotation dataset with balanced distribution between genders and ethnic origins. The experiments demonstrate that it is possible to improve the privacy and equality of opportunity while retaining competitive performance independently of the task.
Joint Entity Linking with Deep Reinforcement Learning
Authors: Zheng Fang, Yanan Cao, Dongjie Zhang, Qian Li, Zhen-Yu Zhang, Yanbing Liu
Date: 2019-02-01 00:00:00
Tasks: Deep Reinforcement Learning, Entity Disambiguation, Entity Linking, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Entity linking is the task of aligning mentions to corresponding entities in
a given knowledge base. Previous studies have highlighted the necessity for
entity linking systems to capture the global coherence. However, there are two
common weaknesses in previous global models. First, most of them calculate the
pairwise scores between all candidate entities and select the most relevant
group of entities as the final result. In this process, the consistency among
wrong entities as well as that among right ones are involved, which may
introduce noise data and increase the model complexity. Second, the cues of
previously disambiguated entities, which could contribute to the disambiguation
of the subsequent mentions, are usually ignored by previous models. To address
these problems, we convert the global linking into a sequence decision problem
and propose a reinforcement learning model which makes decisions from a global
perspective. Our model makes full use of the previous referred entities and
explores the long-term influence of current selection on subsequent decisions.
We conduct experiments on different types of datasets, the results show that
our model outperforms state-of-the-art systems and has better generalization
performance.
VrR-VG: Refocusing Visually-Relevant Relationships
Authors: Yuanzhi Liang, Yalong Bai, Wei zhang, Xueming Qian, Li Zhu, Tao Mei
Date: 2019-02-01 00:00:00
Proceeding: ICCV 2019 10
Tasks: Image Captioning, Question Answering, Scene Graph Generation, Scene Understanding, Visual Question Answering, Visual Question Answering (VQA)

Relationships encode the interactions among individual instances, and play a critical role in deep visual scene understanding. Suffering from the high predictability with non-visual information, existing methods tend to fit the statistical bias rather than ``learning'' to ``infer'' the relationships from images. To encourage further development in visual relationships, we propose a novel method to automatically mine more valuable relationships by pruning visually-irrelevant ones. We construct a new scene-graph dataset named Visually-Relevant Relationships Dataset (VrR-VG) based on Visual Genome. Compared with existing datasets, the performance gap between learnable and statistical method is more significant in VrR-VG, and frequency-based analysis does not work anymore. Moreover, we propose to learn a relationship-aware representation by jointly considering instances, attributes and relationships. By applying the representation-aware feature learned on VrR-VG, the performances of image captioning and visual question answering are systematically improved with a large margin, which demonstrates the gain of our dataset and the features embedding schema. VrR-VG is available via http://vrr-vg.com/.
Generative Smoke Removal
Authors: Oleksii Sidorov, Congcong Wang, Faouzi Alaya Cheikh
Date: 2019-02-01 00:00:00
Tasks: Image-to-Image Translation, Translation

In minimally invasive surgery, the use of tissue dissection tools causes smoke, which inevitably degrades the image quality. This could reduce the visibility of the operation field for surgeons and introduces errors for the computer vision algorithms used in surgical navigation systems. In this paper, we propose a novel approach for computational smoke removal using supervised image-to-image translation. We demonstrate that straightforward application of existing generative algorithms allows removing smoke but decreases image quality and introduces synthetic noise (grid-structure). Thus, we propose to solve this issue by modification of GAN's architecture and adding perceptual image quality metric to the loss function. Obtained results demonstrate that proposed method efficiently removes smoke as well as preserves perceptually sufficient image quality.
Fast Re-Optimization via Structural Diversity
Authors: Benjamin Doerr, Carola Doerr, Frank Neumann
Date: 2019-02-01 00:00:00
Tasks: Diversity, Evolutionary Algorithms

When a problem instance is perturbed by a small modification, one would hope
to find a good solution for the new instance by building on a known good
solution for the previous one. Via a rigorous mathematical analysis, we show
that evolutionary algorithms, despite usually being robust problem solvers, can
have unexpected difficulties to solve such re-optimization problems. When
started with a random Hamming neighbor of the optimum, the (1+1) evolutionary
algorithm takes $\Omega(n^2)$ time to optimize the LeadingOnes benchmark
function, which is the same asymptotic optimization time when started in a
randomly chosen solution. There is hence no significant advantage from
re-optimizing a structurally good solution.
  We then propose a way to overcome such difficulties. As our mathematical
analysis reveals, the reason for this undesired behavior is that during the
optimization structurally good solutions can easily be replaced by structurally
worse solutions of equal or better fitness. We propose a simple diversity
mechanism that prevents this behavior, thereby reducing the re-optimization
time for LeadingOnes to $O(\gamma\delta n)$, where $\gamma$ is the population
size used by the diversity mechanism and $\delta \le \gamma$ the Hamming
distance of the new optimum from the previous solution. We show similarly fast
re-optimization times for the optimization of linear functions with changing
constraints and for the minimum spanning tree problem.
Deep Hyperspectral Prior: Denoising, Inpainting, Super-Resolution
Authors: Oleksii Sidorov, Jon Yngve Hardeberg
Date: 2019-02-01 00:00:00
Tasks: Denoising, Image Restoration, Super-Resolution

Deep learning algorithms have demonstrated state-of-the-art performance in various tasks of image restoration. This was made possible through the ability of CNNs to learn from large exemplar sets. However, the latter becomes an issue for hyperspectral image processing where datasets commonly consist of just a few images. In this work, we propose a new approach to denoising, inpainting, and super-resolution of hyperspectral image data using intrinsic properties of a CNN without any training. The performance of the given algorithm is shown to be comparable to the performance of trained networks, while its application is not restricted by the availability of training data. This work is an extension of original "deep prior" algorithm to HSI domain and 3D-convolutional networks.
End-to-end Lane Detection through Differentiable Least-Squares Fitting
Authors: Wouter Van Gansbeke, Bert de Brabandere, Davy Neven, Marc Proesmans, Luc van Gool
Date: 2019-02-01 00:00:00
Tasks: Lane Detection

Lane detection is typically tackled with a two-step pipeline in which a segmentation mask of the lane markings is predicted first, and a lane line model (like a parabola or spline) is fitted to the post-processed mask next. The problem with such a two-step approach is that the parameters of the network are not optimized for the true task of interest (estimating the lane curvature parameters) but for a proxy task (segmenting the lane markings), resulting in sub-optimal performance. In this work, we propose a method to train a lane detector in an end-to-end manner, directly regressing the lane parameters. The architecture consists of two components: a deep network that predicts a segmentation-like weight map for each lane line, and a differentiable least-squares fitting module that returns for each map the parameters of the best-fitting curve in the weighted least-squares sense. These parameters can subsequently be supervised with a loss function of choice. Our method relies on the observation that it is possible to backpropagate through a least-squares fitting procedure. This leads to an end-to-end method where the features are optimized for the true task of interest: the network implicitly learns to generate features that prevent instabilities during the model fitting step, as opposed to two-step pipelines that need to handle outliers with heuristics. Additionally, the system is not just a black box but offers a degree of interpretability because the intermediately generated segmentation-like weight maps can be inspected and visualized. Code and a video is available at github.com/wvangansbeke/LaneDetection_End2End.
Causal Simulations for Uplift Modeling
Authors: Jeroen Berrevoets, Wouter Verbeke
Date: 2019-02-01 00:00:00

Uplift modeling requires experimental data, preferably collected in random
fashion. This places a logistical and financial burden upon any organisation
aspiring such models. Once deployed, uplift models are subject to effects from
concept drift. Hence, methods are being developed that are able to learn from
newly gained experience, as well as handle drifting environments. As these new
methods attempt to eliminate the need for experimental data, another approach
to test such methods must be formulated. Therefore, we propose a method to
simulate environments that offer causal relationships in their parameters.
Understanding MCMC Dynamics as Flows on the Wasserstein Space
Authors: Chang Liu, Jingwei Zhuo, Jun Zhu
Date: 2019-02-01 00:00:00
Tasks: Novel Concepts, Variational Inference

It is known that the Langevin dynamics used in MCMC is the gradient flow of the KL divergence on the Wasserstein space, which helps convergence analysis and inspires recent particle-based variational inference methods (ParVIs). But no more MCMC dynamics is understood in this way. In this work, by developing novel concepts, we propose a theoretical framework that recognizes a general MCMC dynamics as the fiber-gradient Hamiltonian flow on the Wasserstein space of a fiber-Riemannian Poisson manifold. The "conservation + convergence" structure of the flow gives a clear picture on the behavior of general MCMC dynamics. The framework also enables ParVI simulation of MCMC dynamics, which enriches the ParVI family with more efficient dynamics, and also adapts ParVI advantages to MCMCs. We develop two ParVI methods for a particular MCMC dynamics and demonstrate the benefits in experiments.