pwc-archive/papers-with-abstracts huggingface dataset:
Stable Backward Diffusion Models that Minimise Convex Energies
Authors: Leif Bergerhoff, Marcelo Cárdenas, Joachim Weickert, Martin Welk
Date: 2019-03-08 00:00:00
Tasks: Deblurring, Image Enhancement

The inverse problem of backward diffusion is known to be ill-posed and highly unstable. Backward diffusion processes appear naturally in image enhancement and deblurring applications. It is therefore greatly desirable to establish a backward diffusion model which implements a smart stabilisation approach that can be used in combination with an easy to handle numerical scheme. So far, existing stabilisation strategies in literature require sophisticated numerics to solve the underlying initial value problem. We derive a class of space-discrete one-dimensional backward diffusion as gradient descent of energies where we gain stability by imposing range constraints. Interestingly, these energies are even convex. Furthermore, we establish a comprehensive theory for the time-continuous evolution and we show that stability carries over to a simple explicit time discretisation of our model. Finally, we confirm the stability and usefulness of our technique in experiments in which we enhance the contrast of digital greyscale and colour images.
Is Deeper Better only when Shallow is Good?
Authors: Eran Malach, Shai Shalev-Shwartz
Date: 2019-03-08 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Learning Theory, Open-Ended Question Answering

Understanding the power of depth in feed-forward neural networks is an
ongoing challenge in the field of deep learning theory. While current works
account for the importance of depth for the expressive power of
neural-networks, it remains an open question whether these benefits are
exploited during a gradient-based optimization process. In this work we explore
the relation between expressivity properties of deep networks and the ability
to train them efficiently using gradient-based algorithms. We give a depth
separation argument for distributions with fractal structure, showing that they
can be expressed efficiently by deep networks, but not with shallow ones. These
distributions have a natural coarse-to-fine structure, and we show that the
balance between the coarse and fine details has a crucial effect on whether the
optimization process is likely to succeed. We prove that when the distribution
is concentrated on the fine details, gradient-based algorithms are likely to
fail. Using this result we prove that, at least in some distributions, the
success of learning deep networks depends on whether the distribution can be
well approximated by shallower networks, and we conjecture that this property
holds in general.
NEARBY Platform for Automatic Asteroids Detection and EURONEAR Surveys
Authors: Dorian Gorgan, Ovidiu Vaduvescu, Teodor Stefanut, Victor Bacu, Adrian Sabou, Denisa Copandean Balazs, Constantin Nandra, Costin Boldea, Afrodita Boldea, Marian Predatu, Viktoria Pinter, Adrian Stanica
Date: 2019-03-08 00:00:00
Tasks: Astronomy, Survey

The survey of the nearby space and continuous monitoring of the Near Earth
Objects (NEOs) and especially Near Earth Asteroids (NEAs) are essential for the
future of our planet and should represent a priority for our solar system
research and nearby space exploration. More computing power and sophisticated
digital tracking algorithms are needed to cope with the larger astronomy
imaging cameras dedicated for survey telescopes. The paper presents the NEARBY
platform that aims to experiment new algorithms for automatic image reduction,
detection and validation of moving objects in astronomical surveys,
specifically NEAs. The NEARBY platform has been developed and experimented
through a collaborative research work between the Technical University of
Cluj-Napoca (UTCN) and the University of Craiova, Romania, using observing
infrastructure of the Instituto de Astrofisica de Canarias (IAC) and Isaac
Newton Group (ING), La Palma, Spain. The NEARBY platform has been developed and
deployed on the UTCN's cloud infrastructure and the acquired images are
processed remotely by the astronomers who transfer it from ING through the web
interface of the NEARBY platform. The paper analyzes and highlights the main
aspects of the NEARBY platform development, and the results and conclusions on
the EURONEAR surveys.
Auto-Encoding Progressive Generative Adversarial Networks For 3D Multi Object Scenes
Authors: Vedant Singh, Manan Oza, Himanshu Vaghela, Pratik Kanani
Date: 2019-03-08 00:00:00
Tasks: Object, Representation Learning, Scene Generation

3D multi object generative models allow us to synthesize a large range of
novel 3D multi object scenes and also identify objects, shapes, layouts and
their positions. But multi object scenes are difficult to create because of the
dataset being multimodal in nature. The conventional 3D generative adversarial
models are not efficient in generating multi object scenes, they usually tend
to generate either one object or generate fuzzy results of multiple objects.
Auto-encoder models have much scope in feature extraction and representation
learning using the unsupervised paradigm in probabilistic spaces. We try to
make use of this property in our proposed model. In this paper we propose a
novel architecture using 3DConvNets trained with the progressive training
paradigm that has been able to generate realistic high resolution 3D scenes of
rooms, bedrooms, offices etc. with various pieces of furniture and objects. We
make use of the adversarial auto-encoder along with the WGAN-GP loss parameter
in our discriminator loss function. Finally this new approach to multi object
scene generation has also been able to generate more number of objects per
scene.
Improving Device-Edge Cooperative Inference of Deep Learning via 2-Step Pruning
Authors: Wenqi Shi, Yunzhong Hou, Sheng Zhou, Zhisheng Niu, Yang Zhang, Lu Geng
Date: 2019-03-08 00:00:00

Deep neural networks (DNNs) are state-of-the-art solutions for many machine
learning applications, and have been widely used on mobile devices. Running
DNNs on resource-constrained mobile devices often requires the help from edge
servers via computation offloading. However, offloading through a
bandwidth-limited wireless link is non-trivial due to the tight interplay
between the computation resources on mobile devices and wireless resources.
Existing studies have focused on cooperative inference where DNN models are
partitioned at different neural network layers, and the two parts are executed
at the mobile device and the edge server, respectively. Since the output data
size of a DNN layer can be larger than that of the raw data, offloading
intermediate data between layers can suffer from high transmission latency
under limited wireless bandwidth. In this paper, we propose an efficient and
flexible 2-step pruning framework for DNN partition between mobile devices and
edge servers. In our framework, the DNN model only needs to be pruned once in
the training phase where unimportant convolutional filters are removed
iteratively. By limiting the pruning region, our framework can greatly reduce
either the wireless transmission workload of the device or the total
computation workload. A series of pruned models are generated in the training
phase, from which the framework can automatically select to satisfy varying
latency and accuracy requirements. Furthermore, coding for the intermediate
data is added to provide extra transmission workload reduction. Our experiments
show that the proposed framework can achieve up to 25.6$\times$ reduction on
transmission workload, 6.01$\times$ acceleration on total computation and
4.81$\times$ reduction on end-to-end latency as compared to partitioning the
original DNN model without pruning.
Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injection
Authors: Amit Moryossef, Roee Aharoni, Yoav Goldberg
Date: 2019-03-08 00:00:00
Tasks: Machine Translation, Translation

When translating from a language that does not morphologically mark
information such as gender and number into a language that does, translation
systems must "guess" this missing information, often leading to incorrect
translations in the given context. We propose a black-box approach for
injecting the missing information to a pre-trained neural machine translation
system, allowing to control the morphological variations in the generated
translations without changing the underlying model or training data. We
evaluate our method on an English to Hebrew translation task, and show that it
is effective in injecting the gender and number information and that supplying
the correct information improves the translation accuracy in up to 2.3 BLEU on
a female-speaker test set for a state-of-the-art online black-box system.
Finally, we perform a fine-grained syntactic analysis of the generated
translations that shows the effectiveness of our method.
On Boosting Semantic Street Scene Segmentation with Weak Supervision
Authors: Panagiotis Meletis, Gijs Dubbelman
Date: 2019-03-08 00:00:00
Tasks: GPU, Scene Segmentation, Segmentation, Semantic Segmentation

Training convolutional networks for semantic segmentation requires per-pixel ground truth labels, which are very time consuming and hence costly to obtain. Therefore, in this work, we research and develop a hierarchical deep network architecture and the corresponding loss for semantic segmentation that can be trained from weak supervision, such as bounding boxes or image level labels, as well as from strong per-pixel supervision. We demonstrate that the hierarchical structure and the simultaneous training on strong (per-pixel) and weak (bounding boxes) labels, even from separate datasets, constantly increases the performance against per-pixel only training. Moreover, we explore the more challenging case of adding weak image-level labels. We collect street scene images and weak labels from the immense Open Images dataset to generate the OpenScapes dataset, and we use this novel dataset to increase segmentation performance on two established per-pixel labeled datasets, Cityscapes and Vistas. We report performance gains up to +13.2% mIoU on crucial street scene classes, and inference speed of 20 fps on a Titan V GPU for Cityscapes at 512 x 1024 resolution. Our network and OpenScapes dataset are shared with the research community.
Support and Invertibility in Domain-Invariant Representations
Authors: Fredrik D. Johansson, David Sontag, Rajesh Ranganath
Date: 2019-03-08 00:00:00
Tasks: Domain Adaptation, Generalization Bounds, Unsupervised Domain Adaptation

Learning domain-invariant representations has become a popular approach to unsupervised domain adaptation and is often justified by invoking a particular suite of theoretical results. We argue that there are two significant flaws in such arguments. First, the results in question hold only for a fixed representation and do not account for information lost in non-invertible transformations. Second, domain invariance is often a far too strict requirement and does not always lead to consistent estimation, even under strong and favorable assumptions. In this work, we give generalization bounds for unsupervised domain adaptation that hold for any representation function by acknowledging the cost of non-invertibility. In addition, we show that penalizing distance between densities is often wasteful and propose a bound based on measuring the extent to which the support of the source domain covers the target domain. We perform experiments on well-known benchmarks that illustrate the short-comings of current standard practice.
Random Matrix-Improved Estimation of the Wasserstein Distance between two Centered Gaussian Distributions
Authors: Malik Tiomoko, Romain Couillet
Date: 2019-03-08 00:00:00

This article proposes a method to consistently estimate functionals
$\frac1p\sum_{i=1}^pf(\lambda_i(C_1C_2))$ of the eigenvalues of the product of
two covariance matrices $C_1,C_2\in\mathbb{R}^{p\times p}$ based on the
empirical estimates $\lambda_i(\hat C_1\hat C_2)$ ($\hat
C_a=\frac1{n_a}\sum_{i=1}^{n_a} x_i^{(a)}x_i^{(a){{\sf T}}}$), when the size
$p$ and number $n_a$ of the (zero mean) samples $x_i^{(a)}$ are similar. As a
corollary, a consistent estimate of the Wasserstein distance (related to the
case $f(t)=\sqrt{t}$) between centered Gaussian distributions is derived.
  The new estimate is shown to largely outperform the classical sample
covariance-based `plug-in' estimator. Based on this finding, a practical
application to covariance estimation is then devised which demonstrates
potentially significant performance gains with respect to state-of-the-art
alternatives.
Joint Learning of Brain Lesion and Anatomy Segmentation from Heterogeneous Datasets
Authors: Nicolas Roulet, Diego Fernandez Slezak, Enzo Ferrante
Date: 2019-03-08 00:00:00
Tasks: Anatomy, Benchmarking

Brain lesion and anatomy segmentation in magnetic resonance images are
fundamental tasks in neuroimaging research and clinical practice. Given enough
training data, convolutional neuronal networks (CNN) proved to outperform all
existent techniques in both tasks independently. However, to date, little work
has been done regarding simultaneous learning of brain lesion and anatomy
segmentation from disjoint datasets.
  In this work we focus on training a single CNN model to predict brain tissue
and lesion segmentations using heterogeneous datasets labeled independently,
according to only one of these tasks (a common scenario when using publicly
available datasets). We show that label contradiction issues can arise in this
case, and propose a novel adaptive cross entropy (ACE) loss function that makes
such training possible. We provide quantitative evaluation in two different
scenarios, benchmarking the proposed method in comparison with a multi-network
approach. Our experiments suggest that ACE loss enables training of single
models when standard cross entropy and Dice loss functions tend to fail.
Moreover, we show that it is possible to achieve competitive results when
comparing with multiple networks trained for independent tasks.
Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks
Authors: Tim J. Adler, Lynton Ardizzone, Anant Vemuri, Leonardo Ayala, Janek Gröhl, Thomas Kirchner, Sebastian Wirkert, Jakob Kruse, Carsten Rother, Ullrich Köthe, Lena Maier-Hein
Date: 2019-03-08 00:00:00
Tasks: parameter estimation

Purpose: Optical imaging is evolving as a key technique for advanced sensing
in the operating room. Recent research has shown that machine learning
algorithms can be used to address the inverse problem of converting pixel-wise
multispectral reflectance measurements to underlying tissue parameters, such as
oxygenation. Assessment of the specific hardware used in conjunction with such
algorithms, however, has not properly addressed the possibility that the
problem may be ill-posed.
  Methods: We present a novel approach to the assessment of optical imaging
modalities, which is sensitive to the different types of uncertainties that may
occur when inferring tissue parameters. Based on the concept of invertible
neural networks, our framework goes beyond point estimates and maps each
multispectral measurement to a full posterior probability distribution which is
capable of representing ambiguity in the solution via multiple modes.
Performance metrics for a hardware setup can then be computed from the
characteristics of the posteriors.
  Results: Application of the assessment framework to the specific use case of
camera selection for physiological parameter estimation yields the following
insights: (1) Estimation of tissue oxygenation from multispectral images is a
well-posed problem, while (2) blood volume fraction may not be recovered
without ambiguity. (3) In general, ambiguity may be reduced by increasing the
number of spectral bands in the camera.
  Conclusion: Our method could help to optimize optical camera design in an
application-specific manner.
Accelerating Generalized Linear Models with MLWeaving: A One-Size-Fits-All System for Any-precision Learning (Technical Report)
Authors: Zeke Wang, Kaan Kara, Hantian Zhang, Gustavo Alonso, Onur Mutlu, Ce Zhang
Date: 2019-03-08 00:00:00
Tasks: All, CPU, Quantization, Retrieval

Learning from the data stored in a database is an important function
increasingly available in relational engines. Methods using lower precision
input data are of special interest given their overall higher efficiency but,
in databases, these methods have a hidden cost: the quantization of the real
value into a smaller number is an expensive step. To address the issue, in this
paper we present MLWeaving, a data structure and hardware acceleration
technique intended to speed up learning of generalized linear models in
databases. ML-Weaving provides a compact, in-memory representation enabling the
retrieval of data at any level of precision. MLWeaving also takes advantage of
the increasing availability of FPGA-based accelerators to provide a highly
efficient implementation of stochastic gradient descent. The solution adopted
in MLWeaving is more efficient than existing designs in terms of space (since
it can process any resolution on the same design) and resources (via the use of
bit-serial multipliers). MLWeaving also enables the runtime tuning of
precision, instead of a fixed precision level during the training. We
illustrate this using a simple, dynamic precision schedule. Experimental
results show MLWeaving achieves up to16 performance improvement over
low-precision CPU implementations of first-order methods.
Event-Based Modeling with High-Dimensional Imaging Biomarkers for Estimating Spatial Progression of Dementia
Authors: Vikram Venkatraghavan, Florian Dubost, Esther E. Bron, Wiro J. Niessen, Marleen de Bruijne, Stefan Klein
Date: 2019-03-08 00:00:00

Event-based models (EBM) are a class of disease progression models that can
be used to estimate temporal ordering of neuropathological changes from
cross-sectional data. Current EBMs only handle scalar biomarkers, such as
regional volumes, as inputs. However, regional aggregates are a crude summary
of the underlying high-resolution images, potentially limiting the accuracy of
EBM. Therefore, we propose a novel method that exploits high-dimensional
voxel-wise imaging biomarkers: n-dimensional discriminative EBM (nDEBM). nDEBM
is based on an insight that mixture modeling, which is a key element of
conventional EBMs, can be replaced by a more scalable semi-supervised support
vector machine (SVM) approach. This SVM is used to estimate the degree of
abnormality of each region which is then used to obtain subject-specific
disease progression patterns. These patterns are in turn used for estimating
the mean ordering by fitting a generalized Mallows model. In order to validate
the biomarker ordering obtained using nDEBM, we also present a framework for
Simulation of Imaging Biomarkers' Temporal Evolution (SImBioTE) that mimics
neurodegeneration in brain regions. SImBioTE trains variational auto-encoders
(VAE) in different brain regions independently to simulate images at varying
stages of disease progression. We also validate nDEBM clinically using data
from the Alzheimer's Disease Neuroimaging Initiative (ADNI). In both
experiments, nDEBM using high-dimensional features gave better performance than
state-of-the-art EBM methods using regional volume biomarkers. This suggests
that nDEBM is a promising approach for disease progression modeling.
Unsupervised Medical Image Translation Using Cycle-MedGAN
Authors: Karim Armanious, Chenming Jiang, Sherif Abdulatif, Thomas Küstner, Sergios Gatidis, Bin Yang
Date: 2019-03-08 00:00:00
Tasks: Image-to-Image Translation, Translation

Image-to-image translation is a new field in computer vision with multiple
potential applications in the medical domain. However, for supervised image
translation frameworks, co-registered datasets, paired in a pixel-wise sense,
are required. This is often difficult to acquire in realistic medical
scenarios. On the other hand, unsupervised translation frameworks often result
in blurred translated images with unrealistic details. In this work, we propose
a new unsupervised translation framework which is titled Cycle-MedGAN. The
proposed framework utilizes new non-adversarial cycle losses which direct the
framework to minimize the textural and perceptual discrepancies in the
translated images. Qualitative and quantitative comparisons against other
unsupervised translation approaches demonstrate the performance of the proposed
framework for PET-CT translation and MR motion correction.
Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-based Image Retrieval
Authors: Anjan Dutta, Zeynep Akata
Date: 2019-03-08 00:00:00
Proceeding: CVPR 2019 6
Tasks: feature selection, Image Retrieval, Retrieval, Sketch-Based Image Retrieval

Zero-shot sketch-based image retrieval (SBIR) is an emerging task in computer
vision, allowing to retrieve natural images relevant to sketch queries that
might not been seen in the training phase. Existing works either require
aligned sketch-image pairs or inefficient memory fusion layer for mapping the
visual information to a semantic space. In this work, we propose a semantically
aligned paired cycle-consistent generative (SEM-PCYC) model for zero-shot SBIR,
where each branch maps the visual information to a common semantic space via an
adversarial training. Each of these branches maintains a cycle consistency that
only requires supervision at category levels, and avoids the need of
highly-priced aligned sketch-image pairs. A classification criteria on the
generators' outputs ensures the visual to semantic space mapping to be
discriminating. Furthermore, we propose to combine textual and hierarchical
side information via a feature selection auto-encoder that selects
discriminating side information within a same end-to-end model. Our results
demonstrate a significant boost in zero-shot SBIR performance over the
state-of-the-art on the challenging Sketchy and TU-Berlin datasets.
Large-Margin Multiple Kernel Learning for Discriminative Features Selection and Representation Learning
Authors: Babak Hosseini, Barbara Hammer
Date: 2019-03-08 00:00:00
Tasks: Binary Classification, feature selection, General Classification, Representation Learning

Multiple kernel learning (MKL) algorithms combine different base kernels to
obtain a more efficient representation in the feature space. Focusing on
discriminative tasks, MKL has been used successfully for feature selection and
finding the significant modalities of the data. In such applications, each base
kernel represents one dimension of the data or is derived from one specific
descriptor. Therefore, MKL finds an optimal weighting scheme for the given
kernels to increase the classification accuracy. Nevertheless, the majority of
the works in this area focus on only binary classification problems or aim for
linear separation of the classes in the kernel space, which are not realistic
assumptions for many real-world problems. In this paper, we propose a novel
multi-class MKL framework which improves the state-of-the-art by enhancing the
local separation of the classes in the feature space. Besides, by using a
sparsity term, our large-margin multiple kernel algorithm (LMMK) performs
discriminative feature selection by aiming to employ a small subset of the base
kernels. Based on our empirical evaluations on different real-world datasets,
LMMK provides a competitive classification accuracy compared with the
state-of-the-art algorithms in MKL. Additionally, it learns a sparse set of
non-zero kernel weights which leads to a more interpretable feature selection
and representation learning.