pwc-archive/papers-with-abstracts huggingface dataset:
Super-Resolution using Convolutional Neural Networks without Any Checkerboard Artifacts
Authors: Yusuke Sugawara, Sayaka Shiota, Hitoshi Kiya
Date: 2018-06-07 00:00:00
Tasks: Super-Resolution

It is well-known that a number of excellent super-resolution (SR) methods
using convolutional neural networks (CNNs) generate checkerboard artifacts. A
condition to avoid the checkerboard artifacts is proposed in this paper. So
far, checkerboard artifacts have been mainly studied for linear multirate
systems, but the condition to avoid checkerboard artifacts can not be applied
to CNNs due to the non-linearity of CNNs. We extend the avoiding condition for
CNNs, and apply the proposed structure to some typical SR methods to confirm
the effectiveness of the new scheme. Experiment results demonstrate that the
proposed structure can perfectly avoid to generate checkerboard artifacts under
two loss conditions: mean square error and perceptual loss, while keeping
excellent properties that the SR methods have.
Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising
Authors: Borja Balle, Yu-Xiang Wang
Date: 2018-05-16 00:00:00
Proceeding: ICML 2018 7
Tasks: Denoising

The Gaussian mechanism is an essential building block used in multitude of
differentially private data analysis algorithms. In this paper we revisit the
Gaussian mechanism and show that the original analysis has several important
limitations. Our analysis reveals that the variance formula for the original
mechanism is far from tight in the high privacy regime ($\varepsilon \to 0$)
and it cannot be extended to the low privacy regime ($\varepsilon \to \infty$).
We address these limitations by developing an optimal Gaussian mechanism whose
variance is calibrated directly using the Gaussian cumulative density function
instead of a tail bound approximation. We also propose to equip the Gaussian
mechanism with a post-processing step based on adaptive estimation techniques
by leveraging that the distribution of the perturbation is known. Our
experiments show that analytical calibration removes at least a third of the
variance of the noise compared to the classical Gaussian mechanism, and that
denoising dramatically improves the accuracy of the Gaussian mechanism in the
high-dimensional regime.
Augment and Reduce: Stochastic Inference for Large Categorical Distributions
Authors: Francisco J. R. Ruiz, Michalis K. Titsias, Adji B. Dieng, David M. Blei
Date: 2018-02-12 00:00:00
Proceeding: ICML 2018
Tasks: General Classification, Recommendation Systems, Variational Inference

Categorical distributions are ubiquitous in machine learning, e.g., in
classification, language models, and recommendation systems. However, when the
number of possible outcomes is very large, using categorical distributions
becomes computationally expensive, as the complexity scales linearly with the
number of outcomes. To address this problem, we propose augment and reduce
(A&R), a method to alleviate the computational complexity. A&R uses two ideas:
latent variable augmentation and stochastic variational inference. It maximizes
a lower bound on the marginal likelihood of the data. Unlike existing methods
which are specific to softmax, A&R is more general and is amenable to other
categorical models, such as multinomial probit. On several large-scale
classification problems, we show that A&R provides a tighter bound on the
marginal likelihood and has better predictive performance than existing
approaches.
Re-evaluating Evaluation
Authors: David Balduzzi, Karl Tuyls, Julien Perolat, Thore Graepel
Date: 2018-06-07 00:00:00
Proceeding: NeurIPS 2018 12

Progress in machine learning is measured by careful evaluation on problems of
outstanding common interest. However, the proliferation of benchmark suites and
environments, adversarial attacks, and other complications has diluted the
basic evaluation model by overwhelming researchers with choices. Deliberate or
accidental cherry picking is increasingly likely, and designing well-balanced
evaluation suites requires increasing effort. In this paper we take a step back
and propose Nash averaging. The approach builds on a detailed analysis of the
algebraic structure of evaluation in two basic scenarios: agent-vs-agent and
agent-vs-task. The key strength of Nash averaging is that it automatically
adapts to redundancies in evaluation data, so that results are not biased by
the incorporation of easy tasks or weak agents. Nash averaging thus encourages
maximally inclusive evaluation -- since there is no harm (computational cost
aside) from including all available tasks and agents.
Visual Place Recognition with Probabilistic Vertex Voting
Authors: Mathias Gehrig, Elena Stumm, Timo Hinzmann, Roland Siegwart
Date: 2016-10-11 00:00:00
Tasks: Retrieval, Visual Place Recognition

We propose a novel scoring concept for visual place recognition based on
nearest neighbor descriptor voting and demonstrate how the algorithm naturally
emerges from the problem formulation. Based on the observation that the number
of votes for matching places can be evaluated using a binomial distribution
model, loop closures can be detected with high precision. By casting the
problem into a probabilistic framework, we not only remove the need for
commonly employed heuristic parameters but also provide a powerful score to
classify matching and non-matching places. We present methods for both a 2D-2D
pose-graph vertex matching and a 2D-3D landmark matching based on the above
scoring. The approach maintains accuracy while being efficient enough for
online application through the use of compact (low dimensional) descriptors and
fast nearest neighbor retrieval techniques. The proposed methods are evaluated
on several challenging datasets in varied environments, showing
state-of-the-art results with high precision and high recall.
Effects of Word Embeddings on Neural Network-based Pitch Accent Detection
Authors: Sabrina Stehwien, Ngoc Thang Vu, Antje Schweitzer
Date: 2018-05-14 00:00:00
Tasks: Cross-corpus, Word Embeddings

Pitch accent detection often makes use of both acoustic and lexical features
based on the fact that pitch accents tend to correlate with certain words. In
this paper, we extend a pitch accent detector that involves a convolutional
neural network to include word embeddings, which are state-of-the-art vector
representations of words. We examine the effect these features have on
within-corpus and cross-corpus experiments on three English datasets. The
results show that while word embeddings can improve the performance in
corpus-dependent experiments, they also have the potential to make
generalization to unseen data more challenging.
Path-Level Network Transformation for Efficient Architecture Search
Authors: Han Cai, Jiacheng Yang, Wei-Nan Zhang, Song Han, Yong Yu
Date: 2018-06-07 00:00:00
Proceeding: ICML 2018 7
Tasks: GPU, image-classification, Image Classification, Neural Architecture Search, Reinforcement Learning

We introduce a new function-preserving transformation for efficient neural
architecture search. This network transformation allows reusing previously
trained networks and existing successful architectures that improves sample
efficiency. We aim to address the limitation of current network transformation
operations that can only perform layer-level architecture modifications, such
as adding (pruning) filters or inserting (removing) a layer, which fails to
change the topology of connection paths. Our proposed path-level transformation
operations enable the meta-controller to modify the path topology of the given
network while keeping the merits of reusing weights, and thus allow efficiently
designing effective structures with complex path topologies like Inception
models. We further propose a bidirectional tree-structured reinforcement
learning meta-controller to explore a simple yet highly expressive
tree-structured architecture space that can be viewed as a generalization of
multi-branch architectures. We experimented on the image classification
datasets with limited computational resources (about 200 GPU-hours), where we
observed improved parameter efficiency and better test results (97.70% test
accuracy on CIFAR-10 with 14.3M parameters and 74.6% top-1 accuracy on ImageNet
in the mobile setting), demonstrating the effectiveness and transferability of
our designed architectures.
Multi-objective Analysis of MAP-Elites Performance
Authors: Eivind Samuelsen, Kyrre Glette
Date: 2018-03-14 00:00:00

In certain complex optimization tasks, it becomes necessary to use multiple
measures to characterize the performance of different algorithms. This paper
presents a method that combines ordinal effect sizes with Pareto dominance to
analyze such cases. Since the method is ordinal, it can also generalize across
different optimization tasks even when the performance measurements are
differently scaled. Through a case study, we show that this method can discover
and quantify relations that would be difficult to deduce using a conventional
measure-by-measure analysis. This case study applies the method to the
evolution of robot controller repertoires using the MAP-Elites algorithm. Here,
we analyze the search performance across a large set of parametrizations;
varying mutation size and operator type, as well as map resolution, across four
different robot morphologies. We show that the average magnitude of mutations
has a bigger effect on outcomes than their precise distributions.
Reducing Deep Network Complexity via Sparse Hierarchical Fourier Interaction Networks
Authors: Andrew Kiruluta, Samantha Williams
Date: 2017-12-15 00:00:00
Tasks: General Classification

This paper presents a Sparse Hierarchical Fourier Interaction Networks, an architectural building block that unifies three complementary principles of frequency domain modeling: A hierarchical patch wise Fourier transform that affords simultaneous access to local detail and global context; A learnable, differentiable top K masking mechanism which retains only the most informative spectral coefficients, thereby exploiting the natural compressibility of visual and linguistic signals.
Layer-wise Learning of Stochastic Neural Networks with Information Bottleneck
Authors: Thanh T. Nguyen, Jaesik Choi
Date: 2017-12-04 00:00:00
Tasks: Adversarial Robustness

Information Bottleneck (IB) is a generalization of rate-distortion theory that naturally incorporates compression and relevance trade-offs for learning. Though the original IB has been extensively studied, there has not been much understanding of multiple bottlenecks which better fit in the context of neural networks. In this work, we propose Information Multi-Bottlenecks (IMBs) as an extension of IB to multiple bottlenecks which has a direct application to training neural networks by considering layers as multiple bottlenecks and weights as parameterized encoders and decoders. We show that the multiple optimality of IMB is not simultaneously achievable for stochastic encoders. We thus propose a simple compromised scheme of IMB which in turn generalizes maximum likelihood estimate (MLE) principle in the context of stochastic neural networks. We demonstrate the effectiveness of IMB on classification tasks and adversarial robustness in MNIST and CIFAR10.
Recommendations with Negative Feedback via Pairwise Deep Reinforcement Learning
Authors: Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Long Xia, Jiliang Tang, Dawei Yin
Date: 2018-02-19 00:00:00
Tasks: Deep Reinforcement Learning, Recommendation Systems, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recommender systems play a crucial role in mitigating the problem of
information overload by suggesting users' personalized items or services. The
vast majority of traditional recommender systems consider the recommendation
procedure as a static process and make recommendations following a fixed
strategy. In this paper, we propose a novel recommender system with the
capability of continuously improving its strategies during the interactions
with users. We model the sequential interactions between users and a
recommender system as a Markov Decision Process (MDP) and leverage
Reinforcement Learning (RL) to automatically learn the optimal strategies via
recommending trial-and-error items and receiving reinforcements of these items
from users' feedback. Users' feedback can be positive and negative and both
types of feedback have great potentials to boost recommendations. However, the
number of negative feedback is much larger than that of positive one; thus
incorporating them simultaneously is challenging since positive feedback could
be buried by negative one. In this paper, we develop a novel approach to
incorporate them into the proposed deep recommender system (DEERS) framework.
The experimental results based on real-world e-commerce data demonstrate the
effectiveness of the proposed framework. Further experiments have been
conducted to understand the importance of both positive and negative feedback
in recommendations.
Challenges in Monocular Visual Odometry: Photometric Calibration, Motion Bias and Rolling Shutter Effect
Authors: Nan Yang, Rui Wang, Xiang Gao, Daniel Cremers
Date: 2017-05-11 00:00:00
Tasks: Monocular Visual Odometry, Simultaneous Localization and Mapping, Visual Odometry

Monocular visual odometry (VO) and simultaneous localization and mapping
(SLAM) have seen tremendous improvements in accuracy, robustness and
efficiency, and have gained increasing popularity over recent years.
Nevertheless, not so many discussions have been carried out to reveal the
influences of three very influential yet easily overlooked aspects: photometric
calibration, motion bias and rolling shutter effect. In this work, we evaluate
these three aspects quantitatively on the state of the art of direct,
feature-based and semi-direct methods, providing the community with useful
practical knowledge both for better applying existing methods and developing
new algorithms of VO and SLAM. Conclusions (some of which are
counter-intuitive) are drawn with both technical and empirical analyses to all
of our experiments. Possible improvements on existing methods are directed or
proposed, such as a sub-pixel accuracy refinement of ORB-SLAM which boosts its
performance.
Upper Bounds on the Runtime of the Univariate Marginal Distribution Algorithm on OneMax
Authors: Carsten Witt
Date: 2017-03-31 00:00:00

A runtime analysis of the Univariate Marginal Distribution Algorithm (UMDA)
is presented on the OneMax function for wide ranges of its parameters $\mu$ and
$\lambda$. If $\mu\ge c\log n$ for some constant $c>0$ and
$\lambda=(1+\Theta(1))\mu$, a general bound $O(\mu n)$ on the expected runtime
is obtained. This bound crucially assumes that all marginal probabilities of
the algorithm are confined to the interval $[1/n,1-1/n]$. If $\mu\ge c'
\sqrt{n}\log n$ for a constant $c'>0$ and $\lambda=(1+\Theta(1))\mu$, the
behavior of the algorithm changes and the bound on the expected runtime becomes
$O(\mu\sqrt{n})$, which typically even holds if the borders on the marginal
probabilities are omitted.
  The results supplement the recently derived lower bound
$\Omega(\mu\sqrt{n}+n\log n)$ by Krejca and Witt (FOGA 2017) and turn out as
tight for the two very different values $\mu=c\log n$ and $\mu=c'\sqrt{n}\log
n$. They also improve the previously best known upper bound $O(n\log n\log\log
n)$ by Dang and Lehre (GECCO 2015).
An Evaluation of Trajectory Prediction Approaches and Notes on the TrajNet Benchmark
Authors: Stefan Becker, Ronny Hug, Wolfgang Hübner, Michael Arens
Date: 2018-05-19 00:00:00
Tasks: Trajectory Prediction

In recent years, there is a shift from modeling the tracking problem based on
Bayesian formulation towards using deep neural networks. Towards this end, in
this paper the effectiveness of various deep neural networks for predicting
future pedestrian paths are evaluated. The analyzed deep networks solely rely,
like in the traditional approaches, on observed tracklets without human-human
interaction information. The evaluation is done on the publicly available
TrajNet benchmark dataset, which builds up a repository of considerable and
popular datasets for trajectory-based activity forecasting. We show that a
Recurrent-Encoder with a Dense layer stacked on top, referred to as
RED-predictor, is able to achieve sophisticated results compared to elaborated
models in such scenarios. Further, we investigate failure cases and give
explanations for observed phenomena and give some recommendations for
overcoming demonstrated shortcomings.
Asynchronous Stochastic Quasi-Newton MCMC for Non-Convex Optimization
Authors: Umut Şimşekli, Çağatay Yıldız, Thanh Huy Nguyen, Gaël Richard, A. Taylan Cemgil
Date: 2018-06-07 00:00:00
Proceeding: ICML 2018

Recent studies have illustrated that stochastic gradient Markov Chain Monte
Carlo techniques have a strong potential in non-convex optimization, where
local and global convergence guarantees can be shown under certain conditions.
By building up on this recent theory, in this study, we develop an
asynchronous-parallel stochastic L-BFGS algorithm for non-convex optimization.
The proposed algorithm is suitable for both distributed and shared-memory
settings. We provide formal theoretical analysis and show that the proposed
method achieves an ergodic convergence rate of ${\cal O}(1/\sqrt{N})$ ($N$
being the total number of iterations) and it can achieve a linear speedup under
certain conditions. We perform several experiments on both synthetic and real
datasets. The results support our theory and show that the proposed algorithm
provides a significant speedup over the recently proposed synchronous
distributed L-BFGS algorithm.
Writing Style Invariant Deep Learning Model for Historical Manuscripts Alignment
Authors: Majeed Kassis, Jumana Nassour, Jihad El-Sana
Date: 2018-06-07 00:00:00
Tasks: Deep Learning

Historical manuscript alignment is a widely known problem in document
analysis. Finding the differences between manuscript editions is mostly done
manually. In this paper, we present a writer independent deep learning model
which is trained on several writing styles, and able to achieve high detection
accuracy when tested on writing styles not present in training data. We test
our model using cross validation, each time we train the model on five
manuscripts, and test it on the other two manuscripts, never seen in the
training data. We've applied cross validation on seven manuscripts, netting 21
different tests, achieving average accuracy of $\%92.17$. We also present a new
alignment algorithm based on dynamic sized sliding window, which is able to
successfully handle complex cases.
Nonparametric Density Flows for MRI Intensity Normalisation
Authors: Daniel C. Castro, Ben Glocker
Date: 2018-06-07 00:00:00
Tasks: Medical Image Analysis

With the adoption of powerful machine learning methods in medical image
analysis, it is becoming increasingly desirable to aggregate data that is
acquired across multiple sites. However, the underlying assumption of many
analysis techniques that corresponding tissues have consistent intensities in
all images is often violated in multi-centre databases. We introduce a novel
intensity normalisation scheme based on density matching, wherein the
histograms are modelled as Dirichlet process Gaussian mixtures. The source
mixture model is transformed to minimise its $L^2$ divergence towards a target
model, then the voxel intensities are transported through a mass-conserving
flow to maintain agreement with the moving density. In a multi-centre study
with brain MRI data, we show that the proposed technique produces excellent
correspondence between the matched densities and histograms. We further
demonstrate that our method makes tissue intensity statistics substantially
more compatible between images than a baseline affine transformation and is
comparable to state-of-the-art while providing considerably smoother
transformations. Finally, we validate that nonlinear intensity normalisation is
a step toward effective imaging data harmonisation.
Dimensionality-Driven Learning with Noisy Labels
Authors: Xingjun Ma, Yisen Wang, Michael E. Houle, Shuo Zhou, Sarah M. Erfani, Shu-Tao Xia, Sudanthi Wijewickrema, James Bailey
Date: 2018-06-07 00:00:00
Proceeding: ICML 2018 7
Tasks: Image Classification, Learning with noisy labels

Datasets with significant proportions of noisy (incorrect) class labels
present challenges for training accurate Deep Neural Networks (DNNs). We
propose a new perspective for understanding DNN generalization for such
datasets, by investigating the dimensionality of the deep representation
subspace of training samples. We show that from a dimensionality perspective,
DNNs exhibit quite distinctive learning styles when trained with clean labels
versus when trained with a proportion of noisy labels. Based on this finding,
we develop a new dimensionality-driven learning strategy, which monitors the
dimensionality of subspaces during training and adapts the loss function
accordingly. We empirically demonstrate that our approach is highly tolerant to
significant proportions of noisy labels, and can effectively learn
low-dimensional local subspaces that capture the data distribution.
Learning Multi-Modal Self-Awareness Models for Autonomous Vehicles from Human Driving
Authors: Mahdyar Ravanbakhsh, Mohamad Baydoun, Damian Campo, Pablo Marin, David Martin, Lucio Marcenaro, Carlo S. Regazzoni
Date: 2018-06-07 00:00:00
Tasks: Anomaly Detection, Autonomous Vehicles, Decision Making

This paper presents a novel approach for learning self-awareness models for
autonomous vehicles. The proposed technique is based on the availability of
synchronized multi-sensor dynamic data related to different maneuvering tasks
performed by a human operator. It is shown that different machine learning
approaches can be used to first learn single modality models using coupled
Dynamic Bayesian Networks; such models are then correlated at event level to
discover contextual multi-modal concepts. In the presented case, visual
perception and localization are used as modalities. Cross-correlations among
modalities in time is discovered from data and are described as probabilistic
links connecting shared and private multi-modal DBNs at the event (discrete)
level. Results are presented on experiments performed on an autonomous vehicle,
highlighting potentiality of the proposed approach to allow anomaly detection
and autonomous decision making based on learned self-awareness models.
AdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimation
Authors: Jogendra Nath Kundu, Phani Krishna Uppala, Anuj Pahuja, R. Venkatesh Babu
Date: 2018-03-05 00:00:00
Proceeding: CVPR 2018 6
Tasks: Depth Estimation, Domain Adaptation, Monocular Depth Estimation, Unsupervised Domain Adaptation

Supervised deep learning methods have shown promising results for the task of
monocular depth estimation; but acquiring ground truth is costly, and prone to
noise as well as inaccuracies. While synthetic datasets have been used to
circumvent above problems, the resultant models do not generalize well to
natural scenes due to the inherent domain shift. Recent adversarial approaches
for domain adaption have performed well in mitigating the differences between
the source and target domains. But these methods are mostly limited to a
classification setup and do not scale well for fully-convolutional
architectures. In this work, we propose AdaDepth - an unsupervised domain
adaptation strategy for the pixel-wise regression task of monocular depth
estimation. The proposed approach is devoid of above limitations through a)
adversarial learning and b) explicit imposition of content consistency on the
adapted target representation. Our unsupervised approach performs competitively
with other established approaches on depth estimation tasks and achieves
state-of-the-art results in a semi-supervised setting.