pwc-archive/papers-with-abstracts huggingface dataset:
SHADE: Information Based Regularization for Deep Learning
Authors: Michael Blot, Thomas Robert, Nicolas Thome, Matthieu Cord
Date: 2018-04-29 00:00:00
Tasks: Deep Learning, General Classification

Regularization is a big issue for training deep neural networks. In this
paper, we propose a new information-theory-based regularization scheme named
SHADE for SHAnnon DEcay. The originality of the approach is to define a prior
based on conditional entropy, which explicitly decouples the learning of
invariant representations in the regularizer and the learning of correlations
between inputs and labels in the data fitting term. Our second contribution is
to derive a stochastic version of the regularizer compatible with deep
learning, resulting in a tractable training scheme. We empirically validate the
efficiency of our approach to improve classification performances compared to
common regularization schemes on several standard architectures.
Beyond Unfolding: Exact Recovery of Latent Convex Tensor Decomposition under Reshuffling
Authors: Chao Li, Mohammad Emtiyaz Khan, Zhun Sun, Gang Niu, Bo Han, Shengli Xie, Qibin Zhao
Date: 2018-05-22 00:00:00
Tasks: Image Steganography, Tensor Decomposition

Exact recovery of tensor decomposition (TD) methods is a desirable property in both unsupervised learning and scientific data analysis. The numerical defects of TD methods, however, limit their practical applications on real-world data. As an alternative, convex tensor decomposition (CTD) was proposed to alleviate these problems, but its exact-recovery property is not properly addressed so far. To this end, we focus on latent convex tensor decomposition (LCTD), a practically widely-used CTD model, and rigorously prove a sufficient condition for its exact-recovery property. Furthermore, we show that such property can be also achieved by a more general model than LCTD. In the new model, we generalize the classic tensor (un-)folding into reshuffling operation, a more flexible mapping to relocate the entries of the matrix into a tensor. Armed with the reshuffling operations and exact-recovery property, we explore a totally novel application for (generalized) LCTD, i.e., image steganography. Experimental results on synthetic data validate our theory, and results on image steganography show that our method outperforms the state-of-the-art methods.
Design and Analysis of Diversity-Based Parent Selection Schemes for Speeding Up Evolutionary Multi-objective Optimisation
Authors: Edgar Covantes Osuna, Wanru Gao, Frank Neumann, Dirk Sudholt
Date: 2018-05-03 00:00:00
Tasks: Diversity, Evolutionary Algorithms

Parent selection in evolutionary algorithms for multi-objective optimisation
is usually performed by dominance mechanisms or indicator functions that prefer
non-dominated points. We propose to refine the parent selection on evolutionary
multi-objective optimisation with diversity-based metrics. The aim is to focus
on individuals with a high diversity contribution located in poorly explored
areas of the search space, so the chances of creating new non-dominated
individuals are better than in highly populated areas. We show by means of
rigorous runtime analysis that the use of diversity-based parent selection
mechanisms in the Simple Evolutionary Multi-objective Optimiser (SEMO) and
Global SEMO for the well known bi-objective functions ${\rm O{\small
NE}M{\small IN}M{\small AX}}$ and ${\rm LOTZ}$ can significantly improve their
performance. Our theoretical results are accompanied by experimental studies
that show a correspondence between theory and empirical results and motivate
further theoretical investigations in terms of stagnation. We show that
stagnation might occur when favouring individuals with a high diversity
contribution in the parent selection step and provide a discussion on which
scheme to use for more complex problems based on our theoretical and
experimental results.
Structure-preserving Guided Retinal Image Filtering and Its Application for Optic Disc Analysis
Authors: Jun Cheng, Zhengguo Li, Zaiwang Gu, Huazhu Fu, Damon Wing Kee Wong, Jiang Liu
Date: 2018-05-17 00:00:00
Tasks: Optic Cup Segmentation, Sparse Learning

Retinal fundus photographs have been used in the diagnosis of many ocular
diseases such as glaucoma, pathological myopia, age-related macular
degeneration and diabetic retinopathy. With the development of computer
science, computer aided diagnosis has been developed to process and analyse the
retinal images automatically. One of the challenges in the analysis is that the
quality of the retinal image is often degraded. For example, a cataract in
human lens will attenuate the retinal image, just as a cloudy camera lens which
reduces the quality of a photograph. It often obscures the details in the
retinal images and posts challenges in retinal image processing and analysing
tasks. In this paper, we approximate the degradation of the retinal images as a
combination of human-lens attenuation and scattering. A novel
structure-preserving guided retinal image filtering (SGRIF) is then proposed to
restore images based on the attenuation and scattering model. The proposed
SGRIF consists of a step of global structure transferring and a step of global
edge-preserving smoothing. Our results show that the proposed SGRIF method is
able to improve the contrast of retinal images, measured by histogram flatness
measure, histogram spread and variability of local luminosity. In addition, we
further explored the benefits of SGRIF for subsequent retinal image processing
and analysing tasks. In the two applications of deep learning based optic cup
segmentation and sparse learning based cup-to-disc ratio (CDR) computation, our
results show that we are able to achieve more accurate optic cup segmentation
and CDR measurements from images processed by SGRIF.
Variational Learning on Aggregate Outputs with Gaussian Processes
Authors: Ho Chung Leon Law, Dino Sejdinovic, Ewan Cameron, Tim CD Lucas, Seth Flaxman, Katherine Battle, Kenji Fukumizu
Date: 2018-05-22 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Gaussian Processes

While a typical supervised learning framework assumes that the inputs and the
outputs are measured at the same levels of granularity, many applications,
including global mapping of disease, only have access to outputs at a much
coarser level than that of the inputs. Aggregation of outputs makes
generalization to new inputs much more difficult. We consider an approach to
this problem based on variational learning with a model of output aggregation
and Gaussian processes, where aggregation leads to intractability of the
standard evidence lower bounds. We propose new bounds and tractable
approximations, leading to improved prediction accuracy and scalability to
large datasets, while explicitly taking uncertainty into account. We develop a
framework which extends to several types of likelihoods, including the Poisson
model for aggregated count data. We apply our framework to a challenging and
important problem, the fine-scale spatial modelling of malaria incidence, with
over 1 million observations.
Meta-Learning with Hessian-Free Approach in Deep Neural Nets Training
Authors: Boyu Chen, Wenlian Lu, Ernest Fokoue
Date: 2018-05-22 00:00:00
Tasks: Meta-Learning

Meta-learning is a promising method to achieve efficient training method
towards deep neural net and has been attracting increases interests in recent
years. But most of the current methods are still not capable to train complex
neuron net model with long-time training process. In this paper, a novel
second-order meta-optimizer, named Meta-learning with Hessian-Free(MLHF)
approach, is proposed based on the Hessian-Free approach. Two recurrent neural
networks are established to generate the damping and the precondition matrix of
this Hessian-Free framework. A series of techniques to meta-train the MLHF
towards stable and reinforce the meta-training of this optimizer, including the
gradient calculation of $H$. Numerical experiments on deep convolution neural
nets, including CUDA-convnet and ResNet18(v2), with datasets of CIFAR10 and
ILSVRC2012, indicate that the MLHF shows good and continuous training
performance during the whole long-time training process, i.e., both the
rapid-decreasing early stage and the steadily-deceasing later stage, and so is
a promising meta-learning framework towards elevating the training efficiency
in real-world deep neural nets.
High-dimension Tensor Completion via Gradient-based Optimization Under Tensor-train Format
Authors: Longhao Yuan, Qibin Zhao, Lihua Gui, Jianting Cao
Date: 2018-04-05 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Tensor train (TT) decomposition has drawn people's attention due to its
powerful representation ability and performance stability in high-order
tensors. In this paper, we propose a novel approach to recover the missing
entries of incomplete data represented by higher-order tensors. We attempt to
find the low-rank TT decomposition of the incomplete data which captures the
latent features of the whole data and then reconstruct the missing entries. By
applying gradient descent algorithms, tensor completion problem is efficiently
solved by optimization models. We propose two TT-based algorithms: Tensor Train
Weighted Optimization (TT-WOPT) and Tensor Train Stochastic Gradient Descent
(TT-SGD) to optimize TT decomposition factors. In addition, a method named
Visual Data Tensorization (VDT) is proposed to transform visual data into
higher-order tensors, resulting in the performance improvement of our
algorithms. The experiments in synthetic data and visual data show high
efficiency and performance of our algorithms compared to the state-of-the-art
completion algorithms, especially in high-order, high missing rate, and
large-scale tensor completion situations.
Sim-to-Real Robot Learning from Pixels with Progressive Nets
Authors: Andrei A. Rusu, Mel Vecerik, Thomas Rothörl, Nicolas Heess, Razvan Pascanu, Raia Hadsell
Date: 2016-10-13 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Robot Manipulation

Applying end-to-end learning to solve complex, interactive, pixel-driven
control tasks on a robot is an unsolved problem. Deep Reinforcement Learning
algorithms are too slow to achieve performance on a real robot, but their
potential has been demonstrated in simulated environments. We propose using
progressive networks to bridge the reality gap and transfer learned policies
from simulation to the real world. The progressive net approach is a general
framework that enables reuse of everything from low-level visual features to
high-level policies for transfer to new tasks, enabling a compositional, yet
simple, approach to building complex skills. We present an early demonstration
of this approach with a number of experiments in the domain of robot
manipulation that focus on bridging the reality gap. Unlike other proposed
approaches, our real-world experiments demonstrate successful task learning
from raw visual input on a fully actuated robot manipulator. Moreover, rather
than relying on model-based trajectory optimisation, the task learning is
accomplished using only deep reinforcement learning and sparse rewards.
Deep learning for determining a near-optimal topological design without any iteration
Authors: Yonggyun Yu, Taeil Hur, Jaeho Jung, In Gwun Jang
Date: 2018-01-13 00:00:00
Tasks: Decoder, Generative Adversarial Network

In this study, we propose a novel deep learning-based method to predict an
optimized structure for a given boundary condition and optimization setting
without using any iterative scheme. For this purpose, first, using open-source
topology optimization code, datasets of the optimized structures paired with
the corresponding information on boundary conditions and optimization settings
are generated at low (32 x 32) and high (128 x 128) resolutions. To construct
the artificial neural network for the proposed method, a convolutional neural
network (CNN)-based encoder and decoder network is trained using the training
dataset generated at low resolution. Then, as a two-stage refinement, the
conditional generative adversarial network (cGAN) is trained with the optimized
structures paired at both low and high resolutions, and is connected to the
trained CNN-based encoder and decoder network. The performance evaluation
results of the integrated network demonstrate that the proposed method can
determine a near-optimal structure in terms of pixel values and compliance with
negligible computational time.
Conditional Network Embeddings
Authors: Bo Kang, Jefrey Lijffijt, Tijl De Bie
Date: 2018-05-19 00:00:00
Proceeding: ICLR 2019 5
Tasks: General Classification, Link Prediction, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

Network Embeddings (NEs) map the nodes of a given network into
$d$-dimensional Euclidean space $\mathbb{R}^d$. Ideally, this mapping is such
that `similar' nodes are mapped onto nearby points, such that the NE can be
used for purposes such as link prediction (if `similar' means being `more
likely to be connected') or classification (if `similar' means `being more
likely to have the same label'). In recent years various methods for NE have
been introduced, all following a similar strategy: defining a notion of
similarity between nodes (typically some distance measure within the network),
a distance measure in the embedding space, and a loss function that penalizes
large distances for similar nodes and small distances for dissimilar nodes.
  A difficulty faced by existing methods is that certain networks are
fundamentally hard to embed due to their structural properties: (approximate)
multipartiteness, certain degree distributions, assortativity, etc. To overcome
this, we introduce a conceptual innovation to the NE literature and propose to
create \emph{Conditional Network Embeddings} (CNEs); embeddings that maximally
add information with respect to given structural properties (e.g. node degrees,
block densities, etc.). We use a simple Bayesian approach to achieve this, and
propose a block stochastic gradient descent algorithm for fitting it
efficiently. We demonstrate that CNEs are superior for link prediction and
multi-label classification when compared to state-of-the-art methods, and this
without adding significant mathematical or computational complexity. Finally,
we illustrate the potential of CNE for network visualization.
QBF as an Alternative to Courcelle's Theorem
Authors: Michael Lampis, Stefan Mengel, Valia Mitsou
Date: 2018-05-22 00:00:00

We propose reductions to quantified Boolean formulas (QBF) as a new approach
to showing fixed-parameter linear algorithms for problems parameterized by
treewidth. We demonstrate the feasibility of this approach by giving new
algorithms for several well-known problems from artificial intelligence that
are in general complete for the second level of the polynomial hierarchy. By
reduction from QBF we show that all resulting algorithms are essentially
optimal in their dependence on the treewidth. Most of the problems that we
consider were already known to be fixed-parameter linear by using Courcelle's
Theorem or dynamic programming, but we argue that our approach has clear
advantages over these techniques: on the one hand, in contrast to Courcelle's
Theorem, we get concrete and tight guarantees for the runtime dependence on the
treewidth. On the other hand, we avoid tedious dynamic programming and, after
showing some normalization results for CNF-formulas, our upper bounds often
boil down to a few lines.
Context-Aware Sequence-to-Sequence Models for Conversational Systems
Authors: Silje Christensen, Simen Johnsrud, Massimiliano Ruocco, Heri Ramampiaro
Date: 2018-05-22 00:00:00

This work proposes a novel approach based on sequence-to-sequence (seq2seq)
models for context-aware conversational systems. Exist- ing seq2seq models have
been shown to be good for generating natural responses in a data-driven
conversational system. However, they still lack mechanisms to incorporate
previous conversation turns. We investigate RNN-based methods that efficiently
integrate previous turns as a context for generating responses. Overall, our
experimental results based on human judgment demonstrate the feasibility and
effectiveness of the proposed approach.
Avatar-Net: Multi-scale Zero-shot Style Transfer by Feature Decoration
Authors: Lu Sheng, Ziyi Lin, Jing Shao, Xiaogang Wang
Date: 2018-05-10 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Generation, Image Reconstruction, Style Transfer

Zero-shot artistic style transfer is an important image synthesis problem
aiming at transferring arbitrary style into content images. However, the
trade-off between the generalization and efficiency in existing methods impedes
a high quality zero-shot style transfer in real-time. In this paper, we resolve
this dilemma and propose an efficient yet effective Avatar-Net that enables
visually plausible multi-scale transfer for arbitrary style. The key ingredient
of our method is a style decorator that makes up the content features by
semantically aligned style features from an arbitrary style image, which does
not only holistically match their feature distributions but also preserve
detailed style patterns in the decorated features. By embedding this module
into an image reconstruction network that fuses multi-scale style abstractions,
the Avatar-Net renders multi-scale stylization for any style image in one
feed-forward pass. We demonstrate the state-of-the-art effectiveness and
efficiency of the proposed method in generating high-quality stylized images,
with a series of applications include multiple style integration, video
stylization and etc.
Classification Uncertainty of Deep Neural Networks Based on Gradient Information
Authors: Philipp Oberdiek, Matthias Rottmann, Hanno Gottschalk
Date: 2018-05-22 00:00:00
Tasks: Classification, General Classification, Known Unknowns

We study the quantification of uncertainty of Convolutional Neural Networks
(CNNs) based on gradient metrics. Unlike the classical softmax entropy, such
metrics gather information from all layers of the CNN. We show for the EMNIST
digits data set that for several such metrics we achieve the same meta
classification accuracy -- i.e. the task of classifying predictions as correct
or incorrect without knowing the actual label -- as for entropy thresholding.
We apply meta classification to unknown concepts (out-of-distribution samples)
-- EMNIST/Omniglot letters, CIFAR10 and noise -- and demonstrate that meta
classification rates for unknown concepts can be increased when using entropy
together with several gradient based metrics as input quantities for a meta
classifier. Meta classifiers only trained on the uncertainty metrics of known
concepts, i.e. EMNIST digits, usually do not perform equally well for all
unknown concepts. If we however allow the meta classifier to be trained on
uncertainty metrics for some out-of-distribution samples, meta classification
for concepts remote from EMNIST digits (then termed known unknowns) can be
improved considerably.
Paracompositionality, MWEs and Argument Substitution
Authors: Cem Bozsahin, Arzu Burcu Guven
Date: 2018-05-22 00:00:00

Multi-word expressions, verb-particle constructions, idiomatically combining
phrases, and phrasal idioms have something in common: not all of their elements
contribute to the argument structure of the predicate implicated by the
expression.
  Radically lexicalized theories of grammar that avoid string-, term-, logical
form-, and tree-writing, and categorial grammars that avoid wrap operation,
make predictions about the categories involved in verb-particles and phrasal
idioms. They may require singleton types, which can only substitute for one
value, not just for one kind of value. These types are asymmetric: they can be
arguments only. They also narrowly constrain the kind of semantic value that
can correspond to such syntactic categories. Idiomatically combining phrases do
not subcategorize for singleton types, and they exploit another locally
computable and compositional property of a correspondence, that every syntactic
expression can project its head word. Such MWEs can be seen as empirically
realized categorial possibilities, rather than lacuna in a theory of
lexicalizable syntactic categories.
RPC Considered Harmful: Fast Distributed Deep Learning on RDMA
Authors: Jilong Xue, Youshan Miao, Cheng Chen, Ming Wu, Lintao Zhang, Lidong Zhou
Date: 2018-05-22 00:00:00
Tasks: Deep Learning

Deep learning emerges as an important new resource-intensive workload and has
been successfully applied in computer vision, speech, natural language
processing, and so on. Distributed deep learning is becoming a necessity to
cope with growing data and model sizes. Its computation is typically
characterized by a simple tensor data abstraction to model multi-dimensional
matrices, a data-flow graph to model computation, and iterative executions with
relatively frequent synchronizations, thereby making it substantially different
from Map/Reduce style distributed big data computation.
  RPC, commonly used as the communication primitive, has been adopted by
popular deep learning frameworks such as TensorFlow, which uses gRPC. We show
that RPC is sub-optimal for distributed deep learning computation, especially
on an RDMA-capable network. The tensor abstraction and data-flow graph, coupled
with an RDMA network, offers the opportunity to reduce the unnecessary overhead
(e.g., memory copy) without sacrificing programmability and generality. In
particular, from a data access point of view, a remote machine is abstracted
just as a "device" on an RDMA channel, with a simple memory interface for
allocating, reading, and writing memory regions. Our graph analyzer looks at
both the data flow graph and the tensors to optimize memory allocation and
remote data access using this interface. The result is up to 25 times speedup
in representative deep learning benchmarks against the standard gRPC in
TensorFlow and up to 169% improvement even against an RPC implementation
optimized for RDMA, leading to faster convergence in the training process.
Bayesian Inference of Regular Expressions from Human-Generated Example Strings
Authors: Long Ouyang
Date: 2018-05-22 00:00:00
Tasks: Bayesian Inference

In programming by example, users "write" programs by generating a small
number of input-output examples and asking the computer to synthesize
consistent programs. We consider a challenging problem in this domain: learning
regular expressions (regexes) from positive and negative example strings. This
problem is challenging, as (1) user-generated examples may not be informative
enough to sufficiently constrain the hypothesis space, and (2) even if
user-generated examples are in principle informative, there is still a massive
search space to examine. We frame regex induction as the problem of inferring a
probabilistic regular grammar and propose an efficient inference approach that
uses a novel stochastic process recognition model. This model incrementally
"grows" a grammar using positive examples as a scaffold. We show that this
approach is competitive with human ability to learn regexes from examples.
Cubes3D: Neural Network based Optical Flow in Omnidirectional Image Scenes
Authors: André Apitzsch, Roman Seidel, Gangolf Hirtz
Date: 2018-04-24 00:00:00
Tasks: Object, Optical Flow Estimation

Optical flow estimation with convolutional neural networks (CNNs) has
recently solved various tasks of computer vision successfully. In this paper we
adapt a state-of-the-art approach for optical flow estimation to
omnidirectional images. We investigate CNN architectures to determine high
motion variations caused by the geometry of fish-eye images. Further we
determine the qualitative influence of texture on the non-rigid object to the
motion vectors. For evaluation of the results we create ground truth motion
fields synthetically. The ground truth contains cubes with static background.
We test variations of pre-trained FlowNet 2.0 architectures by indicating
common error metrics. We generate competitive results for the motion of the
foreground with inhomogeneous texture on the moving object.