pwc-archive/papers-with-abstracts huggingface dataset:
Image denoising with generalized Gaussian mixture model patch priors
Authors: Charles-Alban Deledalle, Shibin Parameswaran, Truong Q. Nguyen
Date: 2018-02-05 00:00:00
Tasks: Denoising, Image Denoising, Image Restoration

Patch priors have become an important component of image restoration. A
powerful approach in this category of restoration algorithms is the popular
Expected Patch Log-Likelihood (EPLL) algorithm. EPLL uses a Gaussian mixture
model (GMM) prior learned on clean image patches as a way to regularize
degraded patches. In this paper, we show that a generalized Gaussian mixture
model (GGMM) captures the underlying distribution of patches better than a GMM.
Even though GGMM is a powerful prior to combine with EPLL, the non-Gaussianity
of its components presents major challenges to be applied to a computationally
intensive process of image restoration. Specifically, each patch has to undergo
a patch classification step and a shrinkage step. These two steps can be
efficiently solved with a GMM prior but are computationally impractical when
using a GGMM prior. In this paper, we provide approximations and computational
recipes for fast evaluation of these two steps, so that EPLL can embed a GGMM
prior on an image with more than tens of thousands of patches. Our main
contribution is to analyze the accuracy of our approximations based on thorough
theoretical analysis. Our evaluations indicate that the GGMM prior is
consistently a better fit formodeling image patch distribution and performs
better on average in image denoising task.
Second-Order Asymptotically Optimal Statistical Classification
Authors: Lin Zhou, Vincent Y. F. Tan, Mehul Motani
Date: 2018-06-03 00:00:00
Tasks: Classification, General Classification, Two-sample testing, Vocal Bursts Type Prediction

Motivated by real-world machine learning applications, we analyze
approximations to the non-asymptotic fundamental limits of statistical
classification. In the binary version of this problem, given two training
sequences generated according to two {\em unknown} distributions $P_1$ and
$P_2$, one is tasked to classify a test sequence which is known to be generated
according to either $P_1$ or $P_2$. This problem can be thought of as an
analogue of the binary hypothesis testing problem but in the present setting,
the generating distributions are unknown. Due to finite sample considerations,
we consider the second-order asymptotics (or dispersion-type) tradeoff between
type-I and type-II error probabilities for tests which ensure that (i) the
type-I error probability for {\em all} pairs of distributions decays
exponentially fast and (ii) the type-II error probability for a {\em
particular} pair of distributions is non-vanishing. We generalize our results
to classification of multiple hypotheses with the rejection option.
When and where do feed-forward neural networks learn localist representations?
Authors: Ella M. Gale, Nicolas Martin, Jeffrey S. Bowers
Date: 2018-06-11 00:00:00
Proceeding: ICLR 2018 1

According to parallel distributed processing (PDP) theory in psychology,
neural networks (NN) learn distributed rather than interpretable localist
representations. This view has been held so strongly that few researchers have
analysed single units to determine if this assumption is correct. However,
recent results from psychology, neuroscience and computer science have shown
the occasional existence of local codes emerging in artificial and biological
neural networks. In this paper, we undertake the first systematic survey of
when local codes emerge in a feed-forward neural network, using generated input
and output data with known qualities. We find that the number of local codes
that emerge from a NN follows a well-defined distribution across the number of
hidden layer neurons, with a peak determined by the size of input data, number
of examples presented and the sparsity of input data. Using a 1-hot output code
drastically decreases the number of local codes on the hidden layer. The number
of emergent local codes increases with the percentage of dropout applied to the
hidden layer, suggesting that the localist encoding may offer a resilience to
noisy networks. This data suggests that localist coding can emerge from
feed-forward PDP networks and suggests some of the conditions that may lead to
interpretable localist representations in the cortex. The findings highlight
how local codes should not be dismissed out of hand.
Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
Authors: Lars Mescheder, Sebastian Nowozin, Andreas Geiger
Date: 2017-01-17 00:00:00
Proceeding: ICML 2017 8

Variational Autoencoders (VAEs) are expressive latent variable models that
can be used to learn complex probability distributions from training data.
However, the quality of the resulting model crucially relies on the
expressiveness of the inference model. We introduce Adversarial Variational
Bayes (AVB), a technique for training Variational Autoencoders with arbitrarily
expressive inference models. We achieve this by introducing an auxiliary
discriminative network that allows to rephrase the maximum-likelihood-problem
as a two-player game, hence establishing a principled connection between VAEs
and Generative Adversarial Networks (GANs). We show that in the nonparametric
limit our method yields an exact maximum-likelihood assignment for the
parameters of the generative model, as well as the exact posterior distribution
over the latent variables given an observation. Contrary to competing
approaches which combine VAEs with GANs, our approach has a clear theoretical
justification, retains most advantages of standard Variational Autoencoders and
is easy to implement.
KBLRN : End-to-End Learning of Knowledge Base Representations with Latent, Relational, and Numerical Features
Authors: Alberto Garcia-Duran, Mathias Niepert
Date: 2017-09-14 00:00:00
Tasks: Knowledge Base Completion, Representation Learning

We present KBLRN, a framework for end-to-end learning of knowledge base
representations from latent, relational, and numerical features. KBLRN
integrates feature types with a novel combination of neural representation
learning and probabilistic product of experts models. To the best of our
knowledge, KBLRN is the first approach that learns representations of knowledge
bases by integrating latent, relational, and numerical features. We show that
instances of KBLRN outperform existing methods on a range of knowledge base
completion tasks. We contribute a novel data sets enriching commonly used
knowledge base completion benchmarks with numerical features. The data sets are
available under a permissive BSD-3 license. We also investigate the impact
numerical features have on the KB completion performance of KBLRN.
3D Face Reconstruction with Geometry Details from a Single Image
Authors: Luo Jiang, Juyong Zhang, Bailin Deng, Hao Li, Ligang Liu
Date: 2017-02-18 00:00:00
Tasks: 3D Face Reconstruction, Face Model, Face Reconstruction

3D face reconstruction from a single image is a classical and challenging
problem, with wide applications in many areas. Inspired by recent works in face
animation from RGB-D or monocular video inputs, we develop a novel method for
reconstructing 3D faces from unconstrained 2D images, using a coarse-to-fine
optimization strategy. First, a smooth coarse 3D face is generated from an
example-based bilinear face model, by aligning the projection of 3D face
landmarks with 2D landmarks detected from the input image. Afterwards, using
local corrective deformation fields, the coarse 3D face is refined using
photometric consistency constraints, resulting in a medium face shape. Finally,
a shape-from-shading method is applied on the medium face to recover fine
geometric details. Our method outperforms state-of-the-art approaches in terms
of accuracy and detail recovery, which is demonstrated in extensive experiments
using real world models and publicly available datasets.
Gear Training: A new way to implement high-performance model-parallel training
Authors: Hao Dong, Shuai Li, Dongchang Xu, Yi Ren, Di Zhang
Date: 2018-06-11 00:00:00
Tasks: GPU

The training of Deep Neural Networks usually needs tremendous computing
resources. Therefore many deep models are trained in large cluster instead of
single machine or GPU. Though major researchs at present try to run whole model
on all machines by using asynchronous asynchronous stochastic gradient descent
(ASGD), we present a new approach to train deep model parallely -- split the
model and then seperately train different parts of it in different speed.
Convergence Rates for Projective Splitting
Authors: Patrick R. Johnstone, Jonathan Eckstein
Date: 2018-06-11 00:00:00

Projective splitting is a family of methods for solving inclusions involving
sums of maximal monotone operators. First introduced by Eckstein and Svaiter in
2008, these methods have enjoyed significant innovation in recent years,
becoming one of the most flexible operator splitting frameworks available.
While weak convergence of the iterates to a solution has been established,
there have been few attempts to study convergence rates of projective
splitting. The purpose of this paper is to do so under various assumptions. To
this end, there are three main contributions. First, in the context of convex
optimization, we establish an $O(1/k)$ ergodic function convergence rate.
Second, for strongly monotone inclusions, strong convergence is established as
well as an ergodic $O(1/\sqrt{k})$ convergence rate for the distance of the
iterates to the solution. Finally, for inclusions featuring strong monotonicity
and cocoercivity, linear convergence is established.
Learning Classifiers with Fenchel-Young Losses: Generalized Entropies, Margins, and Algorithms
Authors: Mathieu Blondel, André F. T. Martins, Vlad Niculae
Date: 2018-05-24 00:00:00

This paper studies Fenchel-Young losses, a generic way to construct convex
loss functions from a regularization function. We analyze their properties in
depth, showing that they unify many well-known loss functions and allow to
create useful new ones easily. Fenchel-Young losses constructed from a
generalized entropy, including the Shannon and Tsallis entropies, induce
predictive probability distributions. We formulate conditions for a generalized
entropy to yield losses with a separation margin, and probability distributions
with sparse support. Finally, we derive efficient algorithms, making
Fenchel-Young losses appealing both in theory and practice.
Low-Rank Inducing Norms with Optimality Interpretations
Authors: Christian Grussler, Pontus Giselsson
Date: 2016-12-09 00:00:00
Tasks: Matrix Completion

Optimization problems with rank constraints appear in many diverse fields
such as control, machine learning and image analysis. Since the rank constraint
is non-convex, these problems are often approximately solved via convex
relaxations. Nuclear norm regularization is the prevailing convexifying
technique for dealing with these types of problem. This paper introduces a
family of low-rank inducing norms and regularizers which includes the nuclear
norm as a special case. A posteriori guarantees on solving an underlying rank
constrained optimization problem with these convex relaxations are provided. We
evaluate the performance of the low-rank inducing norms on three matrix
completion problems. In all examples, the nuclear norm heuristic is
outperformed by convex relaxations based on other low-rank inducing norms. For
two of the problems there exist low-rank inducing norms that succeed in
recovering the partially unknown matrix, while the nuclear norm fails. These
low-rank inducing norms are shown to be representable as semi-definite
programs. Moreover, these norms have cheaply computable proximal mappings,
which makes it possible to also solve problems of large size using first-order
methods.
Retinal Optic Disc Segmentation using Conditional Generative Adversarial Network
Authors: Vivek Kumar Singh, Hatem Rashwan, Farhan Akram, Nidhi Pandey, Md. Mostaf Kamal Sarker, Adel Saleh, Saddam Abdulwahab, Najlaa Maaroof, Santiago Romani, Domenec Puig
Date: 2018-06-11 00:00:00
Tasks: Generative Adversarial Network, GPU, Image Segmentation, Optic Disc Segmentation, Segmentation, Semantic Segmentation

This paper proposed a retinal image segmentation method based on conditional
Generative Adversarial Network (cGAN) to segment optic disc. The proposed model
consists of two successive networks: generator and discriminator. The generator
learns to map information from the observing input (i.e., retinal fundus color
image), to the output (i.e., binary mask). Then, the discriminator learns as a
loss function to train this mapping by comparing the ground-truth and the
predicted output with observing the input image as a condition.Experiments were
performed on two publicly available dataset; DRISHTI GS1 and RIM-ONE. The
proposed model outperformed state-of-the-art-methods by achieving around 0.96%
and 0.98% of Jaccard and Dice coefficients, respectively. Moreover, an image
segmentation is performed in less than a second on recent GPU.
Multi-task learning of daily work and study round-trips from survey data
Authors: Mehdi Katranji, Sami Kraiem, Laurent Moalic, Guilhem Sanmarty, Alexandre Caminada, Fouad Hadj Selem
Date: 2018-06-11 00:00:00
Tasks: Decision Making, Multi-Task Learning

In this study, we present a machine learning approach to infer the worker and
student mobility flows on daily basis from static censuses. The rapid
urbanization has made the estimation of the human mobility flows a critical
task for transportation and urban planners. The primary objective of this paper
is to complete individuals' census data with working and studying trips,
allowing its merging with other mobility data to better estimate the complete
origin-destination matrices. Worker and student mobility flows are among the
most weekly regular displacements and consequently generate road congestion
problems. Estimating their round-trips eases the decision-making processes for
local authorities. Worker and student censuses often contain home location,
work places and educational institutions. We thus propose a neural network
model that learns the temporal distribution of displacements from other
mobility sources and tries to predict them on new censuses data. The inclusion
of multi-task learning in our neural network results in a significant error
rate control in comparison to single task learning.
Dual Pattern Learning Networks by Empirical Dual Prediction Risk Minimization
Authors: Haimin Zhang, Min Xu
Date: 2018-06-11 00:00:00
Tasks: image-classification, Image Classification

Motivated by the observation that humans can learn patterns from two given
images at one time, we propose a dual pattern learning network architecture in
this paper. Unlike conventional networks, the proposed architecture has two
input branches and two loss functions. Instead of minimizing the empirical risk
of a given dataset, dual pattern learning networks is trained by minimizing the
empirical dual prediction loss. We show that this can improve the performance
for single image classification. This architecture forces the network to learn
discriminative class-specific features by analyzing and comparing two input
images. In addition, the dual input structure allows the network to have a
considerably large number of image pairs, which can help address the
overfitting issue due to limited training data. Moreover, we propose to
associate each input branch with a random interest value for learning
corresponding image during training. This method can be seen as a stochastic
regularization technique, and can further lead to generalization performance
improvement. State-of-the-art deep networks can be adapted to dual pattern
learning networks without increasing the same number of parameters. Extensive
experiments on CIFAR-10, CIFAR- 100, FI-8, Google commands dataset, and MNIST
demonstrate that our DPLNets exhibit better performance than original networks.
The experimental results on subsets of CIFAR- 10, CIFAR-100, and MNIST
demonstrate that dual pattern learning networks have good generalization
performance on small datasets.
Natasha 2: Faster Non-Convex Optimization Than SGD
Authors: Zeyuan Allen-Zhu
Date: 2017-08-29 00:00:00
Proceeding: NeurIPS 2018 12

We design a stochastic algorithm to train any smooth neural network to
$\varepsilon$-approximate local minima, using $O(\varepsilon^{-3.25})$
backpropagations. The best result was essentially $O(\varepsilon^{-4})$ by SGD.
  More broadly, it finds $\varepsilon$-approximate local minima of any smooth
nonconvex function in rate $O(\varepsilon^{-3.25})$, with only oracle access to
stochastic gradients.
Large-scale Bisample Learning on ID Versus Spot Face Recognition
Authors: Xiangyu Zhu, Hao liu, Zhen Lei, Hailin Shi, Fan Yang, Dong Yi, Guo-Jun Qi, Stan Z. Li
Date: 2018-06-08 00:00:00
Tasks: Face Recognition, General Classification

In real-world face recognition applications, there is a tremendous amount of
data with two images for each person. One is an ID photo for face enrollment,
and the other is a probe photo captured on spot. Most existing methods are
designed for training data with limited breadth (a relatively small number of
classes) and sufficient depth (many samples for each class). They would meet
great challenges on ID versus Spot (IvS) data, including the under-represented
intra-class variations and an excessive demand on computing devices. In this
paper, we propose a deep learning based large-scale bisample learning (LBL)
method for IvS face recognition. To tackle the bisample problem with only two
samples for each class, a classification-verification-classification (CVC)
training strategy is proposed to progressively enhance the IvS performance.
Besides, a dominant prototype softmax (DP-softmax) is incorporated to make the
deep learning scalable on large-scale classes. We conduct LBL on a IvS face
dataset with more than two million identities. Experimental results show the
proposed method achieves superior performance to previous ones, validating the
effectiveness of LBL on IvS face recognition.
Multi-Task Deep Networks for Depth-Based 6D Object Pose and Joint Registration in Crowd Scenarios
Authors: Juil Sock, Kwang In Kim, Caner Sahin, Tae-Kyun Kim
Date: 2018-06-11 00:00:00
Tasks: 3D Pose Estimation, Multi-Task Learning, Object, Pose Estimation

In bin-picking scenarios, multiple instances of an object of interest are
stacked in a pile randomly, and hence, the instances are inherently subjected
to the challenges: severe occlusion, clutter, and similar-looking distractors.
Most existing methods are, however, for single isolated object instances, while
some recent methods tackle crowd scenarios as post-refinement which accounts
multiple object relations. In this paper, we address recovering 6D poses of
multiple instances in bin-picking scenarios in depth modality by multi-task
learning in deep neural networks. Our architecture jointly learns multiple
sub-tasks: 2D detection, depth, and 3D pose estimation of individual objects;
and joint registration of multiple objects. For training data generation, depth
images of physically plausible object pose configurations are generated by a 3D
object model in a physics simulation, which yields diverse occlusion patterns
to learn. We adopt a state-of-the-art object detector, and 2D offsets are
further estimated via a network to refine misaligned 2D detections. The depth
and 3D pose estimator is designed to generate multiple hypotheses per
detection. This allows the joint registration network to learn occlusion
patterns and remove physically implausible pose hypotheses. We apply our
architecture on both synthetic (our own and Sileane dataset) and real (a public
Bin-Picking dataset) data, showing that it significantly outperforms
state-of-the-art methods by 15-31% in average precision.
First Experiments with Neural Translation of Informal to Formal Mathematics
Authors: Qingxiang Wang, Cezary Kaliszyk, Josef Urban
Date: 2018-05-10 00:00:00
Tasks: Machine Translation, NMT, Translation

We report on our experiments to train deep neural networks that automatically
translate informalized LaTeX-written Mizar texts into the formal Mizar
language. To the best of our knowledge, this is the first time when neural
networks have been adopted in the formalization of mathematics. Using Luong et
al.'s neural machine translation model (NMT), we tested our aligned
informal-formal corpora against various hyperparameters and evaluated their
results. Our experiments show that our best performing model configurations are
able to generate correct Mizar statements on 65.73\% of the inference data,
with the union of all models covering 79.17\%. These results indicate that
formalization through artificial neural network is a promising approach for
automated formalization of mathematics. We present several case studies to
illustrate our results.
Object Detection using Domain Randomization and Generative Adversarial Refinement of Synthetic Images
Authors: Fernando Camaro Nogues, Andrew Huie, Sakyasingha Dasgupta
Date: 2018-05-30 00:00:00
Tasks: object-detection, Object Detection, Translation

In this work, we present an application of domain randomization and
generative adversarial networks (GAN) to train a near real-time object detector
for industrial electric parts, entirely in a simulated environment. Large scale
availability of labelled real world data is typically rare and difficult to
obtain in many industrial settings. As such here, only a few hundred of
unlabelled real images are used to train a Cyclic-GAN network, in combination
with various degree of domain randomization procedures. We demonstrate that
this enables robust translation of synthetic images to the real world domain.
We show that a combination of the original synthetic (simulation) and GAN
translated images, when used for training a Mask-RCNN object detection network
achieves greater than 0.95 mean average precision in detecting and classifying
a collection of industrial electric parts. We evaluate the performance across
different combinations of training data.
Automatic Target Recovery for Hindi-English Code Mixed Puns
Authors: Srishti Aggarwal, Kritik Mathur, Radhika Mamidi
Date: 2018-06-11 00:00:00

In order for our computer systems to be more human-like, with a higher
emotional quotient, they need to be able to process and understand intrinsic
human language phenomena like humour. In this paper, we consider a subtype of
humour - puns, which are a common type of wordplay-based jokes. In particular,
we consider code-mixed puns which have become increasingly mainstream on social
media, in informal conversations and advertisements and aim to build a system
which can automatically identify the pun location and recover the target of
such puns. We first study and classify code-mixed puns into two categories
namely intra-sentential and intra-word, and then propose a four-step algorithm
to recover the pun targets for puns belonging to the intra-sentential category.
Our algorithm uses language models, and phonetic similarity-based features to
get the desired results. We test our approach on a small set of code-mixed
punning advertisements, and observe that our system is successfully able to
recover the targets for 67% of the puns.