pwc-archive/papers-with-abstracts huggingface dataset:
Robust Bregman Clustering
Authors: Aurélie Fischer, Clément Levrard, Claire Brécheteau
Date: 2018-12-11 00:00:00
Tasks: Clustering

Using a trimming approach, we investigate a k-means type method based on Bregman divergences for clustering data possibly corrupted with clutter noise. The main interest of Bregman divergences is that the standard Lloyd algorithm adapts to these distortion measures, and they are well-suited for clustering data sampled according to mixture models from exponential families. We prove that there exists an optimal codebook, and that an empirically optimal codebook converges a.s. to an optimal codebook in the distortion sense. Moreover, we obtain the sub-Gaussian rate of convergence for k-means 1 $\sqrt$ n under mild tail assumptions. Also, we derive a Lloyd-type algorithm with a trimming parameter that can be selected from data according to some heuristic, and present some experimental results.
Proximal Mean-field for Neural Network Quantization
Authors: Thalaiyasingam Ajanthan, Puneet K. Dokania, Richard Hartley, Philip H. S. Torr
Date: 2018-12-11 00:00:00
Proceeding: ICCV 2019 10
Tasks: Image Classification, Quantization

Compressing large Neural Networks (NN) by quantizing the parameters, while maintaining the performance is highly desirable due to reduced memory and time complexity. In this work, we cast NN quantization as a discrete labelling problem, and by examining relaxations, we design an efficient iterative optimization procedure that involves stochastic gradient descent followed by a projection. We prove that our simple projected gradient descent approach is, in fact, equivalent to a proximal version of the well-known mean-field method. These findings would allow the decades-old and theoretically grounded research on MRF optimization to be used to design better network quantization schemes. Our experiments on standard classification datasets (MNIST, CIFAR10/100, TinyImageNet) with convolutional and residual architectures show that our algorithm obtains fully-quantized networks with accuracies very close to the floating-point reference networks.
Layer-Parallel Training of Deep Residual Neural Networks
Authors: S. Günther, L. Ruthotto, J. B. Schroder, E. C. Cyr, N. R. Gauger
Date: 2018-12-11 00:00:00
Tasks: General Classification, image-classification, Image Classification

Residual neural networks (ResNets) are a promising class of deep neural networks that have shown excellent performance for a number of learning tasks, e.g., image classification and recognition. Mathematically, ResNet architectures can be interpreted as forward Euler discretizations of a nonlinear initial value problem whose time-dependent control variables represent the weights of the neural network. Hence, training a ResNet can be cast as an optimal control problem of the associated dynamical system. For similar time-dependent optimal control problems arising in engineering applications, parallel-in-time methods have shown notable improvements in scalability. This paper demonstrates the use of those techniques for efficient and effective training of ResNets. The proposed algorithms replace the classical (sequential) forward and backward propagation through the network layers by a parallel nonlinear multigrid iteration applied to the layer domain. This adds a new dimension of parallelism across layers that is attractive when training very deep networks. From this basic idea, we derive multiple layer-parallel methods. The most efficient version employs a simultaneous optimization approach where updates to the network parameters are based on inexact gradient information in order to speed up the training process. Using numerical examples from supervised classification, we demonstrate that the new approach achieves similar training performance to traditional methods, but enables layer-parallelism and thus provides speedup over layer-serial methods through greater concurrency.
Multichannel Semantic Segmentation with Unsupervised Domain Adaptation
Authors: Kohei Watanabe, Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada
Date: 2018-12-11 00:00:00
Tasks: Domain Adaptation, Segmentation, Semantic Segmentation, Unsupervised Domain Adaptation

Most contemporary robots have depth sensors, and research on semantic
segmentation with RGBD images has shown that depth images boost the accuracy of
segmentation. Since it is time-consuming to annotate images with semantic
labels per pixel, it would be ideal if we could avoid this laborious work by
utilizing an existing dataset or a synthetic dataset which we can generate on
our own. Robot motions are often tested in a synthetic environment, where
multichannel (eg, RGB + depth + instance boundary) images plus their
pixel-level semantic labels are available. However, models trained simply on
synthetic images tend to demonstrate poor performance on real images. In order
to address this, we propose two approaches that can efficiently exploit
multichannel inputs combined with an unsupervised domain adaptation (UDA)
algorithm. One is a fusion-based approach that uses depth images as inputs. The
other is a multitask learning approach that uses depth images as outputs. We
demonstrated that the segmentation results were improved by using a multitask
learning approach with a post-process and created a benchmark for this task.
Towards Automatic Personality Prediction Using Facebook Like Categories
Authors: Raad Bin Tareaf, Philipp Berger, Patrick Hennig, Christoph Meinel
Date: 2018-12-11 00:00:00
Tasks: Prediction

We demonstrate that effortlessly accessible digital records of behavior such
as Facebook Likes can be obtained and utilized to automatically distinguish a
wide range of highly delicate personal traits including: life satisfaction,
cultural ethnicity, political views, age, gender and personality traits. The
analysis presented based on a dataset of over 738,000 users who conferred their
Facebook Likes, social network activities, egocentric network, demographic
characteristics, and the results of various psychometric tests for our extended
personality analysis. The proposed model uses unique mapping technique between
each Facebook Like object to the corresponding Facebook page
category/sub-category object, which is then evaluated as features for a set of
machine learning algorithms to predict individual psycho-demographic profiles
from Likes. The model , distinguishes between a religious and non-religious
individual in 83% of circumstances, Asian and European in 87% of situations,
and between emotional stable and emotion unstable in 81% of situations. We
provide exemplars of correlations between attributes and Likes and present
suggestions for future directions.
Closing the U.S. gender wage gap requires understanding its heterogeneity
Authors: Philipp Bach, Victor Chernozhukov, Martin Spindler
Date: 2018-12-11 00:00:00
Tasks: regression

In 2016, the majority of full-time employed women in the U.S. earned significantly less than comparable men. The extent to which women were affected by gender inequality in earnings, however, depended greatly on socio-economic characteristics, such as marital status or educational attainment. In this paper, we analyzed data from the 2016 American Community Survey using a high-dimensional wage regression and applying double lasso to quantify heterogeneity in the gender wage gap. We found that the gap varied substantially across women and was driven primarily by marital status, having children at home, race, occupation, industry, and educational attainment. We recommend that policy makers use these insights to design policies that will reduce discrimination and unequal pay more effectively.
Learning latent representations for style control and transfer in end-to-end speech synthesis
Authors: Ya-Jie Zhang, Shifeng Pan, Lei He, Zhen-Hua Ling
Date: 2018-12-11 00:00:00
Tasks: Speech Synthesis, Style Transfer

In this paper, we introduce the Variational Autoencoder (VAE) to an
end-to-end speech synthesis model, to learn the latent representation of
speaking styles in an unsupervised manner. The style representation learned
through VAE shows good properties such as disentangling, scaling, and
combination, which makes it easy for style control. Style transfer can be
achieved in this framework by first inferring style representation through the
recognition network of VAE, then feeding it into TTS network to guide the style
in synthesizing speech. To avoid Kullback-Leibler (KL) divergence collapse in
training, several techniques are adopted. Finally, the proposed model shows
good performance of style control and outperforms Global Style Token (GST)
model in ABX preference tests on style transfer.
Convolutional Neural Networks with Transformed Input based on Robust Tensor Network Decomposition
Authors: Jenn-Bing Ong, Wee-Keong Ng, C. -C. Jay Kuo
Date: 2018-11-20 00:00:00
Tasks: Privacy Preserving, Tensor Networks

Tensor network decomposition, originated from quantum physics to model
entangled many-particle quantum systems, turns out to be a promising
mathematical technique to efficiently represent and process big data in
parsimonious manner. In this study, we show that tensor networks can
systematically partition structured data, e.g. color images, for distributed
storage and communication in privacy-preserving manner. Leveraging the sea of
big data and metadata privacy, empirical results show that neighbouring
subtensors with implicit information stored in tensor network formats cannot be
identified for data reconstruction. This technique complements the existing
encryption and randomization techniques which store explicit data
representation at one place and highly susceptible to adversarial attacks such
as side-channel attacks and de-anonymization. Furthermore, we propose a theory
for adversarial examples that mislead convolutional neural networks to
misclassification using subspace analysis based on singular value decomposition
(SVD). The theory is extended to analyze higher-order tensors using
tensor-train SVD (TT-SVD); it helps to explain the level of susceptibility of
different datasets to adversarial attacks, the structural similarity of
different adversarial attacks including global and localized attacks, and the
efficacy of different adversarial defenses based on input transformation. An
efficient and adaptive algorithm based on robust TT-SVD is then developed to
detect strong and static adversarial attacks.
Adversarial Autoencoders with Constant-Curvature Latent Manifolds
Authors: Daniele Grattarola, Lorenzo Livi, Cesare Alippi
Date: 2018-12-11 00:00:00
Tasks: Link Prediction

Constant-curvature Riemannian manifolds (CCMs) have been shown to be ideal
embedding spaces in many application domains, as their non-Euclidean geometry
can naturally account for some relevant properties of data, like hierarchy and
circularity. In this work, we introduce the CCM adversarial autoencoder
(CCM-AAE), a probabilistic generative model trained to represent a data
distribution on a CCM. Our method works by matching the aggregated posterior of
the CCM-AAE with a probability distribution defined on a CCM, so that the
encoder implicitly learns to represent data on the CCM to fool the
discriminator network. The geometric constraint is also explicitly imposed by
jointly training the CCM-AAE to maximise the membership degree of the
embeddings to the CCM. While a few works in recent literature make use of
either hyperspherical or hyperbolic manifolds for different learning tasks,
ours is the first unified framework to seamlessly deal with CCMs of different
curvatures. We show the effectiveness of our model on three different datasets
characterised by non-trivial geometry: semi-supervised classification on MNIST,
link prediction on two popular citation datasets, and graph-based molecule
generation using the QM9 chemical database. Results show that our method
improves upon other autoencoders based on Euclidean and non-Euclidean
geometries on all tasks taken into account.
Analytic heuristics for a fast DSC-MRI
Authors: Marco Virgulin, Marco Castellaro, Enrico Grisan, Fabio Marcuzzi
Date: 2018-12-11 00:00:00
Tasks: compressed sensing

In this paper we propose a deterministic approach for the reconstruction of
Dynamic Susceptibility Contrast magnetic resonance imaging data and compare it
with the compressed sensing solution existing in the literature for the same
problem. Our study is based on the mathematical analysis of the problem, which
is computationally intractable because of its non polynomial complexity, but
suggests simple heuristics that perform quite well. We give results on real
images and on artificial phantoms with added noise.
Deep RBFNet: Point Cloud Feature Learning using Radial Basis Functions
Authors: Weikai Chen, Xiaoguang Han, Guanbin Li, Chao Chen, Jun Xing, Yajie Zhao, Hao Li
Date: 2018-12-11 00:00:00
Tasks: 3D Object Recognition, Object Recognition

Three-dimensional object recognition has recently achieved great progress
thanks to the development of effective point cloud-based learning frameworks,
such as PointNet and its extensions. However, existing methods rely heavily on
fully connected layers, which introduce a significant amount of parameters,
making the network harder to train and prone to overfitting problems. In this
paper, we propose a simple yet effective framework for point set feature
learning by leveraging a nonlinear activation layer encoded by Radial Basis
Function (RBF) kernels. Unlike PointNet variants, that fail to recognize local
point patterns, our approach explicitly models the spatial distribution of
point clouds by aggregating features from sparsely distributed RBF kernels. A
typical RBF kernel, e.g. Gaussian function, naturally penalizes long-distance
response and is only activated by neighboring points. Such localized response
generates highly discriminative features given different point distributions.
In addition, our framework allows the joint optimization of kernel distribution
and its receptive field, automatically evolving kernel configurations in an
end-to-end manner. We demonstrate that the proposed network with a single RBF
layer can outperform the state-of-the-art Pointnet++ in terms of classification
accuracy for 3D object recognition tasks. Moreover, the introduction of
nonlinear mappings significantly reduces the number of network parameters and
computational cost, enabling significantly faster training and a deployable
point cloud recognition solution on portable devices with limited resources.
Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis
Authors: Côme Huré, Huyên Pham, Achref Bachouch, Nicolas Langrené
Date: 2018-12-11 00:00:00
Tasks: Deep Reinforcement Learning, Quantization, Reinforcement Learning

This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming. Unlike classical approximate dynamic programming approaches, we first approximate the optimal policy by means of neural networks in the spirit of deep reinforcement learning, and then the value function by Monte Carlo regression. This is achieved in the dynamic programming recursion by performance or hybrid iteration, and regress now methods from numerical probabilities. We provide a theoretical justification of these algorithms. Consistency and rate of convergence for the control and value function estimates are analyzed and expressed in terms of the universal approximation error of the neural networks, and of the statistical error when estimating network function, leaving aside the optimization error. Numerical results on various applications are presented in a companion paper (arxiv.org/abs/1812.05916) and illustrate the performance of the proposed algorithms.
Deep Density-based Image Clustering
Authors: Yazhou Ren, Ni Wang, Mingxia Li, Zenglin Xu
Date: 2018-12-11 00:00:00
Tasks: Clustering, Deep Clustering, Image Clustering

Recently, deep clustering, which is able to perform feature learning that
favors clustering tasks via deep neural networks, has achieved remarkable
performance in image clustering applications. However, the existing deep
clustering algorithms generally need the number of clusters in advance, which
is usually unknown in real-world tasks. In addition, the initial cluster
centers in the learned feature space are generated by $k$-means. This only
works well on spherical clusters and probably leads to unstable clustering
results. In this paper, we propose a two-stage deep density-based image
clustering (DDC) framework to address these issues. The first stage is to train
a deep convolutional autoencoder (CAE) to extract low-dimensional feature
representations from high-dimensional image data, and then apply t-SNE to
further reduce the data to a 2-dimensional space favoring density-based
clustering algorithms. The second stage is to apply the developed density-based
clustering technique on the 2-dimensional embedded data to automatically
recognize an appropriate number of clusters with arbitrary shapes. Concretely,
a number of local clusters are generated to capture the local structures of
clusters, and then are merged via their density relationship to form the final
clustering result. Experiments demonstrate that the proposed DDC achieves
comparable or even better clustering performance than state-of-the-art deep
clustering methods, even though the number of clusters is not given.
Deep Unfolding of a Proximal Interior Point Method for Image Restoration
Authors: Carla Bertocchi, Emilie Chouzenoux, Marie-Caroline Corbineau, Jean-Christophe Pesquet, Marco Prato
Date: 2018-12-11 00:00:00
Tasks: Deblurring, Image Deblurring, Image Restoration

Variational methods are widely applied to ill-posed inverse problems for they have the ability to embed prior knowledge about the solution. However, the level of performance of these methods significantly depends on a set of parameters, which can be estimated through computationally expensive and time-consuming methods. In contrast, deep learning offers very generic and efficient architectures, at the expense of explainability, since it is often used as a black-box, without any fine control over its output. Deep unfolding provides a convenient approach to combine variational-based and deep learning approaches. Starting from a variational formulation for image restoration, we develop iRestNet, a neural network architecture obtained by unfolding a proximal interior point algorithm. Hard constraints, encoding desirable properties for the restored image, are incorporated into the network thanks to a logarithmic barrier, while the barrier parameter, the stepsize, and the penalization weight are learned by the network. We derive explicit expressions for the gradient of the proximity operator for various choices of constraints, which allows training iRestNet with gradient descent and backpropagation. In addition, we provide theoretical results regarding the stability of the network for a common inverse problem example. Numerical experiments on image deblurring problems show that the proposed approach compares favorably with both state-of-the-art variational and machine learning methods in terms of image quality.
Solving Pictorial Jigsaw Puzzle by Stigmergy-inspired Internet-based Human Collective Intelligence
Authors: Bo Shen, Wei zhang, Haiyan Zhao, Zhi Jin, Yanhong Wu
Date: 2018-11-28 00:00:00

The pictorial jigsaw (PJ) puzzle is a well-known leisure game for humans.
Usually, a PJ puzzle game is played by one or several human players
face-to-face in the physical space. In this paper, we focus on how to solve PJ
puzzles in the cyberspace by a group of physically distributed human players.
We propose an approach to solving PJ puzzle by stigmergy-inspired
Internet-based human collective intelligence. The core of the approach is a
continuously executing loop, named the EIF loop, which consists of three
activities: exploration, integration, and feedback. In exploration, each player
tries to solve the PJ puzzle alone, without direct interactions with other
players. At any time, the result of a player's exploration is a partial
solution to the PJ puzzle, and a set of rejected neighboring relation between
pieces. The results of all players' exploration are integrated in real time
through integration, with the output of a continuously updated collective
opinion graph (COG). And through feedback, each player is provided with
personalized feedback information based on the current COG and the player's
exploration result, in order to accelerate his/her puzzle-solving process.
Exploratory experiments show that: (1) supported by this approach, the time to
solve PJ puzzle is nearly linear to the reciprocal of the number of players,
and shows better scalability to puzzle size than that of face-to-face
collaboration for 10-player groups; (2) for groups with 2 to 10 players, the
puzzle-solving time decreases 31.36%-64.57% on average, compared with the best
single players in the experiments.
Domain-Aware SE Network for Sketch-based Image Retrieval with Multiplicative Euclidean Margin Softmax
Authors: Peng Lu, Gao Huang, Hangyu Lin, Wenming Yang, Guodong Guo, Yanwei Fu
Date: 2018-12-11 00:00:00
Tasks: Image Retrieval, Retrieval, Sketch-Based Image Retrieval

This paper proposes a novel approach for Sketch-Based Image Retrieval (SBIR), for which the key is to bridge the gap between sketches and photos in terms of the data representation. Inspired by channel-wise attention explored in recent years, we present a Domain-Aware Squeeze-and-Excitation (DASE) network, which seamlessly incorporates the prior knowledge of sample sketch or photo into SE module and make the SE module capable of emphasizing appropriate channels according to domain signal. Accordingly, the proposed network can switch its mode to achieve a better domain feature with lower intra-class discrepancy. Moreover, while previous works simply focus on minimizing intra-class distance and maximizing inter-class distance, we introduce a loss function, named Multiplicative Euclidean Margin Softmax (MEMS), which introduces multiplicative Euclidean margin into feature space and ensure that the maximum intra-class distance is smaller than the minimum inter-class distance. This facilitates learning a highly discriminative feature space and ensures a more accurate image retrieval result. Extensive experiments are conducted on two widely used SBIR benchmark datasets. Our approach achieves better results on both datasets, surpassing the state-of-the-art methods by a large margin.
Classification-Reconstruction Learning for Open-Set Recognition
Authors: Ryota Yoshihashi, Wen Shao, Rei Kawakami, ShaoDi You, Makoto Iida, Takeshi Naemura
Date: 2018-12-11 00:00:00
Proceeding: CVPR 2019 6
Tasks: Classification, General Classification, open-set classification, Open Set Learning

Open-set classification is a problem of handling `unknown' classes that are not contained in the training dataset, whereas traditional classifiers assume that only known classes appear in the test environment. Existing open-set classifiers rely on deep networks trained in a supervised manner on known classes in the training set; this causes specialization of learned representations to known classes and makes it hard to distinguish unknowns from knowns. In contrast, we train networks for joint classification and reconstruction of input data. This enhances the learned representation so as to preserve information useful for separating unknowns from knowns, as well as to discriminate classes of knowns. Our novel Classification-Reconstruction learning for Open-Set Recognition (CROSR) utilizes latent representations for reconstruction and enables robust unknown detection without harming the known-class classification accuracy. Extensive experiments reveal that the proposed method outperforms existing deep open-set classifiers in multiple standard datasets and is robust to diverse outliers. The code is available in https://nae-lab.org/~rei/research/crosr/.