pwc-archive/papers-with-abstracts huggingface dataset:
Improved asynchronous parallel optimization analysis for stochastic incremental methods
Authors: Rémi Leblond, Fabian Pedregosa, Simon Lacoste-Julien
Date: 2018-01-11 00:00:00

As datasets continue to increase in size and multi-core computer
architectures are developed, asynchronous parallel optimization algorithms
become more and more essential to the field of Machine Learning. Unfortunately,
conducting the theoretical analysis asynchronous methods is difficult, notably
due to the introduction of delay and inconsistency in inherently sequential
algorithms. Handling these issues often requires resorting to simplifying but
unrealistic assumptions. Through a novel perspective, we revisit and clarify a
subtle but important technical issue present in a large fraction of the recent
convergence rate proofs for asynchronous parallel optimization algorithms, and
propose a simplification of the recently introduced "perturbed iterate"
framework that resolves it. We demonstrate the usefulness of our new framework
by analyzing three distinct asynchronous parallel incremental optimization
algorithms: Hogwild (asynchronous SGD), KROMAGNON (asynchronous SVRG) and
ASAGA, a novel asynchronous parallel version of the incremental gradient
algorithm SAGA that enjoys fast linear convergence rates. We are able to both
remove problematic assumptions and obtain better theoretical results. Notably,
we prove that ASAGA and KROMAGNON can obtain a theoretical linear speedup on
multi-core systems even without sparsity assumptions. We present results of an
implementation on a 40-core architecture illustrating the practical speedups as
well as the hardware overhead. Finally, we investigate the overlap constant, an
ill-understood but central quantity for the theoretical analysis of
asynchronous parallel algorithms. We find that it encompasses much more
complexity than suggested in previous work, and often is order-of-magnitude
bigger than traditionally thought.
Statistical learning for wind power : a modeling and stability study towards forecasting
Authors: Aurélie Fischer, Lucie Montuelle, Mathilde Mougeot, Dominique Picard
Date: 2016-10-04 00:00:00

We focus on wind power modeling using machine learning techniques. We show on
real data provided by the wind energy company Ma{\"i}a Eolis, that parametric
models, even following closely the physical equation relating wind production
to wind speed are outperformed by intelligent learning algorithms. In
particular, the CART-Bagging algorithm gives very stable and promising results.
Besides, as a step towards forecast, we quantify the impact of using
deteriorated wind measures on the performances. We show also on this
application that the default methodology to select a subset of predictors
provided in the standard random forest package can be refined, especially when
there exists among the predictors one variable which has a major impact.
Generative Single Image Reflection Separation
Authors: Donghoon Lee, Ming-Hsuan Yang, Songhwai Oh
Date: 2018-01-12 00:00:00

Single image reflection separation is an ill-posed problem since two scenes,
a transmitted scene and a reflected scene, need to be inferred from a single
observation. To make the problem tractable, in this work we assume that
categories of two scenes are known. It allows us to address the problem by
generating both scenes that belong to the categories while their contents are
constrained to match with the observed image. A novel network architecture is
proposed to render realistic images of both scenes based on adversarial
learning. The network can be trained in a weakly supervised manner, i.e., it
learns to separate an observed image without corresponding ground truth images
of transmission and reflection scenes which are difficult to collect in
practice. Experimental results on real and synthetic datasets demonstrate that
the proposed algorithm performs favorably against existing methods.
Trust-Aware Decision Making for Human-Robot Collaboration: Model Learning and Planning
Authors: Min Chen, Stefanos Nikolaidis, Harold Soh, David Hsu, Siddhartha Srinivasa
Date: 2018-01-12 00:00:00
Tasks: Decision Making

Trust in autonomy is essential for effective human-robot collaboration and
user adoption of autonomous systems such as robot assistants. This paper
introduces a computational model which integrates trust into robot
decision-making. Specifically, we learn from data a partially observable Markov
decision process (POMDP) with human trust as a latent variable. The trust-POMDP
model provides a principled approach for the robot to (i) infer the trust of a
human teammate through interaction, (ii) reason about the effect of its own
actions on human trust, and (iii) choose actions that maximize team performance
over the long term. We validated the model through human subject experiments on
a table-clearing task in simulation (201 participants) and with a real robot
(20 participants). In our studies, the robot builds human trust by manipulating
low-risk objects first. Interestingly, the robot sometimes fails intentionally
in order to modulate human trust and achieve the best team performance. These
results show that the trust-POMDP calibrates trust to improve human-robot team
performance over the long term. Further, they highlight that maximizing trust
alone does not always lead to the best performance.
Hierarchical Motion Consistency Constraint for Efficient Geometrical Verification in UAV Image Matching
Authors: San Jiang, Wanshou Jiang
Date: 2018-01-12 00:00:00

This paper proposes a strategy for efficient geometrical verification in
unmanned aerial vehicle (UAV) image matching. First, considering the complex
transformation model between correspondence set in the image-space, feature
points of initial candidate matches are projected onto an elevation plane in
the object-space, with assistant of UAV flight control data and camera mounting
angles. Spatial relationships are simplified as a 2D-translation in which a
motion establishes the relation of two correspondence points. Second, a
hierarchical motion consistency constraint, termed HMCC, is designed to
eliminate outliers from initial candidate matches, which includes three major
steps, namely the global direction consistency constraint, the local
direction-change consistency constraint and the global length consistency
constraint. To cope with scenarios with high outlier ratios, the HMCC is
achieved by using a voting scheme. Finally, an efficient geometrical
verification strategy is proposed by using the HMCC as a pre-processing step to
increase inlier ratios before the consequent application of the basic RANSAC
algorithm. The performance of the proposed strategy is verified through
comprehensive comparison and analysis by using real UAV datasets captured with
different photogrammetric systems. Experimental results demonstrate that the
generated motions have noticeable separation ability, and the HMCC-RANSAC
algorithm can efficiently eliminate outliers based on the motion consistency
constraint, with a speedup ratio reaching to 6 for oblique UAV images. Even
though the completeness sacrifice of approximately 7 percent of points is
observed from image orientation tests, competitive orientation accuracy is
achieved from all used datasets. For geometrical verification of both nadir and
oblique UAV images, the proposed method can be a more efficient solution.
How should a fixed budget of dwell time be spent in scanning electron microscopy to optimize image quality?
Authors: Patrick Trampert, Faysal Bourghorbel, Pavel Potocek, Maurice Peemen, Christian Schlinkmann, Tim Dahmen, Philipp Slusallek
Date: 2018-01-12 00:00:00
Tasks: Denoising, Super-Resolution

In scanning electron microscopy, the achievable image quality is often
limited by a maximum feasible acquisition time per dataset. Particularly with
regard to three-dimensional or large field-of-view imaging, a compromise must
be found between a high amount of shot noise, which leads to a low
signal-to-noise ratio, and excessive acquisition times. Assuming a fixed
acquisition time per frame, we compared three different strategies for
algorithm-assisted image acquisition in scanning electron microscopy. We
evaluated (1) raster scanning with a reduced dwell time per pixel followed by a
state-of-the-art Denoising algorithm, (2) raster scanning with a decreased
resolution in conjunction with a state-of-the-art Super Resolution algorithm,
and (3) a sparse scanning approach where a fixed percentage of pixels is
visited by the beam in combination with state-of-the-art inpainting algorithms.
Additionally, we considered increased beam currents for each of the strategies.
The experiments showed that sparse scanning using an appropriate reconstruction
technique was superior to the other strategies.
Deep learning is a good steganalysis tool when embedding key is reused for different images, even if there is a cover source-mismatch
Authors: Lionel Pibre, Pasquet Jérôme, Dino Ienco, Marc Chaumont
Date: 2015-11-16 00:00:00
Tasks: Steganalysis

Since the BOSS competition, in 2010, most steganalysis approaches use a
learning methodology involving two steps: feature extraction, such as the Rich
Models (RM), for the image representation, and use of the Ensemble Classifier
(EC) for the learning step. In 2015, Qian et al. have shown that the use of a
deep learning approach that jointly learns and computes the features, is very
promising for the steganalysis. In this paper, we follow-up the study of Qian
et al., and show that, due to intrinsic joint minimization, the results
obtained from a Convolutional Neural Network (CNN) or a Fully Connected Neural
Network (FNN), if well parameterized, surpass the conventional use of a RM with
an EC. First, numerous experiments were conducted in order to find the best "
shape " of the CNN. Second, experiments were carried out in the clairvoyant
scenario in order to compare the CNN and FNN to an RM with an EC. The results
show more than 16% reduction in the classification error with our CNN or FNN.
Third, experiments were also performed in a cover-source mismatch setting. The
results show that the CNN and FNN are naturally robust to the mismatch problem.
In Addition to the experiments, we provide discussions on the internal
mechanisms of a CNN, and weave links with some previously stated ideas, in
order to understand the impressive results we obtained.
CT sinogram-consistency learning for metal-induced beam hardening correction
Authors: Hyung Suk Park, Sung Min Lee, Hwa Pyung Kim, Jin Keun Seo
Date: 2017-08-02 00:00:00
Tasks: CT Reconstruction

This paper proposes a sinogram consistency learning method to deal with
beam-hardening related artifacts in polychromatic computerized tomography (CT).
The presence of highly attenuating materials in the scan field causes an
inconsistent sinogram, that does not match the range space of the Radon
transform. When the mismatched data are entered into the range space during CT
reconstruction, streaking and shading artifacts are generated owing to the
inherent nature of the inverse Radon transform. The proposed learning method
aims to repair inconsistent sinograms by removing the primary metal-induced
beam-hardening factors along the metal trace in the sinogram. Taking account of
the fundamental difficulty in obtaining sufficient training data in a medical
environment, the learning method is designed to use simulated training data and
a patient-type specific learning model is used to simplify the learning
process. The feasibility of the proposed method is investigated using a
dataset, consisting of real CT scan of pelvises containing hip prostheses. The
anatomical areas in training and test data are different, in order to
demonstrate that the proposed method extracts the beam hardening features,
selectively. The results show that our method successfully corrects sinogram
inconsistency by extracting beam-hardening sources by means of deep learning.
This paper proposed a deep learning method of sinogram correction for beam
hardening reduction in CT for the first time. Conventional methods for beam
hardening reduction are based on regularizations, and have the fundamental
drawback of being not easily able to use manifold CT images, while a deep
learning approach has the potential to do so.
Deep Stereo Matching with Explicit Cost Aggregation Sub-Architecture
Authors: Lidong Yu, Yucheng Wang, Yuwei Wu, Yunde Jia
Date: 2018-01-12 00:00:00
Tasks: Stereo Matching, Stereo Matching Hand

Deep neural networks have shown excellent performance for stereo matching.
Many efforts focus on the feature extraction and similarity measurement of the
matching cost computation step while less attention is paid on cost aggregation
which is crucial for stereo matching. In this paper, we present a
learning-based cost aggregation method for stereo matching by a novel
sub-architecture in the end-to-end trainable pipeline. We reformulate the cost
aggregation as a learning process of the generation and selection of cost
aggregation proposals which indicate the possible cost aggregation results. The
cost aggregation sub-architecture is realized by a two-stream network: one for
the generation of cost aggregation proposals, the other for the selection of
the proposals. The criterion for the selection is determined by the low-level
structure information obtained from a light convolutional network. The
two-stream network offers a global view guidance for the cost aggregation to
rectify the mismatching value stemming from the limited view of the matching
cost computation. The comprehensive experiments on challenge datasets such as
KITTI and Scene Flow show that our method outperforms the state-of-the-art
methods.
Quantum transport senses community structure in networks
Authors: Chenchao Zhao, Jun S. Song
Date: 2017-11-14 00:00:00
Tasks: Clustering

Quantum time evolution exhibits rich physics, attributable to the interplay
between the density and phase of a wave function. However, unlike classical
heat diffusion, the wave nature of quantum mechanics has not yet been
extensively explored in modern data analysis. We propose that the Laplace
transform of quantum transport (QT) can be used to construct an ensemble of
maps from a given complex network to a circle $S^1$, such that closely-related
nodes on the network are grouped into sharply concentrated clusters on $S^1$.
The resulting QT clustering (QTC) algorithm is as powerful as the
state-of-the-art spectral clustering in discerning complex geometric patterns
and more robust when clusters show strong density variations or heterogeneity
in size. The observed phenomenon of QTC can be interpreted as a collective
behavior of the microscopic nodes that evolve as macroscopic cluster orbitals
in an effective tight-binding model recapitulating the network. Python source
code implementing the algorithm and examples are available at
https://github.com/jssong-lab/QTC.
Mechanisms of dimensionality reduction and decorrelation in deep neural networks
Authors: Haiping Huang
Date: 2017-10-04 00:00:00
Tasks: Dimensionality Reduction

Deep neural networks are widely used in various domains. However, the nature
of computations at each layer of the deep networks is far from being well
understood. Increasing the interpretability of deep neural networks is thus
important. Here, we construct a mean-field framework to understand how compact
representations are developed across layers, not only in deterministic deep
networks with random weights but also in generative deep networks where an
unsupervised learning is carried out. Our theory shows that the deep
computation implements a dimensionality reduction while maintaining a finite
level of weak correlations between neurons for possible feature extraction.
Mechanisms of dimensionality reduction and decorrelation are unified in the
same framework. This work may pave the way for understanding how a sensory
hierarchy works.
A3T: Adversarially Augmented Adversarial Training
Authors: Akram Erraqabi, Aristide Baratin, Yoshua Bengio, Simon Lacoste-Julien
Date: 2018-01-12 00:00:00
Tasks: Adversarial Robustness, BIG-bench Machine Learning, General Classification

Recent research showed that deep neural networks are highly sensitive to
so-called adversarial perturbations, which are tiny perturbations of the input
data purposely designed to fool a machine learning classifier. Most
classification models, including deep learning models, are highly vulnerable to
adversarial attacks. In this work, we investigate a procedure to improve
adversarial robustness of deep neural networks through enforcing representation
invariance. The idea is to train the classifier jointly with a discriminator
attached to one of its hidden layer and trained to filter the adversarial
noise. We perform preliminary experiments to test the viability of the approach
and to compare it to other standard adversarial training methods.
Noisy Expectation-Maximization: Applications and Generalizations
Authors: Osonde Osoba, Bart Kosko
Date: 2018-01-12 00:00:00

We present a noise-injected version of the Expectation-Maximization (EM)
algorithm: the Noisy Expectation Maximization (NEM) algorithm. The NEM
algorithm uses noise to speed up the convergence of the EM algorithm. The NEM
theorem shows that injected noise speeds up the average convergence of the EM
algorithm to a local maximum of the likelihood surface if a positivity
condition holds. The generalized form of the noisy expectation-maximization
(NEM) algorithm allow for arbitrary modes of noise injection including adding
and multiplying noise to the data.
  We demonstrate these noise benefits on EM algorithms for the Gaussian mixture
model (GMM) with both additive and multiplicative NEM noise injection. A
separate theorem (not presented here) shows that the noise benefit for
independent identically distributed additive noise decreases with sample size
in mixture models. This theorem implies that the noise benefit is most
pronounced if the data is sparse. Injecting blind noise only slowed
convergence.
3D Face Reconstruction from Light Field Images: A Model-free Approach
Authors: Mingtao Feng, Syed Zulqarnain Gilani, Yaonan Wang, Ajmal Mian
Date: 2017-11-16 00:00:00
Proceeding: ECCV 2018 9
Tasks: 3D Face Reconstruction, Face Reconstruction

Reconstructing 3D facial geometry from a single RGB image has recently
instigated wide research interest. However, it is still an ill-posed problem
and most methods rely on prior models hence undermining the accuracy of the
recovered 3D faces. In this paper, we exploit the Epipolar Plane Images (EPI)
obtained from light field cameras and learn CNN models that recover horizontal
and vertical 3D facial curves from the respective horizontal and vertical EPIs.
Our 3D face reconstruction network (FaceLFnet) comprises a densely connected
architecture to learn accurate 3D facial curves from low resolution EPIs. To
train the proposed FaceLFnets from scratch, we synthesize photo-realistic light
field images from 3D facial scans. The curve by curve 3D face estimation
approach allows the networks to learn from only 14K images of 80 identities,
which still comprises over 11 Million EPIs/curves. The estimated facial curves
are merged into a single pointcloud to which a surface is fitted to get the
final 3D face. Our method is model-free, requires only a few training samples
to learn FaceLFnet and can reconstruct 3D faces with high accuracy from single
light field images under varying poses, expressions and lighting conditions.
Comparison on the BU-3DFE and BU-4DFE datasets show that our method reduces
reconstruction errors by over 20% compared to recent state of the art.
Comparative Study on Generative Adversarial Networks
Authors: Saifuddin Hitawala
Date: 2018-01-12 00:00:00
Tasks: General Classification, image-classification, Image Classification

In recent years, there have been tremendous advancements in the field of
machine learning. These advancements have been made through both academic as
well as industrial research. Lately, a fair amount of research has been
dedicated to the usage of generative models in the field of computer vision and
image classification. These generative models have been popularized through a
new framework called Generative Adversarial Networks. Moreover, many modified
versions of this framework have been proposed in the last two years. We study
the original model proposed by Goodfellow et al. as well as modifications over
the original model and provide a comparative analysis of these models.
PrivyNet: A Flexible Framework for Privacy-Preserving Deep Neural Network Training
Authors: Meng Li, Liangzhen Lai, Naveen Suda, Vikas Chandra, David Z. Pan
Date: 2017-09-18 00:00:00
Proceeding: ICLR 2018 1
Tasks: General Classification, image-classification, Image Classification, Privacy Preserving

Massive data exist among user local platforms that usually cannot support
deep neural network (DNN) training due to computation and storage resource
constraints. Cloud-based training schemes provide beneficial services but
suffer from potential privacy risks due to excessive user data collection. To
enable cloud-based DNN training while protecting the data privacy
simultaneously, we propose to leverage the intermediate representations of the
data, which is achieved by splitting the DNNs and deploying them separately
onto local platforms and the cloud. The local neural network (NN) is used to
generate the feature representations. To avoid local training and protect data
privacy, the local NN is derived from pre-trained NNs. The cloud NN is then
trained based on the extracted intermediate representations for the target
learning task. We validate the idea of DNN splitting by characterizing the
dependency of privacy loss and classification accuracy on the local NN topology
for a convolutional NN (CNN) based image classification task. Based on the
characterization, we further propose PrivyNet to determine the local NN
topology, which optimizes the accuracy of the target learning task under the
constraints on privacy loss, local computation, and storage. The efficiency and
effectiveness of PrivyNet are demonstrated with the CIFAR-10 dataset.
Non-negative Matrix Factorization: Robust Extraction of Extended Structures
Authors: Bīn Rén, Laurent Pueyo, Guangtun Ben Zhu, John Debes, Gaspard Duchêne
Date: 2017-12-29 00:00:00

We apply the vectorized Non-negative Matrix Factorization (NMF) method to
post-processing of direct imaging data for exoplanetary systems such as
circumstellar disks. NMF is an iterative approach, which first creates a
non-orthogonal and non-negative basis of components using given reference
images, then models a target with the components. The constructed model is then
rescaled with a factor to compensate for the contribution from a disk. We
compare NMF with existing methods (classical reference differential imaging
method, and the Karhunen-Lo\`eve image projection algorithm) using synthetic
circumstellar disks, and demonstrate the superiority of NMF: with no need for
prior selection of references, NMF can detect fainter circumstellar disks,
better preserve low order disk morphology, and does not require forward
modeling. As an application to a well-known disk example, we process the
archival Hubble Space Telescope (HST) STIS coronagraphic observations of
HD~181327 with different methods and compare them. NMF is able to extract some
circumstellar material inside the primary ring for the first time. In the
appendix, we mathematically investigate the stability of NMF components during
iteration, and the linearity of NMF modeling.