pwc-archive/papers-with-abstracts huggingface dataset:
A Hierarchical Neural Network for Sequence-to-Sequences Learning
Authors: Si Zuo, Zhimin Xu
Date: 2018-11-23 00:00:00
Tasks: Machine Translation, NMT, Translation

In recent years, the sequence-to-sequence learning neural networks with
attention mechanism have achieved great progress. However, there are still
challenges, especially for Neural Machine Translation (NMT), such as lower
translation quality on long sentences. In this paper, we present a hierarchical
deep neural network architecture to improve the quality of long sentences
translation. The proposed network embeds sequence-to-sequence neural networks
into a two-level category hierarchy by following the coarse-to-fine paradigm.
Long sentences are input by splitting them into shorter sequences, which can be
well processed by the coarse category network as the long distance dependencies
for short sentences is able to be handled by network based on
sequence-to-sequence neural network. Then they are concatenated and corrected
by the fine category network. The experiments shows that our method can achieve
superior results with higher BLEU(Bilingual Evaluation Understudy) scores,
lower perplexity and better performance in imitating expression style and words
usage than the traditional networks.
Kernel-Based Training of Generative Networks
Authors: Kalliopi Basioti, George V. Moustakides, Emmanouil Z. Psarakis
Date: 2018-11-23 00:00:00

Generative adversarial networks (GANs) are designed with the help of min-max
optimization problems that are solved with stochastic gradient-type algorithms
which are known to be non-robust. In this work we revisit a non-adversarial
method based on kernels which relies on a pure minimization problem and propose
a simple stochastic gradient algorithm for the computation of its solution.
Using simplified tools from Stochastic Approximation theory we demonstrate that
batch versions of the algorithm or smoothing of the gradient do not improve
convergence. These observations allow for the development of a training
algorithm that enjoys reduced computational complexity and increased robustness
while exhibiting similar synthesis characteristics as classical GANs.
How does Lipschitz Regularization Influence GAN Training?
Authors: Yipeng Qin, Niloy Mitra, Peter Wonka
Date: 2018-11-23 00:00:00
Proceeding: ECCV 2020 8

Despite the success of Lipschitz regularization in stabilizing GAN training, the exact reason of its effectiveness remains poorly understood. The direct effect of $K$-Lipschitz regularization is to restrict the $L2$-norm of the neural network gradient to be smaller than a threshold $K$ (e.g., $K=1$) such that $\|\nabla f\| \leq K$. In this work, we uncover an even more important effect of Lipschitz regularization by examining its impact on the loss function: It degenerates GAN loss functions to almost linear ones by restricting their domain and interval of attainable gradient values. Our analysis shows that loss functions are only successful if they are degenerated to almost linear ones. We also show that loss functions perform poorly if they are not degenerated and that a wide range of functions can be used as loss function as long as they are sufficiently degenerated by regularization. Basically, Lipschitz regularization ensures that all loss functions effectively work in the same way. Empirically, we verify our proposition on the MNIST, CIFAR10 and CelebA datasets.
Contributions to Biclustering of Microarray Data Using Formal Concept Analysis
Authors: Amina Houari
Date: 2018-11-23 00:00:00

Biclustering is an unsupervised data mining technique that aims to unveil
patterns (biclusters) from gene expression data matrices. In the framework of
this thesis, we propose new biclustering algorithms for microarray data. The
latter is done using data mining techniques. The objective is to identify
positively and negatively correlated biclusters.
  This thesis is divided into two part: In the first part, we present an
overview of the pattern-mining techniques and the biclustering of microarray
data. In the second part, we present our proposed biclustering algorithms where
we rely on two axes. In the first axis, we initially focus on extracting
biclusters of positive correlations. For this, we use both Formal Concept
Analysis and Association Rules. In the second axis, we focus on the extraction
of negatively correlated biclusters.
  The performed experimental studies highlight the very promising results
offered by the proposed algorithms. Our biclustering algorithms are evaluated
and compared statistically and biologically.
An Adaptive Approach for Automated Grapevine Phenotyping using VGG-based Convolutional Neural Networks
Authors: Jonatan Grimm, Katja Herzog, Florian Rist, Anna Kicherer, Reinhard Töpfer, Volker Steinhage
Date: 2018-11-23 00:00:00
Tasks: Object, object-detection, Object Detection

In (grapevine) breeding programs and research, periodic phenotyping and
multi-year monitoring of different grapevine traits, like growth or yield, is
needed especially in the field. This demand imply objective, precise and
automated methods using sensors and adaptive software. This work presents a
proof-of-concept analyzing RGB images of different growth stages of grapevines
with the aim to detect and quantify promising plant organs which are related to
yield. The input images are segmented by a Fully Convolutional Neural Network
(FCN) into object and background pixels. The objects are plant organs like
young shoots, pedicels, flower buds or grapes, which are principally suitable
for yield estimation. In the ground truth of the training images, each object
is separately annotated as a connected segment of object pixels, which enables
end-to-end learning of the object features. Based on the CNN-based
segmentation, the number of objects is determined by detecting and counting
connected components of object pixels using region labeling. In an evaluation
on six different data sets, the system achieves an IoU of up to 87.3% for the
segmentation and an F1 score of up to 88.6% for the object detection.
Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior
Authors: Zi Wang, Beomjoon Kim, Leslie Pack Kaelbling
Date: 2018-11-23 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Bayesian Optimization, Motion Planning, Task and Motion Planning

Bayesian optimization usually assumes that a Bayesian prior is given.
However, the strong theoretical guarantees in Bayesian optimization are often
regrettably compromised in practice because of unknown parameters in the prior.
In this paper, we adopt a variant of empirical Bayes and show that, by
estimating the Gaussian process prior from offline data sampled from the same
prior and constructing unbiased estimators of the posterior, variants of both
GP-UCB and probability of improvement achieve a near-zero regret bound, which
decreases to a constant proportional to the observational noise as the number
of offline data and the number of online evaluations increase. Empirically, we
have verified our approach on challenging simulated robotic problems featuring
task and motion planning.
Hyperdimensional Computing Nanosystem
Authors: Abbas Rahimi, Tony F. Wu, Haitong Li, Jan M. Rabaey, H. -S. Philip Wong, Max M. Shulaker, Subhasish Mitra
Date: 2018-11-23 00:00:00

One viable solution for continuous reduction in energy-per-operation is to
rethink functionality to cope with uncertainty by adopting computational
approaches that are inherently robust to uncertainty. It requires a novel look
at data representations, associated operations, and circuits, and at materials
and substrates that enable them. 3D integrated nanotechnologies combined with
novel brain-inspired computational paradigms that support fast learning and
fault tolerance could lead the way. Recognizing the very size of the brain's
circuits, hyperdimensional (HD) computing can model neural activity patterns
with points in a HD space, that is, with hypervectors as large randomly
generated patterns. At its very core, HD computing is about manipulating and
comparing these patterns inside memory. Emerging nanotechnologies such as
carbon nanotube field effect transistors (CNFETs) and resistive RAM (RRAM), and
their monolithic 3D integration offer opportunities for hardware
implementations of HD computing through tight integration of logic and memory,
energy-efficient computation, and unique device characteristics. We
experimentally demonstrate and characterize an end-to-end HD computing
nanosystem built using monolithic 3D integration of CNFETs and RRAM. With our
nanosystem, we experimentally demonstrate classification of 21 languages with
measured accuracy of up to 98% on >20,000 sentences (6.4 million characters),
training using one text sample (~100,000 characters) per language, and
resilient operation (98% accuracy) despite 78% hardware errors in HD
representation (outputs stuck at 0 or 1). By exploiting the unique properties
of the underlying nanotechnologies, we show that HD computing, when implemented
with monolithic 3D integration, can be up to 420X more energy-efficient while
using 25X less area compared to traditional silicon CMOS implementations.
Learning Attractor Dynamics for Generative Memory
Authors: Yan Wu, Greg Wayne, Karol Gregor, Timothy Lillicrap
Date: 2018-11-23 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Retrieval

A central challenge faced by memory systems is the robust retrieval of a
stored pattern in the presence of interference due to other stored patterns and
noise. A theoretically well-founded solution to robust retrieval is given by
attractor dynamics, which iteratively clean up patterns during recall. However,
incorporating attractor dynamics into modern deep learning systems poses
difficulties: attractor basins are characterised by vanishing gradients, which
are known to make training neural networks difficult. In this work, we avoid
the vanishing gradient problem by training a generative distributed memory
without simulating the attractor dynamics. Based on the idea of memory writing
as inference, as proposed in the Kanerva Machine, we show that a
likelihood-based Lyapunov function emerges from maximising the variational
lower-bound of a generative memory. Experiments shows it converges to correct
patterns upon iterative retrieval and achieves competitive performance as both
a memory model and a generative model.
High Dimensional Classification through $\ell_0$-Penalized Empirical Risk Minimization
Authors: Le-Yu Chen, Sokbae Lee
Date: 2018-11-23 00:00:00
Tasks: Binary Classification, Classification, General Classification, Vocal Bursts Intensity Prediction

We consider a high dimensional binary classification problem and construct a
classification procedure by minimizing the empirical misclassification risk
with a penalty on the number of selected features. We derive non-asymptotic
probability bounds on the estimated sparsity as well as on the excess
misclassification risk. In particular, we show that our method yields a sparse
solution whose l0-norm can be arbitrarily close to true sparsity with high
probability and obtain the rates of convergence for the excess
misclassification risk. The proposed procedure is implemented via the method of
mixed integer linear programming. Its numerical performance is illustrated in
Monte Carlo experiments.
Competency Questions and SPARQL-OWL Queries Dataset and Analysis
Authors: Dawid Wisniewski, Jedrzej Potoniec, Agnieszka Lawrynowicz, C. Maria Keet
Date: 2018-11-23 00:00:00
Tasks: Management

Competency Questions (CQs) are natural language questions outlining and
constraining the scope of knowledge represented by an ontology. Despite that
CQs are a part of several ontology engineering methodologies, we have observed
that the actual publication of CQs for the available ontologies is very limited
and even scarcer is the publication of their respective formalisations in terms
of, e.g., SPARQL queries. This paper aims to contribute to addressing the
engineering shortcomings of using CQs in ontology development, to facilitate
wider use of CQs. In order to understand the relation between CQs and the
queries over the ontology to test the CQs on an ontology, we gather, analyse,
and publicly release a set of 234 CQs and their translations to SPARQL-OWL for
several ontologies in different domains developed by different groups. We
analysed the CQs in two principal ways. The first stage focused on a linguistic
analysis of the natural language text itself, i.e., a lexico-syntactic analysis
without any presuppositions of ontology elements, and a subsequent step of
semantic analysis in order to find patterns. This increased diversity of CQ
sources resulted in a 5-fold increase of hitherto published patterns, to 106
distinct CQ patterns, which have a limited subset of few patterns shared across
the CQ sets from the different ontologies. Next, we analysed the relation
between the found CQ patterns and the 46 SPARQL-OWL query signatures, which
revealed that one CQ pattern may be realised by more than one SPARQL-OWL query
signature, and vice versa. We hope that our work will contribute to
establishing common practices, templates, automation, and user tools that will
support CQ formulation, formalisation, execution, and general management.
Complementary Segmentation of Primary Video Objects with Reversible Flows
Authors: Jia Li, Junjie Wu, Anlin Zheng, Yafei Song, Yu Zhang, Xiaowu Chen
Date: 2018-11-23 00:00:00
Tasks: Superpixels, Video Semantic Segmentation

Segmenting primary objects in a video is an important yet challenging problem
in computer vision, as it exhibits various levels of foreground/background
ambiguities. To reduce such ambiguities, we propose a novel formulation via
exploiting foreground and background context as well as their complementary
constraint. Under this formulation, a unified objective function is further
defined to encode each cue. For implementation, we design a Complementary
Segmentation Network (CSNet) with two separate branches, which can
simultaneously encode the foreground and background information along with
joint spatial constraints. The CSNet is trained on massive images with manually
annotated salient objects in an end-to-end manner. By applying CSNet on each
video frame, the spatial foreground and background maps can be initialized. To
enforce temporal consistency effectively and efficiently, we divide each frame
into superpixels and construct neighborhood reversible flow that reflects the
most reliable temporal correspondences between superpixels in far-away frames.
With such flow, the initialized foregroundness and backgroundness can be
propagated along the temporal dimension so that primary video objects gradually
pop-out and distractors are well suppressed. Extensive experimental results on
three video datasets show that the proposed approach achieves impressive
performance in comparisons with 18 state-of-the-art models.
MURAUER: Mapping Unlabeled Real Data for Label AUstERity
Authors: Georg Poier, Michael Opitz, David Schinagl, Horst Bischof
Date: 2018-11-23 00:00:00
Tasks: 3D Hand Pose Estimation, Hand Pose Estimation, Pose Estimation

Data labeling for learning 3D hand pose estimation models is a huge effort.
Readily available, accurately labeled synthetic data has the potential to
reduce the effort. However, to successfully exploit synthetic data, current
state-of-the-art methods still require a large amount of labeled real data. In
this work, we remove this requirement by learning to map from the features of
real data to the features of synthetic data mainly using a large amount of
synthetic and unlabeled real data. We exploit unlabeled data using two
auxiliary objectives, which enforce that (i) the mapped representation is pose
specific and (ii) at the same time, the distributions of real and synthetic
data are aligned. While pose specifity is enforced by a self-supervisory signal
requiring that the representation is predictive for the appearance from
different views, distributions are aligned by an adversarial term. In this way,
we can significantly improve the results of the baseline system, which does not
use unlabeled data and outperform many recent approaches already with about 1%
of the labeled real data. This presents a step towards faster deployment of
learning based hand pose estimation, making it accessible for a larger range of
applications.
The Error is the Feature: how to Forecast Lightning using a Model Prediction Error
Authors: Christian Schön, Jens Dittrich, Richard Müller
Date: 2018-11-23 00:00:00
Tasks: BIG-bench Machine Learning, Optical Flow Estimation, Weather Forecasting

Despite the progress within the last decades, weather forecasting is still a
challenging and computationally expensive task. Current satellite-based
approaches to predict thunderstorms are usually based on the analysis of the
observed brightness temperatures in different spectral channels and emit a
warning if a critical threshold is reached. Recent progress in data science
however demonstrates that machine learning can be successfully applied to many
research fields in science, especially in areas dealing with large datasets. We
therefore present a new approach to the problem of predicting thunderstorms
based on machine learning. The core idea of our work is to use the error of
two-dimensional optical flow algorithms applied to images of meteorological
satellites as a feature for machine learning models. We interpret that optical
flow error as an indication of convection potentially leading to thunderstorms
and lightning. To factor in spatial proximity we use various manual convolution
steps. We also consider effects such as the time of day or the geographic
location. We train different tree classifier models as well as a neural network
to predict lightning within the next few hours (called nowcasting in
meteorology) based on these features. In our evaluation section we compare the
predictive power of the different models and the impact of different features
on the classification result. Our results show a high accuracy of 96% for
predictions over the next 15 minutes which slightly decreases with increasing
forecast period but still remains above 83% for forecasts of up to five hours.
The high false positive rate of nearly 6% however needs further investigation
to allow for an operational use of our approach.
Differential Private Stack Generalization with an Application to Diabetes Prediction
Authors: Quanming Yao, Xiawei Guo, James T. Kwok, WeiWei Tu, Yuqiang Chen, Wenyuan Dai, Qiang Yang
Date: 2018-11-23 00:00:00
Tasks: Diabetes Prediction, Ensemble Learning, Feature Importance, Privacy Preserving, Transfer Learning

To meet the standard of differential privacy, noise is usually added into the original data, which inevitably deteriorates the predicting performance of subsequent learning algorithms. In this paper, motivated by the success of improving predicting performance by ensemble learning, we propose to enhance privacy-preserving logistic regression by stacking. We show that this can be done either by sample-based or feature-based partitioning. However, we prove that when privacy-budgets are the same, feature-based partitioning requires fewer samples than sample-based one, and thus likely has better empirical performance. As transfer learning is difficult to be integrated with a differential privacy guarantee, we further combine the proposed method with hypothesis transfer learning to address the problem of learning across different organizations. Finally, we not only demonstrate the effectiveness of our method on two benchmark data sets, i.e., MNIST and NEWS20, but also apply it into a real application of cross-organizational diabetes prediction from RUIJIN data set, where privacy is of significant concern.
LSD$_2$ -- Joint Denoising and Deblurring of Short and Long Exposure Images with CNNs
Authors: Janne Mustaniemi, Juho Kannala, Jiri Matas, Simo Särkkä, Janne Heikkilä
Date: 2018-11-23 00:00:00
Tasks: Deblurring, Denoising

The paper addresses the problem of acquiring high-quality photographs with handheld smartphone cameras in low-light imaging conditions. We propose an approach based on capturing pairs of short and long exposure images in rapid succession and fusing them into a single high-quality photograph. Unlike existing methods, we take advantage of both images simultaneously and perform a joint denoising and deblurring using a convolutional neural network. A novel approach is introduced to generate realistic short-long exposure image pairs. The method produces good images in extremely challenging conditions and outperforms existing denoising and deblurring methods. It also enables exposure fusion in the presence of motion blur.
Defect Detection from UAV Images based on Region-Based CNNs
Authors: Meng Lan, YiPeng Zhang, Lefei Zhang, Bo Du
Date: 2018-11-23 00:00:00
Tasks: Defect Detection, object-detection, Object Detection

With the wide applications of Unmanned Aerial Vehicle (UAV) in engineering
such as the inspection of the electrical equipment from distance, the demands
of efficient object detection algorithms for abundant images acquired by UAV
have also been significantly increased in recent years. In this work, we study
the performance of the region-based CNN for the electrical equipment defect
detection by using the UAV images. In order to train the detection model, we
collect a UAV images dataset composes of four classes of electrical equipment
defects with thousands of annotated labels. Then, based on the region-based
faster R-CNN model, we present a multi-class defects detection model for
electrical equipment which is more efficient and accurate than traditional
single class detection methods. Technically, we have replaced the RoI pooling
layer with a similar operation in Tensorflow and promoted the mini-batch to 128
per image in the training procedure. These improvements have slightly increased
the speed of detection without any accuracy loss. Therefore, the modified
region-based CNN could simultaneously detect multi-class of defects of the
electrical devices in nearly real time. Experimental results on the real word
electrical equipment images demonstrate that the proposed method achieves
better performance than the traditional object detection algorithms in defect
detection.
Parallel sequential Monte Carlo for stochastic gradient-free nonconvex optimization
Authors: Ömer Deniz Akyildiz, Dan Crisan, Joaquín Míguez
Date: 2018-11-23 00:00:00
Tasks: Stochastic Optimization

We introduce and analyze a parallel sequential Monte Carlo methodology for the numerical solution of optimization problems that involve the minimization of a cost function that consists of the sum of many individual components. The proposed scheme is a stochastic zeroth order optimization algorithm which demands only the capability to evaluate small subsets of components of the cost function. It can be depicted as a bank of samplers that generate particle approximations of several sequences of probability measures. These measures are constructed in such a way that they have associated probability density functions whose global maxima coincide with the global minima of the original cost function. The algorithm selects the best performing sampler and uses it to approximate a global minimum of the cost function. We prove analytically that the resulting estimator converges to a global minimum of the cost function almost surely and provide explicit convergence rates in terms of the number of generated Monte Carlo samples and the dimension of the search space. We show, by way of numerical examples, that the algorithm can tackle cost functions with multiple minima or with broad "flat" regions which are hard to minimize using gradient-based techniques.