pwc-archive/papers-with-abstracts huggingface dataset:
Advanced Super-Resolution using Lossless Pooling Convolutional Networks
Authors: Farzad Toutounchi, Ebroul Izquierdo
Date: 2018-12-14 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

In this paper, we present a novel deep learning-based approach for still
image super-resolution, that unlike the mainstream models does not rely solely
on the input low resolution image for high quality upsampling, and takes
advantage of a set of artificially created auxiliary self-replicas of the input
image that are incorporated in the neural network to create an enhanced and
accurate upscaling scheme. Inclusion of the proposed lossless pooling layers,
and the fusion of the input self-replicas enable the model to exploit the high
correlation between multiple instances of the same content, and eventually
result in significant improvements in the quality of the super-resolution,
which is confirmed by extensive evaluations.
More Effective Ontology Authoring with Test-Driven Development
Authors: C. Maria Keet, Kieren Davies, Agnieszka Lawrynowicz
Date: 2018-12-14 00:00:00
Tasks: test driven development

Ontology authoring is a complex process, where commonly the automated
reasoner is invoked for verification of newly introduced changes, therewith
amounting to a time-consuming test-last approach. Test-Driven Development (TDD)
for ontology authoring is a recent {\em test-first} approach that aims to
reduce authoring time and increase authoring efficiency. Current TDD testing
falls short on coverage of OWL features and possible test outcomes, the
rigorous foundation thereof, and evaluations to ascertain its effectiveness.
  We aim to address these issues in one instantiation of TDD for ontology
authoring. We first propose a succinct, logic-based model of TDD testing and
present novel TDD algorithms so as to cover also any OWL 2 class expression for
the TBox and for the principal ABox assertions, and prove their correctness.
The algorithms use methods from the OWL API directly such that reclassification
is not necessary for test execution, therewith reducing ontology authoring
time. The algorithms were implemented in TDDonto2, a Prot\'eg\'e plugin.
TDDonto2 was evaluated on editing efficiency and by users. The editing
efficiency study demonstrated that it is faster than a typical ontology
authoring interface, especially for medium size and large ontologies. The user
evaluation demonstrated that modellers make significantly less errors with
TDDonto2 compared to the standard Prot\'eg\'e interface and complete their
tasks better using less time. Thus, the results indicate that Test-Driven
Development is a promising approach in an ontology development methodology.
Nonparametric inference of interaction laws in systems of agents from trajectory data
Authors: Fei Lu, Mauro Maggioni, Sui Tang, Ming Zhong
Date: 2018-12-14 00:00:00

Inferring the laws of interaction between particles and agents in complex
dynamical systems from observational data is a fundamental challenge in a wide
variety of disciplines. We propose a non-parametric statistical learning
approach to estimate the governing laws of distance-based interactions, with no
reference or assumption about their analytical form, from data consisting
trajectories of interacting agents. We demonstrate the effectiveness of our
learning approach both by providing theoretical guarantees, and by testing the
approach on a variety of prototypical systems in various disciplines. These
systems include homogeneous and heterogeneous agents systems, ranging from
particle systems in fundamental physics to agent-based systems modeling opinion
dynamics under the social influence, prey-predator dynamics, flocking and
swarming, and phototaxis in cell dynamics.
Products of Many Large Random Matrices and Gradients in Deep Neural Networks
Authors: Boris Hanin, Mihai Nica
Date: 2018-12-14 00:00:00

We study products of random matrices in the regime where the number of terms
and the size of the matrices simultaneously tend to infinity. Our main theorem
is that the logarithm of the $\ell_2$ norm of such a product applied to any
fixed vector is asymptotically Gaussian. The fluctuations we find can be
thought of as a finite temperature correction to the limit in which first the
size and then the number of matrices tend to infinity. Depending on the scaling
limit considered, the mean and variance of the limiting Gaussian depend only on
either the first two or the first four moments of the measure from which matrix
entries are drawn. We also obtain explicit error bounds on the moments of the
norm and the Kolmogorov-Smirnov distance to a Gaussian. Finally, we apply our
result to obtain precise information about the stability of gradients in
randomly initialized deep neural networks with ReLU activations. This provides
a quantitative measure of the extent to which the exploding and vanishing
gradient problem occurs in a fully connected neural network with ReLU
activations and a given architecture.
Class Mean Vector Component and Discriminant Analysis
Authors: Alexandros Iosifidis
Date: 2018-12-14 00:00:00
Tasks: Dimensionality Reduction

The kernel matrix used in kernel methods encodes all the information required for solving complex nonlinear problems defined on data representations in the input space using simple, but implicitly defined, solutions. Spectral analysis on the kernel matrix defines an explicit nonlinear mapping of the input data representations to a subspace of the kernel space, which can be used for directly applying linear methods. However, the selection of the kernel subspace is crucial for the performance of the proceeding processing steps. In this paper, we propose a component analysis method for kernel-based dimensionality reduction that optimally preserves the pair-wise distances of the class means in the feature space. We provide extensive analysis on the connection of the proposed criterion to those used in kernel principal component analysis and kernel discriminant analysis, leading to a discriminant analysis version of the proposed method. Our analysis also provides more insights on the properties of the feature spaces obtained by applying these methods.
Measuring Similarity: Computationally Reproducing the Scholar's Interests
Authors: Ashley Lee, Jo Guldi, Andras Zsom
Date: 2018-12-14 00:00:00
Tasks: Articles, Classification, Document Classification, General Classification, text-classification, Text Classification

Computerized document classification already orders the news articles that
Apple's "News" app or Google's "personalized search" feature groups together to
match a reader's interests. The invisible and therefore illegible decisions
that go into these tailored searches have been the subject of a critique by
scholars who emphasize that our intelligence about documents is only as good as
our ability to understand the criteria of search. This article will attempt to
unpack the procedures used in computational classification of texts,
translating them into term legible to humanists, and examining opportunities to
render the computational text classification process subject to expert critique
and improvement.
Why ReLU Units Sometimes Die: Analysis of Single-Unit Error Backpropagation in Neural Networks
Authors: Scott C. Douglas, Jiutian Yu
Date: 2018-12-14 00:00:00

Recently, neural networks in machine learning use rectified linear units
(ReLUs) in early processing layers for better performance. Training these
structures sometimes results in "dying ReLU units" with near-zero outputs. We
first explore this condition via simulation using the CIFAR-10 dataset and
variants of two popular convolutive neural network architectures. Our
explorations show that the output activation probability Pr[y>0] is generally
less than 0.5 at system convergence for layers that do not employ skip
connections, and this activation probability tends to decrease as one
progresses from input layer to output layer. Employing a simplified model of a
single ReLU unit trained by a variant of error backpropagation, we then perform
a statistical convergence analysis to explore the model's evolutionary
behavior. Our analysis describes the potentially-slower convergence speeds of
dying ReLU units, and this issue can occur regardless of how the weights are
initialized.
Probabilistic Class-Specific Discriminant Analysis
Authors: Alexandros Iosifidis
Date: 2018-12-14 00:00:00
Tasks: Classification, General Classification

In this paper we formulate a probabilistic model for class-specific discriminant subspace learning. The proposed model can naturally incorporate the multi-modal structure of the negative class, which is neglected by existing class-specific methods. Moreover, it can be directly used to define a class-specific probabilistic classification rule in the discriminant subspace. We show that existing class-specific discriminant analysis methods are special cases of the proposed probabilistic model and, by casting them as probabilistic models, they can be extended to class-specific classifiers. We illustrate the performance of the proposed model in both verification and classification problems.
Scaling shared model governance via model splitting
Authors: Miljan Martic, Jan Leike, Andrew Trask, Matteo Hessel, Shane Legg, Pushmeet Kohli
Date: 2018-12-14 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Learning, model, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Currently the only techniques for sharing governance of a deep learning model
are homomorphic encryption and secure multiparty computation. Unfortunately,
neither of these techniques is applicable to the training of large neural
networks due to their large computational and communication overheads. As a
scalable technique for shared model governance, we propose splitting deep
learning model between multiple parties. This paper empirically investigates
the security guarantee of this technique, which is introduced as the problem of
model completion: Given the entire training data set or an environment
simulator, and a subset of the parameters of a trained deep learning model, how
much training is required to recover the model's original performance? We
define a metric for evaluating the hardness of the model completion problem and
study it empirically in both supervised learning on ImageNet and reinforcement
learning on Atari and DeepMind~Lab. Our experiments show that (1) the model
completion problem is harder in reinforcement learning than in supervised
learning because of the unavailability of the trained agent's trajectories, and
(2) its hardness depends not primarily on the number of parameters of the
missing part, but more so on their type and location. Our results suggest that
model splitting might be a feasible technique for shared model governance in
some settings where training is very expensive.
EEG-based Communication with a Predictive Text Algorithm
Authors: Daniel Omeiza, Kayode Sakariyah Adewole, Daniel Nkemelu
Date: 2018-12-14 00:00:00
Tasks: Decision Making, EEG, Electroencephalogram (EEG)

Several changes occur in the brain in response to voluntary and involuntary activities performed by a person. The ability to retrieve data from the brain within a time space provides a basis for in-depth analyses that offer insight on what changes occur in the brain during its decision-making processes. In this work, we present the technical description and software implementation of an electroencephalographic (EEG) based communication system. We read EEG data in real-time with which we compute the likelihood that a voluntary eye blink has been made by a person and use the decision to trigger buttons on a user interface in order to produce text. Relevant texts are suggested using a modification of the T9 algorithm. Our results indicate that EEG-based technology can be effectively applied in facilitating speech for people with severe speech and muscular disabilities, providing a foundation for future work in the area.
A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms, Experimental Analysis, Prospects and Challenges (with Appendices on Mathematical Background and Detailed Algorithms Explanation)
Authors: Juan Luis Suárez-Díaz, Salvador García, Francisco Herrera
Date: 2018-12-14 00:00:00
Tasks: Dimensionality Reduction, General Classification, Metric Learning

Distance metric learning is a branch of machine learning that aims to learn distances from the data, which enhances the performance of similarity-based algorithms. This tutorial provides a theoretical background and foundations on this topic and a comprehensive experimental analysis of the most-known algorithms. We start by describing the distance metric learning problem and its main mathematical foundations, divided into three main blocks: convex analysis, matrix analysis and information theory. Then, we will describe a representative set of the most popular distance metric learning methods used in classification. All the algorithms studied in this paper will be evaluated with exhaustive testing in order to analyze their capabilities in standard classification problems, particularly considering dimensionality reduction and kernelization. The results, verified by Bayesian statistical tests, highlight a set of outstanding algorithms. Finally, we will discuss several potential future prospects and challenges in this field. This tutorial will serve as a starting point in the domain of distance metric learning from both a theoretical and practical perspective.
Context-encoding Variational Autoencoder for Unsupervised Anomaly Detection
Authors: David Zimmerer, Simon A. A. Kohl, Jens Petersen, Fabian Isensee, Klaus H. Maier-Hein
Date: 2018-12-14 00:00:00
Tasks: Anomaly Detection, Unsupervised Anomaly Detection

Unsupervised learning can leverage large-scale data sources without the need
for annotations. In this context, deep learning-based auto encoders have shown
great potential in detecting anomalies in medical images. However,
state-of-the-art anomaly scores are still based on the reconstruction error,
which lacks in two essential parts: it ignores the model-internal
representation employed for reconstruction, and it lacks formal assertions and
comparability between samples. We address these shortcomings by proposing the
Context-encoding Variational Autoencoder (ceVAE) which combines reconstruction-
with density-based anomaly scoring. This improves the sample- as well as
pixel-wise results. In our experiments on the BraTS-2017 and ISLES-2015
segmentation benchmarks, the ceVAE achieves unsupervised ROC-AUCs of 0.95 and
0.89, respectively, thus outperforming state-of-the-art methods by a
considerable margin.
Detecting Reliable Novel Word Senses: A Network-Centric Approach
Authors: Abhik Jana, Animesh Mukherjee, Pawan Goyal
Date: 2018-12-14 00:00:00

In this era of Big Data, due to expeditious exchange of information on the
web, words are being used to denote newer meanings, causing linguistic shift.
With the recent availability of large amounts of digitized texts, an automated
analysis of the evolution of language has become possible. Our study mainly
focuses on improving the detection of new word senses. This paper presents a
unique proposal based on network features to improve the precision of new word
sense detection. For a candidate word where a new sense (birth) has been
detected by comparing the sense clusters induced at two different time points,
we further compare the network properties of the subgraphs induced from novel
sense cluster across these two time points. Using the mean fractional change in
edge density, structural similarity and average path length as features in an
SVM classifier, manual evaluation gives precision values of 0.86 and 0.74 for
the task of new sense detection, when tested on 2 distinct time-point pairs, in
comparison to the precision values in the range of 0.23-0.32, when the proposed
scheme is not used. The outlined method can therefore be used as a new post-hoc
step to improve the precision of novel word sense detection in a robust and
reliable way where the underlying framework uses a graph structure. Another
important observation is that even though our proposal is a post-hoc step, it
can be used in isolation and that itself results in a very decent performance
achieving a precision of 0.54-0.62. Finally, we show that our method is able to
detect the well-known historical shifts in 80% cases.
Model-free Training of End-to-end Communication Systems
Authors: Fayçal Ait Aoudia, Jakob Hoydis
Date: 2018-12-14 00:00:00

The idea of end-to-end learning of communication systems through neural network-based autoencoders has the shortcoming that it requires a differentiable channel model. We present in this paper a novel learning algorithm which alleviates this problem. The algorithm enables training of communication systems with an unknown channel model or with non-differentiable components. It iterates between training of the receiver using the true gradient, and training of the transmitter using an approximation of the gradient. We show that this approach works as well as model-based training for a variety of channels and tasks. Moreover, we demonstrate the algorithm's practical viability through hardware implementation on software-defined radios where it achieves state-of-the-art performance over a coaxial cable and wireless channel.
DATELINE: Deep Plackett-Luce Model with Uncertainty Measurements
Authors: Bo Han
Date: 2018-12-14 00:00:00

The aggregation of k-ary preferences is a historical and important problem,
since it has many real-world applications, such as peer grading, presidential
elections and restaurant ranking. Meanwhile, variants of Plackett-Luce model
has been applied to aggregate k-ary preferences. However, there are two urgent
issues still existing in the current variants. First, most of them ignore
feature information. Namely, they consider k-ary preferences instead of
instance-dependent k-ary preferences. Second, these variants barely consider
the uncertainty in k-ary preferences provided by agnostic crowds. In this
paper, we propose Deep plAckeTt-luce modEL wIth uNcertainty mEasurements
(DATELINE), which can address both issues simultaneously. To address the first
issue, we employ deep neural networks mapping each instance into its ranking
score in Plackett-Luce model. Then, we present a weighted Plackett-Luce model
to solve the second issue, where the weight is a dynamic uncertainty vector
measuring the worker quality. More importantly, we provide theoretical
guarantees for DATELINE to justify its robustness.
The Coherent Point Drift for Clustered Point Sets
Authors: Dmitry Lachinov, Vadim Turlapov
Date: 2018-12-14 00:00:00
Tasks: Clustering

The problem of non-rigid point set registration is a key problem for many
computer vision tasks. In many cases the nature of the data or capabilities of
the point detection algorithms can give us some prior information on point sets
distribution. In non-rigid case this information is able to drastically improve
registration results by limiting number of possible solutions. In this paper we
explore use of prior information about point sets clustering, such information
can be obtained with preliminary segmentation. We extend existing probabilistic
framework for fitting two level Gaussian mixture model and derive closed form
solution for maximization step of the EM algorithm. This enables us to improve
method accuracy with almost no performance loss. We evaluate our approach and
compare the Cluster Coherent Point Drift with other existing non-rigid point
set registration methods and show it's advantages for digital medicine tasks,
especially for heart template model personalization using patient's medical
data.
Evolutionary Neural Architecture Search for Image Restoration
Authors: Gerard Jacques van Wyk, Anna Sergeevna Bosman
Date: 2018-12-14 00:00:00
Tasks: GPU, image-classification, Image Classification, Image Restoration, Image-to-Image Regression, Neural Architecture Search

Convolutional neural network (CNN) architectures have traditionally been
explored by human experts in a manual search process that is time-consuming and
ineffectively explores the massive space of potential solutions. Neural
architecture search (NAS) methods automatically search the space of neural
network hyperparameters in order to find optimal task-specific architectures.
NAS methods have discovered CNN architectures that achieve state-of-the-art
performance in image classification among other tasks, however the application
of NAS to image-to-image regression problems such as image restoration is
sparse. This paper proposes a NAS method that performs computationally
efficient evolutionary search of a minimally constrained network architecture
search space. The performance of architectures discovered by the proposed
method is evaluated on a variety of image restoration tasks applied to the
ImageNet64x64 dataset, and compared with human-engineered CNN architectures.
The best neural architectures discovered using only 2 GPU-hours of evolutionary
search exhibit comparable performance to the human-engineered baseline
architecture.
Multi-hypothesis contextual modeling for semantic segmentation
Authors: Hasan F. Ates, Sercan Sunetci
Date: 2018-12-14 00:00:00
Tasks: Segmentation, Semantic Segmentation

Semantic segmentation (i.e. image parsing) aims to annotate each image pixel
with its corresponding semantic class label. Spatially consistent labeling of
the image requires an accurate description and modeling of the local contextual
information. Segmentation result is typically improved by Markov Random Field
(MRF) optimization on the initial labels. However this improvement is limited
by the accuracy of initial result and how the contextual neighborhood is
defined. In this paper, we develop generalized and flexible contextual models
for segmentation neighborhoods in order to improve parsing accuracy. Instead of
using a fixed segmentation and neighborhood definition, we explore various
contextual models for fusion of complementary information available in
alternative segmentations of the same image. In other words, we propose a novel
MRF framework that describes and optimizes the contextual dependencies between
multiple segmentations. Simulation results on two common datasets demonstrate
significant improvement in parsing accuracy over the baseline approaches.
Imitation Learning for End to End Vehicle Longitudinal Control with Forward Camera
Authors: Laurent George, Thibault Buhet, Emilie Wirbel, Gaetan Le-Gall, Xavier Perrotton
Date: 2018-12-14 00:00:00
Tasks: Data Augmentation, Imitation Learning

In this paper we present a complete study of an end-to-end imitation learning
system for speed control of a real car, based on a neural network with a Long
Short Term Memory (LSTM). To achieve robustness and generalization from expert
demonstrations, we propose data augmentation and label augmentation that are
relevant for imitation learning in longitudinal control context. Based on front
camera image only, our system is able to correctly control the speed of a car
in simulation environment, and in a real car on a challenging test track. The
system also shows promising results in open road context.