pwc-archive/papers-with-abstracts huggingface dataset:
Analysis of KNN Information Estimators for Smooth Distributions
Authors: Puning Zhao, Lifeng Lai
Date: 2018-10-27 00:00:00

KSG mutual information estimator, which is based on the distances of each sample to its k-th nearest neighbor, is widely used to estimate mutual information between two continuous random variables. Existing work has analyzed the convergence rate of this estimator for random variables whose densities are bounded away from zero in its support. In practice, however, KSG estimator also performs well for a much broader class of distributions, including not only those with bounded support and densities bounded away from zero, but also those with bounded support but densities approaching zero, and those with unbounded support. In this paper, we analyze the convergence rate of the error of KSG estimator for smooth distributions, whose support of density can be both bounded and unbounded. As KSG mutual information estimator can be viewed as an adaptive recombination of KL entropy estimators, in our analysis, we also provide convergence analysis of KL entropy estimator for a broad class of distributions.
Monitoring the shape of weather, soundscapes, and dynamical systems: a new statistic for dimension-driven data analysis on large data sets
Authors: Henry Kvinge, Elin Farnell, Michael Kirby, Chris Peterson
Date: 2018-10-27 00:00:00
Tasks: Dimensionality Reduction

Dimensionality-reduction methods are a fundamental tool in the analysis of
large data sets. These algorithms work on the assumption that the "intrinsic
dimension" of the data is generally much smaller than the ambient dimension in
which it is collected. Alongside their usual purpose of mapping data into a
smaller dimension with minimal information loss, dimensionality-reduction
techniques implicitly or explicitly provide information about the dimension of
the data set.
  In this paper, we propose a new statistic that we call the $\kappa$-profile
for analysis of large data sets. The $\kappa$-profile arises from a
dimensionality-reduction optimization problem: namely that of finding a
projection into $k$-dimensions that optimally preserves the secants between
points in the data set. From this optimal projection we extract $\kappa,$ the
norm of the shortest projected secant from among the set of all normalized
secants. This $\kappa$ can be computed for any $k$; thus the tuple of $\kappa$
values (indexed by dimension) becomes a $\kappa$-profile. Algorithms such as
the Secant-Avoidance Projection algorithm and the Hierarchical Secant-Avoidance
Projection algorithm, provide a computationally feasible means of estimating
the $\kappa$-profile for large data sets, and thus a method of understanding
and monitoring their behavior. As we demonstrate in this paper, the
$\kappa$-profile serves as a useful statistic in several representative
settings: weather data, soundscape data, and dynamical systems data.
MCA-based Rule Mining Enables Interpretable Inference in Clinical Psychiatry
Authors: Qingzhu Gao, Humberto Gonzalez, Parvez Ahammad
Date: 2018-10-26 00:00:00
Tasks: BIG-bench Machine Learning, Interpretable Machine Learning

Development of interpretable machine learning models for clinical healthcare
applications has the potential of changing the way we understand, treat, and
ultimately cure, diseases and disorders in many areas of medicine. These models
can serve not only as sources of predictions and estimates, but also as
discovery tools for clinicians and researchers to reveal new knowledge from the
data. High dimensionality of patient information (e.g., phenotype, genotype,
and medical history), lack of objective measurements, and the heterogeneity in
patient populations often create significant challenges in developing
interpretable machine learning models for clinical psychiatry in practice. In
this paper we take a step towards the development of such interpretable models.
First, by developing a novel categorical rule mining method based on
Multivariate Correspondence Analysis (MCA) capable of handling datasets with
large numbers of features, and second, by applying this method to build
transdiagnostic Bayesian Rule List models to screen for psychiatric disorders
using the Consortium for Neuropsychiatric Phenomics dataset. We show that our
method is not only at least 100 times faster than state-of-the-art rule mining
techniques for datasets with 50 features, but also provides interpretability
and comparable prediction accuracy across several benchmark datasets.
Deep Convolutional Neural Network Applied to Quality Assessment for Video Tracking
Authors: Roger Gomez Nieto, Eugenio Tamura Morimitsu
Date: 2018-10-26 00:00:00

Surveillance videos often suffer from blur and exposure distortions that
occur during acquisition and storage, which can adversely influence following
automatic image analysis results on video-analytic tasks. The purpose of this
paper is to deploy an algorithm that can automatically assess the presence of
exposure distortion in videos. In this work we to design and build one
architecture for deep learning applied to recognition of distortions in a
video. The goal is to know if the video present exposure distortions. Such an
algorithm could be used to enhance or restoration image or to create an object
tracker distortion-aware.
Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach
Authors: Behnam Gholami, Pritish Sahu, Ognjen Rudovic, Konstantinos Bousmalis, Vladimir Pavlovic
Date: 2018-10-26 00:00:00
Proceeding: ICLR 2019 5
Tasks: Disentanglement, Domain Adaptation, Multi-target Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation (uDA) models focus on pairwise adaptation
settings where there is a single, labeled, source and a single target domain.
However, in many real-world settings one seeks to adapt to multiple, but
somewhat similar, target domains. Applying pairwise adaptation approaches to
this setting may be suboptimal, as they fail to leverage shared information
among multiple domains. In this work we propose an information theoretic
approach for domain adaptation in the novel context of multiple target domains
with unlabeled instances and one source domain with labeled instances. Our
model aims to find a shared latent space common to all domains, while
simultaneously accounting for the remaining private, domain-specific factors.
Disentanglement of shared and private information is accomplished using a
unified information-theoretic approach, which also serves to establish a
stronger link between the latent representations and the observed data. The
resulting model, accompanied by an efficient optimization algorithm, allows
simultaneous adaptation from a single source to multiple target domains. We
test our approach on three challenging publicly-available datasets, showing
that it outperforms several popular domain adaptation methods.
Mobile Sensor Data Anonymization
Authors: Mohammad Malekzadeh, Richard G. Clegg, Andrea Cavallaro, Hamed Haddadi
Date: 2018-10-26 00:00:00
Tasks: Activity Recognition, Decoder, User Identification

Motion sensors such as accelerometers and gyroscopes measure the instant
acceleration and rotation of a device, in three dimensions. Raw data streams
from motion sensors embedded in portable and wearable devices may reveal
private information about users without their awareness. For example, motion
data might disclose the weight or gender of a user, or enable their
re-identification. To address this problem, we propose an on-device
transformation of sensor data to be shared for specific applications, such as
monitoring selected daily activities, without revealing information that
enables user identification. We formulate the anonymization problem using an
information-theoretic approach and propose a new multi-objective loss function
for training deep autoencoders. This loss function helps minimizing
user-identity information as well as data distortion to preserve the
application-specific utility. The training process regulates the encoder to
disregard user-identifiable patterns and tunes the decoder to shape the output
independently of users in the training set. The trained autoencoder can be
deployed on a mobile or wearable device to anonymize sensor data even for users
who are not included in the training dataset. Data from 24 users transformed by
the proposed anonymizing autoencoder lead to a promising trade-off between
utility and privacy, with an accuracy for activity recognition above 92% and an
accuracy for user identification below 7%.
Efficiently Combining Human Demonstrations and Interventions for Safe Training of Autonomous Systems in Real-Time
Authors: Vinicius G. Goecks, Gregory M. Gremillion, Vernon J. Lawhern, John Valasek, Nicholas R. Waytowich
Date: 2018-10-26 00:00:00
Tasks: Imitation Learning

This paper investigates how to utilize different forms of human interaction
to safely train autonomous systems in real-time by learning from both human
demonstrations and interventions. We implement two components of the
Cycle-of-Learning for Autonomous Systems, which is our framework for combining
multiple modalities of human interaction. The current effort employs human
demonstrations to teach a desired behavior via imitation learning, then
leverages intervention data to correct for undesired behaviors produced by the
imitation learner to teach novel tasks to an autonomous agent safely, after
only minutes of training. We demonstrate this method in an autonomous perching
task using a quadrotor with continuous roll, pitch, yaw, and throttle commands
and imagery captured from a downward-facing camera in a high-fidelity simulated
environment. Our method improves task completion performance for the same
amount of human interaction when compared to learning from demonstrations
alone, while also requiring on average 32% less data to achieve that
performance. This provides evidence that combining multiple modes of human
interaction can increase both the training speed and overall performance of
policies for autonomous systems.
Quantifying Learning Guarantees for Convex but Inconsistent Surrogates
Authors: Kirill Struminsky, Simon Lacoste-Julien, Anton Osokin
Date: 2018-10-26 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: General Classification, Multi-class Classification

We study consistency properties of machine learning methods based on
minimizing convex surrogates. We extend the recent framework of Osokin et al.
(2017) for the quantitative analysis of consistency properties to the case of
inconsistent surrogates. Our key technical contribution consists in a new lower
bound on the calibration function for the quadratic surrogate, which is
non-trivial (not always zero) for inconsistent cases. The new bound allows to
quantify the level of inconsistency of the setting and shows how learning with
inconsistent surrogates can have guarantees on sample complexity and
optimization difficulty. We apply our theory to two concrete cases: multi-class
classification with the tree-structured loss and ranking with the mean average
precision loss. The results show the approximation-computation trade-offs
caused by inconsistent surrogates and their potential benefits.
Automatic Graphics Program Generation using Attention-Based Hierarchical Decoder
Authors: Zhihao Zhu, Zhan Xue, Zejian yuan
Date: 2018-10-26 00:00:00
Tasks: Code Generation, Decoder

Recent progress on deep learning has made it possible to automatically
transform the screenshot of Graphic User Interface (GUI) into code by using the
encoder-decoder framework. While the commonly adopted image encoder (e.g., CNN
network), might be capable of extracting image features to the desired level,
interpreting these abstract image features into hundreds of tokens of code puts
a particular challenge on the decoding power of the RNN-based code generator.
Considering the code used for describing GUI is usually hierarchically
structured, we propose a new attention-based hierarchical code generation
model, which can describe GUI images in a finer level of details, while also
being able to generate hierarchically structured code in consistency with the
hierarchical layout of the graphic elements in the GUI. Our model follows the
encoder-decoder framework, all the components of which can be trained jointly
in an end-to-end manner. The experimental results show that our method
outperforms other current state-of-the-art methods on both a publicly available
GUI-code dataset as well as a dataset established by our own.
A Theoretical Framework of Approximation Error Analysis of Evolutionary Algorithms
Authors: Jun He, Yu Chen, Yuren Zhou
Date: 2018-10-26 00:00:00
Tasks: Evolutionary Algorithms

In the empirical study of evolutionary algorithms, the solution quality is
evaluated by either the fitness value or approximation error. The latter
measures the fitness difference between an approximation solution and the
optimal solution. Since the approximation error analysis is more convenient
than the direct estimation of the fitness value, this paper focuses on
approximation error analysis. However, it is straightforward to extend all
related results from the approximation error to the fitness value. Although the
evaluation of solution quality plays an essential role in practice, few
rigorous analyses have been conducted on this topic. This paper aims at
establishing a novel theoretical framework of approximation error analysis of
evolutionary algorithms for discrete optimization. This framework is divided
into two parts. The first part is about exact expressions of the approximation
error. Two methods, Jordan form and Schur's triangularization, are presented to
obtain an exact expression. The second part is about upper bounds on
approximation error. Two methods, convergence rate and auxiliary matrix
iteration, are proposed to estimate the upper bound. The applicability of this
framework is demonstrated through several examples.
Automatic differentiation in ML: Where we are and where we should be going
Authors: Bart van Merriënboer, Olivier Breuleux, Arnaud Bergeron, Pascal Lamblin
Date: 2018-10-26 00:00:00
Proceeding: NeurIPS 2018 12

We review the current state of automatic differentiation (AD) for array
programming in machine learning (ML), including the different approaches such
as operator overloading (OO) and source transformation (ST) used for AD,
graph-based intermediate representations for programs, and source languages.
Based on these insights, we introduce a new graph-based intermediate
representation (IR) which specifically aims to efficiently support
fully-general AD for array programming. Unlike existing dataflow programming
representations in ML frameworks, our IR naturally supports function calls,
higher-order functions and recursion, making ML models easier to implement. The
ability to represent closures allows us to perform AD using ST without a tape,
making the resulting derivative (adjoint) program amenable to ahead-of-time
optimization using tools from functional language compilers, and enabling
higher-order derivatives. Lastly, we introduce a proof of concept compiler
toolchain called Myia which uses a subset of Python as a front end.
Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication
Authors: William Severa, Craig M. Vineyard, Ryan Dellana, Stephen J. Verzi, James B. Aimone
Date: 2018-10-26 00:00:00
Tasks: General Classification, image-classification, Image Classification, Semantic Segmentation

This paper presents a new technique for training networks for low-precision
communication. Targeting minimal communication between nodes not only enables
the use of emerging spiking neuromorphic platforms, but may additionally
streamline processing conventionally. Low-power and embedded neuromorphic
processors potentially offer dramatic performance-per-Watt improvements over
traditional von Neumann processors, however programming these brain-inspired
platforms generally requires platform-specific expertise which limits their
applicability. To date, the majority of artificial neural networks have not
operated using discrete spike-like communication.
  We present a method for training deep spiking neural networks using an
iterative modification of the backpropagation optimization algorithm. This
method, which we call Whetstone, effectively and reliably configures a network
for a spiking hardware target with little, if any, loss in performance.
Whetstone networks use single time step binary communication and do not require
a rate code or other spike-based coding scheme, thus producing networks
comparable in timing and size to conventional ANNs, albeit with binarized
communication. We demonstrate Whetstone on a number of image classification
networks, describing how the sharpening process interacts with different
training optimizers and changes the distribution of activity within the
network. We further note that Whetstone is compatible with several
non-classification neural network applications, such as autoencoders and
semantic segmentation. Whetstone is widely extendable and currently implemented
using custom activation functions within the Keras wrapper to the popular
TensorFlow machine learning framework.
Spectrogram-channels u-net: a source separation model viewing each channel as the spectrogram of each source
Authors: Jaehoon Oh, Duyeon Kim, Se-Young Yun
Date: 2018-10-26 00:00:00
Tasks: Information Retrieval, Music Information Retrieval, Retrieval, Singer Identification, Voice Conversion

Sound source separation has attracted attention from Music Information
Retrieval(MIR) researchers, since it is related to many MIR tasks such as
automatic lyric transcription, singer identification, and voice conversion. In
this paper, we propose an intuitive spectrogram-based model for source
separation by adapting U-Net. We call it Spectrogram-Channels U-Net, which
means each channel of the output corresponds to the spectrogram of separated
source itself. The proposed model can be used for not only singing voice
separation but also multi-instrument separation by changing only the number of
output channels. In addition, we propose a loss function that balances volumes
between different sources. Finally, we yield performance that is
state-of-the-art on both separation tasks.
An Acceleration Scheme to The Local Directional Pattern
Authors: Yasin Musa Ayami, Aboubayda Shabat
Date: 2018-10-26 00:00:00

This study seeks to improve the running time of the Local Directional Pattern
(LDP) during feature extraction using a newly proposed acceleration scheme to
LDP. LDP is considered to be computationally expensive. To confirm this, the
running time of the LDP to gray level co-occurrence matrix (GLCM) were it was
established that the running time for LDP was two orders of magnitude higher
than that of the GLCM. In this study, the performance of the newly proposed
acceleration scheme was evaluated against LDP and Local Binary patter (LBP)
using images from the publicly available extended Cohn-Kanade (CK+) dataset.
Based on our findings, the proposed acceleration scheme significantly improves
the running time of the LDP by almost 3 times during feature extraction
Noise Sensitivity of Local Descriptors vs ConvNets: An application to Facial Recognition
Authors: Yasin Musa Ayami, Aboubayda Shabat, Delene Heukelman
Date: 2018-10-26 00:00:00
Tasks: Sensitivity

The Local Binary Patterns (LBP) is a local descriptor proposed by Ojala et al
to discriminate texture due to its discriminative power. However, the LBP is
sensitive to noise and illumination changes. Consequently, several extensions
to the LBP such as Median Binary Pattern (MBP) and methods such as Local
Directional Pattern (LDP) have been proposed to address its drawbacks. Though
studies by Zhou et al, suggest that the LDP exhibits poor performance in
presence of random noise. Recently, convolution neural networks (ConvNets) were
introduced which are increasingly becoming popular for feature extraction due
to their discriminative power. This study aimed at evaluating the sensitivity
of ResNet50, a ConvNet pre-trained model and local descriptors (LBP and LDP) to
noise using the Extended Yale B face dataset with 5 different levels of noise
added to the dataset. In our findings, it was observed that despite adding
different levels of noise to the dataset, ResNet50 proved to be more robust
than the local descriptors (LBP and LDP).
Learning and Interpreting Multi-Multi-Instance Learning Networks
Authors: Alessandro Tibo, Manfred Jaeger, Paolo Frasconi
Date: 2018-10-26 00:00:00
Tasks: General Classification, image-classification, Image Classification, text-classification, Text Classification

We introduce an extension of the multi-instance learning problem where examples are organized as nested bags of instances (e.g., a document could be represented as a bag of sentences, which in turn are bags of words). This framework can be useful in various scenarios, such as text and image classification, but also supervised learning over graphs. As a further advantage, multi-multi instance learning enables a particular way of interpreting predictions and the decision function. Our approach is based on a special neural network layer, called bag-layer, whose units aggregate bags of inputs of arbitrary size. We prove theoretically that the associated class of functions contains all Boolean functions over sets of sets of instances and we provide empirical evidence that functions of this kind can be actually learned on semi-synthetic datasets. We finally present experiments on text classification, on citation graphs, and social graph data, which show that our model obtains competitive results with respect to accuracy when compared to other approaches such as convolutional networks on graphs, while at the same time it supports a general approach to interpret the learnt model, as well as explain individual predictions.
Neural Network-Based Approach to Phase Space Integration
Authors: Matthew D. Klimek, Maxim Perelstein
Date: 2018-10-26 00:00:00

Monte Carlo methods are widely used in particle physics to integrate and sample probability distributions (differential cross sections or decay rates) on multi-dimensional phase spaces. We present a Neural Network (NN) algorithm optimized to perform this task. The algorithm has been applied to several examples of direct relevance for particle physics, including situations with non-trivial features such as sharp resonances and soft/collinear enhancements. Excellent performance has been demonstrated in all examples, with the properly trained NN achieving unweighting efficiencies of between 30% and 75%. In contrast to traditional Monte Carlo algorithms such as VEGAS, the NN-based approach does not require that the phase space coordinates be aligned with resonant or other features in the cross section.
Efficient Distributed Hessian Free Algorithm for Large-scale Empirical Risk Minimization via Accumulating Sample Strategy
Authors: Majid Jahani, Xi He, Chenxin Ma, Aryan Mokhtari, Dheevatsa Mudigere, Alejandro Ribeiro, Martin Takáč
Date: 2018-10-26 00:00:00

In this paper, we propose a Distributed Accumulated Newton Conjugate gradiEnt (DANCE) method in which sample size is gradually increasing to quickly obtain a solution whose empirical loss is under satisfactory statistical accuracy. Our proposed method is multistage in which the solution of a stage serves as a warm start for the next stage which contains more samples (including the samples in the previous stage). The proposed multistage algorithm reduces the number of passes over data to achieve the statistical accuracy of the full training set. Moreover, our algorithm in nature is easy to be distributed and shares the strong scaling property indicating that acceleration is always expected by using more computing nodes. Various iteration complexity results regarding descent direction computation, communication efficiency and stopping criteria are analyzed under convex setting. Our numerical results illustrate that the proposed method outperforms other comparable methods for solving learning problems including neural networks.