pwc-archive/papers-with-abstracts huggingface dataset:
Robust Multiple Kernel k-means Clustering using Min-Max Optimization
Authors: Seojin Bang, Yao-Liang Yu, Wei Wu
Date: 2018-03-06 00:00:00
Tasks: Clustering, Disease Prediction, Multiview Learning

Multiple kernel learning is a type of multiview learning that combines different data modalities by capturing view-specific patterns using kernels. Although supervised multiple kernel learning has been extensively studied, until recently, only a few unsupervised approaches have been proposed. In the meanwhile, adversarial learning has recently received much attention. Many works have been proposed to defend against adversarial examples. However, little is known about the effect of adversarial perturbation in the context of multiview learning, and even less in the unsupervised case. In this study, we show that adversarial features added to a view can make the existing approaches with the min-max formulation in multiple kernel clustering yield unfavorable clusters. To address this problem and inspired by recent works in adversarial learning, we propose a multiple kernel clustering method with the min-max framework that aims to be robust to such adversarial perturbation. We evaluate the robustness of our method on simulation data under different types of adversarial perturbations and show that it outperforms several compared existing methods. In the real data analysis, We demonstrate the utility of our method on a real-world problem.
A Reductions Approach to Fair Classification
Authors: Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, Hanna Wallach
Date: 2018-03-06 00:00:00
Proceeding: ICML 2018 7
Tasks: Binary Classification, Classification, Fairness, General Classification

We present a systematic approach for achieving fairness in a binary
classification setting. While we focus on two well-known quantitative
definitions of fairness, our approach encompasses many other previously studied
definitions as special cases. The key idea is to reduce fair classification to
a sequence of cost-sensitive classification problems, whose solutions yield a
randomized classifier with the lowest (empirical) error subject to the desired
constraints. We introduce two reductions that work for any representation of
the cost-sensitive classifier and compare favorably to prior baselines on a
variety of data sets, while overcoming several of their disadvantages.
Categorical Mixture Models on VGGNet activations
Authors: Sean Billings
Date: 2018-03-06 00:00:00
Tasks: Object Recognition

In this project, I use unsupervised learning techniques in order to cluster a
set of yelp restaurant photos under meaningful topics. In order to do this, I
extract layer activations from a pre-trained implementation of the popular
VGGNet convolutional neural network. First, I explore using LDA with the
activations of convolutional layers as features. Secondly, I explore using the
object-recognition powers of VGGNet trained on ImageNet in order to extract
meaningful objects from the photos, and then perform LDA to group the photos
under topic-archetypes. I find that this second approach finds meaningful
archetypes, which match the human intuition for photo topics such as
restaurant, food, and drinks. Furthermore, these clusters align well and
distinctly with the actual yelp photo labels.
On Nonlinear Dimensionality Reduction, Linear Smoothing and Autoencoding
Authors: Daniel Ting, Michael. I. Jordan
Date: 2018-03-06 00:00:00
Tasks: Dimensionality Reduction

We develop theory for nonlinear dimensionality reduction (NLDR). A number of
NLDR methods have been developed, but there is limited understanding of how
these methods work and the relationships between them. There is limited basis
for using existing NLDR theory for deriving new algorithms. We provide a novel
framework for analysis of NLDR via a connection to the statistical theory of
linear smoothers. This allows us to both understand existing methods and derive
new ones. We use this connection to smoothing to show that asymptotically,
existing NLDR methods correspond to discrete approximations of the solutions of
sets of differential equations given a boundary condition. In particular, we
can characterize many existing methods in terms of just three limiting
differential operators and boundary conditions. Our theory also provides a way
to assert that one method is preferable to another; indeed, we show Local
Tangent Space Alignment is superior within a class of methods that assume a
global coordinate chart defines an isometric embedding of the manifold.
Matched Filters for Noisy Induced Subgraph Detection
Authors: Daniel L. Sussman, Youngser Park, Carey E. Priebe, Vince Lyzinski
Date: 2018-03-06 00:00:00
Tasks: Graph Matching

The problem of finding the vertex correspondence between two noisy graphs with different number of vertices where the smaller graph is still large has many applications in social networks, neuroscience, and computer vision. We propose a solution to this problem via a graph matching matched filter: centering and padding the smaller adjacency matrix and applying graph matching methods to align it to the larger network. The centering and padding schemes can be incorporated into any algorithm that matches using adjacency matrices. Under a statistical model for correlated pairs of graphs, which yields a noisy copy of the small graph within the larger graph, the resulting optimization problem can be guaranteed to recover the true vertex correspondence between the networks. However, there are currently no efficient algorithms for solving this problem. To illustrate the possibilities and challenges of such problems, we use an algorithm that can exploit a partially known correspondence and show via varied simulations and applications to {\it Drosophila} and human connectomes that this approach can achieve good performance.
Masked Conditional Neural Networks for Audio Classification
Authors: Fady Medhat, David Chesmore, John Robinson
Date: 2018-03-06 00:00:00
Tasks: Audio Classification, Classification, General Classification

We present the ConditionaL Neural Network (CLNN) and the Masked ConditionaL
Neural Network (MCLNN) designed for temporal signal recognition. The CLNN takes
into consideration the temporal nature of the sound signal and the MCLNN
extends upon the CLNN through a binary mask to preserve the spatial locality of
the features and allows an automated exploration of the features combination
analogous to hand-crafting the most relevant features for the recognition task.
MCLNN has achieved competitive recognition accuracies on the GTZAN and the
ISMIR2004 music datasets that surpass several state-of-the-art neural network
based architectures and hand-crafted methods applied on both datasets.
Variational Bayes In Private Settings (VIPS)
Authors: Mijung Park, James Foulds, Kamalika Chaudhuri, Max Welling
Date: 2016-11-01 00:00:00
Tasks: Bayesian Inference, Data Augmentation, Privacy Preserving

Many applications of Bayesian data analysis involve sensitive information,
motivating methods which ensure that privacy is protected. We introduce a
general privacy-preserving framework for Variational Bayes (VB), a widely used
optimization-based Bayesian inference method. Our framework respects
differential privacy, the gold-standard privacy criterion, and encompasses a
large class of probabilistic models, called the Conjugate Exponential (CE)
family. We observe that we can straightforwardly privatise VB's approximate
posterior distributions for models in the CE family, by perturbing the expected
sufficient statistics of the complete-data likelihood. For a broadly-used class
of non-CE models, those with binomial likelihoods, we show how to bring such
models into the CE family, such that inferences in the modified model resemble
the private variational Bayes algorithm as closely as possible, using the
Polya-Gamma data augmentation scheme. The iterative nature of variational Bayes
presents a further challenge since iterations increase the amount of noise
needed. We overcome this by combining: (1) an improved composition method for
differential privacy, called the moments accountant, which provides a tight
bound on the privacy cost of multiple VB iterations and thus significantly
decreases the amount of additive noise; and (2) the privacy amplification
effect of subsampling mini-batches from large-scale data in stochastic
learning. We empirically demonstrate the effectiveness of our method in CE and
non-CE models including latent Dirichlet allocation, Bayesian logistic
regression, and sigmoid belief networks, evaluated on real-world datasets.
Long-term Forecasting using Higher Order Tensor RNNs
Authors: Rose Yu, Stephan Zheng, Anima Anandkumar, Yisong Yue
Date: 2017-10-31 00:00:00
Proceeding: ICLR 2018 1
Tasks: Time Series, Time Series Analysis

We present Higher-Order Tensor RNN (HOT-RNN), a novel family of neural sequence architectures for multivariate forecasting in environments with nonlinear dynamics. Long-term forecasting in such systems is highly challenging, since there exist long-term temporal dependencies, higher-order correlations and sensitivity to error propagation. Our proposed recurrent architecture addresses these issues by learning the nonlinear dynamics directly using higher-order moments and higher-order state transition functions. Furthermore, we decompose the higher-order structure using the tensor-train decomposition to reduce the number of parameters while preserving the model performance. We theoretically establish the approximation guarantees and the variance bound for HOT-RNN for general sequence inputs. We also demonstrate 5% ~ 12% improvements for long-term prediction over general RNN and LSTM architectures on a range of simulated environments with nonlinear dynamics, as well on real-world time series data.
Sequence-based Multi-lingual Low Resource Speech Recognition
Authors: Siddharth Dalmia, Ramon Sanabria, Florian Metze, Alan W. black
Date: 2018-02-21 00:00:00
Tasks: speech-recognition, Speech Recognition

Techniques for multi-lingual and cross-lingual speech recognition can help in
low resource scenarios, to bootstrap systems and enable analysis of new
languages and domains. End-to-end approaches, in particular sequence-based
techniques, are attractive because of their simplicity and elegance. While it
is possible to integrate traditional multi-lingual bottleneck feature
extractors as front-ends, we show that end-to-end multi-lingual training of
sequence models is effective on context independent models trained using
Connectionist Temporal Classification (CTC) loss. We show that our model
improves performance on Babel languages by over 6% absolute in terms of
word/phoneme error rate when compared to mono-lingual systems built in the same
setting for these languages. We also show that the trained model can be adapted
cross-lingually to an unseen language using just 25% of the target data. We
show that training on multiple languages is important for very low resource
cross-lingual target scenarios, but not for multi-lingual testing scenarios.
Here, it appears beneficial to include large well prepared datasets.
Trifo-VIO: Robust and Efficient Stereo Visual Inertial Odometry using Points and Lines
Authors: Feng Zheng, Grace Tsai, Zhe Zhang, Shaoshan Liu, Chen-Chi Chu, Hongbing Hu
Date: 2018-03-06 00:00:00

In this paper, we present the Trifo Visual Inertial Odometry (Trifo-VIO), a
tightly-coupled filtering-based stereo VIO system using both points and lines.
Line features help improve system robustness in challenging scenarios when
point features cannot be reliably detected or tracked, e.g. low-texture
environment or lighting change. In addition, we propose a novel lightweight
filtering-based loop closing technique to reduce accumulated drift without
global bundle adjustment or pose graph optimization. We formulate loop closure
as EKF updates to optimally relocate the current sliding window maintained by
the filter to past keyframes. We also present the Trifo Ironsides dataset, a
new visual-inertial dataset, featuring high-quality synchronized stereo camera
and IMU data from the Ironsides sensor [3] with various motion types and
textures and millimeter-accuracy groundtruth. To validate the performance of
the proposed system, we conduct extensive comparison with state-of-the-art
approaches (OKVIS, VINS-MONO and S-MSCKF) using both the public EuRoC dataset
and the Trifo Ironsides dataset.
Visualizing Convolutional Neural Network Protein-Ligand Scoring
Authors: Joshua Hochuli, Alec Helbling, Tamar Skaist, Matthew Ragoza, David Ryan Koes
Date: 2018-03-06 00:00:00
Tasks: Drug Design

Protein-ligand scoring is an important step in a structure-based drug design
pipeline. Selecting a correct binding pose and predicting the binding affinity
of a protein-ligand complex enables effective virtual screening. Machine
learning techniques can make use of the increasing amounts of structural data
that are becoming publicly available. Convolutional neural network (CNN)
scoring functions in particular have shown promise in pose selection and
affinity prediction for protein-ligand complexes. Neural networks are known for
being difficult to interpret. Understanding the decisions of a particular
network can help tune parameters and training data to maximize performance.
Visualization of neural networks helps decompose complex scoring functions into
pictures that are more easily parsed by humans. Here we present three methods
for visualizing how individual protein-ligand complexes are interpreted by 3D
convolutional neural networks. We also present a visualization of the
convolutional filters and their weights. We describe how the intuition provided
by these visualizations aids in network design.
Arbitrary Discrete Sequence Anomaly Detection with Zero Boundary LSTM
Authors: Chase Roberts, Manish Nair
Date: 2018-03-06 00:00:00
Tasks: Anomaly Detection

We propose a simple mathematical definition and new neural architecture for
finding anomalies within discrete sequence datasets. Our model comprises of a
modified LSTM autoencoder and an array of One-Class SVMs. The LSTM takes in
elements from a sequence and creates context vectors that are used to predict
the probability distribution of the following element. These context vectors
are then used to train an array of One-Class SVMs. These SVMs are used to
determine an outlier boundary in context space.We show that our method is
consistently more stable and also outperforms standard LSTM and sliding window
anomaly detection systems on two generated datasets.
Learning SMaLL Predictors
Authors: Vikas K. Garg, Ofer Dekel, Lin Xiao
Date: 2018-03-06 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: BIG-bench Machine Learning

We present a new machine learning technique for training small
resource-constrained predictors. Our algorithm, the Sparse Multiprototype
Linear Learner (SMaLL), is inspired by the classic machine learning problem of
learning $k$-DNF Boolean formulae. We present a formal derivation of our
algorithm and demonstrate the benefits of our approach with a detailed
empirical study.
Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
Authors: Tianhao Zhang, Zoe McCarthy, Owen Jow, Dennis Lee, Xi Chen, Ken Goldberg, Pieter Abbeel
Date: 2017-10-12 00:00:00
Tasks: Imitation Learning

Imitation learning is a powerful paradigm for robot skill acquisition.
However, obtaining demonstrations suitable for learning a policy that maps from
raw pixels to actions can be challenging. In this paper we describe how
consumer-grade Virtual Reality headsets and hand tracking hardware can be used
to naturally teleoperate robots to perform complex tasks. We also describe how
imitation learning can learn deep neural network policies (mapping from pixels
to actions) that can acquire the demonstrated skills. Our experiments showcase
the effectiveness of our approach for learning visuomotor skills.
Generalizable Adversarial Examples Detection Based on Bi-model Decision Mismatch
Authors: João Monteiro, Isabela Albuquerque, Zahid Akhtar, Tiago H. Falk
Date: 2018-02-21 00:00:00

Modern applications of artificial neural networks have yielded remarkable
performance gains in a wide range of tasks. However, recent studies have
discovered that such modelling strategy is vulnerable to Adversarial Examples,
i.e. examples with subtle perturbations often too small and imperceptible to
humans, but that can easily fool neural networks. Defense techniques against
adversarial examples have been proposed, but ensuring robust performance
against varying or novel types of attacks remains an open problem. In this
work, we focus on the detection setting, in which case attackers become
identifiable while models remain vulnerable. Particularly, we employ the
decision layer of independently trained models as features for posterior
detection. The proposed framework does not require any prior knowledge of
adversarial examples generation techniques, and can be directly employed along
with unmodified off-the-shelf models. Experiments on the standard MNIST and
CIFAR10 datasets deliver empirical evidence that such detection approach
generalizes well across not only different adversarial examples generation
methods but also quality degradation attacks. Non-linear binary classifiers
trained on top of our proposed features can achieve a high detection rate
(>90%) in a set of white-box attacks and maintain such performance when tested
against unseen attacks.
Learning Memory Access Patterns
Authors: Milad Hashemi, Kevin Swersky, Jamie A. Smith, Grant Ayers, Heiner Litz, Jichuan Chang, Christos Kozyrakis, Parthasarathy Ranganathan
Date: 2018-03-06 00:00:00
Proceeding: ICML 2018 7
Tasks: BIG-bench Machine Learning

The explosion in workload complexity and the recent slow-down in Moore's law
scaling call for new approaches towards efficient computing. Researchers are
now beginning to use recent advances in machine learning in software
optimizations, augmenting or replacing traditional heuristics and data
structures. However, the space of machine learning for computer hardware
architecture is only lightly explored. In this paper, we demonstrate the
potential of deep learning to address the von Neumann bottleneck of memory
performance. We focus on the critical problem of learning memory access
patterns, with the goal of constructing accurate and efficient memory
prefetchers. We relate contemporary prefetching strategies to n-gram models in
natural language processing, and show how recurrent neural networks can serve
as a drop-in replacement. On a suite of challenging benchmark datasets, we find
that neural networks consistently demonstrate superior performance in terms of
precision and recall. This work represents the first step towards practical
neural-network based prefetching, and opens a wide range of exciting directions
for machine learning in computer architecture research.
Applying Deep Learning to the Newsvendor Problem
Authors: Afshin Oroojlooyjadid, Lawrence Snyder, Martin Takáč
Date: 2016-07-07 00:00:00
Tasks: Deep Learning

The newsvendor problem is one of the most basic and widely applied inventory
models. There are numerous extensions of this problem. If the probability
distribution of the demand is known, the problem can be solved analytically.
  However, approximating the probability distribution is not easy and is prone
to error; therefore, the resulting solution to the newsvendor problem may be
not optimal. To address this issue, we propose an algorithm based on deep
learning that optimizes the order quantities for all products based on features
of the demand data. Our algorithm integrates the forecasting and
inventory-optimization steps, rather than solving them separately, as is
typically done, and does not require knowledge of the probability distributions
of the demand. Numerical experiments on real-world data suggest that our
algorithm outperforms other approaches, including data-driven and machine
learning approaches, especially for demands with high volatility. Finally, in
order to show how this approach can be used for other inventory optimization
problems, we provide an extension for (r,Q) policies.
Deep Super Learner: A Deep Ensemble for Classification Problems
Authors: Steven Young, Tamer Abdou, Ayse Bener
Date: 2018-03-06 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification

Deep learning has become very popular for tasks such as predictive modeling
and pattern recognition in handling big data. Deep learning is a powerful
machine learning method that extracts lower level features and feeds them
forward for the next layer to identify higher level features that improve
performance. However, deep neural networks have drawbacks, which include many
hyper-parameters and infinite architectures, opaqueness into results, and
relatively slower convergence on smaller datasets. While traditional machine
learning algorithms can address these drawbacks, they are not typically capable
of the performance levels achieved by deep neural networks. To improve
performance, ensemble methods are used to combine multiple base learners. Super
learning is an ensemble that finds the optimal combination of diverse learning
algorithms. This paper proposes deep super learning as an approach which
achieves log loss and accuracy results competitive to deep neural networks
while employing traditional machine learning algorithms in a hierarchical
structure. The deep super learner is flexible, adaptable, and easy to train
with good performance across different tasks using identical hyper-parameter
values. Using traditional machine learning requires fewer hyper-parameters,
allows transparency into results, and has relatively fast convergence on
smaller datasets. Experimental results show that the deep super learner has
superior performance compared to the individual base learners, single-layer
ensembles, and in some cases deep neural networks. Performance of the deep
super learner may further be improved with task-specific tuning.
Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification
Authors: Ivo M. Baltruschat, Hannes Nickisch, Michael Grass, Tobias Knopp, Axel Saalbach
Date: 2018-03-06 00:00:00
Tasks: Classification, General Classification, Transfer Learning, X-ray Classification

The increased availability of X-ray image archives (e.g. the ChestX-ray14
dataset from the NIH Clinical Center) has triggered a growing interest in deep
learning techniques. To provide better insight into the different approaches,
and their applications to chest X-ray classification, we investigate a powerful
network architecture in detail: the ResNet-50. Building on prior work in this
domain, we consider transfer learning with and without fine-tuning as well as
the training of a dedicated X-ray network from scratch. To leverage the high
spatial resolution of X-ray data, we also include an extended ResNet-50
architecture, and a network integrating non-image data (patient age, gender and
acquisition type) in the classification process. In a concluding experiment, we
also investigate multiple ResNet depths (i.e. ResNet-38 and ResNet-101). In a
systematic evaluation, using 5-fold re-sampling and a multi-label loss
function, we compare the performance of the different approaches for pathology
classification by ROC statistics and analyze differences between the
classifiers using rank correlation. Overall, we observe a considerable spread
in the achieved performance and conclude that the X-ray-specific ResNet-38,
integrating non-image data yields the best overall results. Furthermore, class
activation maps are used to understand the classification process, and a
detailed analysis of the impact of non-image features is provided.