pwc-archive/papers-with-abstracts huggingface dataset:
Summarizing Event Sequences with Serial Episodes: A Statistical Model and an Application
Authors: Soumyajit Mitra, P. S. Sastry
Date: 2019-04-01 00:00:00
Tasks: General Classification, Temporal Sequences, text-classification, Text Classification

In this paper we address the problem of discovering a small set of frequent
serial episodes from sequential data so as to adequately characterize or
summarize the data. We discuss an algorithm based on the Minimum Description
Length (MDL) principle and the algorithm is a slight modification of an earlier
method, called CSC-2. We present a novel generative model for sequence data
containing prominent pairs of serial episodes and, using this, provide some
statistical justification for the algorithm. We believe this is the first
instance of such a statistical justification for an MDL based algorithm for
summarizing event sequence data. We then present a novel application of this
data mining algorithm in text classification. By considering text documents as
temporal sequences of words, the data mining algorithm can find a set of
characteristic episodes for all the training data as a whole. The words that
are part of these characteristic episodes could then be considered the only
relevant words for the dictionary thus resulting in a considerably reduced
feature vector dimension. We show, through simulation experiments using
benchmark data sets, that the discovered frequent episodes can be used to
achieve more than four-fold reduction in dictionary size without losing any
classification accuracy.
VINE: Visualizing Statistical Interactions in Black Box Models
Authors: Matthew Britton
Date: 2019-04-01 00:00:00

As machine learning becomes more pervasive, there is an urgent need for
interpretable explanations of predictive models. Prior work has developed
effective methods for visualizing global model behavior, as well as generating
local (instance-specific) explanations. However, relatively little work has
addressed regional explanations - how groups of similar instances behave in a
complex model, and the related issue of visualizing statistical feature
interactions. The lack of utilities available for these analytical needs
hinders the development of models that are mission-critical, transparent, and
align with social goals. We present VINE (Visual INteraction Effects), a novel
algorithm to extract and visualize statistical interaction effects in black box
models. We also present a novel evaluation metric for visualizations in the
interpretable ML space.
Deep Clustering With Intra-class Distance Constraint for Hyperspectral Images
Authors: Jinguang Sun, Wanli Wang, Xian Wei, Li Fang, Xiaoliang Tang, Yusheng Xu, Hui Yu, Wei Yao
Date: 2019-04-01 00:00:00
Tasks: Clustering, Deep Clustering

The high dimensionality of hyperspectral images often results in the
degradation of clustering performance. Due to the powerful ability of deep
feature extraction and non-linear feature representation, the clustering
algorithm based on deep learning has become a hot research topic in the field
of hyperspectral remote sensing. However, most deep clustering algorithms for
hyperspectral images utilize deep neural networks as feature extractor without
considering prior knowledge constraints that are suitable for clustering. To
solve this problem, we propose an intra-class distance constrained deep
clustering algorithm for high-dimensional hyperspectral images. The proposed
algorithm constrains the feature mapping procedure of the auto-encoder network
by intra-class distance so that raw images are transformed from the original
high-dimensional space to the low-dimensional feature space that is more
conducive to clustering. Furthermore, the related learning process is treated
as a joint optimization problem of deep feature extraction and clustering.
Experimental results demonstrate the intense competitiveness of the proposed
algorithm in comparison with state-of-the-art clustering methods of
hyperspectral images.
A Novel GAN-based Fault Diagnosis Approach for Imbalanced Industrial Time Series
Authors: Wenqian Jiang, Cheng Cheng, Beitong Zhou, Guijun Ma, Ye Yuan
Date: 2019-04-01 00:00:00
Tasks: Decoder, Fault Diagnosis, Time Series, Time Series Analysis

This paper proposes a novel fault diagnosis approach based on generative
adversarial networks (GAN) for imbalanced industrial time series where normal
samples are much larger than failure cases. We combine a well-designed feature
extractor with GAN to help train the whole network. Aimed at obtaining data
distribution and hidden pattern in both original distinguishing features and
latent space, the encoder-decoder-encoder three-sub-network is employed in GAN,
based on Deep Convolution Generative Adversarial Networks (DCGAN) but without
Tanh activation layer and only trained on normal samples. In order to verify
the validity and feasibility of our approach, we test it on rolling bearing
data from Case Western Reserve University and further verify it on data
collected from our laboratory. The results show that our proposed approach can
achieve excellent performance in detecting faulty by outputting much larger
evaluation scores.
Adaptive Bayesian Linear Regression for Automated Machine Learning
Authors: Weilin Zhou, Frederic Precioso
Date: 2019-04-01 00:00:00
Tasks: AutoML, Bayesian Optimization, BIG-bench Machine Learning, Hyperparameter Optimization, Meta-Learning, Model Selection, regression

To solve a machine learning problem, one typically needs to perform data
preprocessing, modeling, and hyperparameter tuning, which is known as model
selection and hyperparameter optimization.The goal of automated machine
learning (AutoML) is to design methods that can automatically perform model
selection and hyperparameter optimization without human interventions for a
given dataset. In this paper, we propose a meta-learning method that can search
for a high-performance machine learning pipeline from the predefined set of
candidate pipelines for supervised classification datasets in an efficient way
by leveraging meta-data collected from previous experiments. More specifically,
our method combines an adaptive Bayesian regression model with a neural network
basis function and the acquisition function from Bayesian optimization. The
adaptive Bayesian regression model is able to capture knowledge from previous
meta-data and thus make predictions of the performances of machine learning
pipelines on a new dataset. The acquisition function is then used to guide the
search of possible pipelines based on the predictions.The experiments
demonstrate that our approach can quickly identify high-performance pipelines
for a range of test datasets and outperforms the baseline methods.
Sound source ranging using a feed-forward neural network with fitting-based early stopping
Authors: Jing Chi, Xiaolei Li, Haozhong Wang, Dazhi Gao, Peter Gerstoft
Date: 2019-04-01 00:00:00

When a feed-forward neural network (FNN) is trained for source ranging in an
ocean waveguide, it is difficult evaluating the range accuracy of the FNN on
unlabeled test data. A fitting-based early stopping (FEAST) method is
introduced to evaluate the range error of the FNN on test data where the
distance of source is unknown. Based on FEAST, when the evaluated range error
of the FNN reaches the minimum on test data, stopping training, which will help
to improve the ranging accuracy of the FNN on the test data. The FEAST is
demonstrated on simulated and experimental data.
Transfer Learning for Clinical Time Series Analysis using Deep Neural Networks
Authors: Priyanka Gupta, Pankaj Malhotra, Jyoti Narwariya, Lovekesh Vig, Gautam Shroff
Date: 2019-04-01 00:00:00
Tasks: Domain Adaptation, Time Series, Time Series Analysis, Transfer Learning

Deep neural networks have shown promising results for various clinical prediction tasks. However, training deep networks such as those based on Recurrent Neural Networks (RNNs) requires large labeled data, significant hyper-parameter tuning effort and expertise, and high computational resources. In this work, we investigate as to what extent can transfer learning address these issues when using deep RNNs to model multivariate clinical time series. We consider two scenarios for transfer learning using RNNs: i) domain-adaptation, i.e., leveraging a deep RNN - namely, TimeNet - pre-trained for feature extraction on time series from diverse domains, and adapting it for feature extraction and subsequent target tasks in healthcare domain, ii) task-adaptation, i.e., pre-training a deep RNN - namely, HealthNet - on diverse tasks in healthcare domain, and adapting it to new target tasks in the same domain. We evaluate the above approaches on publicly available MIMIC-III benchmark dataset, and demonstrate that (a) computationally-efficient linear models trained using features extracted via pre-trained RNNs outperform or, in the worst case, perform as well as deep RNNs and statistical hand-crafted features based models trained specifically for target task; (b) models obtained by adapting pre-trained models for target tasks are significantly more robust to the size of labeled data compared to task-specific RNNs, while also being computationally efficient. We, therefore, conclude that pre-trained deep models like TimeNet and HealthNet allow leveraging the advantages of deep learning for clinical time series analysis tasks, while also minimize dependence on hand-crafted features, deal robustly with scarce labeled training data scenarios without overfitting, as well as reduce dependence on expertise and resources required to train deep networks from scratch.
On the Power and Limitations of Random Features for Understanding Neural Networks
Authors: Gilad Yehudai, Ohad Shamir
Date: 2019-04-01 00:00:00
Proceeding: NeurIPS 2019 12

Recently, a spate of papers have provided positive theoretical results for training over-parameterized neural networks (where the network size is larger than what is needed to achieve low error). The key insight is that with sufficient over-parameterization, gradient-based methods will implicitly leave some components of the network relatively unchanged, so the optimization dynamics will behave as if those components are essentially fixed at their initial random values. In fact, fixing these explicitly leads to the well-known approach of learning with random features. In other words, these techniques imply that we can successfully learn with neural networks, whenever we can successfully learn with random features. In this paper, we first review these techniques, providing a simple and self-contained analysis for one-hidden-layer networks. We then argue that despite the impressive positive results, random feature approaches are also inherently limited in what they can explain. In particular, we rigorously show that random features cannot be used to learn even a single ReLU neuron with standard Gaussian inputs, unless the network size (or magnitude of the weights) is exponentially large. Since a single neuron is learnable with gradient-based methods, we conclude that we are still far from a satisfying general explanation for the empirical success of neural networks.
Defending against adversarial attacks by randomized diversification
Authors: Olga Taran, Shideh Rezaeifar, Taras Holotyak, Slava Voloshynovskiy
Date: 2019-04-01 00:00:00
Proceeding: CVPR 2019 6

The vulnerability of machine learning systems to adversarial attacks
questions their usage in many applications. In this paper, we propose a
randomized diversification as a defense strategy. We introduce a multi-channel
architecture in a gray-box scenario, which assumes that the architecture of the
classifier and the training data set are known to the attacker. The attacker
does not only have access to a secret key and to the internal states of the
system at the test time. The defender processes an input in multiple channels.
Each channel introduces its own randomization in a special transform domain
based on a secret key shared between the training and testing stages. Such a
transform based randomization with a shared key preserves the gradients in
key-defined sub-spaces for the defender but it prevents gradient back
propagation and the creation of various bypass systems for the attacker. An
additional benefit of multi-channel randomization is the aggregation that fuses
soft-outputs from all channels, thus increasing the reliability of the final
score. The sharing of a secret key creates an information advantage to the
defender. Experimental evaluation demonstrates an increased robustness of the
proposed method to a number of known state-of-the-art attacks.
DSL: Discriminative Subgraph Learning via Sparse Self-Representation
Authors: Lin Zhang, Petko Bogdanov
Date: 2019-03-24 00:00:00
Tasks: feature selection

The goal in network state prediction (NSP) is to classify the global state
(label) associated with features embedded in a graph. This graph structure
encoding feature relationships is the key distinctive aspect of NSP compared to
classical supervised learning. NSP arises in various applications: gene
expression samples embedded in a protein-protein interaction (PPI) network,
temporal snapshots of infrastructure or sensor networks, and fMRI coherence
network samples from multiple subjects to name a few. Instances from these
domains are typically ``wide'' (more features than samples), and thus, feature
sub-selection is required for robust and generalizable prediction. How to best
employ the network structure in order to learn succinct connected subgraphs
encompassing the most discriminative features becomes a central challenge in
NSP. Prior work employs connected subgraph sampling or graph smoothing within
optimization frameworks, resulting in either large variance of quality or weak
control over the connectivity of selected subgraphs.
  In this work we propose an optimization framework for discriminative subgraph
learning (DSL) which simultaneously enforces (i) sparsity, (ii) connectivity
and (iii) high discriminative power of the resulting subgraphs of features. Our
optimization algorithm is a single-step solution for the NSP and the associated
feature selection problem. It is rooted in the rich literature on
maximal-margin optimization, spectral graph methods and sparse subspace
self-representation. DSL simultaneously ensures solution interpretability and
superior predictive power (up to 16% improvement in challenging instances
compared to baselines), with execution times up to an hour for large instances.
Tree Search Network for Sparse Regression
Authors: Kyung-Su Kim, Sae-Young Chung
Date: 2019-04-01 00:00:00
Tasks: regression

We consider the classical sparse regression problem of recovering a sparse
signal $x_0$ given a measurement vector $y = \Phi x_0+w$. We propose a tree
search algorithm driven by the deep neural network for sparse regression (TSN).
TSN improves the signal reconstruction performance of the deep neural network
designed for sparse regression by performing a tree search with pruning. It is
observed in both noiseless and noisy cases, TSN recovers synthetic and real
signals with lower complexity than a conventional tree search and is superior
to existing algorithms by a large margin for various types of the sensing
matrix $\Phi$, widely used in sparse regression.
STYLE-ANALYZER: fixing code style inconsistencies with interpretable unsupervised algorithms
Authors: Vadim Markovtsev, Waren Long, Hugo Mougard, Konstantin Slavnov, Egor Bulychev
Date: 2019-04-01 00:00:00

Source code reviews are manual, time-consuming, and expensive. Human
involvement should be focused on analyzing the most relevant aspects of the
program, such as logic and maintainability, rather than amending style, syntax,
or formatting defects. Some tools with linting capabilities can format code
automatically and report various stylistic violations for supported programming
languages. They are based on rules written by domain experts, hence, their
configuration is often tedious, and it is impractical for the given set of
rules to cover all possible corner cases. Some machine learning-based solutions
exist, but they remain uninterpretable black boxes. This paper introduces
STYLE-ANALYZER, a new open source tool to automatically fix code formatting
violations using the decision tree forest model which adapts to each codebase
and is fully unsupervised. STYLE-ANALYZER is built on top of our novel assisted
code review framework, Lookout. It accurately mines the formatting style of
each analyzed Git repository and expresses the found format patterns with
compact human-readable rules. STYLE-ANALYZER can then suggest style
inconsistency fixes in the form of code review comments. We evaluate the output
quality and practical relevance of STYLE-ANALYZER by demonstrating that it can
reproduce the original style with high precision, measured on 19 popular
JavaScript projects, and by showing that it yields promising results in fixing
real style mistakes. STYLE-ANALYZER includes a web application to visualize how
the rules are triggered. We release STYLE-ANALYZER as a reusable and extendable
open source software package on GitHub for the benefit of the community.
LUTNet: Rethinking Inference in FPGA Soft Logic
Authors: Erwei Wang, James J. Davis, Peter Y. K. Cheung, George A. Constantinides
Date: 2019-04-01 00:00:00

Research has shown that deep neural networks contain significant redundancy,
and that high classification accuracies can be achieved even when weights and
activations are quantised down to binary values. Network binarisation on FPGAs
greatly increases area efficiency by replacing resource-hungry multipliers with
lightweight XNOR gates. However, an FPGA's fundamental building block, the
K-LUT, is capable of implementing far more than an XNOR: it can perform any
K-input Boolean operation. Inspired by this observation, we propose LUTNet, an
end-to-end hardware-software framework for the construction of area-efficient
FPGA-based neural network accelerators using the native LUTs as inference
operators. We demonstrate that the exploitation of LUT flexibility allows for
far heavier pruning than possible in prior works, resulting in significant area
savings while achieving comparable accuracy. Against the state-of-the-art
binarised neural network implementation, we achieve twice the area efficiency
for several standard network models when inferencing popular datasets. We also
demonstrate that even greater energy efficiency improvements are obtainable.
Controlling for Biasing Signals in Images for Prognostic Models: Survival Predictions for Lung Cancer with Deep Learning
Authors: Wouter A. C. van Amsterdam, Marinus J. C. Eijkemans
Date: 2019-04-01 00:00:00
Tasks: Deep Learning, Prognosis

Deep learning has shown remarkable results for image analysis and is expected
to aid individual treatment decisions in health care. To achieve this, deep
learning methods need to be promoted from the level of mere associations to
being able to answer causal questions. We present a scenario with real-world
medical images (CT-scans of lung cancers) and simulated outcome data. Through
the sampling scheme, the images contain two distinct factors of variation that
represent a collider and a prognostic factor. We show that when this collider
can be quantified, unbiased individual prognosis predictions are attainable
with deep learning. This is achieved by (1) setting a dual task for the network
to predict both the outcome and the collider and (2) enforcing independence of
the activation distributions of the last layer with ordinary least squares. Our
method provides an example of combining deep learning and structural causal
models for unbiased individual prognosis predictions.
Engineering problems in machine learning systems
Authors: Hiroshi Kuwajima, Hirotoshi Yasuoka, Toshihiro Nakae
Date: 2019-04-01 00:00:00
Tasks: BIG-bench Machine Learning

Fatal accidents are a major issue hindering the wide acceptance of safety-critical systems that employ machine learning and deep learning models, such as automated driving vehicles. In order to use machine learning in a safety-critical system, it is necessary to demonstrate the safety and security of the system through engineering processes. However, thus far, no such widely accepted engineering concepts or frameworks have been established for these systems. The key to using a machine learning model in a deductively engineered system is decomposing the data-driven training of machine learning models into requirement, design, and verification, particularly for machine learning models used in safety-critical systems. Simultaneously, open problems and relevant technical fields are not organized in a manner that enables researchers to select a theme and work on it. In this study, we identify, classify, and explore the open problems in engineering (safety-critical) machine learning systems --- that is, in terms of requirement, design, and verification of machine learning models and systems --- as well as discuss related works and research directions, using automated driving vehicles as an example. Our results show that machine learning models are characterized by a lack of requirements specification, lack of design specification, lack of interpretability, and lack of robustness. We also perform a gap analysis on a conventional system quality standard SQuARE with the characteristics of machine learning models to study quality models for machine learning systems. We find that a lack of requirements specification and lack of robustness have the greatest impact on conventional quality models.
RAPID: Early Classification of Explosive Transients using Deep Learning
Authors: Daniel Muthukrishna, Gautham Narayan, Kaisey S. Mandel, Rahul Biswas, Renée Hložek
Date: 2019-03-29 00:00:00
Tasks: Classification, Deep Learning, Early  Classification, General Classification, Time Series, Time Series Analysis, Time Series Classification

We present RAPID (Real-time Automated Photometric IDentification), a novel time-series classification tool capable of automatically identifying transients from within a day of the initial alert, to the full lifetime of a light curve. Using a deep recurrent neural network with Gated Recurrent Units (GRUs), we present the first method specifically designed to provide early classifications of astronomical time-series data, typing 12 different transient classes. Our classifier can process light curves with any phase coverage, and it does not rely on deriving computationally expensive features from the data, making RAPID well-suited for processing the millions of alerts that ongoing and upcoming wide-field surveys such as the Zwicky Transient Facility (ZTF), and the Large Synoptic Survey Telescope (LSST) will produce. The classification accuracy improves over the lifetime of the transient as more photometric data becomes available, and across the 12 transient classes, we obtain an average area under the receiver operating characteristic curve of 0.95 and 0.98 at early and late epochs, respectively. We demonstrate RAPID's ability to effectively provide early classifications of observed transients from the ZTF data stream. We have made RAPID available as an open-source software package (https://astrorapid.readthedocs.io) for machine learning-based alert-brokers to use for the autonomous and quick classification of several thousand light curves within a few seconds.