pwc-archive/papers-with-abstracts huggingface dataset:
mQAPViz: A divide-and-conquer multi-objective optimization algorithm to compute large data visualizations
Authors: Claudio Sanhueza, Francia Jiménez, Regina Berretta, Pablo Moscato
Date: 2018-04-02 00:00:00

Algorithms for data visualizations are essential tools for transforming data
into useful narratives. Unfortunately, very few visualization algorithms can
handle the large datasets of many real-world scenarios. In this study, we
address the visualization of these datasets as a Multi-Objective Optimization
Problem. We propose mQAPViz, a divide-and-conquer multi-objective optimization
algorithm to compute large-scale data visualizations. Our method employs the
Multi-Objective Quadratic Assignment Problem (mQAP) as the mathematical
foundation to solve the visualization task at hand. The algorithm applies
advanced sampling techniques originating from the field of machine learning and
efficient data structures to scale to millions of data objects. The algorithm
allocates objects onto a 2D grid layout. Experimental results on real-world and
large datasets demonstrate that mQAPViz is a competitive alternative to
existing techniques.
Convolutional Dictionary Learning: A Comparative Review and New Algorithms
Authors: Cristina Garcia-Cardona, Brendt Wohlberg
Date: 2017-09-09 00:00:00
Tasks: Dictionary Learning

Convolutional sparse representations are a form of sparse representation with
a dictionary that has a structure that is equivalent to convolution with a set
of linear filters. While effective algorithms have recently been developed for
the convolutional sparse coding problem, the corresponding dictionary learning
problem is substantially more challenging. Furthermore, although a number of
different approaches have been proposed, the absence of thorough comparisons
between them makes it difficult to determine which of them represents the
current state of the art. The present work both addresses this deficiency and
proposes some new approaches that outperform existing ones in certain contexts.
A thorough set of performance comparisons indicates a very wide range of
performance differences among the existing and proposed methods, and clearly
identifies those that are the most effective.
Cell Identity Codes: Understanding Cell Identity from Gene Expression Profiles using Deep Neural Networks
Authors: Farzad Abdolhosseini, Behrooz Azarkhalili, Abbas Maazallahi, Aryan Kamal, Seyed Abolfazl Motahari, Ali Sharifi-Zarchi, Hamidreza Chitsaz
Date: 2018-06-13 00:00:00

Understanding cell identity is an important task in many biomedical areas.
Expression patterns of specific marker genes have been used to characterize
some limited cell types, but exclusive markers are not available for many cell
types. A second approach is to use machine learning to discriminate cell types
based on the whole gene expression profiles (GEPs). The accuracies of simple
classification algorithms such as linear discriminators or support vector
machines are limited due to the complexity of biological systems. We used deep
neural networks to analyze 1040 GEPs from 16 different human tissues and cell
types. After comparing different architectures, we identified a specific
structure of deep autoencoders that can encode a GEP into a vector of 30
numeric values, which we call the cell identity code (CIC). The original GEP
can be reproduced from the CIC with an accuracy comparable to technical
replicates of the same experiment. Although we use an unsupervised approach to
train the autoencoder, we show different values of the CIC are connected to
different biological aspects of the cell, such as different pathways or
biological processes. This network can use CIC to reproduce the GEP of the cell
types it has never seen during the training. It also can resist some noise in
the measurement of the GEP. Furthermore, we introduce classifier autoencoder,
an architecture that can accurately identify cell type based on the GEP or the
CIC.
Learning Visual Knowledge Memory Networks for Visual Question Answering
Authors: Zhou Su, Chen Zhu, Yinpeng Dong, Dongqi Cai, Yurong Chen, Jianguo Li
Date: 2018-06-13 00:00:00
Proceeding: CVPR 2018 6
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Visual question answering (VQA) requires joint comprehension of images and
natural language questions, where many questions can't be directly or clearly
answered from visual content but require reasoning from structured human
knowledge with confirmation from visual content. This paper proposes visual
knowledge memory network (VKMN) to address this issue, which seamlessly
incorporates structured human knowledge and deep visual features into memory
networks in an end-to-end learning framework. Comparing to existing methods for
leveraging external knowledge for supporting VQA, this paper stresses more on
two missing mechanisms. First is the mechanism for integrating visual contents
with knowledge facts. VKMN handles this issue by embedding knowledge triples
(subject, relation, target) and deep visual features jointly into the visual
knowledge features. Second is the mechanism for handling multiple knowledge
facts expanding from question and answer pairs. VKMN stores joint embedding
using key-value pair structure in the memory networks so that it is easy to
handle multiple facts. Experiments show that the proposed method achieves
promising results on both VQA v1.0 and v2.0 benchmarks, while outperforms
state-of-the-art methods on the knowledge-reasoning related questions.
DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling
Authors: Burak Demirel, Arunselvan Ramaswamy, Daniel E. Quevedo, Holger Karl
Date: 2018-03-08 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Scheduling

We consider networked control systems consisting of multiple independent
controlled subsystems, operating over a shared communication network. Such
systems are ubiquitous in cyber-physical systems, Internet of Things, and
large-scale industrial systems. In many large-scale settings, the size of the
communication network is smaller than the size of the system. In consequence,
scheduling issues arise. The main contribution of this paper is to develop a
deep reinforcement learning-based \emph{control-aware} scheduling
(\textsc{DeepCAS}) algorithm to tackle these issues. We use the following
(optimal) design strategy: First, we synthesize an optimal controller for each
subsystem; next, we design a learning algorithm that adapts to the chosen
subsystems (plants) and controllers. As a consequence of this adaptation, our
algorithm finds a schedule that minimizes the \emph{control loss}. We present
empirical results to show that \textsc{DeepCAS} finds schedules with better
performance than periodic ones.
Curve Reconstruction via the Global Statistics of Natural Curves
Authors: Ehud Barnea, Ohad Ben-Shahar
Date: 2017-11-08 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Inpainting

Reconstructing the missing parts of a curve has been the subject of much
computational research, with applications in image inpainting, object
synthesis, etc. Different approaches for solving that problem are typically
based on processes that seek visually pleasing or perceptually plausible
completions. In this work we focus on reconstructing the underlying physically
likely shape by utilizing the global statistics of natural curves. More
specifically, we develop a reconstruction model that seeks the mean physical
curve for a given inducer configuration. This simple model is both
straightforward to compute and it is receptive to diverse additional
information, but it requires enough samples for all curve configurations, a
practical requirement that limits its effective utilization. To address this
practical issue we explore and exploit statistical geometrical properties of
natural curves, and in particular, we show that in many cases the mean curve is
scale invariant and oftentimes it is extensible. This, in turn, allows to boost
the number of examples and thus the robustness of the statistics and its
applicability. The reconstruction results are not only more physically
plausible but they also lead to important insights on the reconstruction
problem, including an elegant explanation why certain inducer configurations
are more likely to yield consistent perceptual completions than others.
Double Path Networks for Sequence to Sequence Learning
Authors: Kaitao Song, Xu Tan, Di He, Jianfeng Lu, Tao Qin, Tie-Yan Liu
Date: 2018-06-13 00:00:00
Proceeding: COLING 2018 8
Tasks: Decoder

Encoder-decoder based Sequence to Sequence learning (S2S) has made remarkable
progress in recent years. Different network architectures have been used in the
encoder/decoder. Among them, Convolutional Neural Networks (CNN) and Self
Attention Networks (SAN) are the prominent ones. The two architectures achieve
similar performances but use very different ways to encode and decode context:
CNN use convolutional layers to focus on the local connectivity of the
sequence, while SAN uses self-attention layers to focus on global semantics. In
this work we propose Double Path Networks for Sequence to Sequence learning
(DPN-S2S), which leverage the advantages of both models by using double path
information fusion. During the encoding step, we develop a double path
architecture to maintain the information coming from different paths with
convolutional layers and self-attention layers separately. To effectively use
the encoded context, we develop a cross attention module with gating and use it
to automatically pick up the information needed during the decoding step. By
deeply integrating the two paths with cross attention, both types of
information are combined and well exploited. Experiments show that our proposed
method can significantly improve the performance of sequence to sequence
learning over state-of-the-art systems.
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam
Authors: Mohammad Emtiyaz Khan, Didrik Nielsen, Voot Tangkaratt, Wu Lin, Yarin Gal, Akash Srivastava
Date: 2018-06-13 00:00:00
Proceeding: ICML 2018 7
Tasks: Reinforcement Learning, Stochastic Optimization, Variational Inference

Uncertainty computation in deep learning is essential to design robust and
reliable systems. Variational inference (VI) is a promising approach for such
computation, but requires more effort to implement and execute compared to
maximum-likelihood methods. In this paper, we propose new natural-gradient
algorithms to reduce such efforts for Gaussian mean-field VI. Our algorithms
can be implemented within the Adam optimizer by perturbing the network weights
during gradient evaluations, and uncertainty estimates can be cheaply obtained
by using the vector that adapts the learning rate. This requires lower memory,
computation, and implementation effort than existing VI methods, while
obtaining uncertainty estimates of comparable quality. Our empirical results
confirm this and further suggest that the weight-perturbation in our algorithm
could be useful for exploration in reinforcement learning and stochastic
optimization.
Learning Structural Node Embeddings Via Diffusion Wavelets
Authors: Claire Donnat, Marinka Zitnik, David Hallac, Jure Leskovec
Date: 2017-10-27 00:00:00
Proceeding: KDD 2018 6

Nodes residing in different parts of a graph can have similar structural
roles within their local network topology. The identification of such roles
provides key insight into the organization of networks and can be used for a
variety of machine learning tasks. However, learning structural representations
of nodes is a challenging problem, and it has typically involved manually
specifying and tailoring topological features for each node. In this paper, we
develop GraphWave, a method that represents each node's network neighborhood
via a low-dimensional embedding by leveraging heat wavelet diffusion patterns.
Instead of training on hand-selected features, GraphWave learns these
embeddings in an unsupervised way. We mathematically prove that nodes with
similar network neighborhoods will have similar GraphWave embeddings even
though these nodes may reside in very different parts of the network, and our
method scales linearly with the number of edges. Experiments in a variety of
different settings demonstrate GraphWave's real-world potential for capturing
structural roles in networks, and our approach outperforms existing
state-of-the-art baselines in every experiment, by as much as 137%.
Analyzing the Robustness of Nearest Neighbors to Adversarial Examples
Authors: Yizhen Wang, Somesh Jha, Kamalika Chaudhuri
Date: 2017-06-13 00:00:00
Proceeding: ICML 2018 7

Motivated by safety-critical applications, test-time attacks on classifiers via adversarial examples has recently received a great deal of attention. However, there is a general lack of understanding on why adversarial examples arise; whether they originate due to inherent properties of data or due to lack of training samples remains ill-understood. In this work, we introduce a theoretical framework analogous to bias-variance theory for understanding these effects. We use our framework to analyze the robustness of a canonical non-parametric classifier - the k-nearest neighbors. Our analysis shows that its robustness properties depend critically on the value of k - the classifier may be inherently non-robust for small k, but its robustness approaches that of the Bayes Optimal classifier for fast-growing k. We propose a novel modified 1-nearest neighbor classifier, and guarantee its robustness in the large sample limit. Our experiments suggest that this classifier may have good robustness properties even for reasonable data set sizes.
Polynomial Regression As an Alternative to Neural Nets
Authors: Xi Cheng, Bohdan Khomtchouk, Norman Matloff, Pete Mohanty
Date: 2018-06-13 00:00:00
Tasks: regression

Despite the success of neural networks (NNs), there is still a concern among
many over their "black box" nature. Why do they work? Here we present a simple
analytic argument that NNs are in fact essentially polynomial regression
models. This view will have various implications for NNs, e.g. providing an
explanation for why convergence problems arise in NNs, and it gives rough
guidance on avoiding overfitting. In addition, we use this phenomenon to
predict and confirm a multicollinearity property of NNs not previously reported
in the literature. Most importantly, given this loose correspondence, one may
choose to routinely use polynomial models instead of NNs, thus avoiding some
major problems of the latter, such as having to set many tuning parameters and
dealing with convergence issues. We present a number of empirical results; in
each case, the accuracy of the polynomial approach matches or exceeds that of
NN approaches. A many-featured, open-source software package, polyreg, is
available.
Learning from Mutants: Using Code Mutation to Learn and Monitor Invariants of a Cyber-Physical System
Authors: Yuqi Chen, Christopher M. Poskitt, Jun Sun
Date: 2018-01-03 00:00:00

Cyber-physical systems (CPS) consist of sensors, actuators, and controllers
all communicating over a network; if any subset becomes compromised, an
attacker could cause significant damage. With access to data logs and a model
of the CPS, the physical effects of an attack could potentially be detected
before any damage is done. Manually building a model that is accurate enough in
practice, however, is extremely difficult. In this paper, we propose a novel
approach for constructing models of CPS automatically, by applying supervised
machine learning to data traces obtained after systematically seeding their
software components with faults ("mutants"). We demonstrate the efficacy of
this approach on the simulator of a real-world water purification plant,
presenting a framework that automatically generates mutants, collects data
traces, and learns an SVM-based model. Using cross-validation and statistical
model checking, we show that the learnt model characterises an invariant
physical property of the system. Furthermore, we demonstrate the usefulness of
the invariant by subjecting the system to 55 network and code-modification
attacks, and showing that it can detect 85% of them from the data logs
generated at runtime.
A Study of Enhancement, Augmentation, and Autoencoder Methods for Domain Adaptation in Distant Speech Recognition
Authors: Hao Tang, Wei-Ning Hsu, Francois Grondin, James Glass
Date: 2018-06-13 00:00:00
Tasks: Data Augmentation, Distant Speech Recognition, Domain Adaptation, Speech Enhancement, speech-recognition, Speech Recognition

Speech recognizers trained on close-talking speech do not generalize to
distant speech and the word error rate degradation can be as large as 40%
absolute. Most studies focus on tackling distant speech recognition as a
separate problem, leaving little effort to adapting close-talking speech
recognizers to distant speech. In this work, we review several approaches from
a domain adaptation perspective. These approaches, including speech
enhancement, multi-condition training, data augmentation, and autoencoders, all
involve a transformation of the data between domains. We conduct experiments on
the AMI data set, where these approaches can be realized under the same
controlled setting. These approaches lead to different amounts of improvement
under their respective assumptions. The purpose of this paper is to quantify
and characterize the performance gap between the two domains, setting up the
basis for studying adaptation of speech recognizers from close-talking speech
to distant speech. Our results also have implications for improving distant
speech recognition.
Sequence Prediction with Neural Segmental Models
Authors: Hao Tang
Date: 2017-09-05 00:00:00
Tasks: General Classification, Prediction

Segments that span contiguous parts of inputs, such as phonemes in speech,
named entities in sentences, actions in videos, occur frequently in sequence
prediction problems. Segmental models, a class of models that explicitly
hypothesizes segments, have allowed the exploration of rich segment features
for sequence prediction. However, segmental models suffer from slow decoding,
hampering the use of computationally expensive features.
  In this thesis, we introduce discriminative segmental cascades, a multi-pass
inference framework that allows us to improve accuracy by adding higher-order
features and neural segmental features while maintaining efficiency. We also
show that instead of including more features to obtain better accuracy,
segmental cascades can be used to speed up training and decoding.
  Segmental models, similarly to conventional speech recognizers, are typically
trained in multiple stages. In the first stage, a frame classifier is trained
with manual alignments, and then in the second stage, segmental models are
trained with manual alignments and the out- puts of the frame classifier.
However, obtaining manual alignments are time-consuming and expensive. We
explore end-to-end training for segmental models with various loss functions,
and show how end-to-end training with marginal log loss can eliminate the need
for detailed manual alignments.
  We draw the connections between the marginal log loss and a popular
end-to-end training approach called connectionist temporal classification. We
present a unifying framework for various end-to-end graph search-based models,
such as hidden Markov models, connectionist temporal classification, and
segmental models. Finally, we discuss possible extensions of segmental models
to large-vocabulary sequence prediction tasks.
Partial AUC Maximization via Nonlinear Scoring Functions
Authors: Naonori Ueda, Akinori Fujino
Date: 2018-06-13 00:00:00
Tasks: Anomaly Detection, Binary Classification, Classification, Diagnostic, General Classification

We propose a method for maximizing a partial area under a receiver operating
characteristic (ROC) curve (pAUC) for binary classification tasks. In binary
classification tasks, accuracy is the most commonly used as a measure of
classifier performance. In some applications such as anomaly detection and
diagnostic testing, accuracy is not an appropriate measure since prior
probabilties are often greatly biased. Although in such cases the pAUC has been
utilized as a performance measure, few methods have been proposed for directly
maximizing the pAUC. This optimization is achieved by using a scoring function.
The conventional approach utilizes a linear function as the scoring function.
In contrast we newly introduce nonlinear scoring functions for this purpose.
Specifically, we present two types of nonlinear scoring functions based on
generative models and deep neural networks. We show experimentally that
nonlinear scoring fucntions improve the conventional methods through the
application of a binary classification of real and bogus objects obtained with
the Hyper Suprime-Cam on the Subaru telescope.
3D Pose Estimation for Fine-Grained Object Categories
Authors: Yaming Wang, Xiao Tan, Yi Yang, Xiao Liu, Errui Ding, Feng Zhou, Larry S. Davis
Date: 2018-06-12 00:00:00
Tasks: 3D Pose Estimation, Object, Pose Estimation

Existing object pose estimation datasets are related to generic object types
and there is so far no dataset for fine-grained object categories. In this
work, we introduce a new large dataset to benchmark pose estimation for
fine-grained objects, thanks to the availability of both 2D and 3D fine-grained
data recently. Specifically, we augment two popular fine-grained recognition
datasets (StanfordCars and CompCars) by finding a fine-grained 3D CAD model for
each sub-category and manually annotating each object in images with 3D pose.
We show that, with enough training data, a full perspective model with
continuous parameters can be estimated using 2D appearance information alone.
We achieve this via a framework based on Faster/Mask R-CNN. This goes beyond
previous works on category-level pose estimation, which only estimate
discrete/continuous viewpoint angles or recover rotation matrices often with
the help of key points. Furthermore, with fine-grained 3D models available, we
incorporate a dense 3D representation named as location field into the
CNN-based pose estimation framework to further improve the performance. The new
dataset is available at www.umiacs.umd.edu/~wym/3dpose.html
State Space Representations of Deep Neural Networks
Authors: Michael Hauser, Sean Gunn, Samer Saab Jr, Asok Ray
Date: 2018-06-11 00:00:00

This paper deals with neural networks as dynamical systems governed by
differential or difference equations. It shows that the introduction of skip
connections into network architectures, such as residual networks and dense
networks, turns a system of static equations into a system of dynamical
equations with varying levels of smoothness on the layer-wise transformations.
Closed form solutions for the state space representations of general dense
networks, as well as $k^{th}$ order smooth networks, are found in general
settings. Furthermore, it is shown that imposing $k^{th}$ order smoothness on a
network architecture with $d$-many nodes per layer increases the state space
dimension by a multiple of $k$, and so the effective embedding dimension of the
data manifold is $k \cdot d$-many dimensions. It follows that network
architectures of these types reduce the number of parameters needed to maintain
the same embedding dimension by a factor of $k^2$ when compared to an
equivalent first-order, residual network, significantly motivating the
development of network architectures of these types. Numerical simulations were
run to validate parts of the developed theory.