pwc-archive/papers-with-abstracts huggingface dataset:
An approach to predictively securing critical cloud infrastructures through probabilistic modeling
Authors: Satvik Jain, Arun Balaji Buduru, Anshuman Chhabra
Date: 2018-10-29 00:00:00
Tasks: Management

Cloud infrastructures are being increasingly utilized in critical
infrastructures such as banking/finance, transportation and utility management.
Sophistication and resources used in recent security breaches including those
on critical infrastructures show that attackers are no longer limited by
monetary/computational constraints. In fact, they may be aided by entities with
large financial and human resources. Hence there is urgent need to develop
predictive approaches for cyber defense to strengthen cloud infrastructures
specifically utilized by critical infrastructures. Extensive research has been
done in the past on applying techniques such as Game Theory, Machine Learning
and Bayesian Networks among others for the predictive defense of critical
infrastructures. However a major drawback of these approaches is that they do
not incorporate probabilistic human behavior which limits their predictive
ability. In this paper, a stochastic approach is proposed to predict less
secure states in critical cloud systems which might lead to potential security
breaches. These less-secure states are deemed as `risky' states in our
approach. Markov Decision Process (MDP) is used to accurately incorporate user
behavior(s) as well as operational behavior of the cloud infrastructure through
a set of features. The developed reward/cost mechanism is then used to select
appropriate `actions' to identify risky states at future time steps by learning
an optimal policy. Experimental results show that the proposed framework
performs well in identifying future `risky' states. Through this work we
demonstrate the effectiveness of using probabilistic modeling (MDP) to
predictively secure critical cloud infrastructures.
The Expressive Power of Parameterized Quantum Circuits
Authors: Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, DaCheng Tao
Date: 2018-10-29 00:00:00
Tasks: Tensor Networks

Parameterized quantum circuits (PQCs) have been broadly used as a hybrid
quantum-classical machine learning scheme to accomplish generative tasks.
However, whether PQCs have better expressive power than classical generative
neural networks, such as restricted or deep Boltzmann machines, remains an open
issue. In this paper, we prove that PQCs with a simple structure already
outperform any classical neural network for generative tasks, unless the
polynomial hierarchy collapses. Our proof builds on known results from tensor
networks and quantum circuits (in particular, instantaneous quantum polynomial
circuits). In addition, PQCs equipped with ancillary qubits for post-selection
have even stronger expressive power than those without post-selection. We
employ them as an application for Bayesian learning, since it is possible to
learn prior probabilities rather than assuming they are known. We expect that
it will find many more applications in semi-supervised learning where prior
distributions are normally assumed to be unknown. Lastly, we conduct several
numerical experiments using the Rigetti Forest platform to demonstrate the
performance of the proposed Bayesian quantum circuit.
AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks
Authors: Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, Jian Tang
Date: 2018-10-29 00:00:00
Tasks: Click-Through Rate Prediction, Recommendation Systems

Click-through rate (CTR) prediction, which aims to predict the probability of a user clicking on an ad or an item, is critical to many online applications such as online advertising and recommender systems. The problem is very challenging since (1) the input features (e.g., the user id, user age, item id, item category) are usually sparse and high-dimensional, and (2) an effective prediction relies on high-order combinatorial features (\textit{a.k.a.} cross features), which are very time-consuming to hand-craft by domain experts and are impossible to be enumerated. Therefore, there have been efforts in finding low-dimensional representations of the sparse and high-dimensional raw features and their meaningful combinations. In this paper, we propose an effective and efficient method called the \emph{AutoInt} to automatically learn the high-order feature interactions of input features. Our proposed algorithm is very general, which can be applied to both numerical and categorical input features. Specifically, we map both the numerical and categorical features into the same low-dimensional space. Afterwards, a multi-head self-attentive neural network with residual connections is proposed to explicitly model the feature interactions in the low-dimensional space. With different layers of the multi-head self-attentive neural networks, different orders of feature combinations of input features can be modeled. The whole model can be efficiently fit on large-scale raw data in an end-to-end fashion. Experimental results on four real-world datasets show that our proposed approach not only outperforms existing state-of-the-art approaches for prediction but also offers good explainability. Code is available at: \url{https://github.com/DeepGraphLearning/RecommenderSystems}.
Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language
Authors: Seonghyeon Nam, Yunji Kim, Seon Joo Kim
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Generative Adversarial Network

This paper addresses the problem of manipulating images using natural
language description. Our task aims to semantically modify visual attributes of
an object in an image according to the text describing the new visual
appearance. Although existing methods synthesize images having new attributes,
they do not fully preserve text-irrelevant contents of the original image. In
this paper, we propose the text-adaptive generative adversarial network (TAGAN)
to generate semantically manipulated images while preserving text-irrelevant
contents. The key to our method is the text-adaptive discriminator that creates
word-level local discriminators according to input text to classify
fine-grained attributes independently. With this discriminator, the generator
learns to generate images where only regions that correspond to the given text
are modified. Experimental results show that our method outperforms existing
methods on CUB and Oxford-102 datasets, and our results were mostly preferred
on a user study. Extensive analysis shows that our method is able to
effectively disentangle visual attributes and produce pleasing outputs.
Rademacher Complexity for Adversarially Robust Generalization
Authors: Dong Yin, Kannan Ramchandran, Peter Bartlett
Date: 2018-10-29 00:00:00
Tasks: BIG-bench Machine Learning

Many machine learning models are vulnerable to adversarial attacks; for example, adding adversarial perturbations that are imperceptible to humans can often make machine learning models produce wrong predictions with high confidence. Moreover, although we may obtain robust models on the training dataset via adversarial training, in some problems the learned models cannot generalize well to the test data. In this paper, we focus on $\ell_\infty$ attacks, and study the adversarially robust generalization problem through the lens of Rademacher complexity. For binary linear classifiers, we prove tight bounds for the adversarial Rademacher complexity, and show that the adversarial Rademacher complexity is never smaller than its natural counterpart, and it has an unavoidable dimension dependence, unless the weight vector has bounded $\ell_1$ norm. The results also extend to multi-class linear classifiers. For (nonlinear) neural networks, we show that the dimension dependence in the adversarial Rademacher complexity also exists. We further consider a surrogate adversarial loss for one-hidden layer ReLU network and prove margin bounds for this setting. Our results indicate that having $\ell_1$ norm constraints on the weight matrices might be a potential way to improve generalization in the adversarial setting. We demonstrate experimental results that validate our theoretical findings.
Probabilistic Multilevel Clustering via Composite Transportation Distance
Authors: Nhat Ho, Viet Huynh, Dinh Phung, Michael. I. Jordan
Date: 2018-10-29 00:00:00
Tasks: Clustering

We propose a novel probabilistic approach to multilevel clustering problems
based on composite transportation distance, which is a variant of
transportation distance where the underlying metric is Kullback-Leibler
divergence. Our method involves solving a joint optimization problem over
spaces of probability measures to simultaneously discover grouping structures
within groups and among groups. By exploiting the connection of our method to
the problem of finding composite transportation barycenters, we develop fast
and efficient optimization algorithms even for potentially large-scale
multilevel datasets. Finally, we present experimental results with both
synthetic and real data to demonstrate the efficiency and scalability of the
proposed approach.
Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference
Authors: Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, Gerald Tesauro
Date: 2018-10-29 00:00:00
Proceeding: ICLR 2019 5
Tasks: Continual Learning, Meta-Learning, Reinforcement Learning

Lack of performance when it comes to continual learning over non-stationary distributions of data remains a major challenge in scaling neural network learning to more human realistic settings. In this work we propose a new conceptualization of the continual learning problem in terms of a temporally symmetric trade-off between transfer and interference that can be optimized by enforcing gradient alignment across examples. We then propose a new algorithm, Meta-Experience Replay (MER), that directly exploits this view by combining experience replay with optimization based meta-learning. This method learns parameters that make interference based on future gradients less likely and transfer based on future gradients more likely. We conduct experiments across continual lifelong supervised learning benchmarks and non-stationary reinforcement learning environments demonstrating that our approach consistently outperforms recently proposed baselines for continual learning. Our experiments show that the gap between the performance of MER and baseline algorithms grows both as the environment gets more non-stationary and as the fraction of the total experiences stored gets smaller.
Mean-field theory of graph neural networks in graph partitioning
Authors: Tatsuro Kawamoto, Masashi Tsubaki, Tomoyuki Obuchi
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Bayesian Inference, General Classification, Graph Neural Network, graph partitioning

A theoretical performance analysis of the graph neural network (GNN) is
presented. For classification tasks, the neural network approach has the
advantage in terms of flexibility that it can be employed in a data-driven
manner, whereas Bayesian inference requires the assumption of a specific model.
A fundamental question is then whether GNN has a high accuracy in addition to
this flexibility. Moreover, whether the achieved performance is predominately a
result of the backpropagation or the architecture itself is a matter of
considerable interest. To gain a better insight into these questions, a
mean-field theory of a minimal GNN architecture is developed for the graph
partitioning problem. This demonstrates a good agreement with numerical
experiments.
Semi-Supervised Translation with MMD Networks
Authors: Mark Hamilton
Date: 2018-10-28 00:00:00
Tasks: Single Image Deraining, Translation

This work aims to improve semi-supervised learning in a neural network
architecture by introducing a hybrid supervised and unsupervised cost function.
The unsupervised component is trained using a differentiable estimator of the
Maximum Mean Discrepancy (MMD) distance between the network output and the
target dataset. We introduce the notion of an $n$-channel network and several
methods to improve performance of these nets based on supervised
pre-initialization, and multi-scale kernels. This work investigates the
effectiveness of these methods on language translation where very few quality
translations are known \textit{a priori}. We also present a thorough
investigation of the hyper-parameter space of this method on both synthetic
data.
Sparse Logistic Regression Learns All Discrete Pairwise Graphical Models
Authors: Shanshan Wu, Sujay Sanghavi, Alexandros G. Dimakis
Date: 2018-10-28 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: All, regression

We characterize the effectiveness of a classical algorithm for recovering the Markov graph of a general discrete pairwise graphical model from i.i.d. samples. The algorithm is (appropriately regularized) maximum conditional log-likelihood, which involves solving a convex program for each node; for Ising models this is $\ell_1$-constrained logistic regression, while for more general alphabets an $\ell_{2,1}$ group-norm constraint needs to be used. We show that this algorithm can recover any arbitrary discrete pairwise graphical model, and also characterize its sample complexity as a function of model width, alphabet size, edge parameter accuracy, and the number of variables. We show that along every one of these axes, it matches or improves on all existing results and algorithms for this problem. Our analysis applies a sharp generalization error bound for logistic regression when the weight vector has an $\ell_1$ constraint (or $\ell_{2,1}$ constraint) and the sample vector has an $\ell_{\infty}$ constraint (or $\ell_{2, \infty}$ constraint). We also show that the proposed convex programs can be efficiently solved in $\tilde{O}(n^2)$ running time (where $n$ is the number of variables) under the same statistical guarantees. We provide experimental results to support our analysis.
Accurate, Efficient and Scalable Graph Embedding
Authors: Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, Viktor Prasanna
Date: 2018-10-28 00:00:00
Tasks: Clustering, Graph Embedding, Graph Learning, Graph Sampling, Node Classification

The Graph Convolutional Network (GCN) model and its variants are powerful graph embedding tools for facilitating classification and clustering on graphs. However, a major challenge is to reduce the complexity of layered GCNs and make them parallelizable and scalable on very large graphs -- state-of the art techniques are unable to achieve scalability without losing accuracy and efficiency. In this paper, we propose novel parallelization techniques for graph sampling-based GCNs that achieve superior scalable performance on very large graphs without compromising accuracy. Specifically, our GCN guarantees work-efficient training and produces order of magnitude savings in computation and communication. To scale GCN training on tightly-coupled shared memory systems, we develop parallelization strategies for the key steps in training: For the graph sampling step, we exploit parallelism within and across multiple sampling instances, and devise an efficient data structure for concurrent accesses that provides theoretical guarantee of near-linear speedup with number of processing units. For the feature propagation step within the sampled graph, we improve cache utilization and reduce DRAM communication by data partitioning. We prove that our partitioning strategy is a 2-approximation for minimizing the communication time compared to the optimal strategy. We demonstrate that our parallel graph embedding outperforms state-of-the-art methods in scalability (with respect to number of processors, graph size and GCN model size), efficiency and accuracy on several large datasets. On a 40-core Xeon platform, our parallel training achieves $64\times$ speedup (with AVX) in the sampling step and $25\times$ speedup in the feature propagation step, compared to the serial implementation, resulting in a net speedup of $21\times$.
Smoothed Analysis of Discrete Tensor Decomposition and Assemblies of Neurons
Authors: Nima Anari, Constantinos Daskalakis, Wolfgang Maass, Christos H. Papadimitriou, Amin Saberi, Santosh Vempala
Date: 2018-10-28 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Tensor Decomposition

We analyze linear independence of rank one tensors produced by tensor powers
of randomly perturbed vectors. This enables efficient decomposition of sums of
high-order tensors. Our analysis builds upon [BCMV14] but allows for a wider
range of perturbation models, including discrete ones. We give an application
to recovering assemblies of neurons.
  Assemblies are large sets of neurons representing specific memories or
concepts. The size of the intersection of two assemblies has been shown in
experiments to represent the extent to which these memories co-occur or these
concepts are related; the phenomenon is called association of assemblies. This
suggests that an animal's memory is a complex web of associations, and poses
the problem of recovering this representation from cognitive data. Motivated by
this problem, we study the following more general question: Can we reconstruct
the Venn diagram of a family of sets, given the sizes of their $\ell$-wise
intersections? We show that as long as the family of sets is randomly
perturbed, it is enough for the number of measurements to be polynomially
larger than the number of nonempty regions of the Venn diagram to fully
reconstruct the diagram.
Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training
Authors: Hila Gonen, Yoav Goldberg
Date: 2018-10-28 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition

We focus on the problem of language modeling for code-switched language, in the context of automatic speech recognition (ASR). Language modeling for code-switched language is challenging for (at least) three reasons: (1) lack of available large-scale code-switched data for training; (2) lack of a replicable evaluation setup that is ASR directed yet isolates language modeling performance from the other intricacies of the ASR system; and (3) the reliance on generative modeling. We tackle these three issues: we propose an ASR-motivated evaluation setup which is decoupled from an ASR system and the choice of vocabulary, and provide an evaluation dataset for English-Spanish code-switching. This setup lends itself to a discriminative training approach, which we demonstrate to work better than generative language modeling. Finally, we explore a variety of training protocols and verify the effectiveness of training with large amounts of monolingual data followed by fine-tuning with small amounts of code-switched data, for both the generative and discriminative cases.
An Efficient Implementation of Riemannian Manifold Hamiltonian Monte Carlo for Gaussian Process Models
Authors: Ulrich Paquet, Marco Fraccaro
Date: 2018-10-28 00:00:00

This technical report presents pseudo-code for a Riemannian manifold
Hamiltonian Monte Carlo (RMHMC) method to efficiently simulate samples from
$N$-dimensional posterior distributions $p(x|y)$, where $x \in R^N$ is drawn
from a Gaussian Process (GP) prior, and observations $y_n$ are independent
given $x_n$. Sufficient technical and algorithmic details are provided for the
implementation of RMHMC for distributions arising from GP priors.
Active Learning of Uniformly Accurate Inter-atomic Potentials for Materials Simulation
Authors: Linfeng Zhang, De-Ye Lin, Han Wang, Roberto Car, Weinan E
Date: 2018-10-28 00:00:00
Tasks: Active Learning, BIG-bench Machine Learning

An active learning procedure called Deep Potential Generator (DP-GEN) is
proposed for the construction of accurate and transferable machine
learning-based models of the potential energy surface (PES) for the molecular
modeling of materials. This procedure consists of three main components:
exploration, generation of accurate reference data, and training. Application
to the sample systems of Al, Mg and Al-Mg alloys demonstrates that DP-GEN can
produce uniformly accurate PES models with a minimal number of reference data.
Automatically Evolving CNN Architectures Based on Blocks
Authors: Yanan Sun, Bing Xue, Mengjie Zhang, Gary G. Yen
Date: 2018-10-28 00:00:00
Tasks: General Classification

The performance of Convolutional Neural Networks (CNNs) highly relies on
their architectures. In order to design a CNN with promising performance,
extended expertise in both CNNs and the investigated problem is required, which
is not necessarily held by every user interested in CNNs or the problem domain.
In this paper, we propose to automatically evolve CNN architectures by using a
genetic algorithm based on ResNet blocks and DenseNet blocks. The proposed
algorithm is \textbf{completely} automatic in designing CNN architectures,
particularly, neither pre-processing before it starts nor post-processing on
the designed CNN is needed. Furthermore, the proposed algorithm does not
require users with domain knowledge on CNNs, the investigated problem or even
genetic algorithms. The proposed algorithm is evaluated on CIFAR10 and CIFAR100
against 18 state-of-the-art peer competitors. Experimental results show that it
outperforms state-of-the-art CNNs hand-crafted and CNNs designed by automatic
peer competitors in terms of the classification accuracy, and achieves the
competitive classification accuracy against semi-automatic peer competitors. In
addition, the proposed algorithm consumes much less time than most peer
competitors in finding the best CNN architectures.
Learning with Bad Training Data via Iterative Trimmed Loss Minimization
Authors: Yanyao Shen, Sujay Sanghavi
Date: 2018-10-28 00:00:00

In this paper, we study a simple and generic framework to tackle the problem
of learning model parameters when a fraction of the training samples are
corrupted. We first make a simple observation: in a variety of such settings,
the evolution of training accuracy (as a function of training epochs) is
different for clean and bad samples. Based on this we propose to iteratively
minimize the trimmed loss, by alternating between (a) selecting samples with
lowest current loss, and (b) retraining a model on only these samples. We prove
that this process recovers the ground truth (with linear convergence rate) in
generalized linear models with standard statistical assumptions.
Experimentally, we demonstrate its effectiveness in three settings: (a) deep
image classifiers with errors only in labels, (b) generative adversarial
networks with bad training images, and (c) deep image classifiers with
adversarial (image, label) pairs (i.e., backdoor attacks). For the well-studied
setting of random label noise, our algorithm achieves state-of-the-art
performance without having access to any a-priori guaranteed clean samples.
Sequential anatomy localization in fetal echocardiography videos
Authors: Arijit Patra, J. A. Noble
Date: 2018-10-28 00:00:00
Tasks: Anatomical Landmark Detection, Anatomy, Diagnostic

Fetal heart motion is an important diagnostic indicator for structural
detection and functional assessment of congenital heart disease. We propose an
approach towards integrating deep convolutional and recurrent architectures
that utilize localized spatial and temporal features of different anatomical
substructures within a global spatiotemporal context for interpretation of
fetal echocardiography videos. We formulate our task as a cardiac structure
localization problem with convolutional architectures for aggregating global
spatial context and detecting anatomical structures on spatial region
proposals. This information is aggregated temporally by recurrent architectures
to quantify the progressive motion patterns. We experimentally show that the
resulting architecture combines anatomical landmark detection at the
frame-level over multiple video sequences-with temporal progress of the
associated anatomical motions to encode local spatiotemporal fetal heart
dynamics and is validated on a real-world clinical dataset.