pwc-archive/papers-with-abstracts huggingface dataset:
A determinantal point process for column subset selection
Authors: Ayoub Belhadji, Rémi Bardenet, Pierre Chainais
Date: 2018-12-23 00:00:00
Tasks: Dimensionality Reduction, feature selection

Dimensionality reduction is a first step of many machine learning pipelines.
Two popular approaches are principal component analysis, which projects onto a
small number of well chosen but non-interpretable directions, and feature
selection, which selects a small number of the original features. Feature
selection can be abstracted as a numerical linear algebra problem called the
column subset selection problem (CSSP). CSSP corresponds to selecting the best
subset of columns of a matrix $X \in \mathbb{R}^{N \times d}$, where
\emph{best} is often meant in the sense of minimizing the approximation error,
i.e., the norm of the residual after projection of $X$ onto the space spanned
by the selected columns. Such an optimization over subsets of $\{1,\dots,d\}$
is usually impractical. One workaround that has been vastly explored is to
resort to polynomial-cost, random subset selection algorithms that favor small
values of this approximation error. We propose such a randomized algorithm,
based on sampling from a projection determinantal point process (DPP), a
repulsive distribution over a fixed number $k$ of indices $\{1,\dots,d\}$ that
favors diversity among the selected columns. We give bounds on the ratio of the
expected approximation error for this DPP over the optimal error of PCA. These
bounds improve over the state-of-the-art bounds of \emph{volume sampling} when
some realistic structural assumptions are satisfied for $X$. Numerical
experiments suggest that our bounds are tight, and that our algorithms have
comparable performance with the \emph{double phase} algorithm, often considered
to be the practical state-of-the-art. Column subset selection with DPPs thus
inherits the best of both worlds: good empirical performance and tight error
bounds.
Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology
Authors: Bastian Rieck, Matteo Togninalli, Christian Bock, Michael Moor, Max Horn, Thomas Gumbsch, Karsten Borgwardt
Date: 2018-12-23 00:00:00
Proceeding: ICLR 2019 5
Tasks: Topological Data Analysis

While many approaches to make neural networks more fathomable have been proposed, they are restricted to interrogating the network with input data. Measures for characterizing and monitoring structural properties, however, have not been developed. In this work, we propose neural persistence, a complexity measure for neural network architectures based on topological data analysis on weighted stratified graphs. To demonstrate the usefulness of our approach, we show that neural persistence reflects best practices developed in the deep learning community such as dropout and batch normalization. Moreover, we derive a neural persistence-based stopping criterion that shortens the training process while achieving comparable accuracies as early stopping based on validation loss.
Detecting British Columbia Coastal Rainfall Patterns by Clustering Gaussian Processes
Authors: Forrest Paton, Paul D. McNicholas
Date: 2018-12-23 00:00:00
Tasks: Clustering, Gaussian Processes, Time Series, Time Series Analysis

Functional data analysis is a statistical framework where data are assumed to follow some functional form. This method of analysis is commonly applied to time series data, where time, measured continuously or in discrete intervals, serves as the location for a function's value. Gaussian processes are a generalization of the multivariate normal distribution to function space and, in this paper, they are used to shed light on coastal rainfall patterns in British Columbia (BC). Specifically, this work addressed the question over how one should carry out an exploratory cluster analysis for the BC, or any similar, coastal rainfall data. An approach is developed for clustering multiple processes observed on a comparable interval, based on how similar their underlying covariance kernel is. This approach provides interesting insights into the BC data, and these insights can be framed in terms of El Ni\~{n}o and La Ni\~{n}a; however, the result is not simply one cluster representing El Ni\~{n}o years and another for La Ni\~{n}a years. From one perspective, the results show that clustering annual rainfall can potentially be used to identify extreme weather patterns.
Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks
Authors: Amanpreet Singh, Tushar Jain, Sainbayar Sukhbaatar
Date: 2018-12-23 00:00:00
Proceeding: ICLR 2019 5
Tasks: Starcraft

Learning when to communicate and doing that effectively is essential in
multi-agent tasks. Recent works show that continuous communication allows
efficient training with back-propagation in multi-agent scenarios, but have
been restricted to fully-cooperative tasks. In this paper, we present
Individualized Controlled Continuous Communication Model (IC3Net) which has
better training efficiency than simple continuous communication model, and can
be applied to semi-cooperative and competitive settings along with the
cooperative settings. IC3Net controls continuous communication with a gating
mechanism and uses individualized rewards foreach agent to gain better
performance and scalability while fixing credit assignment issues. Using
variety of tasks including StarCraft BroodWars explore and combat scenarios, we
show that our network yields improved performance and convergence rates than
the baselines as the scale increases. Our results convey that IC3Net agents
learn when to communicate based on the scenario and profitability.
Enhancing Discrete Choice Models with Representation Learning
Authors: Brian Sifringer, Virginie Lurkin, Alexandre Alahi
Date: 2018-12-23 00:00:00
Tasks: Discrete Choice Models, parameter estimation, Representation Learning

In discrete choice modeling (DCM), model misspecifications may lead to limited predictability and biased parameter estimates. In this paper, we propose a new approach for estimating choice models in which we divide the systematic part of the utility specification into (i) a knowledge-driven part, and (ii) a data-driven one, which learns a new representation from available explanatory variables. Our formulation increases the predictive power of standard DCM without sacrificing their interpretability. We show the effectiveness of our formulation by augmenting the utility specification of the Multinomial Logit (MNL) and the Nested Logit (NL) models with a new non-linear representation arising from a Neural Network (NN), leading to new choice models referred to as the Learning Multinomial Logit (L-MNL) and Learning Nested Logit (L-NL) models. Using multiple publicly available datasets based on revealed and stated preferences, we show that our models outperform the traditional ones, both in terms of predictive performance and accuracy in parameter estimation. All source code of the models are shared to promote open science.
A Multi-Objective Anytime Rule Mining System to Ease Iterative Feedback from Domain Experts
Authors: Tobias Baum, Steffen Herbold, Kurt Schneider
Date: 2018-12-23 00:00:00

Data extracted from software repositories is used intensively in Software
Engineering research, for example, to predict defects in source code. In our
research in this area, with data from open source projects as well as an
industrial partner, we noticed several shortcomings of conventional data mining
approaches for classification problems: (1) Domain experts' acceptance is of
critical importance, and domain experts can provide valuable input, but it is
hard to use this feedback. (2) The evaluation of the model is not a simple
matter of calculating AUC or accuracy. Instead, there are multiple objectives
of varying importance, but their importance cannot be easily quantified.
Furthermore, the performance of the model cannot be evaluated on a per-instance
level in our case, because it shares aspects with the set cover problem. To
overcome these problems, we take a holistic approach and develop a rule mining
system that simplifies iterative feedback from domain experts and can easily
incorporate the domain-specific evaluation needs. A central part of the system
is a novel multi-objective anytime rule mining algorithm. The algorithm is
based on the GRASP-PR meta-heuristic but extends it with ideas from several
other approaches. We successfully applied the system in the industrial context.
In the current article, we focus on the description of the algorithm and the
concepts of the system. We provide an implementation of the system for reuse.
Leveraging Class Similarity to Improve Deep Neural Network Robustness
Authors: Pooran Singh Negi, David Chan, Mohammad Mahoor
Date: 2018-12-23 00:00:00

Traditionally artificial neural networks (ANNs) are trained by minimizing the
cross-entropy between a provided groundtruth delta distribution (encoded as
one-hot vector) and the ANN's predictive softmax distribution. It seems,
however, unacceptable to penalize networks equally for missclassification
between classes. Confusing the class "Automobile" with the class "Truck" should
be penalized less than confusing the class "Automobile" with the class
"Donkey". To avoid such representation issues and learn cleaner classification
boundaries in the network, this paper presents a variation of cross-entropy
loss which depends not only on the sample class but also on a data-driven prior
"class-similarity distribution" across the classes encoded in a matrix form. We
explore learning the class-similarity distribution using a datadriven method
and then show that by training with our modified similarity-driven loss, we
obtain slightly better generalization performance over multiple architectures
and datasets as well as improved performance on noisy testing scenarios.
End-to-end Learning for Graph Decomposition
Authors: Jie Song, Bjoern Andres, Michael Black, Otmar Hilliges, Siyu Tang
Date: 2018-12-23 00:00:00
Proceeding: ICCV 2019 10
Tasks: Clustering, Multi-Person Pose Estimation, Pose Estimation

We propose a novel end-to-end trainable framework for the graph decomposition
problem. The minimum cost multicut problem is first converted to an
unconstrained binary cubic formulation where cycle consistency constraints are
incorporated into the objective function. The new optimization problem can be
viewed as a Conditional Random Field (CRF) in which the random variables are
associated with the binary edge labels of the initial graph and the hard
constraints are introduced in the CRF as high-order potentials. The parameters
of a standard Neural Network and the fully differentiable CRF are optimized in
an end-to-end manner. Furthermore, our method utilizes the cycle constraints as
meta-supervisory signals during the learning of the deep feature
representations by taking the dependencies between the output random variables
into account. We present analyses of the end-to-end learned representations,
showing the impact of the joint training, on the task of clustering images of
MNIST. We also validate the effectiveness of our approach both for the feature
learning and the final clustering on the challenging task of real-world
multi-person pose estimation.
Parallelized Interactive Machine Learning on Autonomous Vehicles
Authors: Xi Chen, Caylin Hickey
Date: 2018-12-23 00:00:00
Tasks: Autonomous Vehicles, BIG-bench Machine Learning, Deep Reinforcement Learning, Reinforcement Learning

Deep reinforcement learning (deep RL) has achieved superior performance in
complex sequential tasks by learning directly from image input. A deep neural
network is used as a function approximator and requires no specific state
information. However, one drawback of using only images as input is that this
approach requires a prohibitively large amount of training time and data for
the model to learn the state feature representation and approach reasonable
performance. This is not feasible in real-world applications, especially when
the data are expansive and training phase could introduce disasters that affect
human safety. In this work, we use a human demonstration approach to speed up
training for learning features and use the resulting pre-trained model to
replace the neural network in the deep RL Deep Q-Network (DQN), followed by
human interaction to further refine the model. We empirically evaluate our
approach by using only a human demonstration model and modified DQN with human
demonstration model included in the Microsoft AirSim car simulator. Our results
show that (1) pre-training with human demonstration in a supervised learning
approach is better and much faster at discovering features than DQN alone, (2)
initializing the DQN with a pre-trained model provides a significant
improvement in training time and performance even with limited human
demonstration, and (3) providing the ability for humans to supply suggestions
during DQN training can speed up the network's convergence on an optimal
policy, as well as allow it to learn more complex policies that are harder to
discover by random exploration.
Optimizing Answer Set Computation via Heuristic-Based Decomposition
Authors: Francesco Calimeri, Simona Perri, Jessica Zangari
Date: 2018-12-23 00:00:00
Tasks: Tree Decomposition

Answer Set Programming (ASP) is a purely declarative formalism developed in
the field of logic programming and nonmonotonic reasoning: computational
problems are encoded by logic programs whose answer sets, corresponding to
solutions, are computed by an ASP system. Different, semantically equivalent,
programs can be defined for the same problem; however, performance of systems
evaluating them might significantly vary. We propose an approach for
automatically transforming an input logic program into an equivalent one that
can be evaluated more efficiently. One can make use of existing
tree-decomposition techniques for rewriting selected rules into a set of
multiple ones; the idea is to guide and adaptively apply them on the basis of
proper new heuristics, to obtain a smart rewriting algorithm to be integrated
into an ASP system. The method is rather general: it can be adapted to any
system and implement different preference policies. Furthermore, we define a
set of new heuristics tailored at optimizing grounding, one of the main phases
of the ASP computation; we use them in order to implement the approach into the
ASP system DLV, in particular into its grounding subsystem I-DLV, and carry out
an extensive experimental activity for assessing the impact of the proposal.
Under consideration in Theory and Practice of Logic Programming (TPLP).
Increasing the adversarial robustness and explainability of capsule networks with $γ$-capsules
Authors: David Peer, Sebastian Stabinger, Antonio Rodriguez-Sanchez
Date: 2018-12-23 00:00:00
Tasks: Adversarial Robustness, Inductive Bias

In this paper we introduce a new inductive bias for capsule networks and call networks that use this prior $\gamma$-capsule networks. Our inductive bias that is inspired by TE neurons of the inferior temporal cortex increases the adversarial robustness and the explainability of capsule networks. A theoretical framework with formal definitions of $\gamma$-capsule networks and metrics for evaluation are also provided. Under our framework we show that common capsule networks do not necessarily make use of this inductive bias. For this reason we introduce a novel routing algorithm and use a different training algorithm to be able to implement $\gamma$-capsule networks. We then show experimentally that $\gamma$-capsule networks are indeed more transparent and more robust against adversarial attacks than regular capsule networks.
Advanced Image Processing for Astronomical Images
Authors: Diganta Misra, Sparsha Mishra, Bhargav Appasani
Date: 2018-12-23 00:00:00
Tasks: Astronomy

Image Processing in Astronomy is a major field of research and involves a lot
of techniques pertaining to improve analyzing the properties of the celestial
objects or obtaining preliminary inference from the image data. In this paper,
we provide a comprehensive case study of advanced image processing techniques
applied to Astronomical Galaxy Images for improved analysis, accurate
inferences and faster analysis.
Nonlinear Robust Filtering of Sampled-Data Dynamical Systems
Authors: Masoud Abbaszadeh, Horacio J. Marquez
Date: 2018-12-23 00:00:00

This work is concerned with robust filtering of nonlinear sampled-data
systems with and without exact discrete-time models. A linear matrix inequality
(LMI) based approach is proposed for the design of robust $H_{\infty}$
observers for a class of Lipschitz nonlinear systems. Two type of systems are
considered, Lipschitz nonlinear discrete-time systems and Lipschitz nonlinear
sampled-data systems with Euler approximate discrete-time models. Observer
convergence when the exact discrete-time model of the system is available is
shown. Then, practical convergence of the proposed observer is proved using the
Euler approximate discrete-time model. As an additional feature, maximizing the
admissible Lipschitz constant, the solution of the proposed LMI optimization
problem guaranties robustness against some nonlinear uncertainty. The robust
H_infty observer synthesis problem is solved for both cases. The maximum
disturbance attenuation level is achieved through LMI optimization. At the end,
a path to extending the results to higher-order approximate discretizations is
provided.
Image Processing on IOPA Radiographs: A comprehensive case study on Apical Periodontitis
Authors: Diganta Misra, Vanshika Arora
Date: 2018-12-23 00:00:00
Tasks: Diagnostic, Medical Diagnosis

With the recent advancements in Image Processing Techniques and development
of new robust computer vision algorithms, new areas of research within Medical
Diagnosis and Biomedical Engineering are picking up pace. This paper provides a
comprehensive in-depth case study of Image Processing, Feature Extraction and
Analysis of Apical Periodontitis diagnostic cases in IOPA (Intra Oral
Peri-Apical) Radiographs, a common case in oral diagnostic pipeline. This paper
provides a detailed analytical approach towards improving the diagnostic
procedure with improved and faster results with higher accuracy targeting to
eliminate True Negative and False Positive cases.
Computations in Stochastic Acceptors
Authors: Karl-Heinz Zimmermann
Date: 2018-12-23 00:00:00
Tasks: BIG-bench Machine Learning, parameter estimation

Machine learning provides algorithms that can learn from data and make
inferences or predictions on data. Stochastic acceptors or probabilistic
automata are stochastic automata without output that can model components in
machine learning scenarios. In this paper, we provide dynamic programming
algorithms for the computation of input marginals and the acceptance
probabilities in stochastic acceptors. Furthermore, we specify an algorithm for
the parameter estimation of the conditional probabilities using the
expectation-maximization technique and a more efficient implementation related
to the Baum-Welch algorithm.
Scene Graph Reasoning with Prior Visual Relationship for Visual Question Answering
Authors: Zhuoqian Yang, Zengchang Qin, Jing Yu, Yue Hu
Date: 2018-12-23 00:00:00
Tasks: Cross-Modal Information Retrieval, Information Retrieval, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

One of the key issues of Visual Question Answering (VQA) is to reason with semantic clues in the visual content under the guidance of the question, how to model relational semantics still remains as a great challenge. To fully capture visual semantics, we propose to reason over a structured visual representation - scene graph, with embedded objects and inter-object relationships. This shows great benefit over vanilla vector representations and implicit visual relationship learning. Based on existing visual relationship models, we propose a visual relationship encoder that projects visual relationships into a learned deep semantic space constrained by visual context and language priors. Upon the constructed graph, we propose a Scene Graph Convolutional Network (SceneGCN) to jointly reason the object properties and relational semantics for the correct answer. We demonstrate the model's effectiveness and interpretability on the challenging GQA dataset and the classical VQA 2.0 dataset, remarkably achieving state-of-the-art 54.56% accuracy on GQA compared to the existing best model.
Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input
Authors: Junliang Guo, Xu Tan, Di He, Tao Qin, Linli Xu, Tie-Yan Liu
Date: 2018-12-23 00:00:00
Tasks: Decoder, Machine Translation, Sentence, Translation, Word Embeddings

Non-autoregressive translation (NAT) models, which remove the dependence on
previous target tokens from the inputs of the decoder, achieve significantly
inference speedup but at the cost of inferior accuracy compared to
autoregressive translation (AT) models. Previous work shows that the quality of
the inputs of the decoder is important and largely impacts the model accuracy.
In this paper, we propose two methods to enhance the decoder inputs so as to
improve NAT models. The first one directly leverages a phrase table generated
by conventional SMT approaches to translate source tokens to target tokens,
which are then fed into the decoder as inputs. The second one transforms
source-side word embeddings to target-side word embeddings through
sentence-level alignment and word-level adversary learning, and then feeds the
transformed word embeddings into the decoder as inputs. Experimental results
show our method largely outperforms the NAT baseline~\citep{gu2017non} by
$5.11$ BLEU scores on WMT14 English-German task and $4.72$ BLEU scores on WMT16
English-Romanian task.
Markov Game Modeling of Moving Target Defense for Strategic Detection of Threats in Cloud Networks
Authors: Ankur Chowdhary, Sailik Sengupta, Dijiang Huang, Subbarao Kambhampati
Date: 2018-12-23 00:00:00

The processing and storage of critical data in large-scale cloud networks
necessitate the need for scalable security solutions. It has been shown that
deploying all possible security measures incurs a cost on performance by using
up valuable computing and networking resources which are the primary selling
points for cloud service providers. Thus, there has been a recent interest in
developing Moving Target Defense (MTD) mechanisms that helps one optimize the
joint objective of maximizing security while ensuring that the impact on
performance is minimized. Often, these techniques model the problem of
multi-stage attacks by stealthy adversaries as a single-step attack detection
game using graph connectivity measures as a heuristic to measure performance,
thereby (1) losing out on valuable information that is inherently present in
graph-theoretic models designed for large cloud networks, and (2) coming up
with certain strategies that have asymmetric impacts on performance. In this
work, we leverage knowledge in attack graphs of a cloud network in formulating
a zero-sum Markov Game and use the Common Vulnerability Scoring System (CVSS)
to come up with meaningful utility values for this game. Then, we show that the
optimal strategy of placing detecting mechanisms against an adversary is
equivalent to computing the mixed Min-max Equilibrium of the Markov Game. We
compare the gains obtained by using our method to other techniques presently
used in cloud network security, thereby showing its effectiveness. Finally, we
highlight how the method was used for a small real-world cloud system.