pwc-archive/papers-with-abstracts huggingface dataset:
The Limits of Morality in Strategic Games
Authors: Rui Cao, Pavel Naumov
Date: 2019-01-22 00:00:00

A coalition is blameable for an outcome if the coalition had a strategy to
prevent it. It has been previously suggested that the cost of prevention, or
the cost of sacrifice, can be used to measure the degree of blameworthiness.
The paper adopts this approach and proposes a modal logical system for
reasoning about the degree of blameworthiness. The main technical result is a
completeness theorem for the proposed system.
A tensorized logic programming language for large-scale data
Authors: Ryosuke Kojima, Taisuke Sato
Date: 2019-01-20 00:00:00
Tasks: Knowledge Graphs

We introduce a new logic programming language T-PRISM based on tensor
embeddings. Our embedding scheme is a modification of the distribution
semantics in PRISM, one of the state-of-the-art probabilistic logic programming
languages, by replacing distribution functions with multidimensional arrays,
i.e., tensors. T-PRISM consists of two parts: logic programming part and
numerical computation part. The former provides flexible and interpretable
modeling at the level of first order logic, and the latter part provides
scalable computation utilizing parallelization and hardware acceleration with
GPUs. Combing these two parts provides a remarkably wide range of high-level
declarative modeling from symbolic reasoning to deep learning. To embody this
programming language, we also introduce a new semantics, termed tensorized
semantics, which combines the traditional least model semantics in logic
programming with the embeddings of tensors. In T-PRISM, we first derive a set
of equations related to tensors from a given program using logical inference,
i.e., Prolog execution in a symbolic space and then solve the derived equations
in a continuous space by TensorFlow. Using our preliminary implementation of
T-PRISM, we have successfully dealt with a wide range of modeling. We have
succeeded in dealing with real large-scale data in the declarative modeling.
This paper presents a DistMult model for knowledge graphs using the FB15k and
WN18 datasets.
Human-centric Transfer Learning Explanation via Knowledge Graph [Extended Abstract]
Authors: Yuxia Geng, Jiaoyan Chen, Ernesto Jimenez-Ruiz, Huajun Chen
Date: 2019-01-20 00:00:00
Tasks: Transfer Learning, Zero-Shot Learning

Transfer learning which aims at utilizing knowledge learned from one problem
(source domain) to solve another different but related problem (target domain)
has attracted wide research attentions. However, the current transfer learning
methods are mostly uninterpretable, especially to people without ML expertise.
In this extended abstract, we brief introduce two knowledge graph (KG) based
frameworks towards human understandable transfer learning explanation. The
first one explains the transferability of features learned by Convolutional
Neural Network (CNN) from one domain to another through pre-training and
fine-tuning, while the second justifies the model of a target domain predicted
by models from multiple source domains in zero-shot learning (ZSL). Both
methods utilize KG and its reasoning capability to provide rich and human
understandable explanations to the transfer procedure.
A Universal Logic Operator for Interpretable Deep Convolution Networks
Authors: KamWoh Ng, Lixin Fan, Chee Seng Chan
Date: 2019-01-20 00:00:00

Explaining neural network computation in terms of probabilistic/fuzzy logical
operations has attracted much attention due to its simplicity and high
interpretability. Different choices of logical operators such as AND, OR and
XOR give rise to another dimension for network optimization, and in this paper,
we study the open problem of learning a universal logical operator without
prescribing to any logical operations manually. Insightful observations along
this exploration furnish deep convolution networks with a novel logical
interpretation.
Visualized Insights into the Optimization Landscape of Fully Convolutional Networks
Authors: Jianjie Lu, Kai-yu Tong
Date: 2019-01-20 00:00:00

Many image processing tasks involve image-to-image mapping, which can be
addressed well by fully convolutional networks (FCN) without any heavy
preprocessing. Although empirically designing and training FCNs can achieve
satisfactory results, reasons for the improvement in performance are slightly
ambiguous. Our study is to make progress in understanding their generalization
abilities through visualizing the optimization landscapes. The visualization of
objective functions is obtained by choosing a solution and projecting its
vicinity onto a 3D space. We compare three FCN-based networks (two existing
models and a new proposed in this paper for comparison) on multiple datasets.
It has been observed in practice that the connections from the pre-pooled
feature maps to the post-upsampled can achieve better results. We investigate
the cause and provide experiments to shows that the skip-layer connections in
FCN can promote flat optimization landscape, which is well known to generalize
better. Additionally, we explore the relationship between the models
generalization ability and loss surface under different batch sizes. Results
show that large-batch training makes the model converge to sharp minimizers
with chaotic vicinities while small-batch method leads the model to flat
minimizers with smooth and nearly convex regions. Our work may contribute to
insights and analysis for designing and training FCNs.
Explainable Failure Predictions with RNN Classifiers based on Time Series Data
Authors: Ioana Giurgiu, Anika Schumann
Date: 2019-01-20 00:00:00
Tasks: Time Series, Time Series Analysis

Given key performance indicators collected with fine granularity as time
series, our aim is to predict and explain failures in storage environments.
Although explainable predictive modeling based on spiky telemetry data is key
in many domains, current approaches cannot tackle this problem. Deep learning
methods suitable for sequence modeling and learning temporal dependencies, such
as RNNs, are effective, but opaque from an explainability perspective. Our
approach first extracts the anomalous spikes from time series as events and
then builds an RNN classifier with attention mechanisms to embed the
irregularity and frequency of these events. A preliminary evaluation on real
world storage environments shows that our approach can predict failures within
a 3-day prediction window with comparable accuracy as traditional RNN-based
classifiers. At the same time it can explain the predictions by returning the
key anomalous events which led to those failure predictions.
Data Interpolations in Deep Generative Models under Non-Simply-Connected Manifold Topology
Authors: Jiseob Kim, Byoung-Tak Zhang
Date: 2019-01-20 00:00:00

Exploiting the deep generative model's remarkable ability of learning the
data-manifold structure, some recent researches proposed a geometric data
interpolation method based on the geodesic curves on the learned data-manifold.
However, this interpolation method often gives poor results due to a
topological difference between the model and the dataset. The model defines a
family of simply-connected manifolds, whereas the dataset generally contains
disconnected regions or holes that make them non-simply-connected. To
compensate this difference, we propose a novel density regularizer that make
the interpolation path circumvent the holes denoted by low probability density.
We confirm that our method gives consistently better interpolation results from
the experiments with real-world image datasets.
On Network Science and Mutual Information for Explaining Deep Neural Networks
Authors: Brian Davis, Umang Bhatt, Kartikeya Bhardwaj, Radu Marculescu, José M. F. Moura
Date: 2019-01-20 00:00:00
Tasks: Deep Learning

In this paper, we present a new approach to interpret deep learning models. By coupling mutual information with network science, we explore how information flows through feedforward networks. We show that efficiently approximating mutual information allows us to create an information measure that quantifies how much information flows between any two neurons of a deep learning model. To that end, we propose NIF, Neural Information Flow, a technique for codifying information flow that exposes deep learning model internals and provides feature attributions.
Quantifying Interpretability and Trust in Machine Learning Systems
Authors: Philipp Schmidt, Felix Biessmann
Date: 2019-01-20 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making

Decisions by Machine Learning (ML) models have become ubiquitous. Trusting
these decisions requires understanding how algorithms take them. Hence
interpretability methods for ML are an active focus of research. A central
problem in this context is that both the quality of interpretability methods as
well as trust in ML predictions are difficult to measure. Yet evaluations,
comparisons and improvements of trust and interpretability require quantifiable
measures. Here we propose a quantitative measure for the quality of
interpretability methods. Based on that we derive a quantitative measure of
trust in ML decisions. Building on previous work we propose to measure
intuitive understanding of algorithmic decisions using the information transfer
rate at which humans replicate ML model predictions. We provide empirical
evidence from crowdsourcing experiments that the proposed metric robustly
differentiates interpretability methods. The proposed metric also demonstrates
the value of interpretability for ML assisted human decision making: in our
experiments providing explanations more than doubled productivity in annotation
tasks. However unbiased human judgement is critical for doctors, judges, policy
makers and others. Here we derive a trust metric that identifies when human
decisions are overly biased towards ML predictions. Our results complement
existing qualitative work on trust and interpretability by quantifiable
measures that can serve as objectives for further improving methods in this
field of research.
Convolution Forgetting Curve Model for Repeated Learning
Authors: Yanlu Xie, Yue Chen, Man Li
Date: 2019-01-19 00:00:00
Tasks: model

Most of mathematic forgetting curve models fit well with the forgetting data
under the learning condition of one time rather than repeated. In the paper, a
convolution model of forgetting curve is proposed to simulate the memory
process during learning. In this model, the memory ability (i.e. the central
procedure in the working memory model) and learning material (i.e. the input in
the working memory model) is regarded as the system function and the input
function, respectively. The status of forgetting (i.e. the output in the
working memory model) is regarded as output function or the convolution result
of the memory ability and learning material. The model is applied to simulate
the forgetting curves in different situations. The results show that the model
is able to simulate the forgetting curves not only in one time learning
condition but also in multi-times condition. The model is further verified in
the experiments of Mandarin tone learning for Japanese learners. And the
predicted curve fits well on the test points.
Similarity Learning for Provably Accurate Sparse Linear Classification
Authors: Aurelien Bellet, Amaury Habrard, Marc Sebban
Date: 2012-06-27 00:00:00
Tasks: Classification, General Classification

In recent years, the crucial importance of metrics in machine learning
algorithms has led to an increasing interest for optimizing distance and
similarity functions. Most of the state of the art focus on learning
Mahalanobis distances (requiring to fulfill a constraint of positive
semi-definiteness) for use in a local k-NN algorithm. However, no theoretical
link is established between the learned metrics and their performance in
classification. In this paper, we make use of the formal framework of good
similarities introduced by Balcan et al. to design an algorithm for learning a
non PSD linear similarity optimized in a nonlinear feature space, which is then
used to build a global linear classifier. We show that our approach has uniform
stability and derive a generalization bound on the classification error.
Experiments performed on various datasets confirm the effectiveness of our
approach compared to state-of-the-art methods and provide evidence that (i) it
is fast, (ii) robust to overfitting and (iii) produces very sparse classifiers.
SVD Based Image Processing Applications: State of The Art, Contributions and Research Challenges
Authors: Rowayda A. Sadek
Date: 2012-11-29 00:00:00
Tasks: Survey

Singular Value Decomposition (SVD) has recently emerged as a new paradigm for
processing different types of images. SVD is an attractive algebraic transform
for image processing applications. The paper proposes an experimental survey
for the SVD as an efficient transform in image processing applications. Despite
the well-known fact that SVD offers attractive properties in imaging, the
exploring of using its properties in various image applications is currently at
its infancy. Since the SVD has many attractive properties have not been
utilized, this paper contributes in using these generous properties in newly
image applications and gives a highly recommendation for more research
challenges. In this paper, the SVD properties for images are experimentally
presented to be utilized in developing new SVD-based image processing
applications. The paper offers survey on the developed SVD based image
applications. The paper also proposes some new contributions that were
originated from SVD properties analysis in different image processing. The aim
of this paper is to provide a better understanding of the SVD in image
processing and identify important various applications and open research
directions in this increasingly important area; SVD based image processing in
the future research.
Effectiveness of Tree-based Ensembles for Anomaly Discovery: Insights, Batch and Streaming Active Learning
Authors: Shubhomoy Das, Md. Rakibul Islam, Nitthilan Kannappan Jayakodi, Janardhan Rao Doppa
Date: 2019-01-23 00:00:00
Tasks: Active Learning, Anomaly Detection, Computer Security, Drift Detection

In many real-world AD applications including computer security and fraud prevention, the anomaly detector must be configurable by the human analyst to minimize the effort on false positives. One important way to configure the detector is by providing true labels (nominal or anomaly) for a few instances. Recent work on active anomaly discovery has shown that greedily querying the top-scoring instance and tuning the weights of ensemble detectors based on label feedback allows us to quickly discover true anomalies. This paper makes four main contributions to improve the state-of-the-art in anomaly discovery using tree-based ensembles. First, we provide an important insight that explains the practical successes of unsupervised tree-based ensembles and active learning based on greedy query selection strategy. We also present empirical results on real-world data to support our insights and theoretical analysis to support active learning. Second, we develop a novel batch active learning algorithm to improve the diversity of discovered anomalies based on a formalism called compact description to describe the discovered anomalies. Third, we develop a novel active learning algorithm to handle streaming data setting. We present a data drift detection algorithm that not only detects the drift robustly, but also allows us to take corrective actions to adapt the anomaly detector in a principled manner. Fourth, we present extensive experiments to evaluate our insights and our tree-based active anomaly discovery algorithms in both batch and streaming data settings. Our results show that active learning allows us to discover significantly more anomalies than state-of-the-art unsupervised baselines, our batch active learning algorithm discovers diverse anomalies, and our algorithms under the streaming-data setup are competitive with the batch setup.
How to Explain Individual Classification Decisions
Authors: David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, Klaus-Robert Mueller
Date: 2009-12-06 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification

After building a classifier with modern tools of machine learning we
typically have a black box at hand that is able to predict well for unseen
data. Thus, we get an answer to the question what is the most likely label of a
given unseen data point. However, most methods will provide no answer why the
model predicted the particular label for a single instance and what features
were most influential for that particular instance. The only method that is
currently able to provide such explanations are decision trees. This paper
proposes a procedure which (based on a set of assumptions) allows to explain
the decisions of any classification method.
CoCoNet: A Collaborative Convolutional Network
Authors: Tapabrata Chakraborti, Brendan McCane, Steven Mills, Umapada Pal
Date: 2019-01-28 00:00:00
Tasks: Fine-Grained Visual Categorization, Fine-Grained Visual Recognition, Transfer Learning

We present an end-to-end deep network for fine-grained visual categorization called Collaborative Convolutional Network (CoCoNet). The network uses a collaborative layer after the convolutional layers to represent an image as an optimal weighted collaboration of features learned from training samples as a whole rather than one at a time. This gives CoCoNet more power to encode the fine-grained nature of the data with limited samples. We perform a detailed study of the performance with 1-stage and 2-stage transfer learning. The ablation study shows that the proposed method outperforms its constituent parts consistently. CoCoNet also outperforms few state-of-the-art competing methods. Experiments have been performed on the fine-grained bird species classification problem as a representative example, but the method may be applied to other similar tasks. We also introduce a new public dataset for fine-grained species recognition, that of Indian endemic birds and have reported initial results on it.
Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets
Authors: Robert Cornish, Paul Vanetti, Alexandre Bouchard-Côté, George Deligiannidis, Arnaud Doucet
Date: 2019-01-28 00:00:00
Tasks: Bayesian Inference

Bayesian inference via standard Markov Chain Monte Carlo (MCMC) methods is too computationally intensive to handle large datasets, since the cost per step usually scales like $\Theta(n)$ in the number of data points $n$. We propose the Scalable Metropolis-Hastings (SMH) kernel that exploits Gaussian concentration of the posterior to require processing on average only $O(1)$ or even $O(1/\sqrt{n})$ data points per step. This scheme is based on a combination of factorized acceptance probabilities, procedures for fast simulation of Bernoulli processes, and control variate ideas. Contrary to many MCMC subsampling schemes such as fixed step-size Stochastic Gradient Langevin Dynamics, our approach is exact insofar as the invariant distribution is the true posterior and not an approximation to it. We characterise the performance of our algorithm theoretically, and give realistic and verifiable conditions under which it is geometrically ergodic. This theory is borne out by empirical results that demonstrate overall performance benefits over standard Metropolis-Hastings and various subsampling algorithms.
CapsAttacks: Robust and Imperceptible Adversarial Attacks on Capsule Networks
Authors: Alberto Marchisio, Giorgio Nanfa, Faiq Khalid, Muhammad Abdullah Hanif, Maurizio Martina, Muhammad Shafique
Date: 2019-01-28 00:00:00
Tasks: image-classification, Image Classification, Traffic Sign Recognition

Capsule Networks preserve the hierarchical spatial relationships between objects, and thereby bears a potential to surpass the performance of traditional Convolutional Neural Networks (CNNs) in performing tasks like image classification. A large body of work has explored adversarial examples for CNNs, but their effectiveness on Capsule Networks has not yet been well studied. In our work, we perform an analysis to study the vulnerabilities in Capsule Networks to adversarial attacks. These perturbations, added to the test inputs, are small and imperceptible to humans, but can fool the network to mispredict. We propose a greedy algorithm to automatically generate targeted imperceptible adversarial examples in a black-box attack scenario. We show that this kind of attacks, when applied to the German Traffic Sign Recognition Benchmark (GTSRB), mislead Capsule Networks. Moreover, we apply the same kind of adversarial attacks to a 5-layer CNN and a 9-layer CNN, and analyze the outcome, compared to the Capsule Networks to study differences in their behavior.
It could be worse, it could be raining: reliable automatic meteorological forecasting
Authors: Matteo Cristani, Francesco Domenichini, Claudio Tomazzoli, Luca Viganò, Margherita Zorzi
Date: 2019-01-28 00:00:00
Tasks: Diagnostic

Meteorological forecasting provides reliable prediction about the future
weather within a given interval of time. Meteorological forecasting can be
viewed as a form of hybrid diagnostic reasoning and can be mapped onto an
integrated conceptual framework. The automation of the forecasting process
would be helpful in a number of contexts, in particular: when the amount of
data is too wide to be dealt with manually; to support forecasters education;
when forecasting about underpopulated geographic areas is not interesting for
everyday life (and then is out from human forecasters' tasks) but is central
for tourism sponsorship. We present logic MeteoLOG, a framework that models the
main steps of the reasoner the forecaster adopts to provide a bulletin.
MeteoLOG rests on several traditions, mainly on fuzzy, temporal and
probabilistic logics. On this basis, we also introduce the algorithm
Tournament, that transforms a set of MeteoLOG rules into a defeasible theory,
that can be implemented into an automatic reasoner. We finally propose an
example that models a real world forecasting scenario.
Multi-modal dialog for browsing large visual catalogs using exploration-exploitation paradigm in a joint embedding space
Authors: Indrani Bhattacharya, Arkabandhu Chowdhury, Vikas Raykar
Date: 2019-01-28 00:00:00

We present a multi-modal dialog system to assist online shoppers in visually
browsing through large catalogs. Visual browsing is different from visual
search in that it allows the user to explore the wide range of products in a
catalog, beyond the exact search matches. We focus on a slightly asymmetric
version of the complete multi-modal dialog where the system can understand both
text and image queries but responds only in images. We formulate our problem of
"showing $k$ best images to a user" based on the dialog context so far, as
sampling from a Gaussian Mixture Model in a high dimensional joint multi-modal
embedding space, that embed both the text and the image queries. Our system
remembers the context of the dialog and uses an exploration-exploitation
paradigm to assist in visual browsing. We train and evaluate the system on a
multi-modal dialog dataset that we generate from large catalog data. Our
experiments are promising and show that the agent is capable of learning and
can display relevant results with an average cosine similarity of 0.85 to the
ground truth. Our preliminary human evaluation also corroborates the fact that
such a multi-modal dialog system for visual browsing is well-received and is
capable of engaging human users.