pwc-archive/papers-with-abstracts huggingface dataset:
Using Noisy Extractions to Discover Causal Knowledge
Authors: Dhanya Sridhar, Jay Pujara, Lise Getoor
Date: 2017-11-16 00:00:00
Tasks: Causal Discovery

Knowledge bases (KB) constructed through information extraction from text
play an important role in query answering and reasoning. In this work, we study
a particular reasoning task, the problem of discovering causal relationships
between entities, known as causal discovery. There are two contrasting types of
approaches to discovering causal knowledge. One approach attempts to identify
causal relationships from text using automatic extraction techniques, while the
other approach infers causation from observational data. However, extractions
alone are often insufficient to capture complex patterns and full observational
data is expensive to obtain. We introduce a probabilistic method for fusing
noisy extractions with observational data to discover causal knowledge. We
propose a principled approach that uses the probabilistic soft logic (PSL)
framework to encode well-studied constraints to recover long-range patterns and
consistent predictions, while cheaply acquired extractions provide a proxy for
unseen observations. We apply our method gene regulatory networks and show the
promise of exploiting KB signals in causal discovery, suggesting a critical,
new area of research.
Locality preserving projection on SPD matrix Lie group: algorithm and analysis
Authors: Yangyang Li, Ruqian Lu
Date: 2017-03-28 00:00:00
Tasks: Action Recognition, Dimensionality Reduction, Face Recognition, Temporal Action Localization

Symmetric positive definite (SPD) matrices used as feature descriptors in
image recognition are usually high dimensional. Traditional manifold learning
is only applicable for reducing the dimension of high-dimensional vector-form
data. For high-dimensional SPD matrices, directly using manifold learning
algorithms to reduce the dimension of matrix-form data is impossible. The SPD
matrix must first be transformed into a long vector, and then the dimension of
this vector must be reduced. However, this approach breaks the spatial
structure of the SPD matrix space. To overcome this limitation, we propose a
new dimension reduction algorithm on SPD matrix space to transform
high-dimensional SPD matrices into low-dimensional SPD matrices. Our work is
based on the fact that the set of all SPD matrices with the same size has a Lie
group structure, and we aim to transform the manifold learning to the SPD
matrix Lie group. We use the basic idea of the manifold learning algorithm
called locality preserving projection (LPP) to construct the corresponding
Laplacian matrix on the SPD matrix Lie group. Thus, we call our approach
Lie-LPP to emphasize its Lie group character. We present a detailed algorithm
analysis and show through experiments that Lie-LPP achieves effective results
on human action recognition and human face recognition.
RelNet: End-to-End Modeling of Entities & Relations
Authors: Trapit Bansal, Arvind Neelakantan, Andrew McCallum
Date: 2017-06-22 00:00:00
Tasks: Question Answering, Relational Reasoning

We introduce RelNet: a new model for relational reasoning. RelNet is a memory
augmented neural network which models entities as abstract memory slots and is
equipped with an additional relational memory which models relations between
all memory pairs. The model thus builds an abstract knowledge graph on the
entities and relations present in a document which can then be used to answer
questions about the document. It is trained end-to-end: only supervision to the
model is in the form of correct answers to the questions. We test the model on
the 20 bAbI question-answering tasks with 10k examples per task and find that
it solves all the tasks with a mean error of 0.3%, achieving 0% error on 11 of
the 20 tasks.
Self-critical Sequence Training for Image Captioning
Authors: Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, Vaibhava Goel
Date: 2016-12-02 00:00:00
Proceeding: CVPR 2017 7
Tasks: Image Captioning, Policy Gradient Methods, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recently it has been shown that policy-gradient methods for reinforcement
learning can be utilized to train deep end-to-end systems directly on
non-differentiable metrics for the task at hand. In this paper we consider the
problem of optimizing image captioning systems using reinforcement learning,
and show that by carefully optimizing our systems using the test metrics of the
MSCOCO task, significant gains in performance can be realized. Our systems are
built using a new optimization approach that we call self-critical sequence
training (SCST). SCST is a form of the popular REINFORCE algorithm that, rather
than estimating a "baseline" to normalize the rewards and reduce variance,
utilizes the output of its own test-time inference algorithm to normalize the
rewards it experiences. Using this approach, estimating the reward signal (as
actor-critic methods must do) and estimating normalization (as REINFORCE
algorithms typically do) is avoided, while at the same time harmonizing the
model with respect to its test-time inference procedure. Empirically we find
that directly optimizing the CIDEr metric with SCST and greedy decoding at
test-time is highly effective. Our results on the MSCOCO evaluation sever
establish a new state-of-the-art on the task, improving the best result in
terms of CIDEr from 104.9 to 114.7.
Applying Ricci Flow to High Dimensional Manifold Learning
Authors: Yangyang Li, Ruqian Lu
Date: 2017-03-03 00:00:00
Tasks: Dimensionality Reduction, Vocal Bursts Intensity Prediction

Traditional manifold learning algorithms often bear an assumption that the
local neighborhood of any point on embedded manifold is roughly equal to the
tangent space at that point without considering the curvature. The curvature
indifferent way of manifold processing often makes traditional dimension
reduction poorly neighborhood preserving. To overcome this drawback we propose
a new algorithm called RF-ML to perform an operation on the manifold with help
of Ricci flow before reducing the dimension of manifold.
Sliced Wasserstein Distance for Learning Gaussian Mixture Models
Authors: Soheil Kolouri, Gustavo K. Rohde, Heiko Hoffmann
Date: 2017-11-15 00:00:00
Proceeding: CVPR 2018 6

Gaussian mixture models (GMM) are powerful parametric tools with many
applications in machine learning and computer vision. Expectation maximization
(EM) is the most popular algorithm for estimating the GMM parameters. However,
EM guarantees only convergence to a stationary point of the log-likelihood
function, which could be arbitrarily worse than the optimal solution. Inspired
by the relationship between the negative log-likelihood function and the
Kullback-Leibler (KL) divergence, we propose an alternative formulation for
estimating the GMM parameters using the sliced Wasserstein distance, which
gives rise to a new algorithm. Specifically, we propose minimizing the
sliced-Wasserstein distance between the mixture model and the data distribution
with respect to the GMM parameters. In contrast to the KL-divergence, the
energy landscape for the sliced-Wasserstein distance is more well-behaved and
therefore more suitable for a stochastic gradient descent scheme to obtain the
optimal GMM parameters. We show that our formulation results in parameter
estimates that are more robust to random initializations and demonstrate that
it can estimate high-dimensional data distributions more faithfully than the EM
algorithm.
Wald-Kernel: Learning to Aggregate Information for Sequential Inference
Authors: Diyan Teng, Emre Ertin
Date: 2015-08-31 00:00:00
Tasks: Decision Making, Two-sample testing

Sequential hypothesis testing is a desirable decision making strategy in any
time sensitive scenario. Compared with fixed sample-size testing, sequential
testing is capable of achieving identical probability of error requirements
using less samples in average. For a binary detection problem, it is well known
that for known density functions accumulating the likelihood ratio statistics
is time optimal under a fixed error rate constraint. This paper considers the
problem of learning a binary sequential detector from training samples when
density functions are unavailable. We formulate the problem as a constrained
likelihood ratio estimation which can be solved efficiently through convex
optimization by imposing Reproducing Kernel Hilbert Space (RKHS) structure on
the log-likelihood ratio function. In addition, we provide a computationally
efficient approximated solution for large scale data set. The proposed
algorithm, namely Wald-Kernel, is tested on a synthetic data set and two real
world data sets, together with previous approaches for likelihood ratio
estimation. Our empirical results show that the classifier trained through the
proposed technique achieves smaller average sampling cost than previous
approaches proposed in the literature for the same error rate.
Crowdsourcing Question-Answer Meaning Representations
Authors: Julian Michael, Gabriel Stanovsky, Luheng He, Ido Dagan, Luke Zettlemoyer
Date: 2017-11-16 00:00:00
Proceeding: NAACL 2018 6
Tasks: Sentence

We introduce Question-Answer Meaning Representations (QAMRs), which represent
the predicate-argument structure of a sentence as a set of question-answer
pairs. We also develop a crowdsourcing scheme to show that QAMRs can be labeled
with very little training, and gather a dataset with over 5,000 sentences and
100,000 questions. A detailed qualitative analysis demonstrates that the
crowd-generated question-answer pairs cover the vast majority of
predicate-argument relationships in existing datasets (including PropBank,
NomBank, QA-SRL, and AMR) along with many previously under-resourced ones,
including implicit arguments and relations. The QAMR data and annotation code
is made publicly available to enable future work on how best to model these
complex phenomena.
Systems, Actors and Agents: Operation in a multicomponent environment
Authors: Mark Burgin
Date: 2017-11-16 00:00:00
Tasks: Diversity

Multi-agent approach has become popular in computer science and technology.
However, the conventional models of multi-agent and multicomponent systems
implicitly or explicitly assume existence of absolute time or even do not
include time in the set of defining parameters. At the same time, it is proved
theoretically and validated experimentally that there are different times and
time scales in a variety of real systems - physical, chemical, biological,
social, informational, etc. Thus, the goal of this work is construction of a
multi-agent multicomponent system models with concurrency of processes and
diversity of actions. To achieve this goal, a mathematical system actor model
is elaborated and its properties are studied.
Towards Scalable Spectral Clustering via Spectrum-Preserving Sparsification
Authors: Yongyu Wang, Zhuo Feng
Date: 2017-10-12 00:00:00
Tasks: Clustering

The eigendeomposition of nearest-neighbor (NN) graph Laplacian matrices is
the main computational bottleneck in spectral clustering. In this work, we
introduce a highly-scalable, spectrum-preserving graph sparsification algorithm
that enables to build ultra-sparse NN (u-NN) graphs with guaranteed
preservation of the original graph spectrums, such as the first few
eigenvectors of the original graph Laplacian. Our approach can immediately lead
to scalable spectral clustering of large data networks without sacrificing
solution quality. The proposed method starts from constructing low-stretch
spanning trees (LSSTs) from the original graphs, which is followed by
iteratively recovering small portions of "spectrally critical" off-tree edges
to the LSSTs by leveraging a spectral off-tree embedding scheme. To determine
the suitable amount of off-tree edges to be recovered to the LSSTs, an
eigenvalue stability checking scheme is proposed, which enables to robustly
preserve the first few Laplacian eigenvectors within the sparsified graph.
Additionally, an incremental graph densification scheme is proposed for
identifying extra edges that have been missing in the original NN graphs but
can still play important roles in spectral clustering tasks. Our experimental
results for a variety of well-known data sets show that the proposed method can
dramatically reduce the complexity of NN graphs, leading to significant
speedups in spectral clustering.
Polynomial Time Algorithms for Dual Volume Sampling
Authors: Chengtao Li, Stefanie Jegelka, Suvrit Sra
Date: 2017-03-08 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Experimental Design

We study dual volume sampling, a method for selecting k columns from an n x m
short and wide matrix (n <= k <= m) such that the probability of selection is
proportional to the volume spanned by the rows of the induced submatrix. This
method was proposed by Avron and Boutsidis (2013), who showed it to be a
promising method for column subset selection and its multiple applications.
However, its wider adoption has been hampered by the lack of polynomial time
sampling algorithms. We remove this hindrance by developing an exact
(randomized) polynomial time sampling algorithm as well as its derandomization.
Thereafter, we study dual volume sampling via the theory of real stable
polynomials and prove that its distribution satisfies the "Strong Rayleigh"
property. This result has numerous consequences, including a provably
fast-mixing Markov chain sampler that makes dual volume sampling much more
attractive to practitioners. This sampler is closely related to classical
algorithms for popular experimental design methods that are to date lacking
theoretical analysis but are known to empirically work well.
Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
Authors: Asit Mishra, Debbie Marr
Date: 2017-11-15 00:00:00
Proceeding: ICLR 2018 1
Tasks: image-classification, Image Classification, Knowledge Distillation, Model Compression, object-detection, Object Detection

Deep learning networks have achieved state-of-the-art accuracies on computer
vision workloads like image classification and object detection. The performant
systems, however, typically involve big models with numerous parameters. Once
trained, a challenging aspect for such top performing models is deployment on
resource constrained inference systems - the models (often deep networks or
wide networks or both) are compute and memory intensive. Low-precision numerics
and model compression using knowledge distillation are popular techniques to
lower both the compute requirements and memory footprint of these deployed
models. In this paper, we study the combination of these two techniques and
show that the performance of low-precision networks can be significantly
improved by using knowledge distillation techniques. Our approach, Apprentice,
achieves state-of-the-art accuracies using ternary precision and 4-bit
precision for variants of ResNet architecture on ImageNet dataset. We present
three schemes using which one can apply knowledge distillation techniques to
various stages of the train-and-deploy pipeline.
Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
Authors: Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, Andrew McCallum
Date: 2017-11-15 00:00:00
Proceeding: ICLR 2018 1
Tasks: Navigate, Reinforcement Learning, Relation, valid

Knowledge bases (KB), both automatically and manually constructed, are often
incomplete --- many valid facts can be inferred from the KB by synthesizing
existing information. A popular approach to KB completion is to infer new
relations by combinatory reasoning over the information found along other paths
connecting a pair of entities. Given the enormous size of KBs and the
exponential number of paths, previous path-based models have considered only
the problem of predicting a missing relation given two entities or evaluating
the truth of a proposed triple. Additionally, these methods have traditionally
used random paths between fixed entity pairs or more recently learned to pick
paths between them. We propose a new algorithm MINERVA, which addresses the
much more difficult and practical task of answering questions where the
relation is known, but only one entity. Since random walks are impractical in a
setting with combinatorially many destinations from a start node, we present a
neural reinforcement learning approach which learns how to navigate the graph
conditioned on the input query to find predictive paths. Empirically, this
approach obtains state-of-the-art results on several datasets, significantly
outperforming prior methods.
Knowledge transfer for surgical activity prediction
Authors: Olga Dergachyova, Xavier Morandi, Pierre Jannin
Date: 2017-11-15 00:00:00
Tasks: Activity Prediction, Prediction, Transfer Learning

Lack of training data hinders automatic recognition and prediction of
surgical activities necessary for situation-aware operating rooms. We propose
using knowledge transfer to compensate for data deficit and improve prediction.
We used two approaches to extract and transfer surgical process knowledge.
First, we encoded semantic information about surgical terms using word
embedding which boosted learning process. Secondly, we passed knowledge between
different clinical datasets of neurosurgical procedures using transfer
learning. Transfer learning was shown to be more effective than a simple
combination of data, especially for less similar procedures. The combination of
two methods provided 22% improvement of activity prediction. We also made
several pertinent observations about surgical practices.
AOGNets: Compositional Grammatical Architectures for Deep Learning
Authors: Xilai Li, Xi Song, Tianfu Wu
Date: 2017-11-15 00:00:00
Proceeding: CVPR 2019 6
Tasks: Adversarial Defense, Deep Learning, Image Classification, object-detection, Object Detection, Representation Learning, Sentence

Neural architectures are the foundation for improving performance of deep
neural networks (DNNs). This paper presents deep compositional grammatical
architectures which harness the best of two worlds: grammar models and DNNs.
The proposed architectures integrate compositionality and reconfigurability of
the former and the capability of learning rich features of the latter in a
principled way. We utilize AND-OR Grammar (AOG) as network generator in this
paper and call the resulting networks AOGNets. An AOGNet consists of a number
of stages each of which is composed of a number of AOG building blocks. An AOG
building block splits its input feature map into N groups along feature
channels and then treat it as a sentence of N words. It then jointly realizes a
phrase structure grammar and a dependency grammar in bottom-up parsing the
"sentence" for better feature exploration and reuse. It provides a unified
framework for the best practices developed in state-of-the-art DNNs. In
experiments, AOGNet is tested in the CIFAR-10, CIFAR-100 and ImageNet-1K
classification benchmark and the MS-COCO object detection and segmentation
benchmark. In CIFAR-10, CIFAR-100 and ImageNet-1K, AOGNet obtains better
performance than ResNet and most of its variants, ResNeXt and its attention
based variants such as SENet, DenseNet and DualPathNet. AOGNet also obtains the
best model interpretability score using network dissection. AOGNet further
shows better potential in adversarial defense. In MS-COCO, AOGNet obtains
better performance than the ResNet and ResNeXt backbones in Mask R-CNN.
FSL-BM: Fuzzy Supervised Learning with Binary Meta-Feature for Classification
Authors: Kamran Kowsari, Nima Bari, Roman Vichr, Farhad A. Goodarzi
Date: 2017-09-26 00:00:00
Tasks: Binary Classification, General Classification

This paper introduces a novel real-time Fuzzy Supervised Learning with Binary
Meta-Feature (FSL-BM) for big data classification task. The study of real-time
algorithms addresses several major concerns, which are namely: accuracy, memory
consumption, and ability to stretch assumptions and time complexity. Attaining
a fast computational model providing fuzzy logic and supervised learning is one
of the main challenges in the machine learning. In this research paper, we
present FSL-BM algorithm as an efficient solution of supervised learning with
fuzzy logic processing using binary meta-feature representation using Hamming
Distance and Hash function to relax assumptions. While many studies focused on
reducing time complexity and increasing accuracy during the last decade, the
novel contribution of this proposed solution comes through integration of
Hamming Distance, Hash function, binary meta-features, binary classification to
provide real time supervised method. Hash Tables (HT) component gives a fast
access to existing indices; and therefore, the generation of new indices in a
constant time complexity, which supersedes existing fuzzy supervised algorithms
with better or comparable results. To summarize, the main contribution of this
technique for real-time Fuzzy Supervised Learning is to represent hypothesis
through binary input as meta-feature space and creating the Fuzzy Supervised
Hash table to train and validate model.
Detecting Adversarial Samples from Artifacts
Authors: Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, Andrew B. Gardner
Date: 2017-03-01 00:00:00
Tasks: Density Estimation

Deep neural networks (DNNs) are powerful nonlinear architectures that are
known to be robust to random perturbations of the input. However, these models
are vulnerable to adversarial perturbations--small input changes crafted
explicitly to fool the model. In this paper, we ask whether a DNN can
distinguish adversarial samples from their normal and noisy counterparts. We
investigate model confidence on adversarial samples by looking at Bayesian
uncertainty estimates, available in dropout neural networks, and by performing
density estimation in the subspace of deep features learned by the model. The
result is a method for implicit adversarial detection that is oblivious to the
attack algorithm. We evaluate this method on a variety of standard datasets
including MNIST and CIFAR-10 and show that it generalizes well across different
architectures and attacks. Our findings report that 85-93% ROC-AUC can be
achieved on a number of standard classification tasks with a negative class
that consists of both normal and noisy samples.
Constrained Bayesian Optimization for Automatic Chemical Design
Authors: Ryan-Rhys Griffiths, José Miguel Hernández-Lobato
Date: 2017-09-16 00:00:00
Tasks: Bayesian Optimization

Automatic Chemical Design is a framework for generating novel molecules with optimized properties. The original scheme, featuring Bayesian optimization over the latent space of a variational autoencoder, suffers from the pathology that it tends to produce invalid molecular structures. First, we demonstrate empirically that this pathology arises when the Bayesian optimization scheme queries latent points far away from the data on which the variational autoencoder has been trained. Secondly, by reformulating the search procedure as a constrained Bayesian optimization problem, we show that the effects of this pathology can be mitigated, yielding marked improvements in the validity of the generated molecules. We posit that constrained Bayesian optimization is a good approach for solving this class of training set mismatch in many generative tasks involving Bayesian optimization over the latent space of a variational autoencoder.