pwc-archive/papers-with-abstracts huggingface dataset:
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions
Authors: Qing Li, Jianlong Fu, Dongfei Yu, Tao Mei, Jiebo Luo
Date: 2018-01-27 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Attribute, Image Captioning, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Visual Question Answering (VQA) has attracted attention from both computer
vision and natural language processing communities. Most existing approaches
adopt the pipeline of representing an image via pre-trained CNNs, and then
using the uninterpretable CNN features in conjunction with the question to
predict the answer. Although such end-to-end models might report promising
performance, they rarely provide any insight, apart from the answer, into the
VQA process. In this work, we propose to break up the end-to-end VQA into two
steps: explaining and reasoning, in an attempt towards a more explainable VQA
by shedding light on the intermediate results between these two steps. To that
end, we first extract attributes and generate descriptions as explanations for
an image using pre-trained attribute detectors and image captioning models,
respectively. Next, a reasoning module utilizes these explanations in place of
the image to infer an answer to the question. The advantages of such a
breakdown include: (1) the attributes and captions can reflect what the system
extracts from the image, thus can provide some explanations for the predicted
answer; (2) these intermediate results can help us identify the inabilities of
both the image understanding part and the answer inference part when the
predicted answer is wrong. We conduct extensive experiments on a popular VQA
dataset and dissect all results according to several measurements of the
explanation quality. Our system achieves comparable performance with the
state-of-the-art, yet with added benefits of explainability and the inherent
ability to further improve with higher quality explanations.
A Sheaf Model of Contradictions and Disagreements. Preliminary Report and Discussion
Authors: Wlodek Zadrozny, Luciana Garbayo
Date: 2018-01-27 00:00:00
Tasks: Topological Data Analysis

We introduce a new formal model -- based on the mathematical construct of
sheaves -- for representing contradictory information in textual sources. This
model has the advantage of letting us (a) identify the causes of the
inconsistency; (b) measure how strong it is; (c) and do something about it,
e.g. suggest ways to reconcile inconsistent advice. This model naturally
represents the distinction between contradictions and disagreements. It is
based on the idea of representing natural language sentences as formulas with
parameters sitting on lattices, creating partial orders based on predicates
shared by theories, and building sheaves on these partial orders with products
of lattices as stalks. Degrees of disagreement are measured by the existence of
global and local sections.
  Limitations of the sheaf approach and connections to recent work in natural
language processing, as well as the topics of contextuality in physics, data
fusion, topological data analysis and epistemology are also discussed.
Scalable Twin Neural Networks for Classification of Unbalanced Data
Authors: Jayadeva, Himanshu Pant, Sumit Soman, Mayank Sharma
Date: 2017-04-30 00:00:00
Tasks: Classification, General Classification

Twin Support Vector Machines (TWSVMs) have emerged an efficient alternative
to Support Vector Machines (SVM) for learning from imbalanced datasets. The
TWSVM learns two non-parallel classifying hyperplanes by solving a couple of
smaller sized problems. However, it is unsuitable for large datasets, as it
involves matrix operations. In this paper, we discuss a Twin Neural Network
(Twin NN) architecture for learning from large unbalanced datasets. The Twin NN
also learns an optimal feature map, allowing for better discrimination between
classes. We also present an extension of this network architecture for
multiclass datasets. Results presented in the paper demonstrate that the Twin
NN generalizes well and scales well on large unbalanced datasets.
Graph Memory Networks for Molecular Activity Prediction
Authors: Trang Pham, Truyen Tran, Svetha Venkatesh
Date: 2018-01-08 00:00:00
Tasks: Activity Prediction, Drug Design, Multi-Task Learning, Prediction

Molecular activity prediction is critical in drug design. Machine learning
techniques such as kernel methods and random forests have been successful for
this task. These models require fixed-size feature vectors as input while the
molecules are variable in size and structure. As a result, fixed-size
fingerprint representation is poor in handling substructures for large
molecules. In addition, molecular activity tests, or a so-called BioAssays, are
relatively small in the number of tested molecules due to its complexity. Here
we approach the problem through deep neural networks as they are flexible in
modeling structured data such as grids, sequences and graphs. We train multiple
BioAssays using a multi-task learning framework, which combines information
from multiple sources to improve the performance of prediction, especially on
small datasets. We propose Graph Memory Network (GraphMem), a memory-augmented
neural network to model the graph structure in molecules. GraphMem consists of
a recurrent controller coupled with an external memory whose cells dynamically
interact and change through a multi-hop reasoning process. Applied to the
molecules, the dynamic interactions enable an iterative refinement of the
representation of molecular graphs with multiple bond types. GraphMem is
capable of jointly training on multiple datasets by using a specific-task query
fed to the controller as an input. We demonstrate the effectiveness of the
proposed model for separately and jointly training on more than 100K
measurements, spanning across 9 BioAssay activity tests.
Improving Word Vector with Prior Knowledge in Semantic Dictionary
Authors: Wei Li, Yunfang Wu, Xueqiang Lv
Date: 2018-01-27 00:00:00
Tasks: NER

Using low dimensional vector space to represent words has been very effective
in many NLP tasks. However, it doesn't work well when faced with the problem of
rare and unseen words. In this paper, we propose to leverage the knowledge in
semantic dictionary in combination with some morphological information to build
an enhanced vector space. We get an improvement of 2.3% over the
state-of-the-art Heidel Time system in temporal expression recognition, and
obtain a large gain in other name entity recognition (NER) tasks. The semantic
dictionary Hownet alone also shows promising results in computing lexical
similarity.
A Learning and Masking Approach to Secure Learning
Authors: Linh Nguyen, Sky Wang, Arunesh Sinha
Date: 2017-09-13 00:00:00
Tasks: Autonomous Driving

Deep Neural Networks (DNNs) have been shown to be vulnerable against
adversarial examples, which are data points cleverly constructed to fool the
classifier. Such attacks can be devastating in practice, especially as DNNs are
being applied to ever increasing critical tasks like image recognition in
autonomous driving. In this paper, we introduce a new perspective on the
problem. We do so by first defining robustness of a classifier to adversarial
exploitation. Next, we show that the problem of adversarial example generation
can be posed as learning problem. We also categorize attacks in literature into
high and low perturbation attacks; well-known attacks like fast-gradient sign
method (FGSM) and our attack produce higher perturbation adversarial examples
while the more potent but computationally inefficient Carlini-Wagner (CW)
attack is low perturbation. Next, we show that the dual approach of the attack
learning problem can be used as a defensive technique that is effective against
high perturbation attacks. Finally, we show that a classifier masking method
achieved by adding noise to the a neural network's logit output protects
against low distortion attacks such as the CW attack. We also show that both
our learning and masking defense can work simultaneously to protect against
multiple attacks. We demonstrate the efficacy of our techniques by
experimenting with the MNIST and CIFAR-10 datasets.
Hierarchical Scene Parsing by Weakly Supervised Learning with Image Descriptions
Authors: Ruimao Zhang, Liang Lin, Guangrun Wang, Meng Wang, WangMeng Zuo
Date: 2017-09-27 00:00:00
Tasks: Descriptive, Object, Scene Labeling, Scene Parsing, Scene Understanding, Sentence, Weakly-supervised Learning

This paper investigates a fundamental problem of scene understanding: how to
parse a scene image into a structured configuration (i.e., a semantic object
hierarchy with object interaction relations). We propose a deep architecture
consisting of two networks: i) a convolutional neural network (CNN) extracting
the image representation for pixel-wise object labeling and ii) a recursive
neural network (RsNN) discovering the hierarchical object structure and the
inter-object relations. Rather than relying on elaborative annotations (e.g.,
manually labeled semantic maps and relations), we train our deep model in a
weakly-supervised learning manner by leveraging the descriptive sentences of
the training images. Specifically, we decompose each sentence into a semantic
tree consisting of nouns and verb phrases, and apply these tree structures to
discover the configurations of the training images. Once these scene
configurations are determined, then the parameters of both the CNN and RsNN are
updated accordingly by back propagation. The entire model training is
accomplished through an Expectation-Maximization method. Extensive experiments
show that our model is capable of producing meaningful scene configurations and
achieving more favorable scene labeling results on two benchmarks (i.e., PASCAL
VOC 2012 and SYSU-Scenes) compared with other state-of-the-art
weakly-supervised deep learning methods. In particular, SYSU-Scenes contains
more than 5000 scene images with their semantic sentence descriptions, which is
created by us for advancing research on scene parsing.
Approximate Inference via Weighted Rademacher Complexity
Authors: Jonathan Kuck, Ashish Sabharwal, Stefano Ermon
Date: 2018-01-27 00:00:00
Tasks: LEMMA

Rademacher complexity is often used to characterize the learnability of a
hypothesis class and is known to be related to the class size. We leverage this
observation and introduce a new technique for estimating the size of an
arbitrary weighted set, defined as the sum of weights of all elements in the
set. Our technique provides upper and lower bounds on a novel generalization of
Rademacher complexity to the weighted setting in terms of the weighted set
size. This generalizes Massart's Lemma, a known upper bound on the Rademacher
complexity in terms of the unweighted set size. We show that the weighted
Rademacher complexity can be estimated by solving a randomly perturbed
optimization problem, allowing us to derive high-probability bounds on the size
of any weighted set. We apply our method to the problems of calculating the
partition function of an Ising model and computing propositional model counts
(#SAT). Our experiments demonstrate that we can produce tighter bounds than
competing methods in both the weighted and unweighted settings.
Plug-and-Play CNN for Crowd Motion Analysis: An Application in Abnormal Event Detection
Authors: Mahdyar Ravanbakhsh, Moin Nabi, Hossein Mousavi, Enver Sangineto, Nicu Sebe
Date: 2016-10-02 00:00:00
Tasks: Anomaly Detection, Event Detection, Optical Flow Estimation

Most of the crowd abnormal event detection methods rely on complex
hand-crafted features to represent the crowd motion and appearance.
Convolutional Neural Networks (CNN) have shown to be a powerful tool with
excellent representational capacities, which can leverage the need for
hand-crafted features. In this paper, we show that keeping track of the changes
in the CNN feature across time can facilitate capturing the local abnormality.
We specifically propose a novel measure-based method which allows measuring the
local abnormality in a video by combining semantic information (inherited from
existing CNN models) with low-level Optical-Flow. One of the advantage of this
method is that it can be used without the fine-tuning costs. The proposed
method is validated on challenging abnormality detection datasets and the
results show the superiority of our method compared to the state-of-the-art
methods.
A Two-point Method for PTZ Camera Calibration in Sports
Authors: Jianhui Chen, Fangrui Zhu, James J. Little
Date: 2018-01-26 00:00:00
Tasks: Camera Calibration, Vocal Bursts Valence Prediction

Calibrating narrow field of view soccer cameras is challenging because there
are very few field markings in the image. Unlike previous solutions, we propose
a two-point method, which requires only two point correspondences given the
prior knowledge of base location and orientation of a pan-tilt-zoom (PTZ)
camera. We deploy this new calibration method to annotate pan-tilt-zoom data
from soccer videos. The collected data are used as references for new images.
We also propose a fast random forest method to predict pan-tilt angles without
image-to-image feature matching, leading to an efficient calibration method for
new images. We demonstrate our system on synthetic data and two real soccer
datasets. Our two-point approach achieves superior performance over the
state-of-the-art method.
Vocabulary-informed Extreme Value Learning
Authors: Yanwei Fu, Hanze Dong, Yu-feng Ma, Zhengjun Zhang, xiangyang xue
Date: 2017-05-28 00:00:00
Tasks: Open Set Learning

The novel unseen classes can be formulated as the extreme values of known
classes. This inspired the recent works on open-set recognition
\cite{Scheirer_2013_TPAMI,Scheirer_2014_TPAMIb,EVM}, which however can have no
way of naming the novel unseen classes. To solve this problem, we propose the
Extreme Value Learning (EVL) formulation to learn the mapping from visual
feature to semantic space. To model the margin and coverage distributions of
each class, the Vocabulary-informed Learning (ViL) is adopted by using vast
open vocabulary in the semantic space. Essentially, by incorporating the EVL
and ViL, we for the first time propose a novel semantic embedding paradigm --
Vocabulary-informed Extreme Value Learning (ViEVL), which embeds the visual
features into semantic space in a probabilistic way. The learned embedding can
be directly used to solve supervised learning, zero-shot and open set
recognition simultaneously. Experiments on two benchmark datasets demonstrate
the effectiveness of proposed frameworks.
A Simple Theoretical Model of Importance for Summarization
Authors: Maxime Peyrard
Date: 2018-01-26 00:00:00
Proceeding: ACL 2019 7
Tasks: Informativeness, model

Research on summarization has mainly been driven by empirical approaches, crafting systems to perform well on standard datasets with the notion of information Importance remaining latent. We argue that establishing theoretical models of Importance will advance our understanding of the task and help to further improve summarization systems. To this end, we propose simple but rigorous definitions of several concepts that were previously used only intuitively in summarization: Redundancy, Relevance, and Informativeness. Importance arises as a single quantity naturally unifying these concepts. Additionally, we provide intuitions to interpret the proposed quantities and experiments to demonstrate the potential of the framework to inform and guide subsequent works.
Underdamped Langevin MCMC: A non-asymptotic analysis
Authors: Xiang Cheng, Niladri S. Chatterji, Peter L. Bartlett, Michael. I. Jordan
Date: 2017-07-12 00:00:00

We study the underdamped Langevin diffusion when the log of the target
distribution is smooth and strongly concave. We present a MCMC algorithm based
on its discretization and show that it achieves $\varepsilon$ error (in
2-Wasserstein distance) in $\mathcal{O}(\sqrt{d}/\varepsilon)$ steps. This is a
significant improvement over the best known rate for overdamped Langevin MCMC,
which is $\mathcal{O}(d/\varepsilon^2)$ steps under the same
smoothness/concavity assumptions.
  The underdamped Langevin MCMC scheme can be viewed as a version of
Hamiltonian Monte Carlo (HMC) which has been observed to outperform overdamped
Langevin MCMC methods in a number of application areas. We provide quantitative
rates that support this empirical wisdom.
Object category learning and retrieval with weak supervision
Authors: Steven Hickson, Anelia Angelova, Irfan Essa, Rahul Sukthankar
Date: 2018-01-26 00:00:00
Tasks: Clustering, Deep Clustering, Object, Retrieval

We consider the problem of retrieving objects from image data and learning to
classify them into meaningful semantic categories with minimal supervision. To
that end, we propose a fully differentiable unsupervised deep clustering
approach to learn semantic classes in an end-to-end fashion without individual
class labeling using only unlabeled object proposals. The key contributions of
our work are 1) a kmeans clustering objective where the clusters are learned as
parameters of the network and are represented as memory units, and 2)
simultaneously building a feature representation, or embedding, while learning
to cluster it. This approach shows promising results on two popular computer
vision datasets: on CIFAR10 for clustering objects, and on the more complex and
challenging Cityscapes dataset for semantically discovering classes which
visually correspond to cars, people, and bicycles. Currently, the only
supervision provided is segmentation objectness masks, but this method can be
extended to use an unsupervised objectness-based object generation mechanism
which will make the approach completely unsupervised.
Efficient Hierarchical Graph-Based Segmentation of RGBD Videos
Authors: Steven Hickson, Stan Birchfield, Irfan Essa, Henrik Christensen
Date: 2018-01-26 00:00:00
Proceeding: CVPR 2014 6
Tasks: Clustering, Graph Matching, Segmentation, Video Segmentation

We present an efficient and scalable algorithm for segmenting 3D RGBD point
clouds by combining depth, color, and temporal information using a multistage,
hierarchical graph-based approach. Our algorithm processes a moving window over
several point clouds to group similar regions over a graph, resulting in an
initial over-segmentation. These regions are then merged to yield a dendrogram
using agglomerative clustering via a minimum spanning tree algorithm. Bipartite
graph matching at a given level of the hierarchical tree yields the final
segmentation of the point clouds by maintaining region identities over
arbitrarily long periods of time. We show that a multistage segmentation with
depth then color yields better results than a linear combination of depth and
color. Due to its incremental processing, our algorithm can process videos of
any length and in a streaming pipeline. The algorithm's ability to produce
robust, efficient segmentation is demonstrated with numerous experimental
results on challenging sequences from our own as well as public RGBD data sets.
Interpretable Categorization of Heterogeneous Time Series Data
Authors: Ritchie Lee, Mykel J. Kochenderfer, Ole J. Mengshoel, Joshua Silbermann
Date: 2017-08-30 00:00:00
Tasks: Clustering, Collision Avoidance, Time Series, Time Series Analysis

Understanding heterogeneous multivariate time series data is important in
many applications ranging from smart homes to aviation. Learning models of
heterogeneous multivariate time series that are also human-interpretable is
challenging and not adequately addressed by the existing literature. We propose
grammar-based decision trees (GBDTs) and an algorithm for learning them. GBDTs
extend decision trees with a grammar framework. Logical expressions derived
from a context-free grammar are used for branching in place of simple
thresholds on attributes. The added expressivity enables support for a wide
range of data types while retaining the interpretability of decision trees. In
particular, when a grammar based on temporal logic is used, we show that GBDTs
can be used for the interpretable classi cation of high-dimensional and
heterogeneous time series data. Furthermore, we show how GBDTs can also be used
for categorization, which is a combination of clustering and generating
interpretable explanations for each cluster. We apply GBDTs to analyze the
classic Australian Sign Language dataset as well as data on near mid-air
collisions (NMACs). The NMAC data comes from aircraft simulations used in the
development of the next-generation Airborne Collision Avoidance System (ACAS
X).
DeepWheat: Estimating Phenotypic Traits from Crop Images with Deep Learning
Authors: Shubhra Aich, Anique Josuttes, Ilya Ovsyannikov, Keegan Strueby, Imran Ahmed, Hema Sudhakar Duddu, Curtis Pozniak, Steve Shirtliffe, Ian Stavness
Date: 2017-09-30 00:00:00
Tasks: Deep Learning

In this paper, we investigate estimating emergence and biomass traits from
color images and elevation maps of wheat field plots. We employ a
state-of-the-art deconvolutional network for segmentation and convolutional
architectures, with residual and Inception-like layers, to estimate traits via
high dimensional nonlinear regression. Evaluation was performed on two
different species of wheat, grown in field plots for an experimental plant
breeding study. Our framework achieves satisfactory performance with mean and
standard deviation of absolute difference of 1.05 and 1.40 counts for emergence
and 1.45 and 2.05 for biomass estimation. Our results for counting wheat plants
from field images are better than the accuracy reported for the similar, but
arguably less difficult, task of counting leaves from indoor images of rosette
plants. Our results for biomass estimation, even with a very small dataset,
improve upon all previously proposed approaches in the literature.
Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
Authors: Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, Thomas Griffiths
Date: 2018-01-26 00:00:00
Proceeding: ICLR 2018 1
Tasks: Meta-Learning

Meta-learning allows an intelligent agent to leverage prior learning episodes
as a basis for quickly improving performance on a novel task. Bayesian
hierarchical modeling provides a theoretical framework for formalizing
meta-learning as inference for a set of parameters that are shared across
tasks. Here, we reformulate the model-agnostic meta-learning algorithm (MAML)
of Finn et al. (2017) as a method for probabilistic inference in a hierarchical
Bayesian model. In contrast to prior methods for meta-learning via hierarchical
Bayes, MAML is naturally applicable to complex function approximators through
its use of a scalable gradient descent procedure for posterior inference.
Furthermore, the identification of MAML as hierarchical Bayes provides a way to
understand the algorithm's operation as a meta-learning procedure, as well as
an opportunity to make use of computational strategies for efficient inference.
We use this opportunity to propose an improvement to the MAML algorithm that
makes use of techniques from approximate inference and curvature estimation.