pwc-archive/papers-with-abstracts huggingface dataset:
Transitive Invariance for Self-supervised Visual Representation Learning
Authors: Xiaolong Wang, Kaiming He, Abhinav Gupta
Date: 2017-08-09 00:00:00
Proceeding: ICCV 2017 10
Tasks: Multi-Task Learning, object-detection, Object Detection, Representation Learning, Self-Supervised Learning, Surface Normal Estimation, Triplet

Learning visual representations with self-supervised learning has become
popular in computer vision. The idea is to design auxiliary tasks where labels
are free to obtain. Most of these tasks end up providing data to learn specific
kinds of invariance useful for recognition. In this paper, we propose to
exploit different self-supervised approaches to learn representations invariant
to (i) inter-instance variations (two objects in the same class should have
similar features) and (ii) intra-instance variations (viewpoint, pose,
deformations, illumination, etc). Instead of combining two approaches with
multi-task learning, we argue to organize and reason the data with multiple
variations. Specifically, we propose to generate a graph with millions of
objects mined from hundreds of thousands of videos. The objects are connected
by two types of edges which correspond to two types of invariance: "different
instances but a similar viewpoint and category" and "different viewpoints of
the same instance". By applying simple transitivity on the graph with these
edges, we can obtain pairs of images exhibiting richer visual invariance. We
use this data to train a Triplet-Siamese network with VGG16 as the base
architecture and apply the learned representations to different recognition
tasks. For object detection, we achieve 63.2% mAP on PASCAL VOC 2007 using Fast
R-CNN (compare to 67.3% with ImageNet pre-training). For the challenging COCO
dataset, our method is surprisingly close (23.5%) to the ImageNet-supervised
counterpart (24.4%) using the Faster R-CNN framework. We also show that our
network can perform significantly better than the ImageNet network in the
surface normal estimation task.
Gradient Methods for Submodular Maximization
Authors: Hamed Hassani, Mahdi Soltanolkotabi, Amin Karbasi
Date: 2017-08-13 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Active Learning

In this paper, we study the problem of maximizing continuous submodular
functions that naturally arise in many learning applications such as those
involving utility functions in active learning and sensing, matrix
approximations and network inference. Despite the apparent lack of convexity in
such functions, we prove that stochastic projected gradient methods can provide
strong approximation guarantees for maximizing continuous submodular functions
with convex constraints. More specifically, we prove that for monotone
continuous DR-submodular functions, all fixed points of projected gradient
ascent provide a factor $1/2$ approximation to the global maxima. We also study
stochastic gradient and mirror methods and show that after
$\mathcal{O}(1/\epsilon^2)$ iterations these methods reach solutions which
achieve in expectation objective values exceeding
$(\frac{\text{OPT}}{2}-\epsilon)$. An immediate application of our results is
to maximize submodular functions that are defined stochastically, i.e. the
submodular function is defined as an expectation over a family of submodular
functions with an unknown distribution. We will show how stochastic gradient
methods are naturally well-suited for this setting, leading to a factor $1/2$
approximation when the function is monotone. In particular, it allows us to
approximately maximize discrete, monotone submodular optimization problems via
projected gradient descent on a continuous relaxation, directly connecting the
discrete and continuous domains. Finally, experiments on real data demonstrate
that our projected gradient methods consistently achieve the best utility
compared to other continuous baselines while remaining competitive in terms of
computational effort.
Reinterpreting Importance-Weighted Autoencoders
Authors: Chris Cremer, Quaid Morris, David Duvenaud
Date: 2017-04-10 00:00:00

The standard interpretation of importance-weighted autoencoders is that they
maximize a tighter lower bound on the marginal likelihood than the standard
evidence lower bound. We give an alternate interpretation of this procedure:
that it optimizes the standard variational lower bound, but using a more
complex distribution. We formally derive this result, present a tighter lower
bound, and visualize the implicit importance-weighted distribution.
Deep Edge-Aware Saliency Detection
Authors: Jing Zhang, Yuchao Dai, Fatih Porikli, Mingyi He
Date: 2017-08-15 00:00:00
Tasks: Descriptive, Saliency Detection

There has been profound progress in visual saliency thanks to the deep
learning architectures, however, there still exist three major challenges that
hinder the detection performance for scenes with complex compositions, multiple
salient objects, and salient objects of diverse scales. In particular, output
maps of the existing methods remain low in spatial resolution causing blurred
edges due to the stride and pooling operations, networks often neglect
descriptive statistical and handcrafted priors that have potential to
complement saliency detection results, and deep features at different layers
stay mainly desolate waiting to be effectively fused to handle multi-scale
salient objects. In this paper, we tackle these issues by a new fully
convolutional neural network that jointly learns salient edges and saliency
labels in an end-to-end fashion. Our framework first employs convolutional
layers that reformulate the detection task as a dense labeling problem, then
integrates handcrafted saliency features in a hierarchical manner into lower
and higher levels of the deep network to leverage available information for
multi-scale response, and finally refines the saliency map through dilated
convolutions by imposing context. In this way, the salient edge priors are
efficiently incorporated and the output resolution is significantly improved
while keeping the memory requirements low, leading to cleaner and sharper
object boundaries. Extensive experimental analyses on ten benchmarks
demonstrate that our framework achieves consistently superior performance and
attains robustness for complex scenes in comparison to the very recent
state-of-the-art approaches.
Distant Supervision for Relation Extraction beyond the Sentence Boundary
Authors: Chris Quirk, Hoifung Poon
Date: 2016-09-15 00:00:00
Proceeding: EACL 2017 4
Tasks: Articles, Relation, Relation Extraction, Sentence

The growing demand for structured knowledge has led to great interest in
relation extraction, especially in cases with limited supervision. However,
existing distance supervision approaches only extract relations expressed in
single sentences. In general, cross-sentence relation extraction is
under-explored, even in the supervised-learning setting. In this paper, we
propose the first approach for applying distant supervision to cross- sentence
relation extraction. At the core of our approach is a graph representation that
can incorporate both standard dependencies and discourse relations, thus
providing a unifying way to model relations within and across sentences. We
extract features from multiple paths in this graph, increasing accuracy and
robustness when confronted with linguistic variation and analysis error.
Experiments on an important extraction task for precision medicine show that
our approach can learn an accurate cross-sentence extractor, using only a small
existing knowledge base and unlabeled text from biomedical research articles.
Compared to the existing distant supervision paradigm, our approach extracted
twice as many relations at similar precision, thus demonstrating the prevalence
of cross-sentence relations and the promise of our approach.
Continuous Representation of Location for Geolocation and Lexical Dialectology using Mixture Density Networks
Authors: Afshin Rahimi, Timothy Baldwin, Trevor Cohn
Date: 2017-08-14 00:00:00
Proceeding: EMNLP 2017 9
Tasks: regression

We propose a method for embedding two-dimensional locations in a continuous
vector space using a neural network-based model incorporating mixtures of
Gaussian distributions, presenting two model variants for text-based
geolocation and lexical dialectology. Evaluated over Twitter data, the proposed
model outperforms conventional regression-based geolocation and provides a
better estimate of uncertainty. We also show the effectiveness of the
representation for predicting words from location in lexical dialectology, and
evaluate it using the DARE dataset.
Graph Classification via Deep Learning with Virtual Nodes
Authors: Trang Pham, Truyen Tran, Hoa Dam, Svetha Venkatesh
Date: 2017-08-14 00:00:00
Tasks: Classification, Deep Learning, General Classification, Graph Classification

Learning representation for graph classification turns a variable-size graph
into a fixed-size vector (or matrix). Such a representation works nicely with
algebraic manipulations. Here we introduce a simple method to augment an
attributed graph with a virtual node that is bidirectionally connected to all
existing nodes. The virtual node represents the latent aspects of the graph,
which are not immediately available from the attributes and local connectivity
structures. The expanded graph is then put through any node representation
method. The representation of the virtual node is then the representation of
the entire graph. In this paper, we use the recently introduced Column Network
for the expanded graph, resulting in a new end-to-end graph classification
model dubbed Virtual Column Network (VCN). The model is validated on two tasks:
(i) predicting bio-activity of chemical compounds, and (ii) finding software
vulnerability from source code. Results demonstrate that VCN is competitive
against well-established rivals.
Benchmark Environments for Multitask Learning in Continuous Domains
Authors: Peter Henderson, Wei-Di Chang, Florian Shkurti, Johanna Hansen, David Meger, Gregory Dudek
Date: 2017-08-14 00:00:00
Tasks: Lifelong learning, OpenAI Gym

As demand drives systems to generalize to various domains and problems, the
study of multitask, transfer and lifelong learning has become an increasingly
important pursuit. In discrete domains, performance on the Atari game suite has
emerged as the de facto benchmark for assessing multitask learning. However, in
continuous domains there is a lack of agreement on standard multitask
evaluation environments which makes it difficult to compare different
approaches fairly. In this work, we describe a benchmark set of tasks that we
have developed in an extendable framework based on OpenAI Gym. We run a simple
baseline using Trust Region Policy Optimization and release the framework
publicly to be expanded and used for the systematic comparison of multitask,
transfer, and lifelong learning in continuous domains.
Evolving imputation strategies for missing data in classification problems with TPOT
Authors: Unai Garciarena, Roberto Santana, Alexander Mendiburu
Date: 2017-06-04 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, Imputation, Missing Values

Missing data has a ubiquitous presence in real-life applications of machine
learning techniques. Imputation methods are algorithms conceived for restoring
missing values in the data, based on other entries in the database. The choice
of the imputation method has an influence on the performance of the machine
learning technique, e.g., it influences the accuracy of the classification
algorithm applied to the data. Therefore, selecting and applying the right
imputation method is important and usually requires a substantial amount of
human intervention. In this paper we propose the use of genetic programming
techniques to search for the right combination of imputation and classification
algorithms. We build our work on the recently introduced Python-based TPOT
library, and incorporate a heterogeneous set of imputation algorithms as part
of the machine learning pipeline search. We show that genetic programming can
automatically find increasingly better pipelines that include the most
effective combinations of imputation methods, feature pre-processing, and
classifiers for a variety of classification problems with missing data.
A Copy-Augmented Sequence-to-Sequence Architecture Gives Good Performance on Task-Oriented Dialogue
Authors: Mihail Eric, Christopher D. Manning
Date: 2017-01-15 00:00:00
Proceeding: EACL 2017 4
Tasks: Response Generation

Task-oriented dialogue focuses on conversational agents that participate in
user-initiated dialogues on domain-specific topics. In contrast to chatbots,
which simply seek to sustain open-ended meaningful discourse, existing
task-oriented agents usually explicitly model user intent and belief states.
This paper examines bypassing such an explicit representation by depending on a
latent neural embedding of state and learning selective attention to dialogue
history together with copying to incorporate relevant prior context. We
complement recent work by showing the effectiveness of simple
sequence-to-sequence neural architectures with a copy mechanism. Our model
outperforms more complex memory-augmented models by 7% in per-response
generation and is on par with the current state-of-the-art on DSTC2.
TheoSea: Marching Theory to Light
Authors: Mark A. Stalzer, Chao Ju
Date: 2017-08-14 00:00:00

There is sufficient information in the far-field of a radiating dipole
antenna to rediscover the Maxwell Equations and the wave equations of light,
including the speed of light $c.$ TheoSea is a Julia program that does this in
about a second, and the key insight is that the compactness of theories drives
the search. The program is a computational embodiment of the scientific method:
observation, consideration of candidate theories, and validation.
Fixed effects testing in high-dimensional linear mixed models
Authors: Jelena Bradic, Gerda Claeskens, Thomas Gueuning
Date: 2017-08-14 00:00:00
Tasks: Decision Making, Marketing, Model Selection, Vocal Bursts Intensity Prediction

Many scientific and engineering challenges -- ranging from pharmacokinetic
drug dosage allocation and personalized medicine to marketing mix (4Ps)
recommendations -- require an understanding of the unobserved heterogeneity in
order to develop the best decision making-processes. In this paper, we develop
a hypothesis test and the corresponding p-value for testing for the
significance of the homogeneous structure in linear mixed models. A robust
matching moment construction is used for creating a test that adapts to the
size of the model sparsity. When unobserved heterogeneity at a cluster level is
constant, we show that our test is both consistent and unbiased even when the
dimension of the model is extremely high. Our theoretical results rely on a new
family of adaptive sparse estimators of the fixed effects that do not require
consistent estimation of the random effects. Moreover, our inference results do
not require consistent model selection. We showcase that moment matching can be
extended to nonlinear mixed effects models and to generalized linear mixed
effects models. In numerical and real data experiments, we find that the
developed method is extremely accurate, that it adapts to the size of the
underlying model and is decidedly powerful in the presence of irrelevant
covariates.
Risk-Averse Multi-Armed Bandit Problems under Mean-Variance Measure
Authors: Sattar Vakili, Qing Zhao
Date: 2016-04-18 00:00:00

The multi-armed bandit problems have been studied mainly under the measure of
expected total reward accrued over a horizon of length $T$. In this paper, we
address the issue of risk in multi-armed bandit problems and develop parallel
results under the measure of mean-variance, a commonly adopted risk measure in
economics and mathematical finance. We show that the model-specific regret and
the model-independent regret in terms of the mean-variance of the reward
process are lower bounded by $\Omega(\log T)$ and $\Omega(T^{2/3})$,
respectively. We then show that variations of the UCB policy and the DSEE
policy developed for the classic risk-neutral MAB achieve these lower bounds.
Distance and Similarity Measures Effect on the Performance of K-Nearest Neighbor Classifier -- A Review
Authors: V. B. Surya Prasath, Haneen Arafat Abu Alfeilat, Ahmad B. A. Hassanat, Omar Lasassmeh, Ahmad S. Tarawneh, Mahmoud Bashir Alhasanat, Hamzeh S. Eyal Salman
Date: 2017-08-14 00:00:00

The K-nearest neighbor (KNN) classifier is one of the simplest and most common classifiers, yet its performance competes with the most complex classifiers in the literature. The core of this classifier depends mainly on measuring the distance or similarity between the tested examples and the training examples. This raises a major question about which distance measures to be used for the KNN classifier among a large number of distance and similarity measures available? This review attempts to answer this question through evaluating the performance (measured by accuracy, precision and recall) of the KNN using a large number of distance measures, tested on a number of real-world datasets, with and without adding different levels of noise. The experimental results show that the performance of KNN classifier depends significantly on the distance used, and the results showed large gaps between the performances of different distances. We found that a recently proposed non-convex distance performed the best when applied on most datasets comparing to the other tested distances. In addition, the performance of the KNN with this top performing distance degraded only about $20\%$ while the noise level reaches $90\%$, this is true for most of the distances used as well. This means that the KNN classifier using any of the top $10$ distances tolerate noise to a certain degree. Moreover, the results show that some distances are less affected by the added noise comparing to other distances.
Situation Recognition with Graph Neural Networks
Authors: Ruiyu Li, Makarand Tapaswi, Renjie Liao, Jiaya Jia, Raquel Urtasun, Sanja Fidler
Date: 2017-08-14 00:00:00
Proceeding: ICCV 2017 10
Tasks: Grounded Situation Recognition, Situation Recognition

We address the problem of recognizing situations in images. Given an image,
the task is to predict the most salient verb (action), and fill its semantic
roles such as who is performing the action, what is the source and target of
the action, etc. Different verbs have different roles (e.g. attacking has
weapon), and each role can take on many possible values (nouns). We propose a
model based on Graph Neural Networks that allows us to efficiently capture
joint dependencies between roles using neural networks defined on a graph.
Experiments with different graph connectivities show that our approach that
propagates information between roles significantly outperforms existing work,
as well as multiple baselines. We obtain roughly 3-5% improvement over previous
work in predicting the full situation. We also provide a thorough qualitative
analysis of our model and influence of different roles in the verbs.
An ELU Network with Total Variation for Image Denoising
Authors: Tianyang Wang, Zhengrui Qin, Michelle Zhu
Date: 2017-08-14 00:00:00
Tasks: Denoising, Image Denoising

In this paper, we propose a novel convolutional neural network (CNN) for
image denoising, which uses exponential linear unit (ELU) as the activation
function. We investigate the suitability by analyzing ELU's connection with
trainable nonlinear reaction diffusion model (TNRD) and residual denoising. On
the other hand, batch normalization (BN) is indispensable for residual
denoising and convergence purpose. However, direct stacking of BN and ELU
degrades the performance of CNN. To mitigate this issue, we design an
innovative combination of activation layer and normalization layer to exploit
and leverage the ELU network, and discuss the corresponding rationale.
Moreover, inspired by the fact that minimizing total variation (TV) can be
applied to image denoising, we propose a TV regularized L2 loss to evaluate the
training effect during the iterations. Finally, we conduct extensive
experiments, showing that our model outperforms some recent and popular
approaches on Gaussian denoising with specific or randomized noise levels for
both gray and color images.
Collaborative Filtering using Denoising Auto-Encoders for Market Basket Data
Authors: Andres G. Abad, Luis I. Reyes-Castro
Date: 2017-08-14 00:00:00
Tasks: Collaborative Filtering, Denoising, Navigate, Recommendation Systems

Recommender systems (RS) help users navigate large sets of items in the
search for "interesting" ones. One approach to RS is Collaborative Filtering
(CF), which is based on the idea that similar users are interested in similar
items. Most model-based approaches to CF seek to train a
machine-learning/data-mining model based on sparse data; the model is then used
to provide recommendations. While most of the proposed approaches are effective
for small-size situations, the combinatorial nature of the problem makes it
impractical for medium-to-large instances. In this work we present a novel
approach to CF that works by training a Denoising Auto-Encoder (DAE) on
corrupted baskets, i.e., baskets from which one or more items have been
removed. The DAE is then forced to learn to reconstruct the original basket
given its corrupted input. Due to recent advancements in optimization and other
technologies for training neural-network models (such as DAE), the proposed
method results in a scalable and practical approach to CF. The contribution of
this work is twofold: (1) to identify missing items in observed baskets and,
thus, directly providing a CF model; and, (2) to construct a generative model
of baskets which may be used, for instance, in simulation analysis or as part
of a more complex analytical method.
Attacking Automatic Video Analysis Algorithms: A Case Study of Google Cloud Video Intelligence API
Authors: Hossein Hosseini, Baicen Xiao, Andrew Clark, Radha Poovendran
Date: 2017-08-14 00:00:00
Tasks: Video Classification

Due to the growth of video data on Internet, automatic video analysis has
gained a lot of attention from academia as well as companies such as Facebook,
Twitter and Google. In this paper, we examine the robustness of video analysis
algorithms in adversarial settings. Specifically, we propose targeted attacks
on two fundamental classes of video analysis algorithms, namely video
classification and shot detection. We show that an adversary can subtly
manipulate a video in such a way that a human observer would perceive the
content of the original video, but the video analysis algorithm will return the
adversary's desired outputs.
  We then apply the attacks on the recently released Google Cloud Video
Intelligence API. The API takes a video file and returns the video labels
(objects within the video), shot changes (scene changes within the video) and
shot labels (description of video events over time). Through experiments, we
show that the API generates video and shot labels by processing only the first
frame of every second of the video. Hence, an adversary can deceive the API to
output only her desired video and shot labels by periodically inserting an
image into the video at the rate of one frame per second. We also show that the
pattern of shot changes returned by the API can be mostly recovered by an
algorithm that compares the histograms of consecutive frames. Based on our
equivalent model, we develop a method for slightly modifying the video frames,
in order to deceive the API into generating our desired pattern of shot
changes. We perform extensive experiments with different videos and show that
our attacks are consistently successful across videos with different
characteristics. At the end, we propose introducing randomness to video
analysis algorithms as a countermeasure to our attacks.