pwc-archive/papers-with-abstracts huggingface dataset:
PatternNet: Visual Pattern Mining with Deep Neural Network
Authors: Hongzhi Li, Joseph G. Ellis, Lei Zhang, Shih-Fu Chang
Date: 2017-03-18 00:00:00
Tasks: image-classification, Image Classification

Visual patterns represent the discernible regularity in the visual world.
They capture the essential nature of visual objects or scenes. Understanding
and modeling visual patterns is a fundamental problem in visual recognition
that has wide ranging applications. In this paper, we study the problem of
visual pattern mining and propose a novel deep neural network architecture
called PatternNet for discovering these patterns that are both discriminative
and representative. The proposed PatternNet leverages the filters in the last
convolution layer of a convolutional neural network to find locally consistent
visual patches, and by combining these filters we can effectively discover
unique visual patterns. In addition, PatternNet can discover visual patterns
efficiently without performing expensive image patch sampling, and this
advantage provides an order of magnitude speedup compared to most other
approaches. We evaluate the proposed PatternNet subjectively by showing
randomly selected visual patterns which are discovered by our method and
quantitatively by performing image classification with the identified visual
patterns and comparing our performance with the current state-of-the-art. We
also directly evaluate the quality of the discovered visual patterns by
leveraging the identified patterns as proposed objects in an image and compare
with other relevant methods. Our proposed network and procedure, PatterNet, is
able to outperform competing methods for the tasks described.
Automatic formation of the structure of abstract machines in hierarchical reinforcement learning with state clustering
Authors: Aleksandr I. Panov, Aleksey Skrynnik
Date: 2018-06-13 00:00:00
Tasks: Clustering, Hierarchical Reinforcement Learning, Q-Learning, Reinforcement Learning

We introduce a new approach to hierarchy formation and task decomposition in
hierarchical reinforcement learning. Our method is based on the Hierarchy Of
Abstract Machines (HAM) framework because HAM approach is able to design
efficient controllers that will realize specific behaviors in real robots. The
key to our algorithm is the introduction of the internal or "mental"
environment in which the state represents the structure of the HAM hierarchy.
The internal action in this environment leads to changes the hierarchy of HAMs.
We propose the classical Q-learning procedure in the internal environment which
allows the agent to obtain an optimal hierarchy. We extends the HAM framework
by adding on-model approach to select the appropriate sub-machine to execute
action sequences for certain class of external environment states. Preliminary
experiments demonstrated the prospects of the method.
Natural Language to Structured Query Generation via Meta-Learning
Authors: Po-Sen Huang, Chenglong Wang, Rishabh Singh, Wen-tau Yih, Xiaodong He
Date: 2018-03-02 00:00:00
Proceeding: NAACL 2018 6
Tasks: Meta-Learning

In conventional supervised training, a model is trained to fit all the
training examples. However, having a monolithic model may not always be the
best strategy, as examples could vary widely. In this work, we explore a
different learning protocol that treats each example as a unique pseudo-task,
by reducing the original learning problem to a few-shot meta-learning scenario
with the help of a domain-dependent relevance function. When evaluated on the
WikiSQL dataset, our approach leads to faster convergence and achieves
1.1%-5.4% absolute accuracy gains over the non-meta-learning counterparts.
A Flexible Convolutional Solver with Application to Photorealistic Style Transfer
Authors: Gilles Puy, Patrick Pérez
Date: 2018-06-13 00:00:00
Tasks: Rolling Shutter Correction, Style Transfer

We propose a new flexible deep convolutional neural network (convnet) to
perform fast visual style transfer. In contrast to existing convnets that
address the same task, our architecture derives directly from the structure of
the gradient descent originally used to solve the style transfer problem [Gatys
et al., 2016]. Like existing convnets, ours approximately solves the original
problem much faster than the gradient descent. However, our network is uniquely
flexible by design: it can be manipulated at runtime to enforce new constraints
on the final solution. In particular, we show how to modify it to obtain a
photorealistic result with no retraining. We study the modifications made by
[Luan et al., 2017] to the original cost function of [Gatys et al., 2016] to
achieve photorealistic style transfer. These modifications affect directly the
gradient descent and can be reported on-the-fly in our network. These
modifications are possible as the proposed architecture stems from unrolling
the gradient descent.
How Predictable is Your State? Leveraging Lexical and Contextual Information for Predicting Legislative Floor Action at the State Level
Authors: Vlad Eidelman, Anastassia Kornilova, Daniel Argyle
Date: 2018-06-13 00:00:00
Proceeding: COLING 2018 8

Modeling U.S. Congressional legislation and roll-call votes has received
significant attention in previous literature. However, while legislators across
50 state governments and D.C. propose over 100,000 bills each year, and on
average enact over 30% of them, state level analysis has received relatively
less attention due in part to the difficulty in obtaining the necessary data.
Since each state legislature is guided by their own procedures, politics and
issues, however, it is difficult to qualitatively asses the factors that affect
the likelihood of a legislative initiative succeeding. Herein, we present
several methods for modeling the likelihood of a bill receiving floor action
across all 50 states and D.C. We utilize the lexical content of over 1 million
bills, along with contextual legislature and legislator derived features to
build our predictive models, allowing a comparison of the factors that are
important to the lawmaking process. Furthermore, we show that these signals
hold complementary predictive power, together achieving an average improvement
in accuracy of 18% over state specific baselines.
Solving the Steiner Tree Problem in graphs with Variable Neighborhood Descent
Authors: Matthieu De Laere, San Tu Pham, Patrick De Causmaecker
Date: 2018-06-13 00:00:00
Tasks: Steiner Tree Problem

The Steiner Tree Problem (STP) in graphs is an important problem with various
applications in many areas such as design of integrated circuits, evolution
theory, networking, etc. In this paper, we propose an algorithm to solve the
STP. The algorithm includes a reducer and a solver using Variable Neighborhood
Descent (VND), interacting with each other during the search. New constructive
heuristics and a vertex score system for intensification purpose are proposed.
The algorithm is tested on a set of benchmarks which shows encouraging results.
Learning Privacy Preserving Encodings through Adversarial Training
Authors: Francesco Pittaluga, Sanjeev J. Koppal, Ayan Chakrabarti
Date: 2018-02-14 00:00:00
Tasks: Attribute, Privacy Preserving

We present a framework to learn privacy-preserving encodings of images that
inhibit inference of chosen private attributes, while allowing recovery of
other desirable information. Rather than simply inhibiting a given fixed
pre-trained estimator, our goal is that an estimator be unable to learn to
accurately predict the private attributes even with knowledge of the encoding
function. We use a natural adversarial optimization-based formulation for
this---training the encoding function against a classifier for the private
attribute, with both modeled as deep neural networks. The key contribution of
our work is a stable and convergent optimization approach that is successful at
learning an encoder with our desired properties---maintaining utility while
inhibiting inference of private attributes, not just within the adversarial
optimization, but also by classifiers that are trained after the encoder is
fixed. We adopt a rigorous experimental protocol for verification wherein
classifiers are trained exhaustively till saturation on the fixed encoders. We
evaluate our approach on tasks of real-world complexity---learning
high-dimensional encodings that inhibit detection of different scene
categories---and find that it yields encoders that are resilient at maintaining
privacy.
Online learning over a finite action set with limited switching
Authors: Jason Altschuler, Kunal Talwar
Date: 2018-03-05 00:00:00
Tasks: Multi-Armed Bandits

This paper studies the value of switching actions in the Prediction From
Experts (PFE) problem and Adversarial Multi-Armed Bandits (MAB) problem. First,
we revisit the well-studied and practically motivated setting of PFE with
switching costs. Many algorithms are known to achieve the minimax optimal order
of $O(\sqrt{T \log n})$ in expectation for both regret and number of switches,
where $T$ is the number of iterations and $n$ the number of actions. However,
no high probability (h.p.) guarantees are known. Our main technical
contribution is the first algorithms which with h.p. achieve this optimal order
for both regret and switches. This settles an open problem of [Devroye et al.,
2015], and directly implies the first h.p. guarantees for several problems of
interest.
  Next, to investigate the value of switching actions at a more granular level,
we introduce the setting of switching budgets, in which algorithms are limited
to $S \leq T$ switches between actions. This entails a limited number of free
switches, in contrast to the unlimited number of expensive switches in the
switching cost setting. Using the above result and several reductions, we unify
previous work and completely characterize the complexity of this switching
budget setting up to small polylogarithmic factors: for both PFE and MAB, for
all switching budgets $S \leq T$, and for both expectation and h.p. guarantees.
For PFE, we show the optimal rate is $\tilde{\Theta}(\sqrt{T\log n})$ for $S =
\Omega(\sqrt{T\log n})$, and $\min(\tilde{\Theta}(\tfrac{T\log n}{S}), T)$ for
$S = O(\sqrt{T \log n})$. Interestingly, the bandit setting does not exhibit
such a phase transition; instead we show the minimax rate decays steadily as
$\min(\tilde{\Theta}(\tfrac{T\sqrt{n}}{\sqrt{S}}), T)$ for all ranges of $S
\leq T$. These results recover and generalize the known minimax rates for the
(arbitrary) switching cost setting.
Benchmarks for Image Classification and Other High-dimensional Pattern Recognition Problems
Authors: Tarun Yellamraju, Jonas Hepp, Mireille Boutin
Date: 2018-06-13 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Vocal Bursts Intensity Prediction

A good classification method should yield more accurate results than simple
heuristics. But there are classification problems, especially high-dimensional
ones like the ones based on image/video data, for which simple heuristics can
work quite accurately; the structure of the data in such problems is easy to
uncover without any sophisticated or computationally expensive method. On the
other hand, some problems have a structure that can only be found with
sophisticated pattern recognition methods. We are interested in quantifying the
difficulty of a given high-dimensional pattern recognition problem. We consider
the case where the patterns come from two pre-determined classes and where the
objects are represented by points in a high-dimensional vector space. However,
the framework we propose is extendable to an arbitrarily large number of
classes. We propose classification benchmarks based on simple random projection
heuristics. Our benchmarks are 2D curves parameterized by the classification
error and computational cost of these simple heuristics. Each curve divides the
plane into a "positive- gain" and a "negative-gain" region. The latter contains
methods that are ill-suited for the given classification problem. The former is
divided into two by the curve asymptote; methods that lie in the small region
under the curve but right of the asymptote merely provide a computational gain
but no structural advantage over the random heuristics. We prove that the curve
asymptotes are optimal (i.e. at Bayes error) in some cases, and thus no
sophisticated method can provide a structural advantage over the random
heuristics. Such classification problems, an example of which we present in our
numerical experiments, provide poor ground for testing new pattern
classification methods.
Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors
Authors: Soumya Ghosh, Jiayu Yao, Finale Doshi-Velez
Date: 2018-06-13 00:00:00
Proceeding: ICML 2018 7
Tasks: Model Selection, Open-Ended Question Answering, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Bayesian Neural Networks (BNNs) have recently received increasing attention
for their ability to provide well-calibrated posterior uncertainties. However,
model selection---even choosing the number of nodes---remains an open question.
Recent work has proposed the use of a horseshoe prior over node pre-activations
of a Bayesian neural network, which effectively turns off nodes that do not
help explain the data. In this work, we propose several modeling and inference
advances that consistently improve the compactness of the model learned while
maintaining predictive performance, especially in smaller-sample settings
including reinforcement learning.
Online Self-supervised Scene Segmentation for Micro Aerial Vehicles
Authors: Shreyansh Daftry, Yashasvi Agrawal, Larry Matthies
Date: 2018-06-13 00:00:00
Tasks: Scene Segmentation, Scene Understanding

Recently, there have been numerous advances in the development of payload and
power constrained lightweight Micro Aerial Vehicles (MAVs). As these robots
aspire for high-speed autonomous flights in complex dynamic environments,
robust scene understanding at long-range becomes critical. The problem is
heavily characterized by either the limitations imposed by sensor capabilities
for geometry-based methods, or the need for large-amounts of manually annotated
training data required by data-driven methods. This motivates the need to build
systems that have the capability to alleviate these problems by exploiting the
complimentary strengths of both geometry and data-driven methods. In this
paper, we take a step in this direction and propose a generic framework for
adaptive scene segmentation using self-supervised online learning. We present
this in the context of vision-based autonomous MAV flight, and demonstrate the
efficacy of our proposed system through extensive experiments on benchmark
datasets and real-world field tests.
Improving Robustness of ML Classifiers against Realizable Evasion Attacks Using Conserved Features
Authors: Liang Tong, Bo Li, Chen Hajaj, Chaowei Xiao, Ning Zhang, Yevgeniy Vorobeychik
Date: 2017-08-28 00:00:00
Tasks: Intrusion Detection, Malware Detection

Machine learning (ML) techniques are increasingly common in security applications, such as malware and intrusion detection. However, ML models are often susceptible to evasion attacks, in which an adversary makes changes to the input (such as malware) in order to avoid being detected. A conventional approach to evaluate ML robustness to such attacks, as well as to design robust ML, is by considering simplified feature-space models of attacks, where the attacker changes ML features directly to effect evasion, while minimizing or constraining the magnitude of this change. We investigate the effectiveness of this approach to designing robust ML in the face of attacks that can be realized in actual malware (realizable attacks). We demonstrate that in the context of structure-based PDF malware detection, such techniques appear to have limited effectiveness, but they are effective with content-based detectors. In either case, we show that augmenting the feature space models with conserved features (those that cannot be unilaterally modified without compromising malicious functionality) significantly improves performance. Finally, we show that feature space models enable generalized robustness when faced with a variety of realizable attacks, as compared to classifiers which are tuned to be robust to a specific realizable attack.
On the Connection Between Learning Two-Layers Neural Networks and Tensor Decomposition
Authors: Marco Mondelli, Andrea Montanari
Date: 2018-02-20 00:00:00
Tasks: Tensor Decomposition

We establish connections between the problem of learning a two-layer neural
network and tensor decomposition. We consider a model with feature vectors
$\boldsymbol x \in \mathbb R^d$, $r$ hidden units with weights $\{\boldsymbol
w_i\}_{1\le i \le r}$ and output $y\in \mathbb R$, i.e., $y=\sum_{i=1}^r
\sigma( \boldsymbol w_i^{\mathsf T}\boldsymbol x)$, with activation functions
given by low-degree polynomials. In particular, if $\sigma(x) =
a_0+a_1x+a_3x^3$, we prove that no polynomial-time learning algorithm can
outperform the trivial predictor that assigns to each example the response
variable $\mathbb E(y)$, when $d^{3/2}\ll r\ll d^2$. Our conclusion holds for a
`natural data distribution', namely standard Gaussian feature vectors
$\boldsymbol x$, and output distributed according to a two-layer neural network
with random isotropic weights, and under a certain complexity-theoretic
assumption on tensor decomposition. Roughly speaking, we assume that no
polynomial-time algorithm can substantially outperform current methods for
tensor decomposition based on the sum-of-squares hierarchy.
  We also prove generalizations of this statement for higher degree polynomial
activations, and non-random weight vectors. Remarkably, several existing
algorithms for learning two-layer networks with rigorous guarantees are based
on tensor decomposition. Our results support the idea that this is indeed the
core computational difficulty in learning such networks, under the stated
generative model for the data. As a side result, we show that under this model
learning the network requires accurate learning of its weights, a property that
does not hold in a more general setting.
A comparison of methods for model selection when estimating individual treatment effects
Authors: Alejandro Schuler, Michael Baiocchi, Robert Tibshirani, Nigam Shah
Date: 2018-04-14 00:00:00
Tasks: Model Selection

Practitioners in medicine, business, political science, and other fields are
increasingly aware that decisions should be personalized to each patient,
customer, or voter. A given treatment (e.g. a drug or advertisement) should be
administered only to those who will respond most positively, and certainly not
to those who will be harmed by it. Individual-level treatment effects can be
estimated with tools adapted from machine learning, but different models can
yield contradictory estimates. Unlike risk prediction models, however,
treatment effect models cannot be easily evaluated against each other using a
held-out test set because the true treatment effect itself is never directly
observed. Besides outcome prediction accuracy, several metrics that can
leverage held-out data to evaluate treatment effects models have been proposed,
but they are not widely used. We provide a didactic framework that elucidates
the relationships between the different approaches and compare them all using a
variety of simulations of both randomized and observational data. Our results
show that researchers estimating heterogenous treatment effects need not limit
themselves to a single model-fitting algorithm. Instead of relying on a single
method, multiple models fit by a diverse set of algorithms should be evaluated
against each other using an objective function learned from the validation set.
The model minimizing that objective should be used for estimating the
individual treatment effect for future individuals.
Embarrassingly Parallel Inference for Gaussian Processes
Authors: Michael Minyi Zhang, Sinead A. Williamson
Date: 2017-02-27 00:00:00
Tasks: Gaussian Processes, Mixture-of-Experts, regression

Training Gaussian process-based models typically involves an $ O(N^3)$ computational bottleneck due to inverting the covariance matrix. Popular methods for overcoming this matrix inversion problem cannot adequately model all types of latent functions, and are often not parallelizable. However, judicious choice of model structure can ameliorate this problem. A mixture-of-experts model that uses a mixture of $K$ Gaussian processes offers modeling flexibility and opportunities for scalable inference. Our embarrassingly parallel algorithm combines low-dimensional matrix inversions with importance sampling to yield a flexible, scalable mixture-of-experts model that offers comparable performance to Gaussian process regression at a much lower computational cost.
SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions
Authors: Arman Cohan, Bart Desmet, Andrew Yates, Luca Soldaini, Sean MacAvaney, Nazli Goharian
Date: 2018-06-13 00:00:00
Proceeding: COLING 2018 8
Tasks: text-classification, Text Classification

Mental health is a significant and growing public health concern. As language
usage can be leveraged to obtain crucial insights into mental health
conditions, there is a need for large-scale, labeled, mental health-related
datasets of users who have been diagnosed with one or more of such conditions.
In this paper, we investigate the creation of high-precision patterns to
identify self-reported diagnoses of nine different mental health conditions,
and obtain high-quality labeled data without the need for manual labelling. We
introduce the SMHD (Self-reported Mental Health Diagnoses) dataset and make it
available. SMHD is a novel large dataset of social media posts from users with
one or multiple mental health conditions along with matched control users. We
examine distinctions in users' language, as measured by linguistic and
psychological variables. We further explore text classification methods to
identify individuals with mental conditions through their language.
Finding your Lookalike: Measuring Face Similarity Rather than Face Identity
Authors: Amir Sadovnik, Wassim Gharbi, Thanh Vu, Andrew Gallagher
Date: 2018-06-13 00:00:00
Tasks: Face Recognition, General Classification

Face images are one of the main areas of focus for computer vision, receiving
on a wide variety of tasks. Although face recognition is probably the most
widely researched, many other tasks such as kinship detection, facial
expression classification and facial aging have been examined. In this work we
propose the new, subjective task of quantifying perceived face similarity
between a pair of faces. That is, we predict the perceived similarity between
facial images, given that they are not of the same person. Although this task
is clearly correlated with face recognition, it is different and therefore
justifies a separate investigation. Humans often remark that two persons look
alike, even in cases where the persons are not actually confused with one
another. In addition, because face similarity is different than traditional
image similarity, there are challenges in data collection and labeling, and
dealing with diverging subjective opinions between human labelers. We present
evidence that finding facial look-alikes and recognizing faces are two distinct
tasks. We propose a new dataset for facial similarity and introduce the
Lookalike network, directed towards similar face classification, which
outperforms the ad hoc usage of a face recognition network directed at the same
task.