pwc-archive/papers-with-abstracts huggingface dataset:
URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection
Authors: Hung Le, Quang Pham, Doyen Sahoo, Steven C. H. Hoi
Date: 2018-02-09 00:00:00
Tasks: BIG-bench Machine Learning, Feature Engineering, Word Embeddings

Malicious URLs host unsolicited content and are used to perpetrate
cybercrimes. It is imperative to detect them in a timely manner. Traditionally,
this is done through the usage of blacklists, which cannot be exhaustive, and
cannot detect newly generated malicious URLs. To address this, recent years
have witnessed several efforts to perform Malicious URL Detection using Machine
Learning. The most popular and scalable approaches use lexical properties of
the URL string by extracting Bag-of-words like features, followed by applying
machine learning models such as SVMs. There are also other features designed by
experts to improve the prediction performance of the model. These approaches
suffer from several limitations: (i) Inability to effectively capture semantic
meaning and sequential patterns in URL strings; (ii) Requiring substantial
manual feature engineering; and (iii) Inability to handle unseen features and
generalize to test data. To address these challenges, we propose URLNet, an
end-to-end deep learning framework to learn a nonlinear URL embedding for
Malicious URL Detection directly from the URL. Specifically, we apply
Convolutional Neural Networks to both characters and words of the URL String to
learn the URL embedding in a jointly optimized framework. This approach allows
the model to capture several types of semantic information, which was not
possible by the existing models. We also propose advanced word-embeddings to
solve the problem of too many rare words observed in this task. We conduct
extensive experiments on a large-scale dataset and show a significant
performance gain over existing methods. We also conduct ablation studies to
evaluate the performance of various components of URLNet.
Interpretable Counting for Visual Question Answering
Authors: Alexander Trott, Caiming Xiong, Richard Socher
Date: 2017-12-23 00:00:00
Proceeding: ICLR 2018 1
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Questions that require counting a variety of objects in images remain a major
challenge in visual question answering (VQA). The most common approaches to VQA
involve either classifying answers based on fixed length representations of
both the image and question or summing fractional counts estimated from each
section of the image. In contrast, we treat counting as a sequential decision
process and force our model to make discrete choices of what to count.
Specifically, the model sequentially selects from detected objects and learns
interactions between objects that influence subsequent selections. A
distinction of our approach is its intuitive and interpretable output, as
discrete counts are automatically grounded in the image. Furthermore, our
method outperforms the state of the art architecture for VQA on multiple
metrics that evaluate counting.
Constrained Neural Style Transfer for Decorated Logo Generation
Authors: Gantugs Atarsaikhan, Brian Kenji Iwana, Seiichi Uchida
Date: 2018-03-02 00:00:00
Tasks: Style Transfer

Making decorated logos requires image editing skills, without sufficient
skills, it could be a time-consuming task. While there are many on-line web
services to make new logos, they have limited designs and duplicates can be
made. We propose using neural style transfer with clip art and text for the
creation of new and genuine logos. We introduce a new loss function based on
distance transform of the input image, which allows the preservation of the
silhouettes of text and objects. The proposed method constrains style transfer
only around the designated area. We demonstrate the characteristics of proposed
method. Finally, we show the results of logo generation with various input
images.
Autostacker: A Compositional Evolutionary Learning System
Authors: Boyuan Chen, Harvey Wu, Warren Mo, Ishanu Chattopadhyay, Hod Lipson
Date: 2018-03-02 00:00:00
Tasks: AutoML, BIG-bench Machine Learning, Hyperparameter Optimization

We introduce an automatic machine learning (AutoML) modeling architecture
called Autostacker, which combines an innovative hierarchical stacking
architecture and an Evolutionary Algorithm (EA) to perform efficient parameter
search. Neither prior domain knowledge about the data nor feature preprocessing
is needed. Using EA, Autostacker quickly evolves candidate pipelines with high
predictive accuracy. These pipelines can be used as is or as a starting point
for human experts to build on. Autostacker finds innovative combinations and
structures of machine learning models, rather than selecting a single model and
optimizing its hyperparameters. Compared with other AutoML systems on fifteen
datasets, Autostacker achieves state-of-art or competitive performance both in
terms of test accuracy and time cost.
MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving
Authors: Sauhaarda Chowdhuri, Tushar Pankaj, Karl Zipser
Date: 2017-09-16 00:00:00
Tasks: Autonomous Driving, Multi-Task Learning

Autonomous driving requires operation in different behavioral modes ranging
from lane following and intersection crossing to turning and stopping. However,
most existing deep learning approaches to autonomous driving do not consider
the behavioral mode in the training strategy. This paper describes a technique
for learning multiple distinct behavioral modes in a single deep neural network
through the use of multi-modal multi-task learning. We study the effectiveness
of this approach, denoted MultiNet, using self-driving model cars for driving
in unstructured environments such as sidewalks and unpaved roads. Using labeled
data from over one hundred hours of driving our fleet of 1/10th scale model
cars, we trained different neural networks to predict the steering angle and
driving speed of the vehicle in different behavioral modes. We show that in
each case, MultiNet networks outperform networks trained on individual modes
while using a fraction of the total number of parameters.
Matrices with Gaussian noise: optimal estimates for singular subspace perturbation
Authors: Sean O'Rourke, Van Vu, Ke Wang
Date: 2018-03-02 00:00:00
Tasks: Matrix Completion

The Davis-Kahan-Wedin $\sin \Theta$ theorem describes how the singular subspaces of a matrix change when subjected to a small perturbation. This classic result is sharp in the worst case scenario. In this paper, we prove a stochastic version of the Davis-Kahan-Wedin $\sin \Theta$ theorem when the perturbation is a Gaussian random matrix. Under certain structural assumptions, we obtain an optimal bound that significantly improves upon the classic Davis-Kahan-Wedin $\sin \Theta$ theorem. One of our key tools is a new perturbation bound for the singular values, which may be of independent interest.
Selective Inference for Change Point Detection in Multi-dimensional Sequences
Authors: Yuta Umezu, Ichiro Takeuchi
Date: 2017-06-01 00:00:00
Tasks: Change Point Detection, Selection bias

We study the problem of detecting change points (CPs) that are characterized
by a subset of dimensions in a multi-dimensional sequence. A method for
detecting those CPs can be formulated as a two-stage method: one for selecting
relevant dimensions, and another for selecting CPs. It has been difficult to
properly control the false detection probability of these CP detection methods
because selection bias in each stage must be properly corrected. Our main
contribution in this paper is to formulate a CP detection problem as a
selective inference problem, and show that exact (non-asymptotic) inference is
possible for a class of CP detection methods. We demonstrate the performances
of the proposed selective inference framework through numerical simulations and
its application to our motivating medical data analysis problem.
Meta-Learning for Semi-Supervised Few-Shot Classification
Authors: Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenenbaum, Hugo Larochelle, Richard S. Zemel
Date: 2018-03-02 00:00:00
Proceeding: ICLR 2018 1
Tasks: General Classification, Meta-Learning

In few-shot classification, we are interested in learning algorithms that
train a classifier from only a handful of labeled examples. Recent progress in
few-shot classification has featured meta-learning, in which a parameterized
model for a learning algorithm is defined and trained on episodes representing
different classification problems, each with a small labeled training set and
its corresponding test set. In this work, we advance this few-shot
classification paradigm towards a scenario where unlabeled examples are also
available within each episode. We consider two situations: one where all
unlabeled examples are assumed to belong to the same set of classes as the
labeled examples of the episode, as well as the more challenging situation
where examples from other distractor classes are also provided. To address this
paradigm, we propose novel extensions of Prototypical Networks (Snell et al.,
2017) that are augmented with the ability to use unlabeled examples when
producing prototypes. These models are trained in an end-to-end way on
episodes, to learn to leverage the unlabeled examples successfully. We evaluate
these methods on versions of the Omniglot and miniImageNet benchmarks, adapted
to this new framework augmented with unlabeled examples. We also propose a new
split of ImageNet, consisting of a large set of classes, with a hierarchical
structure. Our experiments confirm that our Prototypical Networks can learn to
improve their predictions due to unlabeled examples, much like a
semi-supervised algorithm would.
Estimation Considerations in Contextual Bandits
Authors: Maria Dimakopoulou, Zhengyuan Zhou, Susan Athey, Guido Imbens
Date: 2017-11-19 00:00:00
Tasks: Causal Inference, Econometrics, Multi-Armed Bandits

Contextual bandit algorithms are sensitive to the estimation method of the
outcome model as well as the exploration method used, particularly in the
presence of rich heterogeneity or complex outcome models, which can lead to
difficult estimation problems along the path of learning. We study a
consideration for the exploration vs. exploitation framework that does not
arise in multi-armed bandits but is crucial in contextual bandits; the way
exploration and exploitation is conducted in the present affects the bias and
variance in the potential outcome model estimation in subsequent stages of
learning. We develop parametric and non-parametric contextual bandits that
integrate balancing methods from the causal inference literature in their
estimation to make it less prone to problems of estimation bias. We provide the
first regret bound analyses for contextual bandits with balancing in the domain
of linear contextual bandits that match the state of the art regret bounds. We
demonstrate the strong practical advantage of balanced contextual bandits on a
large number of supervised learning datasets and on a synthetic example that
simulates model mis-specification and prejudice in the initial training data.
Additionally, we develop contextual bandits with simpler assignment policies by
leveraging sparse model estimation methods from the econometrics literature and
demonstrate empirically that in the early stages they can improve the rate of
learning and decrease regret.
Adversarial Dropout Regularization
Authors: Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, Kate Saenko
Date: 2017-11-05 00:00:00
Proceeding: ICLR 2018 1
Tasks: Domain Adaptation, General Classification, image-classification, Image Classification, Semantic Segmentation, Unsupervised Domain Adaptation

We present a method for transferring neural representations from label-rich
source domains to unlabeled target domains. Recent adversarial methods proposed
for this task learn to align features across domains by fooling a special
domain critic network. However, a drawback of this approach is that the critic
simply labels the generated features as in-domain or not, without considering
the boundaries between classes. This can lead to ambiguous features being
generated near class boundaries, reducing target classification accuracy. We
propose a novel approach, Adversarial Dropout Regularization (ADR), to
encourage the generator to output more discriminative features for the target
domain. Our key idea is to replace the critic with one that detects
non-discriminative features, using dropout on the classifier network. The
generator then learns to avoid these areas of the feature space and thus
creates better features. We apply our ADR approach to the problem of
unsupervised domain adaptation for image classification and semantic
segmentation tasks, and demonstrate significant improvement over the state of
the art. We also show that our approach can be used to train Generative
Adversarial Networks for semi-supervised learning.
Shampoo: Preconditioned Stochastic Tensor Optimization
Authors: Vineet Gupta, Tomer Koren, Yoram Singer
Date: 2018-02-26 00:00:00
Proceeding: ICML 2018 7
Tasks: Stochastic Optimization

Preconditioned gradient methods are among the most general and powerful tools
in optimization. However, preconditioning requires storing and manipulating
prohibitively large matrices. We describe and analyze a new structure-aware
preconditioning algorithm, called Shampoo, for stochastic optimization over
tensor spaces. Shampoo maintains a set of preconditioning matrices, each of
which operates on a single dimension, contracting over the remaining
dimensions. We establish convergence guarantees in the stochastic convex
setting, the proof of which builds upon matrix trace inequalities. Our
experiments with state-of-the-art deep learning models show that Shampoo is
capable of converging considerably faster than commonly used optimizers.
Although it involves a more complex update rule, Shampoo's runtime per step is
comparable to that of simple gradient methods such as SGD, AdaGrad, and Adam.
SD-CNN: a Shallow-Deep CNN for Improved Breast Cancer Diagnosis
Authors: Fei Gao, Teresa Wu, Jing Li, Bin Zheng, Lingxiang Ruan, Desheng Shang, Bhavika Patel
Date: 2018-03-01 00:00:00
Tasks: Diagnostic

Breast cancer is the second leading cause of cancer death among women
worldwide. Nevertheless, it is also one of the most treatable malignances if
detected early. Screening for breast cancer with digital mammography (DM) has
been widely used. However it demonstrates limited sensitivity for women with
dense breasts. An emerging technology in the field is contrast-enhanced digital
mammography (CEDM), which includes a low energy (LE) image similar to DM, and a
recombined image leveraging tumor neoangiogenesis similar to breast magnetic
resonance imaging (MRI). CEDM has shown better diagnostic accuracy than DM.
While promising, CEDM is not yet widely available across medical centers. In
this research, we propose a Shallow-Deep Convolutional Neural Network (SD-CNN)
where a shallow CNN is developed to derive "virtual" recombined images from LE
images, and a deep CNN is employed to extract novel features from LE,
recombined or "virtual" recombined images for ensemble models to classify the
cases as benign vs. cancer. To evaluate the validity of our approach, we first
develop a deep-CNN using 49 CEDM cases collected from Mayo Clinic to prove the
contributions from recombined images for improved breast cancer diagnosis (0.86
in accuracy using LE imaging vs. 0.90 in accuracy using both LE and recombined
imaging). We then develop a shallow-CNN using the same 49 CEDM cases to learn
the nonlinear mapping from LE to recombined images. Next, we use 69 DM cases
collected from the hospital located at Zhejiang University, China to generate
"virtual" recombined images. Using DM alone provides 0.91 in accuracy, whereas
SD-CNN improves the diagnostic accuracy to 0.95.
Evolutionary Generative Adversarial Networks
Authors: Chaoyue Wang, Chang Xu, Xin Yao, DaCheng Tao
Date: 2018-03-01 00:00:00
Tasks: Diversity

Generative adversarial networks (GAN) have been effective for learning
generative models for real-world data. However, existing GANs (GAN and its
variants) tend to suffer from training problems such as instability and mode
collapse. In this paper, we propose a novel GAN framework called evolutionary
generative adversarial networks (E-GAN) for stable GAN training and improved
generative performance. Unlike existing GANs, which employ a pre-defined
adversarial objective function alternately training a generator and a
discriminator, we utilize different adversarial training objectives as mutation
operations and evolve a population of generators to adapt to the environment
(i.e., the discriminator). We also utilize an evaluation mechanism to measure
the quality and diversity of generated samples, such that only well-performing
generator(s) are preserved and used for further training. In this way, E-GAN
overcomes the limitations of an individual adversarial training objective and
always preserves the best offspring, contributing to progress in and the
success of GANs. Experiments on several datasets demonstrate that E-GAN
achieves convincing generative performance and reduces the training problems
inherent in existing GANs.
Semi-parametric Topological Memory for Navigation
Authors: Nikolay Savinov, Alexey Dosovitskiy, Vladlen Koltun
Date: 2018-03-01 00:00:00
Proceeding: ICLR 2018 1
Tasks: Navigate

We introduce a new memory architecture for navigation in previously unseen
environments, inspired by landmark-based navigation in animals. The proposed
semi-parametric topological memory (SPTM) consists of a (non-parametric) graph
with nodes corresponding to locations in the environment and a (parametric)
deep network capable of retrieving nodes from the graph based on observations.
The graph stores no metric information, only connectivity of locations
corresponding to the nodes. We use SPTM as a planning module in a navigation
system. Given only 5 minutes of footage of a previously unseen maze, an
SPTM-based navigation agent can build a topological map of the environment and
use it to confidently navigate towards goals. The average success rate of the
SPTM agent in goal-directed navigation across test environments is higher than
the best-performing baseline by a factor of three. A video of the agent is
available at https://youtu.be/vRF7f4lhswo
Kernel Embedding Approaches to Orbit Determination of Spacecraft Clusters
Authors: Srinagesh Sharma, James W. Cutler
Date: 2018-03-01 00:00:00
Tasks: Domain Generalization

This paper presents a novel formulation and solution of orbit determination
over finite time horizons as a learning problem. We present an approach to
orbit determination under very broad conditions that are satisfied for n-body
problems. These weak conditions allow us to perform orbit determination with
noisy and highly non-linear observations such as those presented by range-rate
only (Doppler only) observations. We show that domain generalization and
distribution regression techniques can learn to estimate orbits of a group of
satellites and identify individual satellites especially with prior
understanding of correlations between orbits and provide asymptotic convergence
conditions. The approach presented requires only visibility and observability
of the underlying state from observations and is particularly useful for
autonomous spacecraft operations using low-cost ground stations or sensors. We
validate the orbit determination approach using observations of two spacecraft
(GRIFEX and MCubed-2) along with synthetic datasets of multiple spacecraft
deployments and lunar orbits. We also provide a comparison with the standard
techniques (EKF) under highly noisy conditions.
Specialized Support Vector Machines for Open-set Recognition
Authors: Pedro Ribeiro Mendes Júnior, Terrance E. Boult, Jacques Wainer, Anderson Rocha
Date: 2016-06-13 00:00:00
Tasks: General Classification, Open Set Learning, Robust classification

Often, when dealing with real-world recognition problems, we do not need, and often cannot have, knowledge of the entire set of possible classes that might appear during operational testing. In such cases, we need to think of robust classification methods able to deal with the "unknown" and properly reject samples belonging to classes never seen during training. Notwithstanding, almost all existing classifiers to date were mostly developed for the closed-set scenario, i.e., the classification setup in which it is assumed that all test samples belong to one of the classes with which the classifier was trained. In the open-set scenario, however, a test sample can belong to none of the known classes and the classifier must properly reject it by classifying it as unknown. In this work, we extend upon the well-known Support Vector Machines (SVM) classifier and introduce the Specialized Support Vector Machines (SSVM), which is suitable for recognition in open-set setups. SSVM balances the empirical risk and the risk of the unknown and ensures that the region of the feature space in which a test sample would be classified as known (one of the known classes) is always bounded, ensuring a finite risk of the unknown. In this work, we also highlight the properties of the SVM classifier related to the open-set scenario, and provide necessary and sufficient conditions for an RBF SVM to have bounded open-space risk.
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for Non-Convex Optimization
Authors: Yi Xu, Rong Jin, Tianbao Yang
Date: 2017-12-04 00:00:00
Tasks: Open-Ended Question Answering

Accelerated gradient (AG) methods are breakthroughs in convex optimization,
improving the convergence rate of the gradient descent method for optimization
with smooth functions. However, the analysis of AG methods for non-convex
optimization is still limited. It remains an open question whether AG methods
from convex optimization can accelerate the convergence of the gradient descent
method for finding local minimum of non-convex optimization problems. This
paper provides an affirmative answer to this question. In particular, we
analyze two renowned variants of AG methods (namely Polyak's Heavy Ball method
and Nesterov's Accelerated Gradient method) for extracting the negative
curvature from random noise, which is central to escaping from saddle points.
By leveraging the proposed AG methods for extracting the negative curvature, we
present a new AG algorithm with double loops for non-convex
optimization~\footnote{this is in contrast to a single-loop AG algorithm
proposed in a recent manuscript~\citep{AGNON}, which directly analyzed the
Nesterov's AG method for non-convex optimization and appeared online on
November 29, 2017. However, we emphasize that our work is an independent work,
which is inspired by our earlier work~\citep{NEON17} and is based on a
different novel analysis.}, which converges to second-order stationary point
$\x$ such that $\|\nabla f(\x)\|\leq \epsilon$ and $\nabla^2 f(\x)\geq
-\sqrt{\epsilon} I$ with $\widetilde O(1/\epsilon^{1.75})$ iteration
complexity, improving that of gradient descent method by a factor of
$\epsilon^{-0.25}$ and matching the best iteration complexity of second-order
Hessian-free methods for non-convex optimization.