pwc-archive/papers-with-abstracts huggingface dataset:
An Aggressive Genetic Programming Approach for Searching Neural Network Structure Under Computational Constraints
Authors: Zhe Li, Xuehan Xiong, Zhou Ren, Ning Zhang, Xiaoyu Wang, Tianbao Yang
Date: 2018-06-03 00:00:00
Tasks: Diversity, Evolutionary Algorithms

Recently, there emerged revived interests of designing automatic programs
(e.g., using genetic/evolutionary algorithms) to optimize the structure of
Convolutional Neural Networks (CNNs) for a specific task. The challenge in
designing such programs lies in how to balance between large search space of
the network structures and high computational costs. Existing works either
impose strong restrictions on the search space or use enormous computing
resources. In this paper, we study how to design a genetic programming approach
for optimizing the structure of a CNN for a given task under limited
computational resources yet without imposing strong restrictions on the search
space. To reduce the computational costs, we propose two general strategies
that are observed to be helpful: (i) aggressively selecting strongest
individuals for survival and reproduction, and killing weaker individuals at a
very early age; (ii) increasing mutation frequency to encourage diversity and
faster evolution. The combined strategy with additional optimization techniques
allows us to explore a large search space but with affordable computational
costs. Our results on standard benchmark datasets (MNIST, SVHN, CIFAR-10,
CIFAR-100) are competitive to similar approaches with significantly reduced
computational costs.
Sample-Efficient Learning of Mixtures
Authors: Hassan Ashtiani, Shai Ben-David, Abbas Mehrabian
Date: 2017-06-06 00:00:00
Tasks: Density Estimation, PAC learning

We consider PAC learning of probability distributions (a.k.a. density
estimation), where we are given an i.i.d. sample generated from an unknown
target distribution, and want to output a distribution that is close to the
target in total variation distance. Let $\mathcal F$ be an arbitrary class of
probability distributions, and let $\mathcal{F}^k$ denote the class of
$k$-mixtures of elements of $\mathcal F$. Assuming the existence of a method
for learning $\mathcal F$ with sample complexity $m_{\mathcal{F}}(\epsilon)$,
we provide a method for learning $\mathcal F^k$ with sample complexity
$O({k\log k \cdot m_{\mathcal F}(\epsilon) }/{\epsilon^{2}})$. Our mixture
learning algorithm has the property that, if the $\mathcal F$-learner is
proper/agnostic, then the $\mathcal F^k$-learner would be proper/agnostic as
well.
  This general result enables us to improve the best known sample complexity
upper bounds for a variety of important mixture classes. First, we show that
the class of mixtures of $k$ axis-aligned Gaussians in $\mathbb{R}^d$ is
PAC-learnable in the agnostic setting with $\widetilde{O}({kd}/{\epsilon ^ 4})$
samples, which is tight in $k$ and $d$ up to logarithmic factors. Second, we
show that the class of mixtures of $k$ Gaussians in $\mathbb{R}^d$ is
PAC-learnable in the agnostic setting with sample complexity
$\widetilde{O}({kd^2}/{\epsilon ^ 4})$, which improves the previous known
bounds of $\widetilde{O}({k^3d^2}/{\epsilon ^ 4})$ and
$\widetilde{O}(k^4d^4/\epsilon ^ 2)$ in its dependence on $k$ and $d$. Finally,
we show that the class of mixtures of $k$ log-concave distributions over
$\mathbb{R}^d$ is PAC-learnable using
$\widetilde{O}(d^{(d+5)/2}\epsilon^{-(d+9)/2}k)$ samples.
Learning graphs from data: A signal representation perspective
Authors: Xiaowen Dong, Dorina Thanou, Michael Rabbat, Pascal Frossard
Date: 2018-06-03 00:00:00
Tasks: Graph Learning

The construction of a meaningful graph topology plays a crucial role in the effective representation, processing, analysis and visualization of structured data. When a natural choice of the graph is not readily available from the data sets, it is thus desirable to infer or learn a graph topology from the data. In this tutorial overview, we survey solutions to the problem of graph learning, including classical viewpoints from statistics and physics, and more recent approaches that adopt a graph signal processing (GSP) perspective. We further emphasize the conceptual similarities and differences between classical and GSP-based graph inference methods, and highlight the potential advantage of the latter in a number of theoretical and practical scenarios. We conclude with several open issues and challenges that are keys to the design of future signal processing and machine learning algorithms for learning graphs from data.
S4ND: Single-Shot Single-Scale Lung Nodule Detection
Authors: Naji Khosravan, Ulas Bagci
Date: 2018-05-06 00:00:00
Tasks: Lung Nodule Detection, object-detection, Object Detection

The state of the art lung nodule detection studies rely on computationally
expensive multi-stage frameworks to detect nodules from CT scans. To address
this computational challenge and provide better performance, in this paper we
propose S4ND, a new deep learning based method for lung nodule detection. Our
approach uses a single feed forward pass of a single network for detection and
provides better performance when compared to the current literature. The whole
detection pipeline is designed as a single $3D$ Convolutional Neural Network
(CNN) with dense connections, trained in an end-to-end manner. S4ND does not
require any further post-processing or user guidance to refine detection
results. Experimentally, we compared our network with the current
state-of-the-art object detection network (SSD) in computer vision as well as
the state-of-the-art published method for lung nodule detection (3D DCNN). We
used publically available $888$ CT scans from LUNA challenge dataset and showed
that the proposed method outperforms the current literature both in terms of
efficiency and accuracy by achieving an average FROC-score of $0.897$. We also
provide an in-depth analysis of our proposed network to shed light on the
unclear paradigms of tiny object detection.
The Power of Localization for Efficiently Learning Linear Separators with Noise
Authors: Pranjal Awasthi, Maria Florina Balcan, Philip M. Long
Date: 2013-07-31 00:00:00
Tasks: Active Learning

We introduce a new approach for designing computationally efficient learning
algorithms that are tolerant to noise, and demonstrate its effectiveness by
designing algorithms with improved noise tolerance guarantees for learning
linear separators.
  We consider both the malicious noise model and the adversarial label noise
model. For malicious noise, where the adversary can corrupt both the label and
the features, we provide a polynomial-time algorithm for learning linear
separators in $\Re^d$ under isotropic log-concave distributions that can
tolerate a nearly information-theoretically optimal noise rate of $\eta =
\Omega(\epsilon)$. For the adversarial label noise model, where the
distribution over the feature vectors is unchanged, and the overall probability
of a noisy label is constrained to be at most $\eta$, we also give a
polynomial-time algorithm for learning linear separators in $\Re^d$ under
isotropic log-concave distributions that can handle a noise rate of $\eta =
\Omega\left(\epsilon\right)$.
  We show that, in the active learning model, our algorithms achieve a label
complexity whose dependence on the error parameter $\epsilon$ is
polylogarithmic. This provides the first polynomial-time active learning
algorithm for learning linear separators in the presence of malicious noise or
adversarial label noise.
TernausNetV2: Fully Convolutional Network for Instance Segmentation
Authors: Vladimir I. Iglovikov, Selim Seferbekov, Alexander V. Buslaev, Alexey Shvets
Date: 2018-06-03 00:00:00
Tasks: Decoder, Instance Segmentation, Segmentation, Semantic Segmentation, Template Matching, Transfer Learning

The most common approaches to instance segmentation are complex and use
two-stage networks with object proposals, conditional random-fields, template
matching or recurrent neural networks. In this work we present TernausNetV2 - a
simple fully convolutional network that allows extracting objects from a
high-resolution satellite imagery on an instance level. The network has popular
encoder-decoder type of architecture with skip connections but has a few
essential modifications that allows using for semantic as well as for instance
segmentation tasks. This approach is universal and allows to extend any network
that has been successfully applied for semantic segmentation to perform
instance segmentation task. In addition, we generalize network encoder that was
pre-trained for RGB images to use additional input channels. It makes possible
to use transfer learning from visual to a wider spectral range. For
DeepGlobe-CVPR 2018 building detection sub-challenge, based on public
leaderboard score, our approach shows superior performance in comparison to
other methods. The source code corresponding pre-trained weights are publicly
available at https://github.com/ternaus/TernausNetV2
Incorrigibility in the CIRL Framework
Authors: Ryan Carey
Date: 2017-09-19 00:00:00

A value learning system has incentives to follow shutdown instructions,
assuming the shutdown instruction provides information (in the technical sense)
about which actions lead to valuable outcomes. However, this assumption is not
robust to model mis-specification (e.g., in the case of programmer errors). We
demonstrate this by presenting some Supervised POMDP scenarios in which errors
in the parameterized reward function remove the incentive to follow shutdown
commands. These difficulties parallel those discussed by Soares et al. (2015)
in their paper on corrigibility. We argue that it is important to consider
systems that follow shutdown commands under some weaker set of assumptions
(e.g., that one small verified module is correctly implemented; as opposed to
an entire prior probability distribution and/or parameterized reward function).
We discuss some difficulties with simple ways to attempt to attain these sorts
of guarantees in a value learning framework.
Latent Tree Learning with Differentiable Parsers: Shift-Reduce Parsing and Chart Parsing
Authors: Jean Maillard, Stephen Clark
Date: 2018-06-03 00:00:00
Proceeding: WS 2018 7

Latent tree learning models represent sentences by composing their words
according to an induced parse tree, all based on a downstream task. These
models often outperform baselines which use (externally provided) syntax trees
to drive the composition order. This work contributes (a) a new latent tree
learning model based on shift-reduce parsing, with competitive downstream
performance and non-trivial induced trees, and (b) an analysis of the trees
learned by our shift-reduce model and by a chart-based model.
Study and development of a Computer-Aided Diagnosis system for classification of chest x-ray images using convolutional neural networks pre-trained for ImageNet and data augmentation
Authors: Vinicius Pavanelli Vianna
Date: 2018-06-03 00:00:00
Tasks: Data Augmentation, General Classification, Transfer Learning

Convolutional neural networks (ConvNets) are the actual standard for image
recognizement and classification. On the present work we develop a Computer
Aided-Diagnosis (CAD) system using ConvNets to classify a x-rays chest images
dataset in two groups: Normal and Pneumonia. The study uses ConvNets models
available on the PyTorch platform: AlexNet, SqueezeNet, ResNet and Inception.
We initially use three training styles: complete from scratch using random
initialization, using a pre-trained ImageNet model training only the last layer
adapted to our problem (transfer learning) and a pre-trained model modified
training all the classifying layers of the model (fine tuning). The last
strategy of training used is with data augmentation techniques that avoid over
fitting problems on ConvNets yielding the better results on this study
The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making
Authors: Luchen Li, Matthieu Komorowski, Aldo A. Faisal
Date: 2018-05-29 00:00:00
Tasks: Decision Making, Heuristic Search, Reinforcement Learning

Off-policy reinforcement learning enables near-optimal policy from suboptimal
experience, thereby provisions opportunity for artificial intelligence
applications in healthcare. Previous works have mainly framed patient-clinician
interactions as Markov decision processes, while true physiological states are
not necessarily fully observable from clinical data. We capture this situation
with partially observable Markov decision process, in which an agent optimises
its actions in a belief represented as a distribution of patient states
inferred from individual history trajectories. A Gaussian mixture model is
fitted for the observed data. Moreover, we take into account the fact that
nuance in pharmaceutical dosage could presumably result in significantly
different effect by modelling a continuous policy through a Gaussian
approximator directly in the policy space, i.e. the actor. To address the
challenge of infinite number of possible belief states which renders exact
value iteration intractable, we evaluate and plan for only every encountered
belief, through heuristic search tree by tightly maintaining lower and upper
bounds of the true value of belief. We further resort to function
approximations to update value bounds estimation, i.e. the critic, so that the
tree search can be improved through more compact bounds at the fringe nodes
that will be back-propagated to the root. Both actor and critic parameters are
learned via gradient-based approaches. Our proposed policy trained from real
intensive care unit data is capable of dictating dosing on vasopressors and
intravenous fluids for sepsis patients that lead to the best patient outcomes.
Analysis of regularized Nyström subsampling for regression functions of low smoothness
Authors: Shuai Lu, Peter Mathé, Sergiy Pereverzyev Jr
Date: 2018-06-03 00:00:00
Tasks: regression

This paper studies a Nystr\"om type subsampling approach to large kernel
learning methods in the misspecified case, where the target function is not
assumed to belong to the reproducing kernel Hilbert space generated by the
underlying kernel. This case is less understood, in spite of its practical
importance. To model such a case, the smoothness of target functions is
described in terms of general source conditions. It is surprising that almost
for the whole range of the source conditions, describing the misspecified case,
the corresponding learning rate bounds can be achieved with just one value of
the regularization parameter. This observation allows a formulation of mild
conditions under which the plain Nystr\"om subsampling can be realized with
subquadratic cost maintaining the guaranteed learning rates.
Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering
Authors: Aishwarya Agrawal, Dhruv Batra, Devi Parikh, Aniruddha Kembhavi
Date: 2017-12-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

A number of studies have found that today's Visual Question Answering (VQA)
models are heavily driven by superficial correlations in the training data and
lack sufficient image grounding. To encourage development of models geared
towards the latter, we propose a new setting for VQA where for every question
type, train and test sets have different prior distributions of answers.
Specifically, we present new splits of the VQA v1 and VQA v2 datasets, which we
call Visual Question Answering under Changing Priors (VQA-CP v1 and VQA-CP v2
respectively). First, we evaluate several existing VQA models under this new
setting and show that their performance degrades significantly compared to the
original VQA setting. Second, we propose a novel Grounded Visual Question
Answering model (GVQA) that contains inductive biases and restrictions in the
architecture specifically designed to prevent the model from 'cheating' by
primarily relying on priors in the training data. Specifically, GVQA explicitly
disentangles the recognition of visual concepts present in the image from the
identification of plausible answer space for a given question, enabling the
model to more robustly generalize across different distributions of answers.
GVQA is built off an existing VQA model -- Stacked Attention Networks (SAN).
Our experiments demonstrate that GVQA significantly outperforms SAN on both
VQA-CP v1 and VQA-CP v2 datasets. Interestingly, it also outperforms more
powerful VQA models such as Multimodal Compact Bilinear Pooling (MCB) in
several cases. GVQA offers strengths complementary to SAN when trained and
evaluated on the original VQA v1 and VQA v2 datasets. Finally, GVQA is more
transparent and interpretable than existing VQA models.
Causal Inference with Noisy and Missing Covariates via Matrix Factorization
Authors: Nathan Kallus, Xiaojie Mao, Madeleine Udell
Date: 2018-06-03 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Causal Inference, Matrix Completion, Missing Values, valid

Valid causal inference in observational studies often requires controlling
for confounders. However, in practice measurements of confounders may be noisy,
and can lead to biased estimates of causal effects. We show that we can reduce
the bias caused by measurement noise using a large number of noisy measurements
of the underlying confounders. We propose the use of matrix factorization to
infer the confounders from noisy covariates, a flexible and principled
framework that adapts to missing values, accommodates a wide variety of data
types, and can augment many causal inference methods. We bound the error for
the induced average treatment effect estimator and show it is consistent in a
linear regression setting, using Exponential Family Matrix Completion
preprocessing. We demonstrate the effectiveness of the proposed procedure in
numerical experiments with both synthetic data and real clinical data.
Fast Approximate Nearest Neighbor Search With The Navigating Spreading-out Graph
Authors: Cong Fu, Chao Xiang, Changxu Wang, Deng Cai
Date: 2017-07-01 00:00:00

Approximate nearest neighbor search (ANNS) is a fundamental problem in
databases and data mining. A scalable ANNS algorithm should be both
memory-efficient and fast. Some early graph-based approaches have shown
attractive theoretical guarantees on search time complexity, but they all
suffer from the problem of high indexing time complexity. Recently, some
graph-based methods have been proposed to reduce indexing complexity by
approximating the traditional graphs; these methods have achieved revolutionary
performance on million-scale datasets. Yet, they still can not scale to
billion-node databases. In this paper, to further improve the search-efficiency
and scalability of graph-based methods, we start by introducing four aspects:
(1) ensuring the connectivity of the graph; (2) lowering the average out-degree
of the graph for fast traversal; (3) shortening the search path; and (4)
reducing the index size. Then, we propose a novel graph structure called
Monotonic Relative Neighborhood Graph (MRNG) which guarantees very low search
complexity (close to logarithmic time). To further lower the indexing
complexity and make it practical for billion-node ANNS problems, we propose a
novel graph structure named Navigating Spreading-out Graph (NSG) by
approximating the MRNG. The NSG takes the four aspects into account
simultaneously. Extensive experiments show that NSG outperforms all the
existing algorithms significantly. In addition, NSG shows superior performance
in the E-commercial search scenario of Taobao (Alibaba Group) and has been
integrated into their search engine at billion-node scale.
Admissible Abstractions for Near-optimal Task and Motion Planning
Authors: William Vega-Brown, Nicholas Roy
Date: 2018-06-03 00:00:00
Tasks: Motion Planning, Task and Motion Planning

We define an admissibility condition for abstractions expressed using angelic
semantics and show that these conditions allow us to accelerate planning while
preserving the ability to find the optimal motion plan. We then derive
admissible abstractions for two motion planning domains with continuous state.
We extract upper and lower bounds on the cost of concrete motion plans using
local metric and topological properties of the problem domain. These bounds
guide the search for a plan while maintaining performance guarantees. We show
that abstraction can dramatically reduce the complexity of search relative to a
direct motion planner. Using our abstractions, we find near-optimal motion
plans in planning problems involving $10^{13}$ states without using a separate
task planner.
NAM: Non-Adversarial Unsupervised Domain Mapping
Authors: Yedid Hoshen, Lior Wolf
Date: 2018-06-03 00:00:00
Proceeding: ECCV 2018 9

Several methods were recently proposed for the task of translating images
between domains without prior knowledge in the form of correspondences. The
existing methods apply adversarial learning to ensure that the distribution of
the mapped source domain is indistinguishable from the target domain, which
suffers from known stability issues. In addition, most methods rely heavily on
`cycle' relationships between the domains, which enforce a one-to-one mapping.
In this work, we introduce an alternative method: Non-Adversarial Mapping
(NAM), which separates the task of target domain generative modeling from the
cross-domain mapping task. NAM relies on a pre-trained generative model of the
target domain, and aligns each source image with an image synthesized from the
target domain, while jointly optimizing the domain mapping function. It has
several key advantages: higher quality and resolution image translations,
simpler and more stable training and reusable target models. Extensive
experiments are presented validating the advantages of our method.
Latent RANSAC
Authors: Simon Korman, Roee Litman
Date: 2018-02-20 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Face Alignment, 3D Plane Detection, Camera Localization, Homography Estimation, Point Cloud Registration, Robust Face Alignment

We present a method that can evaluate a RANSAC hypothesis in constant time,
i.e. independent of the size of the data. A key observation here is that
correct hypotheses are tightly clustered together in the latent parameter
domain. In a manner similar to the generalized Hough transform we seek to find
this cluster, only that we need as few as two votes for a successful detection.
Rapidly locating such pairs of similar hypotheses is made possible by adapting
the recent "Random Grids" range-search technique. We only perform the usual
(costly) hypothesis verification stage upon the discovery of a close pair of
hypotheses. We show that this event rarely happens for incorrect hypotheses,
enabling a significant speedup of the RANSAC pipeline. The suggested approach
is applied and tested on three robust estimation problems: camera localization,
3D rigid alignment and 2D-homography estimation. We perform rigorous testing on
both synthetic and real datasets, demonstrating an improvement in efficiency
without a compromise in accuracy. Furthermore, we achieve state-of-the-art 3D
alignment results on the challenging "Redwood" loop-closure challenge.