pwc-archive/papers-with-abstracts huggingface dataset:
Improved robustness to adversarial examples using Lipschitz regularization of the loss
Authors: Chris Finlay, Adam Oberman, Bilal Abbasi
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Adversarial Robustness

We augment adversarial training (AT) with worst case adversarial training (WCAT) which improves adversarial robustness by 11% over the current state-of-the-art result in the $\ell_2$ norm on CIFAR-10. We obtain verifiable average case and worst case robustness guarantees, based on the expected and maximum values of the norm of the gradient of the loss. We interpret adversarial training as Total Variation Regularization, which is a fundamental tool in mathematical image processing, and WCAT as Lipschitz regularization.
Joint On-line Learning of a Zero-shot Spoken Semantic Parser and a Reinforcement Learning Dialogue Manager
Authors: Matthieu Riou, Bassam Jabaian, Stéphane Huet, Fabrice Lefèvre
Date: 2018-10-01 00:00:00
Tasks: Dialogue Management, Management, Reinforcement Learning, speech-recognition, Speech Recognition, Spoken Language Understanding

Despite many recent advances for the design of dialogue systems, a true
bottleneck remains the acquisition of data required to train its components.
Unlike many other language processing applications, dialogue systems require
interactions with users, therefore it is complex to develop them with
pre-recorded data. Building on previous works, on-line learning is pursued here
as a most convenient way to address the issue. Data collection, annotation and
use in learning algorithms are performed in a single process. The main
difficulties are then: to bootstrap an initial basic system, and to control the
level of additional cost on the user side. Considering that well-performing
solutions can be used directly off the shelf for speech recognition and
synthesis, the study is focused on learning the spoken language understanding
and dialogue management modules only. Several variants of joint learning are
investigated and tested with user trials to confirm that the overall on-line
learning can be obtained after only a few hundred training dialogues and can
overstep an expert-based system.
Robust multivariate and functional archetypal analysis with application to financial time series analysis
Authors: Jesús Moliner, Irene Epifanio
Date: 2018-10-01 00:00:00
Tasks: Time Series, Time Series Analysis

Archetypal analysis approximates data by means of mixtures of actual extreme
cases (archetypoids) or archetypes, which are a convex combination of cases in
the data set. Archetypes lie on the boundary of the convex hull. This makes the
analysis very sensitive to outliers. A robust methodology by means of
M-estimators for classical multivariate and functional data is proposed. This
unsupervised methodology allows complex data to be understood even by
non-experts. The performance of the new procedure is assessed in a simulation
study, where a comparison with a previous methodology for the multivariate case
is also carried out, and our proposal obtains favorable results. Finally,
robust bivariate functional archetypoid analysis is applied to a set of
companies in the S\&P 500 described by two time series of stock quotes. A new
graphic representation is also proposed to visualize the results. The analysis
shows how the information can be easily interpreted and how even non-experts
can gain a qualitative understanding of the data.
Handling Nominals and Inverse Roles using Algebraic Reasoning
Authors: Humaira Farid, Volker Haarslev
Date: 2018-10-01 00:00:00

This paper presents a novel SHOI tableau calculus which incorporates
algebraic reasoning for deciding ontology consistency. Numerical restrictions
imposed by nominals, existential and universal restrictions are encoded into a
set of linear inequalities. Column generation and branch-and-price algorithms
are used to solve these inequalities. Our preliminary experiments indicate that
this calculus performs better on SHOI ontologies than standard tableau methods.
Visual Curiosity: Learning to Ask Questions to Learn Visual Recognition
Authors: Jianwei Yang, Jiasen Lu, Stefan Lee, Dhruv Batra, Devi Parikh
Date: 2018-10-01 00:00:00
Tasks: Question Generation, Question-Generation, Reinforcement Learning

In an open-world setting, it is inevitable that an intelligent agent (e.g., a
robot) will encounter visual objects, attributes or relationships it does not
recognize. In this work, we develop an agent empowered with visual curiosity,
i.e. the ability to ask questions to an Oracle (e.g., human) about the contents
in images (e.g., What is the object on the left side of the red cube?) and
build visual recognition model based on the answers received (e.g., Cylinder).
In order to do this, the agent must (1) understand what it recognizes and what
it does not, (2) formulate a valid, unambiguous and informative language query
(a question) to ask the Oracle, (3) derive the parameters of visual classifiers
from the Oracle response and (4) leverage the updated visual classifiers to ask
more clarified questions. Specifically, we propose a novel framework and
formulate the learning of visual curiosity as a reinforcement learning problem.
In this framework, all components of our agent, visual recognition module (to
see), question generation policy (to ask), answer digestion module (to
understand) and graph memory module (to memorize), are learned entirely
end-to-end to maximize the reward derived from the scene graph obtained by the
agent as a consequence of the dialog with the Oracle. Importantly, the question
generation policy is disentangled from the visual recognition system and
specifics of the environment. Consequently, we demonstrate a sort of double
generalization. Our question generation policy generalizes to new environments
and a new pair of eyes, i.e., new visual system. Trained on a synthetic
dataset, our results show that our agent learns new visual concepts
significantly faster than several heuristic baselines, even when tested on
synthetic environments with novel objects, as well as in a realistic
environment.
ProxQuant: Quantized Neural Networks via Proximal Operators
Authors: Yu Bai, Yu-Xiang Wang, Edo Liberty
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Quantization

To make deep neural networks feasible in resource-constrained environments
(such as mobile devices), it is beneficial to quantize models by using
low-precision weights. One common technique for quantizing neural networks is
the straight-through gradient method, which enables back-propagation through
the quantization mapping. Despite its empirical success, little is understood
about why the straight-through gradient method works.
  Building upon a novel observation that the straight-through gradient method
is in fact identical to the well-known Nesterov's dual-averaging algorithm on a
quantization constrained optimization problem, we propose a more principled
alternative approach, called ProxQuant, that formulates quantized network
training as a regularized learning problem instead and optimizes it via the
prox-gradient method. ProxQuant does back-propagation on the underlying
full-precision vector and applies an efficient prox-operator in between
stochastic gradient steps to encourage quantizedness. For quantizing ResNets
and LSTMs, ProxQuant outperforms state-of-the-art results on binary
quantization and is on par with state-of-the-art on multi-bit quantization. For
binary quantization, our analysis shows both theoretically and experimentally
that ProxQuant is more stable than the straight-through gradient method (i.e.
BinaryConnect), challenging the indispensability of the straight-through
gradient method and providing a powerful alternative.
Dynamic Sparse Graph for Efficient Deep Learning
Authors: Liu Liu, Lei Deng, Xing Hu, Maohua Zhu, Guoqi Li, Yufei Ding, Yuan Xie
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Learning, Dimensionality Reduction

We propose to execute deep neural networks (DNNs) with dynamic and sparse graph (DSG) structure for compressive memory and accelerative execution during both training and inference. The great success of DNNs motivates the pursuing of lightweight models for the deployment onto embedded devices. However, most of the previous studies optimize for inference while neglect training or even complicate it. Training is far more intractable, since (i) the neurons dominate the memory cost rather than the weights in inference; (ii) the dynamic activation makes previous sparse acceleration via one-off optimization on fixed weight invalid; (iii) batch normalization (BN) is critical for maintaining accuracy while its activation reorganization damages the sparsity. To address these issues, DSG activates only a small amount of neurons with high selectivity at each iteration via a dimension-reduction search (DRS) and obtains the BN compatibility via a double-mask selection (DMS). Experiments show significant memory saving (1.7-4.5x) and operation reduction (2.3-4.4x) with little accuracy loss on various benchmarks.
Privacy and Utility Tradeoff in Approximate Differential Privacy
Authors: Quan Geng, Wei Ding, Ruiqi Guo, Sanjiv Kumar
Date: 2018-10-01 00:00:00

We characterize the minimum noise amplitude and power for noise-adding mechanisms in $(\epsilon, \delta)$-differential privacy for single real-valued query function. We derive new lower bounds using the duality of linear programming, and new upper bounds by proposing a new class of $(\epsilon,\delta)$-differentially private mechanisms, the \emph{truncated Laplacian} mechanisms. We show that the multiplicative gap of the lower bounds and upper bounds goes to zero in various high privacy regimes, proving the tightness of the lower and upper bounds and thus establishing the optimality of the truncated Laplacian mechanism. In particular, our results close the previous constant multiplicative gap in the discrete setting. Numeric experiments show the improvement of the truncated Laplacian mechanism over the optimal Gaussian mechanism in all privacy regimes.
Augmented Mitotic Cell Count using Field Of Interest Proposal
Authors: Marc Aubreville, Christof A. Bertram, Robert Klopfleisch, Andreas Maier
Date: 2018-10-01 00:00:00
Tasks: Region Proposal, whole slide images

Histopathological prognostication of neoplasia including most tumor grading
systems are based upon a number of criteria. Probably the most important is the
number of mitotic figures which are most commonly determined as the mitotic
count (MC), i.e. number of mitotic figures within 10 consecutive high power
fields. Often the area with the highest mitotic activity is to be selected for
the MC. However, since mitotic activity is not known in advance, an arbitrary
choice of this region is considered one important cause for high variability in
the prognostication and grading.
  In this work, we present an algorithmic approach that first calculates a
mitotic cell map based upon a deep convolutional network. This map is in a
second step used to construct a mitotic activity estimate. Lastly, we select
the image segment representing the size of ten high power fields with the
overall highest mitotic activity as a region proposal for an expert MC
determination. We evaluate the approach using a dataset of 32 completely
annotated whole slide images, where 22 were used for training of the network
and 10 for test. We find a correlation of r=0.936 in mitotic count estimate.
Classification from Positive, Unlabeled and Biased Negative Data
Authors: Yu-Guan Hsieh, Gang Niu, Masashi Sugiyama
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Binary Classification, Classification, General Classification

In binary classification, there are situations where negative (N) data are too diverse to be fully labeled and we often resort to positive-unlabeled (PU) learning in these scenarios. However, collecting a non-representative N set that contains only a small portion of all possible N data can often be much easier in practice. This paper studies a novel classification framework which incorporates such biased N (bN) data in PU learning. We provide a method based on empirical risk minimization to address this PUbN classification problem. Our approach can be regarded as a novel example-weighting algorithm, with the weight of each example computed through a preliminary step that draws inspiration from PU learning. We also derive an estimation error bound for the proposed method. Experimental results demonstrate the effectiveness of our algorithm in not only PUbN learning scenarios but also ordinary PU learning scenarios on several benchmark datasets.
CHET: Compiler and Runtime for Homomorphic Evaluation of Tensor Programs
Authors: Roshan Dathathri, Olli Saarikivi, Hao Chen, Kim Laine, Kristin Lauter, Saeed Maleki, Madanlal Musuvathi, Todd Mytkowicz
Date: 2018-10-01 00:00:00

Fully Homomorphic Encryption (FHE) refers to a set of encryption schemes that
allow computations to be applied directly on encrypted data without requiring a
secret key. This enables novel application scenarios where a client can safely
offload storage and computation to a third-party cloud provider without having
to trust the software and the hardware vendors with the decryption keys. Recent
advances in both FHE schemes and implementations have moved such applications
from theoretical possibilities into the realm of practicalities.
  This paper proposes a compact and well-reasoned interface called the
Homomorphic Instruction Set Architecture (HISA) for developing FHE
applications. Just as the hardware ISA interface enabled hardware advances to
proceed independent of software advances in the compiler and language runtimes,
HISA decouples compiler optimizations and runtimes for supporting FHE
applications from advancements in the underlying FHE schemes.
  This paper demonstrates the capabilities of HISA by building an end-to-end
software stack for evaluating neural network models on encrypted data. Our
stack includes an end-to-end compiler, runtime, and a set of optimizations. Our
approach shows generated code, on a set of popular neural network
architectures, is faster than hand-optimized implementations.
Network Modeling and Pathway Inference from Incomplete Data ("PathInf")
Authors: Xiang Li, Qitian Chen, Xing Wang, Ning Guo, Nan Wu, Quanzheng Li
Date: 2018-10-01 00:00:00
Tasks: Data Summarization, Diagnostic, Missing Values

In this work, we developed a network inference method from incomplete data
("PathInf") , as massive and non-uniformly distributed missing values is a
common challenge in practical problems. PathInf is a two-stages inference
model. In the first stage, it applies a data summarization model based on
maximum likelihood to deal with the massive distributed missing values by
transforming the observation-wise items in the data into state matrix. In the
second stage, transition pattern (i.e. pathway) among variables is inferred as
a graph inference problem solved by greedy algorithm with constraints. The
proposed method was validated and compared with the state-of-art Bayesian
network method on the simulation data, and shown consistently superior
performance. By applying the PathInf on the lymph vascular metastasis data, we
obtained the holistic pathways of the lymph node metastasis with novel
discoveries on the jumping metastasis among nodes that are physically apart.
The discovery indicates the possible presence of sentinel node groups in the
lung lymph nodes which have been previously speculated yet never found. The
pathway map can also improve the current dissection examination protocol for
better individualized treatment planning, for higher diagnostic accuracy and
reducing the patients trauma.
Adaptive Game-Theoretic Decision Making for Autonomous Vehicle Control at Roundabouts
Authors: Ran Tian, Sisi Li, Nan Li, Ilya Kolmanovsky, Anouck Girard, Yildiray Yildiz
Date: 2018-10-01 00:00:00
Tasks: Decision Making

In this paper, we propose a decision making algorithm for autonomous vehicle
control at a roundabout intersection. The algorithm is based on a
game-theoretic model representing the interactions between the ego vehicle and
an opponent vehicle, and adapts to an online estimated driver type of the
opponent vehicle. Simulation results are reported.
Singularity, Misspecification, and the Convergence Rate of EM
Authors: Raaz Dwivedi, Nhat Ho, Koulik Khamaru, Michael. I. Jordan, Martin J. Wainwright, Bin Yu
Date: 2018-10-01 00:00:00

A line of recent work has analyzed the behavior of the Expectation-Maximization (EM) algorithm in the well-specified setting, in which the population likelihood is locally strongly concave around its maximizing argument. Examples include suitably separated Gaussian mixture models and mixtures of linear regressions. We consider over-specified settings in which the number of fitted components is larger than the number of components in the true distribution. Such misspecified settings can lead to singularity in the Fisher information matrix, and moreover, the maximum likelihood estimator based on $n$ i.i.d. samples in $d$ dimensions can have a non-standard $\mathcal{O}((d/n)^{\frac{1}{4}})$ rate of convergence. Focusing on the simple setting of two-component mixtures fit to a $d$-dimensional Gaussian distribution, we study the behavior of the EM algorithm both when the mixture weights are different (unbalanced case), and are equal (balanced case). Our analysis reveals a sharp distinction between these two cases: in the former, the EM algorithm converges geometrically to a point at Euclidean distance of $\mathcal{O}((d/n)^{\frac{1}{2}})$ from the true parameter, whereas in the latter case, the convergence rate is exponentially slower, and the fixed point has a much lower $\mathcal{O}((d/n)^{\frac{1}{4}})$ accuracy. Analysis of this singular case requires the introduction of some novel techniques: in particular, we make use of a careful form of localization in the associated empirical process, and develop a recursive argument to progressively sharpen the statistical rate.
How Powerful are Graph Neural Networks?
Authors: Keyulu Xu, Weihua Hu, Jure Leskovec, Stefanie Jegelka
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: General Classification, Graph Classification, Graph Regression, Graph Representation Learning, Molecular Property Prediction, Node Classification, Representation Learning

Graph Neural Networks (GNNs) are an effective framework for representation
learning of graphs. GNNs follow a neighborhood aggregation scheme, where the
representation vector of a node is computed by recursively aggregating and
transforming representation vectors of its neighboring nodes. Many GNN variants
have been proposed and have achieved state-of-the-art results on both node and
graph classification tasks. However, despite GNNs revolutionizing graph
representation learning, there is limited understanding of their
representational properties and limitations. Here, we present a theoretical
framework for analyzing the expressive power of GNNs to capture different graph
structures. Our results characterize the discriminative power of popular GNN
variants, such as Graph Convolutional Networks and GraphSAGE, and show that
they cannot learn to distinguish certain simple graph structures. We then
develop a simple architecture that is provably the most expressive among the
class of GNNs and is as powerful as the Weisfeiler-Lehman graph isomorphism
test. We empirically validate our theoretical findings on a number of graph
classification benchmarks, and demonstrate that our model achieves
state-of-the-art performance.
Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
Authors: Juho Lee, Yoonho Lee, Jungtaek Kim, Adam R. Kosiorek, Seungjin Choi, Yee Whye Teh
Date: 2018-10-01 00:00:00
Tasks: 3D Shape Recognition, Decoder, Few-Shot Image Classification, image-classification, Image Classification, Multiple Instance Learning

Many machine learning tasks such as multiple instance learning, 3D shape recognition, and few-shot image classification are defined on sets of instances. Since solutions to such problems do not depend on the order of elements of the set, models used to address them should be permutation invariant. We present an attention-based neural network module, the Set Transformer, specifically designed to model interactions among elements in the input set. The model consists of an encoder and a decoder, both of which rely on attention mechanisms. In an effort to reduce computational complexity, we introduce an attention scheme inspired by inducing point methods from sparse Gaussian process literature. It reduces the computation time of self-attention from quadratic to linear in the number of elements in the set. We show that our model is theoretically attractive and we evaluate it on a range of tasks, demonstrating the state-of-the-art performance compared to recent methods for set-structured data.
Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
Authors: Xue Bin Peng, Angjoo Kanazawa, Sam Toyer, Pieter Abbeel, Sergey Levine
Date: 2018-10-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: continuous-control, Continuous Control, Image Generation, Imitation Learning, Reinforcement Learning

Adversarial learning methods have been proposed for a wide range of applications, but the training of adversarial models can be notoriously unstable. Effectively balancing the performance of the generator and discriminator is critical, since a discriminator that achieves very high accuracy will produce relatively uninformative gradients. In this work, we propose a simple and general technique to constrain information flow in the discriminator by means of an information bottleneck. By enforcing a constraint on the mutual information between the observations and the discriminator's internal representation, we can effectively modulate the discriminator's accuracy and maintain useful and informative gradients. We demonstrate that our proposed variational discriminator bottleneck (VDB) leads to significant improvements across three distinct application areas for adversarial learning algorithms. Our primary evaluation studies the applicability of the VDB to imitation learning of dynamic continuous control skills, such as running. We show that our method can learn such skills directly from \emph{raw} video demonstrations, substantially outperforming prior adversarial imitation learning methods. The VDB can also be combined with adversarial inverse reinforcement learning to learn parsimonious reward functions that can be transferred and re-optimized in new settings. Finally, we demonstrate that VDB can train GANs more effectively for image generation, improving upon a number of prior stabilization methods.
RGB-D Object Detection and Semantic Segmentation for Autonomous Manipulation in Clutter
Authors: Max Schwarz, Anton Milan, Arul Selvam Periyasamy, Sven Behnke
Date: 2018-10-01 00:00:00
Tasks: Disaster Response, Object, object-detection, Object Detection, Segmentation, Semantic Segmentation

Autonomous robotic manipulation in clutter is challenging. A large variety of
objects must be perceived in complex scenes, where they are partially occluded
and embedded among many distractors, often in restricted spaces. To tackle
these challenges, we developed a deep-learning approach that combines object
detection and semantic segmentation. The manipulation scenes are captured with
RGB-D cameras, for which we developed a depth fusion method. Employing
pretrained features makes learning from small annotated robotic data sets
possible. We evaluate our approach on two challenging data sets: one captured
for the Amazon Picking Challenge 2016, where our team NimbRo came in second in
the Stowing and third in the Picking task, and one captured in
disaster-response scenarios. The experiments show that object detection and
semantic segmentation complement each other and can be combined to yield
reliable object perception.