pwc-archive/papers-with-abstracts huggingface dataset:
Interpreting Neural Networks Using Flip Points
Authors: Roozbeh Yousefzadeh, Dianne P. O'Leary
Date: 2019-03-21 00:00:00

Neural networks have been criticized for their lack of easy interpretation,
which undermines confidence in their use for important applications. Here, we
introduce a novel technique, interpreting a trained neural network by
investigating its flip points. A flip point is any point that lies on the
boundary between two output classes: e.g. for a neural network with a binary
yes/no output, a flip point is any input that generates equal scores for "yes"
and "no". The flip point closest to a given input is of particular importance,
and this point is the solution to a well-posed optimization problem. This paper
gives an overview of the uses of flip points and how they are computed. Through
results on standard datasets, we demonstrate how flip points can be used to
provide detailed interpretation of the output produced by a neural network.
Moreover, for a given input, flip points enable us to measure confidence in the
correctness of outputs much more effectively than softmax score. They also
identify influential features of the inputs, identify bias, and find changes in
the input that change the output of the model. We show that distance between an
input and the closest flip point identifies the most influential points in the
training data. Using principal component analysis (PCA) and rank-revealing QR
factorization (RR-QR), the set of directions from each training input to its
closest flip point provides explanations of how a trained neural network
processes an entire dataset: what features are most important for
classification into a given class, which features are most responsible for
particular misclassifications, how an adversary might fool the network, etc.
Although we investigate flip points for neural networks, their usefulness is
actually model-agnostic.
Selective Attention for Context-aware Neural Machine Translation
Authors: Sameen Maruf, André F. T. Martins, Gholamreza Haffari
Date: 2019-03-21 00:00:00
Proceeding: NAACL 2019 6
Tasks: Decoder, Machine Translation, NMT, Sentence, Translation

Despite the progress made in sentence-level NMT, current systems still fall short at achieving fluent, good quality translation for a full document. Recent works in context-aware NMT consider only a few previous sentences as context and may not scale to entire documents. To this end, we propose a novel and scalable top-down approach to hierarchical attention for context-aware NMT which uses sparse attention to selectively focus on relevant sentences in the document context and then attends to key words in those sentences. We also propose single-level attention approaches based on sentence or word-level information in the context. The document-level context representation, produced from these attention modules, is integrated into the encoder or decoder of the Transformer model depending on whether we use monolingual or bilingual context. Our experiments and evaluation on English-German datasets in different document MT settings show that our selective attention approach not only significantly outperforms context-agnostic baselines but also surpasses context-aware baselines in most cases.
Provable Certificates for Adversarial Examples: Fitting a Ball in the Union of Polytopes
Authors: Matt Jordan, Justin Lewis, Alexandros G. Dimakis
Date: 2019-03-20 00:00:00
Proceeding: NeurIPS 2019 12

We propose a novel method for computing exact pointwise robustness of deep neural networks for all convex $\ell_p$ norms. Our algorithm, GeoCert, finds the largest $\ell_p$ ball centered at an input point $x_0$, within which the output class of a given neural network with ReLU nonlinearities remains unchanged. We relate the problem of computing pointwise robustness of these networks to that of computing the maximum norm ball with a fixed center that can be contained in a non-convex polytope. This is a challenging problem in general, however we show that there exists an efficient algorithm to compute this for polyhedral complices. Further we show that piecewise linear neural networks partition the input space into a polyhedral complex. Our algorithm has the ability to almost immediately output a nontrivial lower bound to the pointwise robustness which is iteratively improved until it ultimately becomes tight. We empirically show that our approach generates distance lower bounds that are tighter compared to prior work, under moderate time constraints.
Robust Image Segmentation Quality Assessment
Authors: Leixin Zhou, Wenxiang Deng, Xiaodong Wu
Date: 2019-03-20 00:00:00
Proceeding: MIDL 2019 7
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Deep learning based image segmentation methods have achieved great success, even having human-level accuracy in some applications. However, due to the black box nature of deep learning, the best method may fail in some situations. Thus predicting segmentation quality without ground truth would be very crucial especially in clinical practice. Recently, people proposed to train neural networks to estimate the quality score by regression. Although it can achieve promising prediction accuracy, the network suffers robustness problem, e.g. it is vulnerable to adversarial attacks. In this paper, we propose to alleviate this problem by utilizing the difference between the input image and the reconstructed image, which is conditioned on the segmentation to be assessed, to lower the chance to overfit to the undesired image features from the original input image, and thus to increase the robustness. Results on ACDC17 dataset demonstrated our method is promising.
ToyArchitecture: Unsupervised Learning of Interpretable Models of the World
Authors: Jaroslav Vítků, Petr Dluhoš, Joseph Davidson, Matěj Nikl, Simon Andersson, Přemysl Paška, Jan Šinkora, Petr Hlubuček, Martin Stránský, Martin Hyben, Martin Poliak, Jan Feyereisl, Marek Rosa
Date: 2019-03-20 00:00:00
Tasks: Model-based Reinforcement Learning, Reinforcement Learning

Research in Artificial Intelligence (AI) has focused mostly on two extremes: either on small improvements in narrow AI domains, or on universal theoretical frameworks which are usually uncomputable, incompatible with theories of biological intelligence, or lack practical implementations. The goal of this work is to combine the main advantages of the two: to follow a big picture view, while providing a particular theory and its implementation. In contrast with purely theoretical approaches, the resulting architecture should be usable in realistic settings, but also form the core of a framework containing all the basic mechanisms, into which it should be easier to integrate additional required functionality. In this paper, we present a novel, purposely simple, and interpretable hierarchical architecture which combines multiple different mechanisms into one system: unsupervised learning of a model of the world, learning the influence of one's own actions on the world, model-based reinforcement learning, hierarchical planning and plan execution, and symbolic/sub-symbolic integration in general. The learned model is stored in the form of hierarchical representations with the following properties: 1) they are increasingly more abstract, but can retain details when needed, and 2) they are easy to manipulate in their local and symbolic-like form, thus also allowing one to observe the learning process at each level of abstraction. On all levels of the system, the representation of the data can be interpreted in both a symbolic and a sub-symbolic manner. This enables the architecture to learn efficiently using sub-symbolic methods and to employ symbolic inference.
A Systematic and Meta-analysis Survey of Whale Optimization Algorithm
Authors: Hardi M. Mohammed, Shahla U. Umar, Tarik A. Rashid
Date: 2019-03-20 00:00:00
Tasks: Survey

Whale Optimization Algorithm (WOA) is a nature-inspired meta-heuristic
optimization algorithm, which was proposed by Mirjalili and Lewis in 2016. This
algorithm has shown its ability to solve many problems. Comprehensive surveys
have been conducted about some other nature-inspired algorithms, such as ABC,
PSO, etc.Nonetheless, no survey search work has been conducted on WOA.
Therefore, in this paper, a systematic and meta analysis survey of WOA is
conducted to help researchers to use it in different areas or hybridize it with
other common algorithms. Thus, WOA is presented in depth in terms of
algorithmic backgrounds, its characteristics, limitations, modifications,
hybridizations, and applications. Next, WOA performances are presented to solve
different problems. Then, the statistical results of WOA modifications and
hybridizations are established and compared with the most common optimization
algorithms and WOA. The survey's results indicate that WOA performs better than
other common algorithms in terms of convergence speed and balancing between
exploration and exploitation. WOA modifications and hybridizations also perform
well compared to WOA. In addition, our investigation paves a way to present a
new technique by hybridizing both WOA and BAT algorithms. The BAT algorithm is
used for the exploration phase, whereas the WOA algorithm is used for the
exploitation phase. Finally, statistical results obtained from WOA-BAT are very
competitive and better than WOA in 16 benchmarks functions. WOA-BAT also
outperforms well in 13 functions from CEC2005 and 7 functions from CEC2019.
Distributed Vector Representations of Folksong Motifs
Authors: Aitor Arronte-Alvarez, Francisco Gómez-Martin
Date: 2019-03-20 00:00:00

This article presents a distributed vector representation model for learning
folksong motifs. A skip-gram version of word2vec with negative sampling is used
to represent high quality embeddings. Motifs from the Essen Folksong collection
are compared based on their cosine similarity. A new evaluation method for
testing the quality of the embeddings based on a melodic similarity task is
presented to show how the vector space can represent complex contextual
features, and how it can be utilized for the study of folksong variation.
Byzantine Fault Tolerant Distributed Linear Regression
Authors: Nirupam Gupta, Nitin H. Vaidya
Date: 2019-03-20 00:00:00
Tasks: Distributed Optimization, regression

This paper considers the problem of Byzantine fault tolerance in distributed
linear regression in a multi-agent system. However, the proposed algorithms are
given for a more general class of distributed optimization problems, of which
distributed linear regression is a special case. The system comprises of a
server and multiple agents, where each agent is holding a certain number of
data points and responses that satisfy a linear relationship (could be noisy).
The objective of the server is to determine this relationship, given that some
of the agents in the system (up to a known number) are Byzantine faulty (aka.
actively adversarial). We show that the server can achieve this objective, in a
deterministic manner, by robustifying the original distributed gradient descent
method using norm based filters, namely 'norm filtering' and 'norm-cap
filtering', incurring an additional log-linear computation cost in each
iteration. The proposed algorithms improve upon the existing methods on three
levels: i) no assumptions are required on the probability distribution of data
points, ii) system can be partially asynchronous, and iii) the computational
overhead (in order to handle Byzantine faulty agents) is log-linear in number
of agents and linear in dimension of data points. The proposed algorithms
differ from each other in the assumptions made for their correctness, and the
gradient filter they use.
Affordance Learning In Direct Perception for Autonomous Driving
Authors: Chen Sun, Jean M. Uwabeza Vianney, Dongpu Cao
Date: 2019-03-20 00:00:00
Tasks: Attribute, Autonomous Driving, Autonomous Vehicles, road scene understanding, Scene Understanding

Recent development in autonomous driving involves high-level computer vision
and detailed road scene understanding. Today, most autonomous vehicles are
using mediated perception approach for path planning and control, which highly
rely on high-definition 3D maps and real time sensors. Recent research efforts
aim to substitute the massive HD maps with coarse road attributes. In this
paper, we follow the direct perception based method to train a deep neural
network for affordance learning in autonomous driving. Our goal in this work is
to develop the affordance learning model based on freely available Google
Street View panoramas and Open Street Map road vector attributes. Driving scene
understanding can be achieved by learning affordances from the images captured
by car-mounted cameras. Such scene understanding by learning affordances may be
useful for corroborating base maps such as HD maps so that the required data
storage space is minimized and available for processing in real time. We
compare capability in road attribute identification between human volunteers
and our model by experimental evaluation. Our results indicate that this method
could act as a cheaper way for training data collection in autonomous driving.
The cross validation results also indicate the effectiveness of our model.
Noisy Accelerated Power Method for Eigenproblems with Applications
Authors: Vien V. Mai, Mikael Johansson
Date: 2019-03-20 00:00:00

This paper introduces an efficient algorithm for finding the dominant
generalized eigenvectors of a pair of symmetric matrices. Combining tools from
approximation theory and convex optimization, we develop a simple scalable
algorithm with strong theoretical performance guarantees. More precisely, the
algorithm retains the simplicity of the well-known power method but enjoys the
asymptotic iteration complexity of the powerful Lanczos method. Unlike these
classic techniques, our algorithm is designed to decompose the overall problem
into a series of subproblems that only need to be solved approximately. The
combination of good initializations, fast iterative solvers, and appropriate
error control in solving the subproblems lead to a linear running time in the
input sizes compared to the superlinear time for the traditional methods. The
improved running time immediately offers acceleration for several applications.
As an example, we demonstrate how the proposed algorithm can be used to
accelerate canonical correlation analysis, which is a fundamental statistical
tool for learning of a low-dimensional representation of high-dimensional
objects. Numerical experiments on real-world data sets confirm that our
approach yields significant improvements over the current state-of-the-art.
Batch Policy Learning under Constraints
Authors: Hoang M. Le, Cameron Voloshin, Yisong Yue
Date: 2019-03-20 00:00:00
Tasks: Reinforcement Learning

When learning policies for real-world domains, two important questions arise:
(i) how to efficiently use pre-collected off-policy, non-optimal behavior data;
and (ii) how to mediate among different competing objectives and constraints.
We thus study the problem of batch policy learning under multiple constraints,
and offer a systematic solution. We first propose a flexible meta-algorithm
that admits any batch reinforcement learning and online learning procedure as
subroutines. We then present a specific algorithmic instantiation and provide
performance guarantees for the main objective and all constraints. To certify
constraint satisfaction, we propose a new and simple method for off-policy
policy evaluation (OPE) and derive PAC-style bounds. Our algorithm achieves
strong empirical results in different domains, including in a challenging
problem of simulated car driving subject to multiple constraints such as lane
keeping and smooth driving. We also show experimentally that our OPE method
outperforms other popular OPE techniques on a standalone basis, especially in a
high-dimensional setting.
Accelerating Gradient Boosting Machine
Authors: Haihao Lu, Sai Praneeth Karimireddy, Natalia Ponomareva, Vahab Mirrokni
Date: 2019-03-20 00:00:00

Gradient Boosting Machine (GBM) is an extremely powerful supervised learning algorithm that is widely used in practice. GBM routinely features as a leading algorithm in machine learning competitions such as Kaggle and the KDDCup. In this work, we propose Accelerated Gradient Boosting Machine (AGBM) by incorporating Nesterov's acceleration techniques into the design of GBM. The difficulty in accelerating GBM lies in the fact that weak (inexact) learners are commonly used, and therefore the errors can accumulate in the momentum term. To overcome it, we design a "corrected pseudo residual" and fit best weak learner to this corrected pseudo residual, in order to perform the z-update. Thus, we are able to derive novel computational guarantees for AGBM. This is the first GBM type of algorithm with theoretically-justified accelerated convergence rate. Finally we demonstrate with a number of numerical experiments the effectiveness of AGBM over conventional GBM in obtaining a model with good training and/or testing data fidelity.
LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving
Authors: Gregory P. Meyer, Ankit Laddha, Eric Kee, Carlos Vallespi-Gonzalez, Carl K. Wellington
Date: 2019-03-20 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Object Detection, Autonomous Driving, object-detection, Object Detection

In this paper, we present LaserNet, a computationally efficient method for 3D
object detection from LiDAR data for autonomous driving. The efficiency results
from processing LiDAR data in the native range view of the sensor, where the
input data is naturally compact. Operating in the range view involves well
known challenges for learning, including occlusion and scale variation, but it
also provides contextual information based on how the sensor data was captured.
Our approach uses a fully convolutional network to predict a multimodal
distribution over 3D boxes for each point and then it efficiently fuses these
distributions to generate a prediction for each object. Experiments show that
modeling each detection as a distribution rather than a single deterministic
box leads to better overall detection performance. Benchmark results show that
this approach has significantly lower runtime than other recent detectors and
that it achieves state-of-the-art performance when compared on a large dataset
that has enough data to overcome the challenges of training on the range view.
Using Local Experiences for Global Motion Planning
Authors: Constantinos Chamzas, Anshumali Shrivastava, Lydia E. Kavraki
Date: 2019-03-20 00:00:00
Tasks: Motion Planning

Sampling-based planners are effective in many real-world applications such as
robotics manipulation, navigation, and even protein modeling. However, it is
often challenging to generate a collision-free path in environments where key
areas are hard to sample. In the absence of any prior information,
sampling-based planners are forced to explore uniformly or heuristically, which
can lead to degraded performance. One way to improve performance is to use
prior knowledge of environments to adapt the sampling strategy to the problem
at hand. In this work, we decompose the workspace into local primitives,
memorizing local experiences by these primitives in the form of local samplers,
and store them in a database. We synthesize an efficient global sampler by
retrieving local experiences relevant to the given situation. Our method
transfers knowledge effectively between diverse environments that share local
primitives and speeds up the performance dramatically. Our results show, in
terms of solution time, an improvement of multiple orders of magnitude in two
traditionally challenging high-dimensional problems compared to
state-of-the-art approaches.
Efficient Inner Product Approximation in Hybrid Spaces
Authors: Xiang Wu, Ruiqi Guo, David Simcha, Dave Dopson, Sanjiv Kumar
Date: 2019-03-20 00:00:00
Tasks: Network Embedding

Many emerging use cases of data mining and machine learning operate on large
datasets with data from heterogeneous sources, specifically with both sparse
and dense components. For example, dense deep neural network embedding vectors
are often used in conjunction with sparse textual features to provide high
dimensional hybrid representation of documents. Efficient search in such hybrid
spaces is very challenging as the techniques that perform well for sparse
vectors have little overlap with those that work well for dense vectors.
Popular techniques like Locality Sensitive Hashing (LSH) and its data-dependent
variants also do not give good accuracy in high dimensional hybrid spaces. Even
though hybrid scenarios are becoming more prevalent, currently there exist no
efficient techniques in literature that are both fast and accurate. In this
paper, we propose a technique that approximates the inner product computation
in hybrid vectors, leading to substantial speedup in search while maintaining
high accuracy. We also propose efficient data structures that exploit modern
computer architectures, resulting in orders of magnitude faster search than the
existing baselines. The performance of the proposed method is demonstrated on
several datasets including a very large scale industrial dataset containing one
billion vectors in a billion dimensional space, achieving over 10x speedup and
higher accuracy against competitive baselines.
Implicit Generation and Generalization in Energy-Based Models
Authors: Yilun Du, Igor Mordatch
Date: 2019-03-20 00:00:00
Tasks: General Classification, Image Reconstruction, Robust classification

Energy based models (EBMs) are appealing due to their generality and simplicity in likelihood modeling, but have been traditionally difficult to train. We present techniques to scale MCMC based EBM training on continuous neural networks, and we show its success on the high-dimensional data domains of ImageNet32x32, ImageNet128x128, CIFAR-10, and robotic hand trajectories, achieving better samples than other likelihood models and nearing the performance of contemporary GAN approaches, while covering all modes of the data. We highlight some unique capabilities of implicit generation such as compositionality and corrupt image reconstruction and inpainting. Finally, we show that EBMs are useful models across a wide variety of tasks, achieving state-of-the-art out-of-distribution classification, adversarially robust classification, state-of-the-art continual online class learning, and coherent long term predicted trajectory rollouts.
Im2Pencil: Controllable Pencil Illustration from Photographs
Authors: Yijun Li, Chen Fang, Aaron Hertzmann, Eli Shechtman, Ming-Hsuan Yang
Date: 2019-03-20 00:00:00
Proceeding: CVPR 2019 6
Tasks: Diversity, Translation

We propose a high-quality photo-to-pencil translation method with
fine-grained control over the drawing style. This is a challenging task due to
multiple stroke types (e.g., outline and shading), structural complexity of
pencil shading (e.g., hatching), and the lack of aligned training data pairs.
To address these challenges, we develop a two-branch model that learns separate
filters for generating sketchy outlines and tonal shading from a collection of
pencil drawings. We create training data pairs by extracting clean outlines and
tonal illustrations from original pencil drawings using image filtering
techniques, and we manually label the drawing styles. In addition, our model
creates different pencil styles (e.g., line sketchiness and shading style) in a
user-controllable manner. Experimental results on different types of pencil
drawings show that the proposed algorithm performs favorably against existing
methods in terms of quality, diversity and user evaluations.