pwc-archive/papers-with-abstracts huggingface dataset:
Formal Security Analysis of Neural Networks using Symbolic Intervals
Authors: Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, Suman Jana
Date: 2018-04-28 00:00:00
Tasks: Autonomous Vehicles, Collision Avoidance

Due to the increasing deployment of Deep Neural Networks (DNNs) in real-world
security-critical domains including autonomous vehicles and collision avoidance
systems, formally checking security properties of DNNs, especially under
different attacker capabilities, is becoming crucial. Most existing security
testing techniques for DNNs try to find adversarial examples without providing
any formal security guarantees about the non-existence of such adversarial
examples. Recently, several projects have used different types of
Satisfiability Modulo Theory (SMT) solvers to formally check security
properties of DNNs. However, all of these approaches are limited by the high
overhead caused by the solver.
  In this paper, we present a new direction for formally checking security
properties of DNNs without using SMT solvers. Instead, we leverage interval
arithmetic to compute rigorous bounds on the DNN outputs. Our approach, unlike
existing solver-based approaches, is easily parallelizable. We further present
symbolic interval analysis along with several other optimizations to minimize
overestimations of output bounds.
  We design, implement, and evaluate our approach as part of ReluVal, a system
for formally checking security properties of Relu-based DNNs. Our extensive
empirical results show that ReluVal outperforms Reluplex, a state-of-the-art
solver-based system, by 200 times on average. On a single 8-core machine
without GPUs, within 4 hours, ReluVal is able to verify a security property
that Reluplex deemed inconclusive due to timeout after running for more than 5
days. Our experiments demonstrate that symbolic interval analysis is a
promising new direction towards rigorously analyzing different security
properties of DNNs.
The Neural Painter: Multi-Turn Image Generation
Authors: Ryan Y. Benmalek, Claire Cardie, Serge Belongie, Xiadong He, Jianfeng Gao
Date: 2018-06-16 00:00:00
Tasks: Benchmarking, Conditional Image Generation, Image Generation

In this work we combine two research threads from Vision/ Graphics and
Natural Language Processing to formulate an image generation task conditioned
on attributes in a multi-turn setting. By multiturn, we mean the image is
generated in a series of steps of user-specified conditioning information. Our
proposed approach is practically useful and offers insights into neural
interpretability. We introduce a framework that includes a novel training
algorithm as well as model improvements built for the multi-turn setting. We
demonstrate that this framework generates a sequence of images that match the
given conditioning information and that this task is useful for more detailed
benchmarking and analysis of conditional image generation methods.
Component SPD Matrices: A lower-dimensional discriminative data descriptor for image set classification
Authors: Kai-Xuan Chen, Xiao-Jun Wu
Date: 2018-06-16 00:00:00
Tasks: General Classification

In the domain of pattern recognition, using the SPD (Symmetric Positive
Definite) matrices to represent data and taking the metrics of resulting
Riemannian manifold into account have been widely used for the task of image
set classification. In this paper, we propose a new data representation
framework for image sets named CSPD (Component Symmetric Positive Definite).
Firstly, we obtain sub-image sets by dividing the image set into square blocks
with the same size, and use traditional SPD model to describe them. Then, we
use the results of the Riemannian kernel on SPD matrices as similarities of
corresponding sub-image sets. Finally, the CSPD matrix appears in the form of
the kernel matrix for all the sub-image sets, and CSPDi,j denotes the
similarity between i-th sub-image set and j-th sub-image set. Here, the
Riemannian kernel is shown to satisfy the Mercer's theorem, so our proposed
CSPD matrix is symmetric and positive definite and also lies on a Riemannian
manifold. On three benchmark datasets, experimental results show that CSPD is a
lower-dimensional and more discriminative data descriptor for the task of image
set classification.
Riemannian kernel based Nyström method for approximate infinite-dimensional covariance descriptors with application to image set classification
Authors: Kai-Xuan Chen, Xiao-Jun Wu, Rui Wang, Josef Kittler
Date: 2018-06-16 00:00:00
Tasks: General Classification

In the domain of pattern recognition, using the CovDs (Covariance Descriptors) to represent data and taking the metrics of the resulting Riemannian manifold into account have been widely adopted for the task of image set classification. Recently, it has been proven that infinite-dimensional CovDs are more discriminative than their low-dimensional counterparts. However, the form of infinite-dimensional CovDs is implicit and the computational load is high. We propose a novel framework for representing image sets by approximating infinite-dimensional CovDs in the paradigm of the Nystr\"om method based on a Riemannian kernel. We start by modeling the images via CovDs, which lie on the Riemannian manifold spanned by SPD (Symmetric Positive Definite) matrices. We then extend the Nystr\"om method to the SPD manifold and obtain the approximations of CovDs in RKHS (Reproducing Kernel Hilbert Space). Finally, we approximate infinite-dimensional CovDs via these approximations. Empirically, we apply our framework to the task of image set classification. The experimental results obtained on three benchmark datasets show that our proposed approximate infinite-dimensional CovDs outperform the original CovDs.
Learning Factorized Multimodal Representations
Authors: Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency, Ruslan Salakhutdinov
Date: 2018-06-16 00:00:00
Proceeding: ICLR 2019 5
Tasks: Representation Learning

Learning multimodal representations is a fundamentally complex research problem due to the presence of multiple heterogeneous sources of information. Although the presence of multiple modalities provides additional valuable information, there are two key challenges to address when learning from multimodal data: 1) models must learn the complex intra-modal and cross-modal interactions for prediction and 2) models must be robust to unexpected missing or noisy modalities during testing. In this paper, we propose to optimize for a joint generative-discriminative objective across multimodal data and labels. We introduce a model that factorizes representations into two sets of independent factors: multimodal discriminative and modality-specific generative factors. Multimodal discriminative factors are shared across all modalities and contain joint multimodal features required for discriminative tasks such as sentiment prediction. Modality-specific generative factors are unique for each modality and contain the information required for generating data. Experimental results show that our model is able to learn meaningful multimodal representations that achieve state-of-the-art or competitive performance on six multimodal datasets. Our model demonstrates flexible generative capabilities by conditioning on independent factors and can reconstruct missing modalities without significantly impacting performance. Lastly, we interpret our factorized representations to understand the interactions that influence multimodal learning.
Ensemble Pruning based on Objection Maximization with a General Distributed Framework
Authors: Yijun Bian, Yijun Wang, Yaqiang Yao, Huanhuan Chen
Date: 2018-06-13 00:00:00
Tasks: Diversity, Ensemble Learning, Ensemble Pruning

Ensemble pruning, selecting a subset of individual learners from an original ensemble, alleviates the deficiencies of ensemble learning on the cost of time and space. Accuracy and diversity serve as two crucial factors while they usually conflict with each other. To balance both of them, we formalize the ensemble pruning problem as an objection maximization problem based on information entropy. Then we propose an ensemble pruning method including a centralized version and a distributed version, in which the latter is to speed up the former. At last, we extract a general distributed framework for ensemble pruning, which can be widely suitable for most of the existing ensemble pruning methods and achieve less time consuming without much accuracy degradation. Experimental results validate the efficiency of our framework and methods, particularly concerning a remarkable improvement of the execution speed, accompanied by gratifying accuracy performance.
Revisiting Deep Intrinsic Image Decompositions
Authors: Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, David Wipf
Date: 2017-01-11 00:00:00
Proceeding: CVPR 2018 6

While invaluable for many computer vision applications, decomposing a natural
image into intrinsic reflectance and shading layers represents a challenging,
underdetermined inverse problem. As opposed to strict reliance on conventional
optimization or filtering solutions with strong prior assumptions, deep
learning based approaches have also been proposed to compute intrinsic image
decompositions when granted access to sufficient labeled training data. The
downside is that current data sources are quite limited, and broadly speaking
fall into one of two categories: either dense fully-labeled images in
synthetic/narrow settings, or weakly-labeled data from relatively diverse
natural scenes. In contrast to many previous learning-based approaches, which
are often tailored to the structure of a particular dataset (and may not work
well on others), we adopt core network structures that universally reflect
loose prior knowledge regarding the intrinsic image formation process and can
be largely shared across datasets. We then apply flexibly supervised loss
layers that are customized for each source of ground truth labels. The
resulting deep architecture achieves state-of-the-art results on all of the
major intrinsic image benchmarks, and runs considerably faster than most at
test time.
On the Complexity of Detecting Convexity over a Box
Authors: Amir Ali Ahmadi, Georgina Hall
Date: 2018-06-16 00:00:00
Tasks: Open-Ended Question Answering

It has recently been shown that the problem of testing global convexity of
polynomials of degree four is {strongly} NP-hard, answering an open question of
N.Z. Shor. This result is minimal in the degree of the polynomial when global
convexity is of concern. In a number of applications however, one is interested
in testing convexity only over a compact region, most commonly a box (i.e.,
hyper-rectangle). In this paper, we show that this problem is also strongly
NP-hard, in fact for polynomials of degree as low as three. This result is
minimal in the degree of the polynomial and in some sense justifies why
convexity detection in nonlinear optimization solvers is limited to quadratic
functions or functions with special structure. As a byproduct, our proof shows
that the problem of testing whether all matrices in an interval family are
positive semidefinite is strongly NP-hard. This problem, which was previously
shown to be (weakly) NP-hard by Nemirovski, is of independent interest in the
theory of robust control.
Random Forest for Label Ranking
Authors: Yangming Zhou, Guoping Qiu
Date: 2016-08-27 00:00:00

Label ranking aims to learn a mapping from instances to rankings over a
finite number of predefined labels. Random forest is a powerful and one of the
most successful general-purpose machine learning algorithms of modern times. In
this paper, we present a powerful random forest label ranking method which uses
random decision trees to retrieve nearest neighbors. We have developed a novel
two-step rank aggregation strategy to effectively aggregate neighboring
rankings discovered by the random forest into a final predicted ranking.
Compared with existing methods, the new random forest method has many
advantages including its intrinsically scalable tree data structure, highly
parallel-able computational architecture and much superior performance. We
present extensive experimental results to demonstrate that our new method
achieves the highly competitive performance compared with state-of-the-art
methods for datasets with complete ranking and datasets with only partial
ranking information.
Discrete Sequential Prediction of Continuous Actions for Deep RL
Authors: Luke Metz, Julian Ibarz, Navdeep Jaitly, James Davidson
Date: 2017-05-14 00:00:00
Proceeding: ICLR 2018 1
Tasks: continuous-control, Continuous Control, Prediction, Q-Learning, Structured Prediction

It has long been assumed that high dimensional continuous control problems cannot be solved effectively by discretizing individual dimensions of the action space due to the exponentially large number of bins over which policies would have to be learned. In this paper, we draw inspiration from the recent success of sequence-to-sequence models for structured prediction problems to develop policies over discretized spaces. Central to this method is the realization that complex functions over high dimensional spaces can be modeled by neural networks that predict one dimension at a time. Specifically, we show how Q-values and policies over continuous spaces can be modeled using a next step prediction model over discretized dimensions. With this parameterization, it is possible to both leverage the compositional structure of action spaces during learning, as well as compute maxima over action spaces (approximately). On a simple example task we demonstrate empirically that our method can perform global search, which effectively gets around the local optimization issues that plague DDPG. We apply the technique to off-policy (Q-learning) methods and show that our method can achieve the state-of-the-art for off-policy methods on several continuous control tasks.
Semantic Video Segmentation: A Review on Recent Approaches
Authors: Mohammad Hajizadeh Saffar, Mohsen Fayyaz, Mohammad Sabokrou, Mahmood Fathy
Date: 2018-06-16 00:00:00
Tasks: Segmentation, Semantic Segmentation, Video Segmentation, Video Semantic Segmentation

This paper gives an overview on semantic segmentation consists of an
explanation of this field, it's status and relation with other vision
fundamental tasks, different datasets and common evaluation parameters that
have been used by researchers. This survey also includes an overall review on a
variety of recent approaches (RDF, MRF, CRF, etc.) and their advantages and
challenges and shows the superiority of CNN-based semantic segmentation systems
on CamVid and NYUDv2 datasets. In addition, some areas that is ideal for future
work have mentioned.
BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learning
Authors: Boris Ivanovic, James Harrison, Apoorva Sharma, Mo Chen, Marco Pavone
Date: 2018-06-16 00:00:00
Tasks: continuous-control, Continuous Control, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Model-free Reinforcement Learning (RL) offers an attractive approach to learn
control policies for high-dimensional systems, but its relatively poor sample
complexity often forces training in simulated environments. Even in simulation,
goal-directed tasks whose natural reward function is sparse remain intractable
for state-of-the-art model-free algorithms for continuous control. The
bottleneck in these tasks is the prohibitive amount of exploration required to
obtain a learning signal from the initial state of the system. In this work, we
leverage physical priors in the form of an approximate system dynamics model to
design a curriculum scheme for a model-free policy optimization algorithm. Our
Backward Reachability Curriculum (BaRC) begins policy training from states that
require a small number of actions to accomplish the task, and expands the
initial state distribution backwards in a dynamically-consistent manner once
the policy optimization algorithm demonstrates sufficient performance. BaRC is
general, in that it can accelerate training of any model-free RL algorithm on a
broad class of goal-directed continuous control MDPs. Its curriculum strategy
is physically intuitive, easy-to-tune, and allows incorporating physical priors
to accelerate training without hindering the performance, flexibility, and
applicability of the model-free RL algorithm. We evaluate our approach on two
representative dynamic robotic learning problems and find substantial
performance improvement relative to previous curriculum generation techniques
and naive exploration strategies.
Object Level Visual Reasoning in Videos
Authors: Fabien Baradel, Natalia Neverova, Christian Wolf, Julien Mille, Greg Mori
Date: 2018-06-16 00:00:00
Proceeding: ECCV 2018 9
Tasks: Activity Recognition, Human Activity Recognition, Object, object-detection, Object Detection, Visual Reasoning

Human activity recognition is typically addressed by detecting key concepts
like global and local motion, features related to object classes present in the
scene, as well as features related to the global context. The next open
challenges in activity recognition require a level of understanding that pushes
beyond this and call for models with capabilities for fine distinction and
detailed comprehension of interactions between actors and objects in a scene.
We propose a model capable of learning to reason about semantically meaningful
spatiotemporal interactions in videos. The key to our approach is a choice of
performing this reasoning at the object level through the integration of state
of the art object detection networks. This allows the model to learn detailed
spatial interactions that exist at a semantic, object-interaction relevant
level. We evaluate our method on three standard datasets (Twenty-BN
Something-Something, VLOG and EPIC Kitchens) and achieve state of the art
results on all of them. Finally, we show visualizations of the interactions
learned by the model, which illustrate object classes and their interactions
corresponding to different activity classes.
Knowledge-enriched Two-layered Attention Network for Sentiment Analysis
Authors: Abhishek Kumar, Daisuke Kawahara, Sadao Kurohashi
Date: 2018-05-20 00:00:00
Proceeding: NAACL 2018 6
Tasks: Graph Embedding, Knowledge Graph Embedding, Sentiment Analysis, Vocal Bursts Valence Prediction

We propose a novel two-layered attention network based on Bidirectional Long
Short-Term Memory for sentiment analysis. The novel two-layered attention
network takes advantage of the external knowledge bases to improve the
sentiment prediction. It uses the Knowledge Graph Embedding generated using the
WordNet. We build our model by combining the two-layered attention network with
the supervised model based on Support Vector Regression using a Multilayer
Perceptron network for sentiment analysis. We evaluate our model on the
benchmark dataset of SemEval 2017 Task 5. Experimental results show that the
proposed model surpasses the top system of SemEval 2017 Task 5. The model
performs significantly better by improving the state-of-the-art system at
SemEval 2017 Task 5 by 1.7 and 3.7 points for sub-tracks 1 and 2 respectively.
New Techniques for Preserving Global Structure and Denoising with Low Information Loss in Single-Image Super-Resolution
Authors: Yijie Bei, Alex Damian, Shijia Hu, Sachit Menon, Nikhil Ravi, Cynthia Rudin
Date: 2018-05-09 00:00:00
Tasks: Denoising, Image Super-Resolution, Super-Resolution

This work identifies and addresses two important technical challenges in
single-image super-resolution: (1) how to upsample an image without magnifying
noise and (2) how to preserve large scale structure when upsampling. We
summarize the techniques we developed for our second place entry in Track 1
(Bicubic Downsampling), seventh place entry in Track 2 (Realistic Adverse
Conditions), and seventh place entry in Track 3 (Realistic difficult) in the
2018 NTIRE Super-Resolution Challenge. Furthermore, we present new neural
network architectures that specifically address the two challenges listed
above: denoising and preservation of large-scale structure.
Anticipation in Human-Robot Cooperation: A Recurrent Neural Network Approach for Multiple Action Sequences Prediction
Authors: Paul Schydlo, Mirko Rakovic, Lorenzo Jamone, José Santos-Victor
Date: 2018-02-28 00:00:00
Tasks: Decoder, feature selection, Prediction

Close human-robot cooperation is a key enabler for new developments in
advanced manufacturing and assistive applications. Close cooperation require
robots that can predict human actions and intent, and understand human
non-verbal cues. Recent approaches based on neural networks have led to
encouraging results in the human action prediction problem both in continuous
and discrete spaces. Our approach extends the research in this direction. Our
contributions are three-fold. First, we validate the use of gaze and body pose
cues as a means of predicting human action through a feature selection method.
Next, we address two shortcomings of existing literature: predicting multiple
and variable-length action sequences. This is achieved by introducing an
encoder-decoder recurrent neural network topology in the discrete action
prediction problem. In addition, we theoretically demonstrate the importance of
predicting multiple action sequences as a means of estimating the stochastic
reward in a human robot cooperation scenario. Finally, we show the ability to
effectively train the prediction model on a action prediction dataset,
involving human motion data, and explore the influence of the model's
parameters on its performance. Source code repository:
https://github.com/pschydlo/ActionAnticipation
Image Transformer
Authors: Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Łukasz Kaiser, Noam Shazeer, Alexander Ku, Dustin Tran
Date: 2018-02-15 00:00:00
Tasks: Decoder, Density Estimation, Image Generation, Image Super-Resolution, Super-Resolution

Image generation has been successfully cast as an autoregressive sequence
generation or transformation problem. Recent work has shown that self-attention
is an effective way of modeling textual sequences. In this work, we generalize
a recently proposed model architecture based on self-attention, the
Transformer, to a sequence modeling formulation of image generation with a
tractable likelihood. By restricting the self-attention mechanism to attend to
local neighborhoods we significantly increase the size of images the model can
process in practice, despite maintaining significantly larger receptive fields
per layer than typical convolutional neural networks. While conceptually
simple, our generative models significantly outperform the current state of the
art in image generation on ImageNet, improving the best published negative
log-likelihood on ImageNet from 3.83 to 3.77. We also present results on image
super-resolution with a large magnification ratio, applying an encoder-decoder
configuration of our architecture. In a human evaluation study, we find that
images generated by our super-resolution model fool human observers three times
more often than the previous state of the art.
Characterizing Departures from Linearity in Word Translation
Authors: Ndapa Nakashole, Raphael Flauger
Date: 2018-06-07 00:00:00
Proceeding: ACL 2018 7
Tasks: Machine Translation, Translation, Word Translation

We investigate the behavior of maps learned by machine translation methods.
The maps translate words by projecting between word embedding spaces of
different languages. We locally approximate these maps using linear maps, and
find that they vary across the word embedding space. This demonstrates that the
underlying maps are non-linear. Importantly, we show that the locally linear
maps vary by an amount that is tightly correlated with the distance between the
neighborhoods on which they are trained. Our results can be used to test
non-linear methods, and to drive the design of more accurate maps for word
translation.