pwc-archive/papers-with-abstracts huggingface dataset:
Hyperbolic Attention Networks
Authors: Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, Nando de Freitas
Date: 2018-05-24 00:00:00
Proceeding: ICLR 2019 5
Tasks: Machine Translation, Question Answering, Translation, Visual Question Answering, Visual Question Answering (VQA)

We introduce hyperbolic attention networks to endow neural networks with
enough capacity to match the complexity of data with hierarchical and power-law
structure. A few recent approaches have successfully demonstrated the benefits
of imposing hyperbolic geometry on the parameters of shallow networks. We
extend this line of work by imposing hyperbolic geometry on the activations of
neural networks. This allows us to exploit hyperbolic geometry to reason about
embeddings produced by deep networks. We achieve this by re-expressing the
ubiquitous mechanism of soft attention in terms of operations defined for
hyperboloid and Klein models. Our method shows improvements in terms of
generalization on neural machine translation, learning on graphs and visual
question answering tasks while keeping the neural representations compact.
Entropy and mutual information in models of deep neural networks
Authors: Marylou Gabrié, Andre Manoel, Clément Luneau, Jean Barbier, Nicolas Macris, Florent Krzakala, Lenka Zdeborová
Date: 2018-05-24 00:00:00
Proceeding: NeurIPS 2018 12

We examine a class of deep learning models with a tractable method to compute
information-theoretic quantities. Our contributions are three-fold: (i) We show
how entropies and mutual informations can be derived from heuristic statistical
physics methods, under the assumption that weight matrices are independent and
orthogonally-invariant. (ii) We extend particular cases in which this result is
known to be rigorously exact by providing a proof for two-layers networks with
Gaussian random weights, using the recently introduced adaptive interpolation
method. (iii) We propose an experiment framework with generative models of
synthetic datasets, on which we train deep neural networks with a weight
constraint designed so that the assumption in (i) is verified during learning.
We study the behavior of entropies and mutual informations throughout learning
and conclude that, in the proposed setting, the relationship between
compression and generalization remains elusive.
Efficient Inference in Multi-task Cox Process Models
Authors: Virginia Aglietti, Theodoros Damoulas, Edwin Bonilla
Date: 2018-05-24 00:00:00
Tasks: Gaussian Processes, Point Processes, Variational Inference

We generalize the log Gaussian Cox process (LGCP) framework to model multiple
correlated point data jointly. The observations are treated as realizations of
multiple LGCPs, whose log intensities are given by linear combinations of
latent functions drawn from Gaussian process priors. The combination
coefficients are also drawn from Gaussian processes and can incorporate
additional dependencies. We derive closed-form expressions for the moments of
the intensity functions and develop an efficient variational inference
algorithm that is orders of magnitude faster than competing deterministic and
stochastic approximations of multivariate LGCP, coregionalization models, and
multi-task permanental processes. Our approach outperforms these benchmarks in
multiple problems, offering the current state of the art in modeling
multivariate point processes.
Mining Procedures from Technical Support Documents
Authors: Abhirut Gupta, Abhay Khosla, Gautam Singh, Gargi Dasgupta
Date: 2018-05-24 00:00:00
Tasks: Diagnostic, Question Answering

Guided troubleshooting is an inherent task in the domain of technical support
services. When a customer experiences an issue with the functioning of a
technical service or a product, an expert user helps guide the customer through
a set of steps comprising a troubleshooting procedure. The objective is to
identify the source of the problem through a set of diagnostic steps and
observations, and arrive at a resolution. Procedures containing these set of
diagnostic steps and observations in response to different problems are common
artifacts in the body of technical support documentation. The ability to use
machine learning and linguistics to understand and leverage these procedures
for applications like intelligent chatbots or robotic process automation, is
crucial. Existing research on question answering or intelligent chatbots does
not look within procedures or deep-understand them. In this paper, we outline a
system for mining procedures from technical support documents. We create models
for solving important subproblems like extraction of procedures, identifying
decision points within procedures, identifying blocks of instructions
corresponding to these decision points and mapping instructions within a
decision block. We also release a dataset containing our manual annotations on
publicly available support documents, to promote further research on the
problem.
HotFlip: White-Box Adversarial Examples for Text Classification
Authors: Javid Ebrahimi, Anyi Rao, Daniel Lowd, Dejing Dou
Date: 2017-12-19 00:00:00
Proceeding: ACL 2018 7
Tasks: Classification, General Classification, text-classification, Text Classification

We propose an efficient method to generate white-box adversarial examples to
trick a character-level neural classifier. We find that only a few
manipulations are needed to greatly decrease the accuracy. Our method relies on
an atomic flip operation, which swaps one token for another, based on the
gradients of the one-hot input vectors. Due to efficiency of our method, we can
perform adversarial training which makes the model more robust to attacks at
test time. With the use of a few semantics-preserving constraints, we
demonstrate that HotFlip can be adapted to attack a word-level classifier as
well.
Local SGD Converges Fast and Communicates Little
Authors: Sebastian U. Stich
Date: 2018-05-24 00:00:00
Proceeding: ICLR 2019 5

Mini-batch stochastic gradient descent (SGD) is state of the art in large scale distributed training. The scheme can reach a linear speedup with respect to the number of workers, but this is rarely seen in practice as the scheme often suffers from large network delays and bandwidth limits. To overcome this communication bottleneck recent works propose to reduce the communication frequency. An algorithm of this type is local SGD that runs SGD independently in parallel on different workers and averages the sequences only once in a while. This scheme shows promising results in practice, but eluded thorough theoretical analysis. We prove concise convergence rates for local SGD on convex problems and show that it converges at the same rate as mini-batch SGD in terms of number of evaluated gradients, that is, the scheme achieves linear speedup in the number of workers and mini-batch size. The number of communication rounds can be reduced up to a factor of T^{1/2}---where T denotes the number of total steps---compared to mini-batch SGD. This also holds for asynchronous implementations. Local SGD can also be used for large scale training of deep learning models. The results shown here aim serving as a guideline to further explore the theoretical and practical aspects of local SGD in these applications.
Quantifying Uncertainty in Discrete-Continuous and Skewed Data with Bayesian Deep Learning
Authors: Thomas Vandal, Evan Kodra, Jennifer Dy, Sangram Ganguly, Ramakrishna Nemani, Auroop R. Ganguly
Date: 2018-02-13 00:00:00
Tasks: Management, Super-Resolution, Uncertainty Quantification

Deep Learning (DL) methods have been transforming computer vision with
innovative adaptations to other domains including climate change. For DL to
pervade Science and Engineering (S&E) applications where risk management is a
core component, well-characterized uncertainty estimates must accompany
predictions. However, S&E observations and model-simulations often follow
heavily skewed distributions and are not well modeled with DL approaches, since
they usually optimize a Gaussian, or Euclidean, likelihood loss. Recent
developments in Bayesian Deep Learning (BDL), which attempts to capture
uncertainties from noisy observations, aleatoric, and from unknown model
parameters, epistemic, provide us a foundation. Here we present a
discrete-continuous BDL model with Gaussian and lognormal likelihoods for
uncertainty quantification (UQ). We demonstrate the approach by developing UQ
estimates on `DeepSD', a super-resolution based DL model for Statistical
Downscaling (SD) in climate applied to precipitation, which follows an
extremely skewed distribution. We find that the discrete-continuous models
outperform a basic Gaussian distribution in terms of predictive accuracy and
uncertainty calibration. Furthermore, we find that the lognormal distribution,
which can handle skewed distributions, produces quality uncertainty estimates
at the extremes. Such results may be important across S&E, as well as other
domains such as finance and economics, where extremes are often of significant
interest. Furthermore, to our knowledge, this is the first UQ model in SD where
both aleatoric and epistemic uncertainties are characterized.
Geographical Hidden Markov Tree for Flood Extent Mapping (With Proof Appendix)
Authors: Miao Xie, Zhe Jiang, Arpan Man Sainju
Date: 2018-05-24 00:00:00
Tasks: General Classification, Management

Flood extent mapping plays a crucial role in disaster management and national
water forecasting. Unfortunately, traditional classification methods are often
hampered by the existence of noise, obstacles and heterogeneity in spectral
features as well as implicit anisotropic spatial dependency across class
labels. In this paper, we propose geographical hidden Markov tree, a
probabilistic graphical model that generalizes the common hidden Markov model
from a one dimensional sequence to a two dimensional map. Anisotropic spatial
dependency is incorporated in the hidden class layer with a reverse tree
structure. We also investigate computational algorithms for reverse tree
construction, model parameter learning and class inference. Extensive
evaluations on both synthetic and real world datasets show that proposed model
outperforms multiple baselines in flood mapping, and our algorithms are
scalable on large data sizes.
Keep it Unreal: Bridging the Realism Gap for 2.5D Recognition with Geometry Priors Only
Authors: Sergey Zakharov, Benjamin Planche, Ziyan Wu, Andreas Hutter, Harald Kosch, Slobodan Ilic
Date: 2018-04-24 00:00:00
Tasks: Generative Adversarial Network

With the increasing availability of large databases of 3D CAD models,
depth-based recognition methods can be trained on an uncountable number of
synthetically rendered images. However, discrepancies with the real data
acquired from various depth sensors still noticeably impede progress. Previous
works adopted unsupervised approaches to generate more realistic depth data,
but they all require real scans for training, even if unlabeled. This still
represents a strong requirement, especially when considering
real-life/industrial settings where real training images are hard or impossible
to acquire, but texture-less 3D models are available. We thus propose a novel
approach leveraging only CAD models to bridge the realism gap. Purely trained
on synthetic data, playing against an extensive augmentation pipeline in an
unsupervised manner, our generative adversarial network learns to effectively
segment depth images and recover the clean synthetic-looking depth information
even from partial occlusions. As our solution is not only fully decoupled from
the real domains but also from the task-specific analytics, the pre-processed
scans can be handed to any kind and number of recognition methods also trained
on synthetic data. Through various experiments, we demonstrate how this
simplifies their training and consistently enhances their performance, with
results on par with the same methods trained on real data, and better than
usual approaches doing the reverse mapping.
MobiFace: A Novel Dataset for Mobile Face Tracking in the Wild
Authors: Yiming Lin, Shiyang Cheng, Jie Shen, Maja Pantic
Date: 2018-05-24 00:00:00
Tasks: Face Detection, Object Tracking, Visual Tracking

Face tracking serves as the crucial initial step in mobile applications
trying to analyse target faces over time in mobile settings. However, this
problem has received little attention, mainly due to the scarcity of dedicated
face tracking benchmarks. In this work, we introduce MobiFace, the first
dataset for single face tracking in mobile situations. It consists of 80
unedited live-streaming mobile videos captured by 70 different smartphone users
in fully unconstrained environments. Over $95K$ bounding boxes are manually
labelled. The videos are carefully selected to cover typical smartphone usage.
The videos are also annotated with 14 attributes, including 6 newly proposed
attributes and 8 commonly seen in object tracking. 36 state-of-the-art
trackers, including facial landmark trackers, generic object trackers and
trackers that we have fine-tuned or improved, are evaluated. The results
suggest that mobile face tracking cannot be solved through existing approaches.
In addition, we show that fine-tuning on the MobiFace training data
significantly boosts the performance of deep learning-based trackers,
suggesting that MobiFace captures the unique characteristics of mobile face
tracking. Our goal is to offer the community a diverse dataset to enable the
design and evaluation of mobile face trackers. The dataset, annotations and the
evaluation server will be on \url{https://mobiface.github.io/}.
Standing Wave Decomposition Gaussian Process
Authors: Chi-Ken Lu, Scott Cheng-Hsin Yang, Patrick Shafto
Date: 2018-03-09 00:00:00
Tasks: regression

We propose a Standing Wave Decomposition (SWD) approximation to Gaussian
Process regression (GP). GP involves a costly matrix inversion operation, which
limits applicability to large data analysis. For an input space that can be
approximated by a grid and when correlations among data are short-ranged, the
kernel matrix inversion can be replaced by analytic diagonalization using the
SWD. We show that this approach applies to uni- and multi-dimensional input
data, extends to include longer-range correlations, and the grid can be in a
latent space and used as inducing points. Through simulations, we show that our
approximate method applied to the squared exponential kernel outperforms
existing methods in predictive accuracy per unit time in the regime where data
are plentiful. Our SWD-GP is recommended for regression analyses where there is
a relatively large amount of data and/or there are constraints on computation
time.
Towards Robust Evaluations of Continual Learning
Authors: Sebastian Farquhar, Yarin Gal
Date: 2018-05-24 00:00:00
Tasks: Continual Learning

Experiments used in current continual learning research do not faithfully assess fundamental challenges of learning continually. Instead of assessing performance on challenging and representative experiment designs, recent research has focused on increased dataset difficulty, while still using flawed experiment set-ups. We examine standard evaluations and show why these evaluations make some continual learning approaches look better than they are. We introduce desiderata for continual learning evaluations and explain why their absence creates misleading comparisons. Based on our desiderata we then propose new experiment designs which we demonstrate with various continual learning approaches and datasets. Our analysis calls for a reprioritization of research effort by the community.
Learning convex polyhedra with margin
Authors: Lee-Ad Gottlieb, Eran Kaufman, Aryeh Kontorovich, Gabriel Nivasch
Date: 2018-05-24 00:00:00
Proceeding: NeurIPS 2018 12

We present an improved algorithm for {\em quasi-properly} learning convex polyhedra in the realizable PAC setting from data with a margin. Our learning algorithm constructs a consistent polyhedron as an intersection of about $t \log t$ halfspaces with constant-size margins in time polynomial in $t$ (where $t$ is the number of halfspaces forming an optimal polyhedron). We also identify distinct generalizations of the notion of margin from hyperplanes to polyhedra and investigate how they relate geometrically; this result may have ramifications beyond the learning setting.
Autonomously and Simultaneously Refining Deep Neural Network Parameters by Generative Adversarial Networks
Authors: Burak Kakillioglu, Yantao Lu, Senem Velipasalar
Date: 2018-05-24 00:00:00
Tasks: Generative Adversarial Network

The choice of parameters, and the design of the network architecture are
important factors affecting the performance of deep neural networks. However,
there has not been much work on developing an established and systematic way of
building the structure and choosing the parameters of a neural network, and
this task heavily depends on trial and error and empirical results. Considering
that there are many design and parameter choices, such as the number of neurons
in each layer, the type of activation function, the choice of using drop out or
not, it is very hard to cover every configuration, and find the optimal
structure. In this paper, we propose a novel and systematic method that
autonomously and simultaneously optimizes multiple parameters of any given deep
neural network by using a generative adversarial network (GAN). In our proposed
approach, two different models compete and improve each other progressively
with a GAN-based strategy. Our proposed approach can be used to autonomously
refine the parameters, and improve the accuracy of different deep neural
network architectures. Without loss of generality, the proposed method has been
tested with three different neural network architectures, and three very
different datasets and applications. The results show that the presented
approach can simultaneously and successfully optimize multiple neural network
parameters, and achieve increased accuracy in all three scenarios.
Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation
Authors: Xi Peng, Zhiqiang Tang, Fei Yang, Rogerio Feris, Dimitris Metaxas
Date: 2018-05-24 00:00:00
Proceeding: CVPR 2018 6
Tasks: Data Augmentation, Pose Estimation

Random data augmentation is a critical technique to avoid overfitting in
training deep neural network models. However, data augmentation and network
training are usually treated as two isolated processes, limiting the
effectiveness of network training. Why not jointly optimize the two? We propose
adversarial data augmentation to address this limitation. The main idea is to
design an augmentation network (generator) that competes against a target
network (discriminator) by generating `hard' augmentation operations online.
The augmentation network explores the weaknesses of the target network, while
the latter learns from `hard' augmentations to achieve better performance. We
also design a reward/penalty strategy for effective joint training. We
demonstrate our approach on the problem of human pose estimation and carry out
a comprehensive experimental analysis, showing that our method can
significantly improve state-of-the-art models without additional data efforts.
R-VQA: Learning Visual Relation Facts with Semantic Attention for Visual Question Answering
Authors: Pan Lu, Lei Ji, Wei zhang, Nan Duan, Ming Zhou, Jianyong Wang
Date: 2018-05-24 00:00:00
Tasks: Question Answering, Relation, Semantic Similarity, Semantic Textual Similarity, Visual Question Answering, Visual Question Answering (VQA)

Recently, Visual Question Answering (VQA) has emerged as one of the most
significant tasks in multimodal learning as it requires understanding both
visual and textual modalities. Existing methods mainly rely on extracting image
and question features to learn their joint feature embedding via multimodal
fusion or attention mechanism. Some recent studies utilize external
VQA-independent models to detect candidate entities or attributes in images,
which serve as semantic knowledge complementary to the VQA task. However, these
candidate entities or attributes might be unrelated to the VQA task and have
limited semantic capacities. To better utilize semantic knowledge in images, we
propose a novel framework to learn visual relation facts for VQA. Specifically,
we build up a Relation-VQA (R-VQA) dataset based on the Visual Genome dataset
via a semantic similarity module, in which each data consists of an image, a
corresponding question, a correct answer and a supporting relation fact. A
well-defined relation detector is then adopted to predict visual
question-related relation facts. We further propose a multi-step attention
model composed of visual attention and semantic attention sequentially to
extract related visual knowledge and semantic knowledge. We conduct
comprehensive experiments on the two benchmark datasets, demonstrating that our
model achieves state-of-the-art performance and verifying the benefit of
considering visual relation facts.
Dictionary Learning for Adaptive GPR Landmine Classification
Authors: Fabio Giovanneschi, Kumar Vijay Mishra, Maria Antonia Gonzalez-Huici, Yonina C. Eldar, Joachim H. G. Ender
Date: 2018-05-24 00:00:00
Tasks: Classification, Dictionary Learning, General Classification, GPR, Landmine

Ground penetrating radar (GPR) target detection and classification is a challenging task. Here, we consider online dictionary learning (DL) methods to obtain sparse representations (SR) of the GPR data to enhance feature extraction for target classification via support vector machines. Online methods are preferred because traditional batch DL like K-SVD is not scalable to high-dimensional training sets and infeasible for real-time operation. We also develop Drop-Off MINi-batch Online Dictionary Learning (DOMINODL) which exploits the fact that a lot of the training data may be correlated. The DOMINODL algorithm iteratively considers elements of the training set in small batches and drops off samples which become less relevant. For the case of abandoned anti-personnel landmines classification, we compare the performance of K-SVD with three online algorithms: classical Online Dictionary Learning, its correlation-based variant, and DOMINODL. Our experiments with real data from L-band GPR show that online DL methods reduce learning time by 36-93% and increase mine detection by 4-28% over K-SVD. Our DOMINODL is the fastest and retains similar classification performance as the other two online DL approaches. We use a Kolmogorov-Smirnoff test distance and the Dvoretzky-Kiefer-Wolfowitz inequality for the selection of DL input parameters leading to enhanced classification results. To further compare with state-of-the-art classification approaches, we evaluate a convolutional neural network (CNN) classifier which performs worse than the proposed approach. Moreover, when the acquired samples are randomly reduced by 25%, 50% and 75%, sparse decomposition based classification with DL remains robust while the CNN accuracy is drastically compromised.