pwc-archive/papers-with-abstracts huggingface dataset:
A Biologically Inspired Visual Working Memory for Deep Networks
Authors: Ethan Harris, Mahesan Niranjan, Jonathon Hare
Date: 2019-01-09 00:00:00
Proceeding: ICLR 2019 5
Tasks: Classification, General Classification

The ability to look multiple times through a series of pose-adjusted glimpses
is fundamental to human vision. This critical faculty allows us to understand
highly complex visual scenes. Short term memory plays an integral role in
aggregating the information obtained from these glimpses and informing our
interpretation of the scene. Computational models have attempted to address
glimpsing and visual attention but have failed to incorporate the notion of
memory. We introduce a novel, biologically inspired visual working memory
architecture that we term the Hebb-Rosenblatt memory. We subsequently introduce
a fully differentiable Short Term Attentive Working Memory model (STAWM) which
uses transformational attention to learn a memory over each image it sees. The
state of our Hebb-Rosenblatt memory is embedded in STAWM as the weights space
of a layer. By projecting different queries through this layer we can obtain
goal-oriented latent representations for tasks including classification and
visual reconstruction. Our model obtains highly competitive classification
performance on MNIST and CIFAR-10. As demonstrated through the CelebA dataset,
to perform reconstruction the model learns to make a sequence of updates to a
canvas which constitute a parts-based representation. Classification with the
self supervised representation obtained from MNIST is shown to be in line with
the state of the art models (none of which use a visual attention mechanism).
Finally, we show that STAWM can be trained under the dual constraints of
classification and reconstruction to provide an interpretable visual sketchpad
which helps open the 'black-box' of deep learning.
Individual common dolphin identification via metric embedding learning
Authors: Soren Bouma, Matthew D. M. Pawley, Krista Hupman, Andrew Gilman
Date: 2019-01-09 00:00:00
Tasks: Open Set Learning, Triplet

Photo-identification (photo-id) of dolphin individuals is a commonly used
technique in ecological sciences to monitor state and health of individuals, as
well as to study the social structure and distribution of a population.
Traditional photo-id involves a laborious manual process of matching each
dolphin fin photograph captured in the field to a catalogue of known
individuals.
  We examine this problem in the context of open-set recognition and utilise a
triplet loss function to learn a compact representation of fin images in a
Euclidean embedding, where the Euclidean distance metric represents fin
similarity. We show that this compact representation can be successfully learnt
from a fairly small (in deep learning context) training set and still
generalise well to out-of-sample identities (completely new dolphin
individuals), with top-1 and top-5 test set (37 individuals) accuracy of
$90.5\pm2$ and $93.6\pm1$ percent. In the presence of 1200 distractors, top-1
accuracy dropped by $12\%$; however, top-5 accuracy saw only a $2.8\%$ drop
Enabling FDD Massive MIMO through Deep Learning-based Channel Prediction
Authors: Maximilian Arnold, Sebastian Dörner, Sebastian Cammerer, Sarah Yan, Jakob Hoydis, Stephan ten Brink
Date: 2019-01-08 00:00:00
Tasks: Deep Learning

A major obstacle for widespread deployment of frequency division duplex
(FDD)-based Massive multiple-input multiple-output (MIMO) communications is the
large signaling overhead for reporting full downlink (DL) channel state
information (CSI) back to the basestation (BS), in order to enable closed-loop
precoding. We completely remove this overhead by a deep-learning based channel
extrapolation (or "prediction") approach and demonstrate that a neural network
(NN) at the BS can infer the DL CSI centered around a frequency $f_\text{DL}$
by solely observing uplink (UL) CSI on a different, yet adjacent frequency band
around $f_\text{UL}$; no more pilot/reporting overhead is needed than with a
genuine time division duplex (TDD)-based system. The rationale is that
scatterers and the large-scale propagation environment are sufficiently similar
to allow a NN to learn about the physical connections and constraints between
two neighboring frequency bands, and thus provide a well-operating system even
when classic extrapolation methods, like the Wiener filter (used as a baseline
for comparison throughout) fails. We study its performance for various
state-of-the-art Massive MIMO channel models, and, even more so, evaluate the
scheme using actual Massive MIMO channel measurements, rendering it to be
practically feasible at negligible loss in spectral efficiency when compared to
a genuine TDD-based system.
Weightless Neural Network with Transfer Learning to Detect Distress in Asphalt
Authors: Suayder Milhomem, Tiago da Silva Almeida, Warley Gramacho da Silva, Edeilson Milhomem da Silva, Rafael Lima de Carvalho
Date: 2019-01-03 00:00:00
Tasks: Transfer Learning

The present paper shows a solution to the problem of automatic distress
detection, more precisely the detection of holes in paved roads. To do so, the
proposed solution uses a weightless neural network known as Wisard to decide
whether an image of a road has any kind of cracks. In addition, the proposed
architecture also shows how the use of transfer learning was able to improve
the overall accuracy of the decision system. As a verification step of the
research, an experiment was carried out using images from the streets at the
Federal University of Tocantins, Brazil. The architecture of the developed
solution presents a result of 85.71% accuracy in the dataset, proving to be
superior to approaches of the state-of-the-art.
An Optical Frontend for a Convolutional Neural Network
Authors: Shane Colburn, Yi Chu, Eli Shlizerman, Arka Majumdar
Date: 2018-12-23 00:00:00
Tasks: Benchmarking

The parallelism of optics and the miniaturization of optical components using
nanophotonic structures, such as metasurfaces present a compelling alternative
to electronic implementations of convolutional neural networks. The lack of a
low-power optical nonlinearity, however, requires slow and energy-inefficient
conversions between the electronic and optical domains. Here, we design an
architecture which utilizes a single electrical to optical conversion by
designing a free-space optical frontend unit that implements the linear
operations of the first layer with the subsequent layers realized
electronically. Speed and power analysis of the architecture indicates that the
hybrid photonic-electronic architecture outperforms sole electronic
architecture for large image sizes and kernels. Benchmarking of the
photonic-electronic architecture on a modified version of AlexNet achieves a
classification accuracy of 87% on images from the Kaggle Cats and Dogs
challenge database.
A Fast Randomized Method to Find Homotopy Classes for Socially-Aware Navigation
Authors: Luigi Palmieri, Andrey Rudenko, Kai O. Arras
Date: 2015-10-28 00:00:00

We introduce and show preliminary results of a fast randomized method that
finds a set of K paths lying in distinct homotopy classes. We frame the path
planning task as a graph search problem, where the navigation graph is based on
a Voronoi diagram. The search is biased by a cost function derived from the
social force model that is used to generate and select the paths. We compare
our method to Yen's algorithm, and empirically show that our approach is faster
to find a subset of homotopy classes. Furthermore our approach computes a set
of more diverse paths with respect to the baseline while obtaining a negligible
loss in path quality.
v-SVR Polynomial Kernel for Predicting the Defect Density in New Software Projects
Authors: Cuauhtemoc Lopez-Martin, Mohammad Azzeh, Ali Bou Nassif, Shadi Banitaan
Date: 2018-12-15 00:00:00
Tasks: Benchmarking, regression

An important product measure to determine the effectiveness of software
processes is the defect density (DD). In this study, we propose the application
of support vector regression (SVR) to predict the DD of new software projects
obtained from the International Software Benchmarking Standards Group (ISBSG)
Release 2018 data set. Two types of SVR (e-SVR and v-SVR) were applied to train
and test these projects. Each SVR used four types of kernels. The prediction
accuracy of each SVR was compared to that of a statistical regression (i.e., a
simple linear regression, SLR). Statistical significance test showed that v-SVR
with polynomial kernel was better than that of SLR when new software projects
were developed on mainframes and coded in programming languages of third
generation
UPSNet: A Unified Panoptic Segmentation Network
Authors: Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, Raquel Urtasun
Date: 2019-01-12 00:00:00
Proceeding: CVPR 2019 6
Tasks: Instance Segmentation, Panoptic Segmentation, Segmentation, Semantic Segmentation

In this paper, we propose a unified panoptic segmentation network (UPSNet)
for tackling the newly proposed panoptic segmentation task. On top of a single
backbone residual network, we first design a deformable convolution based
semantic segmentation head and a Mask R-CNN style instance segmentation head
which solve these two subtasks simultaneously. More importantly, we introduce a
parameter-free panoptic head which solves the panoptic segmentation via
pixel-wise classification. It first leverages the logits from the previous two
heads and then innovatively expands the representation for enabling prediction
of an extra unknown class which helps better resolve the conflicts between
semantic and instance segmentation. Additionally, it handles the challenge
caused by the varying number of instances and permits back propagation to the
bottom modules in an end-to-end manner. Extensive experimental results on
Cityscapes, COCO and our internal dataset demonstrate that our UPSNet achieves
state-of-the-art performance with much faster inference. Code has been made
available at: https://github.com/uber-research/UPSNet
LiFF: Light Field Features in Scale and Depth
Authors: Donald G. Dansereau, Bernd Girod, Gordon Wetzstein
Date: 2019-01-13 00:00:00
Proceeding: CVPR 2019 6

Feature detectors and descriptors are key low-level vision tools that many
higher-level tasks build on. Unfortunately these fail in the presence of
challenging light transport effects including partial occlusion, low contrast,
and reflective or refractive surfaces. Building on spatio-angular imaging
modalities offered by emerging light field cameras, we introduce a new and
computationally efficient 4D light field feature detector and descriptor: LiFF.
LiFF is scale invariant and utilizes the full 4D light field to detect features
that are robust to changes in perspective. This is particularly useful for
structure from motion (SfM) and other tasks that match features across
viewpoints of a scene. We demonstrate significantly improved 3D reconstructions
via SfM when using LiFF instead of the leading 2D or 4D features, and show that
LiFF runs an order of magnitude faster than the leading 4D approach. Finally,
LiFF inherently estimates depth for each feature, opening a path for future
research in light field-based SfM.
Faster and better: a machine learning approach to corner detection
Authors: Edward Rosten, Reid Porter, Tom Drummond
Date: 2008-10-14 00:00:00
Tasks: BIG-bench Machine Learning

The repeatability and efficiency of a corner detector determines how likely
it is to be useful in a real-world application. The repeatability is importand
because the same scene viewed from different positions should yield features
which correspond to the same real-world 3D locations [Schmid et al 2000]. The
efficiency is important because this determines whether the detector combined
with further processing can operate at frame rate.
  Three advances are described in this paper. First, we present a new heuristic
for feature detection, and using machine learning we derive a feature detector
from this which can fully process live PAL video using less than 5% of the
available processing time. By comparison, most other detectors cannot even
operate at frame rate (Harris detector 115%, SIFT 195%). Second, we generalize
the detector, allowing it to be optimized for repeatability, with little loss
of efficiency. Third, we carry out a rigorous comparison of corner detectors
based on the above repeatability criterion applied to 3D scenes. We show that
despite being principally constructed for speed, on these stringent tests, our
heuristic detector significantly outperforms existing feature detectors.
Finally, the comparison demonstrates that using machine learning produces
significant improvements in repeatability, yielding a detector that is both
very fast and very high quality.
Passage Re-ranking with BERT
Authors: Rodrigo Nogueira, Kyunghyun Cho
Date: 2019-01-13 00:00:00
Tasks: Language Modeling, Passage Re-Ranking, Passage Retrieval, Re-Ranking, Retrieval

Recently, neural models pretrained on a language modeling task, such as ELMo (Peters et al., 2017), OpenAI GPT (Radford et al., 2018), and BERT (Devlin et al., 2018), have achieved impressive results on various natural language processing tasks such as question-answering and natural language inference. In this paper, we describe a simple re-implementation of BERT for query-based passage re-ranking. Our system is the state of the art on the TREC-CAR dataset and the top entry in the leaderboard of the MS MARCO passage retrieval task, outperforming the previous state of the art by 27% (relative) in MRR@10. The code to reproduce our results is available at https://github.com/nyu-dl/dl4marco-bert
A Modern Retrospective on Probabilistic Numerics
Authors: C. J. Oates, T. J. Sullivan
Date: 2019-01-14 00:00:00

This article attempts to place the emergence of probabilistic numerics as a mathematical-statistical research field within its historical context and to explore how its gradual development can be related both to applications and to a modern formal treatment. We highlight in particular the parallel contributions of Sul'din and Larkin in the 1960s and how their pioneering early ideas have reached a degree of maturity in the intervening period, mediated by paradigms such as average-case analysis and information-based complexity. We provide a subjective assessment of the state of research in probabilistic numerics and highlight some difficulties to be addressed by future works.
Posterior inference unchained with EL_2O
Authors: Uros Seljak, Byeonghee Yu
Date: 2019-01-14 00:00:00
Tasks: Clustering

Statistical inference of analytically non-tractable posteriors is a difficult problem because of marginalization of correlated variables and stochastic methods such as MCMC and VI are commonly used. We argue that stochastic KL divergence minimization used by MCMC and VI is noisy, and we propose instead EL_2O, expectation optimization of L_2 distance squared between the approximate log posterior q and the un-normalized log posterior of p. When sampling from q the solutions agree with stochastic KL divergence minimization based VI in the large sample limit, however EL_2O method is free of sampling noise, has better optimization properties, and requires only as many sample evaluations as the number of parameters we are optimizing if q covers p. As a consequence, increasing the expressivity of q improves both the quality of results and the convergence rate, allowing EL_2O to approach exact inference. Use of automatic differentiation methods enables us to develop Hessian, gradient and gradient free versions of the method, which can determine M(M+2)/2+1, M+1 and 1 parameter(s) of q with a single sample, respectively. EL_2O provides a reliable estimate of the quality of the approximating posterior, and converges rapidly on full rank gaussian approximation for q and extensions beyond it, such as nonlinear transformations and gaussian mixtures. These can handle general posteriors, while still allowing fast analytic marginalizations. We test it on several examples, including a realistic 13 dimensional galaxy clustering analysis, showing that it is several orders of magnitude faster than MCMC, while giving smooth and accurate non-gaussian posteriors, often requiring a few to a few dozen of iterations only.
Bayesian Learning of Neural Network Architectures
Authors: Georgi Dikov, Patrick van der Smagt, Justin Bayer
Date: 2019-01-14 00:00:00
Tasks: Neural Architecture Search

In this paper we propose a Bayesian method for estimating architectural
parameters of neural networks, namely layer size and network depth. We do this
by learning concrete distributions over these parameters. Our results show that
regular networks with a learnt structure can generalise better on small
datasets, while fully stochastic networks can be more robust to parameter
initialisation. The proposed method relies on standard neural variational
learning and, unlike randomised architecture search, does not require a
retraining of the model, thus keeping the computational overhead at minimum.
Data Augmentation with Manifold Exploring Geometric Transformations for Increased Performance and Robustness
Authors: Magdalini Paschali, Walter Simson, Abhijit Guha Roy, Muhammad Ferjad Naeem, Rüdiger Göbl, Christian Wachinger, Nassir Navab
Date: 2019-01-14 00:00:00
Tasks: Data Augmentation, General Classification, Lesion Classification, Skin Lesion Classification

In this paper we propose a novel augmentation technique that improves not
only the performance of deep neural networks on clean test data, but also
significantly increases their robustness to random transformations, both affine
and projective. Inspired by ManiFool, the augmentation is performed by a
line-search manifold-exploration method that learns affine geometric
transformations that lead to the misclassification on an image, while ensuring
that it remains on the same manifold as the training data.
  This augmentation method populates any training dataset with images that lie
on the border of the manifolds between two-classes and maximizes the variance
the network is exposed to during training. Our method was thoroughly evaluated
on the challenging tasks of fine-grained skin lesion classification from
limited data, and breast tumor classification of mammograms. Compared with
traditional augmentation methods, and with images synthesized by Generative
Adversarial Networks our method not only achieves state-of-the-art performance
but also significantly improves the network's robustness.
River Ice Segmentation with Deep Learning
Authors: Abhineet Singh, Hayden Kalke, Mark Loewen, Nilanjan Ray
Date: 2019-01-14 00:00:00
Tasks: Deep Learning, Segmentation, Semantic Segmentation

This paper deals with the problem of computing surface ice concentration for two different types of ice from digital images of river surface. It presents the results of attempting to solve this problem using several state of the art semantic segmentation methods based on deep convolutional neural networks (CNNs). This task presents two main challenges - very limited availability of labeled training data and presence of noisy labels due to the great difficulty of visually distinguishing between the two types of ice, even for human experts. The results are used to analyze the extent to which some of the best deep learning methods currently in existence can handle these challenges. The code and data used in the experiments are made publicly available to facilitate further work in this domain.
Quadratization in discrete optimization and quantum mechanics
Authors: Nike Dattani
Date: 2019-01-14 00:00:00
Tasks: Form

A book about turning high-degree optimization problems into quadratic optimization problems that maintain the same global minimum (ground state). This book explores quadratizations for pseudo-Boolean optimization, perturbative gadgets used in QMA completeness theorems, and also non-perturbative k-local to 2-local transformations used for quantum mechanics, quantum annealing and universal adiabatic quantum computing. The book contains ~70 different Hamiltonian transformations, each of them on a separate page, where the cost (in number of auxiliary binary variables or auxiliary qubits, or number of sub-modular terms, or in graph connectivity, etc.), pros, cons, examples, and references are given. One can therefore look up a quadratization appropriate for the specific term(s) that need to be quadratized, much like using an integral table to look up the integral that needs to be done. This book is therefore useful for writing compilers to transform general optimization problems, into a form that quantum annealing or universal adiabatic quantum computing hardware requires; or for transforming quantum chemistry problems written in the Jordan-Wigner or Bravyi-Kitaev form, into a form where all multi-qubit interactions become 2-qubit pairwise interactions, without changing the desired ground state. Applications cited include computer vision problems (e.g. image de-noising, un-blurring, etc.), number theory (e.g. integer factoring), graph theory (e.g. Ramsey number determination), and quantum chemistry. The book is open source, and anyone can make modifications here: https://github.com/HPQC-LABS/Book_About_Quadratization.
Unsupervised Visual Feature Learning with Spike-timing-dependent Plasticity: How Far are we from Traditional Feature Learning Approaches?
Authors: Pierre Falez, Pierre Tirilly, Ioan Marius Bilasco, Philippe Devienne, Pierre Boulet
Date: 2019-01-14 00:00:00
Tasks: General Classification, image-classification, Image Classification

Spiking neural networks (SNNs) equipped with latency coding and spike-timing
dependent plasticity rules offer an alternative to solve the data and energy
bottlenecks of standard computer vision approaches: they can learn visual
features without supervision and can be implemented by ultra-low power hardware
architectures. However, their performance in image classification has never
been evaluated on recent image datasets. In this paper, we compare SNNs to
auto-encoders on three visual recognition datasets, and extend the use of SNNs
to color images. The analysis of the results helps us identify some bottlenecks
of SNNs: the limits of on-center/off-center coding, especially for color
images, and the ineffectiveness of current inhibition mechanisms. These issues
should be addressed to build effective SNNs for image recognition.
Supervised Learning for Multi-Block Incomplete Data
Authors: Hadrien Lorenzo, Jérôme Saracco, Rodolphe Thiébaut
Date: 2019-01-14 00:00:00
Tasks: Imputation, Variable Selection

In the supervised high dimensional settings with a large number of variables
and a low number of individuals, one objective is to select the relevant
variables and thus to reduce the dimension. That subspace selection is often
managed with supervised tools. However, some data can be missing, compromising
the validity of the sub-space selection. We propose a Partial Least Square
(PLS) based method, called Multi-block Data-Driven sparse PLS mdd-sPLS,
allowing jointly variable selection and subspace estimation while training and
testing missing data imputation through a new algorithm called Koh-Lanta. This
method was challenged through simulations against existing methods such as mean
imputation, nipals, softImpute and imputeMFA. In the context of supervised
analysis of high dimensional data, the proposed method shows the lowest
prediction error of the response variables. So far this is the only method
combining data imputation and response variable prediction. The superiority of
the supervised multi-block mdd-sPLS method increases with the intra-block and
inter-block correlations. The application to a real data-set from a rVSV-ZEBOV
Ebola vaccine trial revealed interesting and biologically relevant results. The
method is implemented in a R-package available on the CRAN and a Python-package
available on pypi.