pwc-archive/papers-with-abstracts huggingface dataset:
Deep Learning based Recommender System: A Survey and New Perspectives
Authors: Shuai Zhang, Lina Yao, Aixin Sun, Yi Tay
Date: 2017-07-24 00:00:00
Tasks: Deep Learning, Information Retrieval, Recommendation Systems, Retrieval

With the ever-growing volume of online information, recommender systems have been an effective strategy to overcome such information overload. The utility of recommender systems cannot be overstated, given its widespread adoption in many web applications, along with its potential impact to ameliorate many problems related to over-choice. In recent years, deep learning has garnered considerable interest in many research fields such as computer vision and natural language processing, owing not only to stellar performance but also the attractive property of learning feature representations from scratch. The influence of deep learning is also pervasive, recently demonstrating its effectiveness when applied to information retrieval and recommender systems research. Evidently, the field of deep learning in recommender system is flourishing. This article aims to provide a comprehensive review of recent research efforts on deep learning based recommender systems. More concretely, we provide and devise a taxonomy of deep learning based recommendation models, along with providing a comprehensive summary of the state-of-the-art. Finally, we expand on current trends and provide new perspectives pertaining to this new exciting development of the field.
Point-of-Interest Recommendation: Exploiting Self-Attentive Autoencoders with Neighbor-Aware Influence
Authors: Chen Ma, Yingxue Zhang, Qinglong Wang, Xue Liu
Date: 2018-09-27 00:00:00
Tasks: Decoder, Recommendation Systems

The rapid growth of Location-based Social Networks (LBSNs) provides a great
opportunity to satisfy the strong demand for personalized Point-of-Interest
(POI) recommendation services. However, with the tremendous increase of users
and POIs, POI recommender systems still face several challenging problems: (1)
the hardness of modeling non-linear user-POI interactions from implicit
feedback; (2) the difficulty of incorporating context information such as POIs'
geographical coordinates. To cope with these challenges, we propose a novel
autoencoder-based model to learn the non-linear user-POI relations, namely
\textit{SAE-NAD}, which consists of a self-attentive encoder (SAE) and a
neighbor-aware decoder (NAD). In particular, unlike previous works equally
treat users' checked-in POIs, our self-attentive encoder adaptively
differentiates the user preference degrees in multiple aspects, by adopting a
multi-dimensional attention mechanism. To incorporate the geographical context
information, we propose a neighbor-aware decoder to make users' reachability
higher on the similar and nearby neighbors of checked-in POIs, which is
achieved by the inner product of POI embeddings together with the radial basis
function (RBF) kernel. To evaluate the proposed model, we conduct extensive
experiments on three real-world datasets with many state-of-the-art baseline
methods and evaluation metrics. The experimental results demonstrate the
effectiveness of our model.
Memory-like Map Decay for Autonomous Vehicles based on Grid Maps
Authors: Thomas Teixeira, Filipe Mutz, Karin Satie Komati, Lucas Veronese, Vinicius B. Cardoso, Claudine Badue, Thiago Oliveira-Santos, Alberto F. de Souza
Date: 2018-10-04 00:00:00
Tasks: Autonomous Vehicles

In this work, we present a novel strategy for correcting imperfections in occupancy grid maps called map decay. The objective of map decay is to correct invalid occupancy probabilities of map cells that are unobservable by sensors. The strategy was inspired by an analogy between the memory architecture believed to exist in the human brain and the maps maintained by an autonomous vehicle. It consists in merging sensory information obtained during runtime (online) with a priori data from a high-precision map constructed offline. In map decay, cells observed by sensors are updated using traditional occupancy grid mapping techniques and unobserved cells are adjusted so that their occupancy probabilities tend to the values found in the offline map. This strategy is grounded in the idea that the most precise information available about an unobservable cell is the value found in the high-precision offline map. Map decay was successfully tested and is still in use in the IARA autonomous vehicle from Universidade Federal do Esp\'irito Santo.
Multi-view X-ray R-CNN
Authors: Jan-Martin O. Steitz, Faraz Saeedan, Stefan Roth
Date: 2018-10-04 00:00:00
Tasks: General Classification, multi-view detection, object-detection, Object Detection

Motivated by the detection of prohibited objects in carry-on luggage as a
part of avionic security screening, we develop a CNN-based object detection
approach for multi-view X-ray image data. Our contributions are two-fold.
First, we introduce a novel multi-view pooling layer to perform a 3D
aggregation of 2D CNN-features extracted from each view. To that end, our
pooling layer exploits the known geometry of the imaging system to ensure
geometric consistency of the feature aggregation. Second, we introduce an
end-to-end trainable multi-view detection pipeline based on Faster R-CNN, which
derives the region proposals and performs the final classification in 3D using
these aggregated multi-view features. Our approach shows significant accuracy
gains compared to single-view detection while even being more efficient than
performing single-view detection in each view.
SNIP: Single-shot Network Pruning based on Connection Sensitivity
Authors: Namhoon Lee, Thalaiyasingam Ajanthan, Philip H. S. Torr
Date: 2018-10-04 00:00:00
Proceeding: ICLR 2019 5
Tasks: Image Classification, Network Pruning, Object Detection, Sensitivity

Pruning large neural networks while maintaining their performance is often
desirable due to the reduced space and time complexity. In existing methods,
pruning is done within an iterative optimization procedure with either
heuristically designed pruning schedules or additional hyperparameters,
undermining their utility. In this work, we present a new approach that prunes
a given network once at initialization prior to training. To achieve this, we
introduce a saliency criterion based on connection sensitivity that identifies
structurally important connections in the network for the given task. This
eliminates the need for both pretraining and the complex pruning schedule while
making it robust to architecture variations. After pruning, the sparse network
is trained in the standard way. Our method obtains extremely sparse networks
with virtually the same accuracy as the reference network on the MNIST,
CIFAR-10, and Tiny-ImageNet classification tasks and is broadly applicable to
various architectures including convolutional, residual and recurrent networks.
Unlike existing methods, our approach enables us to demonstrate that the
retained connections are indeed relevant to the given task.
Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding
Authors: Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, Joshua B. Tenenbaum
Date: 2018-10-04 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Question Answering, Representation Learning, Visual Question Answering, Visual Question Answering (VQA)

We marry two powerful ideas: deep representation learning for visual
recognition and language understanding, and symbolic program execution for
reasoning. Our neural-symbolic visual question answering (NS-VQA) system first
recovers a structural scene representation from the image and a program trace
from the question. It then executes the program on the scene representation to
obtain an answer. Incorporating symbolic structure as prior knowledge offers
three unique advantages. First, executing programs on a symbolic space is more
robust to long program traces; our model can solve complex reasoning tasks
better, achieving an accuracy of 99.8% on the CLEVR dataset. Second, the model
is more data- and memory-efficient: it performs well after learning on a small
number of training data; it can also encode an image into a compact
representation, requiring less storage than existing methods for offline
question answering. Third, symbolic program execution offers full transparency
to the reasoning process; we are thus able to interpret and diagnose each
execution step.
Unsupervised Learning via Meta-Learning
Authors: Kyle Hsu, Sergey Levine, Chelsea Finn
Date: 2018-10-04 00:00:00
Proceeding: ICLR 2019 5
Tasks: Clustering, Disentanglement, Meta-Learning, Unsupervised Few-Shot Image Classification, Unsupervised Few-Shot Learning

A central goal of unsupervised learning is to acquire representations from
unlabeled data or experience that can be used for more effective learning of
downstream tasks from modest amounts of labeled data. Many prior unsupervised
learning works aim to do so by developing proxy objectives based on
reconstruction, disentanglement, prediction, and other metrics. Instead, we
develop an unsupervised meta-learning method that explicitly optimizes for the
ability to learn a variety of tasks from small amounts of data. To do so, we
construct tasks from unlabeled data in an automatic way and run meta-learning
over the constructed tasks. Surprisingly, we find that, when integrated with
meta-learning, relatively simple task construction mechanisms, such as
clustering embeddings, lead to good performance on a variety of downstream,
human-specified tasks. Our experiments across four image datasets indicate that
our unsupervised meta-learning approach acquires a learning algorithm without
any labeled data that is applicable to a wide range of downstream
classification tasks, improving upon the embedding learned by four prior
unsupervised learning methods.
A Practical Approach to Sizing Neural Networks
Authors: Gerald Friedland, Alfredo Metere, Mario Krell
Date: 2018-10-04 00:00:00
Tasks: Memorization

Memorization is worst-case generalization. Based on MacKay's information
theoretic model of supervised machine learning, this article discusses how to
practically estimate the maximum size of a neural network given a training data
set. First, we present four easily applicable rules to analytically determine
the capacity of neural network architectures. This allows the comparison of the
efficiency of different network architectures independently of a task. Second,
we introduce and experimentally validate a heuristic method to estimate the
neural network capacity requirement for a given dataset and labeling. This
allows an estimate of the required size of a neural network for a given
problem. We conclude the article with a discussion on the consequences of
sizing the network wrongly, which includes both increased computation effort
for training as well as reduced generalization capability.
Machine learning electron correlation in a disordered medium
Authors: Jianhua Ma, Puhan Zhang, Yao-Hua Tan, Avik W. Ghosh, Gia-Wei Chern
Date: 2018-10-04 00:00:00
Tasks: BIG-bench Machine Learning

Learning from data has led to a paradigm shift in computational materials
science. In particular, it has been shown that neural networks can learn the
potential energy surface and interatomic forces through examples, thus
bypassing the computationally expensive density functional theory calculations.
Combining many-body techniques with a deep learning approach, we demonstrate
that a fully-connected neural network is able to learn the complex collective
behavior of electrons in strongly correlated systems. Specifically, we consider
the Anderson-Hubbard (AH) model, which is a canonical system for studying the
interplay between electron correlation and strong localization. The ground
states of the AH model on a square lattice are obtained using the real-space
Gutzwiller method. The obtained solutions are used to train a multi-task
multi-layer neural network, which subsequently can accurately predict
quantities such as the local probability of double occupation and the
quasiparticle weight, given the disorder potential in the neighborhood as the
input.
Optimal Learning with Anisotropic Gaussian SVMs
Authors: Hanyuan Hang, Ingo Steinwart
Date: 2018-10-04 00:00:00
Tasks: regression

This paper investigates the nonparametric regression problem using SVMs with
anisotropic Gaussian RBF kernels. Under the assumption that the target
functions are resided in certain anisotropic Besov spaces, we establish the
almost optimal learning rates, more precisely, optimal up to some logarithmic
factor, presented by the effective smoothness. By taking the effective
smoothness into consideration, our almost optimal learning rates are faster
than those obtained with the underlying RKHSs being certain anisotropic Sobolev
spaces. Moreover, if the target function depends only on fewer dimensions,
faster learning rates can be further achieved.
Computer vision-based framework for extracting geological lineaments from optical remote sensing data
Authors: Ehsan Farahbakhsh, Rohitash Chandra, Hugo K. H. Olierook, Richard Scalzo, Chris Clark, Steven M. Reddy, R. Dietmar Muller
Date: 2018-10-04 00:00:00
Tasks: Dimensionality Reduction, Edge Detection

The extraction of geological lineaments from digital satellite data is a
fundamental application in remote sensing. The location of geological
lineaments such as faults and dykes are of interest for a range of
applications, particularly because of their association with hydrothermal
mineralization. Although a wide range of applications have utilized computer
vision techniques, a standard workflow for application of these techniques to
mineral exploration is lacking. We present a framework for extracting
geological lineaments using computer vision techniques which is a combination
of edge detection and line extraction algorithms for extracting geological
lineaments using optical remote sensing data. It features ancillary computer
vision techniques for reducing data dimensionality, removing noise and
enhancing the expression of lineaments. We test the proposed framework on
Landsat 8 data of a mineral-rich portion of the Gascoyne Province in Western
Australia using different dimension reduction techniques and convolutional
filters. To validate the results, the extracted lineaments are compared to our
manual photointerpretation and geologically mapped structures by the Geological
Survey of Western Australia (GSWA). The results show that the best correlation
between our extracted geological lineaments and the GSWA geological lineament
map is achieved by applying a minimum noise fraction transformation and a
Laplacian filter. Application of a directional filter instead shows a stronger
correlation with the output of our manual photointerpretation and known sites
of hydrothermal mineralization. Hence, our framework using either filter can be
used for mineral prospectivity mapping in other regions where faults are
exposed and observable in optical remote sensing data.
Learning Compressed Transforms with Low Displacement Rank
Authors: Anna T. Thomas, Albert Gu, Tri Dao, Atri Rudra, Christopher Ré
Date: 2018-10-04 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: image-classification, Image Classification, Language Modeling, Language Modelling

The low displacement rank (LDR) framework for structured matrices represents
a matrix through two displacement operators and a low-rank residual. Existing
use of LDR matrices in deep learning has applied fixed displacement operators
encoding forms of shift invariance akin to convolutions. We introduce a class
of LDR matrices with more general displacement operators, and explicitly learn
over both the operators and the low-rank component. This class generalizes
several previous constructions while preserving compression and efficient
computation. We prove bounds on the VC dimension of multi-layer neural networks
with structured weight matrices and show empirically that our compact
parameterization can reduce the sample complexity of learning. When replacing
weight layers in fully-connected, convolutional, and recurrent neural networks
for image classification and language modeling tasks, our new classes exceed
the accuracy of existing compression approaches, and on some tasks also
outperform general unstructured layers while using more than 20x fewer
parameters.
Markov Properties of Discrete Determinantal Point Processes
Authors: Kayvan Sadeghi, Alessandro Rinaldo
Date: 2018-10-04 00:00:00
Tasks: Point Processes

Determinantal point processes (DPPs) are probabilistic models for repulsion.
When used to represent the occurrence of random subsets of a finite base set,
DPPs allow to model global negative associations in a mathematically elegant
and direct way. Discrete DPPs have become popular and computationally tractable
models for solving several machine learning tasks that require the selection of
diverse objects, and have been successfully applied in numerous real-life
problems. Despite their popularity, the statistical properties of such models
have not been adequately explored. In this note, we derive the Markov
properties of discrete DPPs and show how they can be expressed using graphical
models.
Progressive Feature Fusion Network for Realistic Image Dehazing
Authors: Kangfu Mei, Aiwen Jiang, Juncheng Li, Mingwen Wang
Date: 2018-10-04 00:00:00
Tasks: 4k, Decoder, GPU, Image Dehazing, Single Image Dehazing

Single image dehazing is a challenging ill-posed restoration problem. Various
prior-based and learning-based methods have been proposed. Most of them follow
a classic atmospheric scattering model which is an elegant simplified physical
model based on the assumption of single-scattering and homogeneous atmospheric
medium. The formulation of haze in realistic environment is more complicated.
In this paper, we propose to take its essential mechanism as "black box", and
focus on learning an input-adaptive trainable end-to-end dehazing model. An
U-Net like encoder-decoder deep network via progressive feature fusions has
been proposed to directly learn highly nonlinear transformation function from
observed hazy image to haze-free ground-truth. The proposed network is
evaluated on two public image dehazing benchmarks. The experiments demonstrate
that it can achieve superior performance when compared with popular
state-of-the-art methods. With efficient GPU memory usage, it can
satisfactorily recover ultra high definition hazed image up to 4K resolution,
which is unaffordable by many deep learning based dehazing algorithms.
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks
Authors: Sanjeev Arora, Nadav Cohen, Noah Golowich, Wei Hu
Date: 2018-10-04 00:00:00
Proceeding: ICLR 2019 5

We analyze speed of convergence to global optimum for gradient descent training a deep linear neural network (parameterized as $x \mapsto W_N W_{N-1} \cdots W_1 x$) by minimizing the $\ell_2$ loss over whitened data. Convergence at a linear rate is guaranteed when the following hold: (i) dimensions of hidden layers are at least the minimum of the input and output dimensions; (ii) weight matrices at initialization are approximately balanced; and (iii) the initial loss is smaller than the loss of any rank-deficient solution. The assumptions on initialization (conditions (ii) and (iii)) are necessary, in the sense that violating any one of them may lead to convergence failure. Moreover, in the important case of output dimension 1, i.e. scalar regression, they are met, and thus convergence to global optimum holds, with constant probability under a random initialization scheme. Our results significantly extend previous analyses, e.g., of deep linear residual networks (Bartlett et al., 2018).
Direct Prediction of Cardiovascular Mortality from Low-dose Chest CT using Deep Learning
Authors: Sanne G. M. van Velzen, Majd Zreik, Nikolas Lessmann, Max A. Viergever, Pim A. de Jong, Helena M. Verkooijen, Ivana Išgum
Date: 2018-10-04 00:00:00

Cardiovascular disease (CVD) is a leading cause of death in the lung cancer
screening population. Chest CT scans made in lung cancer screening are suitable
for identification of participants at risk of CVD. Existing methods analyzing
CT images from lung cancer screening for prediction of CVD events or mortality
use engineered features extracted from the images combined with patient
information. In this work we propose a method that automatically predicts
5-year cardiovascular mortality directly from chest CT scans without the need
for hand-crafting image features. A set of 1,583 participants of the National
Lung Screening Trial was included (1,188 survivors, 395 non-survivors).
Low-dose chest CT images acquired at baseline were analyzed and the follow-up
time was 5 years. To limit the analysis to the heart region, the heart was
first localized by our previously developed algorithm for organ localization
exploiting convolutional neural networks. Thereafter, a convolutional
autoencoder was used to encode the identified heart region. Finally, based on
the extracted encodings subjects were classified into survivors or
non-survivors using a support vector machine classifier. The performance of the
method was assessed in eight cross-validation experiments with 1,433 images
used for training, 50 for validation and 100 for testing. The method achieved a
performance with an area under the ROC curve of 0.72. The results demonstrate
that prediction of cardiovascular mortality directly from low-dose screening
chest CT scans, without hand-crafted features, is feasible, allowing
identification of subjects at risk of fatal CVD events.
Episodic Curiosity through Reachability
Authors: Nikolay Savinov, Anton Raichuk, Raphaël Marinier, Damien Vincent, Marc Pollefeys, Timothy Lillicrap, Sylvain Gelly
Date: 2018-10-04 00:00:00
Proceeding: ICLR 2019 5
Tasks: MuJoCo, Reinforcement Learning

Rewards are sparse in the real world and most of today's reinforcement learning algorithms struggle with such sparsity. One solution to this problem is to allow the agent to create rewards for itself - thus making rewards dense and more suitable for learning. In particular, inspired by curious behaviour in animals, observing something novel could be rewarded with a bonus. Such bonus is summed up with the real task reward - making it possible for RL algorithms to learn from the combined reward. We propose a new curiosity method which uses episodic memory to form the novelty bonus. To determine the bonus, the current observation is compared with the observations in memory. Crucially, the comparison is done based on how many environment steps it takes to reach the current observation from those in memory - which incorporates rich information about environment dynamics. This allows us to overcome the known "couch-potato" issues of prior work - when the agent finds a way to instantly gratify itself by exploiting actions which lead to hardly predictable consequences. We test our approach in visually rich 3D environments in ViZDoom, DMLab and MuJoCo. In navigational tasks from ViZDoom and DMLab, our agent outperforms the state-of-the-art curiosity method ICM. In MuJoCo, an ant equipped with our curiosity module learns locomotion out of the first-person-view curiosity only.