pwc-archive/papers-with-abstracts huggingface dataset:
Motif-based Convolutional Neural Network on Graphs
Authors: Aravind Sankar, Xinyang Zhang, Kevin Chen-Chuan Chang
Date: 2017-11-15 00:00:00
Tasks: General Classification, Node Classification, Translation

This paper introduces a generalization of Convolutional Neural Networks (CNNs) to graphs with irregular linkage structures, especially heterogeneous graphs with typed nodes and schemas. We propose a novel spatial convolution operation to model the key properties of local connectivity and translation invariance, using high-order connection patterns or motifs. We develop a novel deep architecture Motif-CNN that employs an attention model to combine the features extracted from multiple patterns, thus effectively capturing high-order structural and feature information. Our experiments on semi-supervised node classification on real-world social networks and multiple representative heterogeneous graph datasets indicate significant gains of 6-21% over existing graph CNNs and other state-of-the-art techniques.
Sparse Regularization in Marketing and Economics
Authors: Guanhao Feng, Nicholas Polson, Yuexi Wang, Jianeng Xu
Date: 2017-09-01 00:00:00
Tasks: Demand Forecasting, Marketing

Sparse alpha-norm regularization has many data-rich applications in Marketing
and Economics. Alpha-norm, in contrast to lasso and ridge regularization, jumps
to a sparse solution. This feature is attractive for ultra high-dimensional
problems that occur in demand estimation and forecasting. The alpha-norm
objective is nonconvex and requires coordinate descent and proximal operators
to find the sparse solution. We study a typical marketing demand forecasting
problem, grocery store sales for salty snacks, that has many dummy variables as
controls. The key predictors of demand include price, equivalized volume,
promotion, flavor, scent, and brand effects. By comparing with many commonly
used machine learning methods, alpha-norm regularization achieves its goal of
providing accurate out-of-sample estimates for the promotion lift effects.
Finally, we conclude with directions for future research.
Linear Convergence of the Primal-Dual Gradient Method for Convex-Concave Saddle Point Problems without Strong Convexity
Authors: Simon S. Du, Wei Hu
Date: 2018-02-05 00:00:00

We consider the convex-concave saddle point problem $\min_{x}\max_{y}
f(x)+y^\top A x-g(y)$ where $f$ is smooth and convex and $g$ is smooth and
strongly convex. We prove that if the coupling matrix $A$ has full column rank,
the vanilla primal-dual gradient method can achieve linear convergence even if
$f$ is not strongly convex. Our result generalizes previous work which either
requires $f$ and $g$ to be quadratic functions or requires proximal mappings
for both $f$ and $g$. We adopt a novel analysis technique that in each
iteration uses a "ghost" update as a reference, and show that the iterates in
the primal-dual gradient method converge to this "ghost" sequence. Using the
same technique we further give an analysis for the primal-dual stochastic
variance reduced gradient (SVRG) method for convex-concave saddle point
problems with a finite-sum structure.
Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds
Authors: Francis Engelmann, Theodora Kontogianni, Alexander Hermans, Bastian Leibe
Date: 2018-02-05 00:00:00
Tasks: 3D Semantic Segmentation, Segmentation, Semantic Segmentation

Deep learning approaches have made tremendous progress in the field of semantic segmentation over the past few years. However, most current approaches operate in the 2D image space. Direct semantic segmentation of unstructured 3D point clouds is still an open research problem. The recently proposed PointNet architecture presents an interesting step ahead in that it can operate on unstructured point clouds, achieving encouraging segmentation results. However, it subdivides the input points into a grid of blocks and processes each such block individually. In this paper, we investigate the question how such an architecture can be extended to incorporate larger-scale spatial context. We build upon PointNet and propose two extensions that enlarge the receptive field over the 3D scene. We evaluate the proposed strategies on challenging indoor and outdoor datasets and show improved results in both scenarios.
Automatic differentiation in machine learning: a survey
Authors: Atilim Gunes Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, Jeffrey Mark Siskind
Date: 2015-02-20 00:00:00
Tasks: BIG-bench Machine Learning, Survey

Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in
machine learning. Automatic differentiation (AD), also called algorithmic
differentiation or simply "autodiff", is a family of techniques similar to but
more general than backpropagation for efficiently and accurately evaluating
derivatives of numeric functions expressed as computer programs. AD is a small
but established field with applications in areas including computational fluid
dynamics, atmospheric sciences, and engineering design optimization. Until very
recently, the fields of machine learning and AD have largely been unaware of
each other and, in some cases, have independently discovered each other's
results. Despite its relevance, general-purpose AD has been missing from the
machine learning toolbox, a situation slowly changing with its ongoing adoption
under the names "dynamic computational graphs" and "differentiable
programming". We survey the intersection of AD and machine learning, cover
applications where AD has direct relevance, and address the main implementation
techniques. By precisely defining the main differentiation techniques and their
interrelationships, we aim to bring clarity to the usage of the terms
"autodiff", "automatic differentiation", and "symbolic differentiation" as
these are encountered more and more in machine learning settings.
The Sea Exploration Problem: Data-driven Orienteering on a Continuous Surface
Authors: João Pedro Pedroso, Alpar Vajk Kramer, Ke Zhang
Date: 2018-02-05 00:00:00
Tasks: Gaussian Processes

This paper describes a problem arising in sea exploration, where the aim is
to schedule the expedition of a ship for collecting information about the
resources on the seafloor. The aim is to collect data by probing on a set of
carefully chosen locations, so that the information available is optimally
enriched. This problem has similarities with the orienteering problem, where
the aim is to plan a time-limited trip for visiting a set of vertices,
collecting a prize at each of them, in such a way that the total value
collected is maximum. In our problem, the score at each vertex is associated
with an estimation of the level of the resource on the given surface, which is
done by regression using Gaussian processes. Hence, there is a correlation
among scores on the selected vertices; this is a first difference with respect
to the standard orienteering problem. The second difference is the location of
each vertex, which in our problem is a freely chosen point on a given surface.
Diverse Beam Search for Increased Novelty in Abstractive Summarization
Authors: André Cibils, Claudiu Musat, Andreea Hossman, Michael Baeriswyl
Date: 2018-02-05 00:00:00
Tasks: Abstractive Text Summarization, Diversity, Text Summarization

Text summarization condenses a text to a shorter version while retaining the
important informations. Abstractive summarization is a recent development that
generates new phrases, rather than simply copying or rephrasing sentences
within the original text. Recently neural sequence-to-sequence models have
achieved good results in the field of abstractive summarization, which opens
new possibilities and applications for industrial purposes. However, most
practitioners observe that these models still use large parts of the original
text in the output summaries, making them often similar to extractive
frameworks. To address this drawback, we first introduce a new metric to
measure how much of a summary is extracted from the input text. Secondly, we
present a novel method, that relies on a diversity factor in computing the
neural network loss, to improve the diversity of the summaries generated by any
neural abstractive model implementing beam search. Finally, we show that this
method not only makes the system less extractive, but also improves the overall
rouge score of state-of-the-art methods by at least 2 points.
Learning Low-shot facial representations via 2D warping
Authors: Shen Yan
Date: 2017-12-13 00:00:00
Tasks: Face Recognition

In this work, we mainly study the influence of the 2D warping module for
one-shot face recognition.
Road Segmentation in SAR Satellite Images with Deep Fully-Convolutional Neural Networks
Authors: Corentin Henry, Seyed Majid Azimi, Nina Merkle
Date: 2018-02-05 00:00:00
Tasks: Road Segmentation, Segmentation

Remote sensing is extensively used in cartography. As transportation networks
grow and change, extracting roads automatically from satellite images is
crucial to keep maps up-to-date. Synthetic Aperture Radar satellites can
provide high resolution topographical maps. However roads are difficult to
identify in these data as they look visually similar to targets such as rivers
and railways. Most road extraction methods on Synthetic Aperture Radar images
still rely on a prior segmentation performed by classical computer vision
algorithms. Few works study the potential of deep learning techniques, despite
their successful applications to optical imagery. This letter presents an
evaluation of Fully-Convolutional Neural Networks for road segmentation in SAR
images. We study the relative performance of early and state-of-the-art
networks after carefully enhancing their sensitivity towards thin objects by
adding spatial tolerance rules. Our models shows promising results,
successfully extracting most of the roads in our test dataset. This shows that,
although Fully-Convolutional Neural Networks natively lack efficiency for road
segmentation, they are capable of good results if properly tuned. As the
segmentation quality does not scale well with the increasing depth of the
networks, the design of specialized architectures for roads extraction should
yield better performances.
A Method for Restoring the Training Set Distribution in an Image Classifier
Authors: Alexey Chaplygin, Joshua Chacksfield
Date: 2018-02-05 00:00:00
Tasks: General Classification, image-classification, Image Classification

Convolutional Neural Networks are a well-known staple of modern image
classification. However, it can be difficult to assess the quality and
robustness of such models. Deep models are known to perform well on a given
training and estimation set, but can easily be fooled by data that is
specifically generated for the purpose. It has been shown that one can produce
an artificial example that does not represent the desired class, but activates
the network in the desired way. This paper describes a new way of
reconstructing a sample from the training set distribution of an image
classifier without deep knowledge about the underlying distribution. This
enables access to the elements of images that most influence the decision of a
convolutional network and to extract meaningful information about the training
distribution.
Conceptual Text Summarizer: A new model in continuous vector space
Authors: Mohammad Ebrahim Khademi, Mohammad Fakhredanesh, Seyed Mojtaba Hoseini
Date: 2017-10-30 00:00:00
Tasks: Extractive Summarization

Traditional methods of summarization are not cost-effective and possible
today. Extractive summarization is a process that helps to extract the most
important sentences from a text automatically and generates a short informative
summary. In this work, we propose an unsupervised method to summarize Persian
texts. This method is a novel hybrid approach that clusters the concepts of the
text using deep learning and traditional statistical methods. First we produce
a word embedding based on Hamshahri2 corpus and a dictionary of word
frequencies. Then the proposed algorithm extracts the keywords of the document,
clusters its concepts, and finally ranks the sentences to produce the summary.
We evaluated the proposed method on Pasokh single-document corpus using the
ROUGE evaluation measure. Without using any hand-crafted features, our proposed
method achieves state-of-the-art results. We compared our unsupervised method
with the best supervised Persian methods and we achieved an overall improvement
of ROUGE-2 recall score of 7.5%.
Online Compact Convexified Factorization Machine
Authors: Wenpeng Zhang, Xiao Lin, Peilin Zhao
Date: 2018-02-05 00:00:00
Tasks: Binary Classification, Feature Engineering

Factorization Machine (FM) is a supervised learning approach with a powerful
capability of feature engineering. It yields state-of-the-art performance in
various batch learning tasks where all the training data is made available
prior to the training. However, in real-world applications where the data
arrives sequentially in a streaming manner, the high cost of re-training with
batch learning algorithms has posed formidable challenges in the online
learning scenario. The initial challenge is that no prior formulations of FM
could fulfill the requirements in Online Convex Optimization (OCO) -- the
paramount framework for online learning algorithm design. To address the
aforementioned challenge, we invent a new convexification scheme leading to a
Compact Convexified FM (CCFM) that seamlessly meets the requirements in OCO.
However for learning Compact Convexified FM (CCFM) in the online learning
setting, most existing algorithms suffer from expensive projection operations.
To address this subsequent challenge, we follow the general projection-free
algorithmic framework of Online Conditional Gradient and propose an Online
Compact Convex Factorization Machine (OCCFM) algorithm that eschews the
projection operation with efficient linear optimization steps. In support of
the proposed OCCFM in terms of its theoretical foundation, we prove that the
developed algorithm achieves a sub-linear regret bound. To evaluate the
empirical performance of OCCFM, we conduct extensive experiments on 6
real-world datasets for online recommendation and binary classification tasks.
The experimental results show that OCCFM outperforms the state-of-art online
learning algorithms.
Causal Generative Neural Networks
Authors: Olivier Goudet, Diviyan Kalainathan, Philippe Caillou, Isabelle Guyon, David Lopez-Paz, Michèle Sebag
Date: 2017-11-24 00:00:00
Proceeding: ICLR 2018 1
Tasks: Causal Discovery

We present Causal Generative Neural Networks (CGNNs) to learn functional
causal models from observational data. CGNNs leverage conditional
independencies and distributional asymmetries to discover bivariate and
multivariate causal structures. CGNNs make no assumption regarding the lack of
confounders, and learn a differentiable generative model of the data by using
backpropagation. Extensive experiments show their good performances
comparatively to the state of the art in observational causal discovery on both
simulated and real data, with respect to cause-effect inference, v-structure
identification, and multivariate causal discovery.
An Epipolar Line from a Single Pixel
Authors: Tavi Halperin, Michael Werman
Date: 2017-03-28 00:00:00

Computing the epipolar geometry from feature points between cameras with very
different viewpoints is often error prone, as an object's appearance can vary
greatly between images. For such cases, it has been shown that using motion
extracted from video can achieve much better results than using a static image.
This paper extends these earlier works based on the scene dynamics. In this
paper we propose a new method to compute the epipolar geometry from a video
stream, by exploiting the following observation: For a pixel p in Image A, all
pixels corresponding to p in Image B are on the same epipolar line.
Equivalently, the image of the line going through camera A's center and p is an
epipolar line in B. Therefore, when cameras A and B are synchronized, the
momentary images of two objects projecting to the same pixel, p, in camera A at
times t1 and t2, lie on an epipolar line in camera B. Based on this observation
we achieve fast and precise computation of epipolar lines. Calibrating cameras
based on our method of finding epipolar lines is much faster and more robust
than previous methods.
Lie Transform--based Neural Networks for Dynamics Simulation and Learning
Authors: Andrei Ivanov, Alena Sholokhova, Sergei Andrianov, Roman Konoplev-Esgenburg
Date: 2018-02-05 00:00:00
Tasks: Small Data Image Classification, Time Series Analysis

In the article, we discuss the architecture of the polynomial neural network that corresponds to the matrix representation of Lie transform. The matrix form of Lie transform is an approximation of the general solution of the nonlinear system of ordinary differential equations. The proposed architecture can be trained with small data sets, extrapolate predictions outside the training data, and provide a possibility for interpretation. We provide a theoretical explanation of the proposed architecture, as well as demonstrate it in several applications. We present the results of modeling and identification for both simple and well-known dynamical systems, and more complicated examples from price dynamics, chemistry, and accelerator physics. From a practical point of view, we describe the training of a Lie transform--based neural network with a small data set containing only 10 data points. We also demonstrate an interpretation of the fitted neural network by converting it to a system of differential equations.
A Theoretical Investigation of Graph Degree as an Unsupervised Normality Measure
Authors: Caglar Aytekin, Francesco Cricri, Lixin Fan, Emre Aksu
Date: 2018-01-24 00:00:00
Tasks: Anomaly Detection, Clustering, Graph Clustering, Spectral Graph Clustering, Unsupervised Anomaly Detection

For a graph representation of a dataset, a straightforward normality measure
for a sample can be its graph degree. Considering a weighted graph, degree of a
sample is the sum of the corresponding row's values in a similarity matrix. The
measure is intuitive given the abnormal samples are usually rare and they are
dissimilar to the rest of the data. In order to have an in-depth theoretical
understanding, in this manuscript, we investigate the graph degree in spectral
graph clustering based and kernel based point of views and draw connections to
a recent kernel method for the two sample problem. We show that our analyses
guide us to choose fully-connected graphs whose edge weights are calculated via
universal kernels. We show that a simple graph degree based unsupervised
anomaly detection method with the above properties, achieves higher accuracy
compared to other unsupervised anomaly detection methods on average over 10
widely used datasets. We also provide an extensive analysis on the effect of
the kernel parameter on the method's accuracy.
Deep Rewiring: Training very sparse deep networks
Authors: Guillaume Bellec, David Kappel, Wolfgang Maass, Robert Legenstein
Date: 2017-11-14 00:00:00
Proceeding: ICLR 2018 1

Neuromorphic hardware tends to pose limits on the connectivity of deep
networks that one can run on them. But also generic hardware and software
implementations of deep learning run more efficiently for sparse networks.
Several methods exist for pruning connections of a neural network after it was
trained without connectivity constraints. We present an algorithm, DEEP R, that
enables us to train directly a sparsely connected neural network. DEEP R
automatically rewires the network during supervised training so that
connections are there where they are most needed for the task, while its total
number is all the time strictly bounded. We demonstrate that DEEP R can be used
to train very sparse feedforward and recurrent neural networks on standard
benchmark tasks with just a minor loss in performance. DEEP R is based on a
rigorous theoretical foundation that views rewiring as stochastic sampling of
network configurations from a posterior.
Faster Learning by Reduction of Data Access Time
Authors: Vinod Kumar Chauhan, Anuj Sharma, Kalpana Dahiya
Date: 2018-01-18 00:00:00
Tasks: BIG-bench Machine Learning

Nowadays, the major challenge in machine learning is the Big Data challenge.
The big data problems due to large number of data points or large number of
features in each data point, or both, the training of models have become very
slow. The training time has two major components: Time to access the data and
time to process (learn from) the data. So far, the research has focused only on
the second part, i.e., learning from the data. In this paper, we have proposed
one possible solution to handle the big data problems in machine learning. The
idea is to reduce the training time through reducing data access time by
proposing systematic sampling and cyclic/sequential sampling to select
mini-batches from the dataset. To prove the effectiveness of proposed sampling
techniques, we have used Empirical Risk Minimization, which is commonly used
machine learning problem, for strongly convex and smooth case. The problem has
been solved using SAG, SAGA, SVRG, SAAG-II and MBSGD (Mini-batched SGD), each
using two step determination techniques, namely, constant step size and
backtracking line search method. Theoretical results prove the same convergence
for systematic sampling, cyclic sampling and the widely used random sampling
technique, in expectation. Experimental results with bench marked datasets
prove the efficacy of the proposed sampling techniques and show up to six times
faster training.
Comparison of computer systems and ranking criteria for automatic melanoma detection in dermoscopic images
Authors: Kajsa Møllersen, Maciel Zortea, Thomas R. Schopf, Herbert Kirchesch, Fred Godtliebsen
Date: 2018-02-05 00:00:00
Tasks: Diagnostic, General Classification, Segmentation

Melanoma is the deadliest form of skin cancer. Computer systems can assist in
melanoma detection, but are not widespread in clinical practice. In 2016, an
open challenge in classification of dermoscopic images of skin lesions was
announced. A training set of 900 images with corresponding class labels and
semi-automatic/manual segmentation masks was released for the challenge. An
independent test set of 379 images was used to rank the participants. This
article demonstrates the impact of ranking criteria, segmentation method and
classifier, and highlights the clinical perspective. We compare five different
measures for diagnostic accuracy by analysing the resulting ranking of the
computer systems in the challenge. Choice of performance measure had great
impact on the ranking. Systems that were ranked among the top three for one
measure, dropped to the bottom half when changing performance measure. Nevus
Doctor, a computer system previously developed by the authors, was used to
investigate the impact of segmentation and classifier. The unexpected small
impact of automatic versus semi-automatic/manual segmentation suggests that
improvements of the automatic segmentation method w.r.t. resemblance to
semi-automatic/manual segmentation will not improve diagnostic accuracy
substantially. A small set of similar classification algorithms are used to
investigate the impact of classifier on the diagnostic accuracy. The
variability in diagnostic accuracy for different classifier algorithms was
larger than the variability for segmentation methods, and suggests a focus for
future investigations. From a clinical perspective, the misclassification of a
melanoma as benign has far greater cost than the misclassification of a benign
lesion. For computer systems to have clinical impact, their performance should
be ranked by a high-sensitivity measure.