pwc-archive/papers-with-abstracts huggingface dataset:
Concept-Oriented Deep Learning
Authors: Daniel T. Chang
Date: 2018-06-05 00:00:00
Tasks: Continual Learning, Deep Learning, Representation Learning

Concepts are the foundation of human deep learning, understanding, and
knowledge integration and transfer. We propose concept-oriented deep learning
(CODL) which extends (machine) deep learning with concept representations and
conceptual understanding capability. CODL addresses some of the major
limitations of deep learning: interpretability, transferability, contextual
adaptation, and requirement for lots of labeled training data. We discuss the
major aspects of CODL including concept graph, concept representations, concept
exemplars, and concept representation learning systems supporting incremental
and continual learning.
Neural-Kernelized Conditional Density Estimation
Authors: Hiroaki Sasaki, Aapo Hyvärinen
Date: 2018-06-05 00:00:00
Tasks: Density Estimation, Dimensionality Reduction, Representation Learning

Conditional density estimation is a general framework for solving various
problems in machine learning. Among existing methods, non-parametric and/or
kernel-based methods are often difficult to use on large datasets, while
methods based on neural networks usually make restrictive parametric
assumptions on the probability densities. Here, we propose a novel method for
estimating the conditional density based on score matching. In contrast to
existing methods, we employ scalable neural networks, but do not make explicit
parametric assumptions on densities. The key challenge in applying score
matching to neural networks is computation of the first- and second-order
derivatives of a model for the log-density. We tackle this challenge by
developing a new neural-kernelized approach, which can be applied on large
datasets with stochastic gradient descent, while the reproducing kernels allow
for easy computation of the derivatives needed in score matching. We show that
the neural-kernelized function approximator has universal approximation
capability and that our method is consistent in conditional density estimation.
We numerically demonstrate that our method is useful in high-dimensional
conditional density estimation, and compares favourably with existing methods.
Finally, we prove that the proposed method has interesting connections to two
probabilistically principled frameworks of representation learning: Nonlinear
sufficient dimension reduction and nonlinear independent component analysis.
A Machine Learning Framework for Stock Selection
Authors: XingYu Fu, JinHong Du, Yifeng Guo, Mingwen Liu, Tao Dong, XiuWen Duan
Date: 2018-06-05 00:00:00
Tasks: BIG-bench Machine Learning, feature selection

This paper demonstrates how to apply machine learning algorithms to
distinguish good stocks from the bad stocks. To this end, we construct 244
technical and fundamental features to characterize each stock, and label stocks
according to their ranking with respect to the return-to-volatility ratio.
Algorithms ranging from traditional statistical learning methods to recently
popular deep learning method, e.g. Logistic Regression (LR), Random Forest
(RF), Deep Neural Network (DNN), and the Stacking, are trained to solve the
classification task. Genetic Algorithm (GA) is also used to implement feature
selection. The effectiveness of the stock selection strategy is validated in
Chinese stock market in both statistical and practical aspects, showing that:
1) Stacking outperforms other models reaching an AUC score of 0.972; 2) Genetic
Algorithm picks a subset of 114 features and the prediction performances of all
models remain almost unchanged after the selection procedure, which suggests
some features are indeed redundant; 3) LR and DNN are radical models; RF is
risk-neutral model; Stacking is somewhere between DNN and RF. 4) The portfolios
constructed by our models outperform market average in back tests.
The EuroCity Persons Dataset: A Novel Benchmark for Object Detection
Authors: Markus Braun, Sebastian Krebs, Fabian Flohr, Dariu M. Gavrila
Date: 2018-05-18 00:00:00
Tasks: Benchmarking, Object, object-detection, Object Detection

Big data has had a great share in the success of deep learning in computer
vision. Recent works suggest that there is significant further potential to
increase object detection performance by utilizing even bigger datasets. In
this paper, we introduce the EuroCity Persons dataset, which provides a large
number of highly diverse, accurate and detailed annotations of pedestrians,
cyclists and other riders in urban traffic scenes. The images for this dataset
were collected on-board a moving vehicle in 31 cities of 12 European countries.
With over 238200 person instances manually labeled in over 47300 images,
EuroCity Persons is nearly one order of magnitude larger than person datasets
used previously for benchmarking. The dataset furthermore contains a large
number of person orientation annotations (over 211200). We optimize four
state-of-the-art deep learning approaches (Faster R-CNN, R-FCN, SSD and YOLOv3)
to serve as baselines for the new object detection benchmark. In experiments
with previous datasets we analyze the generalization capabilities of these
detectors when trained with the new dataset. We furthermore study the effect of
the training set size, the dataset diversity (day- vs. night-time, geographical
region), the dataset detail (i.e. availability of object orientation
information) and the annotation quality on the detector performance. Finally,
we analyze error sources and discuss the road ahead.
Disease Prediction using Graph Convolutional Networks: Application to Autism Spectrum Disorder and Alzheimer's Disease
Authors: Sarah Parisot, Sofia Ira Ktena, Enzo Ferrante, Matthew Lee, Ricardo Guerrero, Ben Glocker, Daniel Rueckert
Date: 2018-06-05 00:00:00
Tasks: Disease Prediction, General Classification

Graphs are widely used as a natural framework that captures interactions
between individual elements represented as nodes in a graph. In medical
applications, specifically, nodes can represent individuals within a
potentially large population (patients or healthy controls) accompanied by a
set of features, while the graph edges incorporate associations between
subjects in an intuitive manner. This representation allows to incorporate the
wealth of imaging and non-imaging information as well as individual subject
features simultaneously in disease classification tasks. Previous graph-based
approaches for supervised or unsupervised learning in the context of disease
prediction solely focus on pairwise similarities between subjects, disregarding
individual characteristics and features, or rather rely on subject-specific
imaging feature vectors and fail to model interactions between them. In this
paper, we present a thorough evaluation of a generic framework that leverages
both imaging and non-imaging information and can be used for brain analysis in
large populations. This framework exploits Graph Convolutional Networks (GCNs)
and involves representing populations as a sparse graph, where its nodes are
associated with imaging-based feature vectors, while phenotypic information is
integrated as edge weights. The extensive evaluation explores the effect of
each individual component of this framework on disease prediction performance
and further compares it to different baselines. The framework performance is
tested on two large datasets with diverse underlying data, ABIDE and ADNI, for
the prediction of Autism Spectrum Disorder and conversion to Alzheimer's
disease, respectively. Our analysis shows that our novel framework can improve
over state-of-the-art results on both databases, with 70.4% classification
accuracy for ABIDE and 80.0% for ADNI.
Luminoso at SemEval-2018 Task 10: Distinguishing Attributes Using Text Corpora and Relational Knowledge
Authors: Robyn Speer, Joanna Lowry-Duda
Date: 2018-06-05 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: General Knowledge, Relation Extraction

Luminoso participated in the SemEval 2018 task on "Capturing Discriminative
Attributes" with a system based on ConceptNet, an open knowledge graph focused
on general knowledge. In this paper, we describe how we trained a linear
classifier on a small number of semantically-informed features to achieve an
$F_1$ score of 0.7368 on the task, close to the task's high score of 0.75.
Machine Learning for Yield Curve Feature Extraction: Application to Illiquid Corporate Bonds (Preliminary Draft)
Authors: Greg Kirczenow, Ali Fathi, Matt Davison
Date: 2018-06-05 00:00:00
Tasks: BIG-bench Machine Learning, Denoising

This paper studies the application of machine learning in extracting the
market implied features from historical risk neutral corporate bond yields. We
consider the example of a hypothetical illiquid fixed income market. After
choosing a surrogate liquid market, we apply the Denoising Autoencoder
algorithm from the field of computer vision and pattern recognition to learn
the features of the missing yield parameters from the historically implied data
of the instruments traded in the chosen liquid market. The results of the
trained machine learning algorithm are compared with the outputs of a point in-
time 2 dimensional interpolation algorithm known as the Thin Plate Spline.
Finally, the performances of the two algorithms are compared.
EasyConvPooling: Random Pooling with Easy Convolution for Accelerating Training and Testing
Authors: Jianzhong Sheng, Chuanbo Chen, Chenchen Fu, Chun Jason Xue
Date: 2018-06-05 00:00:00

Convolution operations dominate the overall execution time of Convolutional
Neural Networks (CNNs). This paper proposes an easy yet efficient technique for
both Convolutional Neural Network training and testing. The conventional
convolution and pooling operations are replaced by Easy Convolution and Random
Pooling (ECP). In ECP, we randomly select one pixel out of four and only
conduct convolution operations of the selected pixel. As a result, only a
quarter of the conventional convolution computations are needed. Experiments
demonstrate that the proposed EasyConvPooling can achieve 1.45x speedup on
training time and 1.64x on testing time. What's more, a speedup of 5.09x on
pure Easy Convolution operations is obtained compared to conventional
convolution operations.
Level-Based Analysis of the Population-Based Incremental Learning Algorithm
Authors: Per Kristian Lehre, Phan Trung Hai Nguyen
Date: 2018-06-05 00:00:00
Tasks: Incremental Learning

The Population-Based Incremental Learning (PBIL) algorithm uses a convex
combination of the current model and the empirical model to construct the next
model, which is then sampled to generate offspring. The Univariate Marginal
Distribution Algorithm (UMDA) is a special case of the PBIL, where the current
model is ignored. Dang and Lehre (GECCO 2015) showed that UMDA can optimise
LeadingOnes efficiently. The question still remained open if the PBIL performs
equally well. Here, by applying the level-based theorem in addition to
Dvoretzky--Kiefer--Wolfowitz inequality, we show that the PBIL optimises
function LeadingOnes in expected time $\mathcal{O}(n\lambda \log \lambda +
n^2)$ for a population size $\lambda = \Omega(\log n)$, which matches the bound
of the UMDA. Finally, we show that the result carries over to BinVal, giving
the fist runtime result for the PBIL on the BinVal problem.
Evolution leads to a diversity of motion-detection neuronal circuits
Authors: Ali Tehrani-Saleh, Thomas LaBar, Christoph Adami
Date: 2018-04-07 00:00:00
Tasks: Diversity, Motion Detection

A central goal of evolutionary biology is to explain the origins and
distribution of diversity across life. Beyond species or genetic diversity, we
also observe diversity in the circuits (genetic or otherwise) underlying
complex functional traits. However, while the theory behind the origins and
maintenance of genetic and species diversity has been studied for decades,
theory concerning the origin of diverse functional circuits is still in its
infancy. It is not known how many different circuit structures can implement
any given function, which evolutionary factors lead to different circuits, and
whether the evolution of a particular circuit was due to adaptive or
non-adaptive processes. Here, we use digital experimental evolution to study
the diversity of neural circuits that encode motion detection in digital
(artificial) brains. We find that evolution leads to an enormous diversity of
potential neural architectures encoding motion detection circuits, even for
circuits encoding the exact same function. Evolved circuits vary in both
redundancy and complexity (as previously found in genetic circuits) suggesting
that similar evolutionary principles underlie circuit formation using any
substrate. We also show that a simple (designed) motion detection circuit that
is optimally-adapted gains in complexity when evolved further, and that
selection for mutational robustness led this gain in complexity.
Exploiting ConvNet Diversity for Flooding Identification
Authors: Keiller Nogueira, Samuel G. Fadel, Ícaro C. Dourado, Rafael de O. Werneck, Javier A. V. Muñoz, Otávio A. B. Penatti, Rodrigo T. Calumby, Lin Tzy Li, Jefersson A. dos Santos, Ricardo da S. Torres
Date: 2017-11-09 00:00:00
Tasks: Diversity

Flooding is the world's most costly type of natural disaster in terms of both
economic losses and human causalities. A first and essential procedure towards
flood monitoring is based on identifying the area most vulnerable to flooding,
which gives authorities relevant regions to focus. In this work, we propose
several methods to perform flooding identification in high-resolution remote
sensing images using deep learning. Specifically, some proposed techniques are
based upon unique networks, such as dilated and deconvolutional ones, while
other was conceived to exploit diversity of distinct networks in order to
extract the maximum performance of each classifier. Evaluation of the proposed
algorithms were conducted in a high-resolution remote sensing dataset. Results
show that the proposed algorithms outperformed several state-of-the-art
baselines, providing improvements ranging from 1 to 4% in terms of the Jaccard
Index.
Understanding Meanings in Multilingual Customer Feedback
Authors: Chao-Hong Liu, Declan Groves, Akira Hayakawa, Alberto Poncelas, Qun Liu
Date: 2018-06-05 00:00:00
Tasks: General Classification

Understanding and being able to react to customer feedback is the most
fundamental task in providing good customer service. However, there are two
major obstacles for international companies to automatically detect the meaning
of customer feedback in a global multilingual environment. Firstly, there is no
widely acknowledged categorisation (classes) of meaning for customer feedback.
Secondly, the applicability of one meaning categorisation, if it exists, to
customer feedback in multiple languages is questionable. In this paper, we
extracted representative real world samples of customer feedback from Microsoft
Office customers in multiple languages, English, Spanish and Japanese,and
concluded a five-class categorisation(comment, request, bug, complaint and
meaningless) for meaning classification that could be used across languages in
the realm of customer feedback analysis.
State Representation Learning for Control: An Overview
Authors: Timothée Lesort, Natalia Díaz-Rodríguez, Jean-François Goudou, David Filliat
Date: 2018-02-12 00:00:00
Tasks: Reinforcement Learning, Representation Learning

Representation learning algorithms are designed to learn abstract features
that characterize data. State representation learning (SRL) focuses on a
particular kind of representation learning where learned features are in low
dimension, evolve through time, and are influenced by actions of an agent. The
representation is learned to capture the variation in the environment generated
by the agent's actions; this kind of representation is particularly suitable
for robotics and control scenarios. In particular, the low dimension
characteristic of the representation helps to overcome the curse of
dimensionality, provides easier interpretation and utilization by humans and
can help improve performance and speed in policy learning algorithms such as
reinforcement learning.
  This survey aims at covering the state-of-the-art on state representation
learning in the most recent years. It reviews different SRL methods that
involve interaction with the environment, their implementations and their
applications in robotics control tasks (simulated or real). In particular, it
highlights how generic learning objectives are differently exploited in the
reviewed algorithms. Finally, it discusses evaluation methods to assess the
representation learned and summarizes current and future lines of research.
Scikit-learn: Machine Learning in Python
Authors: Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Andreas Müller, Joel Nothman, Gilles Louppe, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, Édouard Duchesnay
Date: 2012-01-02 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Dimensionality Reduction, General Classification, Model Selection

Scikit-learn is a Python module integrating a wide range of state-of-the-art
machine learning algorithms for medium-scale supervised and unsupervised
problems. This package focuses on bringing machine learning to non-specialists
using a general-purpose high-level language. Emphasis is put on ease of use,
performance, documentation, and API consistency. It has minimal dependencies
and is distributed under the simplified BSD license, encouraging its use in
both academic and commercial settings. Source code, binaries, and documentation
can be downloaded from http://scikit-learn.org.
A Projection Method for Metric-Constrained Optimization
Authors: Nate Veldt, David Gleich, Anthony Wirth, James Saunderson
Date: 2018-06-05 00:00:00
Tasks: Clustering, Graph Clustering

We outline a new approach for solving optimization problems which enforce
triangle inequalities on output variables. We refer to this as
metric-constrained optimization, and give several examples where problems of
this form arise in machine learning applications and theoretical approximation
algorithms for graph clustering. Although these problem are interesting from a
theoretical perspective, they are challenging to solve in practice due to the
high memory requirement of black-box solvers. In order to address this
challenge we first prove that the metric-constrained linear program relaxation
of correlation clustering is equivalent to a special case of the metric
nearness problem. We then developed a general solver for metric-constrained
linear and quadratic programs by generalizing and improving a simple projection
algorithm originally developed for metric nearness. We give several novel
approximation guarantees for using our framework to find lower bounds for
optimal solutions to several challenging graph clustering problems. We also
demonstrate the power of our framework by solving optimizing problems involving
up to 10^{8} variables and 10^{11} constraints.
Practical Deep Stereo (PDS): Toward applications-friendly deep stereo matching
Authors: Stepan Tulyakov, Anton Ivanov, Francois Fleuret
Date: 2018-06-05 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Stereo Matching, Stereo Matching Hand

End-to-end deep-learning networks recently demonstrated extremely good
perfor- mance for stereo matching. However, existing networks are difficult to
use for practical applications since (1) they are memory-hungry and unable to
process even modest-size images, (2) they have to be trained for a given
disparity range. The Practical Deep Stereo (PDS) network that we propose
addresses both issues: First, its architecture relies on novel bottleneck
modules that drastically reduce the memory footprint in inference, and
additional design choices allow to handle greater image size during training.
This results in a model that leverages large image context to resolve matching
ambiguities. Second, a novel sub-pixel cross- entropy loss combined with a MAP
estimator make this network less sensitive to ambiguous matches, and applicable
to any disparity range without re-training. We compare PDS to state-of-the-art
methods published over the recent months, and demonstrate its superior
performance on FlyingThings3D and KITTI sets.
Recurrent Convolutional Fusion for RGB-D Object Recognition
Authors: Mohammad Reza Loghmani, Mirco Planamente, Barbara Caputo, Markus Vincze
Date: 2018-06-05 00:00:00
Tasks: Object, Object Categorization, Object Recognition

Providing machines with the ability to recognize objects like humans has
always been one of the primary goals of machine vision. The introduction of
RGB-D cameras has paved the way for a significant leap forward in this
direction thanks to the rich information provided by these sensors. However,
the machine vision community still lacks an effective method to synergically
use the RGB and depth data to improve object recognition. In order to take a
step in this direction, we introduce a novel end-to-end architecture for RGB-D
object recognition called recurrent convolutional fusion (RCFusion). Our method
generates compact and highly discriminative multi-modal features by combining
complementary RGB and depth information representing different levels of
abstraction. Extensive experiments on two popular datasets, RGB-D Object
Dataset and JHUIT-50, show that RCFusion significantly outperforms
state-of-the-art approaches in both the object categorization and instance
recognition tasks.
The Many Faces of Exponential Weights in Online Learning
Authors: Dirk van der Hoeven, Tim van Erven, Wojciech Kotłowski
Date: 2018-02-21 00:00:00
Tasks: Second-order methods

A standard introduction to online learning might place Online Gradient
Descent at its center and then proceed to develop generalizations and
extensions like Online Mirror Descent and second-order methods. Here we explore
the alternative approach of putting Exponential Weights (EW) first. We show
that many standard methods and their regret bounds then follow as a special
case by plugging in suitable surrogate losses and playing the EW posterior
mean. For instance, we easily recover Online Gradient Descent by using EW with
a Gaussian prior on linearized losses, and, more generally, all instances of
Online Mirror Descent based on regular Bregman divergences also correspond to
EW with a prior that depends on the mirror map. Furthermore, appropriate
quadratic surrogate losses naturally give rise to Online Gradient Descent for
strongly convex losses and to Online Newton Step. We further interpret several
recent adaptive methods (iProd, Squint, and a variation of Coin Betting for
experts) as a series of closely related reductions to exp-concave surrogate
losses that are then handled by Exponential Weights. Finally, a benefit of our
EW interpretation is that it opens up the possibility of sampling from the EW
posterior distribution instead of playing the mean. As already observed by
Bubeck and Eldan, this recovers the best-known rate in Online Bandit Linear
Optimization.