pwc-archive/papers-with-abstracts huggingface dataset:
Short-Term Forecasting of Passenger Demand under On-Demand Ride Services: A Spatio-Temporal Deep Learning Approach
Authors: Jintao Ke, Hongyu Zheng, Hai Yang, Xiqun, Chen
Date: 2017-06-20 00:00:00
Tasks: Demand Forecasting, feature selection, Time Series Analysis, Time Series Prediction

Short-term passenger demand forecasting is of great importance to the
on-demand ride service platform, which can incentivize vacant cars moving from
over-supply regions to over-demand regions. The spatial dependences, temporal
dependences, and exogenous dependences need to be considered simultaneously,
however, which makes short-term passenger demand forecasting challenging. We
propose a novel deep learning (DL) approach, named the fusion convolutional
long short-term memory network (FCL-Net), to address these three dependences
within one end-to-end learning architecture. The model is stacked and fused by
multiple convolutional long short-term memory (LSTM) layers, standard LSTM
layers, and convolutional layers. The fusion of convolutional techniques and
the LSTM network enables the proposed DL approach to better capture the
spatio-temporal characteristics and correlations of explanatory variables. A
tailored spatially aggregated random forest is employed to rank the importance
of the explanatory variables. The ranking is then used for feature selection.
The proposed DL approach is applied to the short-term forecasting of passenger
demand under an on-demand ride service platform in Hangzhou, China.
Experimental results, validated on real-world data provided by DiDi Chuxing,
show that the FCL-Net achieves better predictive performance than traditional
approaches including both classical time-series prediction models and neural
network based algorithms (e.g., artificial neural network and LSTM). This paper
is one of the first DL studies to forecast the short-term passenger demand of
an on-demand ride service platform by examining the spatio-temporal
correlations.
Using Artificial Tokens to Control Languages for Multilingual Image Caption Generation
Authors: Satoshi Tsutsui, David Crandall
Date: 2017-06-20 00:00:00
Tasks: Caption Generation

Recent work in computer vision has yielded impressive results in
automatically describing images with natural language. Most of these systems
generate captions in a sin- gle language, requiring multiple language-specific
models to build a multilingual captioning system. We propose a very simple
technique to build a single unified model across languages, using artificial
tokens to control the language, making the captioning system more compact. We
evaluate our approach on generating English and Japanese captions, and show
that a typical neural captioning architecture is capable of learning a single
model that can switch between two different languages.
Learning Graphical Models Using Multiplicative Weights
Authors: Adam Klivans, Raghu Meka
Date: 2017-06-20 00:00:00

We give a simple, multiplicative-weight update algorithm for learning
undirected graphical models or Markov random fields (MRFs). The approach is
new, and for the well-studied case of Ising models or Boltzmann machines, we
obtain an algorithm that uses a nearly optimal number of samples and has
quadratic running time (up to logarithmic factors), subsuming and improving on
all prior work. Additionally, we give the first efficient algorithm for
learning Ising models over general alphabets.
  Our main application is an algorithm for learning the structure of t-wise
MRFs with nearly-optimal sample complexity (up to polynomial losses in
necessary terms that depend on the weights) and running time that is
$n^{O(t)}$. In addition, given $n^{O(t)}$ samples, we can also learn the
parameters of the model and generate a hypothesis that is close in statistical
distance to the true MRF. All prior work runs in time $n^{\Omega(d)}$ for
graphs of bounded degree d and does not generate a hypothesis close in
statistical distance even for t=3. We observe that our runtime has the correct
dependence on n and t assuming the hardness of learning sparse parities with
noise.
  Our algorithm--the Sparsitron-- is easy to implement (has only one parameter)
and holds in the on-line setting. Its analysis applies a regret bound from
Freund and Schapire's classic Hedge algorithm. It also gives the first solution
to the problem of learning sparse Generalized Linear Models (GLMs).
Adaptive Consensus ADMM for Distributed Optimization
Authors: Zheng Xu, Gavin Taylor, Hao Li, Mario Figueiredo, Xiaoming Yuan, Tom Goldstein
Date: 2017-06-09 00:00:00
Proceeding: ICML 2017 8
Tasks: Distributed Optimization

The alternating direction method of multipliers (ADMM) is commonly used for
distributed model fitting problems, but its performance and reliability depend
strongly on user-defined penalty parameters. We study distributed ADMM methods
that boost performance by using different fine-tuned algorithm parameters on
each worker node. We present a O(1/k) convergence rate for adaptive ADMM
methods with node-specific parameters, and propose adaptive consensus ADMM
(ACADMM), which automatically tunes parameters without user oversight.
Statistical Mechanics of Node-perturbation Learning with Noisy Baseline
Authors: Kazuyuki Hara, Kentaro Katahira, Masato Okada
Date: 2017-06-20 00:00:00
Tasks: Reinforcement Learning

Node-perturbation learning is a type of statistical gradient descent
algorithm that can be applied to problems where the objective function is not
explicitly formulated, including reinforcement learning. It estimates the
gradient of an objective function by using the change in the object function in
response to the perturbation. The value of the objective function for an
unperturbed output is called a baseline. Cho et al. proposed node-perturbation
learning with a noisy baseline. In this paper, we report on building the
statistical mechanics of Cho's model and on deriving coupled differential
equations of order parameters that depict learning dynamics. We also show how
to derive the generalization error by solving the differential equations of
order parameters. On the basis of the results, we show that Cho's results are
also apply in general cases and show some general performances of Cho's model.
Analysis of dropout learning regarded as ensemble learning
Authors: Kazuyuki Hara, Daisuke Saitoh, Hayaru Shouno
Date: 2017-06-20 00:00:00
Tasks: Ensemble Learning, Object Recognition, speech-recognition, Speech Recognition

Deep learning is the state-of-the-art in fields such as visual object
recognition and speech recognition. This learning uses a large number of
layers, huge number of units, and connections. Therefore, overfitting is a
serious problem. To avoid this problem, dropout learning is proposed. Dropout
learning neglects some inputs and hidden units in the learning process with a
probability, p, and then, the neglected inputs and hidden units are combined
with the learned network to express the final output. We find that the process
of combining the neglected hidden units with the learned network can be
regarded as ensemble learning, so we analyze dropout learning from this point
of view.
Learning-based Ensemble Average Propagator Estimation
Authors: Chuyang Ye
Date: 2017-06-20 00:00:00

By capturing the anisotropic water diffusion in tissue, diffusion magnetic
resonance imaging (dMRI) provides a unique tool for noninvasively probing the
tissue microstructure and orientation in the human brain. The diffusion profile
can be described by the ensemble average propagator (EAP), which is inferred
from observed diffusion signals. However, accurate EAP estimation using the
number of diffusion gradients that is clinically practical can be challenging.
In this work, we propose a deep learning algorithm for EAP estimation, which is
named learning-based ensemble average propagator estimation (LEAPE). The EAP is
commonly represented by a basis and its associated coefficients, and here we
choose the SHORE basis and design a deep network to estimate the coefficients.
The network comprises two cascaded components. The first component is a
multiple layer perceptron (MLP) that simultaneously predicts the unknown
coefficients. However, typical training loss functions, such as mean squared
errors, may not properly represent the geometry of the possibly non-Euclidean
space of the coefficients, which in particular causes problems for the
extraction of directional information from the EAP. Therefore, to regularize
the training, in the second component we compute an auxiliary output of
approximated fiber orientation (FO) errors with the aid of a second MLP that is
trained separately. We performed experiments using dMRI data that resemble
clinically achievable $q$-space sampling, and observed promising results
compared with the conventional EAP estimation method.
Low Resolution Face Recognition Using a Two-Branch Deep Convolutional Neural Network Architecture
Authors: Erfan Zangeneh, Mohammad Rahmati, Yalda Mohsenzadeh
Date: 2017-06-20 00:00:00
Tasks: Face Recognition, Super-Resolution

We propose a novel couple mappings method for low resolution face recognition
using deep convolutional neural networks (DCNNs). The proposed architecture
consists of two branches of DCNNs to map the high and low resolution face
images into a common space with nonlinear transformations. The branch
corresponding to transformation of high resolution images consists of 14 layers
and the other branch which maps the low resolution face images to the common
space includes a 5-layer super-resolution network connected to a 14-layer
network. The distance between the features of corresponding high and low
resolution images are backpropagated to train the networks. Our proposed method
is evaluated on FERET data set and compared with state-of-the-art competing
methods. Our extensive experimental results show that the proposed method
significantly improves the recognition performance especially for very low
resolution probe face images (11.4% improvement in recognition accuracy).
Furthermore, it can reconstruct a high resolution image from its corresponding
low resolution probe image which is comparable with state-of-the-art
super-resolution methods in terms of visual quality.
Solution of linear ill-posed problems using random dictionaries
Authors: Pawan Gupta, Marianna Pensky
Date: 2016-05-25 00:00:00

In the present paper we consider application of overcomplete dictionaries to
solution of general ill-posed linear inverse problems. In the context of
regression problems, there has been enormous amount of effort to recover an
unknown function using such dictionaries. One of the most popular methods,
lasso and its versions, is based on minimizing empirical likelihood and
unfortunately, requires stringent assumptions on the dictionary, the, so
called, compatibility conditions. Though compatibility conditions are hard to
satisfy, it is well known that this can be accomplished by using random
dictionaries. In the present paper, we show how one can apply random
dictionaries to solution of ill-posed linear inverse problems. We put a
theoretical foundation under the suggested methodology and study its
performance via simulations.
Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach
Authors: Takuo Hamaguchi, Hidekazu Oiwa, Masashi Shimbo, Yuji Matsumoto
Date: 2017-06-18 00:00:00
Tasks: Graph Neural Network, Knowledge Base Completion, Transfer Learning

Knowledge base completion (KBC) aims to predict missing information in a
knowledge base.In this paper, we address the out-of-knowledge-base (OOKB)
entity problem in KBC:how to answer queries concerning test entities not
observed at training time. Existing embedding-based KBC models assume that all
test entities are available at training time, making it unclear how to obtain
embeddings for new entities without costly retraining. To solve the OOKB entity
problem without retraining, we use graph neural networks (Graph-NNs) to compute
the embeddings of OOKB entities, exploiting the limited auxiliary knowledge
provided at test time.The experimental results show the effectiveness of our
proposed model in the OOKB setting.Additionally, in the standard KBC setting in
which OOKB entities are not involved, our model achieves state-of-the-art
performance on the WordNet dataset. The code and dataset are available at
https://github.com/takuo-h/GNN-for-OOKB
N-gram Language Modeling using Recurrent Neural Network Estimation
Authors: Ciprian Chelba, Mohammad Norouzi, Samy Bengio
Date: 2017-03-31 00:00:00
Tasks: Language Modeling, Language Modelling, Sentence

We investigate the effective memory depth of RNN models by using them for
$n$-gram language model (LM) smoothing.
  Experiments on a small corpus (UPenn Treebank, one million words of training
data and 10k vocabulary) have found the LSTM cell with dropout to be the best
model for encoding the $n$-gram state when compared with feed-forward and
vanilla RNN models. When preserving the sentence independence assumption the
LSTM $n$-gram matches the LSTM LM performance for $n=9$ and slightly
outperforms it for $n=13$. When allowing dependencies across sentence
boundaries, the LSTM $13$-gram almost matches the perplexity of the unlimited
history LSTM LM.
  LSTM $n$-gram smoothing also has the desirable property of improving with
increasing $n$-gram order, unlike the Katz or Kneser-Ney back-off estimators.
Using multinomial distributions as targets in training instead of the usual
one-hot target is only slightly beneficial for low $n$-gram orders.
  Experiments on the One Billion Words benchmark show that the results hold at
larger scale: while LSTM smoothing for short $n$-gram contexts does not provide
significant advantages over classic N-gram models, it becomes effective with
long contexts ($n > 5$); depending on the task and amount of data it can match
fully recurrent LSTM models at about $n=13$. This may have implications when
modeling short-format text, e.g. voice search/query LMs.
  Building LSTM $n$-gram LMs may be appealing for some practical situations:
the state in a $n$-gram LM can be succinctly represented with $(n-1)*4$ bytes
storing the identity of the words in the context and batches of $n$-gram
contexts can be processed in parallel. On the downside, the $n$-gram context
encoding computed by the LSTM is discarded, making the model more expensive
than a regular recurrent LSTM LM.
ReasoNet: Learning to Stop Reading in Machine Comprehension
Authors: Yelong Shen, Po-Sen Huang, Jianfeng Gao, Weizhu Chen
Date: 2016-09-17 00:00:00
Tasks: Question Answering, Reading Comprehension, Reinforcement Learning

Teaching a computer to read and answer general questions pertaining to a
document is a challenging yet unsolved problem. In this paper, we describe a
novel neural network architecture called the Reasoning Network (ReasoNet) for
machine comprehension tasks. ReasoNets make use of multiple turns to
effectively exploit and then reason over the relation among queries, documents,
and answers. Different from previous approaches using a fixed number of turns
during inference, ReasoNets introduce a termination state to relax this
constraint on the reasoning depth. With the use of reinforcement learning,
ReasoNets can dynamically determine whether to continue the comprehension
process after digesting intermediate results, or to terminate reading when it
concludes that existing information is adequate to produce an answer. ReasoNets
have achieved exceptional performance in machine comprehension datasets,
including unstructured CNN and Daily Mail datasets, the Stanford SQuAD dataset,
and a structured Graph Reachability dataset.
A Bayesian algorithm for detecting identity matches and fraud in image databases
Authors: Gaurav Thakur
Date: 2017-06-20 00:00:00

A statistical algorithm for categorizing different types of matches and fraud
in image databases is presented. The approach is based on a generative model of
a graph representing images and connections between pairs of identities,
trained using properties of a matching algorithm between images.
Dualing GANs
Authors: Yujia Li, Alexander Schwing, Kuan-Chieh Wang, Richard Zemel
Date: 2017-06-19 00:00:00
Proceeding: NeurIPS 2017 12

Generative adversarial nets (GANs) are a promising technique for modeling a
distribution from samples. It is however well known that GAN training suffers
from instability due to the nature of its maximin formulation. In this paper,
we explore ways to tackle the instability problem by dualizing the
discriminator. We start from linear discriminators in which case conjugate
duality provides a mechanism to reformulate the saddle point objective into a
maximization problem, such that both the generator and the discriminator of
this 'dualing GAN' act in concert. We then demonstrate how to extend this
intuition to non-linear formulations. For GANs with linear discriminators our
approach is able to remove the instability in training, while for GANs with
nonlinear discriminators our approach provides an alternative to the commonly
used GAN training algorithm.
The quality of priority ratios estimation in relation to a selected prioritization procedure and consistency measure for a Pairwise Comparison Matrix
Authors: Paul Thaddeus Kazibudzki
Date: 2017-04-06 00:00:00
Tasks: Decision Making

An overview of current debates and contemporary research devoted to the
modeling of decision making processes and their facilitation directs attention
to the Analytic Hierarchy Process (AHP). At the core of the AHP are various
prioritization procedures (PPs) and consistency measures (CMs) for a Pairwise
Comparison Matrix (PCM) which, in a sense, reflects preferences of decision
makers. Certainly, when judgments about these preferences are perfectly
consistent (cardinally transitive), all PPs coincide and the quality of the
priority ratios (PRs) estimation is exemplary. However, human judgments are
very rarely consistent, thus the quality of PRs estimation may significantly
vary. The scale of these variations depends on the applied PP and utilized CM
for a PCM. This is why it is important to find out which PPs and which CMs for
a PCM lead directly to an improvement of the PRs estimation accuracy. The main
goal of this research is realized through the properly designed, coded and
executed seminal and sophisticated simulation algorithms in Wolfram Mathematica
8.0. These research results convince that the embedded in the AHP and commonly
applied, both genuine PP and CM for PCM may significantly deteriorate the
quality of PRs estimation; however, solutions proposed in this paper can
significantly improve the methodology.
Using deep learning to reveal the neural code for images in primary visual cortex
Authors: William F. Kindel, Elijah D. Christensen, Joel Zylberberg
Date: 2017-06-19 00:00:00

Primary visual cortex (V1) is the first stage of cortical image processing,
and a major effort in systems neuroscience is devoted to understanding how it
encodes information about visual stimuli. Within V1, many neurons respond
selectively to edges of a given preferred orientation: these are known as
simple or complex cells, and they are well-studied. Other neurons respond to
localized center-surround image features. Still others respond selectively to
certain image stimuli, but the specific features that excite them are unknown.
Moreover, even for the simple and complex cells-- the best-understood V1
neurons-- it is challenging to predict how they will respond to natural image
stimuli. Thus, there are important gaps in our understanding of how V1 encodes
images. To fill this gap, we train deep convolutional neural networks to
predict the firing rates of V1 neurons in response to natural image stimuli,
and find that 15% of these neurons are within 10% of their theoretical limit of
predictability. For these well predicted neurons, we invert the predictor
network to identify the image features (receptive fields) that cause the V1
neurons to spike. In addition to those with previously-characterized receptive
fields (Gabor wavelet and center-surround), we identify neurons that respond
predictably to higher-level textural image features that are not localized to
any particular region of the image.
Prototypical Networks for Few-shot Learning
Authors: Jake Snell, Kevin Swersky, Richard S. Zemel
Date: 2017-03-15 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Category-Agnostic Pose Estimation, Few-Shot Image Classification, Few-Shot Learning, General Classification, Inductive Bias, Meta-Learning, One-Shot Learning, Zero-Shot Learning

We propose prototypical networks for the problem of few-shot classification,
where a classifier must generalize to new classes not seen in the training set,
given only a small number of examples of each new class. Prototypical networks
learn a metric space in which classification can be performed by computing
distances to prototype representations of each class. Compared to recent
approaches for few-shot learning, they reflect a simpler inductive bias that is
beneficial in this limited-data regime, and achieve excellent results. We
provide an analysis showing that some simple design decisions can yield
substantial improvements over recent approaches involving complicated
architectural choices and meta-learning. We further extend prototypical
networks to zero-shot learning and achieve state-of-the-art results on the
CU-Birds dataset.
Multi-Target Tracking in Multiple Non-Overlapping Cameras using Constrained Dominant Sets
Authors: Yonatan Tariku Tesfaye, Eyasu Zemene, Andrea Prati, Marcello Pelillo, Mubarak Shah
Date: 2017-06-19 00:00:00
Tasks: Clustering

In this paper, a unified three-layer hierarchical approach for solving
tracking problems in multiple non-overlapping cameras is proposed. Given a
video and a set of detections (obtained by any person detector), we first solve
within-camera tracking employing the first two layers of our framework and,
then, in the third layer, we solve across-camera tracking by merging tracks of
the same person in all cameras in a simultaneous fashion. To best serve our
purpose, a constrained dominant sets clustering (CDSC) technique, a
parametrized version of standard quadratic optimization, is employed to solve
both tracking tasks. The tracking problem is caste as finding constrained
dominant sets from a graph. In addition to having a unified framework that
simultaneously solves within- and across-camera tracking, the third layer helps
link broken tracks of the same person occurring during within-camera tracking.
In this work, we propose a fast algorithm, based on dynamics from evolutionary
game theory, which is efficient and salable to large-scale real-world
applications.
Unsure When to Stop? Ask Your Semantic Neighbors
Authors: Ivo Gonçalves, Sara Silva, Carlos M. Fonseca, Mauro Castelli
Date: 2017-06-19 00:00:00

In iterative supervised learning algorithms it is common to reach a point in
the search where no further induction seems to be possible with the available
data. If the search is continued beyond this point, the risk of overfitting
increases significantly. Following the recent developments in inductive
semantic stochastic methods, this paper studies the feasibility of using
information gathered from the semantic neighborhood to decide when to stop the
search. Two semantic stopping criteria are proposed and experimentally assessed
in Geometric Semantic Genetic Programming (GSGP) and in the Semantic Learning
Machine (SLM) algorithm (the equivalent algorithm for neural networks). The
experiments are performed on real-world high-dimensional regression datasets.
The results show that the proposed semantic stopping criteria are able to
detect stopping points that result in a competitive generalization for both
GSGP and SLM. This approach also yields computationally efficient algorithms as
it allows the evolution of neural networks in less than 3 seconds on average,
and of GP trees in at most 10 seconds. The usage of the proposed semantic
stopping criteria in conjunction with the computation of optimal
mutation/learning steps also results in small trees and neural networks.