pwc-archive/papers-with-abstracts huggingface dataset:
Learning Pixel-wise Labeling from the Internet without Human Interaction
Authors: Yun Liu, Yujun Shi, Jia-Wang Bian, Le Zhang, Ming-Ming Cheng, Jiashi Feng
Date: 2018-05-19 00:00:00
Tasks: Segmentation, Semantic Segmentation

Deep learning stands at the forefront in many computer vision tasks. However,
deep neural networks are usually data-hungry and require a huge amount of
well-annotated training samples. Collecting sufficient annotated data is very
expensive in many applications, especially for pixel-level prediction tasks
such as semantic segmentation. To solve this fundamental issue, we consider a
new challenging vision task, Internetly supervised semantic segmentation, which
only uses Internet data with noisy image-level supervision of corresponding
query keywords for segmentation model training. We address this task by
proposing the following solution. A class-specific attention model unifying
multiscale forward and backward convolutional features is proposed to provide
initial segmentation "ground truth". The model trained with such noisy
annotations is then improved by an online fine-tuning procedure. It achieves
state-of-the-art performance under the weakly-supervised setting on PASCAL
VOC2012 dataset. The proposed framework also paves a new way towards learning
from the Internet without human interaction and could serve as a strong
baseline therein. Code and data will be released upon the paper acceptance.
Autonomous discovery of the goal space to learn a parameterized skill
Authors: Emilio Cartoni, Gianluca Baldassarre
Date: 2018-05-19 00:00:00

A parameterized skill is a mapping from multiple goals/task parameters to the
policy parameters to accomplish them. Existing works in the literature show how
a parameterized skill can be learned given a task space that defines all the
possible achievable goals. In this work, we focus on tasks defined in terms of
final states (goals), and we face on the challenge where the agent aims to
autonomously acquire a parameterized skill to manipulate an initially unknown
environment. In this case, the task space is not known a priori and the agent
has to autonomously discover it. The agent may posit as a task space its whole
sensory space (i.e. the space of all possible sensor readings) as the
achievable goals will certainly be a subset of this space. However, the space
of achievable goals may be a very tiny subspace in relation to the whole
sensory space, thus directly using the sensor space as task space exposes the
agent to the curse of dimensionality and makes existing autonomous skill
acquisition algorithms inefficient. In this work we present an algorithm that
actively discovers the manifold of the achievable goals within the sensor
space. We validate the algorithm by employing it in multiple different
simulated scenarios where the agent actions achieve different types of goals:
moving a redundant arm, pushing an object, and changing the color of an object.
End-to-end driving simulation via angle branched network
Authors: Qing Wang, Long Chen, Wei Tian
Date: 2018-05-19 00:00:00
Tasks: Autonomous Driving, Imitation Learning, Navigate

Imitation learning for end-to-end autonomous driving has drawn attention from
academic communities. Current methods either only use images as the input which
is ambiguous when a car approaches an intersection, or use additional command
information to navigate the vehicle but not automated enough. Focusing on
making the vehicle drive along the given path, we propose a new navigation
command that does not require human's participation and a novel model
architecture called angle branched network. Both the new navigation command and
the angle branched network are easy to understand and effective. Besides, we
find that not only segmentation information but also depth information can
boost the performance of the driving model. We conduct experiments in a 3D
urban simulator and both qualitative and quantitative evaluation results show
the effectiveness of our model.
Learning to Multitask
Authors: Yu Zhang, Ying WEI, Qiang Yang
Date: 2018-05-19 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Graph Neural Network

Multitask learning has shown promising performance in many applications and
many multitask models have been proposed. In order to identify an effective
multitask model for a given multitask problem, we propose a learning framework
called learning to multitask (L2MT). To achieve the goal, L2MT exploits
historical multitask experience which is organized as a training set consists
of several tuples, each of which contains a multitask problem with multiple
tasks, a multitask model, and the relative test error. Based on such training
set, L2MT first uses a proposed layerwise graph neural network to learn task
embeddings for all the tasks in a multitask problem and then learns an
estimation function to estimate the relative test error based on task
embeddings and the representation of the multitask model based on a unified
formulation. Given a new multitask problem, the estimation function is used to
identify a suitable multitask model. Experiments on benchmark datasets show the
effectiveness of the proposed L2MT framework.
An optimal approximation of discrete random variables with respect to the Kolmogorov distance
Authors: Liat Cohen, Dror Fried, Gera Weiss
Date: 2018-05-19 00:00:00

We present an algorithm that takes a discrete random variable $X$ and a
number $m$ and computes a random variable whose support (set of possible
outcomes) is of size at most $m$ and whose Kolmogorov distance from $X$ is
minimal. In addition to a formal theoretical analysis of the correctness and of
the computational complexity of the algorithm, we present a detailed empirical
evaluation that shows how the proposed approach performs in practice in
different applications and domains.
Neural networks with dynamical coefficients and adjustable connections on the basis of integrated backpropagation
Authors: M. N. Nazarov
Date: 2018-05-19 00:00:00

We consider artificial neurons which will update their weight coefficients
with an internal rule based on backpropagation, rather than using it as an
external training procedure. To achieve this we include the backpropagation
error estimate as a separate entity in all the neuron models and perform its
exchange along the synaptic connections. In addition to this we add some
special type of neurons with reference inputs, which will serve as a base
source of error estimates for the whole network. Finally, we introduce a
training control signal for all the neurons, which can enable the correction of
weights and the exchange of error estimates. For recurrent neural networks we
also demonstrate how to integrate backpropagation through time into their
formalism with the help of some stack memory for reference inputs and external
data inputs of neurons. Also, for widely used neural networks, such as long
short-term memory, radial basis function networks, multilayer perceptrons and
convolutional neural networks, we demonstrate their alternative description
within the framework of our new formalism.
Two-stage quality adaptive fingerprint image enhancement using Fuzzy c-means clustering based fingerprint quality analysis
Authors: Ram Prakash Sharma, Somnath Dey
Date: 2018-05-19 00:00:00
Tasks: Clustering, Image Enhancement

Fingerprint recognition techniques are immensely dependent on quality of the
fingerprint images. To improve the performance of recognition algorithm for
poor quality images an efficient enhancement algorithm should be designed.
Performance improvement of recognition algorithm will be more if enhancement
process is adaptive to the fingerprint quality (wet, dry or normal). In this
paper, a quality adaptive fingerprint enhancement algorithm is proposed. The
proposed fingerprint quality assessment algorithm clusters the fingerprint
images in appropriate quality class of dry, wet, normal dry, normal wet and
good quality using fuzzy c-means technique. It considers seven features namely,
mean, moisture, variance, uniformity, contrast, ridge valley area uniformity
and ridge valley uniformity into account for clustering the fingerprint images
in appropriate quality class. Fingerprint images of each quality class undergo
through a two-stage fingerprint quality enhancement process. A quality adaptive
preprocessing method is used as front-end before enhancing the fingerprint
images with Gabor, short term Fourier transform and oriented diffusion
filtering based enhancement techniques. Experimental results show improvement
in the verification results for FVC2004 datasets. Significant improvement in
equal error rate is observed while using quality adaptive preprocessing based
approaches in comparison to the current state-of-the-art enhancement
techniques.
Deep Predictive Coding Network with Local Recurrent Processing for Object Recognition
Authors: Kuan Han, Haiguang Wen, Yizhen Zhang, Di Fu, Eugenio Culurciello, Zhongming Liu
Date: 2018-05-19 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: image-classification, Image Classification, Object Recognition, Prediction

Inspired by "predictive coding" - a theory in neuroscience, we develop a
bi-directional and dynamic neural network with local recurrent processing,
namely predictive coding network (PCN). Unlike feedforward-only convolutional
neural networks, PCN includes both feedback connections, which carry top-down
predictions, and feedforward connections, which carry bottom-up errors of
prediction. Feedback and feedforward connections enable adjacent layers to
interact locally and recurrently to refine representations towards minimization
of layer-wise prediction errors. When unfolded over time, the recurrent
processing gives rise to an increasingly deeper hierarchy of non-linear
transformation, allowing a shallow network to dynamically extend itself into an
arbitrarily deep network. We train and test PCN for image classification with
SVHN, CIFAR and ImageNet datasets. Despite notably fewer layers and parameters,
PCN achieves competitive performance compared to classical and state-of-the-art
models. Further analysis shows that the internal representations in PCN
converge over time and yield increasingly better accuracy in object
recognition. Errors of top-down prediction also reveal visual saliency or
bottom-up attention.
The global optimum of shallow neural network is attained by ridgelet transform
Authors: Sho Sonoda, Isao Ishikawa, Masahiro Ikeda, Kei Hagihara, Yoshihiro Sawano, Takuo Matsubara, Noboru Murata
Date: 2018-05-19 00:00:00

We prove that the global minimum of the backpropagation (BP) training problem
of neural networks with an arbitrary nonlinear activation is given by the
ridgelet transform. A series of computational experiments show that there
exists an interesting similarity between the scatter plot of hidden parameters
in a shallow neural network after the BP training and the spectrum of the
ridgelet transform. By introducing a continuous model of neural networks, we
reduce the training problem to a convex optimization in an infinite dimensional
Hilbert space, and obtain the explicit expression of the global optimizer via
the ridgelet transform.
Estimation of Non-Normalized Mixture Models and Clustering Using Deep Representation
Authors: Takeru Matsuda, Aapo Hyvarinen
Date: 2018-05-19 00:00:00
Tasks: Clustering, Image Clustering

We develop a general method for estimating a finite mixture of non-normalized
models. Here, a non-normalized model is defined to be a parametric distribution
with an intractable normalization constant. Existing methods for estimating
non-normalized models without computing the normalization constant are not
applicable to mixture models because they contain more than one intractable
normalization constant. The proposed method is derived by extending noise
contrastive estimation (NCE), which estimates non-normalized models by
discriminating between the observed data and some artificially generated noise.
We also propose an extension of NCE with multiple noise distributions. Then,
based on the observation that conventional classification learning with neural
networks is implicitly assuming an exponential family as a generative model, we
introduce a method for clustering unlabeled data by estimating a finite mixture
of distributions in an exponential family. Estimation of this mixture model is
attained by the proposed extensions of NCE where the training data of neural
networks are used as noise. Thus, the proposed method provides a
probabilistically principled clustering method that is able to utilize a deep
representation. Application to image clustering using a deep neural network
gives promising results.
Design of Data-Driven Mathematical Laws for Optimal Statistical Classification Systems
Authors: Denise M. Reeves
Date: 2016-12-12 00:00:00
Tasks: Binary Classification, Classification, General Classification

This article will devise data-driven, mathematical laws that generate
optimal, statistical classification systems which achieve minimum error rates
for data distributions with unchanging statistics. Thereby, I will design
learning machines that minimize the expected risk or probability of
misclassification. I will devise a system of fundamental equations of binary
classification for a classification system in statistical equilibrium. I will
use this system of equations to formulate the problem of learning unknown,
linear and quadratic discriminant functions from data as a locus problem,
thereby formulating geometric locus methods within a statistical framework.
Solving locus problems involves finding equations of curves or surfaces defined
by given properties and finding graphs or loci of given equations. I will
devise three systems of data-driven, locus equations that generate optimal,
statistical classification systems. Each class of learning machines satisfies
fundamental statistical laws for a classification system in statistical
equilibrium. Thereby, I will formulate three classes of learning machines that
are scalable modules for optimal, statistical pattern recognition systems, all
of which are capable of performing a wide variety of statistical pattern
recognition tasks, where any given M-class statistical pattern recognition
system exhibits optimal generalization performance for an M-class feature
space.
Diverse Few-Shot Text Classification with Multiple Metrics
Authors: Mo Yu, Xiaoxiao Guo, Jin-Feng Yi, Shiyu Chang, Saloni Potdar, Yu Cheng, Gerald Tesauro, Haoyu Wang, Bo-Wen Zhou
Date: 2018-05-19 00:00:00
Proceeding: NAACL 2018 6
Tasks: Classification, Few-Shot Learning, Few-Shot Text Classification, General Classification, intent-classification, Intent Classification, Meta-Learning, Metric Learning, Sentiment Analysis, text-classification, Text Classification

We study few-shot learning in natural language domains. Compared to many
existing works that apply either metric-based or optimization-based
meta-learning to image domain with low inter-task variance, we consider a more
realistic setting, where tasks are diverse. However, it imposes tremendous
difficulties to existing state-of-the-art metric-based algorithms since a
single metric is insufficient to capture complex task variations in natural
language domain. To alleviate the problem, we propose an adaptive metric
learning approach that automatically determines the best weighted combination
from a set of metrics obtained from meta-training tasks for a newly seen
few-shot task. Extensive quantitative evaluations on real-world sentiment
analysis and dialog intent classification datasets demonstrate that the
proposed method performs favorably against state-of-the-art few shot learning
algorithms in terms of predictive accuracy. We make our code and data available
for further study.
Cross-City Transfer Learning for Deep Spatio-Temporal Prediction
Authors: Leye Wang, Xu Geng, Xiaojuan Ma, Feng Liu, Qiang Yang
Date: 2018-02-01 00:00:00
Tasks: Prediction, Transfer Learning

Spatio-temporal prediction is a key type of tasks in urban computing, e.g.,
traffic flow and air quality. Adequate data is usually a prerequisite,
especially when deep learning is adopted. However, the development levels of
different cities are unbalanced, and still many cities suffer from data
scarcity. To address the problem, we propose a novel cross-city transfer
learning method for deep spatio-temporal prediction tasks, called RegionTrans.
RegionTrans aims to effectively transfer knowledge from a data-rich source city
to a data-scarce target city. More specifically, we first learn an inter-city
region matching function to match each target city region to a similar source
city region. A neural network is designed to effectively extract region-level
representation for spatio-temporal prediction. Finally, an optimization
algorithm is proposed to transfer learned features from the source city to the
target city with the region matching function. Using citywide crowd flow
prediction as a demonstration experiment, we verify the effectiveness of
RegionTrans. Results show that RegionTrans can outperform the state-of-the-art
fine-tuning deep spatio-temporal prediction models by reducing up to 10.7%
prediction error.
GEN Model: An Alternative Approach to Deep Neural Network Models
Authors: Jiawei Zhang, Limeng Cui, Fisher B. Gouza
Date: 2018-05-19 00:00:00
Tasks: Representation Learning

In this paper, we introduce an alternative approach, namely GEN (Genetic
Evolution Network) Model, to the deep learning models. Instead of building one
single deep model, GEN adopts a genetic-evolutionary learning strategy to build
a group of unit models generations by generations. Significantly different from
the wellknown representation learning models with extremely deep structures,
the unit models covered in GEN are of a much shallower architecture. In the
training process, from each generation, a subset of unit models will be
selected based on their performance to evolve and generate the child models in
the next generation. GEN has significant advantages compared with existing deep
representation learning models in terms of both learning effectiveness,
efficiency and interpretability of the learning process and learned results.
Extensive experiments have been done on diverse benchmark datasets, and the
experimental results have demonstrated the outstanding performance of GEN
compared with the state-of-the-art baseline methods in both effectiveness of
efficiency.
Reconciled Polynomial Machine: A Unified Representation of Shallow and Deep Learning Models
Authors: Jiawei Zhang, Limeng Cui, Fisher B. Gouza
Date: 2018-05-19 00:00:00
Tasks: BIG-bench Machine Learning

In this paper, we aim at introducing a new machine learning model, namely
reconciled polynomial machine, which can provide a unified representation of
existing shallow and deep machine learning models. Reconciled polynomial
machine predicts the output by computing the inner product of the feature
kernel function and variable reconciling function. Analysis of several concrete
models, including Linear Models, FM, MVM, Perceptron, MLP and Deep Neural
Networks, will be provided in this paper, which can all be reduced to the
reconciled polynomial machine representations. Detailed analysis of the
learning error by these models will also be illustrated in this paper based on
their reduced representations from the function approximation perspective.
Free-rider Episode Screening via Dual Partition Model
Authors: Xiang Ao, Yang Liu, Zhen Huang, Luo Zuo, Qing He
Date: 2018-05-19 00:00:00
Tasks: model

One of the drawbacks of frequent episode mining is that overwhelmingly many
of the discovered patterns are redundant. Free-rider episode, as a typical
example, consists of a real pattern doped with some additional noise events.
Because of the possible high support of the inside noise events, such
free-rider episodes may have abnormally high support that they cannot be
filtered by frequency based framework. An effective technique for filtering
free-rider episodes is using a partition model to divide an episode into two
consecutive subepisodes and comparing the observed support of such episode with
its expected support under the assumption that these two subepisodes occur
independently. In this paper, we take more complex subepisodes into
consideration and develop a novel partition model named EDP for free-rider
episode filtering from a given set of episodes. It combines (1) a dual
partition strategy which divides an episode to an underlying real pattern and
potential noises; (2) a novel definition of the expected support of a
free-rider episode based on the proposed partition strategy. We can deem the
episode interesting if the observed support is substantially higher than the
expected support estimated by our model. The experiments on synthetic and
real-world datasets demonstrate EDP can effectively filter free-rider episodes
compared with existing state-of-the-arts.
Deep Loopy Neural Network Model for Graph Structured Data Representation Learning
Authors: Jiawei Zhang
Date: 2018-05-19 00:00:00
Tasks: Deep Learning, Representation Learning

Existing deep learning models may encounter great challenges in handling graph structured data. In this paper, we introduce a new deep learning model for graph data specifically, namely the deep loopy neural network. Significantly different from the previous deep models, inside the deep loopy neural network, there exist a large number of loops created by the extensive connections among nodes in the input graph data, which makes model learning an infeasible task. To resolve such a problem, in this paper, we will introduce a new learning algorithm for the deep loopy neural network specifically. Instead of learning the model variables based on the original model, in the proposed learning algorithm, errors will be back-propagated through the edges in a group of extracted spanning trees. Extensive numerical experiments have been done on several real-world graph datasets, and the experimental results demonstrate the effectiveness of both the proposed model and the learning algorithm in handling graph data.
Sequential Multi-Class Labeling in Crowdsourcing
Authors: Qiyu Kang, Wee Peng Tay
Date: 2017-11-06 00:00:00

We consider a crowdsourcing platform where workers' responses to questions
posed by a crowdsourcer are used to determine the hidden state of a multi-class
labeling problem. As workers may be unreliable, we propose to perform
sequential questioning in which the questions posed to the workers are designed
based on previous questions and answers. We propose a Partially-Observable
Markov Decision Process (POMDP) framework to determine the best questioning
strategy, subject to the crowdsourcer's budget constraint. As this POMDP
formulation is in general intractable, we develop a suboptimal approach based
on a $q$-ary Ulam-R\'enyi game. We also propose a sampling heuristic, which can
be used in tandem with standard POMDP solvers, using our Ulam-R\'enyi strategy.
We demonstrate through simulations that our approaches outperform a
non-sequential strategy based on error correction coding and which does not
utilize workers' previous responses.
On Deep Ensemble Learning from a Function Approximation Perspective
Authors: Jiawei Zhang, Limeng Cui, Fisher B. Gouza
Date: 2018-05-19 00:00:00
Tasks: Ensemble Learning

In this paper, we propose to provide a general ensemble learning framework
based on deep learning models. Given a group of unit models, the proposed deep
ensemble learning framework will effectively combine their learning results via
a multilayered ensemble model. In the case when the unit model mathematical
mappings are bounded, sigmoidal and discriminatory, we demonstrate that the
deep ensemble learning framework can achieve a universal approximation of any
functions from the input space to the output space. Meanwhile, to achieve such
a performance, the deep ensemble learning framework also impose a strict
constraint on the number of involved unit models. According to the theoretic
proof provided in this paper, given the input feature space of dimension d, the
required unit model number will be 2d, if the ensemble model involves one
single layer. Furthermore, as the ensemble component goes deeper, the number of
required unit model is proved to be lowered down exponentially.