pwc-archive/papers-with-abstracts huggingface dataset:
Optimal rates for the regularized learning algorithms under general source condition
Authors: Abhishake Rastogi, Sivananthan Sampath
Date: 2016-11-07 00:00:00

We consider the learning algorithms under general source condition with the
polynomial decay of the eigenvalues of the integral operator in vector-valued
function setting. We discuss the upper convergence rates of Tikhonov
regularizer under general source condition corresponding to increasing monotone
index function. The convergence issues are studied for general regularization
schemes by using the concept of operator monotone index functions in minimax
setting. Further we also address the minimum possible error for any learning
algorithm.
Neurogenesis Deep Learning
Authors: Timothy J. Draelos, Nadine E. Miner, Christopher C. Lamb, Jonathan A. Cox, Craig M. Vineyard, Kristofor D. Carlson, William M. Severa, Conrad D. James, James B. Aimone
Date: 2016-12-12 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Hippocampus

Neural machine learning methods, such as deep neural networks (DNN), have
achieved remarkable success in a number of complex data processing tasks. These
methods have arguably had their strongest impact on tasks such as image and
audio processing - data processing domains in which humans have long held clear
advantages over conventional algorithms. In contrast to biological neural
systems, which are capable of learning continuously, deep artificial networks
have a limited ability for incorporating new information in an already trained
network. As a result, methods for continuous learning are potentially highly
impactful in enabling the application of deep networks to dynamic data sets.
Here, inspired by the process of adult neurogenesis in the hippocampus, we
explore the potential for adding new neurons to deep layers of artificial
neural networks in order to facilitate their acquisition of novel information
while preserving previously trained data representations. Our results on the
MNIST handwritten digit dataset and the NIST SD 19 dataset, which includes
lower and upper case letters and digits, demonstrate that neurogenesis is well
suited for addressing the stability-plasticity dilemma that has long challenged
adaptive machine learning algorithms.
Forecasting Interactive Dynamics of Pedestrians with Fictitious Play
Authors: Wei-Chiu Ma, De-An Huang, Namhoon Lee, Kris M. Kitani
Date: 2016-04-05 00:00:00
Proceeding: CVPR 2017 7
Tasks: Decision Making

We develop predictive models of pedestrian dynamics by encoding the coupled
nature of multi-pedestrian interaction using game theory, and deep
learning-based visual analysis to estimate person-specific behavior parameters.
Building predictive models for multi-pedestrian interactions however, is very
challenging due to two reasons: (1) the dynamics of interaction are complex
interdependent processes, where the predicted behavior of one pedestrian can
affect the actions taken by others and (2) dynamics are variable depending on
an individuals physical characteristics (e.g., an older person may walk slowly
while the younger person may walk faster). To address these challenges, we (1)
utilize concepts from game theory to model the interdependent decision making
process of multiple pedestrians and (2) use visual classifiers to learn a
mapping from pedestrian appearance to behavior parameters. We evaluate our
proposed model on several public multiple pedestrian interaction video
datasets. Results show that our strategic planning model explains human
interactions 25% better when compared to state-of-the-art methods.
Side Information in Robust Principal Component Analysis: Algorithms and Applications
Authors: Niannan Xue, Yannis Panagakis, Stefanos Zafeiriou
Date: 2017-02-02 00:00:00
Proceeding: ICCV 2017 10
Tasks: Denoising, Facial Expression Recognition, Facial Expression Recognition (FER), Image Denoising

Robust Principal Component Analysis (RPCA) aims at recovering a low-rank
subspace from grossly corrupted high-dimensional (often visual) data and is a
cornerstone in many machine learning and computer vision applications. Even
though RPCA has been shown to be very successful in solving many rank
minimisation problems, there are still cases where degenerate or suboptimal
solutions are obtained. This is likely to be remedied by taking into account of
domain-dependent prior knowledge. In this paper, we propose two models for the
RPCA problem with the aid of side information on the low-rank structure of the
data. The versatility of the proposed methods is demonstrated by applying them
to four applications, namely background subtraction, facial image denoising,
face and facial expression recognition. Experimental results on synthetic and
five real world datasets indicate the robustness and effectiveness of the
proposed methods on these application domains, largely outperforming six
previous approaches.
Hybrid Clustering based on Content and Connection Structure using Joint Nonnegative Matrix Factorization
Authors: Rundong Du, Barry Drake, Haesun Park
Date: 2017-03-28 00:00:00
Tasks: Clustering, Graph Clustering, Text Clustering

We present a hybrid method for latent information discovery on the data sets
containing both text content and connection structure based on constrained low
rank approximation. The new method jointly optimizes the Nonnegative Matrix
Factorization (NMF) objective function for text clustering and the Symmetric
NMF (SymNMF) objective function for graph clustering. We propose an effective
algorithm for the joint NMF objective function, based on a block coordinate
descent (BCD) framework. The proposed hybrid method discovers content
associations via latent connections found using SymNMF. The method can also be
applied with a natural conversion of the problem when a hypergraph formulation
is used or the content is associated with hypergraph edges.
  Experimental results show that by simultaneously utilizing both content and
connection structure, our hybrid method produces higher quality clustering
results compared to the other NMF clustering methods that uses content alone
(standard NMF) or connection structure alone (SymNMF). We also present some
interesting applications to several types of real world data such as citation
recommendations of papers. The hybrid method proposed in this paper can also be
applied to general data expressed with both feature space vectors and pairwise
similarities and can be extended to the case with multiple feature spaces or
multiple similarity measures.
Algebraic Variety Models for High-Rank Matrix Completion
Authors: Greg Ongie, Rebecca Willett, Robert D. Nowak, Laura Balzano
Date: 2017-03-28 00:00:00
Proceeding: ICML 2017 8
Tasks: Clustering, Low-Rank Matrix Completion, Matrix Completion, Vocal Bursts Intensity Prediction

We consider a generalization of low-rank matrix completion to the case where
the data belongs to an algebraic variety, i.e. each data point is a solution to
a system of polynomial equations. In this case the original matrix is possibly
high-rank, but it becomes low-rank after mapping each column to a higher
dimensional space of monomial features. Many well-studied extensions of linear
models, including affine subspaces and their union, can be described by a
variety model. In addition, varieties can be used to model a richer class of
nonlinear quadratic and higher degree curves and surfaces. We study the
sampling requirements for matrix completion under a variety model with a focus
on a union of affine subspaces. We also propose an efficient matrix completion
algorithm that minimizes a convex or non-convex surrogate of the rank of the
matrix of monomial features. Our algorithm uses the well-known "kernel trick"
to avoid working directly with the high-dimensional monomial matrix. We show
the proposed algorithm is able to recover synthetically generated data up to
the predicted sampling complexity bounds. The proposed algorithm also
outperforms standard low rank matrix completion and subspace clustering
techniques in experiments with real data.
Linguistic Matrix Theory
Authors: Dimitrios Kartsaklis, Sanjaye Ramgoolam, Mehrnoosh Sadrzadeh
Date: 2017-03-28 00:00:00

Recent research in computational linguistics has developed algorithms which
associate matrices with adjectives and verbs, based on the distribution of
words in a corpus of text. These matrices are linear operators on a vector
space of context words. They are used to construct the meaning of composite
expressions from that of the elementary constituents, forming part of a
compositional distributional approach to semantics. We propose a Matrix Theory
approach to this data, based on permutation symmetry along with Gaussian
weights and their perturbations. A simple Gaussian model is tested against word
matrices created from a large corpus of text. We characterize the cubic and
quartic departures from the model, which we propose, alongside the Gaussian
parameters, as signatures for comparison of linguistic corpora. We propose that
perturbed Gaussian models with permutation symmetry provide a promising
framework for characterizing the nature of universality in the statistical
properties of word matrices. The matrix theory framework developed here
exploits the view of statistics as zero dimensional perturbative quantum field
theory. It perceives language as a physical system realizing a universality
class of matrix statistics characterized by permutation symmetry.
Universal Reasoning, Rational Argumentation and Human-Machine Interaction
Authors: Christoph Benzmüller
Date: 2017-03-28 00:00:00

Classical higher-order logic, when utilized as a meta-logic in which various
other (classical and non-classical) logics can be shallowly embedded, is well
suited for realising a universal logic reasoning approach. Universal logic
reasoning in turn, as envisioned already by Leibniz, may support the rigorous
formalisation and deep logical analysis of rational arguments within machines.
A respective universal logic reasoning framework is described and a range of
exemplary applications are discussed. In the future, universal logic reasoning
in combination with appropriate, controlled forms of rational argumentation may
serve as a communication layer between humans and intelligent machines.
Learning Deep Representations Using Convolutional Auto-encoders with Symmetric Skip Connections
Authors: Jianfeng Dong, Xiao-Jiao Mao, Chunhua Shen, Yu-Bin Yang
Date: 2016-11-28 00:00:00
Tasks: Denoising, General Classification, image-classification, Image Classification, Image Reconstruction, Semantic Segmentation, Unsupervised Pre-training

Unsupervised pre-training was a critical technique for training deep neural
networks years ago. With sufficient labeled data and modern training
techniques, it is possible to train very deep neural networks from scratch in a
purely supervised manner nowadays. However, unlabeled data is easier to obtain
and usually of very large scale. How to make use of them better to help
supervised learning is still a well-valued topic. In this paper, we investigate
convolutional denoising auto-encoders to show that unsupervised pre-training
can still improve the performance of high-level image related tasks such as
image classification and semantic segmentation. The architecture we use is a
convolutional auto-encoder network with symmetric shortcut connections. We
empirically show that symmetric shortcut connections are very important for
learning abstract representations via image reconstruction. When no extra
unlabeled data are available, unsupervised pre-training with our network can
regularize the supervised training and therefore lead to better generalization
performance. With the help of unsupervised pre-training, our method achieves
very competitive results in image classification using very simple
all-convolution networks. When labeled data are limited but extra unlabeled
data are available, our method achieves good results in several semi-supervised
learning tasks.
Accelerating Deep Neural Network Training with Inconsistent Stochastic Gradient Descent
Authors: Linnan Wang, Yi Yang, Martin Renqiang Min, Srimat Chakradhar
Date: 2016-03-17 00:00:00

SGD is the widely adopted method to train CNN. Conceptually it approximates
the population with a randomly sampled batch; then it evenly trains batches by
conducting a gradient update on every batch in an epoch. In this paper, we
demonstrate Sampling Bias, Intrinsic Image Difference and Fixed Cycle Pseudo
Random Sampling differentiate batches in training, which then affect learning
speeds on them. Because of this, the unbiased treatment of batches involved in
SGD creates improper load balancing. To address this issue, we present
Inconsistent Stochastic Gradient Descent (ISGD) to dynamically vary training
effort according to learning statuses on batches. Specifically ISGD leverages
techniques in Statistical Process Control to identify a undertrained batch.
Once a batch is undertrained, ISGD solves a new subproblem, a chasing logic
plus a conservative constraint, to accelerate the training on the batch while
avoid drastic parameter changes. Extensive experiments on a variety of datasets
demonstrate ISGD converges faster than SGD. In training AlexNet, ISGD is
21.05\% faster than SGD to reach 56\% top1 accuracy under the exactly same
experiment setup. We also extend ISGD to work on multiGPU or heterogeneous
distributed system based on data parallelism, enabling the batch size to be the
key to scalability. Then we present the study of ISGD batch size to the
learning rate, parallelism, synchronization cost, system saturation and
scalability. We conclude the optimal ISGD batch size is machine dependent.
Various experiments on a multiGPU system validate our claim. In particular,
ISGD trains AlexNet to 56.3% top1 and 80.1% top5 accuracy in 11.5 hours with 4
NVIDIA TITAN X at the batch size of 1536.
Deeply-Supervised CNN for Prostate Segmentation
Authors: Qikui Zhu, Bo Du, Baris Turkbey, Peter L . Choyke, Pingkun Yan
Date: 2017-03-22 00:00:00
Tasks: Segmentation

Prostate segmentation from Magnetic Resonance (MR) images plays an important
role in image guided interven- tion. However, the lack of clear boundary
specifically at the apex and base, and huge variation of shape and texture
between the images from different patients make the task very challenging. To
overcome these problems, in this paper, we propose a deeply supervised
convolutional neural network (CNN) utilizing the convolutional information to
accurately segment the prostate from MR images. The proposed model can
effectively detect the prostate region with additional deeply supervised layers
compared with other approaches. Since some information will be abandoned after
convolution, it is necessary to pass the features extracted from early stages
to later stages. The experimental results show that significant segmentation
accuracy improvement has been achieved by our proposed method compared to other
reported approaches.
Important New Developments in Arabographic Optical Character Recognition (OCR)
Authors: Maxim Romanov, Matthew Thomas Miller, Sarah Bowen Savant, Benjamin Kiessling
Date: 2017-03-28 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR)

The OpenITI team has achieved Optical Character Recognition (OCR) accuracy
rates for classical Arabic-script texts in the high nineties. These numbers are
based on our tests of seven different Arabic-script texts of varying quality
and typefaces, totaling over 7,000 lines. These accuracy rates not only
represent a distinct improvement over the actual accuracy rates of the various
proprietary OCR options for classical Arabic-script texts, but, equally
important, they are produced using an open-source OCR software, thus enabling
us to make this Arabic-script OCR technology freely available to the broader
Islamic, Persian, and Arabic Studies communities.
Is This a Joke? Detecting Humor in Spanish Tweets
Authors: Santiago Castro, Matías Cubero, Diego Garat, Guillermo Moncecchi
Date: 2017-03-28 00:00:00
Tasks: Humor Detection

While humor has been historically studied from a psychological, cognitive and
linguistic standpoint, its study from a computational perspective is an area
yet to be explored in Computational Linguistics. There exist some previous
works, but a characterization of humor that allows its automatic recognition
and generation is far from being specified. In this work we build a
crowdsourced corpus of labeled tweets, annotated according to its humor value,
letting the annotators subjectively decide which are humorous. A humor
classifier for Spanish tweets is assembled based on supervised learning,
reaching a precision of 84% and a recall of 69%.
Mining Best Closed Itemsets for Projection-antimonotonic Constraints in Polynomial Time
Authors: Aleksey Buzmakov, Sergei O. Kuznetsov, Amedeo Napoli
Date: 2017-03-28 00:00:00

The exponential explosion of the set of patterns is one of the main
challenges in pattern mining. This challenge is approached by introducing a
constraint for pattern selection. One of the first constraints proposed in
pattern mining is support (frequency) of a pattern in a dataset. Frequency is
an anti-monotonic function, i.e., given an infrequent pattern, all its
superpatterns are not frequent. However, many other constraints for pattern
selection are neither monotonic nor anti-monotonic, which makes it difficult to
generate patterns satisfying these constraints.
  In order to deal with nonmonotonic constraints we introduce the notion of
"projection antimonotonicity" and SOFIA algorithm that allow generating best
patterns for a class of nonmonotonic constraints. Cosine interest, robustness,
stability of closed itemsets, and the associated delta-measure are among these
constraints. SOFIA starts from light descriptions of transactions in dataset (a
small set of items in the case of itemset description) and then iteratively
adds more information to these descriptions (more items with indication of
tidsets they describe).
Robust SfM with Little Image Overlap
Authors: Yohann Salaun, Renaud Marlet, Pascal Monasse
Date: 2017-03-23 00:00:00

Usual Structure-from-Motion (SfM) techniques require at least trifocal
overlaps to calibrate cameras and reconstruct a scene. We consider here
scenarios of reduced image sets with little overlap, possibly as low as two
images at most seeing the same part of the scene. We propose a new method,
based on line coplanarity hypotheses, for estimating the relative scale of two
independent bifocal calibrations sharing a camera, without the need of any
trifocal information or Manhattan-world assumption. We use it to compute SfM in
a chain of up-to-scale relative motions. For accuracy, we however also make use
of trifocal information for line and/or point features, when present, relaxing
usual trifocal constraints. For robustness to wrong assumptions and mismatches,
we embed all constraints in a parameterless RANSAC-like approach. Experiments
show that we can calibrate datasets that previously could not, and that this
wider applicability does not come at the cost of inaccuracy.
Simulated Data Experiments for Time Series Classification Part 1: Accuracy Comparison with Default Settings
Authors: Anthony Bagnall, Aaron Bostrom, James Large, Jason Lines
Date: 2017-03-28 00:00:00
Tasks: General Classification, Time Series, Time Series Analysis, Time Series Classification

There are now a broad range of time series classification (TSC) algorithms
designed to exploit different representations of the data. These have been
evaluated on a range of problems hosted at the UCR-UEA TSC Archive
(www.timeseriesclassification.com), and there have been extensive comparative
studies. However, our understanding of why one algorithm outperforms another is
still anecdotal at best. This series of experiments is meant to help provide
insights into what sort of discriminatory features in the data lead one set of
algorithms that exploit a particular representation to be better than other
algorithms. We categorise five different feature spaces exploited by TSC
algorithms then design data simulators to generate randomised data from each
representation. We describe what results we expected from each class of
algorithm and data representation, then observe whether these prior beliefs are
supported by the experimental evidence. We provide an open source
implementation of all the simulators to allow for the controlled testing of
hypotheses relating to classifier performance on different data
representations. We identify many surprising results that confounded our
expectations, and use these results to highlight how an over simplified view of
classifier structure can often lead to erroneous prior beliefs. We believe
ensembling can often overcome prior bias, and our results support the belief by
showing that the ensemble approach adopted by the Hierarchical Collective of
Transform based Ensembles (HIVE-COTE) is significantly better than the
alternatives when the data representation is unknown, and is significantly
better than, or not significantly significantly better than, or not
significantly worse than, the best other approach on three out of five of the
individual simulators.
A flexible state space model for learning nonlinear dynamical systems
Authors: Andreas Svensson, Thomas B. Schön
Date: 2016-03-17 00:00:00
Tasks: Gaussian Processes

We consider a nonlinear state-space model with the state transition and
observation functions expressed as basis function expansions. The coefficients
in the basis function expansions are learned from data. Using a connection to
Gaussian processes we also develop priors on the coefficients, for tuning the
model flexibility and to prevent overfitting to data, akin to a Gaussian
process state-space model. The priors can alternatively be seen as a
regularization, and helps the model in generalizing the data without
sacrificing the richness offered by the basis function expansion. To learn the
coefficients and other unknown parameters efficiently, we tailor an algorithm
using state-of-the-art sequential Monte Carlo methods, which comes with
theoretical guarantees on the learning. Our approach indicates promising
results when evaluated on a classical benchmark as well as real data.
Robust Depth-based Person Re-identification
Authors: Ancong Wu, Wei-Shi Zheng, Jian-Huang Lai
Date: 2017-03-28 00:00:00
Tasks: Person Re-Identification

Person re-identification (re-id) aims to match people across non-overlapping
camera views. So far the RGB-based appearance is widely used in most existing
works. However, when people appeared in extreme illumination or changed
clothes, the RGB appearance-based re-id methods tended to fail. To overcome
this problem, we propose to exploit depth information to provide more invariant
body shape and skeleton information regardless of illumination and color
change. More specifically, we exploit depth voxel covariance descriptor and
further propose a locally rotation invariant depth shape descriptor called
Eigen-depth feature to describe pedestrian body shape. We prove that the
distance between any two covariance matrices on the Riemannian manifold is
equivalent to the Euclidean distance between the corresponding Eigen-depth
features. Furthermore, we propose a kernelized implicit feature transfer scheme
to estimate Eigen-depth feature implicitly from RGB image when depth
information is not available. We find that combining the estimated depth
features with RGB-based appearance features can sometimes help to better reduce
visual ambiguities of appearance features caused by illumination and similar
clothes. The effectiveness of our models was validated on publicly available
depth pedestrian datasets as compared to related methods for person
re-identification.