pwc-archive/papers-with-abstracts huggingface dataset:
Experimental Design for Non-Parametric Correction of Misspecified Dynamical Models
Authors: Gal Shulkind, Lior Horesh, Haim Avron
Date: 2017-05-02 00:00:00
Tasks: Experimental Design

We consider a class of misspecified dynamical models where the governing term
is only approximately known. Under the assumption that observations of the
system's evolution are accessible for various initial conditions, our goal is
to infer a non-parametric correction to the misspecified driving term such as
to faithfully represent the system dynamics and devise system evolution
predictions for unobserved initial conditions.
  We model the unknown correction term as a Gaussian Process and analyze the
problem of efficient experimental design to find an optimal correction term
under constraints such as a limited experimental budget. We suggest a novel
formulation for experimental design for this Gaussian Process and show that
approximately optimal (up to a constant factor) designs may be efficiently
derived by utilizing results from the literature on submodular optimization.
Our numerical experiments exemplify the effectiveness of these techniques.
Are Key-Foreign Key Joins Safe to Avoid when Learning High-Capacity Classifiers?
Authors: Vraj Shah, Arun Kumar, Xiaojin Zhu
Date: 2017-04-03 00:00:00
Tasks: Management

Machine learning (ML) over relational data is a booming area of the database
industry and academia. While several projects aim to build scalable and fast ML
systems, little work has addressed the pains of sourcing data and features for
ML tasks. Real-world relational databases typically have many tables (often,
dozens) and data scientists often struggle to even obtain and join all possible
tables that provide features for ML. In this context, Kumar et al. showed
recently that key-foreign key dependencies (KFKDs) between tables often lets us
avoid such joins without significantly affecting prediction accuracy--an idea
they called avoiding joins safely. While initially controversial, this idea has
since been used by multiple companies to reduce the burden of data sourcing for
ML. But their work applied only to linear classifiers. In this work, we verify
if their results hold for three popular complex classifiers: decision trees,
SVMs, and ANNs. We conduct an extensive experimental study using both
real-world datasets and simulations to analyze the effects of avoiding KFK
joins on such models. Our results show that these high-capacity classifiers are
surprisingly and counter-intuitively more robust to avoiding KFK joins compared
to linear classifiers, refuting an intuition from the prior work's analysis. We
explain this behavior intuitively and identify open questions at the
intersection of data management and ML theoretical research. All of our code
and datasets are available for download from
http://cseweb.ucsd.edu/~arunkk/hamlet.
A Random-Fern based Feature Approach for Image Matching
Authors: Yong Khoo, Seo-hyeon Keun
Date: 2017-06-04 00:00:00
Tasks: General Classification, Information Retrieval, Object Recognition, Retrieval

Image or object recognition is an important task in computer vision. With the
hight-speed processing power on modern platforms and the availability of mobile
phones everywhere, millions of photos are uploaded to the internet per minute,
it is critical to establish a generic framework for fast and accurate image
processing for automatic recognition and information retrieval. In this paper,
we proposed an efficient image recognition and matching method that is
originally derived from Naive Bayesian classification method to construct a
probabilistic model. Our method support real-time performance and have very
high ability to distinguish similar images with high details. Experiments are
conducted together with intensive comparison with state-of-the-arts on image
matching, such as Ferns recognition and SIFT recognition. The results
demonstrate satisfactory performance.
InfiniteBoost: building infinite ensembles with gradient descent
Authors: Alex Rogozhnikov, Tatiana Likhomanenko
Date: 2017-06-04 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, regression

In machine learning ensemble methods have demonstrated high accuracy for the
variety of problems in different areas. Two notable ensemble methods widely
used in practice are gradient boosting and random forests. In this paper we
present InfiniteBoost - a novel algorithm, which combines important properties
of these two approaches. The algorithm constructs the ensemble of trees for
which two properties hold: trees of the ensemble incorporate the mistakes done
by others; at the same time the ensemble could contain the infinite number of
trees without the over-fitting effect. The proposed algorithm is evaluated on
the regression, classification, and ranking tasks using large scale, publicly
available datasets.
KATE: K-Competitive Autoencoder for Text
Authors: Yu Chen, Mohammed J. Zaki
Date: 2017-05-04 00:00:00
Tasks: Document Classification, Retrieval, Topic Models

Autoencoders have been successful in learning meaningful representations from
image datasets. However, their performance on text datasets has not been widely
studied. Traditional autoencoders tend to learn possibly trivial
representations of text documents due to their confounding properties such as
high-dimensionality, sparsity and power-law word distributions. In this paper,
we propose a novel k-competitive autoencoder, called KATE, for text documents.
Due to the competition between the neurons in the hidden layer, each neuron
becomes specialized in recognizing specific data patterns, and overall the
model can learn meaningful representations of textual data. A comprehensive set
of experiments show that KATE can learn better representations than traditional
autoencoders including denoising, contractive, variational, and k-sparse
autoencoders. Our model also outperforms deep generative models, probabilistic
topic models, and even word representation models (e.g., Word2Vec) in terms of
several downstream tasks such as document classification, regression, and
retrieval.
Fast Change Point Detection on Dynamic Social Networks
Authors: Yu Wang, Aniket Chakrabarti, David Sivakoff, Srinivasan Parthasarathy
Date: 2017-05-20 00:00:00
Tasks: Change Point Detection, Sociology

A number of real world problems in many domains (e.g. sociology, biology,
political science and communication networks) can be modeled as dynamic
networks with nodes representing entities of interest and edges representing
interactions among the entities at different points in time. A common
representation for such models is the snapshot model - where a network is
defined at logical time-stamps. An important problem under this model is change
point detection. In this work we devise an effective and efficient
three-step-approach for detecting change points in dynamic networks under the
snapshot model. Our algorithm achieves up to 9X speedup over the
state-of-the-art while improving quality on both synthetic and real world
networks.
Data preprocessing methods for robust Fourier ptychographic microscopy
Authors: Yan Zhang, An Pan, Ming Lei, Baoli Yao
Date: 2017-06-04 00:00:00

Fourier ptychographic microscopy (FPM) is a recently proposed computational
imaging technique with both high resolution and wide field-of-view. In current
FP experimental setup, the dark-field images with high-angle illuminations are
easily submerged by stray light and background noise due to the low
signal-to-noise ratio, thus significantly degrading the reconstruction quality
and also imposing a major restriction on the synthetic numerical aperture (NA)
of the FP approach. To this end, an overall and systematic data preprocessing
scheme for noise removal from FP's raw dataset is provided, which involves
sampling analysis as well as underexposed/overexposed treatments, then followed
by the elimination of unknown stray light and suppression of inevitable
background noise, especially Gaussian noise and CCD dark current in our
experiments. The reported non-parametric scheme facilitates great enhancements
of the FP's performance, which has been demonstrated experimentally that the
benefits of noise removal by these methods far outweigh its defects of
concomitant signal loss. In addition, it could be flexibly cooperated with the
existing state-of-the-art algorithms, producing a stronger robustness of the FP
approach in various applications.
Fast Rates for Empirical Risk Minimization of Strict Saddle Problems
Authors: Alon Gonen, Shai Shalev-Shwartz
Date: 2017-01-16 00:00:00

We derive bounds on the sample complexity of empirical risk minimization
(ERM) in the context of minimizing non-convex risks that admit the strict
saddle property. Recent progress in non-convex optimization has yielded
efficient algorithms for minimizing such functions. Our results imply that
these efficient algorithms are statistically stable and also generalize well.
In particular, we derive fast rates which resemble the bounds that are often
attained in the strongly convex setting. We specify our bounds to Principal
Component Analysis and Independent Component Analysis. Our results and
techniques may pave the way for statistical analyses of additional strict
saddle problems.
Guaranteed Sufficient Decrease for Variance Reduced Stochastic Gradient Descent
Authors: Fanhua Shang, Yuanyuan Liu, James Cheng, Kelvin Kai Wing Ng, Yuichi Yoshida
Date: 2017-03-20 00:00:00
Tasks: Stochastic Optimization

In this paper, we propose a novel sufficient decrease technique for variance
reduced stochastic gradient descent methods such as SAG, SVRG and SAGA. In
order to make sufficient decrease for stochastic optimization, we design a new
sufficient decrease criterion, which yields sufficient decrease versions of
variance reduction algorithms such as SVRG-SD and SAGA-SD as a byproduct. We
introduce a coefficient to scale current iterate and satisfy the sufficient
decrease property, which takes the decisions to shrink, expand or move in the
opposite direction, and then give two specific update rules of the coefficient
for Lasso and ridge regression. Moreover, we analyze the convergence properties
of our algorithms for strongly convex problems, which show that both of our
algorithms attain linear convergence rates. We also provide the convergence
guarantees of our algorithms for non-strongly convex problems. Our experimental
results further verify that our algorithms achieve significantly better
performance than their counterparts.
RRV: A Spatiotemporal Descriptor for Rigid Body Motion Recognition
Authors: Yao Guo, Youfu Li, Zhanpeng Shao
Date: 2016-06-18 00:00:00
Tasks: Descriptive, Position

Motion behaviors of a rigid body can be characterized by a 6-dimensional
motion trajectory, which contains position vectors of a reference point on the
rigid body and rotations of this rigid body over time. This paper devises a
Rotation and Relative Velocity (RRV) descriptor by exploring the local
translational and rotational invariants of motion trajectories of rigid bodies,
which is insensitive to noise, invariant to rigid transformation and scaling. A
flexible metric is also introduced to measure the distance between two RRV
descriptors. The RRV descriptor is then applied to characterize motions of a
human body skeleton modeled as articulated interconnections of multiple rigid
bodies. To illustrate the descriptive ability of the RRV descriptor, we explore
it for different rigid body motion recognition tasks. The experimental results
on benchmark datasets demonstrate that this simple RRV descriptor outperforms
the previous ones regarding recognition accuracy without increasing
computational cost.
Nearly Optimal Sampling Algorithms for Combinatorial Pure Exploration
Authors: Lijie Chen, Anupam Gupta, Jian Li, Mingda Qiao, Ruosong Wang
Date: 2017-06-04 00:00:00
Tasks: Multi-Armed Bandits

We study the combinatorial pure exploration problem Best-Set in stochastic
multi-armed bandits. In a Best-Set instance, we are given $n$ arms with unknown
reward distributions, as well as a family $\mathcal{F}$ of feasible subsets
over the arms. Our goal is to identify the feasible subset in $\mathcal{F}$
with the maximum total mean using as few samples as possible. The problem
generalizes the classical best arm identification problem and the top-$k$ arm
identification problem, both of which have attracted significant attention in
recent years. We provide a novel instance-wise lower bound for the sample
complexity of the problem, as well as a nontrivial sampling algorithm, matching
the lower bound up to a factor of $\ln|\mathcal{F}|$. For an important class of
combinatorial families, we also provide polynomial time implementation of the
sampling algorithm, using the equivalence of separation and optimization for
convex program, and approximate Pareto curves in multi-objective optimization.
We also show that the $\ln|\mathcal{F}|$ factor is inevitable in general
through a nontrivial lower bound construction. Our results significantly
improve several previous results for several important combinatorial
constraints, and provide a tighter understanding of the general Best-Set
problem.
  We further introduce an even more general problem, formulated in geometric
terms. We are given $n$ Gaussian arms with unknown means and unit variance.
Consider the $n$-dimensional Euclidean space $\mathbb{R}^n$, and a collection
$\mathcal{O}$ of disjoint subsets. Our goal is to determine the subset in
$\mathcal{O}$ that contains the $n$-dimensional vector of the means. The
problem generalizes most pure exploration bandit problems studied in the
literature. We provide the first nearly optimal sample complexity upper and
lower bounds for the problem.
Actor-Critic for Linearly-Solvable Continuous MDP with Partially Known Dynamics
Authors: Tomoki Nishi, Prashant Doshi, Michael R. James, Danil Prokhorov
Date: 2017-06-04 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

In many robotic applications, some aspects of the system dynamics can be
modeled accurately while others are difficult to obtain or model. We present a
novel reinforcement learning (RL) method for continuous state and action spaces
that learns with partial knowledge of the system and without active
exploration. It solves linearly-solvable Markov decision processes (L-MDPs),
which are well suited for continuous state and action spaces, based on an
actor-critic architecture. Compared to previous RL methods for L-MDPs and path
integral methods which are model based, the actor-critic learning does not need
a model of the uncontrolled dynamics and, importantly, transition noise levels;
however, it requires knowing the control dynamics for the problem. We evaluate
our method on two synthetic test problems, and one real-world problem in
simulation and using real traffic data. Our experiments demonstrate improved
learning and policy performance.
CRNN: A Joint Neural Network for Redundancy Detection
Authors: Xinyu Fu, Eugene Ch'ng, Uwe Aickelin, Simon See
Date: 2017-06-04 00:00:00
Tasks: Benchmarking, General Classification, Sentence, text-classification, Text Classification

This paper proposes a novel framework for detecting redundancy in supervised
sentence categorisation. Unlike traditional singleton neural network, our model
incorporates character-aware convolutional neural network (Char-CNN) with
character-aware recurrent neural network (Char-RNN) to form a convolutional
recurrent neural network (CRNN). Our model benefits from Char-CNN in that only
salient features are selected and fed into the integrated Char-RNN. Char-RNN
effectively learns long sequence semantics via sophisticated update mechanism.
We compare our framework against the state-of-the-art text classification
algorithms on four popular benchmarking corpus. For instance, our model
achieves competing precision rate, recall ratio, and F1 score on the
Google-news data-set. For twenty-news-groups data stream, our algorithm obtains
the optimum on precision rate, recall ratio, and F1 score. For Brown Corpus,
our framework obtains the best F1 score and almost equivalent precision rate
and recall ratio over the top competitor. For the question classification
collection, CRNN produces the optimal recall rate and F1 score and comparable
precision rate. We also analyse three different RNN hidden recurrent cells'
impact on performance and their runtime efficiency. We observe that MGU
achieves the optimal runtime and comparable performance against GRU and LSTM.
For TFIDF based algorithms, we experiment with word2vec, GloVe, and sent2vec
embeddings and report their performance differences.
Face R-CNN
Authors: Hao Wang, Zhifeng Li, Xing Ji, Yitong Wang
Date: 2017-06-04 00:00:00
Tasks: Face Detection, object-detection, Object Detection

Faster R-CNN is one of the most representative and successful methods for
object detection, and has been becoming increasingly popular in various
objection detection applications. In this report, we propose a robust deep face
detection approach based on Faster R-CNN. In our approach, we exploit several
new techniques including new multi-task loss function design, online hard
example mining, and multi-scale training strategy to improve Faster R-CNN in
multiple aspects. The proposed approach is well suited for face detection, so
we call it Face R-CNN. Extensive experiments are conducted on two most popular
and challenging face detection benchmarks, FDDB and WIDER FACE, to demonstrate
the superiority of the proposed approach over state-of-the-arts.
A Survey on Deep Learning in Medical Image Analysis
Authors: Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen A. W. M. van der Laak, Bram van Ginneken, Clara I. Sánchez
Date: 2017-02-19 00:00:00
Tasks: Deep Learning, General Classification, image-classification, Image Classification, Medical Image Analysis, object-detection, Object Detection, Survey

Deep learning algorithms, in particular convolutional networks, have rapidly
become a methodology of choice for analyzing medical images. This paper reviews
the major deep learning concepts pertinent to medical image analysis and
summarizes over 300 contributions to the field, most of which appeared in the
last year. We survey the use of deep learning for image classification, object
detection, segmentation, registration, and other tasks and provide concise
overviews of studies per application area. Open challenges and directions for
future research are discussed.
Brain Intelligence: Go Beyond Artificial Intelligence
Authors: Huimin Lu, Yujie Li, Min Chen, Hyoungseop Kim, Seiichi Serikawa
Date: 2017-06-04 00:00:00
Tasks: Artificial Life, Industrial Robots

Artificial intelligence (AI) is an important technology that supports daily
social life and economic activities. It contributes greatly to the sustainable
growth of Japan's economy and solves various social problems. In recent years,
AI has attracted attention as a key for growth in developed countries such as
Europe and the United States and developing countries such as China and India.
The attention has been focused mainly on developing new artificial intelligence
information communication technology (ICT) and robot technology (RT). Although
recently developed AI technology certainly excels in extracting certain
patterns, there are many limitations. Most ICT models are overly dependent on
big data, lack a self-idea function, and are complicated. In this paper, rather
than merely developing next-generation artificial intelligence technology, we
aim to develop a new concept of general-purpose intelligence cognition
technology called Beyond AI. Specifically, we plan to develop an intelligent
learning model called Brain Intelligence (BI) that generates new ideas about
events without having experienced them by using artificial life with an imagine
function. We will also conduct demonstrations of the developed BI intelligence
learning model on automatic driving, precision medical care, and industrial
robots.
Personalized Age Progression with Bi-level Aging Dictionary Learning
Authors: Xiangbo Shu, Jinhui Tang, Zechao Li, Hanjiang Lai, Liyan Zhang, Shuicheng Yan
Date: 2017-06-04 00:00:00
Tasks: Dictionary Learning, Face Verification

Age progression is defined as aesthetically re-rendering the aging face at
any future age for an individual face. In this work, we aim to automatically
render aging faces in a personalized way. Basically, for each age group, we
learn an aging dictionary to reveal its aging characteristics (e.g., wrinkles),
where the dictionary bases corresponding to the same index yet from two
neighboring aging dictionaries form a particular aging pattern cross these two
age groups, and a linear combination of all these patterns expresses a
particular personalized aging process. Moreover, two factors are taken into
consideration in the dictionary learning process. First, beyond the aging
dictionaries, each person may have extra personalized facial characteristics,
e.g. mole, which are invariant in the aging process. Second, it is challenging
or even impossible to collect faces of all age groups for a particular person,
yet much easier and more practical to get face pairs from neighboring age
groups. To this end, we propose a novel Bi-level Dictionary Learning based
Personalized Age Progression (BDL-PAP) method. Here, bi-level dictionary
learning is formulated to learn the aging dictionaries based on face pairs from
neighboring age groups. Extensive experiments well demonstrate the advantages
of the proposed BDL-PAP over other state-of-the-arts in term of personalized
age progression, as well as the performance gain for cross-age face
verification by synthesizing aging faces.
Adaptive Multiple-Arm Identification
Authors: Jiecao Chen, Xi Chen, Qin Zhang, Yuan Zhou
Date: 2017-06-04 00:00:00
Proceeding: ICML 2017 8

We study the problem of selecting $K$ arms with the highest expected rewards
in a stochastic $n$-armed bandit game. This problem has a wide range of
applications, e.g., A/B testing, crowdsourcing, simulation optimization. Our
goal is to develop a PAC algorithm, which, with probability at least
$1-\delta$, identifies a set of $K$ arms with the aggregate regret at most
$\epsilon$. The notion of aggregate regret for multiple-arm identification was
first introduced in \cite{Zhou:14} , which is defined as the difference of the
averaged expected rewards between the selected set of arms and the best $K$
arms. In contrast to \cite{Zhou:14} that only provides instance-independent
sample complexity, we introduce a new hardness parameter for characterizing the
difficulty of any given instance. We further develop two algorithms and
establish the corresponding sample complexity in terms of this hardness
parameter. The derived sample complexity can be significantly smaller than
state-of-the-art results for a large class of instances and matches the
instance-independent lower bound upto a $\log(\epsilon^{-1})$ factor in the
worst case. We also prove a lower bound result showing that the extra
$\log(\epsilon^{-1})$ is necessary for instance-dependent algorithms using the
introduced hardness parameter.