pwc-archive/papers-with-abstracts huggingface dataset:
On Training Bi-directional Neural Network Language Model with Noise Contrastive Estimation
Authors: Tianxing He, Yu Zhang, Jasha Droppo, Kai Yu
Date: 2016-02-19 00:00:00
Tasks: Language Modeling, Language Modelling

We propose to train bi-directional neural network language model(NNLM) with
noise contrastive estimation(NCE). Experiments are conducted on a rescore task
on the PTB data set. It is shown that NCE-trained bi-directional NNLM
outperformed the one trained by conventional maximum likelihood training. But
still(regretfully), it did not out-perform the baseline uni-directional NNLM.
Toward Mention Detection Robustness with Recurrent Neural Networks
Authors: Thien Huu Nguyen, Avirup Sil, Georgiana Dinu, Radu Florian
Date: 2016-02-24 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Word Embeddings

One of the key challenges in natural language processing (NLP) is to yield
good performance across application domains and languages. In this work, we
investigate the robustness of the mention detection systems, one of the
fundamental tasks in information extraction, via recurrent neural networks
(RNNs). The advantage of RNNs over the traditional approaches is their capacity
to capture long ranges of context and implicitly adapt the word embeddings,
trained on a large corpus, into a task-specific word representation, but still
preserve the original semantic generalization to be helpful across domains. Our
systematic evaluation for RNN architectures demonstrates that RNNs not only
outperform the best reported systems (up to 9\% relative error reduction) in
the general setting but also achieve the state-of-the-art performance in the
cross-domain setting for English. Regarding other languages, RNNs are
significantly better than the traditional methods on the similar task of named
entity recognition for Dutch (up to 22\% relative error reduction).
Recurrent Gaussian Processes
Authors: César Lincoln C. Mattos, Zhenwen Dai, Andreas Damianou, Jeremy Forth, Guilherme A. Barreto, Neil D. Lawrence
Date: 2015-11-20 00:00:00
Tasks: Gaussian Processes

We define Recurrent Gaussian Processes (RGP) models, a general family of
Bayesian nonparametric models with recurrent GP priors which are able to learn
dynamical patterns from sequential data. Similar to Recurrent Neural Networks
(RNNs), RGPs can have different formulations for their internal states,
distinct inference methods and be extended with deep structures. In such
context, we propose a novel deep RGP model whose autoregressive states are
latent, thereby performing representation and dynamical learning
simultaneously. To fully exploit the Bayesian nature of the RGP model we
develop the Recurrent Variational Bayes (REVARB) framework, which enables
efficient inference and strong regularization through coherent propagation of
uncertainty across the RGP layers and states. We also introduce a RGP extension
where variational parameters are greatly reduced by being reparametrized
through RNN-based sequential recognition models. We apply our model to the
tasks of nonlinear system identification and human motion modeling. The
promising obtained results indicate that our RGP model maintains its highly
flexibility while being able to avoid overfitting and being applicable even
when larger datasets are not available.
The Use of Machine Learning Algorithms in Recommender Systems: A Systematic Review
Authors: Ivens Portugal, Paulo Alencar, Donald Cowan
Date: 2015-11-17 00:00:00
Tasks: BIG-bench Machine Learning, Recommendation Systems

Recommender systems use algorithms to provide users with product or service
recommendations. Recently, these systems have been using machine learning
algorithms from the field of artificial intelligence. However, choosing a
suitable machine learning algorithm for a recommender system is difficult
because of the number of algorithms described in the literature. Researchers
and practitioners developing recommender systems are left with little
information about the current approaches in algorithm usage. Moreover, the
development of a recommender system using a machine learning algorithm often
has problems and open questions that must be evaluated, so software engineers
know where to focus research efforts. This paper presents a systematic review
of the literature that analyzes the use of machine learning algorithms in
recommender systems and identifies research opportunities for software
engineering research. The study concludes that Bayesian and decision tree
algorithms are widely used in recommender systems because of their relative
simplicity, and that requirement and design phases of recommender system
development appear to offer opportunities for further research.
Online Dual Coordinate Ascent Learning
Authors: Bicheng Ying, Kun Yuan, Ali H. Sayed
Date: 2016-02-24 00:00:00

The stochastic dual coordinate-ascent (S-DCA) technique is a useful
alternative to the traditional stochastic gradient-descent algorithm for
solving large-scale optimization problems due to its scalability to large data
sets and strong theoretical guarantees. However, the available S-DCA
formulation is limited to finite sample sizes and relies on performing multiple
passes over the same data. This formulation is not well-suited for online
implementations where data keep streaming in. In this work, we develop an {\em
online} dual coordinate-ascent (O-DCA) algorithm that is able to respond to
streaming data and does not need to revisit the past data. This feature embeds
the resulting construction with continuous adaptation, learning, and tracking
abilities, which are particularly attractive for online learning scenarios.
Noisy population recovery in polynomial time
Authors: Anindya De, Michael Saks, Sijian Tang
Date: 2016-02-24 00:00:00

In the noisy population recovery problem of Dvir et al., the goal is to learn
an unknown distribution $f$ on binary strings of length $n$ from noisy samples.
For some parameter $\mu \in [0,1]$, a noisy sample is generated by flipping
each coordinate of a sample from $f$ independently with probability
$(1-\mu)/2$. We assume an upper bound $k$ on the size of the support of the
distribution, and the goal is to estimate the probability of any string to
within some given error $\varepsilon$. It is known that the algorithmic
complexity and sample complexity of this problem are polynomially related to
each other.
  We show that for $\mu > 0$, the sample complexity (and hence the algorithmic
complexity) is bounded by a polynomial in $k$, $n$ and $1/\varepsilon$
improving upon the previous best result of $\mathsf{poly}(k^{\log\log
k},n,1/\varepsilon)$ due to Lovett and Zhang.
  Our proof combines ideas from Lovett and Zhang with a \emph{noise attenuated}
version of M\"{o}bius inversion. In turn, the latter crucially uses the
construction of \emph{robust local inverse} due to Moitra and Saks.
Optimally Confident UCB: Improved Regret for Finite-Armed Bandits
Authors: Tor Lattimore
Date: 2015-07-28 00:00:00

I present the first algorithm for stochastic finite-armed bandits that
simultaneously enjoys order-optimal problem-dependent regret and worst-case
regret. Besides the theoretical results, the new algorithm is simple, efficient
and empirically superb. The approach is based on UCB, but with a carefully
chosen confidence parameter that optimally balances the risk of failing
confidence intervals against the cost of excessive optimism.
A data-driven method for syndrome type identification and classification in traditional Chinese medicine
Authors: Nevin L. Zhang, Chen Fu, Teng Fei Liu, Bao Xin Chen, Kin Man Poon, Pei Xian Chen, Yun Ling Zhang
Date: 2014-10-27 00:00:00
Tasks: Classification, General Classification, Vocal Bursts Type Prediction

Objective: The efficacy of traditional Chinese medicine (TCM) treatments for
Western medicine (WM) diseases relies heavily on the proper classification of
patients into TCM syndrome types. We develop a data-driven method for solving
the classification problem, where syndrome types are identified and quantified
based on patterns detected in unlabeled symptom survey data.
  Method: Latent class analysis (LCA) has been applied in WM research to solve
a similar problem, i.e., to identify subtypes of a patient population in the
absence of a gold standard. A widely known weakness of LCA is that it makes an
unrealistically strong independence assumption. We relax the assumption by
first detecting symptom co-occurrence patterns from survey data and use those
patterns instead of the symptoms as features for LCA. Results: The result of
the investigation is a six-step method: Data collection, symptom co-occurrence
pattern discovery, pattern interpretation, syndrome identification, syndrome
type identification, and syndrome type classification. A software package
called Lantern is developed to support the application of the method. The
method is illustrated using a data set on Vascular Mild Cognitive Impairment
(VMCI).
  Conclusions: A data-driven method for TCM syndrome identification and
classification is presented. The method can be used to answer the following
questions about a Western medicine disease: What TCM syndrome types are there
among the patients with the disease? What is the prevalence of each syndrome
type? What are the statistical characteristics of each syndrome type in terms
of occurrence of symptoms? How can we determine the syndrome type(s) of a
patient?
Identification and classification of TCM syndrome types among patients with vascular mild cognitive impairment using latent tree analysis
Authors: Chen Fu, Nevin L. Zhang, Bao Xin Chen, Zhou Rong Chen, Xiang Lan Jin, Rong Juan Guo, Zhi Gang Chen, Yun Ling Zhang
Date: 2016-01-26 00:00:00
Tasks: Classification, General Classification

Objective: To treat patients with vascular mild cognitive impairment (VMCI)
using TCM, it is necessary to classify the patients into TCM syndrome types and
to apply different treatments to different types. We investigate how to
properly carry out the classification using a novel data-driven method known as
latent tree analysis.
  Method: A cross-sectional survey on VMCI was carried out in several regions
in northern China from 2008 to 2011, which resulted in a data set that involves
803 patients and 93 symptoms. Latent tree analysis was performed on the data to
reveal symptom co-occurrence patterns, and the patients were partitioned into
clusters in multiple ways based on the patterns. The patient clusters were
matched up with syndrome types, and population statistics of the clusters are
used to quantify the syndrome types and to establish classification rules.
  Results: Eight syndrome types are identified: Qi Deficiency, Qi Stagnation,
Blood Deficiency, Blood Stasis, Phlegm-Dampness, Fire-Heat, Yang Deficiency,
and Yin Deficiency. The prevalence and symptom occurrence characteristics of
each syndrome type are determined. Quantitative classification rules are
established for determining whether a patient belongs to each of the syndrome
types.
  Conclusions: A solution for the TCM syndrome classification problem
associated with VMCI is established based on the latent tree analysis of
unlabeled symptom survey data. The results can be used as a reference in clinic
practice to improve the quality of syndrome differentiation and to reduce
diagnosis variances across physicians. They can also be used for patient
selection in research projects aimed at finding biomarkers for the syndrome
types and in randomized control trials aimed at determining the efficacy of TCM
treatments of VMCI.
Time and Activity Sequence Prediction of Business Process Instances
Authors: Mirko Polato, Alessandro Sperduti, Andrea Burattin, Massimiliano de Leoni
Date: 2016-02-24 00:00:00
Tasks: Prediction

The ability to know in advance the trend of running process instances, with
respect to different features, such as the expected completion time, would
allow business managers to timely counteract to undesired situations, in order
to prevent losses. Therefore, the ability to accurately predict future features
of running business process instances would be a very helpful aid when managing
processes, especially under service level agreement constraints. However,
making such accurate forecasts is not easy: many factors may influence the
predicted features.
  Many approaches have been proposed to cope with this problem but all of them
assume that the underling process is stationary. However, in real cases this
assumption is not always true. In this work we present new methods for
predicting the remaining time of running cases. In particular we propose a
method, assuming process stationarity, which outperforms the state-of-the-art
and two other methods which are able to make predictions even with
non-stationary processes. We also describe an approach able to predict the full
sequence of activities that a running case is going to take. All these methods
are extensively evaluated on two real case studies.
Particular object retrieval with integral max-pooling of CNN activations
Authors: Giorgos Tolias, Ronan Sicre, Hervé Jégou
Date: 2015-11-18 00:00:00
Tasks: Image Retrieval, Re-Ranking, Retrieval

Recently, image representation built upon Convolutional Neural Network (CNN)
has been shown to provide effective descriptors for image search, outperforming
pre-CNN features as short-vector representations. Yet such models are not
compatible with geometry-aware re-ranking methods and still outperformed, on
some particular object retrieval benchmarks, by traditional image search
systems relying on precise descriptor matching, geometric re-ranking, or query
expansion. This work revisits both retrieval stages, namely initial search and
re-ranking, by employing the same primitive information derived from the CNN.
We build compact feature vectors that encode several image regions without the
need to feed multiple inputs to the network. Furthermore, we extend integral
images to handle max-pooling on convolutional layer activations, allowing us to
efficiently localize matching objects. The resulting bounding box is finally
used for image re-ranking. As a result, this paper significantly improves
existing CNN-based recognition pipeline: We report for the first time results
competing with traditional methods on the challenging Oxford5k and Paris6k
datasets.
On the Accuracy of Point Localisation in a Circular Camera-Array
Authors: Alireza Ghasemi, Adam Scholefield, Martin Vetterli
Date: 2016-02-24 00:00:00

Although many advances have been made in light-field and camera-array image
processing, there is still a lack of thorough analysis of the localisation
accuracy of different multi-camera systems. By considering the problem from a
frame-quantisation perspective, we are able to quantify the point localisation
error of circular camera configurations. Specifically, we obtain closed form
expressions bounding the localisation error in terms of the parameters
describing the acquisition setup.
  These theoretical results are independent of the localisation algorithm and
thus provide fundamental limits on performance. Furthermore, the new
frame-quantisation perspective is general enough to be extended to more complex
camera configurations.
SHAPE: Linear-Time Camera Pose Estimation With Quadratic Error-Decay
Authors: Alireza Ghasemi, Adam Scholefield, Martin Vetterli
Date: 2016-02-24 00:00:00
Tasks: Camera Pose Estimation, Pose Estimation

We propose a novel camera pose estimation or perspective-n-point (PnP)
algorithm, based on the idea of consistency regions and half-space
intersections. Our algorithm has linear time-complexity and a squared
reconstruction error that decreases at least quadratically, as the number of
feature point correspondences increase.
  Inspired by ideas from triangulation and frame quantisation theory, we define
consistent reconstruction and then present SHAPE, our proposed consistent pose
estimation algorithm. We compare this algorithm with state-of-the-art pose
estimation techniques in terms of accuracy and error decay rate. The
experimental results verify our hypothesis on the optimal worst-case quadratic
decay and demonstrate its promising performance compared to other approaches.
A Bayesian Approach to the Data Description Problem
Authors: Alireza Ghasemi, Hamid R. Rabiee, Mohammad T. Manzuri, M. H. Rohban
Date: 2016-02-24 00:00:00

In this paper, we address the problem of data description using a Bayesian
framework. The goal of data description is to draw a boundary around objects of
a certain class of interest to discriminate that class from the rest of the
feature space. Data description is also known as one-class learning and has a
wide range of applications.
  The proposed approach uses a Bayesian framework to precisely compute the
class boundary and therefore can utilize domain information in form of prior
knowledge in the framework. It can also operate in the kernel space and
therefore recognize arbitrary boundary shapes. Moreover, the proposed method
can utilize unlabeled data in order to improve accuracy of discrimination.
  We evaluate our method using various real-world datasets and compare it with
other state of the art approaches of data description. Experiments show
promising results and improved performance over other data description and
one-class learning algorithms.
Active Learning from Positive and Unlabeled Data
Authors: Alireza Ghasemi, Hamid R. Rabiee, Mohsen Fadaee, Mohammad T. Manzuri, Mohammad H. Rohban
Date: 2016-02-24 00:00:00
Tasks: Active Learning, Informativeness, Multi-class Classification

During recent years, active learning has evolved into a popular paradigm for
utilizing user's feedback to improve accuracy of learning algorithms. Active
learning works by selecting the most informative sample among unlabeled data
and querying the label of that point from user. Many different methods such as
uncertainty sampling and minimum risk sampling have been utilized to select the
most informative sample in active learning. Although many active learning
algorithms have been proposed so far, most of them work with binary or
multi-class classification problems and therefore can not be applied to
problems in which only samples from one class as well as a set of unlabeled
data are available.
  Such problems arise in many real-world situations and are known as the
problem of learning from positive and unlabeled data. In this paper we propose
an active learning algorithm that can work when only samples of one class as
well as a set of unlabelled data are available. Our method works by separately
estimating probability desnity of positive and unlabeled points and then
computing expected value of informativeness to get rid of a hyper-parameter and
have a better measure of informativeness./ Experiments and empirical analysis
show promising results compared to other similar methods.
A fine-grained approach to scene text script identification
Authors: Lluis Gomez, Dimosthenis Karatzas
Date: 2016-02-24 00:00:00
Tasks: Scene Text Recognition, Text Detection

This paper focuses on the problem of script identification in unconstrained
scenarios. Script identification is an important prerequisite to recognition,
and an indispensable condition for automatic text understanding systems
designed for multi-language environments. Although widely studied for document
images and handwritten documents, it remains an almost unexplored territory for
scene text images.
  We detail a novel method for script identification in natural images that
combines convolutional features and the Naive-Bayes Nearest Neighbor
classifier. The proposed framework efficiently exploits the discriminative
power of small stroke-parts, in a fine-grained classification framework.
  In addition, we propose a new public benchmark dataset for the evaluation of
joint text detection and script identification in natural scenes. Experiments
done in this new dataset demonstrate that the proposed method yields state of
the art results, while it generalizes well to different datasets and variable
number of scripts. The evidence provided shows that multi-lingual scene text
recognition in the wild is a viable proposition. Source code of the proposed
method is made available online.
Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications
Authors: Yong-Deok Kim, Eunhyeok Park, Sungjoo Yoo, Taelim Choi, Lu Yang, Dongjun Shin
Date: 2015-11-20 00:00:00
Tasks: CPU, GPU

Although the latest high-end smartphone has powerful CPU and GPU, running
deeper convolutional neural networks (CNNs) for complex tasks such as ImageNet
classification on mobile devices is challenging. To deploy deep CNNs on mobile
devices, we present a simple and effective scheme to compress the entire CNN,
which we call one-shot whole network compression. The proposed scheme consists
of three steps: (1) rank selection with variational Bayesian matrix
factorization, (2) Tucker decomposition on kernel tensor, and (3) fine-tuning
to recover accumulated loss of accuracy, and each step can be easily
implemented using publicly available tools. We demonstrate the effectiveness of
the proposed scheme by testing the performance of various compressed CNNs
(AlexNet, VGGS, GoogLeNet, and VGG-16) on the smartphone. Significant
reductions in model size, runtime, and energy consumption are obtained, at the
cost of small loss in accuracy. In addition, we address the important
implementation level issue on 1?1 convolution, which is a key operation of
inception module of GoogLeNet as well as CNNs compressed by our proposed
scheme.
Asymptotic consistency and order specification for logistic classifier chains in multi-label learning
Authors: Paweł Teisseyre
Date: 2016-02-24 00:00:00
Tasks: Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-Label Learning

Classifier chains are popular and effective method to tackle a multi-label
classification problem. The aim of this paper is to study the asymptotic
properties of the chain model in which the conditional probabilities are of the
logistic form. In particular we find conditions on the number of labels and the
distribution of feature vector under which the estimated mode of the joint
distribution of labels converges to the true mode. Best of our knowledge, this
important issue has not yet been studied in the context of multi-label
learning. We also investigate how the order of model building in a chain
influences the estimation of the joint distribution of labels. We establish the
link between the problem of incorrect ordering in the chain and incorrect model
specification. We propose a procedure of determining the optimal ordering of
labels in the chain, which is based on using measures of correct specification
and allows to find the ordering such that the consecutive logistic models are
best possibly specified. The other important question raised in this paper is
how accurately can we estimate the joint posterior probability when the
ordering of labels is wrong or the logistic models in the chain are incorrectly
specified. The numerical experiments illustrate the theoretical results.