pwc-archive/papers-with-abstracts huggingface dataset:
A Novel Hybrid Approach for Cephalometric Landmark Detection
Authors: Mahshid Majd, Farzaneh Shoeleh
Date: 2015-06-12 00:00:00
Tasks: Template Matching

Cephalometric analysis has an important role in dentistry and especially in
orthodontics as a treatment planning tool to gauge the size and special
relationships of the teeth, jaws and cranium. The first step of using such
analyses is localizing some important landmarks known as cephalometric
landmarks on craniofacial in x-ray image. The past decade has seen a growing
interest in automating this process. In this paper, a novel hybrid approach is
proposed for automatic detection of cephalometric landmarks. Here, the
landmarks are categorized into three main sets according to their anatomical
characteristics and usage in well-known cephalometric analyses. Consequently,
to have a reliable and accurate detection system, three methods named edge
tracing, weighted template matching, and analysis based estimation are
designed, each of which is consistent and well-suited for one category. Edge
tracing method is suggested to predict those landmarks which are located on
edges. Weighted template matching method is well-suited for landmarks located
in an obvious and specific structure which can be extracted or searchable in a
given x-ray image. The last but not the least method is named analysis based
estimation. This method is based on the fact that in cephalometric analyses the
relations between landmarks are used and the locations of some landmarks are
never used individually. Therefore the third suggested method has a novelty in
estimating the desired relations directly. The effectiveness of the proposed
approach is compared with the state of the art methods and the results were
promising especially in real world applications.
Syntax-based Deep Matching of Short Texts
Authors: Mingxuan Wang, Zhengdong Lu, Hang Li, Qun Liu
Date: 2015-03-09 00:00:00
Tasks: Machine Translation, Question Answering, Translation

Many tasks in natural language processing, ranging from machine translation
to question answering, can be reduced to the problem of matching two sentences
or more generally two short texts. We propose a new approach to the problem,
called Deep Match Tree (DeepMatch$_{tree}$), under a general setting. The
approach consists of two components, 1) a mining algorithm to discover patterns
for matching two short-texts, defined in the product space of dependency trees,
and 2) a deep neural network for matching short texts using the mined patterns,
as well as a learning algorithm to build the network having a sparse structure.
We test our algorithm on the problem of matching a tweet and a response in
social media, a hard matching problem proposed in [Wang et al., 2013], and show
that DeepMatch$_{tree}$ can outperform a number of competitor models including
one without using dependency trees and one based on word-embedding, all with
large margins
MCMC Learning
Authors: Varun Kanade, Elchanan Mossel
Date: 2013-07-13 00:00:00

The theory of learning under the uniform distribution is rich and deep, with
connections to cryptography, computational complexity, and the analysis of
boolean functions to name a few areas. This theory however is very limited due
to the fact that the uniform distribution and the corresponding Fourier basis
are rarely encountered as a statistical model.
  A family of distributions that vastly generalizes the uniform distribution on
the Boolean cube is that of distributions represented by Markov Random Fields
(MRF). Markov Random Fields are one of the main tools for modeling high
dimensional data in many areas of statistics and machine learning.
  In this paper we initiate the investigation of extending central ideas,
methods and algorithms from the theory of learning under the uniform
distribution to the setup of learning concepts given examples from MRF
distributions. In particular, our results establish a novel connection between
properties of MCMC sampling of MRFs and learning under the MRF distribution.
Consistency and fluctuations for stochastic gradient Langevin dynamics
Authors: Yee Whye Teh, Alexandre Thiéry, Sebastian Vollmer
Date: 2014-09-01 00:00:00

Applying standard Markov chain Monte Carlo (MCMC) algorithms to large data
sets is computationally expensive. Both the calculation of the acceptance
probability and the creation of informed proposals usually require an iteration
through the whole data set. The recently proposed stochastic gradient Langevin
dynamics (SGLD) method circumvents this problem by generating proposals which
are only based on a subset of the data, by skipping the accept-reject step and
by using decreasing step-sizes sequence $(\delta_m)_{m \geq 0}$.
  %Under appropriate Lyapunov conditions, We provide in this article a rigorous
mathematical framework for analysing this algorithm. We prove that, under
verifiable assumptions, the algorithm is consistent, satisfies a central limit
theorem (CLT) and its asymptotic bias-variance decomposition can be
characterized by an explicit functional of the step-sizes sequence
$(\delta_m)_{m \geq 0}$. We leverage this analysis to give practical
recommendations for the notoriously difficult tuning of this algorithm: it is
asymptotically optimal to use a step-size sequence of the type $\delta_m \asymp
m^{-1/3}$, leading to an algorithm whose mean squared error (MSE) decreases at
rate $\mathcal{O}(m^{-1/3})$
Place classification with a graph regularized deep neural network model
Authors: Yiyi Liao, Sarath Kodagoda, Yue Wang, Lei Shi, Yong liu
Date: 2015-06-12 00:00:00
Tasks: Classification, General Classification

Place classification is a fundamental ability that a robot should possess to
carry out effective human-robot interactions. It is a nontrivial classification
problem which has attracted many research. In recent years, there is a high
exploitation of Artificial Intelligent algorithms in robotics applications.
Inspired by the recent successes of deep learning methods, we propose an
end-to-end learning approach for the place classification problem. With the
deep architectures, this methodology automatically discovers features and
contributes in general to higher classification accuracies. The pipeline of our
approach is composed of three parts. Firstly, we construct multiple layers of
laser range data to represent the environment information in different levels
of granularity. Secondly, each layer of data is fed into a deep neural network
model for classification, where a graph regularization is imposed to the deep
architecture for keeping local consistency between adjacent samples. Finally,
the predicted labels obtained from all the layers are fused based on confidence
trees to maximize the overall confidence. Experimental results validate the
effective- ness of our end-to-end place classification framework in which both
the multi-layer structure and the graph regularization promote the
classification performance. Furthermore, results show that the features
automatically learned from the raw input range data can achieve competitive
results to the features constructed based on statistical and geometrical
information.
Classify Images with Conceptor Network
Authors: Yuhuang Hu, M. S. Ishwarya, Chu Kiong Loo
Date: 2015-06-02 00:00:00
Tasks: regression

This article demonstrates a new conceptor network based classifier in
classifying images. Mathematical descriptions and analysis are presented.
Various tests are experimented using three benchmark datasets: MNIST, CIFAR-10
and CIFAR-100. The experiments displayed that conceptor network can offer
superior results and flexible configurations than conventional classifiers such
as Softmax Regression and Support Vector Machine (SVM).
Hierarchies of Relaxations for Online Prediction Problems with Evolving Constraints
Authors: Alexander Rakhlin, Karthik Sridharan
Date: 2015-03-04 00:00:00
Tasks: Prediction

We study online prediction where regret of the algorithm is measured against
a benchmark defined via evolving constraints. This framework captures online
prediction on graphs, as well as other prediction problems with combinatorial
structure. A key aspect here is that finding the optimal benchmark predictor
(even in hindsight, given all the data) might be computationally hard due to
the combinatorial nature of the constraints. Despite this, we provide
polynomial-time \emph{prediction} algorithms that achieve low regret against
combinatorial benchmark sets. We do so by building improper learning algorithms
based on two ideas that work together. The first is to alleviate part of the
computational burden through random playout, and the second is to employ
Lasserre semidefinite hierarchies to approximate the resulting integer program.
Interestingly, for our prediction algorithms, we only need to compute the
values of the semidefinite programs and not the rounded solutions. However, the
integrality gap for Lasserre hierarchy \emph{does} enter the generic regret
bound in terms of Rademacher complexity of the benchmark set. This establishes
a trade-off between the computation time and the regret bound of the algorithm.
Tree-Cut for Probabilistic Image Segmentation
Authors: Shell X. Hu, Christopher K. I. Williams, Sinisa Todorovic
Date: 2015-06-11 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation, Superpixels

This paper presents a new probabilistic generative model for image
segmentation, i.e. the task of partitioning an image into homogeneous regions.
Our model is grounded on a mid-level image representation, called a region
tree, in which regions are recursively split into subregions until superpixels
are reached. Given the region tree, image segmentation is formalized as
sampling cuts in the tree from the model. Inference for the cuts is exact, and
formulated using dynamic programming. Our tree-cut model can be tuned to sample
segmentations at a particular scale of interest out of many possible multiscale
image segmentations. This generalizes the common notion that there should be
only one correct segmentation per image. Also, it allows moving beyond the
standard single-scale evaluation, where the segmentation result for an image is
averaged against the corresponding set of coarse and fine human annotations, to
conduct a scale-specific evaluation. Our quantitative results are comparable to
those of the leading gPb-owt-ucm method, with the notable advantage that we
additionally produce a distribution over all possible tree-consistent
segmentations of the image.
Rectified Factor Networks
Authors: Djork-Arné Clevert, Andreas Mayr, Thomas Unterthiner, Sepp Hochreiter
Date: 2015-02-23 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Drug Discovery

We propose rectified factor networks (RFNs) to efficiently construct very
sparse, non-linear, high-dimensional representations of the input. RFN models
identify rare and small events in the input, have a low interference between
code units, have a small reconstruction error, and explain the data covariance
structure. RFN learning is a generalized alternating minimization algorithm
derived from the posterior regularization method which enforces non-negative
and normalized posterior means. We proof convergence and correctness of the RFN
learning algorithm. On benchmarks, RFNs are compared to other unsupervised
methods like autoencoders, RBMs, factor analysis, ICA, and PCA. In contrast to
previous sparse coding methods, RFNs yield sparser codes, capture the data's
covariance structure more precisely, and have a significantly smaller
reconstruction error. We test RFNs as pretraining technique for deep networks
on different vision datasets, where RFNs were superior to RBMs and
autoencoders. On gene expression data from two pharmaceutical drug discovery
studies, RFNs detected small and rare gene modules that revealed highly
relevant new biological insights which were so far missed by other unsupervised
methods.
Pose-Invariant 3D Face Alignment
Authors: Amin Jourabloo, Xiaoming Liu
Date: 2015-06-11 00:00:00
Proceeding: ICCV 2015 12
Tasks: 3D Face Alignment, Face Alignment

Face alignment aims to estimate the locations of a set of landmarks for a
given image. This problem has received much attention as evidenced by the
recent advancement in both the methodology and performance. However, most of
the existing works neither explicitly handle face images with arbitrary poses,
nor perform large-scale experiments on non-frontal and profile face images. In
order to address these limitations, this paper proposes a novel face alignment
algorithm that estimates both 2D and 3D landmarks and their 2D visibilities for
a face image with an arbitrary pose. By integrating a 3D deformable model, a
cascaded coupled-regressor approach is designed to estimate both the camera
projection matrix and the 3D landmarks. Furthermore, the 3D model also allows
us to automatically estimate the 2D landmark visibilities via surface normals.
We gather a substantially larger collection of all-pose face images to evaluate
our algorithm and demonstrate superior performances than the state-of-the-art
methods.
Entity-Specific Sentiment Classification of Yahoo News Comments
Authors: Prakhar Biyani, Cornelia Caragea, Narayan Bhamidipati
Date: 2015-06-11 00:00:00
Tasks: Articles, Classification, General Classification, Sentiment Analysis, Sentiment Classification

Sentiment classification is widely used for product reviews and in online
social media such as forums, Twitter, and blogs. However, the problem of
classifying the sentiment of user comments on news sites has not been addressed
yet. News sites cover a wide range of domains including politics, sports,
technology, and entertainment, in contrast to other online social sites such as
forums and review sites, which are specific to a particular domain. A user
associated with a news site is likely to post comments on diverse topics (e.g.,
politics, smartphones, and sports) or diverse entities (e.g., Obama, iPhone, or
Google). Classifying the sentiment of users tied to various entities may help
obtain a holistic view of their personality, which could be useful in
applications such as online advertising, content personalization, and political
campaign planning. In this paper, we formulate the problem of entity-specific
sentiment classification of comments posted on news articles in Yahoo News and
propose novel features that are specific to news comments. Experimental results
show that our models outperform state-of-the-art baselines.
Probabilistic Curve Learning: Coulomb Repulsion and the Electrostatic Gaussian Process
Authors: Ye Wang, David B. Dunson
Date: 2015-06-11 00:00:00
Proceeding: NeurIPS 2015 12

Learning of low dimensional structure in multidimensional data is a canonical
problem in machine learning. One common approach is to suppose that the
observed data are close to a lower-dimensional smooth manifold. There are a
rich variety of manifold learning methods available, which allow mapping of
data points to the manifold. However, there is a clear lack of probabilistic
methods that allow learning of the manifold along with the generative
distribution of the observed data. The best attempt is the Gaussian process
latent variable model (GP-LVM), but identifiability issues lead to poor
performance. We solve these issues by proposing a novel Coulomb repulsive
process (Corp) for locations of points on the manifold, inspired by physical
models of electrostatic interactions among particles. Combining this process
with a GP prior for the mapping function yields a novel electrostatic GP
(electroGP) process. Focusing on the simple case of a one-dimensional manifold,
we develop efficient inference algorithms, and illustrate substantially
improved performance in a variety of experiments including filling in missing
frames in video.
Spectral Representations for Convolutional Neural Networks
Authors: Oren Rippel, Jasper Snoek, Ryan P. Adams
Date: 2015-06-11 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Dimensionality Reduction

Discrete Fourier transforms provide a significant speedup in the computation
of convolutions in deep learning. In this work, we demonstrate that, beyond its
advantages for efficient computation, the spectral domain also provides a
powerful representation in which to model and train convolutional neural
networks (CNNs).
  We employ spectral representations to introduce a number of innovations to
CNN design. First, we propose spectral pooling, which performs dimensionality
reduction by truncating the representation in the frequency domain. This
approach preserves considerably more information per parameter than other
pooling strategies and enables flexibility in the choice of pooling output
dimensionality. This representation also enables a new form of stochastic
regularization by randomized modification of resolution. We show that these
methods achieve competitive results on classification and approximation tasks,
without using any dropout or max-pooling.
  Finally, we demonstrate the effectiveness of complex-coefficient spectral
parameterization of convolutional filters. While this leaves the underlying
model unchanged, it results in a representation that greatly facilitates
optimization. We observe on a variety of popular CNN configurations that this
leads to significantly faster convergence during training.
Recovering communities in the general stochastic block model without knowing the parameters
Authors: Emmanuel Abbe, Colin Sandon
Date: 2015-06-11 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Stochastic Block Model

Most recent developments on the stochastic block model (SBM) rely on the
knowledge of the model parameters, or at least on the number of communities.
This paper introduces efficient algorithms that do not require such knowledge
and yet achieve the optimal information-theoretic tradeoffs identified in
[AS15] for linear size communities. The results are three-fold: (i) in the
constant degree regime, an algorithm is developed that requires only a
lower-bound on the relative sizes of the communities and detects communities
with an optimal accuracy scaling for large degrees; (ii) in the regime where
degrees are scaled by $\omega(1)$ (diverging degrees), this is enhanced into a
fully agnostic algorithm that only takes the graph in question and
simultaneously learns the model parameters (including the number of
communities) and detects communities with accuracy $1-o(1)$, with an overall
quasi-linear complexity; (iii) in the logarithmic degree regime, an agnostic
algorithm is developed that learns the parameters and achieves the optimal
CH-limit for exact recovery, in quasi-linear time. These provide the first
algorithms affording efficiency, universality and information-theoretic
optimality for strong and weak consistency in the general SBM with linear size
communities.
Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)
Authors: Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, Alan Yuille
Date: 2014-12-20 00:00:00
Tasks: 8k, Image Captioning, Retrieval

In this paper, we present a multimodal Recurrent Neural Network (m-RNN) model
for generating novel image captions. It directly models the probability
distribution of generating a word given previous words and an image. Image
captions are generated by sampling from this distribution. The model consists
of two sub-networks: a deep recurrent neural network for sentences and a deep
convolutional network for images. These two sub-networks interact with each
other in a multimodal layer to form the whole m-RNN model. The effectiveness of
our model is validated on four benchmark datasets (IAPR TC-12, Flickr 8K,
Flickr 30K and MS COCO). Our model outperforms the state-of-the-art methods. In
addition, we apply the m-RNN model to retrieval tasks for retrieving images or
sentences, and achieves significant performance improvement over the
state-of-the-art methods which directly optimize the ranking objective function
for retrieval. The project page of this work is:
www.stat.ucla.edu/~junhua.mao/m-RNN.html .
On the Invariance of Dictionary Learning and Sparse Representation to Projecting Data to a Discriminative Space
Authors: Mehrdad J. Gangeh, Ali Ghodsi
Date: 2015-03-06 00:00:00
Tasks: Dictionary Learning

In this paper, it is proved that dictionary learning and sparse
representation is invariant to a linear transformation. It subsumes the special
case of transforming/projecting the data into a discriminative space. This is
important because recently, supervised dictionary learning algorithms have been
proposed, which suggest to include the category information into the learning
of dictionary to improve its discriminative power. Among them, there are some
approaches that propose to learn the dictionary in a discriminative projected
space. To this end, two approaches have been proposed: first, assigning the
discriminative basis as the dictionary and second, perform dictionary learning
in the projected space. Based on the invariance of dictionary learning to any
transformation in general, and to a discriminative space in particular, we
advocate the first approach.
Margin-Based Feed-Forward Neural Network Classifiers
Authors: Han Xiao, Xiaoyan Zhu
Date: 2015-06-11 00:00:00

Margin-Based Principle has been proposed for a long time, it has been proved
that this principle could reduce the structural risk and improve the
performance in both theoretical and practical aspects. Meanwhile, feed-forward
neural network is a traditional classifier, which is very hot at present with a
deeper architecture. However, the training algorithm of feed-forward neural
network is developed and generated from Widrow-Hoff Principle that means to
minimize the squared error. In this paper, we propose a new training algorithm
for feed-forward neural networks based on Margin-Based Principle, which could
effectively promote the accuracy and generalization ability of neural network
classifiers with less labelled samples and flexible network. We have conducted
experiments on four UCI open datasets and achieved good results as expected. In
conclusion, our model could handle more sparse labelled and more high-dimension
dataset in a high accuracy while modification from old ANN method to our method
is easy and almost free of work.
Bootstrapping Skills
Authors: Daniel J. Mankowitz, Timothy A. Mann, Shie Mannor
Date: 2015-06-11 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

The monolithic approach to policy representation in Markov Decision Processes
(MDPs) looks for a single policy that can be represented as a function from
states to actions. For the monolithic approach to succeed (and this is not
always possible), a complex feature representation is often necessary since the
policy is a complex object that has to prescribe what actions to take all over
the state space. This is especially true in large domains with complicated
dynamics. It is also computationally inefficient to both learn and plan in MDPs
using a complex monolithic approach. We present a different approach where we
restrict the policy space to policies that can be represented as combinations
of simpler, parameterized skills---a type of temporally extended action, with a
simple policy representation. We introduce Learning Skills via Bootstrapping
(LSB) that can use a broad family of Reinforcement Learning (RL) algorithms as
a "black box" to iteratively learn parametrized skills. Initially, the learned
skills are short-sighted but each iteration of the algorithm allows the skills
to bootstrap off one another, improving each skill in the process. We prove
that this bootstrapping process returns a near-optimal policy. Furthermore, our
experiments demonstrate that LSB can solve MDPs that, given the same
representational power, could not be solved by a monolithic approach. Thus,
planning with learned skills results in better policies without requiring
complex policy representations.