pwc-archive/papers-with-abstracts huggingface dataset:
Learning Sparse Additive Models with Interactions in High Dimensions
Authors: Hemant Tyagi, Anastasios Kyrillidis, Bernd Gärtner, Andreas Krause
Date: 2016-04-18 00:00:00
Tasks: Additive models, Vocal Bursts Intensity Prediction

A function $f: \mathbb{R}^d \rightarrow \mathbb{R}$ is referred to as a
Sparse Additive Model (SPAM), if it is of the form $f(\mathbf{x}) = \sum_{l \in
\mathcal{S}}\phi_{l}(x_l)$, where $\mathcal{S} \subset [d]$, $|\mathcal{S}| \ll
d$. Assuming $\phi_l$'s and $\mathcal{S}$ to be unknown, the problem of
estimating $f$ from its samples has been studied extensively. In this work, we
consider a generalized SPAM, allowing for second order interaction terms. For
some $\mathcal{S}_1 \subset [d], \mathcal{S}_2 \subset {[d] \choose 2}$, the
function $f$ is assumed to be of the form: $$f(\mathbf{x}) = \sum_{p \in
\mathcal{S}_1}\phi_{p} (x_p) + \sum_{(l,l^{\prime}) \in
\mathcal{S}_2}\phi_{(l,l^{\prime})} (x_{l},x_{l^{\prime}}).$$ Assuming
$\phi_{p},\phi_{(l,l^{\prime})}$, $\mathcal{S}_1$ and, $\mathcal{S}_2$ to be
unknown, we provide a randomized algorithm that queries $f$ and exactly
recovers $\mathcal{S}_1,\mathcal{S}_2$. Consequently, this also enables us to
estimate the underlying $\phi_p, \phi_{(l,l^{\prime})}$. We derive sample
complexity bounds for our scheme and also extend our analysis to include the
situation where the queries are corrupted with noise -- either stochastic, or
arbitrary but bounded. Lastly, we provide simulation results on synthetic data,
that validate our theoretical findings.
Pieces-of-parts for supervoxel segmentation with global context: Application to DCE-MRI tumour delineation
Authors: Benjamin Irving, James M Franklin, Bartlomiej W. Papiez, Ewan M Anderson, Ricky A Sharma, Fergus V Gleeson, Sir Michael Brady, Julia A. Schnabel
Date: 2016-04-18 00:00:00
Tasks: Segmentation

Rectal tumour segmentation in dynamic contrast-enhanced MRI (DCE-MRI) is a
challenging task, and an automated and consistent method would be highly
desirable to improve the modelling and prediction of patient outcomes from
tissue contrast enhancement characteristics - particularly in routine clinical
practice. A framework is developed to automate DCE-MRI tumour segmentation, by
introducing: perfusion-supervoxels to over-segment and classify DCE-MRI volumes
using the dynamic contrast enhancement characteristics; and the pieces-of-parts
graphical model, which adds global (anatomic) constraints that further refine
the supervoxel components that comprise the tumour. The framework was evaluated
on 23 DCE-MRI scans of patients with rectal adenocarcinomas, and achieved a
voxelwise area-under the receiver operating characteristic curve (AUC) of 0.97
compared to expert delineations. Creating a binary tumour segmentation, 21 of
the 23 cases were segmented correctly with a median Dice similarity coefficient
(DSC) of 0.63, which is close to the inter-rater variability of this
challenging task. A sec- ond study is also included to demonstrate the method's
generalisability and achieved a DSC of 0.71. The framework achieves promising
results for the underexplored area of rectal tumour segmentation in DCE-MRI,
and the methods have potential to be applied to other DCE-MRI and supervoxel
segmentation problems
Locally Imposing Function for Generalized Constraint Neural Networks - A Study on Equality Constraints
Authors: Linlin Cao, Ran He, Bao-Gang Hu
Date: 2016-04-18 00:00:00

This work is a further study on the Generalized Constraint Neural Network
(GCNN) model [1], [2]. Two challenges are encountered in the study, that is, to
embed any type of prior information and to select its imposing schemes. The
work focuses on the second challenge and studies a new constraint imposing
scheme for equality constraints. A new method called locally imposing function
(LIF) is proposed to provide a local correction to the GCNN prediction
function, which therefore falls within Locally Imposing Scheme (LIS). In
comparison, the conventional Lagrange multiplier method is considered as
Globally Imposing Scheme (GIS) because its added constraint term exhibits a
global impact to its objective function. Two advantages are gained from LIS
over GIS. First, LIS enables constraints to fire locally and explicitly in the
domain only where they need on the prediction function. Second, constraints can
be implemented within a network setting directly. We attempt to interpret
several constraint methods graphically from a viewpoint of the locality
principle. Numerical examples confirm the advantages of the proposed method. In
solving boundary value problems with Dirichlet and Neumann constraints, the
GCNN model with LIF is possible to achieve an exact satisfaction of the
constraints.
ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation
Authors: Di Lin, Jifeng Dai, Jiaya Jia, Kaiming He, Jian Sun
Date: 2016-04-18 00:00:00
Proceeding: CVPR 2016 6
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Large-scale data is of crucial importance for learning semantic segmentation
models, but annotating per-pixel masks is a tedious and inefficient procedure.
We note that for the topic of interactive image segmentation, scribbles are
very widely used in academic research and commercial software, and are
recognized as one of the most user-friendly ways of interacting. In this paper,
we propose to use scribbles to annotate images, and develop an algorithm to
train convolutional networks for semantic segmentation supervised by scribbles.
Our algorithm is based on a graphical model that jointly propagates information
from scribbles to unmarked pixels and learns network parameters. We present
competitive object semantic segmentation results on the PASCAL VOC dataset by
using scribbles as annotations. Scribbles are also favored for annotating stuff
(e.g., water, sky, grass) that has no well-defined shape, and our method shows
excellent results on the PASCAL-CONTEXT dataset thanks to extra inexpensive
scribble annotations. Our scribble annotations on PASCAL VOC are available at
http://research.microsoft.com/en-us/um/people/jifdai/downloads/scribble_sup
Using Self-Contradiction to Learn Confidence Measures in Stereo Vision
Authors: Christian Mostegel, Markus Rumpler, Friedrich Fraundorfer, Horst Bischof
Date: 2016-04-18 00:00:00
Proceeding: CVPR 2016 6

Learned confidence measures gain increasing importance for outlier removal
and quality improvement in stereo vision. However, acquiring the necessary
training data is typically a tedious and time consuming task that involves
manual interaction, active sensing devices and/or synthetic scenes. To overcome
this problem, we propose a new, flexible, and scalable way for generating
training data that only requires a set of stereo images as input. The key idea
of our approach is to use different view points for reasoning about
contradictions and consistencies between multiple depth maps generated with the
same stereo algorithm. This enables us to generate a huge amount of training
data in a fully automated manner. Among other experiments, we demonstrate the
potential of our approach by boosting the performance of three learned
confidence measures on the KITTI2012 dataset by simply training them on a vast
amount of automatically generated training data rather than a limited amount of
laser ground truth data.
Deep Exemplar 2D-3D Detection by Adapting from Real to Rendered Views
Authors: Francisco Massa, Bryan Russell, Mathieu Aubry
Date: 2015-12-08 00:00:00
Proceeding: CVPR 2016 6

This paper presents an end-to-end convolutional neural network (CNN) for
2D-3D exemplar detection. We demonstrate that the ability to adapt the features
of natural images to better align with those of CAD rendered views is critical
to the success of our technique. We show that the adaptation can be learned by
compositing rendered views of textured object models on natural images. Our
approach can be naturally incorporated into a CNN detection pipeline and
extends the accuracy and speed benefits from recent advances in deep learning
to 2D-3D exemplar detection. We applied our method to two tasks: instance
detection, where we evaluated on the IKEA dataset, and object category
detection, where we out-perform Aubry et al. for "chair" detection on a subset
of the Pascal VOC dataset.
BinaryConnect: Training Deep Neural Networks with binary weights during propagations
Authors: Matthieu Courbariaux, Yoshua Bengio, Jean-Pierre David
Date: 2015-11-02 00:00:00
Proceeding: NeurIPS 2015 12

Deep Neural Networks (DNN) have achieved state-of-the-art results in a wide
range of tasks, with the best results obtained with large training sets and
large models. In the past, GPUs enabled these breakthroughs because of their
greater computational speed. In the future, faster computation at both training
and test time is likely to be crucial for further progress and for consumer
applications on low-power devices. As a result, there is much interest in
research and development of dedicated hardware for Deep Learning (DL). Binary
weights, i.e., weights which are constrained to only two possible values (e.g.
-1 or 1), would bring great benefits to specialized DL hardware by replacing
many multiply-accumulate operations by simple accumulations, as multipliers are
the most space and power-hungry components of the digital implementation of
neural networks. We introduce BinaryConnect, a method which consists in
training a DNN with binary weights during the forward and backward
propagations, while retaining precision of the stored weights in which
gradients are accumulated. Like other dropout schemes, we show that
BinaryConnect acts as regularizer and we obtain near state-of-the-art results
with BinaryConnect on the permutation-invariant MNIST, CIFAR-10 and SVHN.
Normative Multiagent Systems: A Dynamic Generalization
Authors: Xiaowei Huang, Ji Ruan, Qingliang Chen, Kaile Su
Date: 2016-04-18 00:00:00

Social norms are powerful formalism in coordinating autonomous agents'
behaviour to achieve certain objectives. In this paper, we propose a dynamic
normative system to enable the reasoning of the changes of norms under
different circumstances, which cannot be done in the existing static normative
systems. We study two important problems (norm synthesis and norm recognition)
related to the autonomy of the entire system and the agents, and characterise
the computational complexities of solving these problems.
Speed-Constrained Tuning for Statistical Machine Translation Using Bayesian Optimization
Authors: Daniel Beck, Adrià De Gispert, Gonzalo Iglesias, Aurelien Waite, Bill Byrne
Date: 2016-04-18 00:00:00
Proceeding: NAACL 2016 6
Tasks: Bayesian Optimization, Machine Translation, Translation

We address the problem of automatically finding the parameters of a
statistical machine translation system that maximize BLEU scores while ensuring
that decoding speed exceeds a minimum value. We propose the use of Bayesian
Optimization to efficiently tune the speed-related decoding parameters by
easily incorporating speed as a noisy constraint function. The obtained
parameter values are guaranteed to satisfy the speed constraint with an
associated confidence margin. Across three language pairs and two speed
constraint values, we report overall optimization time reduction compared to
grid and random search. We also show that Bayesian Optimization can decouple
speed and BLEU measurements, resulting in a further reduction of overall
optimization time as speed is measured over a small subset of sentences.
Learning Sparse Low-Threshold Linear Classifiers
Authors: Sivan Sabato, Shai Shalev-Shwartz, Nathan Srebro, Daniel Hsu, Tong Zhang
Date: 2012-12-13 00:00:00

We consider the problem of learning a non-negative linear classifier with a
$1$-norm of at most $k$, and a fixed threshold, under the hinge-loss. This
problem generalizes the problem of learning a $k$-monotone disjunction. We
prove that we can learn efficiently in this setting, at a rate which is linear
in both $k$ and the size of the threshold, and that this is the best possible
rate. We provide an efficient online learning algorithm that achieves the
optimal rate, and show that in the batch case, empirical risk minimization
achieves this rate as well. The rates we show are tighter than the uniform
convergence rate, which grows with $k^2$.
Loss minimization and parameter estimation with heavy tails
Authors: Daniel Hsu, Sivan Sabato
Date: 2013-07-07 00:00:00
Tasks: parameter estimation, regression

This work studies applications and generalizations of a simple estimation
technique that provides exponential concentration under heavy-tailed
distributions, assuming only bounded low-order moments. We show that the
technique can be used for approximate minimization of smooth and strongly
convex losses, and specifically for least squares linear regression. For
instance, our $d$-dimensional estimator requires just
$\tilde{O}(d\log(1/\delta))$ random samples to obtain a constant factor
approximation to the optimal least squares loss with probability $1-\delta$,
without requiring the covariates or noise to be bounded or subgaussian. We
provide further applications to sparse linear regression and low-rank
covariance matrix estimation with similar allowances on the noise and covariate
distributions. The core technique is a generalization of the median-of-means
estimator to arbitrary metric spaces.
Empirical study of PROXTONE and PROXTONE$^+$ for Fast Learning of Large Scale Sparse Models
Authors: Ziqiang Shi, Rujie Liu
Date: 2016-04-18 00:00:00

PROXTONE is a novel and fast method for optimization of large scale
non-smooth convex problem \cite{shi2015large}. In this work, we try to use
PROXTONE method in solving large scale \emph{non-smooth non-convex} problems,
for example training of sparse deep neural network (sparse DNN) or sparse
convolutional neural network (sparse CNN) for embedded or mobile device.
PROXTONE converges much faster than first order methods, while first order
method is easy in deriving and controlling the sparseness of the solutions.
Thus in some applications, in order to train sparse models fast, we propose to
combine the merits of both methods, that is we use PROXTONE in the first
several epochs to reach the neighborhood of an optimal solution, and then use
the first order method to explore the possibility of sparsity in the following
training. We call such method PROXTONE plus (PROXTONE$^+$). Both PROXTONE and
PROXTONE$^+$ are tested in our experiments, and which demonstrate both methods
improved convergence speed twice as fast at least on diverse sparse model
learning problems, and at the same time reduce the size to 0.5\% for DNN
models. The source of all the algorithms is available upon request.
Forecasting Volatility in Indian Stock Market using Artificial Neural Network with Multiple Inputs and Outputs
Authors: Tamal Datta Chaudhuri, Indranil Ghosh
Date: 2016-04-18 00:00:00

Volatility in stock markets has been extensively studied in the applied
finance literature. In this paper, Artificial Neural Network models based on
various back propagation algorithms have been constructed to predict volatility
in the Indian stock market through volatility of NIFTY returns and volatility
of gold returns. This model considers India VIX, CBOE VIX, volatility of crude
oil returns (CRUDESDR), volatility of DJIA returns (DJIASDR), volatility of DAX
returns (DAXSDR), volatility of Hang Seng returns (HANGSDR) and volatility of
Nikkei returns (NIKKEISDR) as predictor variables. Three sets of experiments
have been performed over three time periods to judge the effectiveness of the
approach.
Gaussian Copula Variational Autoencoders for Mixed Data
Authors: Suwon Suh, Seungjin Choi
Date: 2016-04-18 00:00:00
Tasks: Decoder

The variational autoencoder (VAE) is a generative model with continuous
latent variables where a pair of probabilistic encoder (bottom-up) and decoder
(top-down) is jointly learned by stochastic gradient variational Bayes. We
first elaborate Gaussian VAE, approximating the local covariance matrix of the
decoder as an outer product of the principal direction at a position determined
by a sample drawn from Gaussian distribution. We show that this model, referred
to as VAE-ROC, better captures the data manifold, compared to the standard
Gaussian VAE where independent multivariate Gaussian was used to model the
decoder. Then we extend the VAE-ROC to handle mixed categorical and continuous
data. To this end, we employ Gaussian copula to model the local dependency in
mixed categorical and continuous data, leading to {\em Gaussian copula
variational autoencoder} (GCVAE). As in VAE-ROC, we use the rank-one
approximation for the covariance in the Gaussian copula, to capture the local
dependency structure in the mixed data. Experiments on various datasets
demonstrate the useful behaviour of VAE-ROC and GCVAE, compared to the standard
VAE.
Fully Convolutional Recurrent Network for Handwritten Chinese Text Recognition
Authors: Zecheng Xie, Zenghui Sun, Lianwen Jin, Ziyong Feng, Shuye Zhang
Date: 2016-04-18 00:00:00
Tasks: Handwriting Recognition, Handwritten Chinese Text Recognition, Language Modeling, Language Modelling

This paper proposes an end-to-end framework, namely fully convolutional
recurrent network (FCRN) for handwritten Chinese text recognition (HCTR).
Unlike traditional methods that rely heavily on segmentation, our FCRN is
trained with online text data directly and learns to associate the pen-tip
trajectory with a sequence of characters. FCRN consists of four parts: a
path-signature layer to extract signature features from the input pen-tip
trajectory, a fully convolutional network to learn informative representation,
a sequence modeling layer to make per-frame predictions on the input sequence
and a transcription layer to translate the predictions into a label sequence.
The FCRN is end-to-end trainable in contrast to conventional methods whose
components are separately trained and tuned. We also present a refined beam
search method that efficiently integrates the language model to decode the FCRN
and significantly improve the recognition results.
  We evaluate the performance of the proposed method on the test sets from the
databases CASIA-OLHWDB and ICDAR 2013 Chinese handwriting recognition
competition, and both achieve state-of-the-art performance with correct rates
of 96.40% and 95.00%, respectively.
Optimal Estimation of Low Rank Density Matrices
Authors: Vladimir Koltchinskii, Dong Xia
Date: 2015-07-17 00:00:00
Tasks: Quantum State Tomography

The density matrices are positively semi-definite Hermitian matrices of unit
trace that describe the state of a quantum system. The goal of the paper is to
develop minimax lower bounds on error rates of estimation of low rank density
matrices in trace regression models used in quantum state tomography (in
particular, in the case of Pauli measurements) with explicit dependence of the
bounds on the rank and other complexity parameters. Such bounds are established
for several statistically relevant distances, including quantum versions of
Kullback-Leibler divergence (relative entropy distance) and of Hellinger
distance (so called Bures distance), and Schatten $p$-norm distances. Sharp
upper bounds and oracle inequalities for least squares estimator with von
Neumann entropy penalization are obtained showing that minimax lower bounds are
attained (up to logarithmic factors) for these distances.
Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation
Authors: Sina Honari, Jason Yosinski, Pascal Vincent, Christopher Pal
Date: 2015-11-23 00:00:00
Proceeding: CVPR 2016 6
Tasks: image-classification, Image Classification

Deep neural networks with alternating convolutional, max-pooling and
decimation layers are widely used in state of the art architectures for
computer vision. Max-pooling purposefully discards precise spatial information
in order to create features that are more robust, and typically organized as
lower resolution spatial feature maps. On some tasks, such as whole-image
classification, max-pooling derived features are well suited; however, for
tasks requiring precise localization, such as pixel level prediction and
segmentation, max-pooling destroys exactly the information required to perform
well. Precise localization may be preserved by shallow convnets without pooling
but at the expense of robustness. Can we have our max-pooled multi-layered cake
and eat it too? Several papers have proposed summation and concatenation based
methods for combining upsampled coarse, abstract features with finer features
to produce robust pixel level predictions. Here we introduce another model ---
dubbed Recombinator Networks --- where coarse features inform finer features
early in their formation such that finer features can make use of several
layers of computation in deciding how to use coarse features. The model is
trained once, end-to-end and performs better than summation-based
architectures, reducing the error from the previous state of the art on two
facial keypoint datasets, AFW and AFLW, by 30\% and beating the current
state-of-the-art on 300W without using extra data. We improve performance even
further by adding a denoising prediction model based on a novel convnet
formulation.
Multi-view Learning as a Nonparametric Nonlinear Inter-Battery Factor Analysis
Authors: Andreas Damianou, Neil D. Lawrence, Carl Henrik Ek
Date: 2016-04-17 00:00:00
Tasks: MULTI-VIEW LEARNING

Factor analysis aims to determine latent factors, or traits, which summarize
a given data set. Inter-battery factor analysis extends this notion to multiple
views of the data. In this paper we show how a nonlinear, nonparametric version
of these models can be recovered through the Gaussian process latent variable
model. This gives us a flexible formalism for multi-view learning where the
latent variables can be used both for exploratory purposes and for learning
representations that enable efficient inference for ambiguous estimation tasks.
Learning is performed in a Bayesian manner through the formulation of a
variational compression scheme which gives a rigorous lower bound on the log
likelihood. Our Bayesian framework provides strong regularization during
training, allowing the structure of the latent space to be determined
efficiently and automatically. We demonstrate this by producing the first (to
our knowledge) published results of learning from dozens of views, even when
data is scarce. We further show experimental results on several different types
of multi-view data sets and for different kinds of tasks, including exploratory
data analysis, generation, ambiguity modelling through latent priors and
classification.
The Variational Gaussian Process
Authors: Dustin Tran, Rajesh Ranganath, David M. Blei
Date: 2015-11-20 00:00:00
Tasks: Representation Learning, Variational Inference

Variational inference is a powerful tool for approximate inference, and it
has been recently applied for representation learning with deep generative
models. We develop the variational Gaussian process (VGP), a Bayesian
nonparametric variational family, which adapts its shape to match complex
posterior distributions. The VGP generates approximate posterior samples by
generating latent inputs and warping them through random non-linear mappings;
the distribution over random mappings is learned during inference, enabling the
transformed outputs to adapt to varying complexity. We prove a universal
approximation theorem for the VGP, demonstrating its representative power for
learning any model. For inference we present a variational objective inspired
by auto-encoders and perform black box inference over a wide class of models.
The VGP achieves new state-of-the-art results for unsupervised learning,
inferring models such as the deep latent Gaussian model and the recently
proposed DRAW.