pwc-archive/papers-with-abstracts huggingface dataset:
Quantum Energy Regression using Scattering Transforms
Authors: Matthew Hirn, Nicolas Poilvert, Stéphane Mallat
Date: 2015-02-06 00:00:00
Tasks: regression

We present a novel approach to the regression of quantum mechanical energies
based on a scattering transform of an intermediate electron density
representation. A scattering transform is a deep convolution network computed
with a cascade of multiscale wavelet transforms. It possesses appropriate
invariant and stability properties for quantum energy regression. This new
framework removes fundamental limitations of Coulomb matrix based energy
regressions, and numerical experiments give state-of-the-art accuracy over
planar molecules.
Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
Authors: Aapo Hyvarinen, Hiroshi Morioka
Date: 2016-05-20 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Contrastive Learning, Time Series, Time Series Analysis

Nonlinear independent component analysis (ICA) provides an appealing
framework for unsupervised feature learning, but the models proposed so far are
not identifiable. Here, we first propose a new intuitive principle of
unsupervised deep learning from time series which uses the nonstationary
structure of the data. Our learning principle, time-contrastive learning (TCL),
finds a representation which allows optimal discrimination of time segments
(windows). Surprisingly, we show how TCL can be related to a nonlinear ICA
model, when ICA is redefined to include temporal nonstationarities. In
particular, we show that TCL combined with linear ICA estimates the nonlinear
ICA model up to point-wise transformations of the sources, and this solution is
unique --- thus providing the first identifiability result for nonlinear ICA
which is rigorous, constructive, as well as very general.
Superpixel Hierarchy
Authors: Xing Wei, Qingxiong Yang, Yihong Gong, Ming-Hsuan Yang, Narendra Ahuja
Date: 2016-05-20 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation, Superpixels

Superpixel segmentation is becoming ubiquitous in computer vision. In
practice, an object can either be represented by a number of segments in finer
levels of detail or included in a surrounding region at coarser levels of
detail, and thus a superpixel segmentation hierarchy is useful for applications
that require different levels of image segmentation detail depending on the
particular image objects segmented. Unfortunately, there is no method that can
generate all scales of superpixels accurately in real-time. As a result, a
simple yet effective algorithm named Super Hierarchy (SH) is proposed in this
paper. It is as accurate as the state-of-the-art but 1-2 orders of magnitude
faster. The proposed method can be directly integrated with recent efficient
edge detectors like the structured forest edges to significantly outperforms
the state-of-the-art in terms of segmentation accuracy. Quantitative and
qualitative evaluation on a number of computer vision applications was
conducted, demonstrating that the proposed method is the top performer.
Random sampling of bandlimited signals on graphs
Authors: Gilles Puy, Nicolas Tremblay, Rémi Gribonval, Pierre Vandergheynst
Date: 2015-11-16 00:00:00
Tasks: Decoder

We study the problem of sampling k-bandlimited signals on graphs. We propose
two sampling strategies that consist in selecting a small subset of nodes at
random. The first strategy is non-adaptive, i.e., independent of the graph
structure, and its performance depends on a parameter called the graph
coherence. On the contrary, the second strategy is adaptive but yields optimal
results. Indeed, no more than O(k log(k)) measurements are sufficient to ensure
an accurate and stable recovery of all k-bandlimited signals. This second
strategy is based on a careful choice of the sampling distribution, which can
be estimated quickly. Then, we propose a computationally efficient decoder to
reconstruct k-bandlimited signals from their samples. We prove that it yields
accurate reconstructions and that it is also stable to noise. Finally, we
conduct several experiments to test these techniques.
A single hidden layer feedforward network with only one neuron in the hidden layer can approximate any univariate function
Authors: Namig J. Guliyev, Vugar E. Ismailov
Date: 2015-12-31 00:00:00

The possibility of approximating a continuous function on a compact subset of
the real line by a feedforward single hidden layer neural network with a
sigmoidal activation function has been studied in many papers. Such networks
can approximate an arbitrary continuous function provided that an unlimited
number of neurons in a hidden layer is permitted. In this paper, we consider
constructive approximation on any finite interval of $\mathbb{R}$ by neural
networks with only one neuron in the hidden layer. We construct algorithmically
a smooth, sigmoidal, almost monotone activation function $\sigma$ providing
approximation to an arbitrary continuous function within any degree of
accuracy. This algorithm is implemented in a computer program, which computes
the value of $\sigma$ at any reasonable point of the real axis.
As Cool as a Cucumber: Towards a Corpus of Contemporary Similes in Serbian
Authors: Nikola Milosevic, Goran Nenadic
Date: 2016-05-20 00:00:00

Similes are natural language expressions used to compare unlikely things,
where the comparison is not taken literally. They are often used in everyday
communication and are an important part of cultural heritage. Having an
up-to-date corpus of similes is challenging, as they are constantly coined
and/or adapted to the contemporary times. In this paper we present a
methodology for semi-automated collection of similes from the world wide web
using text mining techniques. We expanded an existing corpus of traditional
similes (containing 333 similes) by collecting 446 additional expressions. We,
also, explore how crowdsourcing can be used to extract and curate new similes.
Online Optimization of Smoothed Piecewise Constant Functions
Authors: Vincent Cohen-Addad, Varun Kanade
Date: 2016-04-07 00:00:00

We study online optimization of smoothed piecewise constant functions over
the domain [0, 1). This is motivated by the problem of adaptively picking
parameters of learning algorithms as in the recently introduced framework by
Gupta and Roughgarden (2016). Majority of the machine learning literature has
focused on Lipschitz-continuous functions or functions with bounded gradients.
1 This is with good reason---any learning algorithm suffers linear regret even
against piecewise constant functions that are chosen adversarially, arguably
the simplest of non-Lipschitz continuous functions. The smoothed setting we
consider is inspired by the seminal work of Spielman and Teng (2004) and the
recent work of Gupta and Roughgarden---in this setting, the sequence of
functions may be chosen by an adversary, however, with some uncertainty in the
location of discontinuities. We give algorithms that achieve sublinear regret
in the full information and bandit settings.
SPSD Matrix Approximation vis Column Selection: Theories, Algorithms, and Extensions
Authors: Shusen Wang, Luo Luo, Zhihua Zhang
Date: 2014-06-22 00:00:00

Symmetric positive semidefinite (SPSD) matrix approximation is an important
problem with applications in kernel methods. However, existing SPSD matrix
approximation methods such as the Nystr\"om method only have weak error bounds.
In this paper we conduct in-depth studies of an SPSD matrix approximation model
and establish strong relative-error bounds. We call it the prototype model for
it has more efficient and effective extensions, and some of its extensions have
high scalability. Though the prototype model itself is not suitable for
large-scale data, it is still useful to study its properties, on which the
analysis of its extensions relies.
  This paper offers novel theoretical analysis, efficient algorithms, and a
highly accurate extension. First, we establish a lower error bound for the
prototype model and improve the error bound of an existing column selection
algorithm to match the lower bound. In this way, we obtain the first optimal
column selection algorithm for the prototype model. We also prove that the
prototype model is exact under certain conditions. Second, we develop a simple
column selection algorithm with a provable error bound. Third, we propose a
so-called spectral shifting model to make the approximation more accurate when
the eigenvalues of the matrix decay slowly, and the improvement is
theoretically quantified. The spectral shifting method can also be applied to
improve other SPSD matrix approximation models.
Brain Tumor Segmentation with Deep Neural Networks
Authors: Mohammad Havaei, Axel Davy, David Warde-Farley, Antoine Biard, Aaron Courville, Yoshua Bengio, Chris Pal, Pierre-Marc Jodoin, Hugo Larochelle
Date: 2015-05-13 00:00:00
Tasks: Brain Tumor Segmentation, Medical Image Segmentation, Tumor Segmentation

In this paper, we present a fully automatic brain tumor segmentation method
based on Deep Neural Networks (DNNs). The proposed networks are tailored to
glioblastomas (both low and high grade) pictured in MR images. By their very
nature, these tumors can appear anywhere in the brain and have almost any kind
of shape, size, and contrast. These reasons motivate our exploration of a
machine learning solution that exploits a flexible, high capacity DNN while
being extremely efficient. Here, we give a description of different model
choices that we've found to be necessary for obtaining competitive performance.
We explore in particular different architectures based on Convolutional Neural
Networks (CNN), i.e. DNNs specifically adapted to image data.
  We present a novel CNN architecture which differs from those traditionally
used in computer vision. Our CNN exploits both local features as well as more
global contextual features simultaneously. Also, different from most
traditional uses of CNNs, our networks use a final layer that is a
convolutional implementation of a fully connected layer which allows a 40 fold
speed up. We also describe a 2-phase training procedure that allows us to
tackle difficulties related to the imbalance of tumor labels. Finally, we
explore a cascade architecture in which the output of a basic CNN is treated as
an additional source of information for a subsequent CNN. Results reported on
the 2013 BRATS test dataset reveal that our architecture improves over the
currently published state-of-the-art while being over 30 times faster.
Convergence of Contrastive Divergence with Annealed Learning Rate in Exponential Family
Authors: Bai Jiang, Tung-Yu Wu, Wing H. Wong
Date: 2016-05-20 00:00:00

In our recent paper, we showed that in exponential family, contrastive
divergence (CD) with fixed learning rate will give asymptotically consistent
estimates \cite{wu2016convergence}. In this paper, we establish consistency and
convergence rate of CD with annealed learning rate $\eta_t$. Specifically,
suppose CD-$m$ generates the sequence of parameters $\{\theta_t\}_{t \ge 0}$
using an i.i.d. data sample $\mathbf{X}_1^n \sim p_{\theta^*}$ of size $n$,
then $\delta_n(\mathbf{X}_1^n) = \limsup_{t \to \infty} \Vert \sum_{s=t_0}^t
\eta_s \theta_s / \sum_{s=t_0}^t \eta_s - \theta^* \Vert$ converges in
probability to 0 at a rate of $1/\sqrt[3]{n}$. The number ($m$) of MCMC
transitions in CD only affects the coefficient factor of convergence rate. Our
proof is not a simple extension of the one in \cite{wu2016convergence}. which
depends critically on the fact that $\{\theta_t\}_{t \ge 0}$ is a homogeneous
Markov chain conditional on the observed sample $\mathbf{X}_1^n$. Under
annealed learning rate, the homogeneous Markov property is not available and we
have to develop an alternative approach based on super-martingales. Experiment
results of CD on a fully-visible $2\times 2$ Boltzmann Machine are provided to
demonstrate our theoretical results.
Efficient Feature-based Image Registration by Mapping Sparsified Surfaces
Authors: Chun Pang Yung, Gary P. T. Choi, Ke Chen, Lok Ming Lui
Date: 2016-05-20 00:00:00
Tasks: Image Registration

With the advancement in the digital camera technology, the use of high resolution images and videos has been widespread in the modern society. In particular, image and video frame registration is frequently applied in computer graphics and film production. However, conventional registration approaches usually require long computational time for high resolution images and video frames. This hinders the application of the registration approaches in the modern industries. In this work, we first propose a new image representation method to accelerate the registration process by triangulating the images effectively. For each high resolution image or video frame, we compute an optimal coarse triangulation which captures the important features of the image. Then, we apply a surface registration algorithm to obtain a registration map which is used to compute the registration of the high resolution image. Experimental results suggest that our overall algorithm is efficient and capable to achieve a high compression rate while the accuracy of the registration is well retained when compared with the conventional grid-based approach. Also, the computational time of the registration is significantly reduced using our triangulation-based approach.
Fully Convolutional Networks for Semantic Segmentation
Authors: Evan Shelhamer, Jonathan Long, Trevor Darrell
Date: 2016-05-20 00:00:00
Proceeding: CVPR 2015
Tasks: Real-Time Semantic Segmentation, Scene Segmentation, Segmentation, Semantic Segmentation, Video Semantic Segmentation

Convolutional networks are powerful visual models that yield hierarchies of
features. We show that convolutional networks by themselves, trained
end-to-end, pixels-to-pixels, improve on the previous best result in semantic
segmentation. Our key insight is to build "fully convolutional" networks that
take input of arbitrary size and produce correspondingly-sized output with
efficient inference and learning. We define and detail the space of fully
convolutional networks, explain their application to spatially dense prediction
tasks, and draw connections to prior models. We adapt contemporary
classification networks (AlexNet, the VGG net, and GoogLeNet) into fully
convolutional networks and transfer their learned representations by
fine-tuning to the segmentation task. We then define a skip architecture that
combines semantic information from a deep, coarse layer with appearance
information from a shallow, fine layer to produce accurate and detailed
segmentations. Our fully convolutional network achieves improved segmentation
of PASCAL VOC (30% relative improvement to 67.2% mean IU on 2012), NYUDv2, SIFT
Flow, and PASCAL-Context, while inference takes one tenth of a second for a
typical image.
Faster Projection-free Convex Optimization over the Spectrahedron
Authors: Dan Garber
Date: 2016-05-20 00:00:00
Proceeding: NeurIPS 2016 12

Minimizing a convex function over the spectrahedron, i.e., the set of all
positive semidefinite matrices with unit trace, is an important optimization
task with many applications in optimization, machine learning, and signal
processing. It is also notoriously difficult to solve in large-scale since
standard techniques require expensive matrix decompositions. An alternative, is
the conditional gradient method (aka Frank-Wolfe algorithm) that regained much
interest in recent years, mostly due to its application to this specific
setting. The key benefit of the CG method is that it avoids expensive matrix
decompositions all together, and simply requires a single eigenvector
computation per iteration, which is much more efficient. On the downside, the
CG method, in general, converges with an inferior rate. The error for
minimizing a $\beta$-smooth function after $t$ iterations scales like
$\beta/t$. This convergence rate does not improve even if the function is also
strongly convex.
  In this work we present a modification of the CG method tailored for convex
optimization over the spectrahedron. The per-iteration complexity of the method
is essentially identical to that of the standard CG method: only a single
eigenvecor computation is required. For minimizing an $\alpha$-strongly convex
and $\beta$-smooth function, the expected approximation error of the method
after $t$ iterations is: $$O\left({\min\{\frac{\beta{}}{t}
,\left({\frac{\beta\sqrt{\textrm{rank}(\textbf{X}^*)}}{\alpha^{1/4}t}}\right)^{4/3},
\left({\frac{\beta}{\sqrt{\alpha}\lambda_{\min}(\textbf{X}^*)t}}\right)^{2}\}}\right)
,$$ where $\textbf{X}^*$ is the optimal solution. To the best of our knowledge,
this is the first result that attains provably faster convergence rates for a
CG variant for optimization over the spectrahedron. We also present encouraging
preliminary empirical results.
Re-ranking Object Proposals for Object Detection in Automatic Driving
Authors: Zhun Zhong, Mingyi Lei, Shaozi Li, Jianping Fan
Date: 2016-05-19 00:00:00
Tasks: Object, object-detection, Object Detection, Re-Ranking, Semantic Segmentation

Object detection often suffers from a plenty of bootless proposals, selecting
high quality proposals remains a great challenge. In this paper, we propose a
semantic, class-specific approach to re-rank object proposals, which can
consistently improve the recall performance even with less proposals. We first
extract features for each proposal including semantic segmentation, stereo
information, contextual information, CNN-based objectness and low-level cue,
and then score them using class-specific weights learnt by Structured SVM. The
advantages of the proposed model are twofold: 1) it can be easily merged to
existing generators with few computational costs, and 2) it can achieve high
recall rate uner strict critical even using less proposals. Experimental
evaluation on the KITTI benchmark demonstrates that our approach significantly
improves existing popular generators on recall performance. Moreover, in the
experiment conducted for object detection, even with 1,500 proposals, our
approach can still have higher average precision (AP) than baselines with 5,000
proposals.
Modelling Interaction of Sentence Pair with coupled-LSTMs
Authors: Pengfei Liu, Xipeng Qiu, Xuanjing Huang
Date: 2016-05-18 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Sentence

Recently, there is rising interest in modelling the interactions of two
sentences with deep neural networks. However, most of the existing methods
encode two sequences with separate encoders, in which a sentence is encoded
with little or no information from the other sentence. In this paper, we
propose a deep architecture to model the strong interaction of sentence pair
with two coupled-LSTMs. Specifically, we introduce two coupled ways to model
the interdependences of two LSTMs, coupling the local contextualized
interactions of two sentences. We then aggregate these interactions and use a
dynamic pooling to select the most informative features. Experiments on two
very large datasets demonstrate the efficacy of our proposed architecture and
its superiority to state-of-the-art methods.
Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods
Authors: Mehrtash Harandi, Mathieu Salzmann, Richard Hartley
Date: 2016-05-20 00:00:00
Tasks: Dimensionality Reduction

Representing images and videos with Symmetric Positive Definite (SPD)
matrices, and considering the Riemannian geometry of the resulting space, has
been shown to yield high discriminative power in many visual recognition tasks.
Unfortunately, computation on the Riemannian manifold of SPD matrices
-especially of high-dimensional ones- comes at a high cost that limits the
applicability of existing techniques. In this paper, we introduce algorithms
able to handle high-dimensional SPD matrices by constructing a
lower-dimensional SPD manifold. To this end, we propose to model the mapping
from the high-dimensional SPD manifold to the low-dimensional one with an
orthonormal projection. This lets us formulate dimensionality reduction as the
problem of finding a projection that yields a low-dimensional manifold either
with maximum discriminative power in the supervised scenario, or with maximum
variance of the data in the unsupervised one. We show that learning can be
expressed as an optimization problem on a Grassmann manifold and discuss fast
solutions for special cases. Our evaluation on several classification tasks
evidences that our approach leads to a significant accuracy gain over
state-of-the-art methods.
Variational hybridization and transformation for large inaccurate noisy-or networks
Authors: Yusheng Xie, Nan Du, Wei Fan, Jing Zhai, Weicheng Zhu
Date: 2016-05-20 00:00:00
Tasks: Variational Inference

Variational inference provides approximations to the computationally
intractable posterior distribution in Bayesian networks. A prominent medical
application of noisy-or Bayesian network is to infer potential diseases given
observed symptoms. Previous studies focus on approximating a handful of
complicated pathological cases using variational transformation. Our goal is to
use variational transformation as part of a novel hybridized inference for
serving reliable and real time diagnosis at web scale. We propose a hybridized
inference that allows variational parameters to be estimated without disease
posteriors or priors, making the inference faster and much of its computation
recyclable. In addition, we propose a transformation ranking algorithm that is
very stable to large variances in network prior probabilities, a common issue
that arises in medical applications of Bayesian networks. In experiments, we
perform comparative study on a large real life medical network and scalability
study on a much larger (36,000x) synthesized network.
Fine-Grained Classification of Pedestrians in Video: Benchmark and State of the Art
Authors: David Hall, Pietro Perona
Date: 2016-05-20 00:00:00
Proceeding: CVPR 2015 6
Tasks: Benchmarking, General Classification, Pose Estimation

A video dataset that is designed to study fine-grained categorisation of
pedestrians is introduced. Pedestrians were recorded "in-the-wild" from a
moving vehicle. Annotations include bounding boxes, tracks, 14 keypoints with
occlusion information and the fine-grained categories of age (5 classes), sex
(2 classes), weight (3 classes) and clothing style (4 classes). There are a
total of 27,454 bounding box and pose labels across 4222 tracks. This dataset
is designed to train and test algorithms for fine-grained categorisation of
people, it is also useful for benchmarking tracking, detection and pose
estimation of pedestrians. State-of-the-art algorithms for fine-grained
classification and pose estimation were tested using the dataset and the
results are reported as a useful performance baseline.
Evaluation System for a Bayesian Optimization Service
Authors: Ian Dewancker, Michael McCourt, Scott Clark, Patrick Hayes, Alexandra Johnson, George Ke
Date: 2016-05-19 00:00:00
Tasks: Bayesian Optimization, BIG-bench Machine Learning, Hyperparameter Optimization

Bayesian optimization is an elegant solution to the hyperparameter
optimization problem in machine learning. Building a reliable and robust
Bayesian optimization service requires careful testing methodology and sound
statistical analysis. In this talk we will outline our development of an
evaluation framework to rigorously test and measure the impact of changes to
the SigOpt optimization service. We present an overview of our evaluation
system and discuss how this framework empowers our research engineers to
confidently and quickly make changes to our core optimization engine