pwc-archive/papers-with-abstracts huggingface dataset:
Semantic Segmentation with Boundary Neural Fields
Authors: Gedas Bertasius, Jianbo Shi, Lorenzo Torresani
Date: 2015-11-09 00:00:00
Proceeding: CVPR 2016 6
Tasks: Boundary Detection, global-optimization, Object Localization, Segmentation, Semantic Segmentation

The state-of-the-art in semantic segmentation is currently represented by
fully convolutional networks (FCNs). However, FCNs use large receptive fields
and many pooling layers, both of which cause blurring and low spatial
resolution in the deep layers. As a result FCNs tend to produce segmentations
that are poorly localized around object boundaries. Prior work has attempted to
address this issue in post-processing steps, for example using a color-based
CRF on top of the FCN predictions. However, these approaches require additional
parameters and low-level features that are difficult to tune and integrate into
the original network architecture. Additionally, most CRFs use color-based
pixel affinities, which are not well suited for semantic segmentation and lead
to spatially disjoint predictions.
  To overcome these problems, we introduce a Boundary Neural Field (BNF), which
is a global energy model integrating FCN predictions with boundary cues. The
boundary information is used to enhance semantic segment coherence and to
improve object localization. Specifically, we first show that the convolutional
filters of semantic FCNs provide good features for boundary detection. We then
employ the predicted boundaries to define pairwise potentials in our energy.
Finally, we show that our energy decomposes semantic segmentation into multiple
binary problems, which can be relaxed for efficient global optimization. We
report extensive experiments demonstrating that minimization of our global
boundary-based energy yields results superior to prior globalization methods,
both quantitatively as well as qualitatively.
Additive Approximations in High Dimensional Nonparametric Regression via the SALSA
Authors: Kirthevasan Kandasamy, Yao-Liang Yu
Date: 2016-01-31 00:00:00
Tasks: Additive models, regression, Vocal Bursts Intensity Prediction

High dimensional nonparametric regression is an inherently difficult problem
with known lower bounds depending exponentially in dimension. A popular
strategy to alleviate this curse of dimensionality has been to use additive
models of \emph{first order}, which model the regression function as a sum of
independent functions on each dimension. Though useful in controlling the
variance of the estimate, such models are often too restrictive in practical
settings. Between non-additive models which often have large variance and first
order additive models which have large bias, there has been little work to
exploit the trade-off in the middle via additive models of intermediate order.
In this work, we propose SALSA, which bridges this gap by allowing interactions
between variables, but controls model capacity by limiting the order of
interactions. SALSA minimises the residual sum of squares with squared RKHS
norm penalties. Algorithmically, it can be viewed as Kernel Ridge Regression
with an additive kernel. When the regression function is additive, the excess
risk is only polynomial in dimension. Using the Girard-Newton formulae, we
efficiently sum over a combinatorial number of terms in the additive expansion.
Via a comparison on $15$ real datasets, we show that our method is competitive
against $21$ other alternatives.
Unsupervised Deep Embedding for Clustering Analysis
Authors: Junyuan Xie, Ross Girshick, Ali Farhadi
Date: 2015-11-19 00:00:00
Tasks: Clustering, Image Clustering, Unsupervised Image Classification

Clustering is central to many data-driven application domains and has been
studied extensively in terms of distance functions and grouping algorithms.
Relatively little work has focused on learning representations for clustering.
In this paper, we propose Deep Embedded Clustering (DEC), a method that
simultaneously learns feature representations and cluster assignments using
deep neural networks. DEC learns a mapping from the data space to a
lower-dimensional feature space in which it iteratively optimizes a clustering
objective. Our experimental evaluations on image and text corpora show
significant improvement over state-of-the-art methods.
Strongly-Typed Recurrent Neural Networks
Authors: David Balduzzi, Muhammad Ghifary
Date: 2016-02-06 00:00:00

Recurrent neural networks are increasing popular models for sequential
learning. Unfortunately, although the most effective RNN architectures are
perhaps excessively complicated, extensive searches have not found simpler
alternatives. This paper imports ideas from physics and functional programming
into RNN design to provide guiding principles. From physics, we introduce type
constraints, analogous to the constraints that forbids adding meters to
seconds. From functional programming, we require that strongly-typed
architectures factorize into stateless learnware and state-dependent firmware,
reducing the impact of side-effects. The features learned by strongly-typed
nets have a simple semantic interpretation via dynamic average-pooling on
one-dimensional convolutions. We also show that strongly-typed gradients are
better behaved than in classical architectures, and characterize the
representational power of strongly-typed nets. Finally, experiments show that,
despite being more constrained, strongly-typed architectures achieve lower
training and comparable generalization error to classical architectures.
Adaptive Newton Method for Empirical Risk Minimization to Statistical Accuracy
Authors: Aryan Mokhtari, Alejandro Ribeiro
Date: 2016-05-24 00:00:00
Proceeding: NeurIPS 2016 12

We consider empirical risk minimization for large-scale datasets. We
introduce Ada Newton as an adaptive algorithm that uses Newton's method with
adaptive sample sizes. The main idea of Ada Newton is to increase the size of
the training set by a factor larger than one in a way that the minimization
variable for the current training set is in the local neighborhood of the
optimal argument of the next training set. This allows to exploit the quadratic
convergence property of Newton's method and reach the statistical accuracy of
each training set with only one iteration of Newton's method. We show
theoretically and empirically that Ada Newton can double the size of the
training set in each iteration to achieve the statistical accuracy of the full
training set with about two passes over the dataset.
Blind Analysis of CT Image Noise Using Residual Denoised Images
Authors: Sohini Roychowdhury, Nathan Hollraft, Adam Alessio
Date: 2016-05-24 00:00:00
Tasks: Noise Estimation

CT protocol design and quality control would benefit from automated tools to
estimate the quality of generated CT images. These tools could be used to
identify erroneous CT acquisitions or refine protocols to achieve certain
signal to noise characteristics. This paper investigates blind estimation
methods to determine global signal strength and noise levels in chest CT
images. Methods: We propose novel performance metrics corresponding to the
accuracy of noise and signal estimation. We implement and evaluate the noise
estimation performance of six spatial- and frequency- based methods, derived
from conventional image filtering algorithms. Algorithms were tested on patient
data sets from whole-body repeat CT acquisitions performed with a higher and
lower dose technique over the same scan region. Results: The proposed
performance metrics can evaluate the relative tradeoff of filter parameters and
noise estimation performance. The proposed automated methods tend to
underestimate CT image noise at low-flux levels. Initial application of
methodology suggests that anisotropic diffusion and Wavelet-transform based
filters provide optimal estimates of noise. Furthermore, methodology does not
provide accurate estimates of absolute noise levels, but can provide estimates
of relative change and/or trends in noise levels.
Posterior Dispersion Indices
Authors: Alp Kucukelbir, David M. Blei
Date: 2016-05-24 00:00:00

Probabilistic modeling is cyclical: we specify a model, infer its posterior,
and evaluate its performance. Evaluation drives the cycle, as we revise our
model based on how it performs. This requires a metric. Traditionally,
predictive accuracy prevails. Yet, predictive accuracy does not tell the whole
story. We propose to evaluate a model through posterior dispersion. The idea is
to analyze how each datapoint fares in relation to posterior uncertainty around
the hidden structure. We propose a family of posterior dispersion indices (PDI)
that capture this idea. A PDI identifies rich patterns of model mismatch in
three real data examples: voting preferences, supermarket shopping, and
population genetics.
Correlation Clustering and Biclustering with Locally Bounded Errors
Authors: Gregory J. Puleo, Olgica Milenkovic
Date: 2015-06-26 00:00:00
Tasks: Clustering

We consider a generalized version of the correlation clustering problem,
defined as follows. Given a complete graph $G$ whose edges are labeled with $+$
or $-$, we wish to partition the graph into clusters while trying to avoid
errors: $+$ edges between clusters or $-$ edges within clusters. Classically,
one seeks to minimize the total number of such errors. We introduce a new
framework that allows the objective to be a more general function of the number
of errors at each vertex (for example, we may wish to minimize the number of
errors at the worst vertex) and provide a rounding algorithm which converts
"fractional clusterings" into discrete clusterings while causing only a
constant-factor blowup in the number of errors at each vertex. This rounding
algorithm yields constant-factor approximation algorithms for the discrete
problem under a wide variety of objective functions.
An Efficient Multilinear Optimization Framework for Hypergraph Matching
Authors: Quynh Nguyen, Francesco Tudisco, Antoine Gautier, Matthias Hein
Date: 2015-11-09 00:00:00
Tasks: Hypergraph Matching

Hypergraph matching has recently become a popular approach for solving
correspondence problems in computer vision as it allows to integrate
higher-order geometric information. Hypergraph matching can be formulated as a
third-order optimization problem subject to the assignment constraints which
turns out to be NP-hard. In recent work, we have proposed an algorithm for
hypergraph matching which first lifts the third-order problem to a fourth-order
problem and then solves the fourth-order problem via optimization of the
corresponding multilinear form. This leads to a tensor block coordinate ascent
scheme which has the guarantee of providing monotonic ascent in the original
matching score function and leads to state-of-the-art performance both in terms
of achieved matching score and accuracy. In this paper we show that the lifting
step to a fourth-order problem can be avoided yielding a third-order scheme
with the same guarantees and performance but being two times faster. Moreover,
we introduce a homotopy type method which further improves the performance.
Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
Authors: Steven K. Esser, Paul A. Merolla, John V. Arthur, Andrew S. Cassidy, Rathinakumar Appuswamy, Alexander Andreopoulos, David J. Berg, Jeffrey L. McKinstry, Timothy Melano, Davis R. Barch, Carmelo Di Nolfo, Pallab Datta, Arnon Amir, Brian Taba, Myron D. Flickner, Dharmendra S. Modha
Date: 2016-03-28 00:00:00

Deep networks are now able to achieve human-level performance on a broad
spectrum of recognition tasks. Independently, neuromorphic computing has now
demonstrated unprecedented energy-efficiency through a new chip architecture
based on spiking neurons, low precision synapses, and a scalable communication
network. Here, we demonstrate that neuromorphic computing, despite its novel
architectural primitives, can implement deep convolution networks that i)
approach state-of-the-art classification accuracy across 8 standard datasets,
encompassing vision and speech, ii) perform inference while preserving the
hardware's underlying energy-efficiency and high throughput, running on the
aforementioned datasets at between 1200 and 2600 frames per second and using
between 25 and 275 mW (effectively > 6000 frames / sec / W) and iii) can be
specified and trained using backpropagation with the same ease-of-use as
contemporary deep learning. For the first time, the algorithmic power of deep
learning can be merged with the efficiency of neuromorphic processors, bringing
the promise of embedded, intelligent, brain-inspired computing one step closer.
Learning 3D Deformation of Animals from 2D Images
Authors: Angjoo Kanazawa, Shahar Kovalsky, Ronen Basri, David W. Jacobs
Date: 2015-07-28 00:00:00

Understanding how an animal can deform and articulate is essential for a
realistic modification of its 3D model. In this paper, we show that such
information can be learned from user-clicked 2D images and a template 3D model
of the target animal. We present a volumetric deformation framework that
produces a set of new 3D models by deforming a template 3D model according to a
set of user-clicked images. Our framework is based on a novel locally-bounded
deformation energy, where every local region has its own stiffness value that
bounds how much distortion is allowed at that location. We jointly learn the
local stiffness bounds as we deform the template 3D mesh to match each
user-clicked image. We show that this seemingly complex task can be solved as a
sequence of convex optimization problems. We demonstrate the effectiveness of
our approach on cats and horses, which are highly deformable and articulated
animals. Our framework produces new 3D models of animals that are significantly
more plausible than methods without learned stiffness.
Towards Bin Packing (preliminary problem survey, models with multiset estimates)
Authors: Mark Sh. Levin
Date: 2016-05-24 00:00:00
Tasks: Relation, Survey

The paper described a generalized integrated glance to bin packing problems
including a brief literature survey and some new problem formulations for the
cases of multiset estimates of items. A new systemic viewpoint to bin packing
problems is suggested: (a) basic element sets (item set, bin set, item subset
assigned to bin), (b) binary relation over the sets: relation over item set as
compatibility, precedence, dominance; relation over items and bins (i.e.,
correspondence of items to bins). A special attention is targeted to the
following versions of bin packing problems: (a) problem with multiset estimates
of items, (b) problem with colored items (and some close problems). Applied
examples of bin packing problems are considered: (i) planning in paper industry
(framework of combinatorial problems), (ii) selection of information messages,
(iii) packing of messages/information packages in WiMAX communication system
(brief description).
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Authors: Jakob N. Foerster, Yannis M. Assael, Nando de Freitas, Shimon Whiteson
Date: 2016-05-21 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Multi-agent Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We consider the problem of multiple agents sensing and acting in environments
with the goal of maximising their shared utility. In these environments, agents
must learn communication protocols in order to share information that is needed
to solve the tasks. By embracing deep neural networks, we are able to
demonstrate end-to-end learning of protocols in complex environments inspired
by communication riddles and multi-agent computer vision problems with partial
observability. We propose two approaches for learning in these domains:
Reinforced Inter-Agent Learning (RIAL) and Differentiable Inter-Agent Learning
(DIAL). The former uses deep Q-learning, while the latter exploits the fact
that, during learning, agents can backpropagate error derivatives through
(noisy) communication channels. Hence, this approach uses centralised learning
but decentralised execution. Our experiments introduce new environments for
studying the learning of communication protocols and present a set of
engineering innovations that are essential for success in these domains.
ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation
Authors: Francesco Visin, Marco Ciccone, Adriana Romero, Kyle Kastner, Kyunghyun Cho, Yoshua Bengio, Matteo Matteucci, Aaron Courville
Date: 2015-11-22 00:00:00
Tasks: image-classification, Segmentation, Semantic Segmentation, Structured Prediction

We propose a structured prediction architecture, which exploits the local
generic features extracted by Convolutional Neural Networks and the capacity of
Recurrent Neural Networks (RNN) to retrieve distant dependencies. The proposed
architecture, called ReSeg, is based on the recently introduced ReNet model for
image classification. We modify and extend it to perform the more challenging
task of semantic segmentation. Each ReNet layer is composed of four RNN that
sweep the image horizontally and vertically in both directions, encoding
patches or activations, and providing relevant global information. Moreover,
ReNet layers are stacked on top of pre-trained convolutional layers, benefiting
from generic local features. Upsampling layers follow ReNet layers to recover
the original image resolution in the final predictions. The proposed ReSeg
architecture is efficient, flexible and suitable for a variety of semantic
segmentation tasks. We evaluate ReSeg on several widely-used semantic
segmentation datasets: Weizmann Horse, Oxford Flower, and CamVid; achieving
state-of-the-art performance. Results show that ReSeg can act as a suitable
architecture for semantic segmentation tasks, and may have further applications
in other structured prediction problems. The source code and model
hyperparameters are available on https://github.com/fvisin/reseg.
Deconstructing the Ladder Network Architecture
Authors: Mohammad Pezeshki, Linxi Fan, Philemon Brakel, Aaron Courville, Yoshua Bengio
Date: 2015-11-19 00:00:00
Tasks: Denoising

The Manual labeling of data is and will remain a costly endeavor. For this
reason, semi-supervised learning remains a topic of practical importance. The
recently proposed Ladder Network is one such approach that has proven to be
very successful. In addition to the supervised objective, the Ladder Network
also adds an unsupervised objective corresponding to the reconstruction costs
of a stack of denoising autoencoders. Although the empirical results are
impressive, the Ladder Network has many components intertwined, whose
contributions are not obvious in such a complex architecture. In order to help
elucidate and disentangle the different ingredients in the Ladder Network
recipe, this paper presents an extensive experimental investigation of variants
of the Ladder Network in which we replace or remove individual components to
gain more insight into their relative importance. We find that all of the
components are necessary for achieving optimal performance, but they do not
contribute equally. For semi-supervised tasks, we conclude that the most
important contribution is made by the lateral connection, followed by the
application of noise, and finally the choice of what we refer to as the
`combinator function' in the decoder path. We also find that as the number of
labeled training examples increases, the lateral connections and reconstruction
criterion become less important, with most of the improvement in generalization
being due to the injection of noise in each layer. Furthermore, we present a
new type of combinator function that outperforms the original design in both
fully- and semi-supervised tasks, reducing record test error rates on
Permutation-Invariant MNIST to 0.57% for the supervised setting, and to 0.97%
and 1.0% for semi-supervised settings with 1000 and 100 labeled examples
respectively.
A note on privacy preserving iteratively reweighted least squares
Authors: Mijung Park, Max Welling
Date: 2016-05-24 00:00:00
Tasks: Privacy Preserving, Sensitivity

Iteratively reweighted least squares (IRLS) is a widely-used method in
machine learning to estimate the parameters in the generalised linear models.
In particular, IRLS for L1 minimisation under the linear model provides a
closed-form solution in each step, which is a simple multiplication between the
inverse of the weighted second moment matrix and the weighted first moment
vector. When dealing with privacy sensitive data, however, developing a privacy
preserving IRLS algorithm faces two challenges. First, due to the inversion of
the second moment matrix, the usual sensitivity analysis in differential
privacy incorporating a single datapoint perturbation gets complicated and
often requires unrealistic assumptions. Second, due to its iterative nature, a
significant cumulative privacy loss occurs. However, adding a high level of
noise to compensate for the privacy loss hinders from getting accurate
estimates. Here, we develop a practical algorithm that overcomes these
challenges and outputs privatised and accurate IRLS solutions. In our method,
we analyse the sensitivity of each moments separately and treat the matrix
inversion and multiplication as a post-processing step, which simplifies the
sensitivity analysis. Furthermore, we apply the {\it{concentrated differential
privacy}} formalism, a more relaxed version of differential privacy, which
requires adding a significantly less amount of noise for the same level of
privacy guarantee, compared to the conventional and advanced compositions of
differentially private mechanisms.
Monolingually Derived Phrase Scores for Phrase Based SMT Using Neural Networks Vector Representations
Authors: Amir Pouya Aghasadeghi, Mohadeseh Bastan
Date: 2015-06-01 00:00:00
Tasks: Machine Translation, Translation

In this paper, we propose two new features for estimating phrase-based
machine translation parameters from mainly monolingual data. Our method is
based on two recently introduced neural network vector representation models
for words and sentences. It is the first time that these models have been used
in an end to end phrase-based machine translation system. Scores obtained from
our method can recover more than 80% of BLEU loss caused by removing phrase
table probabilities. We also show that our features combined with the phrase
table probabilities improve the BLEU score by absolute 0.74 points.