pwc-archive/papers-with-abstracts huggingface dataset:
An equivalence between high dimensional Bayes optimal inference and M-estimation
Authors: Madhu Advani, Surya Ganguli
Date: 2016-09-22 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Vocal Bursts Intensity Prediction

When recovering an unknown signal from noisy measurements, the computational
difficulty of performing optimal Bayesian MMSE (minimum mean squared error)
inference often necessitates the use of maximum a posteriori (MAP) inference, a
special case of regularized M-estimation, as a surrogate. However, MAP is
suboptimal in high dimensions, when the number of unknown signal components is
similar to the number of measurements. In this work we demonstrate, when the
signal distribution and the likelihood function associated with the noise are
both log-concave, that optimal MMSE performance is asymptotically achievable
via another M-estimation procedure. This procedure involves minimizing convex
loss and regularizer functions that are nonlinearly smoothed versions of the
widely applied MAP optimization problem. Our findings provide a new heuristic
derivation and interpretation for recent optimal M-estimators found in the
setting of linear measurements and additive noise, and further extend these
results to nonlinear measurements with non-additive noise. We numerically
demonstrate superior performance of our optimal M-estimators relative to MAP.
Overall, at the heart of our work is the revelation of a remarkable equivalence
between two seemingly very different computational problems: namely that of
high dimensional Bayesian integration underlying MMSE inference, and high
dimensional convex optimization underlying M-estimation. In essence we show
that the former difficult integral may be computed by solving the latter,
simpler optimization problem.
An Impossibility Result for Reconstruction in a Degree-Corrected Planted-Partition Model
Authors: Lennart Gulikers, Marc Lelarge, Laurent Massoulié
Date: 2015-11-02 00:00:00
Tasks: Stochastic Block Model

We consider the Degree-Corrected Stochastic Block Model (DC-SBM): a random
graph on $n$ nodes, having i.i.d. weights $(\phi_u)_{u=1}^n$ (possibly
heavy-tailed), partitioned into $q \geq 2$ asymptotically equal-sized clusters.
The model parameters are two constants $a,b > 0$ and the finite second moment
of the weights $\Phi^{(2)}$. Vertices $u$ and $v$ are connected by an edge with
probability $\frac{\phi_u \phi_v}{n}a$ when they are in the same class and with
probability $\frac{\phi_u \phi_v}{n}b$ otherwise.
  We prove that it is information-theoretically impossible to estimate the
clusters in a way positively correlated with the true community structure when
$(a-b)^2 \Phi^{(2)} \leq q(a+b)$.
  As by-products of our proof we obtain $(1)$ a precise coupling result for
local neighbourhoods in DC-SBM's, that we use in a follow up paper [Gulikers et
al., 2017] to establish a law of large numbers for local-functionals and $(2)$
that long-range interactions are weak in (power-law) DC-SBM's.
Customized Facial Constant Positive Air Pressure (CPAP) Masks
Authors: Matan Sela, Nadav Toledo, Yaron Honen, Ron Kimmel
Date: 2016-09-22 00:00:00

Sleep apnea is a syndrome that is characterized by sudden breathing halts
while sleeping. One of the common treatments involves wearing a mask that
delivers continuous air flow into the nostrils so as to maintain a steady air
pressure. These masks are designed for an average facial model and are often
difficult to adjust due to poor fit to the actual patient. The incompatibility
is characterized by gaps between the mask and the face, which deteriorates the
impermeability of the mask and leads to air leakage. We suggest a fully
automatic approach for designing a personalized nasal mask interface using a
facial depth scan. The interfaces generated by the proposed method accurately
fit the geometry of the scanned face, and are easy to manufacture. The proposed
method utilizes cheap commodity depth sensors and 3D printing technologies to
efficiently design and manufacture customized masks for patients suffering from
sleep apnea.
Abstractive Meeting Summarization UsingDependency Graph Fusion
Authors: Siddhartha Banerjee, Prasenjit Mitra, Kazunari Sugiyama
Date: 2016-09-22 00:00:00
Tasks: Meeting Summarization, Sentence, Sentence Fusion

Automatic summarization techniques on meeting conversations developed so far
have been primarily extractive, resulting in poor summaries. To improve this,
we propose an approach to generate abstractive summaries by fusing important
content from several utterances. Any meeting is generally comprised of several
discussion topic segments. For each topic segment within a meeting
conversation, we aim to generate a one sentence summary from the most important
utterances using an integer linear programming-based sentence fusion approach.
Experimental results show that our method can generate more informative
summaries than the baselines.
Multi-document abstractive summarization using ILP based multi-sentence compression
Authors: Siddhartha Banerjee, Prasenjit Mitra, Kazunari Sugiyama
Date: 2016-09-22 00:00:00
Tasks: Abstractive Text Summarization, Document Summarization, Informativeness, Multi-Document Summarization, Sentence, Sentence Compression

Abstractive summarization is an ideal form of summarization since it can
synthesize information from multiple documents to create concise informative
summaries. In this work, we aim at developing an abstractive summarizer. First,
our proposed approach identifies the most important document in the
multi-document set. The sentences in the most important document are aligned to
sentences in other documents to generate clusters of similar sentences. Second,
we generate K-shortest paths from the sentences in each cluster using a
word-graph structure. Finally, we select sentences from the set of shortest
paths generated from all the clusters employing a novel integer linear
programming (ILP) model with the objective of maximizing information content
and readability of the final summary. Our ILP model represents the shortest
paths as binary variables and considers the length of the path, information
score and linguistic quality score in the objective function. Experimental
results on the DUC 2004 and 2005 multi-document summarization datasets show
that our proposed approach outperforms all the baselines and state-of-the-art
extractive summarizers as measured by the ROUGE scores. Our method also
outperforms a recent abstractive summarization technique. In manual evaluation,
our approach also achieves promising results on informativeness and
readability.
Generating Abstractive Summaries from Meeting Transcripts
Authors: Siddhartha Banerjee, Prasenjit Mitra, Kazunari Sugiyama
Date: 2016-09-22 00:00:00
Tasks: Document Summarization, Sentence, Text Generation

Summaries of meetings are very important as they convey the essential content
of discussions in a concise form. Generally, it is time consuming to read and
understand the whole documents. Therefore, summaries play an important role as
the readers are interested in only the important context of discussions. In
this work, we address the task of meeting document summarization. Automatic
summarization systems on meeting conversations developed so far have been
primarily extractive, resulting in unacceptable summaries that are hard to
read. The extracted utterances contain disfluencies that affect the quality of
the extractive summaries. To make summaries much more readable, we propose an
approach to generating abstractive summaries by fusing important content from
several utterances. We first separate meeting transcripts into various topic
segments, and then identify the important utterances in each segment using a
supervised learning approach. The important utterances are then combined
together to generate a one-sentence summary. In the text generation step, the
dependency parses of the utterances in each segment are combined together to
create a directed graph. The most informative and well-formed sub-graph
obtained by integer linear programming (ILP) is selected to generate a
one-sentence summary for each topic segment. The ILP formulation reduces
disfluencies by leveraging grammatical relations that are more prominent in
non-conversational style of text, and therefore generates summaries that is
comparable to human-written abstractive summaries. Experimental results show
that our method can generate more informative summaries than the baselines. In
addition, readability assessments by human judges as well as log-likelihood
estimates obtained from the dependency parser show that our generated summaries
are significantly readable and well-formed.
Is the deconvolution layer the same as a convolutional layer?
Authors: Wenzhe Shi, Jose Caballero, Lucas Theis, Ferenc Huszar, Andrew Aitken, Christian Ledig, Zehan Wang
Date: 2016-09-22 00:00:00

In this note, we want to focus on aspects related to two questions most
people asked us at CVPR about the network we presented. Firstly, What is the
relationship between our proposed layer and the deconvolution layer? And
secondly, why are convolutions in low-resolution (LR) space a better choice?
These are key questions we tried to answer in the paper, but we were not able
to go into as much depth and clarity as we would have liked in the space
allowance. To better answer these questions in this note, we first discuss the
relationships between the deconvolution layer in the forms of the transposed
convolution layer, the sub-pixel convolutional layer and our efficient
sub-pixel convolutional layer. We will refer to our efficient sub-pixel
convolutional layer as a convolutional layer in LR space to distinguish it from
the common sub-pixel convolutional layer. We will then show that for a fixed
computational budget and complexity, a network with convolutions exclusively in
LR space has more representation power at the same speed than a network that
first upsamples the input in high resolution space.
TI-POOLING: transformation-invariant pooling for feature learning in Convolutional Neural Networks
Authors: Dmitry Laptev, Nikolay Savinov, Joachim M. Buhmann, Marc Pollefeys
Date: 2016-04-21 00:00:00
Proceeding: CVPR 2016 6

In this paper we present a deep neural network topology that incorporates a
simple to implement transformation invariant pooling operator (TI-POOLING).
This operator is able to efficiently handle prior knowledge on nuisance
variations in the data, such as rotation or scale changes. Most current methods
usually make use of dataset augmentation to address this issue, but this
requires larger number of model parameters and more training data, and results
in significantly increased training time and larger chance of under- or
overfitting. The main reason for these drawbacks is that the learned model
needs to capture adequate features for all the possible transformations of the
input. On the other hand, we formulate features in convolutional neural
networks to be transformation-invariant. We achieve that using parallel siamese
architectures for the considered transformation set and applying the TI-POOLING
operator on their outputs before the fully-connected layers. We show that this
topology internally finds the most optimal "canonical" instance of the input
image for training and therefore limits the redundancy in learned features.
This more efficient use of training data results in better performance on
popular benchmark datasets with smaller number of parameters when comparing to
standard convolutional neural networks with dataset augmentation and to other
baselines.
Symmetric Non-Rigid Structure from Motion for Category-Specific Object Structure Estimation
Authors: Yuan Gao, Alan Yuille
Date: 2016-09-22 00:00:00

Many objects, especially these made by humans, are symmetric, e.g. cars and
aeroplanes. This paper addresses the estimation of 3D structures of symmetric
objects from multiple images of the same object category, e.g. different cars,
seen from various viewpoints. We assume that the deformation between different
instances from the same object category is non-rigid and symmetric. In this
paper, we extend two leading non-rigid structure from motion (SfM) algorithms
to exploit symmetry constraints. We model the both methods as energy
minimization, in which we also recover the missing observations caused by
occlusions. In particularly, we show that by rotating the coordinate system,
the energy can be decoupled into two independent terms, which still exploit
symmetry, to apply matrix factorization separately on each of them for
initialization. The results on the Pascal3D+ dataset show that our methods
significantly improve performance over baseline methods.
Randomized Independent Component Analysis
Authors: Matan Sela, Ron Kimmel
Date: 2016-09-22 00:00:00

Independent component analysis (ICA) is a method for recovering statistically
independent signals from observations of unknown linear combinations of the
sources. Some of the most accurate ICA decomposition methods require searching
for the inverse transformation which minimizes different approximations of the
Mutual Information, a measure of statistical independence of random vectors.
Two such approximations are the Kernel Generalized Variance or the Kernel
Canonical Correlation which has been shown to reach the highest performance of
ICA methods. However, the computational effort necessary just for computing
these measures is cubic in the sample size. Hence, optimizing them becomes even
more computationally demanding, in terms of both space and time. Here, we
propose a couple of alternative novel measures based on randomized features of
the samples - the Randomized Generalized Variance and the Randomized Canonical
Correlation. The computational complexity of calculating the proposed
alternatives is linear in the sample size and provide a controllable
approximation of their Kernel-based non-random versions. We also show that
optimization of the proposed statistical properties yields a comparable
separation error at an order of magnitude faster compared to Kernel-based
measures.
Quantum Neural Machine Learning - Backpropagation and Dynamics
Authors: Carlos Pedro Gonçalves
Date: 2016-09-22 00:00:00
Tasks: BIG-bench Machine Learning, Quantum Machine Learning

The current work addresses quantum machine learning in the context of Quantum
Artificial Neural Networks such that the networks' processing is divided in two
stages: the learning stage, where the network converges to a specific quantum
circuit, and the backpropagation stage where the network effectively works as a
self-programing quantum computing system that selects the quantum circuits to
solve computing problems. The results are extended to general architectures
including recurrent networks that interact with an environment, coupling with
it in the neural links' activation order, and self-organizing in a dynamical
regime that intermixes patterns of dynamical stochasticity and persistent
quasiperiodic dynamics, making emerge a form of noise resilient dynamical
record.
A quantitative analysis of tilt in the Café Wall illusion: a bioplausible model for foveal and peripheral vision
Authors: Nasim Nematzadeh, David M. W. Powers
Date: 2016-09-22 00:00:00

The biological characteristics of human visual processing can be investigated
through the study of optical illusions and their perception, giving rise to
intuitions that may improve computer vision to match human performance.
Geometric illusions are a specific subfamily in which orientations and angles
are misperceived. This paper reports quantifiable predictions of the degree of
tilt for a typical geometric illusion called Caf\'e Wall, in which the mortar
between the tiles seems to tilt or bow. Our study employs a common bioplausible
model of retinal processing and we further develop an analytic processing
pipeline to quantify and thus predict the specific angle of tilt. We further
study the effect of resolution and feature size in order to predict the
different perceived tilts in different areas of the fovea and periphery, where
resolution varies as the eye saccades to different parts of the image. In the
experiments, several different minimal portions of the pattern, modeling
monocular and binocular foveal views, are investigated across multiple scales,
in order to quantify tilts with confidence intervals and explore the difference
between local and global tilt.
Realtime Hierarchical Clustering based on Boundary and Surface Statistics
Authors: Dominik Alexander Klein, Dirk Schulz, Armin Bernd Cremers
Date: 2016-09-22 00:00:00
Tasks: Clustering

Visual grouping is a key mechanism in human scene perception. There, it
belongs to the subconscious, early processing and is key prerequisite for other
high level tasks such as recognition. In this paper, we introduce an efficient,
realtime capable algorithm which likewise agglomerates a valuable hierarchical
clustering of a scene, while using purely local appearance statistics. To speed
up the processing, first we subdivide the image into meaningful, atomic
segments using a fast Watershed transform. Starting from there, our rapid,
agglomerative clustering algorithm prunes and maintains the connectivity graph
between clusters to contain only such pairs, which directly touch in the image
domain and are reciprocal nearest neighbors (RNN) wrt. a distance metric. The
core of this approach is our novel cluster distance: it combines boundary and
surface statistics both in terms of appearance as well as spatial linkage. This
yields state-of-the-art performance, as we demonstrate in conclusive
experiments conducted on BSDS500 and Pascal-Context datasets.
Regularized Dynamic Boltzmann Machine with Delay Pruning for Unsupervised Learning of Temporal Sequences
Authors: Sakyasingha Dasgupta, Takayuki Yoshizumi, Takayuki Osogami
Date: 2016-09-22 00:00:00
Tasks: Temporal Sequences, Time Series, Time Series Analysis

We introduce Delay Pruning, a simple yet powerful technique to regularize
dynamic Boltzmann machines (DyBM). The recently introduced DyBM provides a
particularly structured Boltzmann machine, as a generative model of a
multi-dimensional time-series. This Boltzmann machine can have infinitely many
layers of units but allows exact inference and learning based on its
biologically motivated structure. DyBM uses the idea of conduction delays in
the form of fixed length first-in first-out (FIFO) queues, with a neuron
connected to another via this FIFO queue, and spikes from a pre-synaptic neuron
travel along the queue to the post-synaptic neuron with a constant period of
delay. Here, we present Delay Pruning as a mechanism to prune the lengths of
the FIFO queues (making them zero) by setting some delay lengths to one with a
fixed probability, and finally selecting the best performing model with fixed
delays. The uniqueness of structure and a non-sampling based learning rule in
DyBM, make the application of previously proposed regularization techniques
like Dropout or DropConnect difficult, leading to poor generalization. First,
we evaluate the performance of Delay Pruning to let DyBM learn a
multidimensional temporal sequence generated by a Markov chain. Finally, we
show the effectiveness of delay pruning in learning high dimensional sequences
using the moving MNIST dataset, and compare it with Dropout and DropConnect
methods.
INSIGHT-1 at SemEval-2016 Task 5: Deep Learning for Multilingual Aspect-based Sentiment Analysis
Authors: Sebastian Ruder, Parsa Ghaffari, John G. Breslin
Date: 2016-09-09 00:00:00
Proceeding: SEMEVAL 2016 6
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Aspect Category Detection, Aspect Extraction, Deep Learning, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Sentiment Analysis

This paper describes our deep learning-based approach to multilingual
aspect-based sentiment analysis as part of SemEval 2016 Task 5. We use a
convolutional neural network (CNN) for both aspect extraction and aspect-based
sentiment analysis. We cast aspect extraction as a multi-label classification
problem, outputting probabilities over aspects parameterized by a threshold. To
determine the sentiment towards an aspect, we concatenate an aspect vector with
every word embedding and apply a convolution over it. Our constrained system
(unconstrained for English) achieves competitive results across all languages
and domains, placing first or second in 5 and 7 out of 11 language-domain pairs
for aspect category detection (slot 1) and sentiment polarity (slot 3)
respectively, thereby demonstrating the viability of a deep learning-based
approach for multilingual aspect-based sentiment analysis.
Evolutionary Cost-sensitive Extreme Learning Machine
Authors: Lei Zhang, David Zhang
Date: 2015-05-17 00:00:00
Tasks: Face Recognition

Conventional extreme learning machines solve a Moore-Penrose generalized
inverse of hidden layer activated matrix and analytically determine the output
weights to achieve generalized performance, by assuming the same loss from
different types of misclassification. The assumption may not hold in
cost-sensitive recognition tasks, such as face recognition based access control
system, where misclassifying a stranger as a family member may result in more
serious disaster than misclassifying a family member as a stranger. Though
recent cost-sensitive learning can reduce the total loss with a given cost
matrix that quantifies how severe one type of mistake against another, in many
realistic cases the cost matrix is unknown to users. Motivated by these
concerns, this paper proposes an evolutionary cost-sensitive extreme learning
machine (ECSELM), with the following merits: 1) to our best knowledge, it is
the first proposal of ELM in evolutionary cost-sensitive classification
scenario; 2) it well addresses the open issue of how to define the cost matrix
in cost-sensitive learning tasks; 3) an evolutionary backtracking search
algorithm is induced for adaptive cost matrix optimization. Experiments in a
variety of cost-sensitive tasks well demonstrate the effectiveness of the
proposed approaches, with about 5%~10% improvements.
Bi-directional Attention with Agreement for Dependency Parsing
Authors: Hao Cheng, Hao Fang, Xiaodong He, Jianfeng Gao, Li Deng
Date: 2016-08-06 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Dependency Parsing

We develop a novel bi-directional attention model for dependency parsing,
which learns to agree on headword predictions from the forward and backward
parsing directions. The parsing procedure for each direction is formulated as
sequentially querying the memory component that stores continuous headword
embeddings. The proposed parser makes use of {\it soft} headword embeddings,
allowing the model to implicitly capture high-order parsing history without
dramatically increasing the computational complexity. We conduct experiments on
English, Chinese, and 12 other languages from the CoNLL 2006 shared task,
showing that the proposed model achieves state-of-the-art unlabeled attachment
scores on 6 languages.
How Useful is Region-based Classification of Remote Sensing Images in a Deep Learning Framework?
Authors: Nicolas Audebert, Bertrand Le Saux, Sébastien Lefèvre
Date: 2016-09-22 00:00:00
Tasks: Classification, General Classification, Segmentation, Superpixels

In this paper, we investigate the impact of segmentation algorithms as a
preprocessing step for classification of remote sensing images in a deep
learning framework. Especially, we address the issue of segmenting the image
into regions to be classified using pre-trained deep neural networks as feature
extractors for an SVM-based classifier. An efficient segmentation as a
preprocessing step helps learning by adding a spatially-coherent structure to
the data. Therefore, we compare algorithms producing superpixels with more
traditional remote sensing segmentation algorithms and measure the variation in
terms of classification accuracy. We establish that superpixel algorithms allow
for a better classification accuracy as a homogenous and compact segmentation
favors better generalization of the training samples.