pwc-archive/papers-with-abstracts huggingface dataset:
Clustering is semidefinitely not that hard: Nonnegative SDP for manifold disentangling
Authors: Mariano Tepper, Anirvan M. Sengupta, Dmitri Chklovskii
Date: 2017-06-19 00:00:00
Tasks: Clustering

In solving hard computational problems, semidefinite program (SDP)
relaxations often play an important role because they come with a guarantee of
optimality. Here, we focus on a popular semidefinite relaxation of K-means
clustering which yields the same solution as the non-convex original
formulation for well segregated datasets. We report an unexpected finding: when
data contains (greater than zero-dimensional) manifolds, the SDP solution
captures such geometrical structures. Unlike traditional manifold embedding
techniques, our approach does not rely on manually defining a kernel but rather
enforces locality via a nonnegativity constraint. We thus call our approach
NOnnegative MAnifold Disentangling, or NOMAD. To build an intuitive
understanding of its manifold learning capabilities, we develop a theoretical
analysis of NOMAD on idealized datasets. While NOMAD is convex and the globally
optimal solution can be found by generic SDP solvers with polynomial time
complexity, they are too slow for modern datasets. To address this problem, we
analyze a non-convex heuristic and present a new, convex and yet efficient,
algorithm, based on the conditional gradient method. Our results render NOMAD a
versatile, understandable, and powerful tool for manifold learning.
Combinatorial Penalties: Which structures are preserved by convex relaxations?
Authors: Marwa El Halabi, Francis Bach, Volkan Cevher
Date: 2017-10-17 00:00:00

We consider the homogeneous and the non-homogeneous convex relaxations for
combinatorial penalty functions defined on support sets. Our study identifies
key differences in the tightness of the resulting relaxations through the
notion of the lower combinatorial envelope of a set-function along with new
necessary conditions for support identification. We then propose a general
adaptive estimator for convex monotone regularizers, and derive new sufficient
conditions for support recovery in the asymptotic setting.
Espresso: Efficient Forward Propagation for BCNNs
Authors: Fabrizio Pedersoli, George Tzanetakis, Andrea Tagliasacchi
Date: 2017-05-19 00:00:00
Tasks: CPU, GPU

There are many applications scenarios for which the computational performance
and memory footprint of the prediction phase of Deep Neural Networks (DNNs)
needs to be optimized. Binary Neural Networks (BDNNs) have been shown to be an
effective way of achieving this objective. In this paper, we show how
Convolutional Neural Networks (CNNs) can be implemented using binary
representations. Espresso is a compact, yet powerful library written in C/CUDA
that features all the functionalities required for the forward propagation of
CNNs, in a binary file less than 400KB, without any external dependencies.
Although it is mainly designed to take advantage of massive GPU parallelism,
Espresso also provides an equivalent CPU implementation for CNNs. Espresso
provides special convolutional and dense layers for BCNNs, leveraging
bit-packing and bit-wise computations for efficient execution. These techniques
provide a speed-up of matrix-multiplication routines, and at the same time,
reduce memory usage when storing parameters and activations. We experimentally
show that Espresso is significantly faster than existing implementations of
optimized binary neural networks ($\approx$ 2 orders of magnitude). Espresso is
released under the Apache 2.0 license and is available at
http://github.com/fpeder/espresso.
Making a long story short: A Multi-Importance fast-forwarding egocentric videos with the emphasis on relevant objects
Authors: Michel Melo Silva, Washington Luis Souza Ramos, Felipe Cadar Chamone, João Pedro Klock Ferreira, Mario Fernando Montenegro Campos, Erickson Rangel Nascimento
Date: 2017-11-09 00:00:00
Tasks: Video Stabilization

The emergence of low-cost high-quality personal wearable cameras combined
with the increasing storage capacity of video-sharing websites have evoked a
growing interest in first-person videos, since most videos are composed of
long-running unedited streams which are usually tedious and unpleasant to
watch. State-of-the-art semantic fast-forward methods currently face the
challenge of providing an adequate balance between smoothness in visual flow
and the emphasis on the relevant parts. In this work, we present the
Multi-Importance Fast-Forward (MIFF), a fully automatic methodology to
fast-forward egocentric videos facing these challenges. The dilemma of defining
what is the semantic information of a video is addressed by a learning process
based on the preferences of the user. Results show that the proposed method
keeps over $3$ times more semantic content than the state-of-the-art
fast-forward. Finally, we discuss the need of a particular video stabilization
technique for fast-forward egocentric videos.
A Deep Learning Algorithm for One-step Contour Aware Nuclei Segmentation of Histopathological Images
Authors: Yuxin Cui, Guiying Zhang, Zhonghao Liu, Zheng Xiong, Jianjun Hu
Date: 2018-03-07 00:00:00
Tasks: Data Augmentation, Segmentation, whole slide images

This paper addresses the task of nuclei segmentation in high-resolution
histopathological images. We propose an auto- matic end-to-end deep neural
network algorithm for segmenta- tion of individual nuclei. A nucleus-boundary
model is introduced to predict nuclei and their boundaries simultaneously using
a fully convolutional neural network. Given a color normalized image, the model
directly outputs an estimated nuclei map and a boundary map. A simple, fast and
parameter-free post-processing procedure is performed on the estimated nuclei
map to produce the final segmented nuclei. An overlapped patch extraction and
assembling method is also designed for seamless prediction of nuclei in large
whole-slide images. We also show the effectiveness of data augmentation methods
for nuclei segmentation task. Our experiments showed our method outperforms
prior state-of-the- art methods. Moreover, it is efficient that one 1000X1000
image can be segmented in less than 5 seconds. This makes it possible to
precisely segment the whole-slide image in acceptable time
Realizing Intelligence
Authors: Paul Yaworsky
Date: 2018-03-07 00:00:00

Order exists in the world. The intelligence process enables us to realize
that order, to some extent. We provide a high level description of intelligence
using simple definitions, basic building blocks, a conceptual framework and
general hierarchy. This perspective includes multiple levels of abstraction
occurring in space and in time. The resulting model offers simple, useful ways
to help realize the essence of intelligence.
Fast and Accurate Semantic Mapping through Geometric-based Incremental Segmentation
Authors: Yoshikatsu Nakajima, Keisuke Tateno, Federico Tombari, Hideo Saito
Date: 2018-03-07 00:00:00
Tasks: Computational Efficiency, Segmentation

We propose an efficient and scalable method for incrementally building a
dense, semantically annotated 3D map in real-time. The proposed method assigns
class probabilities to each region, not each element (e.g., surfel and voxel),
of the 3D map which is built up through a robust SLAM framework and
incrementally segmented with a geometric-based segmentation method. Differently
from all other approaches, our method has a capability of running at over 30Hz
while performing all processing components, including SLAM, segmentation, 2D
recognition, and updating class probabilities of each segmentation label at
every incoming frame, thanks to the high efficiency that characterizes the
computationally intensive stages of our framework. By utilizing a specifically
designed CNN to improve the frame-wise segmentation result, we can also achieve
high accuracy. We validate our method on the NYUv2 dataset by comparing with
the state of the art in terms of accuracy and computational efficiency, and by
means of an analysis in terms of time and space complexity.
A bag-to-class divergence approach to multiple-instance learning
Authors: Kajsa Møllersen, Jon Yngve Hardeberg, Fred Godtliebsen
Date: 2018-03-07 00:00:00
Tasks: General Classification, Multiple Instance Learning

In multi-instance (MI) learning, each object (bag) consists of multiple
feature vectors (instances), and is most commonly regarded as a set of points
in a multidimensional space. A different viewpoint is that the instances are
realisations of random vectors with corresponding probability distribution, and
that a bag is the distribution, not the realisations. In MI classification,
each bag in the training set has a class label, but the instances are
unlabelled. By introducing the probability distribution space to bag-level
classification problems, dissimilarities between probability distributions
(divergences) can be applied. The bag-to-bag Kullback-Leibler information is
asymptotically the best classifier, but the typical sparseness of MI training
sets is an obstacle. We introduce bag-to-class divergence to MI learning,
emphasising the hierarchical nature of the random vectors that makes bags from
the same class different. We propose two properties for bag-to-class
divergences, and an additional property for sparse training sets.
Fast Dawid-Skene: A Fast Vote Aggregation Scheme for Sentiment Classification
Authors: Vaibhav B Sinha, Sukrut Rao, Vineeth N. Balasubramanian
Date: 2018-03-07 00:00:00
Tasks: General Classification, Sentiment Analysis, Sentiment Classification

Many real world problems can now be effectively solved using supervised
machine learning. A major roadblock is often the lack of an adequate quantity
of labeled data for training. A possible solution is to assign the task of
labeling data to a crowd, and then infer the true label using aggregation
methods. A well-known approach for aggregation is the Dawid-Skene (DS)
algorithm, which is based on the principle of Expectation-Maximization (EM). We
propose a new simple, yet effective, EM-based algorithm, which can be
interpreted as a `hard' version of DS, that allows much faster convergence
while maintaining similar accuracy in aggregation. We show the use of this
algorithm as a quick and effective technique for online, real-time sentiment
annotation. We also prove that our algorithm converges to the estimated labels
at a linear rate. Our experiments on standard datasets show a significant
speedup in time taken for aggregation - upto $\sim$8x over Dawid-Skene and
$\sim$6x over other fast EM methods, at competitive accuracy performance. The
code for the implementation of the algorithms can be found at
https://github.com/GoodDeeds/Fast-Dawid-Skene
Optimal Subsampling for Large Sample Logistic Regression
Authors: HaiYing Wang, Rong Zhu, Ping Ma
Date: 2017-02-03 00:00:00
Tasks: regression

For massive data, the family of subsampling algorithms is popular to downsize
the data volume and reduce computational burden. Existing studies focus on
approximating the ordinary least squares estimate in linear regression, where
statistical leverage scores are often used to define subsampling probabilities.
In this paper, we propose fast subsampling algorithms to efficiently
approximate the maximum likelihood estimate in logistic regression. We first
establish consistency and asymptotic normality of the estimator from a general
subsampling algorithm, and then derive optimal subsampling probabilities that
minimize the asymptotic mean squared error of the resultant estimator. An
alternative minimization criterion is also proposed to further reduce the
computational cost. The optimal subsampling probabilities depend on the full
data estimate, so we develop a two-step algorithm to approximate the optimal
subsampling procedure. This algorithm is computationally efficient and has a
significant reduction in computing time compared to the full data approach.
Consistency and asymptotic normality of the estimator from a two-step algorithm
are also established. Synthetic and real data sets are used to evaluate the
practical performance of the proposed method.
Comparing Deep Reinforcement Learning and Evolutionary Methods in Continuous Control
Authors: Shangtong Zhang, Osmar R. Zaiane
Date: 2017-11-30 00:00:00
Tasks: continuous-control, Continuous Control, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement Learning and the Evolutionary Strategy are two major approaches
in addressing complicated control problems. Both are strong contenders and have
their own devotee communities. Both groups have been very active in developing
new advances in their own domain and devising, in recent years, leading-edge
techniques to address complex continuous control tasks. Here, in the context of
Deep Reinforcement Learning, we formulate a parallelized version of the
Proximal Policy Optimization method and a Deep Deterministic Policy Gradient
method. Moreover, we conduct a thorough comparison between the state-of-the-art
techniques in both camps fro continuous control; evolutionary methods and Deep
Reinforcement Learning methods. The results show there is no consistent winner.
Beyond black-boxes in Bayesian inverse problems and model validation: applications in solid mechanics of elastography
Authors: Lukas Bruder, Phaedon-Stelios Koutsourelakis
Date: 2018-03-02 00:00:00
Tasks: Variational Inference

The present paper is motivated by one of the most fundamental challenges in
inverse problems, that of quantifying model discrepancies and errors. While
significant strides have been made in calibrating model parameters, the
overwhelming majority of pertinent methods is based on the assumption of a
perfect model. Motivated by problems in solid mechanics which, as all problems
in continuum thermodynamics, are described by conservation laws and
phenomenological constitutive closures, we argue that in order to quantify
model uncertainty in a physically meaningful manner, one should break open the
black-box forward model. In particular we propose formulating an undirected
probabilistic model that explicitly accounts for the governing equations and
their validity. This recasts the solution of both forward and inverse problems
as probabilistic inference tasks where the problem's state variables should not
only be compatible with the data but also with the governing equations as well.
Even though the probability densities involved do not contain any black-box
terms, they live in much higher-dimensional spaces. In combination with the
intractability of the normalization constant of the undirected model employed,
this poses significant challenges which we propose to address with a
linearly-scaling, double-layer of Stochastic Variational Inference. We
demonstrate the capabilities and efficacy of the proposed model in synthetic
forward and inverse problems (with and without model error) in elastography.
Algorithmic detectability threshold of the stochastic block model
Authors: Tatsuro Kawamoto
Date: 2017-10-24 00:00:00
Tasks: model, Stochastic Block Model

The assumption that the values of model parameters are known or correctly
learned, i.e., the Nishimori condition, is one of the requirements for the
detectability analysis of the stochastic block model in statistical inference.
In practice, however, there is no example demonstrating that we can know the
model parameters beforehand, and there is no guarantee that the model
parameters can be learned accurately. In this study, we consider the
expectation--maximization (EM) algorithm with belief propagation (BP) and
derive its algorithmic detectability threshold. Our analysis is not restricted
to the community structure, but includes general modular structures. Because
the algorithm cannot always learn the planted model parameters correctly, the
algorithmic detectability threshold is qualitatively different from the one
with the Nishimori condition.
Learning to Compute Word Embeddings On the Fly
Authors: Dzmitry Bahdanau, Tom Bosc, Stanisław Jastrzębski, Edward Grefenstette, Pascal Vincent, Yoshua Bengio
Date: 2017-06-01 00:00:00
Proceeding: ICLR 2018 1
Tasks: Language Modeling, Language Modelling, Natural Language Inference, Question Answering, Reading Comprehension, Word Embeddings

Words in natural language follow a Zipfian distribution whereby some words
are frequent but most are rare. Learning representations for words in the "long
tail" of this distribution requires enormous amounts of data. Representations
of rare words trained directly on end tasks are usually poor, requiring us to
pre-train embeddings on external data, or treat all rare words as
out-of-vocabulary words with a unique representation. We provide a method for
predicting embeddings of rare words on the fly from small amounts of auxiliary
data with a network trained end-to-end for the downstream task. We show that
this improves results against baselines where embeddings are trained on the end
task for reading comprehension, recognizing textual entailment and language
modeling.
Deep Back-Projection Networks For Super-Resolution
Authors: Muhammad Haris, Greg Shakhnarovich, Norimichi Ukita
Date: 2018-03-07 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Super-Resolution, Super-Resolution, Video Super-Resolution

The feed-forward architectures of recently proposed deep super-resolution
networks learn representations of low-resolution inputs, and the non-linear
mapping from those to high-resolution output. However, this approach does not
fully address the mutual dependencies of low- and high-resolution images. We
propose Deep Back-Projection Networks (DBPN), that exploit iterative up- and
down-sampling layers, providing an error feedback mechanism for projection
errors at each stage. We construct mutually-connected up- and down-sampling
stages each of which represents different types of image degradation and
high-resolution components. We show that extending this idea to allow
concatenation of features across up- and down-sampling stages (Dense DBPN)
allows us to reconstruct further improve super-resolution, yielding superior
results and in particular establishing new state of the art results for large
scaling factors such as 8x across multiple data sets.
Towards the Creation of a Large Corpus of Synthetically-Identified Clinical Notes
Authors: Willie Boag, Tristan Naumann, Peter Szolovits
Date: 2018-03-07 00:00:00
Tasks: De-identification

Clinical notes often describe the most important aspects of a patient's
physiology and are therefore critical to medical research. However, these notes
are typically inaccessible to researchers without prior removal of sensitive
protected health information (PHI), a natural language processing (NLP) task
referred to as deidentification. Tools to automatically de-identify clinical
notes are needed but are difficult to create without access to those very same
notes containing PHI. This work presents a first step toward creating a large
synthetically-identified corpus of clinical notes and corresponding PHI
annotations in order to facilitate the development de-identification tools.
Further, one such tool is evaluated against this corpus in order to understand
the advantages and shortcomings of this approach.
Stochastic Block Models with Multiple Continuous Attributes
Authors: Natalie Stanley, Thomas Bonacci, Roland Kwitt, Marc Niethammer, Peter J. Mucha
Date: 2018-03-07 00:00:00
Tasks: Attribute, Collaborative Filtering, Community Detection, Link Prediction, Stochastic Block Model

The stochastic block model (SBM) is a probabilistic model for community
structure in networks. Typically, only the adjacency matrix is used to perform
SBM parameter inference. In this paper, we consider circumstances in which
nodes have an associated vector of continuous attributes that are also used to
learn the node-to-community assignments and corresponding SBM parameters. While
this assumption is not realistic for every application, our model assumes that
the attributes associated with the nodes in a network's community can be
described by a common multivariate Gaussian model. In this augmented,
attributed SBM, the objective is to simultaneously learn the SBM connectivity
probabilities with the multivariate Gaussian parameters describing each
community. While there are recent examples in the literature that combine
connectivity and attribute information to inform community detection, our model
is the first augmented stochastic block model to handle multiple continuous
attributes. This provides the flexibility in biological data to, for example,
augment connectivity information with continuous measurements from multiple
experimental modalities. Because the lack of labeled network data often makes
community detection results difficult to validate, we highlight the usefulness
of our model for two network prediction tasks: link prediction and
collaborative filtering. As a result of fitting this attributed stochastic
block model, one can predict the attribute vector or connectivity patterns for
a new node in the event of the complementary source of information
(connectivity or attributes, respectively). We also highlight two biological
examples where the attributed stochastic block model provides satisfactory
performance in the link prediction and collaborative filtering tasks.
Rapid Adaptation with Conditionally Shifted Neurons
Authors: Tsendsuren Munkhdalai, Xingdi Yuan, Soroush Mehri, Adam Trischler
Date: 2017-12-28 00:00:00
Proceeding: ICML 2018 7
Tasks: Few-Shot Image Classification

We describe a mechanism by which artificial neural networks can learn rapid
adaptation - the ability to adapt on the fly, with little data, to new tasks -
that we call conditionally shifted neurons. We apply this mechanism in the
framework of metalearning, where the aim is to replicate some of the
flexibility of human learning in machines. Conditionally shifted neurons modify
their activation values with task-specific shifts retrieved from a memory
module, which is populated rapidly based on limited task experience. On
metalearning benchmarks from the vision and language domains, models augmented
with conditionally shifted neurons achieve state-of-the-art results.
Generating Contradictory, Neutral, and Entailing Sentences
Authors: Yikang Shen, Shawn Tan, Chin-wei Huang, Aaron Courville
Date: 2018-03-07 00:00:00
Tasks: Diversity, Natural Language Inference, RTE, Sentence

Learning distributed sentence representations remains an interesting problem
in the field of Natural Language Processing (NLP). We want to learn a model
that approximates the conditional latent space over the representations of a
logical antecedent of the given statement. In our paper, we propose an approach
to generating sentences, conditioned on an input sentence and a logical
inference label. We do this by modeling the different possibilities for the
output sentence as a distribution over the latent representation, which we
train using an adversarial objective. We evaluate the model using two
state-of-the-art models for the Recognizing Textual Entailment (RTE) task, and
measure the BLEU scores against the actual sentences as a probe for the
diversity of sentences produced by our model. The experiment results show that,
given our framework, we have clear ways to improve the quality and diversity of
generated sentences.