pwc-archive/papers-with-abstracts huggingface dataset:
Relaxation of the EM Algorithm via Quantum Annealing
Authors: Hideyuki Miyahara, Koji Tsumura
Date: 2016-06-05 00:00:00

The EM algorithm is a novel numerical method to obtain maximum likelihood
estimates and is often used for practical calculations. However, many of
maximum likelihood estimation problems are nonconvex, and it is known that the
EM algorithm fails to give the optimal estimate by being trapped by local
optima. In order to deal with this difficulty, we propose a deterministic
quantum annealing EM algorithm by introducing the mathematical mechanism of
quantum fluctuations into the conventional EM algorithm because quantum
fluctuations induce the tunnel effect and are expected to relax the difficulty
of nonconvex optimization problems in the maximum likelihood estimation
problems. We show a theorem that guarantees its convergence and give numerical
experiments to verify its efficiency.
Better Image Segmentation by Exploiting Dense Semantic Predictions
Authors: Qiyang Zhao, Lewis D. Griffin
Date: 2016-06-05 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

It is well accepted that image segmentation can benefit from utilizing
multilevel cues. The paper focuses on utilizing the FCNN-based dense semantic
predictions in the bottom-up image segmentation, arguing to take semantic cues
into account from the very beginning. By this we can avoid merging regions of
similar appearance but distinct semantic categories as possible. The semantic
inefficiency problem is handled. We also propose a straightforward way to use
the contour cues to suppress the noise in multilevel cues, thus to improve the
segmentation robustness. The evaluation on the BSDS500 shows that we obtain the
competitive region and boundary performance. Furthermore, since all individual
regions can be assigned with appropriate semantic labels during the
computation, we are capable of extracting the adjusted semantic segmentations.
The experiment on Pascal VOC 2012 shows our improvement to the original
semantic segmentations which derives directly from the dense predictions.
word2vec Parameter Learning Explained
Authors: Xin Rong
Date: 2014-11-11 00:00:00

The word2vec model and application by Mikolov et al. have attracted a great
amount of attention in recent two years. The vector representations of words
learned by word2vec models have been shown to carry semantic meanings and are
useful in various NLP tasks. As an increasing number of researchers would like
to experiment with word2vec or similar techniques, I notice that there lacks a
material that comprehensively explains the parameter learning process of word
embedding models in details, thus preventing researchers that are non-experts
in neural networks from understanding the working mechanism of such models.
  This note provides detailed derivations and explanations of the parameter
update equations of the word2vec models, including the original continuous
bag-of-word (CBOW) and skip-gram (SG) models, as well as advanced optimization
techniques, including hierarchical softmax and negative sampling. Intuitive
interpretations of the gradient equations are also provided alongside
mathematical derivations.
  In the appendix, a review on the basics of neuron networks and
backpropagation is provided. I also created an interactive demo, wevi, to
facilitate the intuitive understanding of the model.
Nighttime Haze Removal with Illumination Correction
Authors: Jing Zhang, Yang Cao, Zengfu Wang
Date: 2016-06-05 00:00:00

Haze removal is important for computational photography and computer vision
applications. However, most of the existing methods for dehazing are designed
for daytime images, and cannot always work well in the nighttime. Different
from the imaging conditions in the daytime, images captured in nighttime haze
condition may suffer from non-uniform illumination due to artificial light
sources, which exhibit low brightness/contrast and color distortion. In this
paper, we present a new nighttime hazy imaging model that takes into account
both the non-uniform illumination from artificial light sources and the
scattering and attenuation effects of haze. Accordingly, we propose an
efficient dehazing algorithm for nighttime hazy images. The proposed algorithm
includes three sequential steps. i) It enhances the overall brightness by
performing a gamma correction step after estimating the illumination from the
original image. ii) Then it achieves a color-balance result by performing a
color correction step after estimating the color characteristics of the
incident light. iii) Finally, it remove the haze effect by applying the dark
channel prior and estimating the point-wise environmental light based on the
previous illumination-balance result. Experimental results show that the
proposed algorithm can achieve illumination-balance and haze-free results with
good color rendition ability.
Learning Discriminative Features via Label Consistent Neural Network
Authors: Zhuolin Jiang, Yaming Wang, Larry Davis, Walt Andrews, Viktor Rozgic
Date: 2016-02-03 00:00:00
Tasks: General Classification

Deep Convolutional Neural Networks (CNN) enforces supervised information only
at the output layer, and hidden layers are trained by back propagating the
prediction error from the output layer without explicit supervision. We propose
a supervised feature learning approach, Label Consistent Neural Network, which
enforces direct supervision in late hidden layers. We associate each neuron in
a hidden layer with a particular class label and encourage it to be activated
for input signals from the same class. More specifically, we introduce a label
consistency regularization called "discriminative representation error" loss
for late hidden layers and combine it with classification error loss to build
our overall objective function. This label consistency constraint alleviates
the common problem of gradient vanishing and tends to faster convergence; it
also makes the features derived from late hidden layers discriminative enough
for classification even using a simple $k$-NN classifier, since input signals
from the same class will have very similar representations. Experimental
results demonstrate that our approach achieves state-of-the-art performances on
several public benchmarks for action and object category recognition.
An Interactive Medical Image Segmentation Framework Using Iterative Refinement
Authors: Pratik Kalshetti, Manas Bundele, Parag Rahangdale, Dinesh Jangra, Chiranjoy Chattopadhyay, Gaurav Harit, Abhay Elhence
Date: 2016-06-05 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Image segmentation is often performed on medical images for identifying
diseases in clinical evaluation. Hence it has become one of the major research
areas. Conventional image segmentation techniques are unable to provide
satisfactory segmentation results for medical images as they contain
irregularities. They need to be pre-processed before segmentation. In order to
obtain the most suitable method for medical image segmentation, we propose a
two stage algorithm. The first stage automatically generates a binary marker
image of the region of interest using mathematical morphology. This marker
serves as the mask image for the second stage which uses GrabCut on the input
image thus resulting in an efficient segmented result. The obtained result can
be further refined by user interaction which can be done using the Graphical
User Interface (GUI). Experimental results show that the proposed method is
accurate and provides satisfactory segmentation results with minimum user
interaction on medical as well as natural images.
Character-Level Question Answering with Attention
Authors: David Golub, Xiaodong He
Date: 2016-04-04 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Data Augmentation, Decoder, Question Answering

We show that a character-level encoder-decoder framework can be successfully
applied to question answering with a structured knowledge base. We use our
model for single-relation question answering and demonstrate the effectiveness
of our approach on the SimpleQuestions dataset (Bordes et al., 2015), where we
improve state-of-the-art accuracy from 63.9% to 70.9%, without use of
ensembles. Importantly, our character-level model has 16x fewer parameters than
an equivalent word-level model, can be learned with significantly less data
compared to previous work, which relies on data augmentation, and is robust to
new entities in testing.
Brundlefly at SemEval-2016 Task 12: Recurrent Neural Networks vs. Joint Inference for Clinical Temporal Information Extraction
Authors: Jason Alan Fries
Date: 2016-06-04 00:00:00
Tasks: Entity Extraction using GAN, General Classification, Structured Prediction, Temporal Information Extraction, Word Embeddings

We submitted two systems to the SemEval-2016 Task 12: Clinical TempEval
challenge, participating in Phase 1, where we identified text spans of time and
event expressions in clinical notes and Phase 2, where we predicted a relation
between an event and its parent document creation time.
  For temporal entity extraction, we find that a joint inference-based approach
using structured prediction outperforms a vanilla recurrent neural network that
incorporates word embeddings trained on a variety of large clinical document
sets. For document creation time relations, we find that a combination of date
canonicalization and distant supervision rules for predicting relations on both
events and time expressions improves classification, though gains are limited,
likely due to the small scale of training data.
Distance Metric Ensemble Learning and the Andrews-Curtis Conjecture
Authors: Krzysztof Krawiec, Jerry Swan
Date: 2016-06-04 00:00:00
Tasks: Ensemble Learning

Motivated by the search for a counterexample to the Poincar\'e conjecture in
three and four dimensions, the Andrews-Curtis conjecture was proposed in 1965.
It is now generally suspected that the Andrews-Curtis conjecture is false, but
small potential counterexamples are not so numerous, and previous work has
attempted to eliminate some via combinatorial search. Progress has however been
limited, with the most successful approach (breadth-first-search using
secondary storage) being neither scalable nor heuristically-informed. A
previous empirical analysis of problem structure examined several heuristic
measures of search progress and determined that none of them provided any
useful guidance for search. In this article, we induce new quality measures
directly from the problem structure and combine them to produce a more
effective search driver via ensemble machine learning. By this means, we
eliminate 19 potential counterexamples, the status of which had been unknown
for some years.
Estimating Treatment Effects using Multiple Surrogates: The Role of the Surrogate Score and the Surrogate Index
Authors: Susan Athey, Raj Chetty, Guido Imbens, Hyunseung Kang
Date: 2016-03-30 00:00:00
Tasks: Causal Inference

Estimating the long-term effects of treatments is of interest in many fields. A common challenge in estimating such treatment effects is that long-term outcomes are unobserved in the time frame needed to make policy decisions. One approach to overcome this missing data problem is to analyze treatments effects on an intermediate outcome, often called a statistical surrogate, if it satisfies the condition that treatment and outcome are independent conditional on the statistical surrogate. The validity of the surrogacy condition is often controversial. Here we exploit that fact that in modern datasets, researchers often observe a large number, possibly hundreds or thousands, of intermediate outcomes, thought to lie on or close to the causal chain between the treatment and the long-term outcome of interest. Even if none of the individual proxies satisfies the statistical surrogacy criterion by itself, using multiple proxies can be useful in causal inference. We focus primarily on a setting with two samples, an experimental sample containing data about the treatment indicator and the surrogates and an observational sample containing information about the surrogates and the primary outcome. We state assumptions under which the average treatment effect be identified and estimated with a high-dimensional vector of proxies that collectively satisfy the surrogacy assumption, and derive the bias from violations of the surrogacy assumption, and show that even if the primary outcome is also observed in the experimental sample, there is still information to be gained from using surrogates.
Generating Natural Language Inference Chains
Authors: Vladyslav Kolesnyk, Tim Rocktäschel, Sebastian Riedel
Date: 2016-06-04 00:00:00
Tasks: Machine Translation, Natural Language Inference, Question Answering, Sentence

The ability to reason with natural language is a fundamental prerequisite for
many NLP tasks such as information extraction, machine translation and question
answering. To quantify this ability, systems are commonly tested whether they
can recognize textual entailment, i.e., whether one sentence can be inferred
from another one. However, in most NLP applications only single source
sentences instead of sentence pairs are available. Hence, we propose a new task
that measures how well a model can generate an entailed sentence from a source
sentence. We take entailment-pairs of the Stanford Natural Language Inference
corpus and train an LSTM with attention. On a manually annotated test set we
found that 82% of generated sentences are correct, an improvement of 10.3% over
an LSTM baseline. A qualitative analysis shows that this model is not only
capable of shortening input sentences, but also inferring new statements via
paraphrasing and phrase entailment. We then apply this model recursively to
input-output pairs, thereby generating natural language inference chains that
can be used to automatically construct an entailment graph from source
sentences. Finally, by swapping source and target sentences we can also train a
model that given an input sentence invents additional information to generate a
new sentence.
Trend Filtering on Graphs
Authors: Yu-Xiang Wang, James Sharpnack, Alex Smola, Ryan J. Tibshirani
Date: 2014-10-28 00:00:00
Tasks: regression

We introduce a family of adaptive estimators on graphs, based on penalizing
the $\ell_1$ norm of discrete graph differences. This generalizes the idea of
trend filtering [Kim et al. (2009), Tibshirani (2014)], used for univariate
nonparametric regression, to graphs. Analogous to the univariate case, graph
trend filtering exhibits a level of local adaptivity unmatched by the usual
$\ell_2$-based graph smoothers. It is also defined by a convex minimization
problem that is readily solved (e.g., by fast ADMM or Newton algorithms). We
demonstrate the merits of graph trend filtering through examples and theory.
Automated Image Captioning for Rapid Prototyping and Resource Constrained Environments
Authors: Karan Sharma, Arun CS Kumar, Suchendra Bhandarkar
Date: 2016-06-04 00:00:00
Tasks: Image Captioning, Word Embeddings

Significant performance gains in deep learning coupled with the exponential
growth of image and video data on the Internet have resulted in the recent
emergence of automated image captioning systems. Ensuring scalability of
automated image captioning systems with respect to the ever increasing volume
of image and video data is a significant challenge. This paper provides a
valuable insight in that the detection of a few significant (top) objects in an
image allows one to extract other relevant information such as actions (verbs)
in the image. We expect this insight to be useful in the design of scalable
image captioning systems. We address two parameters by which the scalability of
image captioning systems could be quantified, i.e., the traditional algorithmic
time complexity which is important given the resource limitations of the user
device and the system development time since the programmers' time is a
critical resource constraint in many real-world scenarios. Additionally, we
address the issue of how word embeddings could be used to infer the verb
(action) from the nouns (objects) in a given image in a zero-shot manner. Our
results show that it is possible to attain reasonably good performance on
predicting actions and captioning images using our approaches with the added
advantage of simplicity of implementation.
Effective Multi-Robot Spatial Task Allocation using Model Approximations
Authors: Okan Aşık, H. Levent Akin
Date: 2016-06-04 00:00:00
Tasks: Clustering

Real-world multi-agent planning problems cannot be solved using
decision-theoretic planning methods due to the exponential complexity. We
approximate firefighting in rescue simulation as a spatially distributed task
and model with multi-agent Markov decision process. We use recent approximation
methods for spatial task problems to reduce the model complexity. Our
approximations are single-agent, static task, shortest path pruning, dynamic
planning horizon, and task clustering. We create scenarios from RoboCup Rescue
Simulation maps and evaluate our methods on these graph worlds. The results
show that our approach is faster and better than comparable methods and has
negligible performance loss compared to the optimal policy. We also show that
our method has a similar performance as DCOP methods on example RCRS scenarios.
LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
Authors: Fisher Yu, Ari Seff, yinda zhang, Shuran Song, Thomas Funkhouser, Jianxiong Xiao
Date: 2015-06-10 00:00:00

While there has been remarkable progress in the performance of visual
recognition algorithms, the state-of-the-art models tend to be exceptionally
data-hungry. Large labeled training datasets, expensive and tedious to produce,
are required to optimize millions of parameters in deep network models. Lagging
behind the growth in model capacity, the available datasets are quickly
becoming outdated in terms of size and density. To circumvent this bottleneck,
we propose to amplify human effort through a partially automated labeling
scheme, leveraging deep learning with humans in the loop. Starting from a large
set of candidate images for each category, we iteratively sample a subset, ask
people to label them, classify the others with a trained model, split the set
into positives, negatives, and unlabeled based on the classification
confidence, and then iterate with the unlabeled set. To assess the
effectiveness of this cascading procedure and enable further progress in visual
recognition research, we construct a new image dataset, LSUN. It contains
around one million labeled images for each of 10 scene categories and 20 object
categories. We experiment with training popular convolutional networks and find
that they achieve substantial performance gains when trained on this dataset.
PAC-Bayes Analysis of Multi-view Learning
Authors: Shiliang Sun, John Shawe-Taylor, Liang Mao
Date: 2014-06-21 00:00:00
Tasks: MULTI-VIEW LEARNING

This paper presents eight PAC-Bayes bounds to analyze the generalization
performance of multi-view classifiers. These bounds adopt data dependent
Gaussian priors which emphasize classifiers with high view agreements. The
center of the prior for the first two bounds is the origin, while the center of
the prior for the third and fourth bounds is given by a data dependent vector.
An important technique to obtain these bounds is two derived logarithmic
determinant inequalities whose difference lies in whether the dimensionality of
data is involved. The centers of the fifth and sixth bounds are calculated on a
separate subset of the training set. The last two bounds use unlabeled data to
represent view agreements and are thus applicable to semi-supervised multi-view
learning. We evaluate all the presented multi-view PAC-Bayes bounds on
benchmark data and compare them with previous single-view PAC-Bayes bounds. The
usefulness and performance of the multi-view bounds are discussed.
Neural Variational Inference for Text Processing
Authors: Yishu Miao, Lei Yu, Phil Blunsom
Date: 2015-11-19 00:00:00
Tasks: Answer Selection, Question Answering, Topic Models, Variational Inference

Recent advances in neural variational inference have spawned a renaissance in
deep latent variable models. In this paper we introduce a generic variational
inference framework for generative and conditional models of text. While
traditional variational methods derive an analytic approximation for the
intractable distributions over latent variables, here we construct an inference
network conditioned on the discrete text input to provide the variational
distribution. We validate this framework on two very different text modelling
applications, generative document modelling and supervised question answering.
Our neural variational document model combines a continuous stochastic document
representation with a bag-of-words generative model and achieves the lowest
reported perplexities on two standard test corpora. The neural answer selection
model employs a stochastic representation layer within an attention mechanism
to extract the semantics between a question and answer pair. On two question
answering benchmarks this model exceeds all previous published benchmarks.
Scalable Algorithms for Tractable Schatten Quasi-Norm Minimization
Authors: Fanhua Shang, Yuanyuan Liu, James Cheng
Date: 2016-06-04 00:00:00
Tasks: Matrix Completion

The Schatten-p quasi-norm $(0<p<1)$ is usually used to replace the standard
nuclear norm in order to approximate the rank function more accurately.
However, existing Schatten-p quasi-norm minimization algorithms involve
singular value decomposition (SVD) or eigenvalue decomposition (EVD) in each
iteration, and thus may become very slow and impractical for large-scale
problems. In this paper, we first define two tractable Schatten quasi-norms,
i.e., the Frobenius/nuclear hybrid and bi-nuclear quasi-norms, and then prove
that they are in essence the Schatten-2/3 and 1/2 quasi-norms, respectively,
which lead to the design of very efficient algorithms that only need to update
two much smaller factor matrices. We also design two efficient proximal
alternating linearized minimization algorithms for solving representative
matrix completion problems. Finally, we provide the global convergence and
performance guarantees for our algorithms, which have better convergence
properties than existing algorithms. Experimental results on synthetic and
real-world data show that our algorithms are more accurate than the
state-of-the-art methods, and are orders of magnitude faster.
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
Authors: Tim Salimans, Diederik P. Kingma
Date: 2016-02-25 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Deep Reinforcement Learning, Image Classification, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We present weight normalization: a reparameterization of the weight vectors
in a neural network that decouples the length of those weight vectors from
their direction. By reparameterizing the weights in this way we improve the
conditioning of the optimization problem and we speed up convergence of
stochastic gradient descent. Our reparameterization is inspired by batch
normalization but does not introduce any dependencies between the examples in a
minibatch. This means that our method can also be applied successfully to
recurrent models such as LSTMs and to noise-sensitive applications such as deep
reinforcement learning or generative models, for which batch normalization is
less well suited. Although our method is much simpler, it still provides much
of the speed-up of full batch normalization. In addition, the computational
overhead of our method is lower, permitting more optimization steps to be taken
in the same amount of time. We demonstrate the usefulness of our method on
applications in supervised image recognition, generative modelling, and deep
reinforcement learning.