pwc-archive/papers-with-abstracts huggingface dataset:
Exploring Nearest Neighbor Approaches for Image Captioning
Authors: Jacob Devlin, Saurabh Gupta, Ross Girshick, Margaret Mitchell, C. Lawrence Zitnick
Date: 2015-05-17 00:00:00
Tasks: Image Captioning

We explore a variety of nearest neighbor baseline approaches for image
captioning. These approaches find a set of nearest neighbor images in the
training set from which a caption may be borrowed for the query image. We
select a caption for the query image by finding the caption that best
represents the "consensus" of the set of candidate captions gathered from the
nearest neighbor images. When measured by automatic evaluation metrics on the
MS COCO caption evaluation server, these approaches perform as well as many
recent approaches that generate novel captions. However, human studies show
that a method that generates novel captions is still preferred over the nearest
neighbor approach.
The Best of Both Worlds: Combining Data-independent and Data-driven Approaches for Action Recognition
Authors: Zhenzhong Lan, Dezhong Yao, Ming Lin, Shoou-I Yu, Alexander Hauptmann
Date: 2015-05-17 00:00:00
Tasks: Action Recognition, Multi-class Classification, Object Recognition, Re-Ranking, Temporal Action Localization

Motivated by the success of data-driven convolutional neural networks (CNNs)
in object recognition on static images, researchers are working hard towards
developing CNN equivalents for learning video features. However, learning video
features globally has proven to be quite a challenge due to its high
dimensionality, the lack of labelled data and the difficulty in processing
large-scale video data. Therefore, we propose to leverage effective techniques
from both data-driven and data-independent approaches to improve action
recognition system.
  Our contribution is three-fold. First, we propose a two-stream Stacked
Convolutional Independent Subspace Analysis (ConvISA) architecture to show that
unsupervised learning methods can significantly boost the performance of
traditional local features extracted from data-independent models. Second, we
demonstrate that by learning on video volumes detected by Improved Dense
Trajectory (IDT), we can seamlessly combine our novel local descriptors with
hand-crafted descriptors. Thus we can utilize available feature enhancing
techniques developed for hand-crafted descriptors. Finally, similar to
multi-class classification framework in CNNs, we propose a training-free
re-ranking technique that exploits the relationship among action classes to
improve the overall performance. Our experimental results on four benchmark
action recognition datasets show significantly improved performance.
CCG Parsing and Multiword Expressions
Authors: Miryam de Lhoneux
Date: 2015-05-17 00:00:00

This thesis presents a study about the integration of information about
Multiword Expressions (MWEs) into parsing with Combinatory Categorial Grammar
(CCG). We build on previous work which has shown the benefit of adding
information about MWEs to syntactic parsing by implementing a similar pipeline
with CCG parsing. More specifically, we collapse MWEs to one token in training
and test data in CCGbank, a corpus which contains sentences annotated with CCG
derivations. Our collapsing algorithm however can only deal with MWEs when they
form a constituent in the data which is one of the limitations of our approach.
  We study the effect of collapsing training and test data. A parsing effect
can be obtained if collapsed data help the parser in its decisions and a
training effect can be obtained if training on the collapsed data improves
results. We also collapse the gold standard and show that our model
significantly outperforms the baseline model on our gold standard, which
indicates that there is a training effect. We show that the baseline model
performs significantly better on our gold standard when the data are collapsed
before parsing than when the data are collapsed after parsing which indicates
that there is a parsing effect. We show that these results can lead to improved
performance on the non-collapsed standard benchmark although we fail to show
that it does so significantly. We conclude that despite the limited settings,
there are noticeable improvements from using MWEs in parsing. We discuss ways
in which the incorporation of MWEs into parsing can be improved and hypothesize
that this will lead to more substantial results.
  We finally show that turning the MWE recognition part of the pipeline into an
experimental part is a useful thing to do as we obtain different results with
different recognizers.
Identifying Cover Songs Using Information-Theoretic Measures of Similarity
Authors: Peter Foster, Simon Dixon, Anssi Klapuri
Date: 2014-07-09 00:00:00
Tasks: Cover song identification, Time Series, Time Series Analysis

This paper investigates methods for quantifying similarity between audio
signals, specifically for the task of of cover song detection. We consider an
information-theoretic approach, where we compute pairwise measures of
predictability between time series. We compare discrete-valued approaches
operating on quantised audio features, to continuous-valued approaches. In the
discrete case, we propose a method for computing the normalised compression
distance, where we account for correlation between time series. In the
continuous case, we propose to compute information-based measures of similarity
as statistics of the prediction error between time series. We evaluate our
methods on two cover song identification tasks using a data set comprised of
300 Jazz standards and using the Million Song Dataset. For both datasets, we
observe that continuous-valued approaches outperform discrete-valued
approaches. We consider approaches to estimating the normalised compression
distance (NCD) based on string compression and prediction, where we observe
that our proposed normalised compression distance with alignment (NCDA)
improves average performance over NCD, for sequential compression algorithms.
Finally, we demonstrate that continuous-valued distances may be combined to
improve performance with respect to baseline approaches. Using a large-scale
filter-and-refine approach, we demonstrate state-of-the-art performance for
cover song identification using the Million Song Dataset.
Shrinkage degree in $L_2$-re-scale boosting for regression
Authors: Lin Xu, Shao-Bo Lin, Yao Wang, Zongben Xu
Date: 2015-05-17 00:00:00
Tasks: regression

Re-scale boosting (RBoosting) is a variant of boosting which can essentially
improve the generalization performance of boosting learning. The key feature of
RBoosting lies in introducing a shrinkage degree to re-scale the ensemble
estimate in each gradient-descent step. Thus, the shrinkage degree determines
the performance of RBoosting.
  The aim of this paper is to develop a concrete analysis concerning how to
determine the shrinkage degree in $L_2$-RBoosting. We propose two feasible ways
to select the shrinkage degree. The first one is to parameterize the shrinkage
degree and the other one is to develope a data-driven approach of it. After
rigorously analyzing the importance of the shrinkage degree in $L_2$-RBoosting
learning, we compare the pros and cons of the proposed methods. We find that
although these approaches can reach the same learning rates, the structure of
the final estimate of the parameterized approach is better, which sometimes
yields a better generalization capability when the number of sample is finite.
With this, we recommend to parameterize the shrinkage degree of
$L_2$-RBoosting. To this end, we present an adaptive parameter-selection
strategy for shrinkage degree and verify its feasibility through both
theoretical analysis and numerical verification.
  The obtained results enhance the understanding of RBoosting and further give
guidance on how to use $L_2$-RBoosting for regression tasks.
Learning Deconvolution Network for Semantic Segmentation
Authors: Hyeonwoo Noh, Seunghoon Hong, Bohyung Han
Date: 2015-05-17 00:00:00
Proceeding: ICCV 2015 12
Tasks: Segmentation, Semantic Segmentation

We propose a novel semantic segmentation algorithm by learning a
deconvolution network. We learn the network on top of the convolutional layers
adopted from VGG 16-layer net. The deconvolution network is composed of
deconvolution and unpooling layers, which identify pixel-wise class labels and
predict segmentation masks. We apply the trained network to each proposal in an
input image, and construct the final semantic segmentation map by combining the
results from all proposals in a simple manner. The proposed algorithm mitigates
the limitations of the existing methods based on fully convolutional networks
by integrating deep deconvolution network and proposal-wise prediction; our
segmentation method typically identifies detailed structures and handles
objects in multiple scales naturally. Our network demonstrates outstanding
performance in PASCAL VOC 2012 dataset, and we achieve the best accuracy
(72.5%) among the methods trained with no external data through ensemble with
the fully convolutional network.
Salient Structure Detection by Context-Guided Visual Search
Authors: Kai-Fu Yang, Hui Li, Chao-Yi Li, Yong-Jie Li
Date: 2015-05-17 00:00:00
Tasks: Bayesian Inference, Object, object-detection, Object Detection, Prediction, RGB Salient Object Detection, Salient Object Detection

We define the task of salient structure (SS) detection to unify the
saliency-related tasks like fixation prediction, salient object detection, and
other detection of structures of interest. In this study, we propose a unified
framework for SS detection by modeling the two-pathway-based guided search
strategy of biological vision. Firstly, context-based spatial prior (CBSP) is
extracted based on the layout of edges in the given scene along a fast visual
pathway, called non-selective pathway. This is a rough and non-selective
estimation of the locations where the potential SSs present. Secondly, another
flow of local feature extraction is executed in parallel along the selective
pathway. Finally, Bayesian inference is used to integrate local cues guided by
CBSP, and to predict the exact locations of SSs in the input scene. The
proposed model is invariant to size and features of objects. Experimental
results on four datasets (two fixation prediction datasets and two salient
object datasets) demonstrate that our system achieves competitive performance
for SS detection (i.e., both the tasks of fixation prediction and salient
object detection) comparing to the state-of-the-art methods.
Evolving Spiking Networks with Variable Resistive Memories
Authors: Gerard David Howard, Larry Bull, Ben De Lacy Costello, Andrew Adamatzky, Ella Gale
Date: 2015-05-17 00:00:00

Neuromorphic computing is a brainlike information processing paradigm that
requires adaptive learning mechanisms. A spiking neuro-evolutionary system is
used for this purpose; plastic resistive memories are implemented as synapses
in spiking neural networks. The evolutionary design process exploits parameter
self-adaptation and allows the topology and synaptic weights to be evolved for
each network in an autonomous manner. Variable resistive memories are the focus
of this research; each synapse has its own conductance profile which modifies
the plastic behaviour of the device and may be altered during evolution. These
variable resistive networks are evaluated on a noisy robotic dynamic-reward
scenario against two static resistive memories and a system containing standard
connections only. Results indicate that the extra behavioural degrees of
freedom available to the networks incorporating variable resistive memories
enable them to outperform the comparative synapse types.
Tight bounds for learning a mixture of two gaussians
Authors: Moritz Hardt, Eric Price
Date: 2014-04-19 00:00:00
Tasks: Dimensionality Reduction, Vocal Bursts Valence Prediction

We consider the problem of identifying the parameters of an unknown mixture
of two arbitrary $d$-dimensional gaussians from a sequence of independent
random samples. Our main results are upper and lower bounds giving a
computationally efficient moment-based estimator with an optimal convergence
rate, thus resolving a problem introduced by Pearson (1894). Denoting by
$\sigma^2$ the variance of the unknown mixture, we prove that
$\Theta(\sigma^{12})$ samples are necessary and sufficient to estimate each
parameter up to constant additive error when $d=1.$ Our upper bound extends to
arbitrary dimension $d>1$ up to a (provably necessary) logarithmic loss in $d$
using a novel---yet simple---dimensionality reduction technique. We further
identify several interesting special cases where the sample complexity is
notably smaller than our optimal worst-case bound. For instance, if the means
of the two components are separated by $\Omega(\sigma)$ the sample complexity
reduces to $O(\sigma^2)$ and this is again optimal.
  Our results also apply to learning each component of the mixture up to small
error in total variation distance, where our algorithm gives strong
improvements in sample complexity over previous work. We also extend our lower
bound to mixtures of $k$ Gaussians, showing that $\Omega(\sigma^{6k-2})$
samples are necessary to estimate each parameter up to constant additive error.
Hyper-parameter optimization of Deep Convolutional Networks for object recognition
Authors: Sachin S. Talathi
Date: 2015-01-30 00:00:00
Tasks: BIG-bench Machine Learning, Object, Object Recognition

Recently sequential model based optimization (SMBO) has emerged as a
promising hyper-parameter optimization strategy in machine learning. In this
work, we investigate SMBO to identify architecture hyper-parameters of deep
convolution networks (DCNs) object recognition. We propose a simple SMBO
strategy that starts from a set of random initial DCN architectures to generate
new architectures, which on training perform well on a given dataset. Using the
proposed SMBO strategy we are able to identify a number of DCN architectures
that produce results that are comparable to state-of-the-art results on object
recognition benchmarks.
Efficient Approximations for the Marginal Likelihood of Incomplete Data Given a Bayesian Network
Authors: David Maxwell Chickering, David Heckerman
Date: 2013-02-13 00:00:00

We discuss Bayesian methods for learning Bayesian networks when data sets are
incomplete. In particular, we examine asymptotic approximations for the
marginal likelihood of incomplete data given a Bayesian network. We consider
the Laplace approximation and the less accurate but more efficient BIC/MDL
approximation. We also consider approximations proposed by Draper (1993) and
Cheeseman and Stutz (1995). These approximations are as efficient as BIC/MDL,
but their accuracy has not been studied in any depth. We compare the accuracy
of these approximations under the assumption that the Laplace approximation is
the most accurate. In experiments using synthetic data generated from discrete
naive-Bayes models having a hidden root node, we find that the CS measure is
the most accurate.
A New Look at Causal Independence
Authors: David Heckerman, John S. Breese
Date: 2013-02-27 00:00:00

Heckerman (1993) defined causal independence in terms of a set of temporal
conditional independence statements. These statements formalized certain types
of causal interaction where (1) the effect is independent of the order that
causes are introduced and (2) the impact of a single cause on the effect does
not depend on what other causes have previously been applied. In this paper, we
introduce an equivalent a temporal characterization of causal independence
based on a functional representation of the relationship between causes and the
effect. In this representation, the interaction between causes and effect can
be written as a nested decomposition of functions. Causal independence can be
exploited by representing this decomposition in the belief network, resulting
in representations that are more efficient for inference than general causal
models. We present empirical results showing the benefits of a
causal-independence representation for belief-network inference.
An Empirical Comparison of Three Inference Methods
Authors: David Heckerman
Date: 2013-03-27 00:00:00
Tasks: Diagnostic, Negation, Pathfinder

In this paper, an empirical evaluation of three inference methods for uncertain reasoning is presented in the context of Pathfinder, a large expert system for the diagnosis of lymph-node pathology. The inference procedures evaluated are (1) Bayes' theorem, assuming evidence is conditionally independent given each hypothesis; (2) odds-likelihood updating, assuming evidence is conditionally independent given each hypothesis and given the negation of each hypothesis; and (3) a inference method related to the Dempster-Shafer theory of belief. Both expert-rating and decision-theoretic metrics are used to compare the diagnostic accuracy of the inference methods.
Separable and transitive graphoids
Authors: Dan Geiger, David Heckerman
Date: 2013-03-27 00:00:00
Tasks: Sentence

We examine three probabilistic formulations of the sentence a and b are
totally unrelated with respect to a given set of variables U. First, two
variables a and b are totally independent if they are independent given any
value of any subset of the variables in U. Second, two variables are totally
uncoupled if U can be partitioned into two marginally independent sets
containing a and b respectively. Third, two variables are totally disconnected
if the corresponding nodes are disconnected in every belief network
representation. We explore the relationship between these three formulations of
unrelatedness and explain their relevance to the process of acquiring
probabilistic knowledge from human experts.
Similarity Networks for the Construction of Multiple-Faults Belief Networks
Authors: David Heckerman
Date: 2013-03-27 00:00:00

A similarity network is a tool for constructing belief networks for the
diagnosis of a single fault. In this paper, we examine modifications to the
similarity-network representation that facilitate the construction of belief
networks for the diagnosis of multiple coexisting faults.
Advances in Probabilistic Reasoning
Authors: Dan Geiger, David Heckerman
Date: 2013-03-20 00:00:00

This paper discuses multiple Bayesian networks representation paradigms for
encoding asymmetric independence assertions. We offer three contributions: (1)
an inference mechanism that makes explicit use of asymmetric independence to
speed up computations, (2) a simplified definition of similarity networks and
extensions of their theory, and (3) a generalized representation scheme that
encodes more types of asymmetric independence assertions than do similarity
networks.
An Approximate Nonmyopic Computation for Value of Information
Authors: David Heckerman, Eric J. Horvitz, Blackford Middleton
Date: 2013-03-20 00:00:00

Value-of-information analyses provide a straightforward means for selecting
the best next observation to make, and for determining whether it is better to
gather additional information or to act immediately. Determining the next best
test to perform, given a state of uncertainty about the world, requires a
consideration of the value of making all possible sequences of observations. In
practice, decision analysts and expert-system designers have avoided the
intractability of exact computation of the value of information by relying on a
myopic approximation. Myopic analyses are based on the assumption that only one
additional test will be performed, even when there is an opportunity to make a
large number of observations. We present a nonmyopic approximation for value of
information that bypasses the traditional myopic analyses by exploiting the
statistical properties of large samples.
Problem Formulation as the Reduction of a Decision Model
Authors: David Heckerman, Eric J. Horvitz
Date: 2013-03-27 00:00:00

In this paper, we extend the QMRDT probabilistic model for the domain of
internal medicine to include decisions about treatments. In addition, we
describe how we can use the comprehensive decision model to construct a simpler
decision model for a specific patient. In so doing, we transform the task of
problem formulation to that of narrowing of a larger problem.
Diagnosis of Multiple Faults: A Sensitivity Analysis
Authors: David Heckerman, Michael Shwe
Date: 2013-03-06 00:00:00
Tasks: Diagnostic, Sensitivity

We compare the diagnostic accuracy of three diagnostic inference models: the
simple Bayes model, the multimembership Bayes model, which is isomorphic to the
parallel combination function in the certainty-factor model, and a model that
incorporates the noisy OR-gate interaction. The comparison is done on 20
clinicopathological conference (CPC) cases from the American Journal of
Medicine-challenging cases describing actual patients often with multiple
disorders. We find that the distributions produced by the noisy OR model agree
most closely with the gold-standard diagnoses, although substantial differences
exist between the distributions and the diagnoses. In addition, we find that
the multimembership Bayes model tends to significantly overestimate the
posterior probabilities of diseases, whereas the simple Bayes model tends to
significantly underestimate the posterior probabilities. Our results suggest
that additional work to refine the noisy OR model for internal medicine will be
worthwhile.
Causal Independence for Knowledge Acquisition and Inference
Authors: David Heckerman
Date: 2013-03-06 00:00:00

I introduce a temporal belief-network representation of causal independence
that a knowledge engineer can use to elicit probabilistic models. Like the
current, atemporal belief-network representation of causal independence, the
new representation makes knowledge acquisition tractable. Unlike the atemproal
representation, however, the temporal representation can simplify inference,
and does not require the use of unobservable variables. The representation is
less general than is the atemporal representation, but appears to be useful for
many practical applications.
Inference Algorithms for Similarity Networks
Authors: Dan Geiger, David Heckerman
Date: 2013-03-06 00:00:00

We examine two types of similarity networks each based on a distinct notion
of relevance. For both types of similarity networks we present an efficient
inference algorithm that works under the assumption that every event has a
nonzero probability of occurrence. Another inference algorithm is developed for
type 1 similarity networks that works under no restriction, albeit less
efficiently.
A Decision-Based View of Causality
Authors: David Heckerman, Ross D. Shachter
Date: 2013-02-27 00:00:00
Tasks: Decision Making

Most traditional models of uncertainty have focused on the associational
relationship among variables as captured by conditional dependence. In order to
successfully manage intelligent systems for decision making, however, we must
be able to predict the effects of actions. In this paper, we attempt to unite
two branches of research that address such predictions: causal modeling and
decision analysis. First, we provide a definition of causal dependence in
decision-analytic terms, which we derive from consequences of causal dependence
cited in the literature. Using this definition, we show how causal dependence
can be represented within an influence diagram. In particular, we identify two
inadequacies of an ordinary influence diagram as a representation for cause. We
introduce a special class of influence diagrams, called causal influence
diagrams, which corrects one of these problems, and identify situations where
the other inadequacy can be eliminated. In addition, we describe the
relationships between Howard Canonical Form and existing graphical
representations of cause.