pwc-archive/papers-with-abstracts huggingface dataset:
High-Order Stochastic Gradient Thermostats for Bayesian Learning of Deep Models
Authors: Chunyuan Li, Changyou Chen, Kai Fan, Lawrence Carin
Date: 2015-12-23 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Learning in deep models using Bayesian methods has generated significant
attention recently. This is largely because of the feasibility of modern
Bayesian methods to yield scalable learning and inference, while maintaining a
measure of uncertainty in the model parameters. Stochastic gradient MCMC
algorithms (SG-MCMC) are a family of diffusion-based sampling methods for
large-scale Bayesian learning. In SG-MCMC, multivariate stochastic gradient
thermostats (mSGNHT) augment each parameter of interest, with a momentum and a
thermostat variable to maintain stationary distributions as target posterior
distributions. As the number of variables in a continuous-time diffusion
increases, its numerical approximation error becomes a practical bottleneck, so
better use of a numerical integrator is desirable. To this end, we propose use
of an efficient symmetric splitting integrator in mSGNHT, instead of the
traditional Euler integrator. We demonstrate that the proposed scheme is more
accurate, robust, and converges faster. These properties are demonstrated to be
desirable in Bayesian deep learning. Extensive experiments on two canonical
models and their deep extensions demonstrate that the proposed scheme improves
general Bayesian posterior sampling, particularly for deep models.
The Max $K$-Armed Bandit: PAC Lower Bounds and Efficient Algorithms
Authors: Yahel David, Nahum Shimkin
Date: 2015-12-23 00:00:00

We consider the Max $K$-Armed Bandit problem, where a learning agent is faced
with several stochastic arms, each a source of i.i.d. rewards of unknown
distribution. At each time step the agent chooses an arm, and observes the
reward of the obtained sample. Each sample is considered here as a separate
item with the reward designating its value, and the goal is to find an item
with the highest possible value. Our basic assumption is a known lower bound on
the {\em tail function} of the reward distributions. Under the PAC framework,
we provide a lower bound on the sample complexity of any
$(\epsilon,\delta)$-correct algorithm, and propose an algorithm that attains
this bound up to logarithmic factors. We analyze the robustness of the proposed
algorithm and in addition, we compare the performance of this algorithm to the
variant in which the arms are not distinguishable by the agent and are chosen
randomly at each stage. Interestingly, when the maximal rewards of the arms
happen to be similar, the latter approach may provide better performance.
Representation and Coding of Signal Geometry
Authors: Petros T. Boufounos, Shantanu Rane, Hassan Mansour
Date: 2015-12-23 00:00:00

Approaches to signal representation and coding theory have traditionally
focused on how to best represent signals using parsimonious representations
that incur the lowest possible distortion. Classical examples include linear
and non-linear approximations, sparse representations, and rate-distortion
theory. Very often, however, the goal of processing is to extract specific
information from the signal, and the distortion should be measured on the
extracted information. The corresponding representation should, therefore,
represent that information as parsimoniously as possible, without necessarily
accurately representing the signal itself.
  In this paper, we examine the problem of encoding signals such that
sufficient information is preserved about their pairwise distances and their
inner products. For that goal, we consider randomized embeddings as an encoding
mechanism and provide a framework to analyze their performance. We also
demonstrate that it is possible to design the embedding such that it represents
different ranges of distances with different precision. These embeddings also
allow the computation of kernel inner products with control on their inner
product-preserving properties. Our results provide a broad framework to design
and analyze embeddins, and generalize existing results in this area, such as
random Fourier kernels and universal embeddings.
Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayes
Authors: Ryan Giordano, Tamara Broderick, Michael Jordan
Date: 2015-06-12 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Form

Mean field variational Bayes (MFVB) is a popular posterior approximation
method due to its fast runtime on large-scale data sets. However, it is well
known that a major failing of MFVB is that it underestimates the uncertainty of
model variables (sometimes severely) and provides no information about model
variable covariance.
  We generalize linear response methods from statistical physics to deliver
accurate uncertainty estimates for model variables---both for individual
variables and coherently across variables. We call our method linear response
variational Bayes (LRVB). When the MFVB posterior approximation is in the
exponential family, LRVB has a simple, analytic form, even for non-conjugate
models. Indeed, we make no assumptions about the form of the true posterior. We
demonstrate the accuracy and scalability of our method on a range of models for
both simulated and real data.
k-Means Clustering Is Matrix Factorization
Authors: Christian Bauckhage
Date: 2015-12-23 00:00:00
Tasks: Clustering

We show that the objective function of conventional k-means clustering can be
expressed as the Frobenius norm of the difference of a data matrix and a low
rank approximation of that data matrix. In short, we show that k-means
clustering is a matrix factorization problem. These notes are meant as a
reference and intended to provide a guided tour towards a result that is often
mentioned but seldom made explicit in the literature.
Connectivity Preserving Multivalued Functions in Digital Topology
Authors: Laurence Boxer, P. Christopher Staecker
Date: 2015-04-09 00:00:00

We study connectivity preserving multivalued functions between digital
images. This notion generalizes that of continuous multivalued functions
studied mostly in the setting of the digital plane $Z^2$. We show that
connectivity preserving multivalued functions, like continuous multivalued
functions, are appropriate models for digital morpholological operations.
Connectivity preservation, unlike continuity, is preserved by compositions, and
generalizes easily to higher dimensions and arbitrary adjacency relations.
Convolutional Architecture Exploration for Action Recognition and Image Classification
Authors: J. T. Turner, David Aha, Leslie Smith, Kalyan Moy Gupta
Date: 2015-12-23 00:00:00
Tasks: Action Recognition, Classification, General Classification, image-classification, Image Classification, Temporal Action Localization

Convolutional Architecture for Fast Feature Encoding (CAFFE) [11] is a
software package for the training, classifying, and feature extraction of
images. The UCF Sports Action dataset is a widely used machine learning dataset
that has 200 videos taken in 720x480 resolution of 9 different sporting
activities: diving, golf, swinging, kicking, lifting, horseback riding,
running, skateboarding, swinging (various gymnastics), and walking. In this
report we report on a caffe feature extraction pipeline of images taken from
the videos of the UCF Sports Action dataset. A similar test was performed on
overfeat, and results were inferior to caffe. This study is intended to explore
the architecture and hyper parameters needed for effective static analysis of
action in videos and classification over a variety of image datasets.
The ERA of FOLE: Foundation
Authors: Robert E. Kent
Date: 2015-12-23 00:00:00
Tasks: Attribute, Form, General Classification

This paper discusses the representation of ontologies in the first-order logical environment {\ttfamily FOLE}. An ontology defines the primitives with which to model the knowledge resources for a community of discourse. These primitives consist of classes, relationships and properties. An ontology uses formal axioms to constrain the interpretation of these primitives. In short, an ontology specifies a logical theory. This paper continues the discussion of the representation and interpretation of ontologies in the first-order logical environment {\ttfamily FOLE}. The formalism and semantics of (many-sorted) first-order logic can be developed in both a \emph{classification form} and an \emph{interpretation form}. Two papers, the current paper, defining the concept of a structure, and ``The {\ttfamily ERA} of {\ttfamily FOLE}: Superstructure'', defining the concept of a sound logic, represent the \emph{classification form}, corresponding to ideas discussed in the ``Information Flow Framework''. Two papers, ``The {\ttfamily FOLE} Table'', defining the concept of a relational table, and ``The {\ttfamily FOLE} Database'', defining the concept of a relational database, represent the \emph{interpretation form}, expanding on material found in the paper ``Database Semantics''. Although the classification form follows the entity-relationship-attribute data model of Chen, the interpretation form incorporates the relational data model of Codd. A fifth paper ``{\ttfamily FOLE} Equivalence'' proves that the classification form is equivalent to the interpretation form. In general, the {\ttfamily FOLE} representation uses a conceptual structures approach, that is completely compatible with the theory of institutions, formal concept analysis and information flow.
Adaptive Algorithms for Online Convex Optimization with Long-term Constraints
Authors: Rodolphe Jenatton, Jim Huang, Cédric Archambeau
Date: 2015-12-23 00:00:00

We present an adaptive online gradient descent algorithm to solve online
convex optimization problems with long-term constraints , which are constraints
that need to be satisfied when accumulated over a finite number of rounds T ,
but can be violated in intermediate rounds. For some user-defined trade-off
parameter $\beta$ $\in$ (0, 1), the proposed algorithm achieves cumulative
regret bounds of O(T^max{$\beta$,1--$\beta$}) and O(T^(1--$\beta$/2)) for the
loss and the constraint violations respectively. Our results hold for convex
losses and can handle arbitrary convex constraints without requiring knowledge
of the number of rounds in advance. Our contributions improve over the best
known cumulative regret bounds by Mahdavi, et al. (2012) that are respectively
O(T^1/2) and O(T^3/4) for general convex domains, and respectively O(T^2/3) and
O(T^2/3) when further restricting to polyhedral domains. We supplement the
analysis with experiments validating the performance of our algorithm in
practice.
A Deep Generative Deconvolutional Image Model
Authors: Yunchen Pu, Xin Yuan, Andrew Stevens, Chunyuan Li, Lawrence Carin
Date: 2015-12-23 00:00:00
Tasks: Dictionary Learning, Image Generation, model

A deep generative model is developed for representation and analysis of
images, based on a hierarchical convolutional dictionary-learning framework.
Stochastic {\em unpooling} is employed to link consecutive layers in the model,
yielding top-down image generation. A Bayesian support vector machine is linked
to the top-layer features, yielding max-margin discrimination. Deep
deconvolutional inference is employed when testing, to infer the latent
features, and the top-layer features are connected with the max-margin
classifier for discrimination tasks. The model is efficiently trained using a
Monte Carlo expectation-maximization (MCEM) algorithm, with implementation on
graphical processor units (GPUs) for efficient large-scale learning, and fast
testing. Excellent results are obtained on several benchmark datasets,
including ImageNet, demonstrating that the proposed model achieves results that
are highly competitive with similarly sized convolutional neural networks.
Latent Variable Modeling with Diversity-Inducing Mutual Angular Regularization
Authors: Pengtao Xie, Yuntian Deng, Eric Xing
Date: 2015-12-23 00:00:00
Tasks: Diversity, Metric Learning

Latent Variable Models (LVMs) are a large family of machine learning models
providing a principled and effective way to extract underlying patterns,
structure and knowledge from observed data. Due to the dramatic growth of
volume and complexity of data, several new challenges have emerged and cannot
be effectively addressed by existing LVMs: (1) How to capture long-tail
patterns that carry crucial information when the popularity of patterns is
distributed in a power-law fashion? (2) How to reduce model complexity and
computational cost without compromising the modeling power of LVMs? (3) How to
improve the interpretability and reduce the redundancy of discovered patterns?
To addresses the three challenges discussed above, we develop a novel
regularization technique for LVMs, which controls the geometry of the latent
space during learning to enable the learned latent components of LVMs to be
diverse in the sense that they are favored to be mutually different from each
other, to accomplish long-tail coverage, low redundancy, and better
interpretability. We propose a mutual angular regularizer (MAR) to encourage
the components in LVMs to have larger mutual angles. The MAR is non-convex and
non-smooth, entailing great challenges for optimization. To cope with this
issue, we derive a smooth lower bound of the MAR and optimize the lower bound
instead. We show that the monotonicity of the lower bound is closely aligned
with the MAR to qualify the lower bound as a desirable surrogate of the MAR.
Using neural network (NN) as an instance, we analyze how the MAR affects the
generalization performance of NN. On two popular latent variable models ---
restricted Boltzmann machine and distance metric learning, we demonstrate that
MAR can effectively capture long-tail patterns, reduce model complexity without
sacrificing expressivity and improve interpretability.
Marginal likelihood and model selection for Gaussian latent tree and forest models
Authors: Mathias Drton, Shaowei Lin, Luca Weihs, Piotr Zwiernik
Date: 2014-12-29 00:00:00
Tasks: Bayesian Inference, Model Selection

Gaussian latent tree models, or more generally, Gaussian latent forest models
have Fisher-information matrices that become singular along interesting
submodels, namely, models that correspond to subforests. For these
singularities, we compute the real log-canonical thresholds (also known as
stochastic complexities or learning coefficients) that quantify the
large-sample behavior of the marginal likelihood in Bayesian inference. This
provides the information needed for a recently introduced generalization of the
Bayesian information criterion. Our mathematical developments treat the general
setting of Laplace integrals whose phase functions are sums of squared
differences between monomials and constants. We clarify how in this case real
log-canonical thresholds can be computed using polyhedral geometry, and we show
how to apply the general theory to the Laplace integrals associated with
Gaussian latent tree and forest models. In simulations and a data example, we
demonstrate how the mathematical knowledge can be applied in model selection.
Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation
Authors: Suhas Sreehari, S. V. Venkatakrishnan, Brendt Wohlberg, Lawrence F. Drummy, Jeffrey P. Simmons, Charles A. Bouman
Date: 2015-12-23 00:00:00
Tasks: Denoising, Electron Tomography

Many material and biological samples in scientific imaging are characterized
by non-local repeating structures. These are studied using scanning electron
microscopy and electron tomography. Sparse sampling of individual pixels in a
2D image acquisition geometry, or sparse sampling of projection images with
large tilt increments in a tomography experiment, can enable high speed data
acquisition and minimize sample damage caused by the electron beam.
  In this paper, we present an algorithm for electron tomographic
reconstruction and sparse image interpolation that exploits the non-local
redundancy in images. We adapt a framework, termed plug-and-play (P&P) priors,
to solve these imaging problems in a regularized inversion setting. The power
of the P&P approach is that it allows a wide array of modern denoising
algorithms to be used as a "prior model" for tomography and image
interpolation. We also present sufficient mathematical conditions that ensure
convergence of the P&P approach, and we use these insights to design a new
non-local means denoising algorithm. Finally, we demonstrate that the algorithm
produces higher quality reconstructions on both simulated and real electron
microscope data, along with improved convergence properties compared to other
methods.
Mid-level Representation for Visual Recognition
Authors: Moin Nabi
Date: 2015-12-23 00:00:00
Tasks: object-detection, Object Detection, Video Understanding

Visual Recognition is one of the fundamental challenges in AI, where the goal
is to understand the semantics of visual data. Employing mid-level
representation, in particular, shifted the paradigm in visual recognition. The
mid-level image/video representation involves discovering and training a set of
mid-level visual patterns (e.g., parts and attributes) and represent a given
image/video utilizing them. The mid-level patterns can be extracted from images
and videos using the motion and appearance information of visual phenomenas.
This thesis targets employing mid-level representations for different
high-level visual recognition tasks, namely (i)image understanding and
(ii)video understanding.
  In the case of image understanding, we focus on object detection/recognition
task. We investigate on discovering and learning a set of mid-level patches to
be used for representing the images of an object category. We specifically
employ the discriminative patches in a subcategory-aware webly-supervised
fashion. We, additionally, study the outcomes provided by employing the
subcategory-based models for undoing dataset bias.
Topical differences between Chinese language Twitter and Sina Weibo
Authors: Qian Zhang, Bruno Gonçalves
Date: 2015-12-22 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction

Sina Weibo, China's most popular microblogging platform, is currently used by
over $500M$ users and is considered to be a proxy of Chinese social life. In
this study, we contrast the discussions occurring on Sina Weibo and on Chinese
language Twitter in order to observe two different strands of Chinese culture:
people within China who use Sina Weibo with its government imposed restrictions
and those outside that are free to speak completely anonymously. We first
propose a simple ad-hoc algorithm to identify topics of Tweets and Weibo.
Different from previous works on micro-message topic detection, our algorithm
considers topics of the same contents but with different \#tags. Our algorithm
can also detect topics for Tweets and Weibos without any \#tags. Using a large
corpus of Weibo and Chinese language tweets, covering the period from January
$1$ to December $31$, $2012$, we obtain a list of topics using clustered \#tags
that we can then use to compare the two platforms. Surprisingly, we find that
there are no common entries among the Top $100$ most popular topics.
Furthermore, only $9.2\%$ of tweets correspond to the Top $1000$ topics on Sina
Weibo platform, and conversely only $4.4\%$ of weibos were found to discuss the
most popular Twitter topics. Our results reveal significant differences in
social attention on the two platforms, with most popular topics on Sina Weibo
relating to entertainment while most tweets corresponded to cultural or
political contents that is practically non existent in Sina Weibo.
Mapping Auto-context Decision Forests to Deep ConvNets for Semantic Segmentation
Authors: David L. Richmond, Dagmar Kainmueller, Michael Y. Yang, Eugene W. Myers, Carsten Rother
Date: 2015-07-27 00:00:00
Tasks: Semantic Segmentation

We consider the task of pixel-wise semantic segmentation given a small set of
labeled training images. Among two of the most popular techniques to address
this task are Decision Forests (DF) and Neural Networks (NN). In this work, we
explore the relationship between two special forms of these techniques: stacked
DFs (namely Auto-context) and deep Convolutional Neural Networks (ConvNet). Our
main contribution is to show that Auto-context can be mapped to a deep ConvNet
with novel architecture, and thereby trained end-to-end. This mapping can be
used as an initialization of a deep ConvNet, enabling training even in the face
of very limited amounts of training data. We also demonstrate an approximate
mapping back from the refined ConvNet to a second stacked DF, with improved
performance over the original. We experimentally verify that these mappings
outperform stacked DFs for two different applications in computer vision and
biology: Kinect-based body part labeling from depth images, and somite
segmentation in microscopy images of developing zebrafish. Finally, we revisit
the core mapping from a Decision Tree (DT) to a NN, and show that it is also
possible to map a fuzzy DT, with sigmoidal split decisions, to a NN. This
addresses multiple limitations of the previous mapping, and yields new insights
into the popular Rectified Linear Unit (ReLU), and more recently proposed
concatenated ReLU (CReLU), activation functions.
Causal Inference by Identification of Vector Autoregressive Processes with Hidden Components
Authors: Philipp Geiger, Kun Zhang, Mingming Gong, Dominik Janzing, Bernhard Schölkopf
Date: 2014-11-14 00:00:00
Tasks: Causal Inference, Time Series, Time Series Analysis, valid

A widely applied approach to causal inference from a non-experimental time
series $X$, often referred to as "(linear) Granger causal analysis", is to
regress present on past and interpret the regression matrix $\hat{B}$ causally.
However, if there is an unmeasured time series $Z$ that influences $X$, then
this approach can lead to wrong causal conclusions, i.e., distinct from those
one would draw if one had additional information such as $Z$. In this paper we
take a different approach: We assume that $X$ together with some hidden $Z$
forms a first order vector autoregressive (VAR) process with transition matrix
$A$, and argue why it is more valid to interpret $A$ causally instead of
$\hat{B}$. Then we examine under which conditions the most important parts of
$A$ are identifiable or almost identifiable from only $X$. Essentially,
sufficient conditions are (1) non-Gaussian, independent noise or (2) no
influence from $X$ to $Z$. We present two estimation algorithms that are
tailored towards conditions (1) and (2), respectively, and evaluate them on
synthetic and real-world data. We discuss how to check the model using $X$.