pwc-archive/papers-with-abstracts huggingface dataset:
Fast Optimization Algorithm on Riemannian Manifolds and Its Application in Low-Rank Representation
Authors: Haoran Chen, Yanfeng Sun, Junbin Gao, Yongli Hu
Date: 2015-12-07 00:00:00
Tasks: Matrix Completion

The paper addresses the problem of optimizing a class of composite functions
on Riemannian manifolds and a new first order optimization algorithm (FOA) with
a fast convergence rate is proposed. Through the theoretical analysis for FOA,
it has been proved that the algorithm has quadratic convergence. The
experiments in the matrix completion task show that FOA has better performance
than other first order optimization methods on Riemannian manifolds. A fast
subspace pursuit method based on FOA is proposed to solve the low-rank
representation model based on augmented Lagrange method on the low rank matrix
variety. Experimental results on synthetic and real data sets are presented to
demonstrate that both FOA and SP-RPRG(ALM) can achieve superior performance in
terms of faster convergence and higher accuracy.
Thinking Required
Authors: Kamil Rocki
Date: 2015-12-07 00:00:00

There exists a theory of a single general-purpose learning algorithm which
could explain the principles its operation. It assumes the initial rough
architecture, a small library of simple innate circuits which are prewired at
birth. and proposes that all significant mental algorithms are learned. Given
current understanding and observations, this paper reviews and lists the
ingredients of such an algorithm from architectural and functional
perspectives.
Rademacher Complexity of the Restricted Boltzmann Machine
Authors: Xiao Zhang
Date: 2015-12-07 00:00:00

Boltzmann machine, as a fundamental construction block of deep belief network
and deep Boltzmann machines, is widely used in deep learning community and
great success has been achieved. However, theoretical understanding of many
aspects of it is still far from clear. In this paper, we studied the Rademacher
complexity of both the asymptotic restricted Boltzmann machine and the
practical implementation with single-step contrastive divergence (CD-1)
procedure. Our results disclose the fact that practical implementation training
procedure indeed increased the Rademacher complexity of restricted Boltzmann
machines. A further research direction might be the investigation of the VC
dimension of a compositional function used in the CD-1 procedure.
Sparsifying Neural Network Connections for Face Recognition
Authors: Yi Sun, Xiaogang Wang, Xiaoou Tang
Date: 2015-12-07 00:00:00
Proceeding: CVPR 2016 6
Tasks: Face Recognition

This paper proposes to learn high-performance deep ConvNets with sparse
neural connections, referred to as sparse ConvNets, for face recognition. The
sparse ConvNets are learned in an iterative way, each time one additional layer
is sparsified and the entire model is re-trained given the initial weights
learned in previous iterations. One important finding is that directly training
the sparse ConvNet from scratch failed to find good solutions for face
recognition, while using a previously learned denser model to properly
initialize a sparser model is critical to continue learning effective features
for face recognition. This paper also proposes a new neural correlation-based
weight selection criterion and empirically verifies its effectiveness in
selecting informative connections from previously learned models in each
iteration. When taking a moderately sparse structure (26%-76% of weights in the
dense model), the proposed sparse ConvNet model significantly improves the face
recognition performance of the previous state-of-the-art DeepID2+ models given
the same training data, while it keeps the performance of the baseline model
with only 12% of the original parameters.
Using SVM to pre-classify government purchases
Authors: Thiago Marzagão
Date: 2015-12-07 00:00:00

The Brazilian government often misclassifies the goods it buys. That makes it
hard to audit government expenditures. We cannot know whether the price paid
for a ballpoint pen (code #7510) was reasonable if the pen was misclassified as
a technical drawing pen (code #6675) or as any other good. This paper shows how
we can use machine learning to reduce misclassification. I trained a support
vector machine (SVM) classifier that takes a product description as input and
returns the most likely category codes as output. I trained the classifier
using 20 million goods purchased by the Brazilian government between 1999-04-01
and 2015-04-02. In 83.3% of the cases the correct category code was one of the
three most likely category codes identified by the classifier. I used the
trained classifier to develop a web app that might help the government reduce
misclassification. I open sourced the code on GitHub; anyone can use and modify
it.
Probabilistic Structural Controllability in Causal Bayesian Networks
Authors: Ardavan Salehi Nobandegani, Ioannis N. Psaromiligkos
Date: 2015-12-07 00:00:00

Humans routinely confront the following key question which could be viewed as
a probabilistic variant of the controllability problem: While faced with an
uncertain environment governed by causal structures, how should they practice
their autonomy by intervening on driver variables, in order to increase (or
decrease) the probability of attaining their desired (or undesired) state for
some target variable? In this paper, for the first time, the problem of
probabilistic controllability in Causal Bayesian Networks (CBNs) is studied.
More specifically, the aim of this paper is two-fold: (i) to introduce and
formalize the problem of probabilistic structural controllability in CBNs, and
(ii) to identify a sufficient set of driver variables for the purpose of
probabilistic structural controllability of a generic CBN. We also elaborate on
the nature of minimality the identified set of driver variables satisfies. In
this context, the term "structural" signifies the condition wherein solely the
structure of the CBN is known.
Driverseat: Crowdstrapping Learning Tasks for Autonomous Driving
Authors: Pranav Rajpurkar, Toki Migimatsu, Jeff Kiske, Royce Cheng-Yue, Sameep Tandon, Tao Wang, Andrew Ng
Date: 2015-12-07 00:00:00
Tasks: Autonomous Driving, Lane Detection

While emerging deep-learning systems have outclassed knowledge-based
approaches in many tasks, their application to detection tasks for autonomous
technologies remains an open field for scientific exploration. Broadly, there
are two major developmental bottlenecks: the unavailability of comprehensively
labeled datasets and of expressive evaluation strategies. Approaches for
labeling datasets have relied on intensive hand-engineering, and strategies for
evaluating learning systems have been unable to identify failure-case
scenarios. Human intelligence offers an untapped approach for breaking through
these bottlenecks. This paper introduces Driverseat, a technology for embedding
crowds around learning systems for autonomous driving. Driverseat utilizes
crowd contributions for (a) collecting complex 3D labels and (b) tagging
diverse scenarios for ready evaluation of learning systems. We demonstrate how
Driverseat can crowdstrap a convolutional neural network on the lane-detection
task. More generally, crowdstrapping introduces a valuable paradigm for any
technology that can benefit from leveraging the powerful combination of human
and computer intelligence.
Fixation prediction with a combined model of bottom-up saliency and vanishing point
Authors: Mengyang Feng, Ali Borji, Huchuan Lu
Date: 2015-12-06 00:00:00

By predicting where humans look in natural scenes, we can understand how they
perceive complex natural scenes and prioritize information for further
high-level visual processing. Several models have been proposed for this
purpose, yet there is a gap between best existing saliency models and human
performance. While many researchers have developed purely computational models
for fixation prediction, less attempts have been made to discover cognitive
factors that guide gaze. Here, we study the effect of a particular type of
scene structural information, known as the vanishing point, and show that human
gaze is attracted to the vanishing point regions. We record eye movements of 10
observers over 532 images, out of which 319 have vanishing points. We then
construct a combined model of traditional saliency and a vanishing point
channel and show that our model outperforms state of the art saliency models
using three scores on our dataset.
Explaining reviews and ratings with PACO: Poisson Additive Co-Clustering
Authors: Chao-yuan Wu, Alex Beutel, Amr Ahmed, Alexander J. Smola
Date: 2015-12-06 00:00:00
Tasks: Clustering, Collaborative Filtering

Understanding a user's motivations provides valuable information beyond the
ability to recommend items. Quite often this can be accomplished by perusing
both ratings and review texts, since it is the latter where the reasoning for
specific preferences is explicitly expressed.
  Unfortunately matrix factorization approaches to recommendation result in
large, complex models that are difficult to interpret and give recommendations
that are hard to clearly explain to users. In contrast, in this paper, we
attack this problem through succinct additive co-clustering. We devise a novel
Bayesian technique for summing co-clusterings of Poisson distributions. With
this novel technique we propose a new Bayesian model for joint collaborative
filtering of ratings and text reviews through a sum of simple co-clusterings.
The simple structure of our model yields easily interpretable recommendations.
Even with a simple, succinct structure, our model outperforms competitors in
terms of predicting ratings with reviews.
Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees
Authors: François-Xavier Briol, Chris. J. Oates, Mark Girolami, Michael A. Osborne
Date: 2015-06-08 00:00:00
Proceeding: NeurIPS 2015 12

There is renewed interest in formulating integration as an inference problem,
motivated by obtaining a full distribution over numerical error that can be
propagated through subsequent computation. Current methods, such as Bayesian
Quadrature, demonstrate impressive empirical performance but lack theoretical
analysis. An important challenge is to reconcile these probabilistic
integrators with rigorous convergence guarantees. In this paper, we present the
first probabilistic integrator that admits such theoretical treatment, called
Frank-Wolfe Bayesian Quadrature (FWBQ). Under FWBQ, convergence to the true
value of the integral is shown to be exponential and posterior contraction
rates are proven to be superexponential. In simulations, FWBQ is competitive
with state-of-the-art methods and out-performs alternatives based on
Frank-Wolfe optimisation. Our approach is applied to successfully quantify
numerical error in the solution to a challenging model choice problem in
cellular biology.
Information Exchange and Learning Dynamics over Weakly-Connected Adaptive Networks
Authors: Bicheng Ying, Ali H. Sayed
Date: 2014-12-04 00:00:00
Tasks: Clustering

The paper examines the learning mechanism of adaptive agents over
weakly-connected graphs and reveals an interesting behavior on how information
flows through such topologies. The results clarify how asymmetries in the
exchange of data can mask local information at certain agents and make them
totally dependent on other agents. A leader-follower relationship develops with
the performance of some agents being fully determined by the performance of
other agents that are outside their domain of influence. This scenario can
arise, for example, due to intruder attacks by malicious agents or as the
result of failures by some critical links. The findings in this work help
explain why strong-connectivity of the network topology, adaptation of the
combination weights, and clustering of agents are important ingredients to
equalize the learning abilities of all agents against such disturbances. The
results also clarify how weak-connectivity can be helpful in reducing the
effect of outlier data on learning performance.
On The Direct Maximization of Quadratic Weighted Kappa
Authors: David Vaughn, Derek Justice
Date: 2015-09-23 00:00:00
Tasks: BIG-bench Machine Learning, Diabetic Retinopathy Detection

In recent years, quadratic weighted kappa has been growing in popularity in
the machine learning community as an evaluation metric in domains where the
target labels to be predicted are drawn from integer ratings, usually obtained
from human experts. For example, it was the metric of choice in several recent,
high profile machine learning contests hosted on Kaggle :
https://www.kaggle.com/c/asap-aes , https://www.kaggle.com/c/asap-sas ,
https://www.kaggle.com/c/diabetic-retinopathy-detection . Yet, little is
understood about the nature of this metric, its underlying mathematical
properties, where it fits among other common evaluation metrics such as mean
squared error (MSE) and correlation, or if it can be optimized analytically,
and if so, how. Much of this is due to the cumbersome way that this metric is
commonly defined. In this paper we first derive an equivalent but much simpler,
and more useful, definition for quadratic weighted kappa, and then employ this
alternate form to address the above issues.
The Next Best Underwater View
Authors: Mark Sheinin, Yoav Y. Schechner
Date: 2015-12-06 00:00:00
Proceeding: CVPR 2016 6

To image in high resolution large and occlusion-prone scenes, a camera must
move above and around. Degradation of visibility due to geometric occlusions
and distances is exacerbated by scattering, when the scene is in a
participating medium. Moreover, underwater and in other media, artificial
lighting is needed. Overall, data quality depends on the observed surface,
medium and the time-varying poses of the camera and light source. This work
proposes to optimize camera/light poses as they move, so that the surface is
scanned efficiently and the descattered recovery has the highest quality. The
work generalizes the next best view concept of robot vision to scattering media
and cooperative movable lighting. It also extends descattering to platforms
that move optimally. The optimization criterion is information gain, taken from
information theory. We exploit the existence of a prior rough 3D model, since
underwater such a model is routinely obtained using sonar. We demonstrate this
principle in a scaled-down setup.
Image reconstruction from dense binary pixels
Authors: Or Litany, Tal Remez, Alex Bronstein
Date: 2015-12-06 00:00:00
Tasks: Image Reconstruction

Recently, the dense binary pixel Gigavision camera had been introduced,
emulating a digital version of the photographic film. While seems to be a
promising solution for HDR imaging, its output is not directly usable and
requires an image reconstruction process. In this work, we formulate this
problem as the minimization of a convex objective combining a
maximum-likelihood term with a sparse synthesis prior. We present MLNet - a
novel feed-forward neural network, producing acceptable output quality at a
fixed complexity and is two orders of magnitude faster than iterative
algorithms. We present state of the art results in the abstract.
Want Answers? A Reddit Inspired Study on How to Pose Questions
Authors: Danish, Yogesh Dahiya, Partha Talukdar
Date: 2015-12-06 00:00:00

Questions form an integral part of our everyday communication, both offline
and online. Getting responses to our questions from others is fundamental to
satisfying our information need and in extending our knowledge boundaries. A
question may be represented using various factors such as social, syntactic,
semantic, etc. We hypothesize that these factors contribute with varying
degrees towards getting responses from others for a given question. We perform
a thorough empirical study to measure effects of these factors using a novel
question and answer dataset from the website Reddit.com. To the best of our
knowledge, this is the first such analysis of its kind on this important topic.
We also use a sparse nonnegative matrix factorization technique to
automatically induce interpretable semantic factors from the question dataset.
We also document various patterns on response prediction we observe during our
analysis in the data. For instance, we found that preference-probing questions
are scantily answered. Our method is robust to capture such latent response
factors. We hope to make our code and datasets publicly available upon
publication of the paper.
Separated by an Un-common Language: Towards Judgment Language Informed Vector Space Modeling
Authors: Ira Leviant, Roi Reichart
Date: 2015-08-01 00:00:00

A common evaluation practice in the vector space models (VSMs) literature is
to measure the models' ability to predict human judgments about lexical
semantic relations between word pairs. Most existing evaluation sets, however,
consist of scores collected for English word pairs only, ignoring the potential
impact of the judgment language in which word pairs are presented on the human
scores. In this paper we translate two prominent evaluation sets, wordsim353
(association) and SimLex999 (similarity), from English to Italian, German and
Russian and collect scores for each dataset from crowdworkers fluent in its
language. Our analysis reveals that human judgments are strongly impacted by
the judgment language. Moreover, we show that the predictions of monolingual
VSMs do not necessarily best correlate with human judgments made with the
language used for model training, suggesting that models and humans are
affected differently by the language they use when making semantic judgments.
Finally, we show that in a large number of setups, multilingual VSM combination
results in improved correlations with human judgments, suggesting that
multilingualism may partially compensate for the judgment language effect on
human judgments.
Fast Algorithms for Game-Theoretic Centrality Measures
Authors: Piotr Lech Szczepański
Date: 2015-12-06 00:00:00

In this dissertation, we analyze the computational properties of
game-theoretic centrality measures. The key idea behind game-theoretic approach
to network analysis is to treat nodes as players in a cooperative game, where
the value of each coalition of nodes is determined by certain graph properties.
Next, the centrality of any individual node is determined by a chosen
game-theoretic solution concept (notably, the Shapley value) in the same way as
the payoff of a player in a cooperative game. On one hand, the advantage of
game-theoretic centrality measures is that nodes are ranked not only according
to their individual roles but also according to how they contribute to the role
played by all possible subsets of nodes. On the other hand, the disadvantage is
that the game-theoretic solution concepts are typically computationally
challenging. The main contribution of this dissertation is that we show that a
wide variety of game-theoretic solution concepts on networks can be computed in
polynomial time. Our focus is on centralities based on the Shapley value and
its various extensions, such as the Semivalues and Coalitional Semivalues.
Furthermore, we prove #P-hardness of computing the Shapley value in
connectivity games and propose an algorithm to compute it. Finally, we analyse
computational properties of generalized version of cooperative games in which
order of player matters. We propose a new representation for such games, called
generalized marginal contribution networks, that allows for polynomial
computation in the size of the representation of two dedicated extensions of
the Shapley value to this class of games.
Variational Particle Approximations
Authors: Ardavan Saeedi, Tejas D. Kulkarni, Vikash Mansinghka, Samuel Gershman
Date: 2014-02-24 00:00:00
Tasks: Spike Sorting, Variational Inference

Approximate inference in high-dimensional, discrete probabilistic models is a
central problem in computational statistics and machine learning. This paper
describes discrete particle variational inference (DPVI), a new approach that
combines key strengths of Monte Carlo, variational and search-based techniques.
DPVI is based on a novel family of particle-based variational approximations
that can be fit using simple, fast, deterministic search techniques. Like Monte
Carlo, DPVI can handle multiple modes, and yields exact results in a
well-defined limit. Like unstructured mean-field, DPVI is based on optimizing a
lower bound on the partition function; when this quantity is not of intrinsic
interest, it facilitates convergence assessment and debugging. Like both Monte
Carlo and combinatorial search, DPVI can take advantage of factorization,
sequential structure, and custom search operators. This paper defines DPVI
particle-based approximation family and partition function lower bounds, along
with the sequential DPVI and local DPVI algorithm templates for optimizing
them. DPVI is illustrated and evaluated via experiments on lattice Markov
Random Fields, nonparametric Bayesian mixtures and block-models, and parametric
as well as non-parametric hidden Markov models. Results include applications to
real-world spike-sorting and relational modeling problems, and show that DPVI
can offer appealing time/accuracy trade-offs as compared to multiple
alternatives.
Similarity Learning via Adaptive Regression and Its Application to Image Retrieval
Authors: Qi Qian, Inci M. Baytas, Rong Jin, Anil Jain, Shenghuo Zhu
Date: 2015-12-06 00:00:00
Tasks: Image Retrieval, Metric Learning, regression, Retrieval

We study the problem of similarity learning and its application to image
retrieval with large-scale data. The similarity between pairs of images can be
measured by the distances between their high dimensional representations, and
the problem of learning the appropriate similarity is often addressed by
distance metric learning. However, distance metric learning requires the
learned metric to be a PSD matrix, which is computational expensive and not
necessary for retrieval ranking problem. On the other hand, the bilinear model
is shown to be more flexible for large-scale image retrieval task, hence, we
adopt it to learn a matrix for estimating pairwise similarities under the
regression framework. By adaptively updating the target matrix in regression,
we can mimic the hinge loss, which is more appropriate for similarity learning
problem. Although the regression problem can have the closed-form solution, the
computational cost can be very expensive. The computational challenges come
from two aspects: the number of images can be very large and image features
have high dimensionality. We address the first challenge by compressing the
data by a randomized algorithm with the theoretical guarantee. For the high
dimensional issue, we address it by taking low rank assumption and applying
alternating method to obtain the partial matrix, which has a global optimal
solution. Empirical studies on real world image datasets (i.e., Caltech and
ImageNet) demonstrate the effectiveness and efficiency of the proposed method.
Generating News Headlines with Recurrent Neural Networks
Authors: Konstantin Lopyrev
Date: 2015-12-05 00:00:00
Tasks: Articles, Decoder

We describe an application of an encoder-decoder recurrent neural network
with LSTM units and attention to generating headlines from the text of news
articles. We find that the model is quite effective at concisely paraphrasing
news articles. Furthermore, we study how the neural network decides which input
words to pay attention to, and specifically we identify the function of the
different neurons in a simplified attention mechanism. Interestingly, our
simplified attention mechanism performs better that the more complex attention
mechanism on a held out set of articles.