pwc-archive/papers-with-abstracts huggingface dataset:
Learning a Metric Embedding for Face Recognition using the Multibatch Method
Authors: Oren Tadmor, Yonatan Wexler, Tal Rosenwein, Shai Shalev-Shwartz, Amnon Shashua
Date: 2016-05-24 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Face Recognition, GPU

This work is motivated by the engineering task of achieving a near
state-of-the-art face recognition on a minimal computing budget running on an
embedded system. Our main technical contribution centers around a novel
training method, called Multibatch, for similarity learning, i.e., for the task
of generating an invariant "face signature" through training pairs of "same"
and "not-same" face images. The Multibatch method first generates signatures
for a mini-batch of $k$ face images and then constructs an unbiased estimate of
the full gradient by relying on all $k^2-k$ pairs from the mini-batch. We prove
that the variance of the Multibatch estimator is bounded by $O(1/k^2)$, under
some mild conditions. In contrast, the standard gradient estimator that relies
on random $k/2$ pairs has a variance of order $1/k$. The smaller variance of
the Multibatch estimator significantly speeds up the convergence rate of
stochastic gradient descent. Using the Multibatch method we train a deep
convolutional neural network that achieves an accuracy of $98.2\%$ on the LFW
benchmark, while its prediction runtime takes only $30$msec on a single ARM
Cortex A9 core. Furthermore, the entire training process took only 12 hours on
a single Titan X GPU.
Classifying discourse in a CSCL platform to evaluate correlations with Teacher Participation and Progress
Authors: Eliana Scheihing, Matthieu Vernier, Javiera Born, Julio Guerra, Luis Carcamo
Date: 2016-05-24 00:00:00

In Computer-Supported learning, monitoring and engaging a group of learners
is a complex task for teachers, especially when learners are working
collaboratively: Are my students motivated? What kind of progress are they
making? Should I intervene? Is my communication and the didactic design adapted
to my students? Our hypothesis is that the analysis of natural language
interactions between students, and between students and teachers, provide very
valuable information and could be used to produce qualitative indicators to
help teachers' decisions. We develop an automatic approach in three steps (1)
to explore the discursive functions of messages in a CSCL platform, (2) to
classify the messages automatically and (3) to evaluate correlations between
discursive attitudes and other variables linked to the learning activity.
Results tend to show that some types of discourse are correlated with a notion
of Progress on the learning activities and the importance of emotive
participation from the Teacher.
Trajectory probability hypothesis density filter
Authors: Ángel F. García-Fernández, Lennart Svensson
Date: 2016-05-24 00:00:00

This paper presents the probability hypothesis density (PHD) filter for sets
of trajectories: the trajectory probability density (TPHD) filter. The TPHD
filter is capable of estimating trajectories in a principled way without
requiring to evaluate all measurement-to-target association hypotheses. The
TPHD filter is based on recursively obtaining the best Poisson approximation to
the multitrajectory filtering density in the sense of minimising the
Kullback-Leibler divergence. We also propose a Gaussian mixture implementation
of the TPHD recursion. Finally, we include simulation results to show the
performance of the proposed algorithm.
Diagnosing editorial strategies of Chilean media on Twitter using an automatic news classifier
Authors: Matthieu Vernier, Luis Carcamo, Eliana Scheihing
Date: 2016-05-24 00:00:00

In Chile, does not exist an independent entity that publishes quantitative or
qualitative surveys to understand the traditional media environment and its
adaptation on the Social Web. Nowadays, Chilean newsreaders are increasingly
using social web platforms as their primary source of information, among which
Twitter plays a central role. Historical media and pure players are developing
different strategies to increase their audience and influence on this platform.
In this article, we propose a methodology based on data mining techniques to
provide a first level of analysis of the new Chilean media environment. We use
a crawling technique to mine news streams of 37 different Chilean media
actively presents on Twitter and propose several indicators to compare them. We
analyze their volumes of production, their potential audience, and using NLP
techniques, we explore the content of their production: their editorial line
and their geographic coverage.
Dense CNN Learning with Equivalent Mappings
Authors: Jianxin Wu, Chen-Wei Xie, Jian-Hao Luo
Date: 2016-05-24 00:00:00
Tasks: Age Estimation, Image Categorization, Object Localization, Semantic Segmentation

Large receptive field and dense prediction are both important for achieving
high accuracy in pixel labeling tasks such as semantic segmentation. These two
properties, however, contradict with each other. A pooling layer (with stride
2) quadruples the receptive field size but reduces the number of predictions to
25\%. Some existing methods lead to dense predictions using computations that
are not equivalent to the original model. In this paper, we propose the
equivalent convolution (eConv) and equivalent pooling (ePool) layers, leading
to predictions that are both dense and equivalent to the baseline CNN model.
Dense prediction models learned using eConv and ePool can transfer the baseline
CNN's parameters as a starting point, and can inverse transfer the learned
parameters in a dense model back to the original one, which has both fast
testing speed and high accuracy. The proposed eConv and ePool layers have
achieved higher accuracy than baseline CNN in various tasks, including semantic
segmentation, object localization, image categorization and apparent age
estimation, not only in those tasks requiring dense pixel labeling.
Information-Theoretic Lower Bounds for Recovery of Diffusion Network Structures
Authors: Keehwan Park, Jean Honorio
Date: 2016-01-28 00:00:00

We study the information-theoretic lower bound of the sample complexity of
the correct recovery of diffusion network structures. We introduce a
discrete-time diffusion model based on the Independent Cascade model for which
we obtain a lower bound of order $\Omega(k \log p)$, for directed graphs of $p$
nodes, and at most $k$ parents per node. Next, we introduce a continuous-time
diffusion model, for which a similar lower bound of order $\Omega(k \log p)$ is
obtained. Our results show that the algorithm of Pouget-Abadie et al. is
statistically optimal for the discrete-time regime. Our work also opens the
question of whether it is possible to devise an optimal algorithm for the
continuous-time regime.
A Kronecker-factored approximate Fisher matrix for convolution layers
Authors: Roger Grosse, James Martens
Date: 2016-02-03 00:00:00
Tasks: Stochastic Optimization

Second-order optimization methods such as natural gradient descent have the
potential to speed up training of neural networks by correcting for the
curvature of the loss function. Unfortunately, the exact natural gradient is
impractical to compute for large models, and most approximations either require
an expensive iterative procedure or make crude approximations to the curvature.
We present Kronecker Factors for Convolution (KFC), a tractable approximation
to the Fisher matrix for convolutional networks based on a structured
probabilistic model for the distribution over backpropagated derivatives.
Similarly to the recently proposed Kronecker-Factored Approximate Curvature
(K-FAC), each block of the approximate Fisher matrix decomposes as the
Kronecker product of small matrices, allowing for efficient inversion. KFC
captures important curvature information while still yielding comparably
efficient updates to stochastic gradient descent (SGD). We show that the
updates are invariant to commonly used reparameterizations, such as centering
of the activations. In our experiments, approximate natural gradient descent
with KFC was able to train convolutional networks several times faster than
carefully tuned SGD. Furthermore, it was able to train the networks in 10-20
times fewer iterations than SGD, suggesting its potential applicability in a
distributed setting.
Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation
Authors: Wang Ling, Tiago Luís, Luís Marujo, Ramón Fernandez Astudillo, Silvio Amir, Chris Dyer, Alan W. black, Isabel Trancoso
Date: 2015-08-09 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Form, Language Modeling, Language Modelling, Part-Of-Speech Tagging

We introduce a model for constructing vector representations of words by
composing characters using bidirectional LSTMs. Relative to traditional word
representation models that have independent vectors for each word type, our
model requires only a single vector per character type and a fixed set of
parameters for the compositional model. Despite the compactness of this model
and, more importantly, the arbitrary nature of the form-function relationship
in language, our "composed" word representations yield state-of-the-art results
in language modeling and part-of-speech tagging. Benefits over traditional
baselines are particularly pronounced in morphologically rich languages (e.g.,
Turkish).
Bayesian leave-one-out cross-validation approximations for Gaussian latent variable models
Authors: Aki Vehtari, Tommi Mononen, Ville Tolvanen, Tuomas Sivula, Ole Winther
Date: 2014-12-23 00:00:00

The future predictive performance of a Bayesian model can be estimated using
Bayesian cross-validation. In this article, we consider Gaussian latent
variable models where the integration over the latent values is approximated
using the Laplace method or expectation propagation (EP). We study the
properties of several Bayesian leave-one-out (LOO) cross-validation
approximations that in most cases can be computed with a small additional cost
after forming the posterior approximation given the full data. Our main
objective is to assess the accuracy of the approximative LOO cross-validation
estimators. That is, for each method (Laplace and EP) we compare the
approximate fast computation with the exact brute force LOO computation.
Secondarily, we evaluate the accuracy of the Laplace and EP approximations
themselves against a ground truth established through extensive Markov chain
Monte Carlo simulation. Our empirical results show that the approach based upon
a Gaussian approximation to the LOO marginal distribution (the so-called cavity
distribution) gives the most accurate and reliable results among the fast
methods.
Kernel-based Reconstruction of Graph Signals
Authors: Daniel Romero, Meng Ma, Georgios B. Giannakis
Date: 2016-05-23 00:00:00
Tasks: regression

A number of applications in engineering, social sciences, physics, and
biology involve inference over networks. In this context, graph signals are
widely encountered as descriptors of vertex attributes or features in
graph-structured data. Estimating such signals in all vertices given noisy
observations of their values on a subset of vertices has been extensively
analyzed in the literature of signal processing on graphs (SPoG). This paper
advocates kernel regression as a framework generalizing popular SPoG modeling
and reconstruction and expanding their capabilities. Formulating signal
reconstruction as a regression task on reproducing kernel Hilbert spaces of
graph signals permeates benefits from statistical learning, offers fresh
insights, and allows for estimators to leverage richer forms of prior
information than existing alternatives. A number of SPoG notions such as
bandlimitedness, graph filters, and the graph Fourier transform are naturally
accommodated in the kernel framework. Additionally, this paper capitalizes on
the so-called representer theorem to devise simpler versions of existing
Thikhonov regularized estimators, and offers a novel probabilistic
interpretation of kernel methods on graphs based on graphical models. Motivated
by the challenges of selecting the bandwidth parameter in SPoG estimators or
the kernel map in kernel-based methods, the present paper further proposes two
multi-kernel approaches with complementary strengths. Whereas the first enables
estimation of the unknown bandwidth of bandlimited signals, the second allows
for efficient graph filter selection. Numerical tests with synthetic as well as
real data demonstrate the merits of the proposed methods relative to
state-of-the-art alternatives.
Genetic Architect: Discovering Genomic Structure with Learned Neural Architectures
Authors: Laura Deming, Sasha Targ, Nate Sauder, Diogo Almeida, Chun Jimmie Ye
Date: 2016-05-23 00:00:00

Each human genome is a 3 billion base pair set of encoding instructions.
Decoding the genome using deep learning fundamentally differs from most tasks,
as we do not know the full structure of the data and therefore cannot design
architectures to suit it. As such, architectures that fit the structure of
genomics should be learned not prescribed. Here, we develop a novel search
algorithm, applicable across domains, that discovers an optimal architecture
which simultaneously learns general genomic patterns and identifies the most
important sequence motifs in predicting functional genomic outcomes. The
architectures we find using this algorithm succeed at using only RNA expression
data to predict gene regulatory structure, learn human-interpretable
visualizations of key sequence motifs, and surpass state-of-the-art results on
benchmark genomics challenges.
Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations
Authors: Behnam Neyshabur, Yuhuai Wu, Ruslan Salakhutdinov, Nathan Srebro
Date: 2016-05-23 00:00:00
Proceeding: NeurIPS 2016 12

We investigate the parameter-space geometry of recurrent neural networks
(RNNs), and develop an adaptation of path-SGD optimization method, attuned to
this geometry, that can learn plain RNNs with ReLU activations. On several
datasets that require capturing long-term dependency structure, we show that
path-SGD can significantly improve trainability of ReLU RNNs compared to RNNs
trained with SGD, even with various recently suggested initialization schemes.
Towards Multi-Agent Communication-Based Language Learning
Authors: Angeliki Lazaridou, Nghia The Pham, Marco Baroni
Date: 2016-05-23 00:00:00

We propose an interactive multimodal framework for language learning. Instead
of being passively exposed to large amounts of natural text, our learners
(implemented as feed-forward neural networks) engage in cooperative referential
games starting from a tabula rasa setup, and thus develop their own language
from the need to communicate in order to succeed at the game. Preliminary
experiments provide promising results, but also suggest that it is important to
ensure that agents trained in this way do not develop an adhoc communication
code only effective for the game they are playing
Completing Low-Rank Matrices with Corrupted Samples from Few Coefficients in General Basis
Authors: Hongyang Zhang, Zhouchen Lin, Chao Zhang
Date: 2015-06-25 00:00:00
Tasks: Clustering, Matrix Completion, Missing Values

Subspace recovery from corrupted and missing data is crucial for various
applications in signal processing and information theory. To complete missing
values and detect column corruptions, existing robust Matrix Completion (MC)
methods mostly concentrate on recovering a low-rank matrix from few corrupted
coefficients w.r.t. standard basis, which, however, does not apply to more
general basis, e.g., Fourier basis. In this paper, we prove that the range
space of an $m\times n$ matrix with rank $r$ can be exactly recovered from few
coefficients w.r.t. general basis, though $r$ and the number of corrupted
samples are both as high as $O(\min\{m,n\}/\log^3 (m+n))$. Our model covers
previous ones as special cases, and robust MC can recover the intrinsic matrix
with a higher rank. Moreover, we suggest a universal choice of the
regularization parameter, which is $\lambda=1/\sqrt{\log n}$. By our
$\ell_{2,1}$ filtering algorithm, which has theoretical guarantees, we can
further reduce the computational cost of our model. As an application, we also
find that the solutions to extended robust Low-Rank Representation and to our
extended robust MC are mutually expressible, so both our theory and algorithm
can be applied to the subspace clustering problem with missing values under
certain conditions. Experiments verify our theories.
A Formal Evaluation of PSNR as Quality Measurement Parameter for Image Segmentation Algorithms
Authors: Fernando A. Fardo, Victor H. Conforto, Francisco C. de Oliveira, Paulo S. Rodrigues
Date: 2016-05-23 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Quality evaluation of image segmentation algorithms are still subject of
debate and research. Currently, there is no generic metric that could be
applied to any algorithm reliably. This article contains an evaluation for the
PSRN (Peak Signal-To-Noise Ratio) as a metric which has been used to evaluate
threshold level selection as well as the number of thresholds in the case of
multi-level segmentation. The results obtained in this study suggest that the
PSNR is not an adequate quality measurement for segmentation algorithms.
Generic Instance Search and Re-identification from One Example via Attributes and Categories
Authors: Ran Tao, Arnold W. M. Smeulders, Shih-Fu Chang
Date: 2016-05-23 00:00:00
Tasks: Instance Search, Person Re-Identification

This paper aims for generic instance search from one example where the
instance can be an arbitrary object like shoes, not just near-planar and
one-sided instances like buildings and logos. First, we evaluate
state-of-the-art instance search methods on this problem. We observe that what
works for buildings loses its generality on shoes. Second, we propose to use
automatically learned category-specific attributes to address the large
appearance variations present in generic instance search. Searching among
instances from the same category as the query, the category-specific attributes
outperform existing approaches by a large margin on shoes and cars and perform
on par with the state-of-the-art on buildings. Third, we treat person
re-identification as a special case of generic instance search. On the popular
VIPeR dataset, we reach state-of-the-art performance with the same method.
Fourth, we extend our method to search objects without restriction to the
specifically known category. We show that the combination of category-level
information and the category-specific attributes is superior to the alternative
method combining category-level information with low-level features such as
Fisher vector.
The red one!: On learning to refer to things based on their discriminative properties
Authors: Angeliki Lazaridou, Nghia The Pham, Marco Baroni
Date: 2016-03-08 00:00:00
Tasks: Attribute

As a first step towards agents learning to communicate about their visual
environment, we propose a system that, given visual representations of a
referent (cat) and a context (sofa), identifies their discriminative
attributes, i.e., properties that distinguish them (has_tail). Moreover,
despite the lack of direct supervision at the attribute level, the model learns
to assign plausible attributes to objects (sofa-has_cushion). Finally, we
present a preliminary experiment confirming the referential success of the
predicted discriminative attributes.
Bounded Rational Decision-Making in Feedforward Neural Networks
Authors: Felix Leibfried, Daniel Alexander Braun
Date: 2016-02-26 00:00:00
Tasks: Decision Making, General Classification

Bounded rational decision-makers transform sensory input into motor output
under limited computational resources. Mathematically, such decision-makers can
be modeled as information-theoretic channels with limited transmission rate.
Here, we apply this formalism for the first time to multilayer feedforward
neural networks. We derive synaptic weight update rules for two scenarios,
where either each neuron is considered as a bounded rational decision-maker or
the network as a whole. In the update rules, bounded rationality translates
into information-theoretically motivated types of regularization in weight
space. In experiments on the MNIST benchmark classification task for
handwritten digits, we show that such information-theoretic regularization
successfully prevents overfitting across different architectures and attains
results that are competitive with other recent techniques like dropout,
dropconnect and Bayes by backprop, for both ordinary and convolutional neural
networks.
On Restricted Nonnegative Matrix Factorization
Authors: Dmitry Chistikov, Stefan Kiefer, Ines Marušić, Mahsa Shirmohammadi, James Worrell
Date: 2016-05-23 00:00:00

Nonnegative matrix factorization (NMF) is the problem of decomposing a given
nonnegative $n \times m$ matrix $M$ into a product of a nonnegative $n \times
d$ matrix $W$ and a nonnegative $d \times m$ matrix $H$. Restricted NMF
requires in addition that the column spaces of $M$ and $W$ coincide. Finding
the minimal inner dimension $d$ is known to be NP-hard, both for NMF and
restricted NMF. We show that restricted NMF is closely related to a question
about the nature of minimal probabilistic automata, posed by Paz in his seminal
1971 textbook. We use this connection to answer Paz's question negatively, thus
falsifying a positive answer claimed in 1974. Furthermore, we investigate
whether a rational matrix $M$ always has a restricted NMF of minimal inner
dimension whose factors $W$ and $H$ are also rational. We show that this holds
for matrices $M$ of rank at most $3$ and we exhibit a rank-$4$ matrix for which
$W$ and $H$ require irrational entries.
Bayesian Model Selection of Stochastic Block Models
Authors: Xiaoran Yan
Date: 2016-05-23 00:00:00
Tasks: model, Model Selection, Stochastic Block Model

A central problem in analyzing networks is partitioning them into modules or
communities. One of the best tools for this is the stochastic block model,
which clusters vertices into blocks with statistically homogeneous pattern of
links. Despite its flexibility and popularity, there has been a lack of
principled statistical model selection criteria for the stochastic block model.
Here we propose a Bayesian framework for choosing the number of blocks as well
as comparing it to the more elaborate degree- corrected block models,
ultimately leading to a universal model selection framework capable of
comparing multiple modeling combinations. We will also investigate its
connection to the minimum description length principle.
Spontaneous vs. Posed smiles - can we tell the difference?
Authors: Bappaditya Mandal, Nizar Ouarti
Date: 2016-05-23 00:00:00
Tasks: Optical Flow Estimation

Smile is an irrefutable expression that shows the physical state of the mind
in both true and deceptive ways. Generally, it shows happy state of the mind,
however, `smiles' can be deceptive, for example people can give a smile when
they feel happy and sometimes they might also give a smile (in a different way)
when they feel pity for others. This work aims to distinguish spontaneous
(felt) smile expressions from posed (deliberate) smiles by extracting and
analyzing both global (macro) motion of the face and subtle (micro) changes in
the facial expression features through both tracking a series of facial
fiducial markers as well as using dense optical flow. Specifically the eyes and
lips features are captured and used for analysis. It aims to automatically
classify all smiles into either `spontaneous' or `posed' categories, by using
support vector machines (SVM). Experimental results on large database show
promising results as compared to other relevant methods.