pwc-archive/papers-with-abstracts huggingface dataset:
A Bayesian alternative to mutual information for the hierarchical clustering of dependent random variables
Authors: Guillaume Marrelec, Arnaud Messé, Pierre Bellec
Date: 2015-01-21 00:00:00
Tasks: Clustering

The use of mutual information as a similarity measure in agglomerative
hierarchical clustering (AHC) raises an important issue: some correction needs
to be applied for the dimensionality of variables. In this work, we formulate
the decision of merging dependent multivariate normal variables in an AHC
procedure as a Bayesian model comparison. We found that the Bayesian
formulation naturally shrinks the empirical covariance matrix towards a matrix
set a priori (e.g., the identity), provides an automated stopping rule, and
corrects for dimensionality using a term that scales up the measure as a
function of the dimensionality of the variables. Also, the resulting log Bayes
factor is asymptotically proportional to the plug-in estimate of mutual
information, with an additive correction for dimensionality in agreement with
the Bayesian information criterion. We investigated the behavior of these
Bayesian alternatives (in exact and asymptotic forms) to mutual information on
simulated and real data. An encouraging result was first derived on
simulations: the hierarchical clustering based on the log Bayes factor
outperformed off-the-shelf clustering techniques as well as raw and normalized
mutual information in terms of classification accuracy. On a toy example, we
found that the Bayesian approaches led to results that were similar to those of
mutual information clustering techniques, with the advantage of an automated
thresholding. On real functional magnetic resonance imaging (fMRI) datasets
measuring brain activity, it identified clusters consistent with the
established outcome of standard procedures. On this application, normalized
mutual information had a highly atypical behavior, in the sense that it
systematically favored very large clusters. These initial experiments suggest
that the proposed Bayesian alternatives to mutual information are a useful new
tool for hierarchical clustering.
Piecewise-Linear Approximation for Feature Subset Selection in a Sequential Logit Model
Authors: Toshiki Sato, Yuichi Takano, Ryuhei Miyashiro
Date: 2015-10-19 00:00:00

This paper concerns a method of selecting a subset of features for a
sequential logit model. Tanaka and Nakagawa (2014) proposed a mixed integer
quadratic optimization formulation for solving the problem based on a quadratic
approximation of the logistic loss function. However, since there is a
significant gap between the logistic loss function and its quadratic
approximation, their formulation may fail to find a good subset of features. To
overcome this drawback, we apply a piecewise-linear approximation to the
logistic loss function. Accordingly, we frame the feature subset selection
problem of minimizing an information criterion as a mixed integer linear
optimization problem. The computational results demonstrate that our
piecewise-linear approximation approach found a better subset of features than
the quadratic approximation approach.
System Descriptions of the First International Competition on Computational Models of Argumentation (ICCMA'15)
Authors: Matthias Thimm, Serena Villata
Date: 2015-10-19 00:00:00
Tasks: Abstract Argumentation

This volume contains the system description of the 18 solvers submitted to
the First International Competition on Computational Models of Argumentation
(ICCMA'15) and therefore gives an overview on state-of-the-art of computational
approaches to abstract argumentation problems. Further information on the
results of the competition and the performance of the individual solvers can be
found on at http://argumentationcompetition.org/2015/.
Estimating Posterior Ratio for Classification: Transfer Learning from Probabilistic Perspective
Authors: Song Liu, Kenji Fukumizu
Date: 2015-06-09 00:00:00
Tasks: Classification, General Classification, Transfer Learning

Transfer learning assumes classifiers of similar tasks share certain
parameter structures. Unfortunately, modern classifiers uses sophisticated
feature representations with huge parameter spaces which lead to costly
transfer. Under the impression that changes from one classifier to another
should be ``simple'', an efficient transfer learning criteria that only learns
the ``differences'' is proposed in this paper. We train a \emph{posterior
ratio} which turns out to minimizes the upper-bound of the target learning
risk. The model of posterior ratio does not have to share the same parameter
space with the source classifier at all so it can be easily modelled and
efficiently trained. The resulting classifier therefore is obtained by simply
multiplying the existing probabilistic-classifier with the learned posterior
ratio.
Clustering is Easy When ....What?
Authors: Shai Ben-David
Date: 2015-10-19 00:00:00
Tasks: Clustering

It is well known that most of the common clustering objectives are NP-hard to
optimize. In practice, however, clustering is being routinely carried out. One
approach for providing theoretical understanding of this seeming discrepancy is
to come up with notions of clusterability that distinguish realistically
interesting input data from worst-case data sets. The hope is that there will
be clustering algorithms that are provably efficient on such "clusterable"
instances. This paper addresses the thesis that the computational hardness of
clustering tasks goes away for inputs that one really cares about. In other
words, that "Clustering is difficult only when it does not matter" (the
\emph{CDNM thesis} for short).
  I wish to present a a critical bird's eye overview of the results published
on this issue so far and to call attention to the gap between available and
desirable results on this issue. A longer, more detailed version of this note
is available as arXiv:1507.05307.
  I discuss which requirements should be met in order to provide formal support
to the the CDNM thesis and then examine existing results in view of these
requirements and list some significant unsolved research challenges in that
direction.
Constrained Convolutional Neural Networks for Weakly Supervised Segmentation
Authors: Deepak Pathak, Philipp Krähenbühl, Trevor Darrell
Date: 2015-06-11 00:00:00
Proceeding: ICCV 2015 12
Tasks: Image Segmentation, Semantic Segmentation, TAG, Weakly supervised segmentation

We present an approach to learn a dense pixel-wise labeling from image-level
tags. Each image-level tag imposes constraints on the output labeling of a
Convolutional Neural Network (CNN) classifier. We propose Constrained CNN
(CCNN), a method which uses a novel loss function to optimize for any set of
linear constraints on the output space (i.e. predicted label distribution) of a
CNN. Our loss formulation is easy to optimize and can be incorporated directly
into standard stochastic gradient descent optimization. The key idea is to
phrase the training objective as a biconvex optimization for linear models,
which we then relax to nonlinear deep networks. Extensive experiments
demonstrate the generality of our new learning framework. The constrained loss
yields state-of-the-art results on weakly supervised semantic image
segmentation. We further demonstrate that adding slightly more supervision can
greatly improve the performance of the learning algorithm.
Latent Space Model for Multi-Modal Social Data
Authors: Yoon-Sik Cho, Greg Ver Steeg, Emilio Ferrara, Aram Galstyan
Date: 2015-10-18 00:00:00
Tasks: model

With the emergence of social networking services, researchers enjoy the
increasing availability of large-scale heterogenous datasets capturing online
user interactions and behaviors. Traditional analysis of techno-social systems
data has focused mainly on describing either the dynamics of social
interactions, or the attributes and behaviors of the users. However,
overwhelming empirical evidence suggests that the two dimensions affect one
another, and therefore they should be jointly modeled and analyzed in a
multi-modal framework. The benefits of such an approach include the ability to
build better predictive models, leveraging social network information as well
as user behavioral signals. To this purpose, here we propose the Constrained
Latent Space Model (CLSM), a generalized framework that combines Mixed
Membership Stochastic Blockmodels (MMSB) and Latent Dirichlet Allocation (LDA)
incorporating a constraint that forces the latent space to concurrently
describe the multiple data modalities. We derive an efficient inference
algorithm based on Variational Expectation Maximization that has a
computational cost linear in the size of the network, thus making it feasible
to analyze massive social datasets. We validate the proposed framework on two
problems: prediction of social interactions from user attributes and behaviors,
and behavior prediction exploiting network information. We perform experiments
with a variety of multi-modal social systems, spanning location-based social
networks (Gowalla), social media services (Instagram, Orkut), e-commerce and
review sites (Amazon, Ciao), and finally citation networks (Cora). The results
indicate significant improvement in prediction accuracy over state of the art
methods, and demonstrate the flexibility of the proposed approach for
addressing a variety of different learning problems commonly occurring with
multi-modal social data.
An optimal randomized incremental gradient method
Authors: Guanghui Lan, Yi Zhou
Date: 2015-07-08 00:00:00

In this paper, we consider a class of finite-sum convex optimization problems
whose objective function is given by the summation of $m$ ($\ge 1$) smooth
components together with some other relatively simple terms. We first introduce
a deterministic primal-dual gradient (PDG) method that can achieve the optimal
black-box iteration complexity for solving these composite optimization
problems using a primal-dual termination criterion. Our major contribution is
to develop a randomized primal-dual gradient (RPDG) method, which needs to
compute the gradient of only one randomly selected smooth component at each
iteration, but can possibly achieve better complexity than PDG in terms of the
total number of gradient evaluations. More specifically, we show that the total
number of gradient evaluations performed by RPDG can be ${\cal O} (\sqrt{m})$
times smaller, both in expectation and with high probability, than those
performed by deterministic optimal first-order methods under favorable
situations. We also show that the complexity of the RPDG method is not
improvable by developing a new lower complexity bound for a general class of
randomized methods for solving large-scale finite-sum convex optimization
problems. Moreover, through the development of PDG and RPDG, we introduce a
novel game-theoretic interpretation for these optimal methods for convex
optimization.
EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding
Authors: Yajie Miao, Mohammad Gowayyed, Florian Metze
Date: 2015-07-29 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

The performance of automatic speech recognition (ASR) has improved
tremendously due to the application of deep neural networks (DNNs). Despite
this progress, building a new ASR system remains a challenging task, requiring
various resources, multiple training stages and significant expertise. This
paper presents our Eesen framework which drastically simplifies the existing
pipeline to build state-of-the-art ASR systems. Acoustic modeling in Eesen
involves learning a single recurrent neural network (RNN) predicting
context-independent targets (phonemes or characters). To remove the need for
pre-generated frame labels, we adopt the connectionist temporal classification
(CTC) objective function to infer the alignments between speech and label
sequences. A distinctive feature of Eesen is a generalized decoding approach
based on weighted finite-state transducers (WFSTs), which enables the efficient
incorporation of lexicons and language models into CTC decoding. Experiments
show that compared with the standard hybrid DNN systems, Eesen achieves
comparable word error rates (WERs), while at the same time speeding up decoding
significantly.
Geometry-aware Deep Transform
Authors: Jiaji Huang, Qiang Qiu, Robert Calderbank, Guillermo Sapiro
Date: 2015-09-17 00:00:00
Proceeding: ICCV 2015 12
Tasks: Deep Learning, Metric Learning

Many recent efforts have been devoted to designing sophisticated deep
learning structures, obtaining revolutionary results on benchmark datasets. The
success of these deep learning methods mostly relies on an enormous volume of
labeled training samples to learn a huge number of parameters in a network;
therefore, understanding the generalization ability of a learned deep network
cannot be overlooked, especially when restricted to a small training set, which
is the case for many applications. In this paper, we propose a novel deep
learning objective formulation that unifies both the classification and metric
learning criteria. We then introduce a geometry-aware deep transform to enable
a non-linear discriminative and robust feature transform, which shows
competitive performance on small training sets for both synthetic and
real-world data. We further support the proposed framework with a formal
$(K,\epsilon)$-robustness analysis.
Real-time Tracking Based on Neuromrophic Vision
Authors: Hongmin Li, Pei Jing, Guoqi Li
Date: 2015-10-18 00:00:00

Real-time tracking is an important problem in computer vision in which most
methods are based on the conventional cameras. Neuromorphic vision is a concept
defined by incorporating neuromorphic vision sensors such as silicon retinas in
vision processing system. With the development of the silicon technology,
asynchronous event-based silicon retinas that mimic neuro-biological
architectures has been developed in recent years. In this work, we combine the
vision tracking algorithm of computer vision with the information encoding
mechanism of event-based sensors which is inspired from the neural rate coding
mechanism. The real-time tracking of single object with the advantage of high
speed of 100 time bins per second is successfully realized. Our method
demonstrates that the computer vision methods could be used for the
neuromorphic vision processing and we can realize fast real-time tracking using
neuromorphic vision sensors compare to the conventional camera.
Large Enforced Sparse Non-Negative Matrix Factorization
Authors: Brendan Gavin, Vijay Gadepally, Jeremy Kepner
Date: 2015-10-18 00:00:00
Tasks: Topic Models

Non-negative matrix factorization (NMF) is a common method for generating
topic models from text data. NMF is widely accepted for producing good results
despite its relative simplicity of implementation and ease of computation. One
challenge with applying NMF to large datasets is that intermediate matrix
products often become dense, stressing the memory and compute elements of a
system. In this article, we investigate a simple but powerful modification of a
common NMF algorithm that enforces the generation of sparse intermediate and
output matrices. This method enables the application of NMF to large datasets
through improved memory and compute performance. Further, we demonstrate
empirically that this method of enforcing sparsity in the NMF either preserves
or improves both the accuracy of the resulting topic model and the convergence
rate of the underlying algorithm.
Clustering Noisy Signals with Structured Sparsity Using Time-Frequency Representation
Authors: Tom Hope, Avishai Wagner, Or Zuk
Date: 2015-10-18 00:00:00
Tasks: Clustering, Dimensionality Reduction, Time Series, Time Series Analysis, Time Series Clustering, Translation

We propose a simple and efficient time-series clustering framework
particularly suited for low Signal-to-Noise Ratio (SNR), by simultaneous
smoothing and dimensionality reduction aimed at preserving clustering
information. We extend the sparse K-means algorithm by incorporating structured
sparsity, and use it to exploit the multi-scale property of wavelets and group
structure in multivariate signals. Finally, we extract features invariant to
translation and scaling with the scattering transform, which corresponds to a
convolutional network with filters given by a wavelet operator, and use the
network's structure in sparse clustering. By promoting sparsity, this transform
can yield a low-dimensional representation of signals that gives improved
clustering results on several real datasets.
Expresso : A user-friendly GUI for Designing, Training and Exploring Convolutional Neural Networks
Authors: Ravi Kiran Sarvadevabhatla, R. Venkatesh Babu
Date: 2015-05-25 00:00:00
Tasks: Deep Learning

With a view to provide a user-friendly interface for designing, training and
developing deep learning frameworks, we have developed Expresso, a GUI tool
written in Python. Expresso is built atop Caffe, the open-source, prize-winning
framework popularly used to develop Convolutional Neural Networks. Expresso
provides a convenient wizard-like graphical interface which guides the user
through various common scenarios -- data import, construction and training of
deep networks, performing various experiments, analyzing and visualizing the
results of these experiments. The multi-threaded nature of Expresso enables
concurrent execution and notification of events related to the aforementioned
scenarios. The GUI sub-components and inter-component interfaces in Expresso
have been designed with extensibility in mind. We believe Expresso's
flexibility and ease of use will come in handy to researchers, newcomers and
seasoned alike, in their explorations related to deep learning.
Neural Reranking Improves Subjective Quality of Machine Translation: NAIST at WAT2015
Authors: Graham Neubig, Makoto Morishita, Satoshi Nakamura
Date: 2015-10-18 00:00:00
Proceeding: WS 2015 10
Tasks: Machine Translation, Reranking, Translation

This year, the Nara Institute of Science and Technology (NAIST)'s submission
to the 2015 Workshop on Asian Translation was based on syntax-based statistical
machine translation, with the addition of a reranking component using neural
attentional machine translation models. Experiments re-confirmed results from
previous work stating that neural MT reranking provides a large gain in
objective evaluation measures such as BLEU, and also confirmed for the first
time that these results also carry over to manual evaluation. We further
perform a detailed analysis of reasons for this increase, finding that the main
contributions of the neural models lie in improvement of the grammatical
correctness of the output, as opposed to improvements in lexical choice of
content words.
Sum-of-Squares Lower Bounds for Sparse PCA
Authors: Tengyu Ma, Avi Wigderson
Date: 2015-07-23 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: BIG-bench Machine Learning

This paper establishes a statistical versus computational trade-off for
solving a basic high-dimensional machine learning problem via a basic convex
relaxation method. Specifically, we consider the {\em Sparse Principal
Component Analysis} (Sparse PCA) problem, and the family of {\em
Sum-of-Squares} (SoS, aka Lasserre/Parillo) convex relaxations. It was well
known that in large dimension $p$, a planted $k$-sparse unit vector can be {\em
in principle} detected using only $n \approx k\log p$ (Gaussian or Bernoulli)
samples, but all {\em efficient} (polynomial time) algorithms known require $n
\approx k^2$ samples. It was also known that this quadratic gap cannot be
improved by the the most basic {\em semi-definite} (SDP, aka spectral)
relaxation, equivalent to a degree-2 SoS algorithms. Here we prove that also
degree-4 SoS algorithms cannot improve this quadratic gap. This average-case
lower bound adds to the small collection of hardness results in machine
learning for this powerful family of convex relaxation algorithms. Moreover,
our design of moments (or "pseudo-expectations") for this lower bound is quite
different than previous lower bounds. Establishing lower bounds for higher
degree SoS algorithms for remains a challenging problem.
Efficient Hand Articulations Tracking using Adaptive Hand Model and Depth map
Authors: Byeongkeun Kang, Yeejin Lee, Truong Q. Nguyen
Date: 2015-10-04 00:00:00

Real-time hand articulations tracking is important for many applications such
as interacting with virtual / augmented reality devices or tablets. However,
most of existing algorithms highly rely on expensive and high power-consuming
GPUs to achieve real-time processing. Consequently, these systems are
inappropriate for mobile and wearable devices. In this paper, we propose an
efficient hand tracking system which does not require high performance GPUs. In
our system, we track hand articulations by minimizing discrepancy between depth
map from sensor and computer-generated hand model. We also initialize hand pose
at each frame using finger detection and classification. Our contributions are:
(a) propose adaptive hand model to consider different hand shapes of users
without generating personalized hand model; (b) improve the highly efficient
frame initialization for robust tracking and automatic initialization; (c)
propose hierarchical random sampling of pixels from each depth map to improve
tracking accuracy while limiting required computations. To the best of our
knowledge, it is the first system that achieves both automatic hand model
adjustment and real-time tracking without using GPUs.
Symbolic Neutrosophic Theory
Authors: Florentin Smarandache
Date: 2015-10-18 00:00:00

Symbolic (or Literal) Neutrosophic Theory is referring to the use of abstract
symbols (i.e. the letters T, I, F, or their refined indexed letters Tj, Ik, Fl)
in neutrosophics. We extend the dialectical triad thesis-antithesis-synthesis
to the neutrosophic tetrad thesis-antithesis-neutrothesis-neutrosynthesis. The
we introduce the neutrosophic system that is a quasi or (t,i,f) classical
system, in the sense that the neutrosophic system deals with quasi-terms
(concepts, attributes, etc.). Then the notions of Neutrosophic Axiom,
Neutrosophic Deducibility, Degree of Contradiction (Dissimilarity) of Two
Neutrosophic Axioms, etc. Afterwards a new type of structures, called (t, i, f)
Neutrosophic Structures, and we show particular cases of such structures in
geometry and in algebra. Also, a short history of the neutrosophic set,
neutrosophic numerical components and neutrosophic literal components,
neutrosophic numbers, etc. We construct examples of splitting the literal
indeterminacy (I) into literal subindeterminacies (I1, I2, and so on, Ir), and
to define a multiplication law of these literal subindeterminacies in order to
be able to build refined I neutrosophic algebraic structures. We define three
neutrosophic actions and their properties. We then introduce the prevalence
order on T,I,F with respect to a given neutrosophic operator. And the
refinement of neutrosophic entities A, neutA, and antiA. Then we extend the
classical logical operators to neutrosophic literal (symbolic) logical
operators and to refined literal (symbolic) logical operators, and we define
the refinement neutrosophic literal (symbolic) space. We introduce the
neutrosophic quadruple numbers (a+bT+cI+dF) and the refined neutrosophic
quadruple numbers. Then we define an absorbance law, based on a prevalence
order, in order to multiply the neutrosophic quadruple numbers.