pwc-archive/papers-with-abstracts huggingface dataset:
DeMIAN: Deep Modality Invariant Adversarial Network
Authors: Kuniaki Saito, Yusuke Mukuta, Yoshitaka Ushiku, Tatsuya Harada
Date: 2016-12-23 00:00:00
Tasks: Domain Adaptation, General Classification, Representation Learning, Zero-Shot Learning

Obtaining common representations from different modalities is important in
that they are interchangeable with each other in a classification problem. For
example, we can train a classifier on image features in the common
representations and apply it to the testing of the text features in the
representations. Existing multi-modal representation learning methods mainly
aim to extract rich information from paired samples and train a classifier by
the corresponding labels; however, collecting paired samples and their labels
simultaneously involves high labor costs. Addressing paired modal samples
without their labels and single modal data with their labels independently is
much easier than addressing labeled multi-modal data. To obtain the common
representations under such a situation, we propose to make the distributions
over different modalities similar in the learned representations, namely
modality-invariant representations. In particular, we propose a novel algorithm
for modality-invariant representation learning, named Deep Modality Invariant
Adversarial Network (DeMIAN), which utilizes the idea of Domain Adaptation
(DA). Using the modality-invariant representations learned by DeMIAN, we
achieved better classification accuracy than with the state-of-the-art methods,
especially for some benchmark datasets of zero-shot learning.
Semi-Supervised Learning with Heterophily
Authors: Wolfgang Gatterbauer
Date: 2014-12-09 00:00:00

We derive a family of linear inference algorithms that generalize existing
graph-based label propagation algorithms by allowing them to propagate
generalized assumptions about "attraction" or "compatibility" between classes
of neighboring nodes (in particular those that involve heterophily between
nodes where "opposites attract"). We thus call this formulation Semi-Supervised
Learning with Heterophily (SSLH) and show how it generalizes and improves upon
a recently proposed approach called Linearized Belief Propagation (LinBP).
Importantly, our framework allows us to reduce the problem of estimating the
relative compatibility between nodes from partially labeled graph to a simple
optimization problem. The result is a very fast algorithm that -- despite its
simplicity -- is surprisingly effective: we can classify unlabeled nodes within
the same graph in the same time as LinBP but with a superior accuracy and
despite our algorithm not knowing the compatibilities.
Distributed Probabilistic Bisection Search using Social Learning
Authors: Athanasios Tsiligkaridis, Theodoros Tsiligkaridis
Date: 2016-08-21 00:00:00

We present a novel distributed probabilistic bisection algorithm using social
learning with application to target localization. Each agent in the network
first constructs a query about the target based on its local information and
obtains a noisy response. Agents then perform a Bayesian update of their
beliefs followed by an averaging of the log beliefs over local neighborhoods.
This two stage algorithm consisting of repeated querying and averaging runs
until convergence. We derive bounds on the rate of convergence of the beliefs
at the correct target location. Numerical simulations show that our method
outperforms current state of the art methods.
Write a Classifier: Predicting Visual Classifiers from Unstructured Text
Authors: Mohamed Elhoseiny, Ahmed Elgammal, Babak Saleh
Date: 2015-12-31 00:00:00
Tasks: regression, Transfer Learning

People typically learn through exposure to visual concepts associated with
linguistic descriptions. For instance, teaching visual object categories to
children is often accompanied by descriptions in text or speech. In a machine
learning context, these observations motivates us to ask whether this learning
process could be computationally modeled to learn visual classifiers. More
specifically, the main question of this work is how to utilize purely textual
description of visual classes with no training images, to learn explicit visual
classifiers for them. We propose and investigate two baseline formulations,
based on regression and domain transfer, that predict a linear classifier.
Then, we propose a new constrained optimization formulation that combines a
regression function and a knowledge transfer function with additional
constraints to predict the parameters of a linear classifier. We also propose a
generic kernelized models where a kernel classifier is predicted in the form
defined by the representer theorem. The kernelized models allow defining and
utilizing any two RKHS (Reproducing Kernel Hilbert Space) kernel functions in
the visual space and text space, respectively. We finally propose a kernel
function between unstructured text descriptions that builds on distributional
semantics, which shows an advantage in our setting and could be useful for
other applications. We applied all the studied models to predict visual
classifiers on two fine-grained and challenging categorization datasets (CU
Birds and Flower Datasets), and the results indicate successful predictions of
our final model over several baselines that we designed.
Detection of Cooperative Interactions in Logistic Regression Models
Authors: Easton Li Xu, Xiaoning Qian, Tie Liu, Shuguang Cui
Date: 2016-02-12 00:00:00
Tasks: feature selection, regression

An important problem in the field of bioinformatics is to identify
interactive effects among profiled variables for outcome prediction. In this
paper, a logistic regression model with pairwise interactions among a set of
binary covariates is considered. Modeling the structure of the interactions by
a graph, our goal is to recover the interaction graph from independently
identically distributed (i.i.d.) samples of the covariates and the outcome.
  When viewed as a feature selection problem, a simple quantity called
influence is proposed as a measure of the marginal effects of the interaction
terms on the outcome. For the case when the underlying interaction graph is
known to be acyclic, it is shown that a simple algorithm that is based on a
maximum-weight spanning tree with respect to the plug-in estimates of the
influences not only has strong theoretical performance guarantees, but can also
outperform generic feature selection algorithms for recovering the interaction
graph from i.i.d. samples of the covariates and the outcome. Our results can
also be extended to the model that includes both individual effects and
pairwise interactions via the help of an auxiliary covariate.
Combined Independent Component Analysis and Canonical Polyadic Decomposition via Joint Diagonalization
Authors: Xiao-Feng Gong, Cheng-Yuan Wang, Ya-Na Hao, Qiu-Hua Lin
Date: 2013-11-05 00:00:00

Recently, there has been a trend to combine independent component analysis
and canonical polyadic decomposition (ICA-CPD) for an enhanced robustness for
the computation of CPD, and ICA-CPD could be further converted into CPD of a
5th-order partially symmetric tensor, by calculating the eigenmatrices of the
4th-order cumulant slices of a trilinear mixture. In this study, we propose a
new 5th-order CPD algorithm constrained with partial symmetry based on joint
diagonalization. As the main steps involved in the proposed algorithm undergo
no updating iterations for the loading matrices, it is much faster than the
existing algorithm based on alternating least squares and enhanced line search,
with competent performances. Simulation results are provided to demonstrate the
performance of the proposed algorithm.
An FFT-based Synchronization Approach to Recognize Human Behaviors using STN-LFP Signal
Authors: Hosein M. Golshan, Adam O. Hebb, Sara J. Hanrahan, Joshua Nedrud, Mohammad H. Mahoor
Date: 2016-12-28 00:00:00
Tasks: General Classification

Classification of human behavior is key to developing closed-loop Deep Brain
Stimulation (DBS) systems, which may be able to decrease the power consumption
and side effects of the existing systems. Recent studies have shown that the
Local Field Potential (LFP) signals from both Subthalamic Nuclei (STN) of the
brain can be used to recognize human behavior. Since the DBS leads implanted in
each STN can collect three bipolar signals, the selection of a suitable pair of
LFPs that achieves optimal recognition performance is still an open problem to
address. Considering the presence of synchronized aggregate activity in the
basal ganglia, this paper presents an FFT-based synchronization approach to
automatically select a relevant pair of LFPs and use the pair together with an
SVM-based MKL classifier for behavior recognition purposes. Our experiments on
five subjects show the superiority of the proposed approach compared to other
methods used for behavior classification.
Hippocampus Temporal Lobe Epilepsy Detection using a Combination of Shape-based Features and Spherical Harmonics Representation
Authors: Zohreh Kohan, Hamidreza Farhidzadeh, Reza Azmi, Behrouz Gholizadeh
Date: 2016-12-01 00:00:00
Tasks: General Classification, Hippocampus, image-classification, Image Classification, Specificity

Most of the temporal lobe epilepsy detection approaches are based on
hippocampus deformation and use complicated features, resulting, detection is
done with complicated features extraction and pre-processing task. In this
paper, a new detection method based on shape-based features and spherical
harmonics is proposed which can analysis the hippocampus shape anomaly and
detection asymmetry. This method consisted of two main parts; (1) shape feature
extraction, and (2) image classification. For evaluation, HFH database is used
which is publicly available in this field. Nine different geometry and 256
spherical harmonic features are introduced then selected Eighteen of them that
detect the asymmetry in hippocampus significantly in a randomly selected subset
of the dataset. Then a support vector machine (SVM) classifier was employed to
classify the remaining images of the dataset to normal and epileptic images
using our selected features. On a dataset of 25 images, 12 images were used for
feature extraction and the rest 13 for classification. The results show that
the proposed method has accuracy, specificity and sensitivity of, respectively,
84%, 100%, and 80%. Therefore, the proposed approach shows acceptable result
and is straightforward also; complicated pre-processing steps were omitted
compared to other methods.
Bank distress in the news: Describing events through deep learning
Authors: Samuel Rönnqvist, Peter Sarlin
Date: 2016-03-17 00:00:00
Tasks: Articles, Deep Learning, Descriptive

While many models are purposed for detecting the occurrence of significant
events in financial systems, the task of providing qualitative detail on the
developments is not usually as well automated. We present a deep learning
approach for detecting relevant discussion in text and extracting natural
language descriptions of events. Supervised by only a small set of event
information, comprising entity names and dates, the model is leveraged by
unsupervised learning of semantic vector representations on extensive text
data. We demonstrate applicability to the study of financial risk based on news
(6.6M articles), particularly bank distress and government interventions (243
events), where indices can signal the level of bank-stress-related reporting at
the entity level, or aggregated at national or European level, while being
coupled with explanations. Thus, we exemplify how text, as timely, widely
available and descriptive data, can serve as a useful complementary source of
information for financial and systemic risk analytics.
Deep Learning without Poor Local Minima
Authors: Kenji Kawaguchi
Date: 2016-05-23 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Deep Learning, Learning Theory

In this paper, we prove a conjecture published in 1989 and also partially
address an open problem announced at the Conference on Learning Theory (COLT)
2015. With no unrealistic assumption, we first prove the following statements
for the squared loss function of deep linear neural networks with any depth and
any widths: 1) the function is non-convex and non-concave, 2) every local
minimum is a global minimum, 3) every critical point that is not a global
minimum is a saddle point, and 4) there exist "bad" saddle points (where the
Hessian has no negative eigenvalue) for the deeper networks (with more than
three layers), whereas there is no bad saddle point for the shallow networks
(with three layers). Moreover, for deep nonlinear neural networks, we prove the
same four statements via a reduction to a deep linear model under the
independence assumption adopted from recent work. As a result, we present an
instance, for which we can answer the following question: how difficult is it
to directly train a deep model in theory? It is more difficult than the
classical machine learning models (because of the non-convexity), but not too
difficult (because of the nonexistence of poor local minima). Furthermore, the
mathematically proven existence of bad saddle points for deeper models would
suggest a possible open problem. We note that even though we have advanced the
theoretical foundations of deep learning and non-convex optimization, there is
still a gap between theory and practice.
The ontogeny of discourse structure mimics the development of literature
Authors: Natalia Bezerra Mota, Sylvia Pinheiro, Mariano Sigman, Diego Fernandez Slezak, Guillermo Cecchi, Mauro Copelli, Sidarta Ribeiro
Date: 2016-12-27 00:00:00
Tasks: Diversity

Discourse varies with age, education, psychiatric state and historical epoch,
but the ontogenetic and cultural dynamics of discourse structure remain to be
quantitatively characterized. To this end we investigated word graphs obtained
from verbal reports of 200 subjects ages 2-58, and 676 literary texts spanning
~5,000 years. In healthy subjects, lexical diversity, graph size, and
long-range recurrence departed from initial near-random levels through a
monotonic asymptotic increase across ages, while short-range recurrence showed
a corresponding decrease. These changes were explained by education and suggest
a hierarchical development of discourse structure: short-range recurrence and
lexical diversity stabilize after elementary school, but graph size and
long-range recurrence only stabilize after high school. This gradual maturation
was blurred in psychotic subjects, who maintained in adulthood a near-random
structure. In literature, monotonic asymptotic changes over time were
remarkable: While lexical diversity, long-range recurrence and graph size
increased away from near-randomness, short-range recurrence declined, from
above to below random levels. Bronze Age texts are structurally similar to
childish or psychotic discourses, but subsequent texts converge abruptly to the
healthy adult pattern around the onset of the Axial Age (800-200 BC), a period
of pivotal cultural change. Thus, individually as well as historically,
discourse maturation increases the range of word recurrence away from
randomness.
A Non-generative Framework and Convex Relaxations for Unsupervised Learning
Authors: Elad Hazan, Tengyu Ma
Date: 2016-10-04 00:00:00
Proceeding: NeurIPS 2016 12

We give a novel formal theoretical framework for unsupervised learning with
two distinctive characteristics. First, it does not assume any generative model
and based on a worst-case performance metric. Second, it is comparative, namely
performance is measured with respect to a given hypothesis class. This allows
to avoid known computational hardness results and improper algorithms based on
convex relaxations. We show how several families of unsupervised learning
models, which were previously only analyzed under probabilistic assumptions and
are otherwise provably intractable, can be efficiently learned in our framework
by convex optimization.
A Hybrid Both Filter and Wrapper Feature Selection Method for Microarray Classification
Authors: Li-Yeh Chuang, Chao-Hsuan Ke, Cheng-Hong Yang
Date: 2016-12-27 00:00:00
Tasks: Classification, feature selection, General Classification, Microarray Classification

Gene expression data is widely used in disease analysis and cancer diagnosis.
However, since gene expression data could contain thousands of genes
simultaneously, successful microarray classification is rather difficult.
Feature selection is an important pre-treatment for any classification process.
Selecting a useful gene subset as a classifier not only decreases the
computational time and cost, but also increases classification accuracy. In
this study, we applied the information gain method as a filter approach, and an
improved binary particle swarm optimization as a wrapper approach to implement
feature selection; selected gene subsets were used to evaluate the performance
of classification. Experimental results show that by employing the proposed
method fewer gene subsets needed to be selected and better classification
accuracy could be obtained.
The Linearization of Belief Propagation on Pairwise Markov Networks
Authors: Wolfgang Gatterbauer
Date: 2015-02-17 00:00:00
Tasks: Node Classification

Belief Propagation (BP) is a widely used approximation for exact
probabilistic inference in graphical models, such as Markov Random Fields
(MRFs). In graphs with cycles, however, no exact convergence guarantees for BP
are known, in general. For the case when all edges in the MRF carry the same
symmetric, doubly stochastic potential, recent works have proposed to
approximate BP by linearizing the update equations around default values, which
was shown to work well for the problem of node classification. The present
paper generalizes all prior work and derives an approach that approximates
loopy BP on any pairwise MRF with the problem of solving a linear equation
system. This approach combines exact convergence guarantees and a fast matrix
implementation with the ability to model heterogenous networks. Experiments on
synthetic graphs with planted edge potentials show that the linearization has
comparable labeling accuracy as BP for graphs with weak potentials, while
speeding-up inference by orders of magnitude.
Reproducible Pattern Recognition Research: The Case of Optimistic SSL
Authors: Jesse H. Krijthe, Marco Loog
Date: 2016-12-27 00:00:00

In this paper, we discuss the approaches we took and trade-offs involved in
making a paper on a conceptual topic in pattern recognition research fully
reproducible. We discuss our definition of reproducibility, the tools used, how
the analysis was set up, show some examples of alternative analyses the code
enables and discuss our views on reproducibility.
Robust 3D Hand Pose Estimation in Single Depth Images: from Single-View CNN to Multi-View CNNs
Authors: Liuhao Ge, Hui Liang, Junsong Yuan, Daniel Thalmann
Date: 2016-06-23 00:00:00
Proceeding: CVPR 2016 6
Tasks: 3D Hand Pose Estimation, Hand Pose Estimation, Pose Estimation

Articulated hand pose estimation plays an important role in human-computer
interaction. Despite the recent progress, the accuracy of existing methods is
still not satisfactory, partially due to the difficulty of embedded
high-dimensional and non-linear regression problem. Different from the existing
discriminative methods that regress for the hand pose with a single depth
image, we propose to first project the query depth image onto three orthogonal
planes and utilize these multi-view projections to regress for 2D heat-maps
which estimate the joint positions on each plane. These multi-view heat-maps
are then fused to produce final 3D hand pose estimation with learned pose
priors. Experiments show that the proposed method largely outperforms
state-of-the-art on a challenging dataset. Moreover, a cross-dataset experiment
also demonstrates the good generalization ability of the proposed method.
Bayesian Nonparametric Models for Synchronous Brain-Computer Interfaces
Authors: Jaime Fernando Delgado Saa, Mujdat Cetin
Date: 2016-12-27 00:00:00
Tasks: Brain Computer Interface, EEG, Electroencephalogram (EEG), General Classification

A brain-computer interface (BCI) is a system that aims for establishing a
non-muscular communication path for subjects who had suffer from a
neurodegenerative disease. Many BCI systems make use of the phenomena of
event-related synchronization and de-synchronization of brain waves as a main
feature for classification of different cognitive tasks. However, the temporal
dynamics of the electroencephalographic (EEG) signals contain additional
information that can be incorporated into the inference engine in order to
improve the performance of the BCIs. This information about the dynamics of the
signals have been exploited previously in BCIs by means of generative and
discriminative methods. In particular, hidden Markov models (HMMs) have been
used in previous works. These methods have the disadvantage that the model
parameters such as the number of hidden states and the number of Gaussian
mixtures need to be fix "a priori". In this work, we propose a Bayesian
nonparametric model for brain signal classification that does not require "a
priori" selection of the number of hidden states and the number of Gaussian
mixtures of a HMM. The results show that the proposed model outperform other
methods based on HMM as well as the winner algorithm of the BCI competition IV.
A Sparse Nonlinear Classifier Design Using AUC Optimization
Authors: Vishal Kakkar, Shirish K. Shevade, S. Sundararajan, Dinesh Garg
Date: 2016-12-27 00:00:00

AUC (Area under the ROC curve) is an important performance measure for
applications where the data is highly imbalanced. Learning to maximize AUC
performance is thus an important research problem. Using a max-margin based
surrogate loss function, AUC optimization problem can be approximated as a
pairwise rankSVM learning problem. Batch learning methods for solving the
kernelized version of this problem suffer from scalability and may not result
in sparse classifiers. Recent years have witnessed an increased interest in the
development of online or single-pass online learning algorithms that design a
classifier by maximizing the AUC performance. The AUC performance of nonlinear
classifiers, designed using online methods, is not comparable with that of
nonlinear classifiers designed using batch learning algorithms on many
real-world datasets. Motivated by these observations, we design a scalable
algorithm for maximizing AUC performance by greedily adding the required number
of basis functions into the classifier model. The resulting sparse classifiers
perform faster inference. Our experimental results show that the level of
sparsity achievable can be order of magnitude smaller than the Kernel RankSVM
model without affecting the AUC performance much.