pwc-archive/papers-with-abstracts huggingface dataset:
Canonical Autocorrelation Analysis
Authors: Maria De-Arteaga, Artur Dubrawski, Peter Huggins
Date: 2015-11-19 00:00:00
Tasks: Anomaly Detection

We present an extension of sparse Canonical Correlation Analysis (CCA)
designed for finding multiple-to-multiple linear correlations within a single
set of variables. Unlike CCA, which finds correlations between two sets of data
where the rows are matched exactly but the columns represent separate sets of
variables, the method proposed here, Canonical Autocorrelation Analysis (CAA),
finds multivariate correlations within just one set of variables. This can be
useful when we look for hidden parsimonious structures in data, each involving
only a small subset of all features. In addition, the discovered correlations
are highly interpretable as they are formed by pairs of sparse linear
combinations of the original features. We show how CAA can be of use as a tool
for anomaly detection when the expected structure of correlations is not
followed by anomalous data. We illustrate the utility of CAA in two application
domains where single-class and unsupervised learning of correlation structures
are particularly relevant: breast cancer diagnosis and radiation threat
detection. When applied to the Wisconsin Breast Cancer data, single-class CAA
is competitive with supervised methods used in literature. On the radiation
threat detection task, unsupervised CAA performs significantly better than an
unsupervised alternative prevalent in the domain, while providing valuable
additional insights for threat analysis.
Fast Parallel SAME Gibbs Sampling on General Discrete Bayesian Networks
Authors: Daniel Seita, Haoyu Chen, John Canny
Date: 2015-11-19 00:00:00

A fundamental task in machine learning and related fields is to perform
inference on Bayesian networks. Since exact inference takes exponential time in
general, a variety of approximate methods are used. Gibbs sampling is one of
the most accurate approaches and provides unbiased samples from the posterior
but it has historically been too expensive for large models. In this paper, we
present an optimized, parallel Gibbs sampler augmented with state replication
(SAME or State Augmented Marginal Estimation) to decrease convergence time. We
find that SAME can improve the quality of parameter estimates while
accelerating convergence. Experiments on both synthetic and real data show that
our Gibbs sampler is substantially faster than the state of the art sampler,
JAGS, without sacrificing accuracy. Our ultimate objective is to introduce the
Gibbs sampler to researchers in many fields to expand their range of feasible
inference problems.
sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings
Authors: Andrew Trask, Phil Michalak, John Liu
Date: 2015-11-19 00:00:00
Tasks: Dependency Parsing, Word Embeddings, Word Sense Disambiguation

Neural word representations have proven useful in Natural Language Processing
(NLP) tasks due to their ability to efficiently model complex semantic and
syntactic word relationships. However, most techniques model only one
representation per word, despite the fact that a single word can have multiple
meanings or "senses". Some techniques model words by using multiple vectors
that are clustered based on context. However, recent neural approaches rarely
focus on the application to a consuming NLP algorithm. Furthermore, the
training process of recent word-sense models is expensive relative to
single-sense embedding processes. This paper presents a novel approach which
addresses these concerns by modeling multiple embeddings for each word based on
supervised disambiguation, which provides a fast and accurate way for a
consuming NLP model to select a sense-disambiguated embedding. We demonstrate
that these embeddings can disambiguate both contrastive senses such as nominal
and verbal senses as well as nuanced senses such as sarcasm. We further
evaluate Part-of-Speech disambiguated embeddings on neural dependency parsing,
yielding a greater than 8% average error reduction in unlabeled attachment
scores across 6 languages.
A Unified Gradient Regularization Family for Adversarial Examples
Authors: Chunchuan Lyu, Kai-Zhu Huang, Hai-Ning Liang
Date: 2015-11-19 00:00:00
Tasks: BIG-bench Machine Learning, Data Augmentation

Adversarial examples are augmented data points generated by imperceptible
perturbation of input samples. They have recently drawn much attention with the
machine learning and data mining community. Being difficult to distinguish from
real examples, such adversarial examples could change the prediction of many of
the best learning models including the state-of-the-art deep learning models.
Recent attempts have been made to build robust models that take into account
adversarial examples. However, these methods can either lead to performance
drops or lack mathematical motivations. In this paper, we propose a unified
framework to build robust machine learning models against adversarial examples.
More specifically, using the unified framework, we develop a family of gradient
regularization methods that effectively penalize the gradient of loss function
w.r.t. inputs. Our proposed framework is appealing in that it offers a unified
view to deal with adversarial examples. It incorporates another
recently-proposed perturbation based approach as a special case. In addition,
we present some visual effects that reveals semantic meaning in those
perturbations, and thus support our regularization method and provide another
explanation for generalizability of adversarial examples. By applying this
technique to Maxout networks, we conduct a series of experiments and achieve
encouraging results on two benchmark datasets. In particular,we attain the best
accuracy on MNIST data (without data augmentation) and competitive performance
on CIFAR-10 data.
Dynamic Adaptive Network Intelligence
Authors: Richard Searle, Megan Bingham-Walker
Date: 2015-11-19 00:00:00
Tasks: Question Answering, Weakly-supervised Learning

Accurate representational learning of both the explicit and implicit
relationships within data is critical to the ability of machines to perform
more complex and abstract reasoning tasks. We describe the efficient weakly
supervised learning of such inferences by our Dynamic Adaptive Network
Intelligence (DANI) model. We report state-of-the-art results for DANI over
question answering tasks in the bAbI dataset that have proved difficult for
contemporary approaches to learning representation (Weston et al., 2015).
Learning Representations Using Complex-Valued Nets
Authors: Andy M. Sarroff, Victor Shepardson, Michael A. Casey
Date: 2015-11-19 00:00:00
Tasks: Representation Learning, Time Series, Time Series Analysis

Complex-valued neural networks (CVNNs) are an emerging field of research in
neural networks due to their potential representational properties for audio,
image, and physiological signals. It is common in signal processing to
transform sequences of real values to the complex domain via a set of complex
basis functions, such as the Fourier transform. We show how CVNNs can be used
to learn complex representations of real valued time-series data. We present
methods and results using a framework that can compose holomorphic and
non-holomorphic functions in a multi-layer network using a theoretical result
called the Wirtinger derivative. We test our methods on a representation
learning task for real-valued signals, recurrent complex-valued networks and
their real-valued counterparts. Our results show that recurrent complex-valued
networks can perform as well as their real-valued counterparts while learning
filters that are representative of the domain of the data.
Handcrafted Local Features are Convolutional Neural Networks
Authors: Zhenzhong Lan, Shoou-I Yu, Ming Lin, Bhiksha Raj, Alexander G. Hauptmann
Date: 2015-11-16 00:00:00
Tasks: Action Recognition, Optical Flow Estimation, Temporal Action Localization, Video Classification

Image and video classification research has made great progress through the
development of handcrafted local features and learning based features. These
two architectures were proposed roughly at the same time and have flourished at
overlapping stages of history. However, they are typically viewed as distinct
approaches. In this paper, we emphasize their structural similarities and show
how such a unified view helps us in designing features that balance efficiency
and effectiveness. As an example, we study the problem of designing efficient
video feature learning algorithms for action recognition.
  We approach this problem by first showing that local handcrafted features and
Convolutional Neural Networks (CNNs) share the same convolution-pooling network
structure. We then propose a two-stream Convolutional ISA (ConvISA) that adopts
the convolution-pooling structure of the state-of-the-art handcrafted video
feature with greater modeling capacities and a cost-effective training
algorithm. Through custom designed network structures for pixels and optical
flow, our method also reflects distinctive characteristics of these two data
sources.
  Our experimental results on standard action recognition benchmarks show that
by focusing on the structure of CNNs, rather than end-to-end training methods,
we are able to design an efficient and powerful video feature learning
algorithm.
Robust Classification by Pre-conditioned LASSO and Transductive Diffusion Component Analysis
Authors: Yanwei Fu, De-An Huang, Leonid Sigal
Date: 2015-11-19 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, Outlier Detection, Robust classification

Modern machine learning-based recognition approaches require large-scale datasets with large number of labelled training images. However, such datasets are inherently difficult and costly to collect and annotate. Hence there is a great and growing interest in automatic dataset collection methods that can leverage the web. % which are collected % in a cheap, efficient and yet unreliable way. Collecting datasets in this way, however, requires robust and efficient ways for detecting and excluding outliers that are common and prevalent. % Outliers are thus a % prominent treat of using these dataset. So far, there have been a limited effort in machine learning community to directly detect outliers for robust classification. Inspired by the recent work on Pre-conditioned LASSO, this paper formulates the outlier detection task using Pre-conditioned LASSO and employs \red{unsupervised} transductive diffusion component analysis to both integrate the topological structure of the data manifold, from labeled and unlabeled instances, and reduce the feature dimensionality. Synthetic experiments as well as results on two real-world classification tasks show that our framework can robustly detect the outliers and improve classification.
Recurrent Reinforcement Learning: A Hybrid Approach
Authors: Xiujun Li, Lihong Li, Jianfeng Gao, Xiaodong He, Jianshu Chen, Li Deng, Ji He
Date: 2015-09-10 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Successful applications of reinforcement learning in real-world problems
often require dealing with partially observable states. It is in general very
challenging to construct and infer hidden states as they often depend on the
agent's entire interaction history and may require substantial domain
knowledge. In this work, we investigate a deep-learning approach to learning
the representation of states in partially observable tasks, with minimal prior
knowledge of the domain. In particular, we propose a new family of hybrid
models that combines the strength of both supervised learning (SL) and
reinforcement learning (RL), trained in a joint fashion: The SL component can
be a recurrent neural networks (RNN) or its long short-term memory (LSTM)
version, which is equipped with the desired property of being able to capture
long-term dependency on history, thus providing an effective way of learning
the representation of hidden states. The RL component is a deep Q-network (DQN)
that learns to optimize the control for maximizing long-term rewards. Extensive
experiments in a direct mailing campaign problem demonstrate the effectiveness
and advantages of the proposed approach, which performs the best among a set of
previous state-of-the-art methods.
face anti-spoofing based on color texture analysis
Authors: Zinelabidine Boulkenafet, Jukka Komulainen, Abdenour Hadid
Date: 2015-11-19 00:00:00
Tasks: Face Anti-Spoofing, Texture Classification

Research on face spoofing detection has mainly been focused on analyzing the
luminance of the face images, hence discarding the chrominance information
which can be useful for discriminating fake faces from genuine ones. In this
work, we propose a new face anti-spoofing method based on color texture
analysis. We analyze the joint color-texture information from the luminance and
the chrominance channels using a color local binary pattern descriptor. More
specifically, the feature histograms are extracted from each image band
separately. Extensive experiments on two benchmark datasets, namely CASIA face
anti-spoofing and Replay-Attack databases, showed excellent results compared to
the state-of-the-art. Most importantly, our inter-database evaluation depicts
that the proposed approach showed very promising generalization capabilities.
Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks
Authors: Stefan Lee, Senthil Purushwalkam, Michael Cogswell, David Crandall, Dhruv Batra
Date: 2015-11-19 00:00:00
Tasks: Diversity

Convolutional Neural Networks have achieved state-of-the-art performance on a
wide range of tasks. Most benchmarks are led by ensembles of these powerful
learners, but ensembling is typically treated as a post-hoc procedure
implemented by averaging independently trained models with model variation
induced by bagging or random initialization. In this paper, we rigorously treat
ensembling as a first-class problem to explicitly address the question: what
are the best strategies to create an ensemble? We first compare a large number
of ensembling strategies, and then propose and evaluate novel strategies, such
as parameter sharing (through a new family of models we call TreeNets) as well
as training under ensemble-aware and diversity-encouraging losses. We
demonstrate that TreeNets can improve ensemble performance and that diverse
ensembles can be trained end-to-end under a unified loss, achieving
significantly higher "oracle" accuracies than classical ensembles.
Good, Better, Best: Choosing Word Embedding Context
Authors: James Cross, Bing Xiang, Bo-Wen Zhou
Date: 2015-11-19 00:00:00
Tasks: Sentence

We propose two methods of learning vector representations of words and
phrases that each combine sentence context with structural features extracted
from dependency trees. Using several variations of neural network classifier,
we show that these combined methods lead to improved performance when used as
input features for supervised term-matching.
Faster method for Deep Belief Network based Object classification using DWT
Authors: Saurabh Sihag, Pranab Kumar Dutta
Date: 2015-11-19 00:00:00
Tasks: General Classification

A Deep Belief Network (DBN) requires large, multiple hidden layers with high
number of hidden units to learn good features from the raw pixels of large
images. This implies more training time as well as computational complexity. By
integrating DBN with Discrete Wavelet Transform (DWT), both training time and
computational complexity can be reduced. The low resolution images obtained
after application of DWT are used to train multiple DBNs. The results obtained
from these DBNs are combined using a weighted voting algorithm. The performance
of this method is found to be competent and faster in comparison with that of
traditional DBNs.
Critical Parameters in Particle Swarm Optimisation
Authors: J. Michael Herrmann, Adam Erskine, Thomas Joyce
Date: 2015-11-19 00:00:00

Particle swarm optimisation is a metaheuristic algorithm which finds
reasonable solutions in a wide range of applied problems if suitable parameters
are used. We study the properties of the algorithm in the framework of random
dynamical systems which, due to the quasi-linear swarm dynamics, yields
analytical results for the stability properties of the particles. Such
considerations predict a relationship between the parameters of the algorithm
that marks the edge between convergent and divergent behaviours. Comparison
with simulations indicates that the algorithm performs best near this margin of
instability.
Gaussian Mixture Embeddings for Multiple Word Prototypes
Authors: Xinchi Chen, Xipeng Qiu, Jingxiang Jiang, Xuanjing Huang
Date: 2015-11-19 00:00:00

Recently, word representation has been increasingly focused on for its
excellent properties in representing the word semantics. Previous works mainly
suffer from the problem of polysemy phenomenon. To address this problem, most
of previous models represent words as multiple distributed vectors. However, it
cannot reflect the rich relations between words by representing words as points
in the embedded space. In this paper, we propose the Gaussian mixture skip-gram
(GMSG) model to learn the Gaussian mixture embeddings for words based on
skip-gram framework. Each word can be regarded as a gaussian mixture
distribution in the embedded space, and each gaussian component represents a
word sense. Since the number of senses varies from word to word, we further
propose the Dynamic GMSG (D-GMSG) model by adaptively increasing the sense
number of words during training. Experiments on four benchmarks show the
effectiveness of our proposed model.
Behavior Query Discovery in System-Generated Temporal Graphs
Authors: Bo Zong, Xusheng Xiao, Zhichun Li, Zhen-Yu Wu, Zhiyun Qian, Xifeng Yan, Ambuj K. Singh, Guofei Jiang
Date: 2015-11-18 00:00:00

Computer system monitoring generates huge amounts of logs that record the
interaction of system entities. How to query such data to better understand
system behaviors and identify potential system risks and malicious behaviors
becomes a challenging task for system administrators due to the dynamics and
heterogeneity of the data. System monitoring data are essentially heterogeneous
temporal graphs with nodes being system entities and edges being their
interactions over time. Given the complexity of such graphs, it becomes
time-consuming for system administrators to manually formulate useful queries
in order to examine abnormal activities, attacks, and vulnerabilities in
computer systems.
  In this work, we investigate how to query temporal graphs and treat query
formulation as a discriminative temporal graph pattern mining problem. We
introduce TGMiner to mine discriminative patterns from system logs, and these
patterns can be taken as templates for building more complex queries. TGMiner
leverages temporal information in graphs to prune graph patterns that share
similar growth trend without compromising pattern quality. Experimental results
on real system data show that TGMiner is 6-32 times faster than baseline
methods. The discovered patterns were verified by system experts; they achieved
high precision (97%) and recall (91%).
Towards Open Set Deep Networks
Authors: Abhijit Bendale, Terrance Boult
Date: 2015-11-19 00:00:00
Proceeding: CVPR 2016 6
Tasks: Open Set Learning

Deep networks have produced significant gains for various visual recognition
problems, leading to high impact academic and commercial applications. Recent
work in deep networks highlighted that it is easy to generate images that
humans would never classify as a particular object class, yet networks classify
such images high confidence as that given class - deep network are easily
fooled with images humans do not consider meaningful. The closed set nature of
deep networks forces them to choose from one of the known classes leading to
such artifacts. Recognition in the real world is open set, i.e. the recognition
system should reject unknown/unseen classes at test time. We present a
methodology to adapt deep networks for open set recognition, by introducing a
new model layer, OpenMax, which estimates the probability of an input being
from an unknown class. A key element of estimating the unknown probability is
adapting Meta-Recognition concepts to the activation patterns in the
penultimate layer of the network. OpenMax allows rejection of "fooling" and
unrelated open set images presented to the system; OpenMax greatly reduces the
number of obvious errors made by a deep network. We prove that the OpenMax
concept provides bounded open space risk, thereby formally providing an open
set recognition solution. We evaluate the resulting open set deep networks
using pre-trained networks from the Caffe Model-zoo on ImageNet 2012 validation
data, and thousands of fooling and open set images. The proposed OpenMax model
significantly outperforms open set recognition accuracy of basic deep networks
as well as deep networks with thresholding of SoftMax probabilities.
Teaching Machines to Read and Comprehend
Authors: Karl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, Phil Blunsom
Date: 2015-06-10 00:00:00
Proceeding: NeurIPS 2015 12
Tasks: Reading Comprehension

Teaching machines to read natural language documents remains an elusive
challenge. Machine reading systems can be tested on their ability to answer
questions posed on the contents of documents that they have seen, but until now
large scale training and test datasets have been missing for this type of
evaluation. In this work we define a new methodology that resolves this
bottleneck and provides large scale supervised reading comprehension data. This
allows us to develop a class of attention based deep neural networks that learn
to read real documents and answer complex questions with minimal prior
knowledge of language structure.
Diffusion Representations
Authors: Moshe Salhov, Amit Bermanis, Guy Wolf, Amir Averbuch
Date: 2015-11-19 00:00:00

Diffusion Maps framework is a kernel based method for manifold learning and
data analysis that defines diffusion similarities by imposing a Markovian
process on the given dataset. Analysis by this process uncovers the intrinsic
geometric structures in the data. Recently, it was suggested to replace the
standard kernel by a measure-based kernel that incorporates information about
the density of the data. Thus, the manifold assumption is replaced by a more
general measure-based assumption.
  The measure-based diffusion kernel incorporates two separate independent
representations. The first determines a measure that correlates with a density
that represents normal behaviors and patterns in the data. The second consists
of the analyzed multidimensional data points.
  In this paper, we present a representation framework for data analysis of
datasets that is based on a closed-form decomposition of the measure-based
kernel. The proposed representation preserves pairwise diffusion distances that
does not depend on the data size while being invariant to scale. For a
stationary data, no out-of-sample extension is needed for embedding newly
arrived data points in the representation space. Several aspects of the
presented methodology are demonstrated on analytically generated data.