pwc-archive/papers-with-abstracts huggingface dataset:
Introducing DeepBalance: Random Deep Belief Network Ensembles to Address Class Imbalance
Authors: Peter Xenopoulos
Date: 2017-09-28 00:00:00
Tasks: feature selection, Fraud Detection

Class imbalance problems manifest in domains such as financial fraud
detection or network intrusion analysis, where the prevalence of one class is
much higher than another. Typically, practitioners are more interested in
predicting the minority class than the majority class as the minority class may
carry a higher misclassification cost. However, classifier performance
deteriorates in the face of class imbalance as oftentimes classifiers may
predict every point as the majority class. Methods for dealing with class
imbalance include cost-sensitive learning or resampling techniques. In this
paper, we introduce DeepBalance, an ensemble of deep belief networks trained
with balanced bootstraps and random feature selection. We demonstrate that our
proposed method outperforms baseline resampling methods such as SMOTE and
under- and over-sampling in metrics such as AUC and sensitivity when applied to
a highly imbalanced financial transaction data. Additionally, we explore
performance and training time implications of various model parameters.
Furthermore, we show that our model is easily parallelizable, which can reduce
training times. Finally, we present an implementation of DeepBalance in R.
Reinforcement Learning in a large scale photonic Recurrent Neural Network
Authors: Julian Bueno, Sheler Maktoobi, Luc Froehly, Ingo Fischer, Maxime Jacquot, Laurent Larger, Daniel Brunner
Date: 2017-11-14 00:00:00
Tasks: BIG-bench Machine Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Photonic Neural Network implementations have been gaining considerable
attention as a potentially disruptive future technology. Demonstrating learning
in large scale neural networks is essential to establish photonic machine
learning substrates as viable information processing systems. Realizing
photonic Neural Networks with numerous nonlinear nodes in a fully parallel and
efficient learning hardware was lacking so far. We demonstrate a network of up
to 2500 diffractively coupled photonic nodes, forming a large scale Recurrent
Neural Network. Using a Digital Micro Mirror Device, we realize reinforcement
learning. Our scheme is fully parallel, and the passive weights maximize energy
efficiency and bandwidth. The computational output efficiently converges and we
achieve very good performance.
TorusE: Knowledge Graph Embedding on a Lie Group
Authors: Takuma Ebisu, Ryutaro Ichise
Date: 2017-11-15 00:00:00
Tasks: Entity Embeddings, Graph Embedding, Knowledge Graph Completion, Knowledge Graph Embedding, Knowledge Graphs, Link Prediction

Knowledge graphs are useful for many artificial intelligence (AI) tasks.
However, knowledge graphs often have missing facts. To populate the graphs,
knowledge graph embedding models have been developed. Knowledge graph embedding
models map entities and relations in a knowledge graph to a vector space and
predict unknown triples by scoring candidate triples. TransE is the first
translation-based method and it is well known because of its simplicity and
efficiency for knowledge graph completion. It employs the principle that the
differences between entity embeddings represent their relations. The principle
seems very simple, but it can effectively capture the rules of a knowledge
graph. However, TransE has a problem with its regularization. TransE forces
entity embeddings to be on a sphere in the embedding vector space. This
regularization warps the embeddings and makes it difficult for them to fulfill
the abovementioned principle. The regularization also affects adversely the
accuracies of the link predictions. On the other hand, regularization is
important because entity embeddings diverge by negative sampling without it.
This paper proposes a novel embedding model, TorusE, to solve the
regularization problem. The principle of TransE can be defined on any Lie
group. A torus, which is one of the compact Lie groups, can be chosen for the
embedding space to avoid regularization. To the best of our knowledge, TorusE
is the first model that embeds objects on other than a real or complex vector
space, and this paper is the first to formally discuss the problem of
regularization of TransE. Our approach outperforms other state-of-the-art
approaches such as TransE, DistMult and ComplEx on a standard link prediction
task. We show that TorusE is scalable to large-size knowledge graphs and is
faster than the original TransE.
Robust Matrix Elastic Net based Canonical Correlation Analysis: An Effective Algorithm for Multi-View Unsupervised Learning
Authors: Peng-Bo Zhang, Zhi-Xin Yang
Date: 2017-11-14 00:00:00
Tasks: feature selection

This paper presents a robust matrix elastic net based canonical correlation
analysis (RMEN-CCA) for multiple view unsupervised learning problems, which
emphasizes the combination of CCA and the robust matrix elastic net (RMEN) used
as coupled feature selection. The RMEN-CCA leverages the strength of the RMEN
to distill naturally meaningful features without any prior assumption and to
measure effectively correlations between different 'views'. We can further
employ directly the kernel trick to extend the RMEN-CCA to the kernel scenario
with theoretical guarantees, which takes advantage of the kernel trick for
highly complicated nonlinear feature learning. Rather than simply incorporating
existing regularization minimization terms into CCA, this paper provides a new
learning paradigm for CCA and is the first to derive a coupled feature
selection based CCA algorithm that guarantees convergence. More significantly,
for CCA, the newly-derived RMEN-CCA bridges the gap between measurement of
relevance and coupled feature selection. Moreover, it is nontrivial to tackle
directly the RMEN-CCA by previous optimization approaches derived from its
sophisticated model architecture. Therefore, this paper further offers a bridge
between a new optimization problem and an existing efficient iterative
approach. As a consequence, the RMEN-CCA can overcome the limitation of CCA and
address large-scale and streaming data problems. Experimental results on four
popular competing datasets illustrate that the RMEN-CCA performs more
effectively and efficiently than do state-of-the-art approaches.
A Sequential Neural Encoder with Latent Structured Description for Modeling Sentences
Authors: Yu-Ping Ruan, Qian Chen, Zhen-Hua Ling
Date: 2017-11-15 00:00:00
Tasks: Chunking, Natural Language Inference, Sentence, Sentence Embeddings, Sentiment Analysis

In this paper, we propose a sequential neural encoder with latent structured
description (SNELSD) for modeling sentences. This model introduces latent
chunk-level representations into conventional sequential neural encoders, i.e.,
recurrent neural networks (RNNs) with long short-term memory (LSTM) units, to
consider the compositionality of languages in semantic modeling. An SNELSD
model has a hierarchical structure that includes a detection layer and a
description layer. The detection layer predicts the boundaries of latent word
chunks in an input sentence and derives a chunk-level vector for each word. The
description layer utilizes modified LSTM units to process these chunk-level
vectors in a recurrent manner and produces sequential encoding outputs. These
output vectors are further concatenated with word vectors or the outputs of a
chain LSTM encoder to obtain the final sentence representation. All the model
parameters are learned in an end-to-end manner without a dependency on
additional text chunking or syntax parsing. A natural language inference (NLI)
task and a sentiment analysis (SA) task are adopted to evaluate the performance
of our proposed model. The experimental results demonstrate the effectiveness
of the proposed SNELSD model on exploring task-dependent chunking patterns
during the semantic modeling of sentences. Furthermore, the proposed method
achieves better performance than conventional chain LSTMs and tree-structured
LSTMs on both tasks.
Deep Inception-Residual Laplacian Pyramid Networks for Accurate Single Image Super-Resolution
Authors: Yongliang Tang, Weiguo Gong, Xi Chen, Weihong Li
Date: 2017-11-15 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

With exploiting contextual information over large image regions in an
efficient way, the deep convolutional neural network has shown an impressive
performance for single image super-resolution (SR). In this paper, we propose a
deep convolutional network by cascading the well-designed inception-residual
blocks within the deep Laplacian pyramid framework to progressively restore the
missing high-frequency details of high-resolution (HR) images. By optimizing
our network structure, the trainable depth of the proposed network gains a
significant improvement, which in turn improves super-resolving accuracy. With
our network depth increasing, however, the saturation and degradation of
training accuracy continues to be a critical problem. As regard to this, we
propose an effective two-stage training strategy, in which we firstly use
images downsampled from the ground-truth HR images as the optimal objective to
train the inception-residual blocks in each pyramid level with an extremely
high learning rate enabled by gradient clipping, and then the ground-truth HR
images are used to fine-tune all the pre-trained inception-residual blocks for
obtaining the final SR model. Furthermore, we present a new loss function
operating in both image space and local rank space to optimize our network for
exploiting the contextual information among different output components.
Extensive experiments on benchmark datasets validate that the proposed method
outperforms existing state-of-the-art SR methods in terms of the objective
evaluation as well as the visual quality.
Spatio-Temporal Saliency Networks for Dynamic Saliency Prediction
Authors: Cagdas Bak, Aysun Kocak, Erkut Erdem, Aykut Erdem
Date: 2016-07-16 00:00:00
Tasks: Optical Flow Estimation, Prediction, Saliency Prediction

Computational saliency models for still images have gained significant
popularity in recent years. Saliency prediction from videos, on the other hand,
has received relatively little interest from the community. Motivated by this,
in this work, we study the use of deep learning for dynamic saliency prediction
and propose the so-called spatio-temporal saliency networks. The key to our
models is the architecture of two-stream networks where we investigate
different fusion mechanisms to integrate spatial and temporal information. We
evaluate our models on the DIEM and UCF-Sports datasets and present highly
competitive results against the existing state-of-the-art models. We also carry
out some experiments on a number of still images from the MIT300 dataset by
exploiting the optical flow maps predicted from these images. Our results show
that considering inherent motion information in this way can be helpful for
static saliency estimation.
Accelerating Cross-Validation in Multinomial Logistic Regression with $\ell_1$-Regularization
Authors: Tomoyuki Obuchi, Yoshiyuki Kabashima
Date: 2017-11-15 00:00:00
Tasks: BIG-bench Machine Learning, regression

We develop an approximate formula for evaluating a cross-validation estimator
of predictive likelihood for multinomial logistic regression regularized by an
$\ell_1$-norm. This allows us to avoid repeated optimizations required for
literally conducting cross-validation; hence, the computational time can be
significantly reduced. The formula is derived through a perturbative approach
employing the largeness of the data size and the model dimensionality. An
extension to the elastic net regularization is also addressed. The usefulness
of the approximate formula is demonstrated on simulated data and the ISOLET
dataset from the UCI machine learning repository.
Adversarially Regularized Autoencoders
Authors: Jake Zhao, Yoon Kim, Kelly Zhang, Alexander M. Rush, Yann Lecun
Date: 2017-06-13 00:00:00
Tasks: Representation Learning, Style Transfer

Deep latent variable models, trained using variational autoencoders or
generative adversarial networks, are now a key technique for representation
learning of continuous structures. However, applying similar methods to
discrete structures, such as text sequences or discretized images, has proven
to be more challenging. In this work, we propose a flexible method for training
deep latent variable models of discrete structures. Our approach is based on
the recently-proposed Wasserstein autoencoder (WAE) which formalizes the
adversarial autoencoder (AAE) as an optimal transport problem. We first extend
this framework to model discrete sequences, and then further explore different
learned priors targeting a controllable representation. This adversarially
regularized autoencoder (ARAE) allows us to generate natural textual outputs as
well as perform manipulations in the latent space to induce change in the
output space. Finally we show that the latent representation can be trained to
perform unaligned textual style transfer, giving improvements both in
automatic/human evaluation compared to existing methods.
A Fast and Robust TSVM for Pattern Classification
Authors: Bin-Bin Gao, Jian-Jun Wang
Date: 2017-11-15 00:00:00
Tasks: Classification, General Classification, Robust classification

Twin support vector machine~(TSVM) is a powerful learning algorithm by solving a pair of smaller SVM-type problems. However, there are still some specific issues such as low efficiency and weak robustness when it is faced with some real applications. In this paper, we propose a Fast and Robust TSVM~(FR-TSVM) to deal with the above issues. In order to alleviate the effects of noisy inputs, we propose an effective fuzzy membership function and reformulate the TSVMs such that different input instances can make different contributions to the learning of the separating hyperplanes. To further speed up the training procedure, we develop an efficient coordinate descent algorithm with shirking to solve the involved a pair of quadratic programming problems (QPPs). Moreover, theoretical foundations of the proposed model are analyzed in details. The experimental results on several artificial and benchmark datasets indicate that the FR-TSVM not only obtains a fast learning speed but also shows a robust classification performance. Code has been made available at: https://github.com/gaobb/FR-TSVM.
Deep Epitome for Unravelling Generalized Hamming Network: A Fuzzy Logic Interpretation of Deep Learning
Authors: Lixin Fan
Date: 2017-11-15 00:00:00

This paper gives a rigorous analysis of trained Generalized Hamming
Networks(GHN) proposed by Fan (2017) and discloses an interesting finding about
GHNs, i.e., stacked convolution layers in a GHN is equivalent to a single yet
wide convolution layer. The revealed equivalence, on the theoretical side, can
be regarded as a constructive manifestation of the universal approximation
theorem Cybenko(1989); Hornik (1991). In practice, it has profound and
multi-fold implications. For network visualization, the constructed deep
epitomes at each layer provide a visualization of network internal
representation that does not rely on the input data. Moreover, deep epitomes
allows the direct extraction of features in just one step, without resorting to
regularized optimizations used in existing visualization tools.
Rank-1 Constrained Multichannel Wiener Filter for Speech Recognition in Noisy Environments
Authors: Ziteng Wang, Emmanuel Vincent, Romain Serizel, Yonghong Yan
Date: 2017-07-01 00:00:00
Tasks: speech-recognition, Speech Recognition

Multichannel linear filters, such as the Multichannel Wiener Filter (MWF) and
the Generalized Eigenvalue (GEV) beamformer are popular signal processing
techniques which can improve speech recognition performance. In this paper, we
present an experimental study on these linear filters in a specific speech
recognition task, namely the CHiME-4 challenge, which features real recordings
in multiple noisy environments. Specifically, the rank-1 MWF is employed for
noise reduction and a new constant residual noise power constraint is derived
which enhances the recognition performance. To fulfill the underlying rank-1
assumption, the speech covariance matrix is reconstructed based on eigenvectors
or generalized eigenvectors. Then the rank-1 constrained MWF is evaluated with
alternative multichannel linear filters under the same framework, which
involves a Bidirectional Long Short-Term Memory (BLSTM) network for mask
estimation. The proposed filter outperforms alternative ones, leading to a 40%
relative Word Error Rate (WER) reduction compared with the baseline Weighted
Delay and Sum (WDAS) beamformer on the real test set, and a 15% relative WER
reduction compared with the GEV-BAN method. The results also suggest that the
speech recognition accuracy correlates more with the Mel-frequency cepstral
coefficients (MFCC) feature variance than with the noise reduction or the
speech distortion level.
Semiblind subgraph reconstruction in Gaussian graphical models
Authors: Tianpei Xie, Sijia Liu, Alfred O. Hero III
Date: 2017-11-15 00:00:00
Tasks: Attribute

Consider a social network where only a few nodes (agents) have meaningful
interactions in the sense that the conditional dependency graph over node
attribute variables (behaviors) is sparse. A company that can only observe the
interactions between its own customers will generally not be able to accurately
estimate its customers' dependency subgraph: it is blinded to any external
interactions of its customers and this blindness creates false edges in its
subgraph. In this paper we address the semiblind scenario where the company has
access to a noisy summary of the complementary subgraph connecting external
agents, e.g., provided by a consolidator. The proposed framework applies to
other applications as well, including field estimation from a network of awake
and sleeping sensors and privacy-constrained information sharing over social
subnetworks. We propose a penalized likelihood approach in the context of a
graph signal obeying a Gaussian graphical models (GGM). We use a convex-concave
iterative optimization algorithm to maximize the penalized likelihood.
Optimizing Kernel Machines using Deep Learning
Authors: Huan Song, Jayaraman J. Thiagarajan, Prasanna Sattigeri, Andreas Spanias
Date: 2017-11-15 00:00:00
Tasks: Computational Efficiency, Deep Learning

Building highly non-linear and non-parametric models is central to several
state-of-the-art machine learning systems. Kernel methods form an important
class of techniques that induce a reproducing kernel Hilbert space (RKHS) for
inferring non-linear models through the construction of similarity functions
from data. These methods are particularly preferred in cases where the training
data sizes are limited and when prior knowledge of the data similarities is
available. Despite their usefulness, they are limited by the computational
complexity and their inability to support end-to-end learning with a
task-specific objective. On the other hand, deep neural networks have become
the de facto solution for end-to-end inference in several learning paradigms.
In this article, we explore the idea of using deep architectures to perform
kernel machine optimization, for both computational efficiency and end-to-end
inferencing. To this end, we develop the DKMO (Deep Kernel Machine
Optimization) framework, that creates an ensemble of dense embeddings using
Nystrom kernel approximations and utilizes deep learning to generate
task-specific representations through the fusion of the embeddings.
Intuitively, the filters of the network are trained to fuse information from an
ensemble of linear subspaces in the RKHS. Furthermore, we introduce the kernel
dropout regularization to enable improved training convergence. Finally, we
extend this framework to the multiple kernel case, by coupling a global fusion
layer with pre-trained deep kernel machines for each of the constituent
kernels. Using case studies with limited training data, and lack of explicit
feature sources, we demonstrate the effectiveness of our framework over
conventional model inferencing techniques.
A Novel SDASS Descriptor for Fully Encoding the Information of 3D Local Surface
Authors: Bao Zhao, Xinyi Le, Juntong Xi
Date: 2017-11-15 00:00:00
Tasks: Attribute

Local feature description is a fundamental yet challenging task in 3D
computer vision. This paper proposes a novel descriptor, named Statistic of
Deviation Angles on Subdivided Space (SDASS), of encoding geometrical and
spatial information of local surface on Local Reference Axis (LRA). In terms of
encoding geometrical information, considering that surface normals, which are
usually used for encoding geometrical information of local surface, are
vulnerable to various nuisances (e.g., noise, varying mesh resolutions etc.),
we propose a robust geometrical attribute, called Local Minimum Axis (LMA), to
replace the normals for generating the geometrical feature in our SDASS
descriptor. For encoding spatial information, we use two spatial features for
fully encoding the spatial information of a local surface based on LRA which
usually presents high overall repeatability than Local Reference Axis (LRF).
Besides, an improved LRA is proposed for increasing the robustness of our SDASS
to noise and varying mesh resolutions. The performance of the SDASS descriptor
is rigorously tested on four popular datasets. The results show that our
descriptor has a high descriptiveness and strong robustness, and its
performance outperform existing algorithms by a large margin. Finally, the
proposed descriptor is applied to 3D registration. The accurate result further
confirms the effectiveness of our SDASS method.
Velocity variations at Columbia Glacier captured by particle filtering of oblique time-lapse images
Authors: Douglas Brinkerhoff, Shad O'Neel
Date: 2017-11-15 00:00:00

We develop a probabilistic method for tracking glacier surface motion based
on time-lapse imagery, which works by sequentially resampling a stochastic
state-space model according to a likelihood determined through correlation
between reference and test images. The method is robust due to its natural
handling of periodic occlusion and its capacity to follow multiple hypothesis
displacements between images, and can improve estimates of velocity magnitude
and direction through the inclusion of observations from an arbitrary number of
cameras. We apply the method to an annual record of images from two cameras
near the terminus of Columbia Glacier. While the method produces velocities at
daily resolution, we verify our results by comparing eleven-day means to
TerraSar-X. We find that Columbia Glacier transitions between a winter state
characterized by moderate velocities and little temporal variability, to an
early summer speed-up in which velocities are sensitive to increases in melt-
and rainwater, to a fall slowdown, where velocities drop to below their winter
mean and become insensitive to external forcing, a pattern consistent with the
development and collapse of efficient and inefficient subglacial hydrologic
networks throughout the year.