pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Learning of Predictors from Unpaired Input-Output Samples
Authors: Jianshu Chen, Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng
Date: 2016-06-15 00:00:00

Unsupervised learning is the most challenging problem in machine learning and
especially in deep learning. Among many scenarios, we study an unsupervised
learning problem of high economic value --- learning to predict without costly
pairing of input data and corresponding labels. Part of the difficulty in this
problem is a lack of solid evaluation measures. In this paper, we take a
practical approach to grounding unsupervised learning by using the same success
criterion as for supervised learning in prediction tasks but we do not require
the presence of paired input-output training data. In particular, we propose an
objective function that aims to make the predicted outputs fit well the
structure of the output while preserving the correlation between the input and
the predicted output. We experiment with a synthetic structural prediction
problem and show that even with simple linear classifiers, the objective
function is already highly non-convex. We further demonstrate the nature of
this non-convex optimization problem as well as potential solutions. In
particular, we show that with regularization via a generative model, learning
with the proposed unsupervised objective function converges to an optimal
solution.
Siamese CBOW: Optimizing Word Embeddings for Sentence Representations
Authors: Tom Kenter, Alexey Borisov, Maarten de Rijke
Date: 2016-06-15 00:00:00
Proceeding: ACL 2016 8
Tasks: Sentence, Sentence Embeddings, Word Embeddings

We present the Siamese Continuous Bag of Words (Siamese CBOW) model, a neural
network for efficient estimation of high-quality sentence embeddings. Averaging
the embeddings of words in a sentence has proven to be a surprisingly
successful and efficient way of obtaining sentence embeddings. However, word
embeddings trained with the methods currently available are not optimized for
the task of sentence representation, and, thus, likely to be suboptimal.
Siamese CBOW handles this problem by training word embeddings directly for the
purpose of being averaged. The underlying neural network learns word embeddings
by predicting, from a sentence representation, its surrounding sentences. We
show the robustness of the Siamese CBOW model by evaluating it on 20 datasets
stemming from a wide variety of sources.
Bidirectional Long-Short Term Memory for Video Description
Authors: Yi Bin, Yang Yang, Zi Huang, Fumin Shen, Xing Xu, Heng Tao Shen
Date: 2016-06-15 00:00:00
Tasks: Language Modeling, Language Modelling, Video Captioning, Video Description

Video captioning has been attracting broad research attention in multimedia
community. However, most existing approaches either ignore temporal information
among video frames or just employ local contextual temporal knowledge. In this
work, we propose a novel video captioning framework, termed as
\emph{Bidirectional Long-Short Term Memory} (BiLSTM), which deeply captures
bidirectional global temporal structure in video. Specifically, we first devise
a joint visual modelling approach to encode video data by combining a forward
LSTM pass, a backward LSTM pass, together with visual features from
Convolutional Neural Networks (CNNs). Then, we inject the derived video
representation into the subsequent language model for initialization. The
benefits are in two folds: 1) comprehensively preserving sequential and visual
information; and 2) adaptively learning dense visual features and sparse
semantic representations for videos and sentences, respectively. We verify the
effectiveness of our proposed video captioning framework on a commonly-used
benchmark, i.e., Microsoft Video Description (MSVD) corpus, and the
experimental results demonstrate that the superiority of the proposed approach
as compared to several state-of-the-art methods.
Cortical Computation via Iterative Constructions
Authors: Christos Papadimitrou, Samantha Petti, Santosh Vempala
Date: 2016-02-26 00:00:00

We study Boolean functions of an arbitrary number of input variables that can
be realized by simple iterative constructions based on constant-size
primitives. This restricted type of construction needs little global
coordination or control and thus is a candidate for neurally feasible
computation. Valiant's construction of a majority function can be realized in
this manner and, as we show, can be generalized to any uniform threshold
function. We study the rate of convergence, finding that while linear
convergence to the correct function can be achieved for any threshold using a
fixed set of primitives, for quadratic convergence, the size of the primitives
must grow as the threshold approaches 0 or 1. We also study finite realizations
of this process and the learnability of the functions realized. We show that
the constructions realized are accurate outside a small interval near the
target threshold, where the size of the construction grows as the inverse
square of the interval width. This phenomenon, that errors are higher closer to
thresholds (and thresholds closer to the boundary are harder to represent), is
a well-known cognitive finding.
Finite-time Analysis for the Knowledge-Gradient Policy
Authors: Yingfei Wang, Warren Powell
Date: 2016-06-15 00:00:00

We consider sequential decision problems in which we adaptively choose one of
finitely many alternatives and observe a stochastic reward. We offer a new
perspective of interpreting Bayesian ranking and selection problems as adaptive
stochastic multi-set maximization problems and derive the first finite-time
bound of the knowledge-gradient policy for adaptive submodular objective
functions. In addition, we introduce the concept of prior-optimality and
provide another insight into the performance of the knowledge gradient policy
based on the submodular assumption on the value of information. We demonstrate
submodularity for the two-alternative case and provide other conditions for
more general problems, bringing out the issue and importance of submodularity
in learning problems. Empirical experiments are conducted to further illustrate
the finite time behavior of the knowledge gradient policy.
High-speed real-time single-pixel microscopy based on Fourier sampling
Authors: Qiang Guo, Hongwei Chen, Yuxi Wang, Yong Guo, Peng Liu, Xiurui Zhu, Zheng Cheng, Zhenming Yu, Minghua Chen, Sigang Yang, Shizhong Xie
Date: 2016-06-15 00:00:00
Tasks: compressed sensing, Image Reconstruction, Image Restoration, Vocal Bursts Intensity Prediction

Single-pixel cameras based on the concepts of compressed sensing (CS)
leverage the inherent structure of images to retrieve them with far fewer
measurements and operate efficiently over a significantly broader spectral
range than conventional silicon-based cameras. Recently, photonic time-stretch
(PTS) technique facilitates the emergence of high-speed single-pixel cameras. A
significant breakthrough in imaging speed of single-pixel cameras enables
observation of fast dynamic phenomena. However, according to CS theory, image
reconstruction is an iterative process that consumes enormous amounts of
computational time and cannot be performed in real time. To address this
challenge, we propose a novel single-pixel imaging technique that can produce
high-quality images through rapid acquisition of their effective spatial
Fourier spectrum. We employ phase-shifting sinusoidal structured illumination
instead of random illumination for spectrum acquisition and apply inverse
Fourier transform to the obtained spectrum for image restoration. We evaluate
the performance of our prototype system by recognizing quick response (QR)
codes and flow cytometric screening of cells. A frame rate of 625 kHz and a
compression ratio of 10% are experimentally demonstrated in accordance with the
recognition rate of the QR code. An imaging flow cytometer enabling
high-content screening with an unprecedented throughput of 100,000 cells/s is
also demonstrated. For real-time imaging applications, the proposed
single-pixel microscope can significantly reduce the time required for image
reconstruction by two orders of magnitude, which can be widely applied in
industrial quality control and label-free biomedical imaging.
Masking Strategies for Image Manifolds
Authors: Hamid Dadkhahi, Marco F. Duarte
Date: 2016-06-15 00:00:00
Tasks: Compressive Sensing

We consider the problem of selecting an optimal mask for an image manifold,
i.e., choosing a subset of the pixels of the image that preserves the
manifold's geometric structure present in the original data. Such masking
implements a form of compressive sensing through emerging imaging sensor
platforms for which the power expense grows with the number of pixels acquired.
Our goal is for the manifold learned from masked images to resemble its full
image counterpart as closely as possible. More precisely, we show that one can
indeed accurately learn an image manifold without having to consider a large
majority of the image pixels. In doing so, we consider two masking methods that
preserve the local and global geometric structure of the manifold,
respectively. In each case, the process of finding the optimal masking pattern
can be cast as a binary integer program, which is computationally expensive but
can be approximated by a fast greedy algorithm. Numerical experiments show that
the relevant manifold structure is preserved through the data-dependent masking
process, even for modest mask sizes.
Natural Scene Character Recognition Using Robust PCA and Sparse Representation
Authors: Zheng Zhang, Yong Xu, Cheng-Lin Liu
Date: 2016-06-15 00:00:00

Natural scene character recognition is challenging due to the cluttered
background, which is hard to separate from text. In this paper, we propose a
novel method for robust scene character recognition. Specifically, we first use
robust principal component analysis (PCA) to denoise character image by
recovering the missing low-rank component and filtering out the sparse noise
term, and then use a simple Histogram of oriented Gradient (HOG) to perform
image feature extraction, and finally, use a sparse representation based
classifier for recognition. In experiments on four public datasets, namely the
Char74K dataset, ICADAR 2003 robust reading dataset, Street View Text (SVT)
dataset and IIIT5K-word dataset, our method was demonstrated to be competitive
with the state-of-the-art methods.
Deep Reinforcement Learning With Macro-Actions
Authors: Ishan P. Durugkar, Clemens Rosenbaum, Stefan Dernbach, Sridhar Mahadevan
Date: 2016-06-15 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning has been shown to be a powerful framework for
learning policies from complex high-dimensional sensory inputs to actions in
complex tasks, such as the Atari domain. In this paper, we explore output
representation modeling in the form of temporal abstraction to improve
convergence and reliability of deep reinforcement learning approaches. We
concentrate on macro-actions, and evaluate these on different Atari 2600 games,
where we show that they yield significant improvements in learning speed.
Additionally, we show that they can even achieve better scores than DQN. We
offer analysis and explanation for both convergence and final results,
revealing a problem deep RL approaches have with sparse reward signals.
A Reconfigurable Low Power High Throughput Architecture for Deep Network Training
Authors: Raqibul Hasan, Tarek Taha
Date: 2016-03-24 00:00:00
Tasks: Anomaly Detection, Dimensionality Reduction, Vocal Bursts Intensity Prediction

General purpose computing systems are used for a large variety of
applications. Extensive supports for flexibility in these systems limit their
energy efficiencies. Neural networks, including deep networks, are widely used
for signal processing and pattern recognition applications. In this paper we
propose a multicore architecture for deep neural network based processing.
Memristor crossbars are utilized to provide low power high throughput execution
of neural networks. The system has both training and recognition (evaluation of
new input) capabilities. The proposed system could be used for classification,
dimensionality reduction, feature extraction, and anomaly detection
applications. The system level area and power benefits of the specialized
architecture is compared with the NVIDIA Telsa K20 GPGPU. Our experimental
evaluations show that the proposed architecture can provide up to five orders
of magnitude more energy efficiency over GPGPUs for deep neural network
processing.
Agreement-based Learning of Parallel Lexicons and Phrases from Non-Parallel Corpora
Authors: Chunyang Liu, Yang Liu, Huanbo Luan, Maosong Sun, Heng Yu
Date: 2016-06-15 00:00:00
Proceeding: ACL 2016 8
Tasks: Translation

We introduce an agreement-based approach to learning parallel lexicons and
phrases from non-parallel corpora. The basic idea is to encourage two
asymmetric latent-variable translation models (i.e., source-to-target and
target-to-source) to agree on identifying latent phrase and word alignments.
The agreement is defined at both word and phrase levels. We develop a Viterbi
EM algorithm for jointly training the two unidirectional models efficiently.
Experiments on the Chinese-English dataset show that agreement-based learning
significantly improves both alignment and translation performance.
Achieving Exact Cluster Recovery Threshold via Semidefinite Programming: Extensions
Authors: Bruce Hajek, Yihong Wu, Jiaming Xu
Date: 2015-02-26 00:00:00
Tasks: Community Detection, Stochastic Block Model

Resolving a conjecture of Abbe, Bandeira and Hall, the authors have recently
shown that the semidefinite programming (SDP) relaxation of the maximum
likelihood estimator achieves the sharp threshold for exactly recovering the
community structure under the binary stochastic block model of two equal-sized
clusters. The same was shown for the case of a single cluster and outliers.
Extending the proof techniques, in this paper it is shown that SDP relaxations
also achieve the sharp recovery threshold in the following cases: (1) Binary
stochastic block model with two clusters of sizes proportional to network size
but not necessarily equal; (2) Stochastic block model with a fixed number of
equal-sized clusters; (3) Binary censored block model with the background graph
being Erd\H{o}s-R\'enyi. Furthermore, a sufficient condition is given for an
SDP procedure to achieve exact recovery for the general case of a fixed number
of clusters plus outliers. These results demonstrate the versatility of SDP
relaxation as a simple, general purpose, computationally feasible methodology
for community detection.
Minimum Risk Training for Neural Machine Translation
Authors: Shiqi Shen, Yong Cheng, Zhongjun He, wei he, Hua Wu, Maosong Sun, Yang Liu
Date: 2015-12-08 00:00:00
Proceeding: ACL 2016 8
Tasks: Machine Translation, Translation

We propose minimum risk training for end-to-end neural machine translation.
Unlike conventional maximum likelihood estimation, minimum risk training is
capable of optimizing model parameters directly with respect to arbitrary
evaluation metrics, which are not necessarily differentiable. Experiments show
that our approach achieves significant improvements over maximum likelihood
estimation on a state-of-the-art neural machine translation system across
various languages pairs. Transparent to architectures, our approach can be
applied to more neural networks and potentially benefit more NLP tasks.
In the Shadows, Shape Priors Shine: Using Occlusion to Improve Multi-Region Segmentation
Authors: Yuka Kihara, Matvey Soloviev, Tsuhan Chen
Date: 2016-06-14 00:00:00
Proceeding: CVPR 2016 6
Tasks: Segmentation

We present a new algorithm for multi-region segmentation of 2D images with
objects that may partially occlude each other. Our algorithm is based on the
observation hat human performance on this task is based both on prior knowledge
about plausible shapes and taking into account the presence of occluding
objects whose shape is already known - once an occluded region is identified,
the shape prior can be used to guess the shape of the missing part. We capture
the former aspect using a deep learning model of shape; for the latter, we
simultaneously minimize the energy of all regions and consider only unoccluded
pixels for data agreement.
  Existing algorithms incorporating object shape priors consider every object
separately in turn and can't distinguish genuine deviation from the expected
shape from parts missing due to occlusion. We show that our method
significantly improves on the performance of a representative algorithm, as
evaluated on both preprocessed natural and synthetic images. Furthermore, on
the synthetic images, we recover the ground truth segmentation with good
accuracy.
Impossibility in Belief Merging
Authors: Amílcar Mata Díaz, Ramón Pino Pérez
Date: 2016-06-14 00:00:00

With the aim of studying social properties of belief merging and having a
better understanding of impossibility, we extend in three ways the framework of
logic-based merging introduced by Konieczny and Pino P\'erez. First, at the
level of representation of the information, we pass from belief bases to
complex epistemic states. Second, the profiles are represented as functions of
finite societies to the set of epistemic states (a sort of vectors) and not as
multisets of epistemic states. Third, we extend the set of rational postulates
in order to consider the epistemic versions of the classical postulates of
Social Choice Theory: Standard Domain, Pareto Property, Independence of
Irrelevant Alternatives and Absence of Dictator. These epistemic versions of
social postulates are given, essentially, in terms of the finite propositional
logic. We state some representation theorems for these operators. These
extensions and representation theorems allow us to establish an epistemic and
very general version of Arrow's Impossibility Theorem. One of the interesting
features of our result, is that it holds for different representations of
epistemic states; for instance conditionals, Ordinal Conditional functions and,
of course, total preorders.
Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
Authors: Mehdi Sajjadi, Mehran Javanmardi, Tolga Tasdizen
Date: 2016-06-14 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Data Augmentation, Semi-Supervised Image Classification

Effective convolutional neural networks are trained on large sets of labeled
data. However, creating large labeled datasets is a very costly and
time-consuming task. Semi-supervised learning uses unlabeled data to train a
model with higher accuracy when there is a limited set of labeled data
available. In this paper, we consider the problem of semi-supervised learning
with convolutional neural networks. Techniques such as randomized data
augmentation, dropout and random max-pooling provide better generalization and
stability for classifiers that are trained using gradient descent. Multiple
passes of an individual sample through the network might lead to different
predictions due to the non-deterministic behavior of these techniques. We
propose an unsupervised loss function that takes advantage of the stochastic
nature of these methods and minimizes the difference between the predictions of
multiple passes of a training sample through the network. We evaluate the
proposed method on several benchmark datasets.
A New Approach to Dimensionality Reduction for Anomaly Detection in Data Traffic
Authors: Tingshan Huang, Harish Sethu, Nagarajan Kandasamy
Date: 2016-06-14 00:00:00
Tasks: Anomaly Detection, Dimensionality Reduction, Management

The monitoring and management of high-volume feature-rich traffic in large
networks offers significant challenges in storage, transmission and
computational costs. The predominant approach to reducing these costs is based
on performing a linear mapping of the data to a low-dimensional subspace such
that a certain large percentage of the variance in the data is preserved in the
low-dimensional representation. This variance-based subspace approach to
dimensionality reduction forces a fixed choice of the number of dimensions, is
not responsive to real-time shifts in observed traffic patterns, and is
vulnerable to normal traffic spoofing. Based on theoretical insights proved in
this paper, we propose a new distance-based approach to dimensionality
reduction motivated by the fact that the real-time structural differences
between the covariance matrices of the observed and the normal traffic is more
relevant to anomaly detection than the structure of the training data alone.
Our approach, called the distance-based subspace method, allows a different
number of reduced dimensions in different time windows and arrives at only the
number of dimensions necessary for effective anomaly detection. We present
centralized and distributed versions of our algorithm and, using simulation on
real traffic traces, demonstrate the qualitative and quantitative advantages of
the distance-based subspace approach.
Training variance and performance evaluation of neural networks in speech
Authors: Ewout van den Berg, Bhuvana Ramabhadran, Michael Picheny
Date: 2016-06-14 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

In this work we study variance in the results of neural network training on a
wide variety of configurations in automatic speech recognition. Although this
variance itself is well known, this is, to the best of our knowledge, the first
paper that performs an extensive empirical study on its effects in speech
recognition. We view training as sampling from a distribution and show that
these distributions can have a substantial variance. These results show the
urgent need to rethink the way in which results in the literature are reported
and interpreted.
Sparsely Connected and Disjointly Trained Deep Neural Networks for Low Resource Behavioral Annotation: Acoustic Classification in Couples' Therapy
Authors: Haoqi Li, Brian Baucom, Panayiotis Georgiou
Date: 2016-06-14 00:00:00
Tasks: General Classification

Observational studies are based on accurate assessment of human state. A
behavior recognition system that models interlocutors' state in real-time can
significantly aid the mental health domain. However, behavior recognition from
speech remains a challenging task since it is difficult to find generalizable
and representative features because of noisy and high-dimensional data,
especially when data is limited and annotated coarsely and subjectively. Deep
Neural Networks (DNN) have shown promise in a wide range of machine learning
tasks, but for Behavioral Signal Processing (BSP) tasks their application has
been constrained due to limited quantity of data. We propose a
Sparsely-Connected and Disjointly-Trained DNN (SD-DNN) framework to deal with
limited data. First, we break the acoustic feature set into subsets and train
multiple distinct classifiers. Then, the hidden layers of these classifiers
become parts of a deeper network that integrates all feature streams. The
overall system allows for full connectivity while limiting the number of
parameters trained at any time and allows convergence possible with even
limited data. We present results on multiple behavior codes in the couples'
therapy domain and demonstrate the benefits in behavior classification
accuracy. We also show the viability of this system towards live behavior
annotations.