pwc-archive/papers-with-abstracts huggingface dataset:
Asynchronous Parallel Empirical Variance Guided Algorithms for the Thresholding Bandit Problem
Authors: Jie Zhong, Yijun Huang, Ji Liu
Date: 2017-04-15 00:00:00

This paper considers the multi-armed thresholding bandit problem --
identifying all arms whose expected rewards are above a predefined threshold
via as few pulls (or rounds) as possible -- proposed by Locatelli et al. [2016]
recently. Although the proposed algorithm in Locatelli et al. [2016] achieves
the optimal round complexity in a certain sense, there still remain unsolved
issues. This paper proposes an asynchronous parallel thresholding algorithm and
its parameter-free version to improve the efficiency and the applicability. On
one hand, the proposed two algorithms use the empirical variance to guide the
pull decision at each round, and significantly improve the round complexity of
the "optimal" algorithm when all arms have bounded high order moments. The
proposed algorithms can be proven to be optimal. On the other hand, most bandit
algorithms assume that the reward can be observed immediately after the pull or
the next decision would not be made before all rewards are observed. Our
proposed asynchronous parallel algorithms allow making the choice of the next
pull with unobserved rewards from earlier pulls, which avoids such an
unrealistic assumption and significantly improves the identification process.
Our theoretical analysis justifies the effectiveness and the efficiency of
proposed asynchronous parallel algorithms.
Deep Learning: Generalization Requires Deep Compositional Feature Space Design
Authors: Mrinal Haloi
Date: 2017-06-06 00:00:00
Tasks: Deep Learning, General Classification, image-classification, Image Classification

Generalization error defines the discriminability and the representation
power of a deep model. In this work, we claim that feature space design using
deep compositional function plays a significant role in generalization along
with explicit and implicit regularizations. Our claims are being established
with several image classification experiments. We show that the information
loss due to convolution and max pooling can be marginalized with the
compositional design, improving generalization performance. Also, we will show
that learning rate decay acts as an implicit regularizer in deep model
training.
Predicting the Quality of Short Narratives from Social Media
Authors: Tong Wang, Ping Chen, Boyang Li
Date: 2017-07-08 00:00:00
Tasks: Active Learning

An important and difficult challenge in building computational models for
narratives is the automatic evaluation of narrative quality. Quality evaluation
connects narrative understanding and generation as generation systems need to
evaluate their own products. To circumvent difficulties in acquiring
annotations, we employ upvotes in social media as an approximate measure for
story quality. We collected 54,484 answers from a crowd-powered
question-and-answer website, Quora, and then used active learning to build a
classifier that labeled 28,320 answers as stories. To predict the number of
upvotes without the use of social network features, we create neural networks
that model textual regions and the interdependence among regions, which serve
as strong benchmarks for future research. To our best knowledge, this is the
first large-scale study for automatic evaluation of narrative quality.
MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network
Authors: Zizhao Zhang, Yuanpu Xie, Fuyong Xing, Mason McGough, Lin Yang
Date: 2017-07-08 00:00:00
Proceeding: CVPR 2017 7
Tasks: Diagnostic, Language Modeling, Language Modelling, Sentence

The inability to interpret the model prediction in semantically and visually
meaningful ways is a well-known shortcoming of most existing computer-aided
diagnosis methods. In this paper, we propose MDNet to establish a direct
multimodal mapping between medical images and diagnostic reports that can read
images, generate diagnostic reports, retrieve images by symptom descriptions,
and visualize attention, to provide justifications of the network diagnosis
process. MDNet includes an image model and a language model. The image model is
proposed to enhance multi-scale feature ensembles and utilization efficiency.
The language model, integrated with our improved attention mechanism, aims to
read and explore discriminative image feature descriptions from reports to
learn a direct mapping from sentence words to image pixels. The overall network
is trained end-to-end by using our developed optimization strategy. Based on a
pathology bladder cancer images and its diagnostic reports (BCIDR) dataset, we
conduct sufficient experiments to demonstrate that MDNet outperforms
comparative baselines. The proposed image model obtains state-of-the-art
performance on two CIFAR datasets as well.
Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projection
Authors: Jian Ni, Georgiana Dinu, Radu Florian
Date: 2017-07-08 00:00:00
Proceeding: ACL 2017 7
Tasks: Cross-Lingual NER, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Word Embeddings

The state-of-the-art named entity recognition (NER) systems are supervised
machine learning models that require large amounts of manually annotated data
to achieve high accuracy. However, annotating NER data by human is expensive
and time-consuming, and can be quite difficult for a new language. In this
paper, we present two weakly supervised approaches for cross-lingual NER with
no human annotation in a target language. The first approach is to create
automatically labeled NER data for a target language via annotation projection
on comparable corpora, where we develop a heuristic scheme that effectively
selects good-quality projection-labeled data from noisy data. The second
approach is to project distributed representations of words (word embeddings)
from a target language to a source language, so that the source-language NER
system can be applied to the target language without re-training. We also
design two co-decoding schemes that effectively combine the outputs of the two
projection-based approaches. We evaluate the performance of the proposed
approaches on both in-house and open NER data for several target languages. The
results show that the combined systems outperform three other weakly supervised
approaches on the CoNLL data.
A Reconfigurable Streaming Deep Convolutional Neural Network Accelerator for Internet of Things
Authors: Li Du, Yuan Du, Yilei Li, Mau-Chung Frank Chang
Date: 2017-07-08 00:00:00

Convolutional neural network (CNN) offers significant accuracy in image
detection. To implement image detection using CNN in the internet of things
(IoT) devices, a streaming hardware accelerator is proposed. The proposed
accelerator optimizes the energy efficiency by avoiding unnecessary data
movement. With unique filter decomposition technique, the accelerator can
support arbitrary convolution window size. In addition, max pooling function
can be computed in parallel with convolution by using separate pooling unit,
thus achieving throughput improvement. A prototype accelerator was implemented
in TSMC 65nm technology with a core size of 5mm2. The accelerator can support
major CNNs and achieve 152GOPS peak throughput and 434GOPS/W energy efficiency
at 350mW, making it a promising hardware accelerator for intelligent IoT
devices.
Hyperspectral Image Restoration via Total Variation Regularized Low-rank Tensor Decomposition
Authors: Yao Wang, Jiangjun Peng, Qian Zhao, Deyu Meng, Yee Leung, Xi-Le Zhao
Date: 2017-07-08 00:00:00
Tasks: Image Restoration, Tensor Decomposition

Hyperspectral images (HSIs) are often corrupted by a mixture of several types
of noise during the acquisition process, e.g., Gaussian noise, impulse noise,
dead lines, stripes, and many others. Such complex noise could degrade the
quality of the acquired HSIs, limiting the precision of the subsequent
processing. In this paper, we present a novel tensor-based HSI restoration
approach by fully identifying the intrinsic structures of the clean HSI part
and the mixed noise part respectively. Specifically, for the clean HSI part, we
use tensor Tucker decomposition to describe the global correlation among all
bands, and an anisotropic spatial-spectral total variation (SSTV)
regularization to characterize the piecewise smooth structure in both spatial
and spectral domains. For the mixed noise part, we adopt the $\ell_1$ norm
regularization to detect the sparse noise, including stripes, impulse noise,
and dead pixels. Despite that TV regulariztion has the ability of removing
Gaussian noise, the Frobenius norm term is further used to model heavy Gaussian
noise for some real-world scenarios. Then, we develop an efficient algorithm
for solving the resulting optimization problem by using the augmented Lagrange
multiplier (ALM) method. Finally, extensive experiments on simulated and
real-world noise HSIs are carried out to demonstrate the superiority of the
proposed method over the existing state-of-the-art ones.
Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
Authors: John Bradshaw, Alexander G. de G. Matthews, Zoubin Ghahramani
Date: 2017-07-08 00:00:00
Tasks: Gaussian Processes

Deep neural networks (DNNs) have excellent representative power and are state
of the art classifiers on many tasks. However, they often do not capture their
own uncertainties well making them less robust in the real world as they
overconfidently extrapolate and do not notice domain shift. Gaussian processes
(GPs) with RBF kernels on the other hand have better calibrated uncertainties
and do not overconfidently extrapolate far from data in their training set.
However, GPs have poor representational power and do not perform as well as
DNNs on complex domains. In this paper we show that GP hybrid deep networks,
GPDNNs, (GPs on top of DNNs and trained end-to-end) inherit the nice properties
of both GPs and DNNs and are much more robust to adversarial examples. When
extrapolating to adversarial examples and testing in domain shift settings,
GPDNNs frequently output high entropy class probabilities corresponding to
essentially "don't know". GPDNNs are therefore promising as deep architectures
that know when they don't know.
Tailoring Artificial Neural Networks for Optimal Learning
Authors: Pau Vilimelis Aceituno, Yan Gang, Yang-Yu Liu
Date: 2017-07-08 00:00:00
Tasks: Time Series, Time Series Analysis

As one of the most important paradigms of recurrent neural networks, the echo state network (ESN) has been applied to a wide range of fields, from robotics to medicine, finance, and language processing. A key feature of the ESN paradigm is its reservoir --- a directed and weighted network of neurons that projects the input time series into a high dimensional space where linear regression or classification can be applied. Despite extensive studies, the impact of the reservoir network on the ESN performance remains unclear. Combining tools from physics, dynamical systems and network science, we attempt to open the black box of ESN and offer insights to understand the behavior of general artificial neural networks. Through spectral analysis of the reservoir network we reveal a key factor that largely determines the ESN memory capacity and hence affects its performance. Moreover, we find that adding short loops to the reservoir network can tailor ESN for specific tasks and optimize learning. We validate our findings by applying ESN to forecast both synthetic and real benchmark time series. Our results provide a new way to design task-specific ESN. More importantly, it demonstrates the power of combining tools from physics, dynamical systems and network science to offer new insights in understanding the mechanisms of general artificial neural networks.
Holistic Planimetric prediction to Local Volumetric prediction for 3D Human Pose Estimation
Authors: Gyeongsik Moon, Ju Yong Chang, Yumin Suh, Kyoung Mu Lee
Date: 2017-06-15 00:00:00
Tasks: 3D Human Pose Estimation, Pose Estimation, Prediction

We propose a novel approach to 3D human pose estimation from a single depth
map. Recently, convolutional neural network (CNN) has become a powerful
paradigm in computer vision. Many of computer vision tasks have benefited from
CNNs, however, the conventional approach to directly regress 3D body joint
locations from an image does not yield a noticeably improved performance. In
contrast, we formulate the problem as estimating per-voxel likelihood of key
body joints from a 3D occupancy grid. We argue that learning a mapping from
volumetric input to volumetric output with 3D convolution consistently improves
the accuracy when compared to learning a regression from depth map to 3D joint
coordinates. We propose a two-stage approach to reduce the computational
overhead caused by volumetric representation and 3D convolution: Holistic 2D
prediction and Local 3D prediction. In the first stage, Planimetric Network
(P-Net) estimates per-pixel likelihood for each body joint in the holistic 2D
space. In the second stage, Volumetric Network (V-Net) estimates the per-voxel
likelihood of each body joints in the local 3D space around the 2D estimations
of the first stage, effectively reducing the computational cost. Our model
outperforms existing methods by a large margin in publicly available datasets.
Improving Multilingual Named Entity Recognition with Wikipedia Entity Type Mapping
Authors: Jian Ni, Radu Florian
Date: 2017-07-08 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Multilingual Named Entity Recognition, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Vocal Bursts Type Prediction

The state-of-the-art named entity recognition (NER) systems are statistical
machine learning models that have strong generalization capability (i.e., can
recognize unseen entities that do not appear in training data) based on lexical
and contextual information. However, such a model could still make mistakes if
its features favor a wrong entity type. In this paper, we utilize Wikipedia as
an open knowledge base to improve multilingual NER systems. Central to our
approach is the construction of high-accuracy, high-coverage multilingual
Wikipedia entity type mappings. These mappings are built from weakly annotated
data and can be extended to new languages with no human annotation or
language-dependent knowledge involved. Based on these mappings, we develop
several approaches to improve an NER system. We evaluate the performance of the
approaches via experiments on NER systems trained for 6 languages. Experimental
results show that the proposed approaches are effective in improving the
accuracy of such systems on unseen entities, especially when a system is
applied to a new domain or it is trained with little training data (up to 18.3
F1 score improvement).
Deep Person Re-Identification with Improved Embedding and Efficient Training
Authors: Haibo Jin, Xiaobo Wang, Shengcai Liao, Stan Z. Li
Date: 2017-05-09 00:00:00
Tasks: Person Re-Identification

Person re-identification task has been greatly boosted by deep convolutional
neural networks (CNNs) in recent years. The core of which is to enlarge the
inter-class distinction as well as reduce the intra-class variance. However, to
achieve this, existing deep models prefer to adopt image pairs or triplets to
form verification loss, which is inefficient and unstable since the number of
training pairs or triplets grows rapidly as the number of training data grows.
Moreover, their performance is limited since they ignore the fact that
different dimension of embedding may play different importance. In this paper,
we propose to employ identification loss with center loss to train a deep model
for person re-identification. The training process is efficient since it does
not require image pairs or triplets for training while the inter-class
distinction and intra-class variance are well handled. To boost the
performance, a new feature reweighting (FRW) layer is designed to explicitly
emphasize the importance of each embedding dimension, thus leading to an
improved embedding. Experiments on several benchmark datasets have shown the
superiority of our method over the state-of-the-art alternatives on both
accuracy and speed.
Estimating network edge probabilities by neighborhood smoothing
Authors: Yuan Zhang, Elizaveta Levina, Ji Zhu
Date: 2015-09-29 00:00:00
Tasks: Denoising, Graphon Estimation, Link Prediction

The estimation of probabilities of network edges from the observed adjacency
matrix has important applications to predicting missing links and network
denoising. It has usually been addressed by estimating the graphon, a function
that determines the matrix of edge probabilities, but this is ill-defined
without strong assumptions on the network structure. Here we propose a novel
computationally efficient method, based on neighborhood smoothing to estimate
the expectation of the adjacency matrix directly, without making the structural
assumptions that graphon estimation requires. The neighborhood smoothing method
requires little tuning, has a competitive mean-squared error rate, and
outperforms many benchmark methods on link prediction in simulated and real
networks.
Deep Learning for Environmentally Robust Speech Recognition: An Overview of Recent Developments
Authors: Zixing Zhang, Jürgen Geiger, Jouni Pohjalainen, Amr El-Desoky Mousa, Wenyu Jin, Björn Schuller
Date: 2017-05-30 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Robust Speech Recognition, speech-recognition, Speech Recognition

Eliminating the negative effect of non-stationary environmental noise is a
long-standing research topic for automatic speech recognition that stills
remains an important challenge. Data-driven supervised approaches, including
ones based on deep neural networks, have recently emerged as potential
alternatives to traditional unsupervised approaches and with sufficient
training, can alleviate the shortcomings of the unsupervised methods in various
real-life acoustic environments. In this light, we review recently developed,
representative deep learning approaches for tackling non-stationary additive
and convolutional degradation of speech with the aim of providing guidelines
for those involved in the development of environmentally robust speech
recognition systems. We separately discuss single- and multi-channel techniques
developed for the front-end and back-end of speech recognition systems, as well
as joint front-end and back-end training frameworks.
Application of Transfer Learning Approaches in Multimodal Wearable Human Activity Recognition
Authors: Hailin Chen, Shengping Cui, Sebastian Li
Date: 2017-07-08 00:00:00
Tasks: Activity Recognition, Ensemble Learning, Human Activity Recognition, Transfer Learning

Through this project, we researched on transfer learning methods and their
applications on real world problems. By implementing and modifying various
methods in transfer learning for our problem, we obtained an insight in the
advantages and disadvantages of these methods, as well as experiences in
developing neural network models for knowledge transfer. Due to time
constraint, we only applied a representative method for each major approach in
transfer learning. As pointed out in the literature review, each method has its
own assumptions, strengths and shortcomings. Thus we believe that an
ensemble-learning approach combining the different methods should yield a
better performance, which can be our future research focus.
Embedding Visual Hierarchy with Deep Networks for Large-Scale Visual Recognition
Authors: Tianyi Zhao, Baopeng Zhang, Wei zhang, Ning Zhou, Jun Yu, Jianping Fan
Date: 2017-07-08 00:00:00
Tasks: Object, Object Recognition

In this paper, a level-wise mixture model (LMM) is developed by embedding
visual hierarchy with deep networks to support large-scale visual recognition
(i.e., recognizing thousands or even tens of thousands of object classes), and
a Bayesian approach is used to adapt a pre-trained visual hierarchy
automatically to the improvements of deep features (that are used for image and
object class representation) when more representative deep networks are learned
along the time. Our LMM model can provide an end-to-end approach for jointly
learning: (a) the deep networks to extract more discriminative deep features
for image and object class representation; (b) the tree classifier for
recognizing large numbers of object classes hierarchically; and (c) the visual
hierarchy adaptation for achieving more accurate indexing of large numbers of
object classes hierarchically. By supporting joint learning of the tree
classifier, the deep networks and the visual hierarchy adaptation, our LMM
algorithm can provide an effective approach for controlling inter-level error
propagation effectively, thus it can achieve better accuracy rates on
large-scale visual recognition. Our experiments are carried on ImageNet1K and
ImageNet10K image sets, and our LMM algorithm can achieve very competitive
results on both the accuracy rates and the computation efficiency as compared
with the baseline methods.
Single-Queue Decoding for Neural Machine Translation
Authors: Raphael Shu, Hideki Nakayama
Date: 2017-07-06 00:00:00
Tasks: Machine Translation, Translation

Neural machine translation models rely on the beam search algorithm for
decoding. In practice, we found that the quality of hypotheses in the search
space is negatively affected owing to the fixed beam size. To mitigate this
problem, we store all hypotheses in a single priority queue and use a universal
score function for hypothesis selection. The proposed algorithm is more
flexible as the discarded hypotheses can be revisited in a later step. We
further design a penalty function to punish the hypotheses that tend to produce
a final translation that is much longer or shorter than expected. Despite its
simplicity, we show that the proposed decoding algorithm is able to select
hypotheses with better qualities and improve the translation performance.
Effective Approaches to Batch Parallelization for Dynamic Neural Network Architectures
Authors: Joseph Suarez, Clare Zhu
Date: 2017-07-08 00:00:00
Tasks: Mixture-of-Experts, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

We present a simple dynamic batching approach applicable to a large class of
dynamic architectures that consistently yields speedups of over 10x. We provide
performance bounds when the architecture is not known a priori and a stronger
bound in the special case where the architecture is a predetermined balanced
tree. We evaluate our approach on Johnson et al.'s recent visual question
answering (VQA) result of his CLEVR dataset by Inferring and Executing Programs
(IEP). We also evaluate on sparsely gated mixture of experts layers and achieve
speedups of up to 1000x over the naive implementation.