pwc-archive/papers-with-abstracts huggingface dataset:
Distance-based Self-Attention Network for Natural Language Inference
Authors: Jinbae Im, Sungzoon Cho
Date: 2017-12-06 00:00:00
Tasks: Machine Translation, Natural Language Inference, Sentence, Translation

Attention mechanism has been used as an ancillary means to help RNN or CNN.
However, the Transformer (Vaswani et al., 2017) recently recorded the
state-of-the-art performance in machine translation with a dramatic reduction
in training time by solely using attention. Motivated by the Transformer,
Directional Self Attention Network (Shen et al., 2017), a fully attention-based
sentence encoder, was proposed. It showed good performance with various data by
using forward and backward directional information in a sentence. But in their
study, not considered at all was the distance between words, an important
feature when learning the local dependency to help understand the context of
input text. We propose Distance-based Self-Attention Network, which considers
the word distance by using a simple distance mask in order to model the local
dependency without losing the ability of modeling global dependency which
attention has inherent. Our model shows good performance with NLI data, and it
records the new state-of-the-art result with SNLI data. Additionally, we show
that our model has a strength in long sentences or documents.
Learning General Latent-Variable Graphical Models with Predictive Belief Propagation
Authors: Borui Wang, Geoffrey Gordon
Date: 2017-12-06 00:00:00

Learning general latent-variable probabilistic graphical models is a key theoretical challenge in machine learning and artificial intelligence. All previous methods, including the EM algorithm and the spectral algorithms, face severe limitations that largely restrict their applicability and affect their performance. In order to overcome these limitations, in this paper we introduce a novel formulation of message-passing inference over junction trees named predictive belief propagation, and propose a new learning and inference algorithm for general latent-variable graphical models based on this formulation. Our proposed algorithm reduces the hard parameter learning problem into a sequence of supervised learning problems, and unifies the learning of different kinds of latent graphical models into a single learning framework, which is local-optima-free and statistically consistent. We then give a proof of the correctness of our algorithm and show in experiments on both synthetic and real datasets that our algorithm significantly outperforms both the EM algorithm and the spectral algorithm while also being orders of magnitude faster to compute.
S-Shaped vs. V-Shaped Transfer Functions for Antlion Optimization Algorithm in Feature Selection Problems
Authors: Majdi Mafarja, Seyedali Mirjalili
Date: 2017-12-06 00:00:00
Tasks: feature selection, General Classification

Feature selection is an important preprocessing step for classification
problems. It deals with selecting near optimal features in the original
dataset. Feature selection is an NP-hard problem, so meta-heuristics can be
more efficient than exact methods. In this work, Ant Lion Optimizer (ALO),
which is a recent metaheuristic algorithm, is employed as a wrapper feature
selection method. Six variants of ALO are proposed where each employ a transfer
function to map a continuous search space to a discrete search space. The
performance of the proposed approaches is tested on eighteen UCI datasets and
compared to a number of existing approaches in the literature: Particle Swarm
Optimization, Gravitational Search Algorithm, and two existing ALO-based
approaches. Computational experiments show that the proposed approaches
efficiently explore the feature space and select the most informative features,
which help to improve the classification accuracy.
Bayesian Policy Gradients via Alpha Divergence Dropout Inference
Authors: Peter Henderson, Thang Doan, Riashat Islam, David Meger
Date: 2017-12-06 00:00:00
Tasks: continuous-control, Continuous Control, MuJoCo, Policy Gradient Methods

Policy gradient methods have had great success in solving continuous control
tasks, yet the stochastic nature of such problems makes deterministic value
estimation difficult. We propose an approach which instead estimates a
distribution by fitting the value function with a Bayesian Neural Network. We
optimize an $\alpha$-divergence objective with Bayesian dropout approximation
to learn and estimate this distribution. We show that using the Monte Carlo
posterior mean of the Bayesian value function distribution, rather than a
deterministic network, improves stability and performance of policy gradient
methods in continuous control MuJoCo simulations.
Learning Semantic Concepts and Order for Image and Sentence Matching
Authors: Yan Huang, Qi Wu, Liang Wang
Date: 2017-12-06 00:00:00
Proceeding: CVPR 2018 6
Tasks: Cross-Modal Retrieval, Sentence

Image and sentence matching has made great progress recently, but it remains
challenging due to the large visual-semantic discrepancy. This mainly arises
from that the representation of pixel-level image usually lacks of high-level
semantic information as in its matched sentence. In this work, we propose a
semantic-enhanced image and sentence matching model, which can improve the
image representation by learning semantic concepts and then organizing them in
a correct semantic order. Given an image, we first use a multi-regional
multi-label CNN to predict its semantic concepts, including objects,
properties, actions, etc. Then, considering that different orders of semantic
concepts lead to diverse semantic meanings, we use a context-gated sentence
generation scheme for semantic order learning. It simultaneously uses the image
global context containing concept relations as reference and the groundtruth
semantic order in the matched sentence as supervision. After obtaining the
improved image representation, we learn the sentence representation with a
conventional LSTM, and then jointly perform image and sentence matching and
sentence generation for model learning. Extensive experiments demonstrate the
effectiveness of our learned semantic concepts and order, by achieving the
state-of-the-art results on two public benchmark datasets.
AttGAN: Facial Attribute Editing by Only Changing What You Want
Authors: Zhenliang He, WangMeng Zuo, Meina Kan, Shiguang Shan, Xilin Chen
Date: 2017-11-29 00:00:00
Tasks: Attribute, Decoder

Facial attribute editing aims to manipulate single or multiple attributes of
a face image, i.e., to generate a new face with desired attributes while
preserving other details. Recently, generative adversarial net (GAN) and
encoder-decoder architecture are usually incorporated to handle this task with
promising results. Based on the encoder-decoder architecture, facial attribute
editing is achieved by decoding the latent representation of the given face
conditioned on the desired attributes. Some existing methods attempt to
establish an attribute-independent latent representation for further attribute
editing. However, such attribute-independent constraint on the latent
representation is excessive because it restricts the capacity of the latent
representation and may result in information loss, leading to over-smooth and
distorted generation. Instead of imposing constraints on the latent
representation, in this work we apply an attribute classification constraint to
the generated image to just guarantee the correct change of desired attributes,
i.e., to "change what you want". Meanwhile, the reconstruction learning is
introduced to preserve attribute-excluding details, in other words, to "only
change what you want". Besides, the adversarial learning is employed for
visually realistic editing. These three components cooperate with each other
forming an effective framework for high quality facial attribute editing,
referred as AttGAN. Furthermore, our method is also directly applicable for
attribute intensity control and can be naturally extended for attribute style
manipulation. Experiments on CelebA dataset show that our method outperforms
the state-of-the-arts on realistic attribute editing with facial details well
preserved.
Towards Recovery of Conditional Vectors from Conditional Generative Adversarial Networks
Authors: Sihao Ding, Andreas Wallin
Date: 2017-12-06 00:00:00
Tasks: Generative Adversarial Network, Image Manipulation

A conditional Generative Adversarial Network allows for generating samples
conditioned on certain external information. Being able to recover latent and
conditional vectors from a condi- tional GAN can be potentially valuable in
various applications, ranging from image manipulation for entertaining purposes
to diagnosis of the neural networks for security purposes. In this work, we
show that it is possible to recover both latent and conditional vectors from
generated images given the generator of a conditional generative adversarial
network. Such a recovery is not trivial due to the often multi-layered
non-linearity of deep neural networks. Furthermore, the effect of such recovery
applied on real natural images are investigated. We discovered that there
exists a gap between the recovery performance on generated and real images,
which we believe comes from the difference between generated data distribution
and real data distribution. Experiments are conducted to evaluate the recovered
conditional vectors and the reconstructed images from these recovered vectors
quantitatively and qualitatively, showing promising results.
Attention Is All You Need
Authors: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin
Date: 2017-06-12 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Abstractive Text Summarization, All, Coreference Resolution, Decoder, Few-Shot 3D Point Cloud Classification, Image-guided Story Ending Generation, LIDAR Semantic Segmentation, Link Prediction, Machine Translation, Multimodal Machine Translation, Natural Language Understanding, Question Answering, Speech Emotion Recognition, Supervised Only 3D Point Cloud Classification, Text Summarization, Translation

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.
Dual Attention Network for Product Compatibility and Function Satisfiability Analysis
Authors: Hu Xu, Sihong Xie, Lei Shu, Philip S. Yu
Date: 2017-12-06 00:00:00

Product compatibility and their functionality are of utmost importance to
customers when they purchase products, and to sellers and manufacturers when
they sell products. Due to the huge number of products available online, it is
infeasible to enumerate and test the compatibility and functionality of every
product. In this paper, we address two closely related problems: product
compatibility analysis and function satisfiability analysis, where the second
problem is a generalization of the first problem (e.g., whether a product works
with another product can be considered as a special function). We first
identify a novel question and answering corpus that is up-to-date regarding
product compatibility and functionality information. To allow automatic
discovery product compatibility and functionality, we then propose a deep
learning model called Dual Attention Network (DAN). Given a QA pair for a
to-be-purchased product, DAN learns to 1) discover complementary products (or
functions), and 2) accurately predict the actual compatibility (or
satisfiability) of the discovered products (or functions). The challenges
addressed by the model include the briefness of QAs, linguistic patterns
indicating compatibility, and the appropriate fusion of questions and answers.
We conduct experiments to quantitatively and qualitatively show that the
identified products and functions have both high coverage and accuracy,
compared with a wide spectrum of baselines.
Deep Anticipation: Light Weight Intelligent Mobile Sensing in IoT by Recurrent Architecture
Authors: Guang Chen, Shu Liu, Kejia Ren, Zhongnan Qu, Changhong Fu, Gereon Hinz, Alois Knoll
Date: 2017-12-06 00:00:00

The rapid growth of IoT era is shaping the future of mobile services.
Advanced communication technology enables a heterogeneous connectivity where
mobile devices broadcast information to everything. Mobile applications such as
robotics and vehicles connecting to cloud and surroundings transfer the
short-range on-board sensor perception system to long-range mobile-sensing
perception system. However, the mobile sensing perception brings new challenges
for how to efficiently analyze and intelligently interpret the deluge of IoT
data in mission- critical services. In this article, we model the challenges as
latency, packet loss and measurement noise which severely deteriorate the
reliability and quality of IoT data. We integrate the artificial intelligence
into IoT to tackle these challenges. We propose a novel architecture that
leverages recurrent neural networks (RNN) and Kalman filtering to anticipate
motions and interac- tions between objects. The basic idea is to learn
environment dynamics by recurrent networks. To improve the robustness of IoT
communication, we use the idea of Kalman filtering and deploy a prediction and
correction step. In this way, the architecture learns to develop a biased
belief between prediction and measurement in the different situation. We
demonstrate our approach with synthetic and real-world datasets with noise that
mimics the challenges of IoT communications. Our method brings a new level of
IoT intelligence. It is also lightweight compared to other state-of-the-art
convolutional recurrent architecture and is ideally suitable for the
resource-limited mobile applications.
On the nonparametric maximum likelihood estimator for Gaussian location mixture densities with application to Gaussian denoising
Authors: Sujayam Saha, Adityanand Guntuboyina
Date: 2017-12-06 00:00:00
Tasks: Clustering, Denoising

We study the Nonparametric Maximum Likelihood Estimator (NPMLE) for estimating Gaussian location mixture densities in $d$-dimensions from independent observations. Unlike usual likelihood-based methods for fitting mixtures, NPMLEs are based on convex optimization. We prove finite sample results on the Hellinger accuracy of every NPMLE. Our results imply, in particular, that every NPMLE achieves near parametric risk (up to logarithmic multiplicative factors) when the true density is a discrete Gaussian mixture without any prior information on the number of mixture components. NPMLEs can naturally be used to yield empirical Bayes estimates of the Oracle Bayes estimator in the Gaussian denoising problem. We prove bounds for the accuracy of the empirical Bayes estimate as an approximation to the Oracle Bayes estimator. Here our results imply that the empirical Bayes estimator performs at nearly the optimal level (up to logarithmic multiplicative factors) for denoising in clustering situations without any prior knowledge of the number of clusters.
HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition
Authors: Rajeev Ranjan, Vishal M. Patel, Rama Chellappa
Date: 2016-03-03 00:00:00
Tasks: Face Detection, Multi-Task Learning, Pose Estimation

We present an algorithm for simultaneous face detection, landmarks
localization, pose estimation and gender recognition using deep convolutional
neural networks (CNN). The proposed method called, HyperFace, fuses the
intermediate layers of a deep CNN using a separate CNN followed by a multi-task
learning algorithm that operates on the fused features. It exploits the synergy
among the tasks which boosts up their individual performances. Additionally, we
propose two variants of HyperFace: (1) HyperFace-ResNet that builds on the
ResNet-101 model and achieves significant improvement in performance, and (2)
Fast-HyperFace that uses a high recall fast face detector for generating region
proposals to improve the speed of the algorithm. Extensive experiments show
that the proposed models are able to capture both global and local information
in faces and performs significantly better than many competitive algorithms for
each of these four tasks.
An analysis of incorporating an external language model into a sequence-to-sequence model
Authors: Anjuli Kannan, Yonghui Wu, Patrick Nguyen, Tara N. Sainath, Zhifeng Chen, Rohit Prabhavalkar
Date: 2017-12-06 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, model, speech-recognition, Speech Recognition

Attention-based sequence-to-sequence models for automatic speech recognition
jointly train an acoustic model, language model, and alignment mechanism. Thus,
the language model component is only trained on transcribed audio-text pairs.
This leads to the use of shallow fusion with an external language model at
inference time. Shallow fusion refers to log-linear interpolation with a
separately trained language model at each step of the beam search. In this
work, we investigate the behavior of shallow fusion across a range of
conditions: different types of language models, different decoding units, and
different tasks. On Google Voice Search, we demonstrate that the use of shallow
fusion with a neural LM with wordpieces yields a 9.1% relative word error rate
reduction (WERR) over our competitive attention-based sequence-to-sequence
model, obviating the need for second-pass rescoring.
SERKET: An Architecture for Connecting Stochastic Models to Realize a Large-Scale Cognitive Model
Authors: Tomoaki Nakamura, Takayuki Nagai, Tadahiro Taniguchi
Date: 2017-12-04 00:00:00
Tasks: parameter estimation

To realize human-like robot intelligence, a large-scale cognitive
architecture is required for robots to understand the environment through a
variety of sensors with which they are equipped. In this paper, we propose a
novel framework named Serket that enables the construction of a large-scale
generative model and its inference easily by connecting sub-modules to allow
the robots to acquire various capabilities through interaction with their
environments and others. We consider that large-scale cognitive models can be
constructed by connecting smaller fundamental models hierarchically while
maintaining their programmatic independence. Moreover, connected modules are
dependent on each other, and parameters are required to be optimized as a
whole. Conventionally, the equations for parameter estimation have to be
derived and implemented depending on the models. However, it becomes harder to
derive and implement those of a larger scale model. To solve these problems, in
this paper, we propose a method for parameter estimation by communicating the
minimal parameters between various modules while maintaining their programmatic
independence. Therefore, Serket makes it easy to construct large-scale models
and estimate their parameters via the connection of modules. Experimental
results demonstrated that the model can be constructed by connecting modules,
the parameters can be optimized as a whole, and they are comparable with the
original models that we have proposed.
Mosquito detection with low-cost smartphones: data acquisition for malaria research
Authors: Yunpeng Li, Davide Zilli, Henry Chan, Ivan Kiskin, Marianne Sinka, Stephen Roberts, Kathy Willis
Date: 2017-11-16 00:00:00

Mosquitoes are a major vector for malaria, causing hundreds of thousands of
deaths in the developing world each year. Not only is the prevention of
mosquito bites of paramount importance to the reduction of malaria transmission
cases, but understanding in more forensic detail the interplay between malaria,
mosquito vectors, vegetation, standing water and human populations is crucial
to the deployment of more effective interventions. Typically the presence and
detection of malaria-vectoring mosquitoes is only quantified by hand-operated
insect traps or signified by the diagnosis of malaria. If we are to gather
timely, large-scale data to improve this situation, we need to automate the
process of mosquito detection and classification as much as possible. In this
paper, we present a candidate mobile sensing system that acts as both a
portable early warning device and an automatic acoustic data acquisition
pipeline to help fuel scientific inquiry and policy. The machine learning
algorithm that powers the mobile system achieves excellent off-line
multi-species detection performance while remaining computationally efficient.
Further, we have conducted preliminary live mosquito detection tests using
low-cost mobile phones and achieved promising results. The deployment of this
system for field usage in Southeast Asia and Africa is planned in the near
future. In order to accelerate processing of field recordings and labelling of
collected data, we employ a citizen science platform in conjunction with
automated methods, the former implemented using the Zooniverse platform,
allowing crowdsourcing on a grand scale.
A Scalable Deep Neural Network Architecture for Multi-Building and Multi-Floor Indoor Localization Based on Wi-Fi Fingerprinting
Authors: Kyeong Soo Kim, Sanghyuk Lee, Kaizhu Huang
Date: 2017-12-06 00:00:00
Tasks: General Classification, Indoor Localization, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

One of the key technologies for future large-scale location-aware services
covering a complex of multi-story buildings --- e.g., a big shopping mall and a
university campus --- is a scalable indoor localization technique. In this
paper, we report the current status of our investigation on the use of deep
neural networks (DNNs) for scalable building/floor classification and
floor-level position estimation based on Wi-Fi fingerprinting. Exploiting the
hierarchical nature of the building/floor estimation and floor-level
coordinates estimation of a location, we propose a new DNN architecture
consisting of a stacked autoencoder for the reduction of feature space
dimension and a feed-forward classifier for multi-label classification of
building/floor/location, on which the multi-building and multi-floor indoor
localization system based on Wi-Fi fingerprinting is built. Experimental
results for the performance of building/floor estimation and floor-level
coordinates estimation of a given location demonstrate the feasibility of the
proposed DNN-based indoor localization system, which can provide near
state-of-the-art performance using a single DNN, for the implementation with
lower complexity and energy consumption at mobile devices.