pwc-archive/papers-with-abstracts huggingface dataset:
HFL-RC System at SemEval-2018 Task 11: Hybrid Multi-Aspects Model for Commonsense Reading Comprehension
Authors: Zhipeng Chen, Yiming Cui, Wentao Ma, Shijin Wang, Ting Liu, Guoping Hu
Date: 2018-03-15 00:00:00
Tasks: Multiple-choice, Reading Comprehension

This paper describes the system which got the state-of-the-art results at
SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge. In
this paper, we present a neural network called Hybrid Multi-Aspects (HMA)
model, which mimic the human's intuitions on dealing with the multiple-choice
reading comprehension. In this model, we aim to produce the predictions in
multiple aspects by calculating attention among the text, question and choices,
and combine these results for final predictions. Experimental results show that
our HMA model could give substantial improvements over the baseline system and
got the first place on the final test set leaderboard with the accuracy of
84.13%.
Dataset for the First Evaluation on Chinese Machine Reading Comprehension
Authors: Yiming Cui, Ting Liu, Zhipeng Chen, Wentao Ma, Shijin Wang, Guoping Hu
Date: 2017-09-25 00:00:00
Proceeding: LREC 2018 5
Tasks: Chinese Reading Comprehension, Diversity, Machine Reading Comprehension, Reading Comprehension

Machine Reading Comprehension (MRC) has become enormously popular recently
and has attracted a lot of attention. However, existing reading comprehension
datasets are mostly in English. To add diversity in reading comprehension
datasets, in this paper we propose a new Chinese reading comprehension dataset
for accelerating related research in the community. The proposed dataset
contains two different types: cloze-style reading comprehension and user query
reading comprehension, associated with large-scale training data as well as
human-annotated validation and hidden test set. Along with this dataset, we
also hosted the first Evaluation on Chinese Machine Reading Comprehension
(CMRC-2017) and successfully attracted tens of participants, which suggest the
potential impact of this dataset.
Consensus Attention-based Neural Networks for Chinese Reading Comprehension
Authors: Yiming Cui, Ting Liu, Zhipeng Chen, Shijin Wang, Guoping Hu
Date: 2016-07-08 00:00:00
Proceeding: COLING 2016 12
Tasks: Chinese Reading Comprehension, Reading Comprehension

Reading comprehension has embraced a booming in recent NLP research. Several
institutes have released the Cloze-style reading comprehension data, and these
have greatly accelerated the research of machine comprehension. In this work,
we firstly present Chinese reading comprehension datasets, which consist of
People Daily news dataset and Children's Fairy Tale (CFT) dataset. Also, we
propose a consensus attention-based neural network architecture to tackle the
Cloze-style reading comprehension problem, which aims to induce a consensus
attention over every words in the query. Experimental results show that the
proposed neural network significantly outperforms the state-of-the-art
baselines in several public datasets. Furthermore, we setup a baseline for
Chinese reading comprehension task, and hopefully this would speed up the
process for future research.
Sylvester Normalizing Flows for Variational Inference
Authors: Rianne van den Berg, Leonard Hasenclever, Jakub M. Tomczak, Max Welling
Date: 2018-03-15 00:00:00
Tasks: Variational Inference

Variational inference relies on flexible approximate posterior distributions.
Normalizing flows provide a general recipe to construct flexible variational
posteriors. We introduce Sylvester normalizing flows, which can be seen as a
generalization of planar flows. Sylvester normalizing flows remove the
well-known single-unit bottleneck from planar flows, making a single
transformation much more flexible. We compare the performance of Sylvester
normalizing flows against planar flows and inverse autoregressive flows and
demonstrate that they compare favorably on several datasets.
Neural Text Generation: Past, Present and Beyond
Authors: Sidi Lu, Yaoming Zhu, Wei-Nan Zhang, Jun Wang, Yong Yu
Date: 2018-03-15 00:00:00
Tasks: Benchmarking, Diversity, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Text Generation

This paper presents a systematic survey on recent development of neural text
generation models. Specifically, we start from recurrent neural network
language models with the traditional maximum likelihood estimation training
scheme and point out its shortcoming for text generation. We thus introduce the
recently proposed methods for text generation based on reinforcement learning,
re-parametrization tricks and generative adversarial nets (GAN) techniques. We
compare different properties of these models and the corresponding techniques
to handle their common problems such as gradient vanishing and generation
diversity. Finally, we conduct a benchmarking experiment with different types
of neural text generation models on two well-known datasets and discuss the
empirical results along with the aforementioned model properties.
Proximal SCOPE for Distributed Sparse Learning: Better Data Partition Implies Faster Convergence Rate
Authors: Shen-Yi Zhao, Gong-Duo Zhang, Ming-Wei Li, Wu-Jun Li
Date: 2018-03-15 00:00:00
Tasks: Sparse Learning

Distributed sparse learning with a cluster of multiple machines has attracted
much attention in machine learning, especially for large-scale applications
with high-dimensional data. One popular way to implement sparse learning is to
use $L_1$ regularization. In this paper, we propose a novel method, called
proximal \mbox{SCOPE}~(\mbox{pSCOPE}), for distributed sparse learning with
$L_1$ regularization. pSCOPE is based on a \underline{c}ooperative
\underline{a}utonomous \underline{l}ocal \underline{l}earning~(\mbox{CALL})
framework. In the \mbox{CALL} framework of \mbox{pSCOPE}, we find that the data
partition affects the convergence of the learning procedure, and subsequently
we define a metric to measure the goodness of a data partition. Based on the
defined metric, we theoretically prove that pSCOPE is convergent with a linear
convergence rate if the data partition is good enough. We also prove that
better data partition implies faster convergence rate. Furthermore, pSCOPE is
also communication efficient. Experimental results on real data sets show that
pSCOPE can outperform other state-of-the-art distributed methods for sparse
learning.
Fast End-to-End Trainable Guided Filter
Authors: Huikai Wu, Shuai Zheng, Junge Zhang, Kaiqi Huang
Date: 2018-03-15 00:00:00
Proceeding: CVPR 2018 6

Dense pixel-wise image prediction has been advanced by harnessing the capabilities of Fully Convolutional Networks (FCNs). One central issue of FCNs is the limited capacity to handle joint upsampling. To address the problem, we present a novel building block for FCNs, namely guided filtering layer, which is designed for efficiently generating a high-resolution output given the corresponding low-resolution one and a high-resolution guidance map. Such a layer contains learnable parameters, which can be integrated with FCNs and jointly optimized through end-to-end training. To further take advantage of end-to-end training, we plug in a trainable transformation function for generating the task-specific guidance map. Based on the proposed layer, we present a general framework for pixel-wise image prediction, named deep guided filtering network (DGF). The proposed network is evaluated on five image processing tasks. Experiments on MIT-Adobe FiveK Dataset demonstrate that DGF runs 10-100 times faster and achieves the state-of-the-art performance. We also show that DGF helps to improve the performance of multiple computer vision tasks.
Arbitrary-Oriented Scene Text Detection via Rotation Proposals
Authors: Jianqi Ma, Weiyuan Shao, Hao Ye, Li Wang, Hong Wang, Yingbin Zheng, xiangyang xue
Date: 2017-03-03 00:00:00
Tasks: Computational Efficiency, Region Proposal, Scene Text Detection, Text Detection

This paper introduces a novel rotation-based framework for arbitrary-oriented
text detection in natural scene images. We present the Rotation Region Proposal
Networks (RRPN), which are designed to generate inclined proposals with text
orientation angle information. The angle information is then adapted for
bounding box regression to make the proposals more accurately fit into the text
region in terms of the orientation. The Rotation Region-of-Interest (RRoI)
pooling layer is proposed to project arbitrary-oriented proposals to a feature
map for a text region classifier. The whole framework is built upon a
region-proposal-based architecture, which ensures the computational efficiency
of the arbitrary-oriented text detection compared with previous text detection
systems. We conduct experiments using the rotation-based framework on three
real-world scene text detection datasets and demonstrate its superiority in
terms of effectiveness and efficiency over previous approaches.
Reconstructing Gaussian sources by spatial sampling
Authors: Vinay Praneeth Boda
Date: 2018-03-15 00:00:00

Consider a Gaussian memoryless multiple source with $m$ components with joint
probability distribution known only to lie in a given class of distributions. A
subset of $k \leq m$ components are sampled and compressed with the objective
of reconstructing all the $m$ components within a specified level of distortion
under a mean-squared error criterion. In Bayesian and nonBayesian settings, the
notion of universal sampling rate distortion function for Gaussian sources is
introduced to capture the optimal tradeoffs among sampling, compression rate
and distortion level. Single-letter characterizations are provided for the
universal sampling rate distortion function. Our achievability proofs highlight
the following structural property: it is optimal to compress and reconstruct
first the sampled components of the GMMS alone, and then form estimates for the
unsampled components based on the former.
Large Margin Deep Networks for Classification
Authors: Gamaleldin F. Elsayed, Dilip Krishnan, Hossein Mobahi, Kevin Regan, Samy Bengio
Date: 2018-03-15 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Classification, Data Augmentation, General Classification

We present a formulation of deep learning that aims at producing a large
margin classifier. The notion of margin, minimum distance to a decision
boundary, has served as the foundation of several theoretically profound and
empirically successful results for both classification and regression tasks.
However, most large margin algorithms are applicable only to shallow models
with a preset feature representation; and conventional margin methods for
neural networks only enforce margin at the output layer. Such methods are
therefore not well suited for deep networks.
  In this work, we propose a novel loss function to impose a margin on any
chosen set of layers of a deep network (including input and hidden layers). Our
formulation allows choosing any norm on the metric measuring the margin. We
demonstrate that the decision boundary obtained by our loss has nice properties
compared to standard classification loss functions. Specifically, we show
improved empirical results on the MNIST, CIFAR-10 and ImageNet datasets on
multiple tasks: generalization from small training sets, corrupted labels, and
robustness against adversarial perturbations. The resulting loss is general and
complementary to existing data augmentation (such as random/adversarial input
transform) and regularization techniques (such as weight decay, dropout, and
batch norm).
On the insufficiency of existing momentum schemes for Stochastic Optimization
Authors: Rahul Kidambi, Praneeth Netrapalli, Prateek Jain, Sham M. Kakade
Date: 2018-03-15 00:00:00
Proceeding: ICLR 2018 1
Tasks: Stochastic Optimization

Momentum based stochastic gradient methods such as heavy ball (HB) and
Nesterov's accelerated gradient descent (NAG) method are widely used in
practice for training deep networks and other supervised learning models, as
they often provide significant improvements over stochastic gradient descent
(SGD). Rigorously speaking, "fast gradient" methods have provable improvements
over gradient descent only for the deterministic case, where the gradients are
exact. In the stochastic case, the popular explanations for their wide
applicability is that when these fast gradient methods are applied in the
stochastic case, they partially mimic their exact gradient counterparts,
resulting in some practical gain. This work provides a counterpoint to this
belief by proving that there exist simple problem instances where these methods
cannot outperform SGD despite the best setting of its parameters. These
negative problem instances are, in an informal sense, generic; they do not look
like carefully constructed pathological instances. These results suggest (along
with empirical evidence) that HB or NAG's practical performance gains are a
by-product of mini-batching.
  Furthermore, this work provides a viable (and provable) alternative, which,
on the same set of problem instances, significantly improves over HB, NAG, and
SGD's performance. This algorithm, referred to as Accelerated Stochastic
Gradient Descent (ASGD), is a simple to implement stochastic algorithm, based
on a relatively less popular variant of Nesterov's Acceleration. Extensive
empirical results in this paper show that ASGD has performance gains over HB,
NAG, and SGD.
On a Formal Model of Safe and Scalable Self-driving Cars
Authors: Shai Shalev-Shwartz, Shaked Shammah, Amnon Shashua
Date: 2017-08-21 00:00:00
Tasks: Autonomous Driving, Self-Driving Cars

In recent years, car makers and tech companies have been racing towards self
driving cars. It seems that the main parameter in this race is who will have
the first car on the road. The goal of this paper is to add to the equation two
additional crucial parameters. The first is standardization of safety assurance
--- what are the minimal requirements that every self-driving car must satisfy,
and how can we verify these requirements. The second parameter is scalability
--- engineering solutions that lead to unleashed costs will not scale to
millions of cars, which will push interest in this field into a niche academic
corner, and drive the entire field into a "winter of autonomous driving". In
the first part of the paper we propose a white-box, interpretable, mathematical
model for safety assurance, which we call Responsibility-Sensitive Safety
(RSS). In the second part we describe a design of a system that adheres to our
safety assurance requirements and is scalable to millions of cars.
Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment
Authors: Zhiwen Shao, Zhilei Liu, Jianfei Cai, Lizhuang Ma
Date: 2018-03-15 00:00:00
Proceeding: ECCV 2018 9
Tasks: Action Unit Detection, Face Alignment, Facial Action Unit Detection

Facial action unit (AU) detection and face alignment are two highly
correlated tasks since facial landmarks can provide precise AU locations to
facilitate the extraction of meaningful local features for AU detection. Most
existing AU detection works often treat face alignment as a preprocessing and
handle the two tasks independently. In this paper, we propose a novel
end-to-end deep learning framework for joint AU detection and face alignment,
which has not been explored before. In particular, multi-scale shared features
are learned firstly, and high-level features of face alignment are fed into AU
detection. Moreover, to extract precise local features, we propose an adaptive
attention learning module to refine the attention map of each AU adaptively.
Finally, the assembled local features are integrated with face alignment
features and global features for AU detection. Experiments on BP4D and DISFA
benchmarks demonstrate that our framework significantly outperforms the
state-of-the-art methods for AU detection.
Rocket Launching: A Universal and Efficient Framework for Training Well-performing Light Net
Authors: Guorui Zhou, Ying Fan, Runpeng Cui, Weijie Bian, Xiaoqiang Zhu, Kun Gai
Date: 2017-08-14 00:00:00
Tasks: Click-Through Rate Prediction

Models applied on real time response task, like click-through rate (CTR)
prediction model, require high accuracy and rigorous response time. Therefore,
top-performing deep models of high depth and complexity are not well suited for
these applications with the limitations on the inference time. In order to
further improve the neural networks' performance given the time and
computational limitations, we propose an approach that exploits a cumbersome
net to help train the lightweight net for prediction. We dub the whole process
rocket launching, where the cumbersome booster net is used to guide the
learning of the target light net throughout the whole training process. We
analyze different loss functions aiming at pushing the light net to behave
similarly to the booster net, and adopt the loss with best performance in our
experiments. We use one technique called gradient block to improve the
performance of the light net and booster net further. Experiments on benchmark
datasets and real-life industrial advertisement data present that our light
model can get performance only previously achievable with more complex models.
Multiple-Human Parsing in the Wild
Authors: Jianshu Li, Jian Zhao, Yunchao Wei, Congyan Lang, Yidong Li, Terence Sim, Shuicheng Yan, Jiashi Feng
Date: 2017-05-19 00:00:00
Tasks: Human Parsing, Multi-Human Parsing

Human parsing is attracting increasing research attention. In this work, we
aim to push the frontier of human parsing by introducing the problem of
multi-human parsing in the wild. Existing works on human parsing mainly tackle
single-person scenarios, which deviates from real-world applications where
multiple persons are present simultaneously with interaction and occlusion. To
address the multi-human parsing problem, we introduce a new multi-human parsing
(MHP) dataset and a novel multi-human parsing model named MH-Parser. The MHP
dataset contains multiple persons captured in real-world scenes with
pixel-level fine-grained semantic annotations in an instance-aware setting. The
MH-Parser generates global parsing maps and person instance masks
simultaneously in a bottom-up fashion with the help of a new Graph-GAN model.
We envision that the MHP dataset will serve as a valuable data resource to
develop new multi-human parsing models, and the MH-Parser offers a strong
baseline to drive future research for multi-human parsing in the wild.
Improving GANs Using Optimal Transport
Authors: Tim Salimans, Han Zhang, Alec Radford, Dimitris Metaxas
Date: 2018-03-15 00:00:00
Proceeding: ICLR 2018 1
Tasks: Image Generation

We present Optimal Transport GAN (OT-GAN), a variant of generative
adversarial nets minimizing a new metric measuring the distance between the
generator distribution and the data distribution. This metric, which we call
mini-batch energy distance, combines optimal transport in primal form with an
energy distance defined in an adversarially learned feature space, resulting in
a highly discriminative distance function with unbiased mini-batch gradients.
Experimentally we show OT-GAN to be highly stable when trained with large
mini-batches, and we present state-of-the-art results on several popular
benchmark problems for image generation.
Consistent Multiple Graph Matching with Multi-layer Random Walks Synchronization
Authors: Han-Mu Park, Kuk-Jin Yoon
Date: 2017-12-07 00:00:00
Tasks: Graph Matching

We address the correspondence search problem among multiple graphs with
complex properties while considering the matching consistency. We describe each
pair of graphs by combining multiple attributes, then jointly match them in a
unified framework. The main contribution of this paper is twofold. First, we
formulate the global correspondence search problem of multi-attributed graphs
by utilizing a set of multi-layer structures. The proposed formulation
describes each pair of graphs as a multi-layer structure, and jointly considers
whole matching pairs. Second, we propose a robust multiple graph matching
method based on the multi-layer random walks framework. The proposed framework
synchronizes movements of random walkers, and leads them to consistent matching
candidates. In our extensive experiments, the proposed method exhibits robust
and accurate performance over the state-of-the-art multiple graph matching
algorithms.
Conditional Adversarial Synthesis of 3D Facial Action Units
Authors: Zhilei Liu, Guoxian Song, Jianfei Cai, Tat-Jen Cham, Juyong Zhang
Date: 2018-02-21 00:00:00
Tasks: Data Augmentation, Deep Learning, Image Generation

Employing deep learning-based approaches for fine-grained facial expression
analysis, such as those involving the estimation of Action Unit (AU)
intensities, is difficult due to the lack of a large-scale dataset of real
faces with sufficiently diverse AU labels for training. In this paper, we
consider how AU-level facial image synthesis can be used to substantially
augment such a dataset. We propose an AU synthesis framework that combines the
well-known 3D Morphable Model (3DMM), which intrinsically disentangles
expression parameters from other face attributes, with models that
adversarially generate 3DMM expression parameters conditioned on given target
AU labels, in contrast to the more conventional approach of generating facial
images directly. In this way, we are able to synthesize new combinations of
expression parameters and facial images from desired AU labels. Extensive
quantitative and qualitative results on the benchmark DISFA dataset demonstrate
the effectiveness of our method on 3DMM facial expression parameter synthesis
and data augmentation for deep learning-based AU intensity estimation.
Shifting Mean Activation Towards Zero with Bipolar Activation Functions
Authors: Lars Eidnes, Arild Nøkland
Date: 2017-09-12 00:00:00
Proceeding: ICLR 2018 1
Tasks: General Classification, Language Modeling, Language Modelling

We propose a simple extension to the ReLU-family of activation functions that
allows them to shift the mean activation across a layer towards zero. Combined
with proper weight initialization, this alleviates the need for normalization
layers. We explore the training of deep vanilla recurrent neural networks
(RNNs) with up to 144 layers, and show that bipolar activation functions help
learning in this setting. On the Penn Treebank and Text8 language modeling
tasks we obtain competitive results, improving on the best reported results for
non-gated networks. In experiments with convolutional neural networks without
batch normalization, we find that bipolar activations produce a faster drop in
training error, and results in a lower test error on the CIFAR-10
classification task.
Achieving Human Parity on Automatic Chinese to English News Translation
Authors: Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, Shujie Liu, Tie-Yan Liu, Renqian Luo, Arul Menezes, Tao Qin, Frank Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingce Xia, Dong-dong Zhang, Zhirui Zhang, Ming Zhou
Date: 2018-03-15 00:00:00
Tasks: Machine Translation, Translation

Machine translation has made rapid advances in recent years. Millions of
people are using it today in online translation systems and mobile applications
in order to communicate across language barriers. The question naturally arises
whether such systems can approach or achieve parity with human translations. In
this paper, we first address the problem of how to define and accurately
measure human parity in translation. We then describe Microsoft's machine
translation system and measure the quality of its translations on the widely
used WMT 2017 news translation task from Chinese to English. We find that our
latest neural machine translation system has reached a new state-of-the-art,
and that the translation quality is at human parity when compared to
professional human translations. We also find that it significantly exceeds the
quality of crowd-sourced non-professional translations.