pwc-archive/papers-with-abstracts huggingface dataset:
3G structure for image caption generation
Authors: Aihong Yuan, Xuelong. Li, Xiaoqiang Lu
Date: 2019-04-21 00:00:00
Tasks: Caption Generation, Sentence

It is a big challenge of computer vision to make machine automatically
describe the content of an image with a natural language sentence. Previous
works have made great progress on this task, but they only use the global or
local image feature, which may lose some important subtle or global information
of an image. In this paper, we propose a model with 3-gated model which fuses
the global and local image features together for the task of image caption
generation. The model mainly has three gated structures. 1) Gate for the global
image feature, which can adaptively decide when and how much the global image
feature should be imported into the sentence generator. 2) The gated recurrent
neural network (RNN) is used as the sentence generator. 3) The gated feedback
method for stacking RNN is employed to increase the capability of nonlinearity
fitting. More specially, the global and local image features are combined
together in this paper, which makes full use of the image information. The
global image feature is controlled by the first gate and the local image
feature is selected by the attention mechanism. With the latter two gates, the
relationship between image and text can be well explored, which improves the
performance of the language part as well as the multi-modal embedding part.
Experimental results show that our proposed method outperforms the
state-of-the-art for image caption generation.
Fact Discovery from Knowledge Base via Facet Decomposition
Authors: Zihao Fu, Yankai Lin, Zhiyuan Liu, Wai Lam
Date: 2019-04-21 00:00:00
Proceeding: NAACL 2019 6
Tasks: Knowledge Base Completion, Relation Prediction

During the past few decades, knowledge bases (KBs) have experienced rapid
growth. Nevertheless, most KBs still suffer from serious incompletion.
Researchers proposed many tasks such as knowledge base completion and relation
prediction to help build the representation of KBs. However, there are some
issues unsettled towards enriching the KBs. Knowledge base completion and
relation prediction assume that we know two elements of the fact triples and we
are going to predict the missing one. This assumption is too restricted in
practice and prevents it from discovering new facts directly. To address this
issue, we propose a new task, namely, fact discovery from knowledge base. This
task only requires that we know the head entity and the goal is to discover
facts associated with the head entity. To tackle this new problem, we propose a
novel framework that decomposes the discovery problem into several facet
discovery components. We also propose a novel auto-encoder based facet
component to estimate some facets of the fact. Besides, we propose a feedback
learning component to share the information between each facet. We evaluate our
framework using a benchmark dataset and the experimental results show that our
framework achieves promising results. We also conduct extensive analysis of our
framework in discovering different kinds of facts. The source code of this
paper can be obtained from https://github.com/thunlp/FFD.
PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text
Authors: Haitian Sun, Tania Bedrax-Weiss, William W. Cohen
Date: 2019-04-21 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Open-Domain Question Answering, Question Answering, Retrieval

We consider open-domain queston answering (QA) where answers are drawn from
either a corpus, a knowledge base (KB), or a combination of both of these. We
focus on a setting in which a corpus is supplemented with a large but
incomplete KB, and on questions that require non-trivial (e.g., ``multi-hop'')
reasoning. We describe PullNet, an integrated framework for (1) learning what
to retrieve (from the KB and/or corpus) and (2) reasoning with this
heterogeneous information to find the best answer. PullNet uses an {iterative}
process to construct a question-specific subgraph that contains information
relevant to the question. In each iteration, a graph convolutional network
(graph CNN) is used to identify subgraph nodes that should be expanded using
retrieval (or ``pull'') operations on the corpus and/or KB. After the subgraph
is complete, a similar graph CNN is used to extract the answer from the
subgraph. This retrieve-and-reason process allows us to answer multi-hop
questions using large KBs and corpora. PullNet is weakly supervised, requiring
question-answer pairs but not gold inference paths. Experimentally PullNet
improves over the prior state-of-the art, and in the setting where a corpus is
used with incomplete KB these improvements are often dramatic. PullNet is also
often superior to prior systems in a KB-only setting or a text-only setting.
NeuronBlocks: Building Your NLP DNN Models Like Playing Lego
Authors: Ming Gong, Linjun Shou, Wutao Lin, Zhijie Sang, Quanjia Yan, Ze Yang, Feixiang Cheng, Daxin Jiang
Date: 2019-04-21 00:00:00
Proceeding: IJCNLP 2019 11

Deep Neural Networks (DNN) have been widely employed in industry to address various Natural Language Processing (NLP) tasks. However, many engineers find it a big overhead when they have to choose from multiple frameworks, compare different types of models, and understand various optimization mechanisms. An NLP toolkit for DNN models with both generality and flexibility can greatly improve the productivity of engineers by saving their learning cost and guiding them to find optimal solutions to their tasks. In this paper, we introduce NeuronBlocks\footnote{Code: \url{https://github.com/Microsoft/NeuronBlocks}} \footnote{Demo: \url{https://youtu.be/x6cOpVSZcdo}}, a toolkit encapsulating a suite of neural network modules as building blocks to construct various DNN models with complex architecture. This toolkit empowers engineers to build, train, and test various NLP models through simple configuration of JSON files. The experiments on several NLP datasets such as GLUE, WikiQA and CoNLL-2003 demonstrate the effectiveness of NeuronBlocks.
GAN-based Generation and Automatic Selection of Explanations for Neural Networks
Authors: Saumitra Mishra, Daniel Stoller, Emmanouil Benetos, Bob L. Sturm, Simon Dixon
Date: 2019-04-21 00:00:00

One way to interpret trained deep neural networks (DNNs) is by inspecting
characteristics that neurons in the model respond to, such as by iteratively
optimising the model input (e.g., an image) to maximally activate specific
neurons. However, this requires a careful selection of hyper-parameters to
generate interpretable examples for each neuron of interest, and current
methods rely on a manual, qualitative evaluation of each setting, which is
prohibitively slow. We introduce a new metric that uses Fr\'echet Inception
Distance (FID) to encourage similarity between model activations for real and
generated data. This provides an efficient way to evaluate a set of generated
examples for each setting of hyper-parameters. We also propose a novel
GAN-based method for generating explanations that enables an efficient search
through the input space and imposes a strong prior favouring realistic outputs.
We apply our approach to a classification model trained to predict whether a
music audio recording contains singing voice. Our results suggest that this
proposed metric successfully selects hyper-parameters leading to interpretable
examples, avoiding the need for manual evaluation. Moreover, we see that
examples synthesised to maximise or minimise the predicted probability of
singing voice presence exhibit vocal or non-vocal characteristics,
respectively, suggesting that our approach is able to generate suitable
explanations for understanding concepts learned by a neural network.
Automatic Temporally Coherent Video Colorization
Authors: Harrish Thasarathan, Kamyar Nazeri, Mehran Ebrahimi
Date: 2019-04-21 00:00:00
Tasks: Colorization, Image Colorization, Image Restoration, Image-to-Image Translation, Translation

Greyscale image colorization for applications in image restoration has seen
significant improvements in recent years. Many of these techniques that use
learning-based methods struggle to effectively colorize sparse inputs. With the
consistent growth of the anime industry, the ability to colorize sparse input
such as line art can reduce significant cost and redundant work for production
studios by eliminating the in-between frame colorization process. Simply using
existing methods yields inconsistent colors between related frames resulting in
a flicker effect in the final video. In order to successfully automate key
areas of large-scale anime production, the colorization of line arts must be
temporally consistent between frames. This paper proposes a method to colorize
line art frames in an adversarial setting, to create temporally coherent video
of large anime by improving existing image to image translation methods. We
show that by adding an extra condition to the generator and discriminator, we
can effectively create temporally consistent video sequences from anime line
arts. Code and models available at: https://github.com/Harry-Thasarathan/TCVC
Metric Learning for Image Registration
Authors: Marc Niethammer, Roland Kwitt, Francois-Xavier Vialard
Date: 2019-04-21 00:00:00
Proceeding: CVPR 2019 6
Tasks: Deep Learning, Deformable Medical Image Registration, Diffeomorphic Medical Image Registration, Image Registration, Medical Image Analysis, Metric Learning

Image registration is a key technique in medical image analysis to estimate
deformations between image pairs. A good deformation model is important for
high-quality estimates. However, most existing approaches use ad-hoc
deformation models chosen for mathematical convenience rather than to capture
observed data variation. Recent deep learning approaches learn deformation
models directly from data. However, they provide limited control over the
spatial regularity of transformations. Instead of learning the entire
registration approach, we learn a spatially-adaptive regularizer within a
registration model. This allows controlling the desired level of regularity and
preserving structural properties of a registration model. For example,
diffeomorphic transformations can be attained. Our approach is a radical
departure from existing deep learning approaches to image registration by
embedding a deep learning model in an optimization-based registration algorithm
to parameterize and data-adapt the registration model itself.
Neural Architecture Search for Deep Face Recognition
Authors: Ning Zhu
Date: 2019-04-21 00:00:00
Tasks: AutoML, Face Recognition, image-classification, Image Classification, Neural Architecture Search, Reinforcement Learning

By the widespread popularity of electronic devices, the emergence of
biometric technology has brought significant convenience to user authentication
compared with the traditional password and mode unlocking. Among many
biological characteristics, the face is a universal and irreplaceable feature
that does not need too much cooperation and can significantly improve the
user's experience at the same time. Face recognition is one of the main
functions of electronic equipment propaganda. Hence it's virtually worth
researching in computer vision. Previous work in this field has focused on two
directions: converting loss function to improve recognition accuracy in
traditional deep convolution neural networks (Resnet); combining the latest
loss function with the lightweight system (MobileNet) to reduce network size at
the minimal expense of accuracy. But none of these has changed the network
structure. With the development of AutoML, neural architecture search (NAS) has
shown excellent performance in the benchmark of image classification. In this
paper, we integrate NAS technology into face recognition to customize a more
suitable network. We quote the framework of neural architecture search which
trains child and controller network alternately. At the same time, we mutate
NAS by incorporating evaluation latency into rewards of reinforcement learning
and utilize policy gradient algorithm to search the architecture automatically
with the most classical cross-entropy loss. The network architectures we
searched out have got state-of-the-art accuracy in the large-scale face
dataset, which achieves 98.77% top-1 in MS-Celeb-1M and 99.89% in LFW with
relatively small network size. To the best of our knowledge, this proposal is
the first attempt to use NAS to solve the problem of Deep Face Recognition and
achieve the best results in this domain.
Few-Shot NLG with Pre-Trained Language Model
Authors: Zhiyu Chen, Harini Eavani, Wenhu Chen, Yinyin Liu, William Yang Wang
Date: 2019-04-21 00:00:00
Proceeding: ACL 2020 6
Tasks: Few-Shot Learning, Language Modeling, Language Modelling, model, Text Generation

Neural-based end-to-end approaches to natural language generation (NLG) from structured data or knowledge are data-hungry, making their adoption for real-world applications difficult with limited data. In this work, we propose the new task of \textit{few-shot natural language generation}. Motivated by how humans tend to summarize tabular data, we propose a simple yet effective approach and show that it not only demonstrates strong performance but also provides good generalization across domains. The design of the model architecture is based on two aspects: content selection from input data and language modeling to compose coherent sentences, which can be acquired from prior knowledge. With just 200 training examples, across multiple domains, we show that our approach achieves very reasonable performances and outperforms the strongest baseline by an average of over 8.0 BLEU points improvement. Our code and data can be found at \url{https://github.com/czyssrs/Few-Shot-NLG}
Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANs
Authors: Javad Amirian, Jean-Bernard Hayet, Julien Pettre
Date: 2019-04-20 00:00:00
Proceeding: CVPR 2019 6
Tasks: Generative Adversarial Network, Human motion prediction, Multi-future Trajectory Prediction, Pedestrian Trajectory Prediction, Self-Driving Cars, Trajectory Forecasting, Trajectory Prediction

This paper proposes a novel approach for predicting the motion of pedestrians
interacting with others. It uses a Generative Adversarial Network (GAN) to
sample plausible predictions for any agent in the scene. As GANs are very
susceptible to mode collapsing and dropping, we show that the recently proposed
Info-GAN allows dramatic improvements in multi-modal pedestrian trajectory
prediction to avoid these issues. We also left out L2-loss in training the
generator, unlike some previous works, because it causes serious mode
collapsing though faster convergence.
  We show through experiments on real and synthetic data that the proposed
method leads to generate more diverse samples and to preserve the modes of the
predictive distribution. In particular, to prove this claim, we have designed a
toy example dataset of trajectories that can be used to assess the performance
of different methods in preserving the predictive distribution modes.
Model-free Deep Reinforcement Learning for Urban Autonomous Driving
Authors: Jianyu Chen, Bodi Yuan, Masayoshi Tomizuka
Date: 2019-04-20 00:00:00
Tasks: Autonomous Driving, Decision Making, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Urban autonomous driving decision making is challenging due to complex road geometry and multi-agent interactions. Current decision making methods are mostly manually designing the driving policy, which might result in sub-optimal solutions and is expensive to develop, generalize and maintain at scale. On the other hand, with reinforcement learning (RL), a policy can be learned and improved automatically without any manual designs. However, current RL methods generally do not work well on complex urban scenarios. In this paper, we propose a framework to enable model-free deep reinforcement learning in challenging urban autonomous driving scenarios. We design a specific input representation and use visual encoding to capture the low-dimensional latent states. Several state-of-the-art model-free deep RL algorithms are implemented into our framework, with several tricks to improve their performance. We evaluate our method in a challenging roundabout task with dense surrounding vehicles in a high-definition driving simulator. The result shows that our method can solve the task well and is significantly better than the baseline.
Energy-based Self-attentive Learning of Abstractive Communities for Spoken Language Understanding
Authors: Guokan Shang, Antoine Jean-Pierre Tixier, Michalis Vazirgiannis, Jean-Pierre Lorré
Date: 2019-04-20 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Clustering, Community Detection, Sentence, Spoken Language Understanding, Triplet

Abstractive community detection is an important spoken language understanding task, whose goal is to group utterances in a conversation according to whether they can be jointly summarized by a common abstractive sentence. This paper provides a novel approach to this task. We first introduce a neural contextual utterance encoder featuring three types of self-attention mechanisms. We then train it using the siamese and triplet energy-based meta-architectures. Experiments on the AMI corpus show that our system outperforms multiple energy-based and non-energy based baselines from the state-of-the-art. Code and data are publicly available.
Compression and Localization in Reinforcement Learning for ATARI Games
Authors: Joel Ruben Antony Moniz, Barun Patra, Sarthak Garg
Date: 2019-04-20 00:00:00
Tasks: Atari Games, Model Compression, Object Localization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Weakly-Supervised Object Localization

Deep neural networks have become commonplace in the domain of reinforcement
learning, but are often expensive in terms of the number of parameters needed.
While compressing deep neural networks has of late assumed great importance to
overcome this drawback, little work has been done to address this problem in
the context of reinforcement learning agents. This work aims at making first
steps towards model compression in an RL agent. In particular, we compress
networks to drastically reduce the number of parameters in them (to sizes less
than 3% of their original size), further facilitated by applying a global max
pool after the final convolution layer, and propose using Actor-Mimic in the
context of compression. Finally, we show that this global max-pool allows for
weakly supervised object localization, improving the ability to identify the
agent's points of focus.
CleanML: A Study for Evaluating the Impact of Data Cleaning on ML Classification Tasks
Authors: Peng Li, Xi Rao, Jennifer Blase, Yue Zhang, Xu Chu, Ce Zhang
Date: 2019-04-20 00:00:00
Tasks: General Classification, Two-sample testing

Data quality affects machine learning (ML) model performances, and data scientists spend considerable amount of time on data cleaning before model training. However, to date, there does not exist a rigorous study on how exactly cleaning affects ML -- ML community usually focuses on developing ML algorithms that are robust to some particular noise types of certain distributions, while database (DB) community has been mostly studying the problem of data cleaning alone without considering how data is consumed by downstream ML analytics. We propose a CleanML study that systematically investigates the impact of data cleaning on ML classification tasks. The open-source and extensible CleanML study currently includes 14 real-world datasets with real errors, five common error types, seven different ML models, and multiple cleaning algorithms for each error type (including both commonly used algorithms in practice as well as state-of-the-art solutions in academic literature). We control the randomness in ML experiments using statistical hypothesis testing, and we also control false discovery rate in our experiments using the Benjamini-Yekutieli (BY) procedure. We analyze the results in a systematic way to derive many interesting and nontrivial observations. We also put forward multiple research directions for researchers.
Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding
Authors: Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao
Date: 2019-04-20 00:00:00
Tasks: Ensemble Learning, Knowledge Distillation, Multi-Task Learning, Natural Language Inference, Natural Language Understanding, Semantic Textual Similarity, Sentiment Analysis

This paper explores the use of knowledge distillation to improve a Multi-Task
Deep Neural Network (MT-DNN) (Liu et al., 2019) for learning text
representations across multiple natural language understanding tasks. Although
ensemble learning can improve model performance, serving an ensemble of large
DNNs such as MT-DNN can be prohibitively expensive. Here we apply the knowledge
distillation method (Hinton et al., 2015) in the multi-task learning setting.
For each task, we train an ensemble of different MT-DNNs (teacher) that
outperforms any single model, and then train a single MT-DNN (student) via
multi-task learning to \emph{distill} knowledge from these ensemble teachers.
We show that the distilled MT-DNN significantly outperforms the original MT-DNN
on 7 out of 9 GLUE tasks, pushing the GLUE benchmark (single model) to 83.7\%
(1.5\% absolute improvement\footnote{ Based on the GLUE leaderboard at
https://gluebenchmark.com/leaderboard as of April 1, 2019.}). The code and
pre-trained models will be made publicly available at
https://github.com/namisan/mt-dnn.
Preference-based Multiobjective Virtual Machine Placement: A Ceteris Paribus Approach
Authors: Abdulaziz Alashaikh, Eisa Alanazi
Date: 2019-04-20 00:00:00

This work adopts the notion of Ceteris Paribus (CP) as an interpretation of
the Decision Maker (DM) preferences and incorporates it in a constrained
multiobjective problem known as virtual machine placement (VMP). VMP is an
essential multiobjective problem in the design and operation of cloud data
centers concerned about placing each virtual machine to a physical machine (a
server) in the data center. We analyze the effectiveness of CP interpretation
on VMP problems and propose an NSGA-II variant with which preferred solutions
are returned at almost no extra time cost.
ChoiceNet: CNN learning through choice of multiple feature map representations
Authors: Farshid Rayhan, Aphrodite Galata, Timothy F. Cootes
Date: 2019-04-20 00:00:00
Tasks: Object Recognition, Semantic Segmentation

We introduce a new architecture called ChoiceNet where each layer of the network is highly connected with skip connections and channelwise concatenations. This enables the network to alleviate the problem of vanishing gradients, reduces the number of parameters without sacrificing performance, and encourages feature reuse. We evaluate our proposed architecture on three benchmark datasets for object recognition tasks (ImageNet, CIFAR- 10, CIFAR-100, SVHN) and on a semantic segmentation dataset (CamVid).