pwc-archive/papers-with-abstracts huggingface dataset:
An Empirical Exploration of Curriculum Learning for Neural Machine Translation
Authors: Xuan Zhang, Gaurav Kumar, Huda Khayrallah, Kenton Murray, Jeremy Gwinnup, Marianna J. Martindale, Paul McNamee, Kevin Duh, Marine Carpuat
Date: 2018-11-02 00:00:00
Tasks: Machine Translation, Translation

Machine translation systems based on deep neural networks are expensive to
train. Curriculum learning aims to address this issue by choosing the order in
which samples are presented during training to help train better models faster.
We adopt a probabilistic view of curriculum learning, which lets us flexibly
evaluate the impact of curricula design, and perform an extensive exploration
on a German-English translation task. Results show that it is possible to
improve convergence time at no loss in translation quality. However, results
are highly sensitive to the choice of sample difficulty criteria, curriculum
schedule and other hyperparameters.
Improving the Robustness of Speech Translation
Authors: Xiang Li, Haiyang Xue, Wei Chen, Yang Liu, Yang Feng, Qun Liu
Date: 2018-11-02 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Machine Translation, NMT, Sentence, speech-recognition, Speech Recognition, Translation

Although neural machine translation (NMT) has achieved impressive progress
recently, it is usually trained on the clean parallel data set and hence cannot
work well when the input sentence is the production of the automatic speech
recognition (ASR) system due to the enormous errors in the source. To solve
this problem, we propose a simple but effective method to improve the
robustness of NMT in the case of speech translation. We simulate the noise
existing in the realistic output of the ASR system and inject them into the
clean parallel data so that NMT can work under similar word distributions
during training and testing. Besides, we also incorporate the Chinese Pinyin
feature which is easy to get in speech translation to further improve the
translation performance. Experiment results show that our method has a more
stable performance and outperforms the baseline by an average of 3.12 BLEU on
multiple noisy test sets, even while achieves a generalization improvement on
the WMT'17 Chinese-English test set.
Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problems
Authors: Ting-Rui Chiang, Yun-Nung Chen
Date: 2018-11-02 00:00:00
Proceeding: NAACL 2019 6
Tasks: Decoder, Math, Math Word Problem Solving, Natural Language Understanding, Retrieval

Solving math word problems is a challenging task that requires accurate natural language understanding to bridge natural language texts and math expressions. Motivated by the intuition about how human generates the equations given the problem texts, this paper presents a neural approach to automatically solve math word problems by operating symbols according to their semantic meanings in texts. This paper views the process of generating equation as a bridge between the semantic world and the symbolic world, where the proposed neural math solver is based on an encoder-decoder framework. In the proposed model, the encoder is designed to understand the semantics of problems, and the decoder focuses on tracking semantic meanings of the generated symbols and then deciding which symbol to generate next. The preliminary experiments are conducted in a dataset Math23K, and our model significantly outperforms both the state-of-the-art single model and the best non-retrieval-based model over about 10% accuracy, demonstrating the effectiveness of bridging the symbolic and semantic worlds from math word problems.
Dirichlet belief networks for topic structure learning
Authors: He Zhao, Lan Du, Wray Buntine, Mingyuan Zhou
Date: 2018-11-02 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Topic Models

Recently, considerable research effort has been devoted to developing deep
architectures for topic models to learn topic structures. Although several deep
models have been proposed to learn better topic proportions of documents, how
to leverage the benefits of deep structures for learning word distributions of
topics has not yet been rigorously studied. Here we propose a new multi-layer
generative process on word distributions of topics, where each layer consists
of a set of topics and each topic is drawn from a mixture of the topics of the
layer above. As the topics in all layers can be directly interpreted by words,
the proposed model is able to discover interpretable topic hierarchies. As a
self-contained module, our model can be flexibly adapted to different kinds of
topic models to improve their modelling accuracy and interpretability.
Extensive experiments on text corpora demonstrate the advantages of the
proposed model.
Training Neural Speech Recognition Systems with Synthetic Speech Augmentation
Authors: Jason Li, Ravi Gadde, Boris Ginsburg, Vitaly Lavrukhin
Date: 2018-11-02 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Building an accurate automatic speech recognition (ASR) system requires a
large dataset that contains many hours of labeled speech samples produced by a
diverse set of speakers. The lack of such open free datasets is one of the main
issues preventing advancements in ASR research. To address this problem, we
propose to augment a natural speech dataset with synthetic speech. We train
very large end-to-end neural speech recognition models using the LibriSpeech
dataset augmented with synthetic speech. These new models achieve state of the
art Word Error Rate (WER) for character-level based models without an external
language model.
Combining Similarity Features and Deep Representation Learning for Stance Detection in the Context of Checking Fake News
Authors: Luís Borges, Bruno Martins, Pável Calado
Date: 2018-11-02 00:00:00
Tasks: Document Classification, General Classification, Natural Language Inference, Representation Learning, Sentence, Stance Detection

Fake news are nowadays an issue of pressing concern, given their recent rise
as a potential threat to high-quality journalism and well-informed public
discourse. The Fake News Challenge (FNC-1) was organized in 2017 to encourage
the development of machine learning-based classification systems for stance
detection (i.e., for identifying whether a particular news article agrees,
disagrees, discusses, or is unrelated to a particular news headline), thus
helping in the detection and analysis of possible instances of fake news. This
article presents a new approach to tackle this stance detection problem, based
on the combination of string similarity features with a deep neural
architecture that leverages ideas previously advanced in the context of
learning efficient text representations, document classification, and natural
language inference. Specifically, we use bi-directional Recurrent Neural
Networks, together with max-pooling over the temporal/sequential dimension and
neural attention, for representing (i) the headline, (ii) the first two
sentences of the news article, and (iii) the entire news article. These
representations are then combined/compared, complemented with similarity
features inspired on other FNC-1 approaches, and passed to a final layer that
predicts the stance of the article towards the headline. We also explore the
use of external sources of information, specifically large datasets of sentence
pairs originally proposed for training and evaluating natural language
inference methods, in order to pre-train specific components of the neural
network architecture (e.g., the RNNs used for encoding sentences). The obtained
results attest to the effectiveness of the proposed ideas and show that our
model, particularly when considering pre-training and the combination of neural
representations together with similarity features, slightly outperforms the
previous state-of-the-art.
Learning Latent Fractional dynamics with Unknown Unknowns
Authors: Gaurav Gupta, Sergio Pequito, Paul Bogdan
Date: 2018-11-02 00:00:00

Despite significant effort in understanding complex systems (CS), we lack a
theory for modeling, inference, analysis and efficient control of time-varying
complex networks (TVCNs) in uncertain environments. From brain activity
dynamics to microbiome, and even chromatin interactions within the genome
architecture, many such TVCNs exhibits a pronounced spatio-temporal fractality.
Moreover, for many TVCNs only limited information (e.g., few variables) is
accessible for modeling, which hampers the capabilities of analytical tools to
uncover the true degrees of freedom and infer the CS model, the hidden states
and their parameters. Another fundamental limitation is that of understanding
and unveiling of unknown drivers of the dynamics that could sporadically excite
the network in ways that straightforward modeling does not work due to our
inability to model non-stationary processes. Towards addressing these
challenges, in this paper, we consider the problem of learning the fractional
dynamical complex networks under unknown unknowns (i.e., hidden drivers) and
partial observability (i.e., only partial data is available). More precisely,
we consider a generalized modeling approach of TVCNs consisting of
discrete-time fractional dynamical equations and propose an iterative framework
to determine the network parameterization and predict the state of the system.
We showcase the performance of the proposed framework in the context of task
classification using real electroencephalogram data.
Learning from Large-scale Noisy Web Data with Ubiquitous Reweighting for Image Classification
Authors: Jia Li, Yafei Song, Jianfeng Zhu, Lele Cheng, Ying Su, Lin Ye, Pengcheng Yuan, Shumin Han
Date: 2018-11-02 00:00:00
Tasks: Diversity, General Classification, image-classification, Image Classification

Many advances of deep learning techniques originate from the efforts of
addressing the image classification task on large-scale datasets. However, the
construction of such clean datasets is costly and time-consuming since the
Internet is overwhelmed by noisy images with inadequate and inaccurate tags. In
this paper, we propose a Ubiquitous Reweighting Network (URNet) that learns an
image classification model from large-scale noisy data. By observing the web
data, we find that there are five key challenges, i.e., imbalanced class sizes,
high intra-classes diversity and inter-class similarity, imprecise instances,
insufficient representative instances, and ambiguous class labels. To alleviate
these challenges, we assume that every training instance has the potential to
contribute positively by alleviating the data bias and noise via reweighting
the influence of each instance according to different class sizes, large
instance clusters, its confidence, small instance bags and the labels. In this
manner, the influence of bias and noise in the web data can be gradually
alleviated, leading to the steadily improving performance of URNet.
Experimental results in the WebVision 2018 challenge with 16 million noisy
training images from 5000 classes show that our approach outperforms
state-of-the-art models and ranks the first place in the image classification
task.
Sequence Generation with Guider Network
Authors: Ruiyi Zhang, Changyou Chen, Zhe Gan, Wenlin Wang, Liqun Chen, Dinghan Shen, Guoyin Wang, Lawrence Carin
Date: 2018-11-02 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

Sequence generation with reinforcement learning (RL) has received significant
attention recently. However, a challenge with such methods is the sparse-reward
problem in the RL training process, in which a scalar guiding signal is often
only available after an entire sequence has been generated. This type of sparse
reward tends to ignore the global structural information of a sequence, causing
generation of sequences that are semantically inconsistent. In this paper, we
present a model-based RL approach to overcome this issue. Specifically, we
propose a novel guider network to model the sequence-generation environment,
which can assist next-word prediction and provide intermediate rewards for
generator optimization. Extensive experiments show that the proposed method
leads to improved performance for both unconditional and conditional
sequence-generation tasks.
Meta-path Augmented Response Generation
Authors: Yan-ran Li, Wenjie Li
Date: 2018-11-02 00:00:00
Tasks: Chatbot, Response Generation

We propose a chatbot, namely Mocha to make good use of relevant entities when
generating responses. Augmented with meta-path information, Mocha is able to
mention proper entities following the conversation flow.
Zero-Shot Transfer VQA Dataset
Authors: Yuanpeng Li, Yi Yang, Jian-Yu Wang, Wei Xu
Date: 2018-11-02 00:00:00
Tasks: Question Answering, Transfer Learning, Visual Question Answering, Visual Question Answering (VQA)

Acquiring a large vocabulary is an important aspect of human intelligence.
Onecommon approach for human to populating vocabulary is to learn words
duringreading or listening, and then use them in writing or speaking. This
ability totransfer from input to output is natural for human, but it is
difficult for machines.Human spontaneously performs this knowledge transfer in
complicated multimodaltasks, such as Visual Question Answering (VQA). In order
to approach human-levelArtificial Intelligence, we hope to equip machines with
such ability. Therefore, toaccelerate this research, we propose a newzero-shot
transfer VQA(ZST-VQA)dataset by reorganizing the existing VQA v1.0 dataset in
the way that duringtraining, some words appear only in one module (i.e.
questions) but not in theother (i.e. answers). In this setting, an intelligent
model should understand andlearn the concepts from one module (i.e. questions),
and at test time, transfer themto the other (i.e. predict the concepts as
answers). We conduct evaluation on thisnew dataset using three existing
state-of-the-art VQA neural models. Experimentalresults show a significant drop
in performance on this dataset, indicating existingmethods do not address the
zero-shot transfer problem. Besides, our analysis findsthat this may be caused
by the implicit bias learned during training.
Data-driven Perception of Neuron Point Process with Unknown Unknowns
Authors: Ruochen Yang, Gaurav Gupta, Paul Bogdan
Date: 2018-11-02 00:00:00
Tasks: Activity Prediction, Time Series Analysis

Identification of patterns from discrete data time-series for statistical
inference, threat detection, social opinion dynamics, brain activity prediction
has received recent momentum. In addition to the huge data size, the associated
challenges are, for example, (i) missing data to construct a closed
time-varying complex network, and (ii) contribution of unknown sources which
are not probed. Towards this end, the current work focuses on statistical
neuron system model with multi-covariates and unknown inputs. Previous research
of neuron activity analysis is mainly limited with effects from the spiking
history of target neuron and the interaction with other neurons in the system
while ignoring the influence of unknown stimuli. We propose to use unknown
unknowns, which describes the effect of unknown stimuli, undetected neuron
activities and all other hidden sources of error. The maximum likelihood
estimation with the fixed-point iteration method is implemented. The
fixed-point iterations converge fast, and the proposed methods can be
efficiently parallelized and offer computational advantage especially when the
input spiking trains are over long time-horizon. The developed framework
provides an intuition into the meaning of having extra degrees-of-freedom in
the data to support the need for unknowns. The proposed algorithm is applied to
simulated spike trains and on real-world experimental data of mouse
somatosensory, mouse retina and cat retina. The model shows a successful
increasing of system likelihood with respect to the conditional intensity
function, and it also reveals the convergence with iterations. Results suggest
that the neural connection model with unknown unknowns can efficiently estimate
the statistical properties of the process by increasing the network likelihood.
Closed Form Variational Objectives For Bayesian Neural Networks with a Single Hidden Layer
Authors: Martin Jankowiak
Date: 2018-11-02 00:00:00
Tasks: Form, General Classification

In this note we consider setups in which variational objectives for Bayesian
neural networks can be computed in closed form. In particular we focus on
single-layer networks in which the activation function is piecewise polynomial
(e.g. ReLU). In this case we show that for a Normal likelihood and structured
Normal variational distributions one can compute a variational lower bound in
closed form. In addition we compute the predictive mean and variance in closed
form. Finally, we also show how to compute approximate lower bounds for other
likelihoods (e.g. softmax classification). In experiments we show how the
resulting variational objectives can help improve training and provide fast
test time predictions.
SDCNet: Video Prediction Using Spatially-Displaced Convolution
Authors: Fitsum A. Reda, Guilin Liu, Kevin J. Shih, Robert Kirby, Jon Barker, David Tarjan, Andrew Tao, Bryan Catanzaro
Date: 2018-11-02 00:00:00
Tasks: Optical Flow Estimation, Prediction, SSIM, Video Prediction

We present an approach for high-resolution video frame prediction by
conditioning on both past frames and past optical flows. Previous approaches
rely on resampling past frames, guided by a learned future optical flow, or on
direct generation of pixels. Resampling based on flow is insufficient because
it cannot deal with disocclusions. Generative models currently lead to blurry
results. Recent approaches synthesis a pixel by convolving input patches with a
predicted kernel. However, their memory requirement increases with kernel size.
Here, we spatially-displaced convolution (SDC) module for video frame
prediction. We learn a motion vector and a kernel for each pixel and synthesize
a pixel by applying the kernel at a displaced location in the source image,
defined by the predicted motion vector. Our approach inherits the merits of
both vector-based and kernel-based approaches, while ameliorating their
respective disadvantages. We train our model on 428K unlabelled 1080p video
game frames. Our approach produces state-of-the-art results, achieving an SSIM
score of 0.904 on high-definition YouTube-8M videos, 0.918 on Caltech
Pedestrian videos. Our model handles large motion effectively and synthesizes
crisp frames with consistent motion.
Quasi-random sampling for multivariate distributions via generative neural networks
Authors: Marius Hofert, Avinash Prasad, Mu Zhu
Date: 2018-11-01 00:00:00
Tasks: Management

Generative moment matching networks (GMMNs) are introduced for generating quasi-random samples from multivariate models with any underlying copula in order to compute estimates under variance reduction. So far, quasi-random sampling for multivariate distributions required a careful design, exploiting specific properties (such as conditional distributions) of the implied parametric copula or the underlying quasi-Monte Carlo (QMC) point set, and was only tractable for a small number of models. Utilizing GMMNs allows one to construct quasi-random samples for a much larger variety of multivariate distributions without such restrictions, including empirical ones from real data with dependence structures not well captured by parametric copulas. Once trained on pseudo-random samples from a parametric model or on real data, these neural networks only require a multivariate standard uniform randomized QMC point set as input and are thus fast in estimating expectations of interest under dependence with variance reduction. Numerical examples are considered to demonstrate the approach, including applications inspired by risk management practice. All results are reproducible with the demos GMMN_QMC_paper, GMMN_QMC_data and GMMN_QMC_timings as part of the R package gnn.
On the Generation of Medical Question-Answer Pairs
Authors: Sheng Shen, Yaliang Li, Nan Du, Xian Wu, Yusheng Xie, Shen Ge, Tao Yang, Kai Wang, Xingzheng Liang, Wei Fan
Date: 2018-11-01 00:00:00
Tasks: Decoder, Diversity, Question Answering

Question answering (QA) has achieved promising progress recently. However, answering a question in real-world scenarios like the medical domain is still challenging, due to the requirement of external knowledge and the insufficient quantity of high-quality training data. In the light of these challenges, we study the task of generating medical QA pairs in this paper. With the insight that each medical question can be considered as a sample from the latent distribution of questions given answers, we propose an automated medical QA pair generation framework, consisting of an unsupervised key phrase detector that explores unstructured material for validity, and a generator that involves a multi-pass decoder to integrate structural knowledge for diversity. A series of experiments have been conducted on a real-world dataset collected from the National Medical Licensing Examination of China. Both automatic evaluation and human annotation demonstrate the effectiveness of the proposed method. Further investigation shows that, by incorporating the generated QA pairs for training, significant improvement in terms of accuracy can be achieved for the examination QA system.
Analyzing different prototype selection techniques for dynamic classifier and ensemble selection
Authors: Rafael M. O. Cruz, Robert Sabourin, George D. C. Cavalcanti
Date: 2018-11-01 00:00:00
Tasks: Classification, General Classification, Prototype Selection

In dynamic selection (DS) techniques, only the most competent classifiers,
for the classification of a specific test sample are selected to predict the
sample's class labels. The more important step in DES techniques is estimating
the competence of the base classifiers for the classification of each specific
test sample. The classifiers' competence is usually estimated using the
neighborhood of the test sample defined on the validation samples, called the
region of competence. Thus, the performance of DS techniques is sensitive to
the distribution of the validation set. In this paper, we evaluate six
prototype selection techniques that work by editing the validation data in
order to remove noise and redundant instances. Experiments conducted using
several state-of-the-art DS techniques over 30 classification problems
demonstrate that by using prototype selection techniques we can improve the
classification accuracy of DS techniques and also significantly reduce the
computational cost involved.
Dialogue Natural Language Inference
Authors: Sean Welleck, Jason Weston, Arthur Szlam, Kyunghyun Cho
Date: 2018-11-01 00:00:00
Proceeding: ACL 2019 7
Tasks: Natural Language Inference

Consistency is a long standing issue faced by dialogue models. In this paper,
we frame the consistency of dialogue agents as natural language inference (NLI)
and create a new natural language inference dataset called Dialogue NLI. We
propose a method which demonstrates that a model trained on Dialogue NLI can be
used to improve the consistency of a dialogue model, and evaluate the method
with human evaluation and with automatic metrics on a suite of evaluation sets
designed to measure a dialogue model's consistency.