pwc-archive/papers-with-abstracts huggingface dataset:
Learning regression and verification networks for long-term visual tracking
Authors: Yunhua Zhang, Dong Wang, Lijun Wang, Jinqing Qi, Huchuan Lu
Date: 2018-09-12 00:00:00
Tasks: General Classification, Object, Region Proposal, regression, Visual Tracking

Compared with short-term tracking, the long-term tracking task requires
determining the tracked object is present or absent, and then estimating the
accurate bounding box if present or conducting image-wide re-detection if
absent. Until now, few attempts have been done although this task is much
closer to designing practical tracking systems. In this work, we propose a
novel long-term tracking framework based on deep regression and verification
networks. The offline-trained regression model is designed using the
object-aware feature fusion and region proposal networks to generate a series
of candidates and estimate their similarity scores effectively. The
verification network evaluates these candidates to output the optimal one as
the tracked object with its classification score, which is online updated to
adapt to the appearance variations based on newly reliable observations. The
similarity and classification scores are combined to obtain a final confidence
value, based on which our tracker can determine the absence of the target
accurately and conduct image-wide re-detection to capture the target
successfully when it reappears. Extensive experiments show that our tracker
achieves the best performance on the VOT2018 long-term challenge and
state-of-the-art results on the OxUvA long-term dataset.
Neural Melody Composition from Lyrics
Authors: Hangbo Bao, Shaohan Huang, Furu Wei, Lei Cui, Yu Wu, Chuanqi Tan, Songhao Piao, Ming Zhou
Date: 2018-09-12 00:00:00
Tasks: Decoder

In this paper, we study a novel task that learns to compose music from
natural language. Given the lyrics as input, we propose a melody composition
model that generates lyrics-conditional melody as well as the exact alignment
between the generated melody and the given lyrics simultaneously. More
specifically, we develop the melody composition model based on the
sequence-to-sequence framework. It consists of two neural encoders to encode
the current lyrics and the context melody respectively, and a hierarchical
decoder to jointly produce musical notes and the corresponding alignment.
Experimental results on lyrics-melody pairs of 18,451 pop songs demonstrate the
effectiveness of our proposed methods. In addition, we apply a singing voice
synthesizer software to synthesize the "singing" of the lyrics and melodies for
human evaluation. Results indicate that our generated melodies are more
melodious and tuneful compared with the baseline method.
Chinese Poetry Generation with a Salient-Clue Mechanism
Authors: Xiaoyuan Yi, Ruoyu Li, Maosong Sun
Date: 2018-09-12 00:00:00
Proceeding: CONLL 2018 10

As a precious part of the human cultural heritage, Chinese poetry has
influenced people for generations. Automatic poetry composition is a challenge
for AI. In recent years, significant progress has been made in this area
benefiting from the development of neural networks. However, the coherence in
meaning, theme or even artistic conception for a generated poem as a whole
still remains a big problem. In this paper, we propose a novel Salient-Clue
mechanism for Chinese poetry generation. Different from previous work which
tried to exploit all the context information, our model selects the most
salient characters automatically from each so-far generated line to gradually
form a salient clue, which is utilized to guide successive poem generation
process so as to eliminate interruptions and improve coherence. Besides, our
model can be flexibly extended to control the generated poem in different
aspects, for example, poetry style, which further enhances the coherence.
Experimental results show that our model is very effective, outperforming three
strong baselines.
Chinese Poetry Generation with a Working Memory Model
Authors: Xiaoyuan Yi, Maosong Sun, Ruoyu Li, Zonghan Yang
Date: 2018-09-12 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction, model

As an exquisite and concise literary form, poetry is a gem of human culture.
Automatic poetry generation is an essential step towards computer creativity.
In recent years, several neural models have been designed for this task.
However, among lines of a whole poem, the coherence in meaning and topics still
remains a big challenge. In this paper, inspired by the theoretical concept in
cognitive psychology, we propose a novel Working Memory model for poetry
generation. Different from previous methods, our model explicitly maintains
topics and informative limited history in a neural memory. During the
generation process, our model reads the most relevant parts from memory slots
to generate the current line. After each line is generated, it writes the most
salient parts of the previous line into memory slots. By dynamic manipulation
of the memory, our model keeps a coherent information flow and learns to
express each topic flexibly and naturally. We experiment on three different
genres of Chinese poetry: quatrain, iambic and chinoiserie lyric. Both
automatic and human evaluation results show that our model outperforms current
state-of-the-art methods.
Cluster Variational Approximations for Structure Learning of Continuous-Time Bayesian Networks from Incomplete Data
Authors: Dominik Linzner, Heinz Koeppl
Date: 2018-09-12 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Time Series, Time Series Analysis

Continuous-time Bayesian networks (CTBNs) constitute a general and powerful
framework for modeling continuous-time stochastic processes on networks. This
makes them particularly attractive for learning the directed structures among
interacting entities. However, if the available data is incomplete, one needs
to simulate the prohibitively complex CTBN dynamics. Existing approximation
techniques, such as sampling and low-order variational methods, either scale
unfavorably in system size, or are unsatisfactory in terms of accuracy.
Inspired by recent advances in statistical physics, we present a new
approximation scheme based on cluster-variational methods significantly
improving upon existing variational approximations. We can analytically
marginalize the parameters of the approximate CTBN, as these are of secondary
importance for structure learning. This recovers a scalable scheme for direct
structure learning from incomplete and noisy time-series data. Our approach
outperforms existing methods in terms of scalability.
Deep Learning Based Multi-modal Addressee Recognition in Visual Scenes with Utterances
Authors: Thao Minh Le, Nobuyuki Shimizu, Takashi Miyazaki, Koichi Shinoda
Date: 2018-09-12 00:00:00

With the widespread use of intelligent systems, such as smart speakers,
addressee recognition has become a concern in human-computer interaction, as
more and more people expect such systems to understand complicated social
scenes, including those outdoors, in cafeterias, and hospitals. Because
previous studies typically focused only on pre-specified tasks with limited
conversational situations such as controlling smart homes, we created a mock
dataset called Addressee Recognition in Visual Scenes with Utterances (ARVSU)
that contains a vast body of image variations in visual scenes with an
annotated utterance and a corresponding addressee for each scenario. We also
propose a multi-modal deep-learning-based model that takes different human
cues, specifically eye gazes and transcripts of an utterance corpus, into
account to predict the conversational addressee from a specific speaker's view
in various real-life conversational scenarios. To the best of our knowledge, we
are the first to introduce an end-to-end deep learning model that combines
vision and transcripts of utterance for addressee recognition. As a result, our
study suggests that future addressee recognition can reach the ability to
understand human intention in many social situations previously unexplored, and
our modality dataset is a first step in promoting research in this field.
Incorporating Syntactic and Semantic Information in Word Embeddings using Graph Convolutional Networks
Authors: Shikhar Vashishth, Manik Bhandari, Prateek Yadav, Piyush Rai, Chiranjib Bhattacharyya, Partha Talukdar
Date: 2018-09-12 00:00:00
Proceeding: ACL 2019 7
Tasks: Learning Word Embeddings, Representation Learning, Word Embeddings

Word embeddings have been widely adopted across several NLP applications. Most existing word embedding methods utilize sequential context of a word to learn its embedding. While there have been some attempts at utilizing syntactic context of a word, such methods result in an explosion of the vocabulary size. In this paper, we overcome this problem by proposing SynGCN, a flexible Graph Convolution based method for learning word embeddings. SynGCN utilizes the dependency context of a word without increasing the vocabulary size. Word embeddings learned by SynGCN outperform existing methods on various intrinsic and extrinsic tasks and provide an advantage when used with ELMo. We also propose SemGCN, an effective framework for incorporating diverse semantic knowledge for further enhancing learned word representations. We make the source code of both models available to encourage reproducible research.
Joint Segmentation and Uncertainty Visualization of Retinal Layers in Optical Coherence Tomography Images using Bayesian Deep Learning
Authors: Suman Sedai, Bhavna Antony, Dwarikanath Mahapatra, Rahil Garnavi
Date: 2018-09-12 00:00:00
Tasks: Segmentation, Uncertainty Visualization

Optical coherence tomography (OCT) is commonly used to analyze retinal layers
for assessment of ocular diseases. In this paper, we propose a method for
retinal layer segmentation and quantification of uncertainty based on Bayesian
deep learning. Our method not only performs end-to-end segmentation of retinal
layers, but also gives the pixel wise uncertainty measure of the segmentation
output. The generated uncertainty map can be used to identify erroneously
segmented image regions which is useful in downstream analysis. We have
validated our method on a dataset of 1487 images obtained from 15 subjects (OCT
volumes) and compared it against the state-of-the-art segmentation algorithms
that does not take uncertainty into account. The proposed uncertainty based
segmentation method results in comparable or improved performance, and most
importantly is more robust against noise.
Music Transformer
Authors: Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, Douglas Eck
Date: 2018-09-12 00:00:00
Proceeding: ICLR 2019 5
Tasks: Music Generation, Music Modeling

Music relies heavily on repetition to build structure and meaning.
Self-reference occurs on multiple timescales, from motifs to phrases to reusing
of entire sections of music, such as in pieces with ABA structure. The
Transformer (Vaswani et al., 2017), a sequence model based on self-attention,
has achieved compelling results in many generation tasks that require
maintaining long-range coherence. This suggests that self-attention might also
be well-suited to modeling music. In musical composition and performance,
however, relative timing is critically important. Existing approaches for
representing relative positional information in the Transformer modulate
attention based on pairwise distance (Shaw et al., 2018). This is impractical
for long sequences such as musical compositions since their memory complexity
for intermediate relative information is quadratic in the sequence length. We
propose an algorithm that reduces their intermediate memory requirement to
linear in the sequence length. This enables us to demonstrate that a
Transformer with our modified relative attention mechanism can generate
minute-long compositions (thousands of steps, four times the length modeled in
Oore et al., 2018) with compelling structure, generate continuations that
coherently elaborate on a given motif, and in a seq2seq setup generate
accompaniments conditioned on melodies. We evaluate the Transformer with our
relative attention mechanism on two datasets, JSB Chorales and
Piano-e-Competition, and obtain state-of-the-art results on the latter.
Safe Navigation with Human Instructions in Complex Scenes
Authors: Zhe Hu, Jia Pan, Tingxiang Fan, Ruigang Yang, Dinesh Manocha
Date: 2018-09-12 00:00:00
Tasks: Collision Avoidance, Motion Planning, object-detection, Object Detection

In this paper, we present a robotic navigation algorithm with natural
language interfaces, which enables a robot to safely walk through a changing
environment with moving persons by following human instructions such as "go to
the restaurant and keep away from people". We first classify human instructions
into three types: the goal, the constraints, and uninformative phrases. Next,
we provide grounding for the extracted goal and constraint items in a dynamic
manner along with the navigation process, to deal with the target objects that
are too far away for sensor observation and the appearance of moving obstacles
like humans. In particular, for a goal phrase (e.g., "go to the restaurant"),
we ground it to a location in a predefined semantic map and treat it as a goal
for a global motion planner, which plans a collision-free path in the workspace
for the robot to follow. For a constraint phrase (e.g., "keep away from
people"), we dynamically add the corresponding constraint into a local planner
by adjusting the values of a local costmap according to the results returned by
the object detection module. The updated costmap is then used to compute a
local collision avoidance control for the safe navigation of the robot. By
combining natural language processing, motion planning, and computer vision,
our developed system is demonstrated to be able to successfully follow natural
language navigation instructions to achieve navigation tasks in both simulated
and real-world scenarios. Videos are available at
https://sites.google.com/view/snhi
Discretely Relaxing Continuous Variables for tractable Variational Inference
Authors: Trefor W. Evans, Prasanth B. Nair
Date: 2018-09-12 00:00:00
Proceeding: NeurIPS 2018
Tasks: Variational Inference

We explore a new research direction in Bayesian variational inference with
discrete latent variable priors where we exploit Kronecker matrix algebra for
efficient and exact computations of the evidence lower bound (ELBO). The
proposed "DIRECT" approach has several advantages over its predecessors; (i) it
can exactly compute ELBO gradients (i.e. unbiased, zero-variance gradient
estimates), eliminating the need for high-variance stochastic gradient
estimators and enabling the use of quasi-Newton optimization methods; (ii) its
training complexity is independent of the number of training points, permitting
inference on large datasets; and (iii) its posterior samples consist of sparse
and low-precision quantized integers which permit fast inference on hardware
limited devices. In addition, our DIRECT models can exactly compute statistical
moments of the parameterized predictive posterior without relying on Monte
Carlo sampling. The DIRECT approach is not practical for all likelihoods,
however, we identify a popular model structure which is practical, and
demonstrate accurate inference using latent variables discretized as extremely
low-precision 4-bit quantized integers. While the ELBO computations considered
in the numerical studies require over $10^{2352}$ log-likelihood evaluations,
we train on datasets with over two-million points in just seconds.
Retrieval-Enhanced Adversarial Training for Neural Response Generation
Authors: Qingfu Zhu, Lei Cui, Wei-Nan Zhang, Furu Wei, Ting Liu
Date: 2018-09-12 00:00:00
Proceeding: ACL 2019 7
Tasks: Decoder, Response Generation, Retrieval

Dialogue systems are usually built on either generation-based or retrieval-based approaches, yet they do not benefit from the advantages of different models. In this paper, we propose a Retrieval-Enhanced Adversarial Training (REAT) method for neural response generation. Distinct from existing approaches, the REAT method leverages an encoder-decoder framework in terms of an adversarial training paradigm, while taking advantage of N-best response candidates from a retrieval-based system to construct the discriminator. An empirical study on a large scale public available benchmark dataset shows that the REAT method significantly outperforms the vanilla Seq2Seq model as well as the conventional adversarial training approach.
Knowledge-Aware Conversational Semantic Parsing Over Web Tables
Authors: Yibo Sun, Duyu Tang, Nan Duan, Jingjing Xu, Xiaocheng Feng, Bing Qin
Date: 2018-09-12 00:00:00
Tasks: Entity Typing, Language Modeling, Language Modelling, Semantic Parsing

Conversational semantic parsing over tables requires knowledge acquiring and
reasoning abilities, which have not been well explored by current
state-of-the-art approaches. Motivated by this fact, we propose a
knowledge-aware semantic parser to improve parsing performance by integrating
various types of knowledge. In this paper, we consider three types of
knowledge, including grammar knowledge, expert knowledge, and external resource
knowledge. First, grammar knowledge empowers the model to effectively replicate
previously generated logical form, which effectively handles the co-reference
and ellipsis phenomena in conversation Second, based on expert knowledge, we
propose a decomposable model, which is more controllable compared with
traditional end-to-end models that put all the burdens of learning on
trial-and-error in an end-to-end way. Third, external resource knowledge, i.e.,
provided by a pre-trained language model or an entity typing model, is used to
improve the representation of question and table for a better semantic
understanding. We conduct experiments on the SequentialQA dataset. Results show
that our knowledge-aware model outperforms the state-of-the-art approaches.
Incremental experimental results also prove the usefulness of various
knowledge. Further analysis shows that our approach has the ability to derive
the meaning representation of a context-dependent utterance by leveraging
previously generated outcomes.
MotherNets: Rapid Deep Ensemble Learning
Authors: Abdul Wasay, Brian Hentschel, Yuze Liao, Sanyuan Chen, Stratos Idreos
Date: 2018-09-12 00:00:00
Tasks: Clustering, Clustering Ensemble, Ensemble Learning, Knowledge Distillation

Ensembles of deep neural networks significantly improve generalization accuracy. However, training neural network ensembles requires a large amount of computational resources and time. State-of-the-art approaches either train all networks from scratch leading to prohibitive training cost that allows only very small ensemble sizes in practice, or generate ensembles by training a monolithic architecture, which results in lower model diversity and decreased prediction accuracy. We propose MotherNets to enable higher accuracy and practical training cost for large and diverse neural network ensembles: A MotherNet captures the structural similarity across some or all members of a deep neural network ensemble which allows us to share data movement and computation costs across these networks. We first train a single or a small set of MotherNets and, subsequently, we generate the target ensemble networks by transferring the function from the trained MotherNet(s). Then, we continue to train these ensemble networks, which now converge drastically faster compared to training from scratch. MotherNets handle ensembles with diverse architectures by clustering ensemble networks of similar architecture and training a separate MotherNet for every cluster. MotherNets also use clustering to control the accuracy vs. training cost tradeoff. We show that compared to state-of-the-art approaches such as Snapshot Ensembles, Knowledge Distillation, and TreeNets, MotherNets provide a new Pareto frontier for the accuracy-training cost tradeoff. Crucially, training cost and accuracy improvements continue to scale as we increase the ensemble size (2 to 3 percent reduced absolute test error rate and up to 35 percent faster training compared to Snapshot Ensembles). We verify these benefits over numerous neural network architectures and large data sets.
Knowledge Based Machine Reading Comprehension
Authors: Yibo Sun, Daya Guo, Duyu Tang, Nan Duan, Zhao Yan, Xiaocheng Feng, Bing Qin
Date: 2018-09-12 00:00:00
Tasks: Machine Reading Comprehension, Question Answering, Question Generation, Question-Generation, Reading Comprehension

Machine reading comprehension (MRC) requires reasoning about both the
knowledge involved in a document and knowledge about the world. However,
existing datasets are typically dominated by questions that can be well solved
by context matching, which fail to test this capability. To encourage the
progress on knowledge-based reasoning in MRC, we present knowledge-based MRC in
this paper, and build a new dataset consisting of 40,047 question-answer pairs.
The annotation of this dataset is designed so that successfully answering the
questions requires understanding and the knowledge involved in a document. We
implement a framework consisting of both a question answering model and a
question generation model, both of which take the knowledge extracted from the
document as well as relevant facts from an external knowledge base such as
Freebase/ProBase/Reverb/NELL. Results show that incorporating side information
from external KB improves the accuracy of the baseline question answer system.
We compare it with a standard MRC model BiDAF, and also provide the difficulty
of the dataset and lay out remaining challenges.
Extracting Fairness Policies from Legal Documents
Authors: Rashmi Nagpal, Chetna Wadhwa, Mallika Gupta, Samiulla Shaikh, Sameep Mehta, Vikram Goyal
Date: 2018-09-12 00:00:00
Tasks: BIG-bench Machine Learning, Fairness

Machine Learning community is recently exploring the implications of bias and
fairness with respect to the AI applications. The definition of fairness for
such applications varies based on their domain of application. The policies
governing the use of such machine learning system in a given context are
defined by the constitutional laws of nations and regulatory policies enforced
by the organizations that are involved in the usage. Fairness related laws and
policies are often spread across the large documents like constitution,
agreements, and organizational regulations. These legal documents have long
complex sentences in order to achieve rigorousness and robustness. Automatic
extraction of fairness policies, or in general, any specific kind of policies
from large legal corpus can be very useful for the study of bias and fairness
in the context of AI applications.
  We attempted to automatically extract fairness policies from publicly
available law documents using two approaches based on semantic relatedness. The
experiments reveal how classical Wordnet-based similarity and vector-based
similarity differ in addressing this task. We have shown that similarity based
on word vectors beats the classical approach with a large margin, whereas other
vector representations of senses and sentences fail to even match the classical
baseline. Further, we have presented thorough error analysis and reasoning to
explain the results with appropriate examples from the dataset for deeper
insights.