pwc-archive/papers-with-abstracts huggingface dataset:
Hierarchical Attention: What Really Counts in Various NLP Tasks
Authors: Zehao Dou, Zhihua Zhang
Date: 2018-08-10 00:00:00
Tasks: Machine Reading Comprehension, Machine Translation, Reading Comprehension, Sentence, Sentence Embedding, Sentence-Embedding, Text Generation, Translation

Attention mechanisms in sequence to sequence models have shown great ability
and wonderful performance in various natural language processing (NLP) tasks,
such as sentence embedding, text generation, machine translation, machine
reading comprehension, etc. Unfortunately, existing attention mechanisms only
learn either high-level or low-level features. In this paper, we think that the
lack of hierarchical mechanisms is a bottleneck in improving the performance of
the attention mechanisms, and propose a novel Hierarchical Attention Mechanism
(Ham) based on the weighted sum of different layers of a multi-level attention.
Ham achieves a state-of-the-art BLEU score of 0.26 on Chinese poem generation
task and a nearly 6.5% averaged improvement compared with the existing machine
reading comprehension models such as BIDAF and Match-LSTM. Furthermore, our
experiments and theorems reveal that Ham has greater generalization and
representation ability than existing attention mechanisms.
Learning to Represent Bilingual Dictionaries
Authors: Muhao Chen, Yingtao Tian, Haochen Chen, Kai-Wei Chang, Steven Skiena, Carlo Zaniolo
Date: 2018-08-10 00:00:00
Proceeding: CONLL 2019 11
Tasks: Multi-Task Learning, Paraphrase Identification, Question Answering, Retrieval, Reverse Dictionary, Sentence, Word Embeddings

Bilingual word embeddings have been widely used to capture the similarity of lexical semantics in different human languages. However, many applications, such as cross-lingual semantic search and question answering, can be largely benefited from the cross-lingual correspondence between sentences and lexicons. To bridge this gap, we propose a neural embedding model that leverages bilingual dictionaries. The proposed model is trained to map the literal word definitions to the cross-lingual target words, for which we explore with different sentence encoding techniques. To enhance the learning process on limited resources, our model adopts several critical learning strategies, including multi-task learning on different bridges of languages, and joint learning of the dictionary model with a bilingual word embedding model. Experimental evaluation focuses on two applications. The results of the cross-lingual reverse dictionary retrieval task show our model's promising ability of comprehending bilingual concepts based on descriptions, and highlight the effectiveness of proposed learning strategies in improving performance. Meanwhile, our model effectively addresses the bilingual paraphrase identification problem and significantly outperforms previous approaches.
This Time with Feeling: Learning Expressive Musical Performance
Authors: Sageev Oore, Ian Simon, Sander Dieleman, Douglas Eck, Karen Simonyan
Date: 2018-08-10 00:00:00
Tasks: Music Generation

Music generation has generally been focused on either creating scores or
interpreting them. We discuss differences between these two problems and
propose that, in fact, it may be valuable to work in the space of direct $\it
performance$ generation: jointly predicting the notes $\it and$ $\it also$
their expressive timing and dynamics. We consider the significance and
qualities of the data set needed for this. Having identified both a problem
domain and characteristics of an appropriate data set, we show an LSTM-based
recurrent network model that subjectively performs quite well on this task.
Critically, we provide generated examples. We also include feedback from
professional composers and musicians about some of these examples.
Unsupervised Keyphrase Extraction from Scientific Publications
Authors: Eirini Papagiannopoulou, Grigorios Tsoumakas
Date: 2018-08-10 00:00:00
Tasks: Keyphrase Extraction, Outlier Detection, Word Embeddings

We propose a novel unsupervised keyphrase extraction approach that filters candidate keywords using outlier detection. It starts by training word embeddings on the target document to capture semantic regularities among the words. It then uses the minimum covariance determinant estimator to model the distribution of non-keyphrase word vectors, under the assumption that these vectors come from the same distribution, indicative of their irrelevance to the semantics expressed by the dimensions of the learned vector representation. Candidate keyphrases only consist of words that are detected as outliers of this dominant distribution. Empirical results show that our approach outperforms state-of-the-art and recent unsupervised keyphrase extraction methods.
LemmaTag: Jointly Tagging and Lemmatizing for Morphologically-Rich Languages with BRNNs
Authors: Daniel Kondratyuk, Tomáš Gavenčiak, Milan Straka, Jan Hajič
Date: 2018-08-10 00:00:00
Tasks: Lemmatization, Part-Of-Speech Tagging, TAG

We present LemmaTag, a featureless neural network architecture that jointly
generates part-of-speech tags and lemmas for sentences by using bidirectional
RNNs with character-level and word-level embeddings. We demonstrate that both
tasks benefit from sharing the encoding part of the network, predicting tag
subcategories, and using the tagger output as an input to the lemmatizer. We
evaluate our model across several languages with complex morphology, which
surpasses state-of-the-art accuracy in both part-of-speech tagging and
lemmatization in Czech, German, and Arabic.
BooST: Boosting Smooth Trees for Partial Effect Estimation in Nonlinear Regressions
Authors: Yuri Fonseca, Marcelo Medeiros, Gabriel Vasconcelos, Alvaro Veiga
Date: 2018-08-10 00:00:00
Tasks: BIG-bench Machine Learning, regression

In this paper, we introduce a new machine learning (ML) model for nonlinear regression called the Boosted Smooth Transition Regression Trees (BooST), which is a combination of boosting algorithms with smooth transition regression trees. The main advantage of the BooST model is the estimation of the derivatives (partial effects) of very general nonlinear models. Therefore, the model can provide more interpretation about the mapping between the covariates and the dependent variable than other tree-based models, such as Random Forests. We present several examples with both simulated and real data.
Partial Adversarial Domain Adaptation
Authors: Zhangjie Cao, Lijia Ma, Mingsheng Long, Jian-Min Wang
Date: 2018-08-10 00:00:00
Proceeding: ECCV 2018 9
Tasks: Domain Adaptation, Partial Domain Adaptation

Domain adversarial learning aligns the feature distributions across the
source and target domains in a two-player minimax game. Existing domain
adversarial networks generally assume identical label space across different
domains. In the presence of big data, there is strong motivation of
transferring deep models from existing big domains to unknown small domains.
This paper introduces partial domain adaptation as a new domain adaptation
scenario, which relaxes the fully shared label space assumption to that the
source label space subsumes the target label space. Previous methods typically
match the whole source domain to the target domain, which are vulnerable to
negative transfer for the partial domain adaptation problem due to the large
mismatch between label spaces. We present Partial Adversarial Domain Adaptation
(PADA), which simultaneously alleviates negative transfer by down-weighing the
data of outlier source classes for training both source classifier and domain
adversary, and promotes positive transfer by matching the feature distributions
in the shared label space. Experiments show that PADA exceeds state-of-the-art
results for partial domain adaptation tasks on several datasets.
Homophonic Quotients of Linguistic Free Groups: German, Korean, and Turkish
Authors: Herbert Gangl, Gizem Karaali, Woohyung Lee
Date: 2018-08-10 00:00:00

In 1993, the homophonic quotient groups for French and English (the quotient
of the free group generated by the French (respectively English) alphabet
determined by relations representing standard pronunciation rules) were
explicitly characterized [5]. In this paper we apply the same methodology to
three different language systems: German, Korean, and Turkish. We argue that
our results point to some interesting differences between these three languages
(or at least their current script systems).
Effective Unsupervised Author Disambiguation with Relative Frequencies
Authors: Tobias Backes
Date: 2018-08-10 00:00:00
Tasks: Clustering

This work addresses the problem of author name homonymy in the Web of
Science. Aiming for an efficient, simple and straightforward solution, we
introduce a novel probabilistic similarity measure for author name
disambiguation based on feature overlap. Using the researcher-ID available for
a subset of the Web of Science, we evaluate the application of this measure in
the context of agglomeratively clustering author mentions. We focus on a
concise evaluation that shows clearly for which problem setups and at which
time during the clustering process our approach works best. In contrast to most
other works in this field, we are sceptical towards the performance of author
name disambiguation methods in general and compare our approach to the trivial
single-cluster baseline. Our results are presented separately for each correct
clustering size as we can explain that, when treating all cases together, the
trivial baseline and more sophisticated approaches are hardly distinguishable
in terms of evaluation results. Our model shows state-of-the-art performance
for all correct clustering sizes without any discriminative training and with
tuning only one convergence parameter.
Connectivity-Driven Brain Parcellation via Consensus Clustering
Authors: Anvar Kurmukov, Ayagoz Mussabayeva, Yulia Denisova, Daniel Moyer, Boris Gutman
Date: 2018-08-10 00:00:00
Tasks: Clustering

We present two related methods for deriving connectivity-based brain atlases
from individual connectomes. The proposed methods exploit a previously proposed
dense connectivity representation, termed continuous connectivity, by first
performing graph-based hierarchical clustering of individual brains, and
subsequently aggregating the individual parcellations into a consensus
parcellation. The search for consensus minimizes the sum of cluster membership
distances, effectively estimating a pseudo-Karcher mean of individual
parcellations. We assess the quality of our parcellations using (1)
Kullback-Liebler and Jensen-Shannon divergence with respect to the dense
connectome representation, (2) inter-hemispheric symmetry, and (3) performance
of the simplified connectome in a biological sex classification task. We find
that the parcellation based-atlas computed using a greedy search at a
hierarchical depth 3 outperforms all other parcellation-based atlases as well
as the standard Dessikan-Killiany anatomical atlas in all three assessments.
Unsupervised Learning of Sentence Representations Using Sequence Consistency
Authors: Siddhartha Brahma
Date: 2018-08-10 00:00:00
Tasks: Form, Sentence, Transfer Learning

Computing universal distributed representations of sentences is a fundamental
task in natural language processing. We propose ConsSent, a simple yet
surprisingly powerful unsupervised method to learn such representations by
enforcing consistency constraints on sequences of tokens. We consider two
classes of such constraints -- sequences that form a sentence and between two
sequences that form a sentence when merged. We learn sentence encoders by
training them to distinguish between consistent and inconsistent examples, the
latter being generated by randomly perturbing consistent examples in six
different ways. Extensive evaluation on several transfer learning and
linguistic probing tasks shows improved performance over strong unsupervised
and supervised baselines, substantially surpassing them in several cases. Our
best results are achieved by training sentence encoders in a multitask setting
and by an ensemble of encoders trained on the individual tasks.
Model Reduction with Memory and the Machine Learning of Dynamical Systems
Authors: Chao Ma, Jianchun Wang, Weinan E
Date: 2018-08-10 00:00:00
Tasks: BIG-bench Machine Learning

The well-known Mori-Zwanzig theory tells us that model reduction leads to
memory effect. For a long time, modeling the memory effect accurately and
efficiently has been an important but nearly impossible task in developing a
good reduced model. In this work, we explore a natural analogy between
recurrent neural networks and the Mori-Zwanzig formalism to establish a
systematic approach for developing reduced models with memory. Two training
models-a direct training model and a dynamically coupled training model-are
proposed and compared. We apply these methods to the Kuramoto-Sivashinsky
equation and the Navier-Stokes equation. Numerical experiments show that the
proposed method can produce reduced model with good performance on both
short-term prediction and long-term statistical properties.
CT Super-resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble(GAN-CIRCLE)
Authors: Chenyu You, Guang Li, Yi Zhang, Xiaoliu Zhang, Hongming Shan, Shenghong Ju, Zhen Zhao, Zhuiyang Zhang, Wenxiang Cong, Michael W. Vannier, Punam K. Saha, Ge Wang
Date: 2018-08-10 00:00:00
Tasks: Computed Tomography (CT), Generative Adversarial Network, Image Restoration, Super-Resolution

Computed tomography (CT) is widely used in screening, diagnosis, and
image-guided therapy for both clinical and research purposes. Since CT involves
ionizing radiation, an overarching thrust of related technical research is
development of novel methods enabling ultrahigh quality imaging with fine
structural details while reducing the X-ray radiation. In this paper, we
present a semi-supervised deep learning approach to accurately recover
high-resolution (HR) CT images from low-resolution (LR) counterparts.
Specifically, with the generative adversarial network (GAN) as the building
block, we enforce the cycle-consistency in terms of the Wasserstein distance to
establish a nonlinear end-to-end mapping from noisy LR input images to denoised
and deblurred HR outputs. We also include the joint constraints in the loss
function to facilitate structural preservation. In this deep imaging process,
we incorporate deep convolutional neural network (CNN), residual learning, and
network in network techniques for feature extraction and restoration. In
contrast to the current trend of increasing network depth and complexity to
boost the CT imaging performance, which limit its real-world applications by
imposing considerable computational and memory overheads, we apply a parallel
$1\times1$ CNN to compress the output of the hidden layer and optimize the
number of layers and the number of filters for each convolutional layer.
Quantitative and qualitative evaluations demonstrate that our proposed model is
accurate, efficient and robust for super-resolution (SR) image restoration from
noisy LR input images. In particular, we validate our composite SR networks on
three large-scale CT datasets, and obtain promising results as compared to the
other state-of-the-art methods.
Relational dynamic memory networks
Authors: Trang Pham, Truyen Tran, Svetha Venkatesh
Date: 2018-08-10 00:00:00

Neural networks excel in detecting regular patterns but are less successful
in representing and manipulating complex data structures, possibly due to the
lack of an external memory. This has led to the recent development of a new
line of architectures known as Memory-Augmented Neural Networks (MANNs), each
of which consists of a neural network that interacts with an external memory
matrix. However, this RAM-like memory matrix is unstructured and thus does not
naturally encode structured objects. Here we design a new MANN dubbed
Relational Dynamic Memory Network (RMDN) to bridge the gap. Like existing
MANNs, RMDN has a neural controller but its memory is structured as
multi-relational graphs. RMDN uses the memory to represent and manipulate
graph-structured data in response to query; and as a neural network, RMDN is
trainable from labeled data. Thus RMDN learns to answer queries about a set of
graph-structured objects without explicit programming. We evaluate the
capability of RMDN on several important prediction problems, including software
vulnerability, molecular bioactivity and chemical-chemical interaction. Results
demonstrate the efficacy of the proposed model.
Affect Estimation in 3D Space Using Multi-Task Active Learning for Regression
Authors: Dongrui Wu, Jian Huang
Date: 2018-08-08 00:00:00
Tasks: Active Learning, regression

Acquisition of labeled training samples for affective computing is usually
costly and time-consuming, as affects are intrinsically subjective, subtle and
uncertain, and hence multiple human assessors are needed to evaluate each
affective sample. Particularly, for affect estimation in the 3D space of
valence, arousal and dominance, each assessor has to perform the evaluations in
three dimensions, which makes the labeling problem even more challenging. Many
sophisticated machine learning approaches have been proposed to reduce the data
labeling requirement in various other domains, but so far few have considered
affective computing. This paper proposes two multi-task active learning for
regression approaches, which select the most beneficial samples to label, by
considering the three affect primitives simultaneously. Experimental results on
the VAM corpus demonstrated that our optimal sample selection approaches can
result in better estimation performance than random selection and several
traditional single-task active learning approaches. Thus, they can help
alleviate the data labeling problem in affective computing, i.e., better
estimation performance can be obtained from fewer labeling queries.
Active Learning for Regression Using Greedy Sampling
Authors: Dongrui Wu, Chin-Teng Lin, Jian Huang
Date: 2018-08-08 00:00:00
Tasks: Active Learning, Diversity, EEG, Electroencephalogram (EEG), regression

Regression problems are pervasive in real-world applications. Generally a
substantial amount of labeled samples are needed to build a regression model
with good generalization ability. However, many times it is relatively easy to
collect a large number of unlabeled samples, but time-consuming or expensive to
label them. Active learning for regression (ALR) is a methodology to reduce the
number of labeled samples, by selecting the most beneficial ones to label,
instead of random selection. This paper proposes two new ALR approaches based
on greedy sampling (GS). The first approach (GSy) selects new samples to
increase the diversity in the output space, and the second (iGS) selects new
samples to increase the diversity in both input and output spaces. Extensive
experiments on 12 UCI and CMU StatLib datasets from various domains, and on 15
subjects on EEG-based driver drowsiness estimation, verified their
effectiveness and robustness.
Improving Generalization via Scalable Neighborhood Component Analysis
Authors: Zhirong Wu, Alexei A. Efros, Stella X. Yu
Date: 2018-08-14 00:00:00
Proceeding: ECCV 2018 9

Current major approaches to visual recognition follow an end-to-end
formulation that classifies an input image into one of the pre-determined set
of semantic categories. Parametric softmax classifiers are a common choice for
such a closed world with fixed categories, especially when big labeled data is
available during training. However, this becomes problematic for open-set
scenarios where new categories are encountered with very few examples for
learning a generalizable parametric classifier. We adopt a non-parametric
approach for visual recognition by optimizing feature embeddings instead of
parametric classifiers. We use a deep neural network to learn the visual
feature that preserves the neighborhood structure in the semantic space, based
on the Neighborhood Component Analysis (NCA) criterion. Limited by its
computational bottlenecks, we devise a mechanism to use augmented memory to
scale NCA for large datasets and very deep networks. Our experiments deliver
not only remarkable performance on ImageNet classification for such a simple
non-parametric method, but most importantly a more generalizable feature
representation for sub-category discovery and few-shot recognition.
Multispectral Pedestrian Detection via Simultaneous Detection and Segmentation
Authors: Chengyang Li, Dan Song, Ruofeng Tong, Min Tang
Date: 2018-08-14 00:00:00
Tasks: Autonomous Driving, Multispectral Object Detection, Pedestrian Detection, Semantic Segmentation

Multispectral pedestrian detection has attracted increasing attention from
the research community due to its crucial competence for many around-the-clock
applications (e.g., video surveillance and autonomous driving), especially
under insufficient illumination conditions. We create a human baseline over the
KAIST dataset and reveal that there is still a large gap between current top
detectors and human performance. To narrow this gap, we propose a network
fusion architecture, which consists of a multispectral proposal network to
generate pedestrian proposals, and a subsequent multispectral classification
network to distinguish pedestrian instances from hard negatives. The unified
network is learned by jointly optimizing pedestrian detection and semantic
segmentation tasks. The final detections are obtained by integrating the
outputs from different modalities as well as the two stages. The approach
significantly outperforms state-of-the-art methods on the KAIST dataset while
remain fast. Additionally, we contribute a sanitized version of training
annotations for the KAIST dataset, and examine the effects caused by different
kinds of annotation errors. Future research of this problem will benefit from
the sanitized version which eliminates the interference of annotation errors.
Jointly Identifying and Fixing Inconsistent Readings from Information Extraction Systems
Authors: Ankur Padia, Francis Ferraro, Tim Finin
Date: 2018-08-14 00:00:00
Proceeding: DeeLIO (ACL) 2022 5
Tasks: Knowledge Base Population

KGCleaner is a framework to identify and correct errors in data produced and delivered by an information extraction system. These tasks have been understudied and KGCleaner is the first to address both. We introduce a multi-task model that jointly learns to predict if an extracted relation is credible and repair it if not. We evaluate our approach and other models as instance of our framework on two collections: a Wikidata corpus of nearly 700K facts and 5M fact-relevant sentences and a collection of 30K facts from the 2015 TAC Knowledge Base Population task. For credibility classification, parameter efficient simple shallow neural network can achieve an absolute performance gain of 30 $F_1$ points on Wikidata and comparable performance on TAC. For the repair task, significant performance (at more than twice) gain can be obtained depending on the nature of the dataset and the models.