pwc-archive/papers-with-abstracts huggingface dataset:
Operations Guided Neural Networks for High Fidelity Data-To-Text Generation
Authors: Feng Nie, Jinpeng Wang, Jin-Ge Yao, Rong pan, Chin-Yew Lin
Date: 2018-09-08 00:00:00
Tasks: Data-to-Text Generation, Decoder, Quantization, Text Generation, Vocal Bursts Intensity Prediction

Recent neural models for data-to-text generation are mostly based on
data-driven end-to-end training over encoder-decoder networks. Even though the
generated texts are mostly fluent and informative, they often generate
descriptions that are not consistent with the input structured data. This is a
critical issue especially in domains that require inference or calculations
over raw data. In this paper, we attempt to improve the fidelity of neural
data-to-text generation by utilizing pre-executed symbolic operations. We
propose a framework called Operation-guided Attention-based
sequence-to-sequence network (OpAtt), with a specifically designed gating
mechanism as well as a quantization module for operation results to utilize
information from pre-executed operations. Experiments on two sports datasets
show our proposed method clearly improves the fidelity of the generated texts
to the input structured data.
Exploiting Invertible Decoders for Unsupervised Sentence Representation Learning
Authors: Shuai Tang, Virginia R. de Sa
Date: 2018-09-08 00:00:00
Proceeding: ACL 2019 7
Tasks: Decoder, Representation Learning, Sentence

The encoder-decoder models for unsupervised sentence representation learning tend to discard the decoder after being trained on a large unlabelled corpus, since only the encoder is needed to map the input sentence into a vector representation. However, parameters learnt in the decoder also contain useful information about language. In order to utilise the decoder after learning, we present two types of decoding functions whose inverse can be easily derived without expensive inverse calculation. Therefore, the inverse of the decoding function serves as another encoder that produces sentence representations. We show that, with careful design of the decoding functions, the model learns good sentence representations, and the ensemble of the representations produced from the encoder and the inverse of the decoder demonstrate even better generalisation ability and solid transferability.
Coupled IGMM-GANs for deep multimodal anomaly detection in human mobility data
Authors: Kathryn Gray, Daniel Smolyak, Sarkhan Badirli, George Mohler
Date: 2018-09-08 00:00:00
Tasks: Anomaly Detection, Feature Engineering, Fraud Detection

Detecting anomalous activity in human mobility data has a number of
applications including road hazard sensing, telematic based insurance, and
fraud detection in taxi services and ride sharing. In this paper we address two
challenges that arise in the study of anomalous human trajectories: 1) a lack
of ground truth data on what defines an anomaly and 2) the dependence of
existing methods on significant pre-processing and feature engineering. While
generative adversarial networks seem like a natural fit for addressing these
challenges, we find that existing GAN based anomaly detection algorithms
perform poorly due to their inability to handle multimodal patterns. For this
purpose we introduce an infinite Gaussian mixture model coupled with
(bi-directional) generative adversarial networks, IGMM-GAN, that is able to
generate synthetic, yet realistic, human mobility data and simultaneously
facilitates multimodal anomaly detection. Through estimation of a generative
probability density on the space of human trajectories, we are able to generate
realistic synthetic datasets that can be used to benchmark existing anomaly
detection methods. The estimated multimodal density also allows for a natural
definition of outlier that we use for detecting anomalous trajectories. We
illustrate our methodology and its improvement over existing GAN anomaly
detection on several human mobility datasets, along with MNIST.
Decentralized Differentially Private Without-Replacement Stochastic Gradient Descent
Authors: Richeng Jin, Xiaofan He, Huaiyu Dai
Date: 2018-09-08 00:00:00
Tasks: BIG-bench Machine Learning

While machine learning has achieved remarkable results in a wide variety of domains, the training of models often requires large datasets that may need to be collected from different individuals. As sensitive information may be contained in the individual's dataset, sharing training data may lead to severe privacy concerns. Therefore, there is a compelling need to develop privacy-aware machine learning methods, for which one effective approach is to leverage the generic framework of differential privacy. Considering that stochastic gradient descent (SGD) is one of the most commonly adopted methods for large-scale machine learning problems, a decentralized differentially private SGD algorithm is proposed in this work. Particularly, we focus on SGD without replacement due to its favorable structure for practical implementation. Both privacy and convergence analysis are provided for the proposed algorithm. Finally, extensive experiments are performed to demonstrate the effectiveness of the proposed method.
Learning to Solve NP-Complete Problems - A Graph Neural Network for Decision TSP
Authors: Marcelo O. R. Prates, Pedro H. C. Avelar, Henrique Lemos, Luis Lamb, Moshe Vardi
Date: 2018-09-08 00:00:00
Tasks: Graph Neural Network

Graph Neural Networks (GNN) are a promising technique for bridging
differential programming and combinatorial domains. GNNs employ trainable
modules which can be assembled in different configurations that reflect the
relational structure of each problem instance. In this paper, we show that GNNs
can learn to solve, with very little supervision, the decision variant of the
Traveling Salesperson Problem (TSP), a highly relevant $\mathcal{NP}$-Complete
problem. Our model is trained to function as an effective message-passing
algorithm in which edges (embedded with their weights) communicate with
vertices for a number of iterations after which the model is asked to decide
whether a route with cost $<C$ exists. We show that such a network can be
trained with sets of dual examples: given the optimal tour cost $C^{*}$, we
produce one decision instance with target cost $x\%$ smaller and one with
target cost $x\%$ larger than $C^{*}$. We were able to obtain $80\%$ accuracy
training with $-2\%,+2\%$ deviations, and the same trained model can generalize
for more relaxed deviations with increasing performance. We also show that the
model is capable of generalizing for larger problem sizes. Finally, we provide
a method for predicting the optimal route cost within $2\%$ deviation from the
ground truth. In summary, our work shows that Graph Neural Networks are
powerful enough to solve $\mathcal{NP}$-Complete problems which combine
symbolic and numeric data.
What If We Simply Swap the Two Text Fragments? A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks
Authors: Haohan Wang, Da Sun, Eric P. Xing
Date: 2018-09-07 00:00:00
Tasks: Sentence, Sentence Embeddings

Nature language inference (NLI) task is a predictive task of determining the
inference relationship of a pair of natural language sentences. With the
increasing popularity of NLI, many state-of-the-art predictive models have been
proposed with impressive performances. However, several works have noticed the
statistical irregularities in the collected NLI data set that may result in an
over-estimated performance of these models and proposed remedies. In this
paper, we further investigate the statistical irregularities, what we refer as
confounding factors, of the NLI data sets. With the belief that some NLI labels
should preserve under swapping operations, we propose a simple yet effective
way (swapping the two text fragments) of evaluating the NLI predictive models
that naturally mitigate the observed problems. Further, we continue to train
the predictive models with our swapping manner and propose to use the deviation
of the model's evaluation performances under different percentages of training
text fragments to be swapped to describe the robustness of a predictive model.
Our evaluation metrics leads to some interesting understandings of recent
published NLI methods. Finally, we also apply the swapping operation on NLI
models to see the effectiveness of this straightforward method in mitigating
the confounding factor problems in training generic sentence embeddings for
other NLP transfer tasks.
DensSiam: End-to-End Densely-Siamese Network with Self-Attention Model for Object Tracking
Authors: Mohamed H. Abdelpakey, Mohamed S. Shehata, Mostafa M. Mohamed
Date: 2018-09-07 00:00:00
Tasks: Object Tracking

Convolutional Siamese neural networks have been recently used to track
objects using deep features. Siamese architecture can achieve real time speed,
however it is still difficult to find a Siamese architecture that maintains the
generalization capability, high accuracy and speed while decreasing the number
of shared parameters especially when it is very deep. Furthermore, a
conventional Siamese architecture usually processes one local neighborhood at a
time, which makes the appearance model local and non-robust to appearance
changes.
  To overcome these two problems, this paper proposes DensSiam, a novel
convolutional Siamese architecture, which uses the concept of dense layers and
connects each dense layer to all layers in a feed-forward fashion with a
similarity-learning function. DensSiam also includes a Self-Attention mechanism
to force the network to pay more attention to the non-local features during
offline training. Extensive experiments are performed on four tracking
benchmarks: OTB2013 and OTB2015 for validation set; and VOT2015, VOT2016 and
VOT2017 for testing set. The obtained results show that DensSiam achieves
superior results on these benchmarks compared to other current state-of-the-art
methods.
Exploiting Edge Features in Graph Neural Networks
Authors: Liyu Gong, Qiang Cheng
Date: 2018-09-07 00:00:00
Tasks: General Classification, Graph Attention, Graph Classification, Graph Learning, Graph Neural Network, Node Classification

Edge features contain important information about graphs. However, current
state-of-the-art neural network models designed for graph learning, e.g. graph
convolutional networks (GCN) and graph attention networks (GAT), adequately
utilize edge features, especially multi-dimensional edge features. In this
paper, we build a new framework for a family of new graph neural network models
that can more sufficiently exploit edge features, including those of undirected
or multi-dimensional edges. The proposed framework can consolidate current
graph neural network models; e.g. graph convolutional networks (GCN) and graph
attention networks (GAT). The proposed framework and new models have the
following novelties: First, we propose to use doubly stochastic normalization
of graph edge features instead of the commonly used row or symmetric
normalization approches used in current graph neural networks. Second, we
construct new formulas for the operations in each individual layer so that they
can handle multi-dimensional edge features. Third, for the proposed new
framework, edge features are adaptive across network layers. As a result, our
proposed new framework and new models can exploit a rich source of graph
information. We apply our new models to graph node classification on several
citation networks, whole graph classification, and regression on several
molecular datasets. Compared with the current state-of-the-art methods, i.e.
GCNs and GAT, our models obtain better performance, which testify to the
importance of exploiting edge features in graph neural networks.
Analysis of Thompson Sampling for Combinatorial Multi-armed Bandit with Probabilistically Triggered Arms
Authors: Alihan Hüyük, Cem Tekin
Date: 2018-09-07 00:00:00
Tasks: Thompson Sampling

We analyze the regret of combinatorial Thompson sampling (CTS) for the
combinatorial multi-armed bandit with probabilistically triggered arms under
the semi-bandit feedback setting. We assume that the learner has access to an
exact optimization oracle but does not know the expected base arm outcomes
beforehand. When the expected reward function is Lipschitz continuous in the
expected base arm outcomes, we derive $O(\sum_{i =1}^m \log T / (p_i
\Delta_i))$ regret bound for CTS, where $m$ denotes the number of base arms,
$p_i$ denotes the minimum non-zero triggering probability of base arm $i$ and
$\Delta_i$ denotes the minimum suboptimality gap of base arm $i$. We also
compare CTS with combinatorial upper confidence bound (CUCB) via numerical
experiments on a cascading bandit problem.
Trick Me If You Can: Human-in-the-loop Generation of Adversarial Examples for Question Answering
Authors: Eric Wallace, Pedro Rodriguez, Shi Feng, Ikuya Yamada, Jordan Boyd-Graber
Date: 2018-09-07 00:00:00
Proceeding: TACL 2019 3
Tasks: Diversity, Information Retrieval, Question Answering, Retrieval

Adversarial evaluation stress tests a model's understanding of natural language. While past approaches expose superficial patterns, the resulting adversarial examples are limited in complexity and diversity. We propose human-in-the-loop adversarial generation, where human authors are guided to break models. We aid the authors with interpretations of model predictions through an interactive user interface. We apply this generation framework to a question answering task called Quizbowl, where trivia enthusiasts craft adversarial questions. The resulting questions are validated via live human--computer matches: although the questions appear ordinary to humans, they systematically stump neural and information retrieval models. The adversarial questions cover diverse phenomena from multi-hop reasoning to entity type distractors, exposing open challenges in robust question answering.
Textual Analogy Parsing: What's Shared and What's Compared among Analogous Facts
Authors: Matthew Lamm, Arun Tejasvi Chaganty, Christopher D. Manning, Dan Jurafsky, Percy Liang
Date: 2018-09-07 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Sentence, Textual Analogy Parsing

To understand a sentence like "whereas only 10% of White Americans live at or
below the poverty line, 28% of African Americans do" it is important not only
to identify individual facts, e.g., poverty rates of distinct demographic
groups, but also the higher-order relations between them, e.g., the disparity
between them. In this paper, we propose the task of Textual Analogy Parsing
(TAP) to model this higher-order meaning. The output of TAP is a frame-style
meaning representation which explicitly specifies what is shared (e.g., poverty
rates) and what is compared (e.g., White Americans vs. African Americans, 10%
vs. 28%) between its component facts. Such a meaning representation can enable
new applications that rely on discourse understanding such as automated chart
generation from quantitative text. We present a new dataset for TAP, baselines,
and a model that successfully uses an ILP to enforce the structural constraints
of the problem.
Neural Machine Translation of Logographic Languages Using Sub-character Level Information
Authors: Longtu Zhang, Mamoru Komachi
Date: 2018-09-07 00:00:00
Tasks: Decoder, Machine Translation, NMT, Translation

Recent neural machine translation (NMT) systems have been greatly improved by
encoder-decoder models with attention mechanisms and sub-word units. However,
important differences between languages with logographic and alphabetic writing
systems have long been overlooked. This study focuses on these differences and
uses a simple approach to improve the performance of NMT systems utilizing
decomposed sub-character level information for logographic languages. Our
results indicate that our approach not only improves the translation
capabilities of NMT systems between Chinese and English, but also further
improves NMT systems between Chinese and Japanese, because it utilizes the
shared information brought by similar sub-character units.
Selective Refinement Network for High Performance Face Detection
Authors: Cheng Chi, Shifeng Zhang, Junliang Xing, Zhen Lei, Stan Z. Li, Xudong Zou
Date: 2018-09-07 00:00:00
Tasks: Face Detection, General Classification, regression, Vocal Bursts Intensity Prediction

High performance face detection remains a very challenging problem,
especially when there exists many tiny faces. This paper presents a novel
single-shot face detector, named Selective Refinement Network (SRN), which
introduces novel two-step classification and regression operations selectively
into an anchor-based face detector to reduce false positives and improve
location accuracy simultaneously. In particular, the SRN consists of two
modules: the Selective Two-step Classification (STC) module and the Selective
Two-step Regression (STR) module. The STC aims to filter out most simple
negative anchors from low level detection layers to reduce the search space for
the subsequent classifier, while the STR is designed to coarsely adjust the
locations and sizes of anchors from high level detection layers to provide
better initialization for the subsequent regressor. Moreover, we design a
Receptive Field Enhancement (RFE) block to provide more diverse receptive
field, which helps to better capture faces in some extreme poses. As a
consequence, the proposed SRN detector achieves state-of-the-art performance on
all the widely used face detection benchmarks, including AFW, PASCAL face,
FDDB, and WIDER FACE datasets. Codes will be released to facilitate further
studies on the face detection problem.
Coherence-Aware Neural Topic Modeling
Authors: Ran Ding, Ramesh Nallapati, Bing Xiang
Date: 2018-09-07 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Topic Models, Variational Inference

Topic models are evaluated based on their ability to describe documents well
(i.e. low perplexity) and to produce topics that carry coherent semantic
meaning. In topic modeling so far, perplexity is a direct optimization target.
However, topic coherence, owing to its challenging computation, is not
optimized for and is only evaluated after training. In this work, under a
neural variational inference framework, we propose methods to incorporate a
topic coherence objective into the training process. We demonstrate that such a
coherence-aware topic model exhibits a similar level of perplexity as baseline
models but achieves substantially higher topic coherence.
Query Attack via Opposite-Direction Feature:Towards Robust Image Retrieval
Authors: Zhedong Zheng, Liang Zheng, Yi Yang, Fei Wu
Date: 2018-09-07 00:00:00
Tasks: Adversarial Attack, General Classification, Image Retrieval, Open Set Learning, Retrieval

Most existing works of adversarial samples focus on attacking image recognition models, while little attention is paid to the image retrieval task. In this paper, we identify two inherent challenges in applying prevailing image recognition attack methods to image retrieval. First, image retrieval demands discriminative visual features, which is significantly different from the one-hot class prediction in image recognition. Second, due to the disjoint and potentially unrelated classes between the training and test set in image retrieval, predicting the query category from predefined training classes is not accurate and leads to a sub-optimal adversarial gradient. To address these limitations, we propose a new white-box attack approach, Opposite-Direction Feature Attack (ODFA), to generate adversarial queries. Opposite-Direction Feature Attack (ODFA) effectively exploits feature-level adversarial gradients and takes advantage of feature distance in the representation space. To our knowledge, we are among the early attempts to design an attack method specifically for image retrieval. When we deploy an attacked image as the query, the true matches are prone to receive low ranks. We demonstrate through extensive experiments that (1) only crafting adversarial queries is sufficient to fool the state-of-the-art retrieval systems; (2) the proposed attack method, ODFA, leads to a higher attack success rate than classification attack methods, validating the necessity of leveraging characteristics of image retrieval; (3) the adversarial queries generated by our method have good transferability to other retrieval models without accessing their parameters, i.e.,the black-box setting.
RetGK: Graph Kernels based on Return Probabilities of Random Walks
Authors: Zhen Zhang, Mianzhi Wang, Yijian Xiang, Yan Huang, Arye Nehorai
Date: 2018-09-07 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Computational Efficiency, General Classification, Graph Classification

Graph-structured data arise in wide applications, such as computer vision,
bioinformatics, and social networks. Quantifying similarities among graphs is a
fundamental problem. In this paper, we develop a framework for computing graph
kernels, based on return probabilities of random walks. The advantages of our
proposed kernels are that they can effectively exploit various node attributes,
while being scalable to large datasets. We conduct extensive graph
classification experiments to evaluate our graph kernels. The experimental
results show that our graph kernels significantly outperform existing
state-of-the-art approaches in both accuracy and computational efficiency.
Unsupervised Sentence Compression using Denoising Auto-Encoders
Authors: Thibault Févry, Jason Phang
Date: 2018-09-07 00:00:00
Proceeding: CONLL 2018 10
Tasks: Denoising, Sentence, Sentence Compression, Text Summarization, Unsupervised Sentence Compression

In sentence compression, the task of shortening sentences while retaining the
original meaning, models tend to be trained on large corpora containing pairs
of verbose and compressed sentences. To remove the need for paired corpora, we
emulate a summarization task and add noise to extend sentences and train a
denoising auto-encoder to recover the original, constructing an end-to-end
training regime without the need for any examples of compressed sentences. We
conduct a human evaluation of our model on a standard text summarization
dataset and show that it performs comparably to a supervised baseline based on
grammatical correctness and retention of meaning. Despite being exposed to no
target data, our unsupervised models learn to generate imperfect but reasonably
readable sentence summaries. Although we underperform supervised models based
on ROUGE scores, our models are competitive with a supervised baseline based on
human evaluation for grammatical correctness and retention of meaning.
dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learning
Authors: Palash Goyal, Sujit Rokka Chhetri, Arquimedes Canedo
Date: 2018-09-07 00:00:00
Tasks: Dynamic Link Prediction, Graph Representation Learning, Link Prediction, Prediction, Representation Learning

Learning graph representations is a fundamental task aimed at capturing various properties of graphs in vector space. The most recent methods learn such representations for static networks. However, real world networks evolve over time and have varying dynamics. Capturing such evolution is key to predicting the properties of unseen networks. To understand how the network dynamics affect the prediction performance, we propose an embedding approach which learns the structure of evolution in dynamic graphs and can predict unseen links with higher precision. Our model, dyngraph2vec, learns the temporal transitions in the network using a deep architecture composed of dense and recurrent layers. We motivate the need of capturing dynamics for prediction on a toy data set created using stochastic block models. We then demonstrate the efficacy of dyngraph2vec over existing state-of-the-art methods on two real world data sets. We observe that learning dynamics can improve the quality of embedding and yield better performance in link prediction.