pwc-archive/papers-with-abstracts huggingface dataset:
Neural Machine Translation into Language Varieties
Authors: Surafel M. Lakew, Aliia Erofeeva, Marcello Federico
Date: 2018-11-02 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Translation

Both research and commercial machine translation have so far neglected the
importance of properly handling the spelling, lexical and grammar divergences
occurring among language varieties. Notable cases are standard national
varieties such as Brazilian and European Portuguese, and Canadian and European
French, which popular online machine translation services are not keeping
distinct. We show that an evident side effect of modeling such varieties as
unique classes is the generation of inconsistent translations. In this work, we
investigate the problem of training neural machine translation from English to
specific pairs of language varieties, assuming both labeled and unlabeled
parallel texts, and low-resource conditions. We report experiments from English
to two pairs of dialects, EuropeanBrazilian Portuguese and European-Canadian
French, and two pairs of standardized varieties, Croatian-Serbian and
Indonesian-Malay. We show significant BLEU score improvements over baseline
systems when translation into similar languages is learned as a multilingual
task with shared representations.
Augmenting Neural Response Generation with Context-Aware Topical Attention
Authors: Nouha Dziri, Ehsan Kamalloo, Kory W. Mathewson, Osmar Zaiane
Date: 2018-11-02 00:00:00
Proceeding: WS 2019 8
Tasks: Decoder, Open-Domain Dialog, Response Generation, Semantic Similarity

Sequence-to-Sequence (Seq2Seq) models have witnessed a notable success in generating natural conversational exchanges. Notwithstanding the syntactically well-formed responses generated by these neural network models, they are prone to be acontextual, short and generic. In this work, we introduce a Topical Hierarchical Recurrent Encoder Decoder (THRED), a novel, fully data-driven, multi-turn response generation system intended to produce contextual and topic-aware responses. Our model is built upon the basic Seq2Seq model by augmenting it with a hierarchical joint attention mechanism that incorporates topical concepts and previous interactions into the response generation. To train our model, we provide a clean and high-quality conversational dataset mined from Reddit comments. We evaluate THRED on two novel automated metrics, dubbed Semantic Similarity and Response Echo Index, as well as with human evaluation. Our experiments demonstrate that the proposed model is able to generate more diverse and contextually relevant responses compared to the strong baselines.
Augmenting Compositional Models for Knowledge Base Completion Using Gradient Representations
Authors: Matthias Lalisse, Paul Smolensky
Date: 2018-11-02 00:00:00
Tasks: Knowledge Base Completion, Knowledge Graphs, Link Prediction, Triplet

Neural models of Knowledge Base data have typically employed compositional representations of graph objects: entity and relation embeddings are systematically combined to evaluate the truth of a candidate Knowedge Base entry. Using a model inspired by Harmonic Grammar, we propose to tokenize triplet embeddings by subjecting them to a process of optimization with respect to learned well-formedness conditions on Knowledge Base triplets. The resulting model, known as Gradient Graphs, leads to sizable improvements when implemented as a companion to compositional models. Also, we show that the "supracompositional" triplet token embeddings it produces have interpretable properties that prove helpful in performing inference on the resulting triplet representations.
Semidefinite relaxations for certifying robustness to adversarial examples
Authors: Aditi Raghunathan, Jacob Steinhardt, Percy Liang
Date: 2018-11-02 00:00:00
Proceeding: NeurIPS 2018 12

Despite their impressive performance on diverse tasks, neural networks fail
catastrophically in the presence of adversarial inputs---imperceptibly but
adversarially perturbed versions of natural inputs. We have witnessed an arms
race between defenders who attempt to train robust networks and attackers who
try to construct adversarial examples. One promise of ending the arms race is
developing certified defenses, ones which are provably robust against all
attackers in some family. These certified defenses are based on convex
relaxations which construct an upper bound on the worst case loss over all
attackers in the family. Previous relaxations are loose on networks that are
not trained against the respective relaxation. In this paper, we propose a new
semidefinite relaxation for certifying robustness that applies to arbitrary
ReLU networks. We show that our proposed relaxation is tighter than previous
relaxations and produces meaningful robustness guarantees on three different
"foreign networks" whose training objectives are agnostic to our proposed
relaxation.
Minimax Estimation of Neural Net Distance
Authors: Kaiyi Ji, Yingbin Liang
Date: 2018-11-02 00:00:00
Proceeding: NeurIPS 2018 12

An important class of distance metrics proposed for training generative
adversarial networks (GANs) is the integral probability metric (IPM), in which
the neural net distance captures the practical GAN training via two neural
networks. This paper investigates the minimax estimation problem of the neural
net distance based on samples drawn from the distributions. We develop the
first known minimax lower bound on the estimation error of the neural net
distance, and an upper bound tighter than an existing bound on the estimator
error for the empirical neural net distance. Our lower and upper bounds match
not only in the order of the sample size but also in terms of the norm of the
parameter matrices of neural networks, which justifies the empirical neural net
distance as a good approximation of the true neural net distance for training
GANs in practice.
What evidence does deep learning model use to classify Skin Lesions?
Authors: Xiaoxiao Li, Junyan Wu, Eric Z. Chen, Hongda Jiang
Date: 2018-11-02 00:00:00
Tasks: Decision Making, Melanoma Diagnosis

Melanoma is a type of skin cancer with the most rapidly increasing incidence.
Early detection of melanoma using dermoscopy images significantly increases
patients' survival rate. However, accurately classifying skin lesions by eye,
especially in the early stage of melanoma, is extremely challenging for the
dermatologists. Hence, the discovery of reliable biomarkers will be meaningful
for melanoma diagnosis. Recent years, the value of deep learning empowered
computer-assisted diagnose has been shown in biomedical imaging based decision
making. However, much research focuses on improving disease detection accuracy
but not exploring the evidence of pathology. In this paper, we propose a method
to interpret the deep learning classification findings. Firstly, we propose an
accurate neural network architecture to classify skin lesions. Secondly, we
utilize a prediction difference analysis method that examines each patch on the
image through patch-wised corrupting to detect the biomarkers. Lastly, we
validate that our biomarker findings are corresponding to the patterns in the
literature. The findings can be significant and useful to guide clinical
diagnosis.
Scalable Deep $k$-Subspace Clustering
Authors: Tong Zhang, Pan Ji, Mehrtash Harandi, Richard Hartley, Ian Reid
Date: 2018-11-02 00:00:00
Tasks: Clustering

Subspace clustering algorithms are notorious for their scalability issues
because building and processing large affinity matrices are demanding. In this
paper, we introduce a method that simultaneously learns an embedding space
along subspaces within it to minimize a notion of reconstruction error, thus
addressing the problem of subspace clustering in an end-to-end learning
paradigm. To achieve our goal, we propose a scheme to update subspaces within a
deep neural network. This in turn frees us from the need of having an affinity
matrix to perform clustering. Unlike previous attempts, our method can easily
scale up to large datasets, making it unique in the context of unsupervised
learning with deep architectures. Our experiments show that our method
significantly improves the clustering accuracy while enjoying cheaper memory
footprints.
TrISec: Training Data-Unaware Imperceptible Security Attacks on Deep Neural Networks
Authors: Faiq Khalid, Muhammad Abdullah Hanif, Semeen Rehman, Rehan Ahmed, Muhammad Shafique
Date: 2018-11-02 00:00:00
Tasks: Autonomous Driving, Autonomous Vehicles, Data Poisoning, Image Generation, Inference Attack, Traffic Sign Detection, Traffic Sign Recognition

Most of the data manipulation attacks on deep neural networks (DNNs) during the training stage introduce a perceptible noise that can be catered by preprocessing during inference or can be identified during the validation phase. Therefore, data poisoning attacks during inference (e.g., adversarial attacks) are becoming more popular. However, many of them do not consider the imperceptibility factor in their optimization algorithms, and can be detected by correlation and structural similarity analysis, or noticeable (e.g., by humans) in a multi-level security system. Moreover, the majority of the inference attack relies on some knowledge about the training dataset. In this paper, we propose a novel methodology which automatically generates imperceptible attack images by using the back-propagation algorithm on pre-trained DNNs, without requiring any information about the training dataset (i.e., completely training data-unaware). We present a case study on traffic sign detection using the VGGNet trained on the German Traffic Sign Recognition Benchmarks dataset in an autonomous driving use case. Our results demonstrate that the generated attack images successfully perform misclassification while remaining imperceptible in both "subjective" and "objective" quality tests.
AiDroid: When Heterogeneous Information Network Marries Deep Neural Network for Real-time Android Malware Detection
Authors: Yanfang Ye, Shifu Hou, Lingwei Chen, Jingwei Lei, Wenqiang Wan, Jiabin Wang, Qi Xiong, Fudong Shao
Date: 2018-11-02 00:00:00
Tasks: Android Malware Detection, Malware Detection, Mobile Security

The explosive growth and increasing sophistication of Android malware call for new defensive techniques that are capable of protecting mobile users against novel threats. In this paper, we first extract the runtime Application Programming Interface (API) call sequences from Android apps, and then analyze higher-level semantic relations within the ecosystem to comprehensively characterize the apps. To model different types of entities (i.e., app, API, IMEI, signature, affiliation) and the rich semantic relations among them, we then construct a structural heterogeneous information network (HIN) and present meta-path based approach to depict the relatedness over apps. To efficiently classify nodes (e.g., apps) in the constructed HIN, we propose the HinLearning method to first obtain in-sample node embeddings and then learn representations of out-of-sample nodes without rerunning/adjusting HIN embeddings at the first attempt. Afterwards, we design a deep neural network (DNN) classifier taking the learned HIN representations as inputs for Android malware detection. A comprehensive experimental study on the large-scale real sample collections from Tencent Security Lab is performed to compare various baselines. Promising experimental results demonstrate that our developed system AiDroid which integrates our proposed method outperforms others in real-time Android malware detection. AiDroid has already been incorporated into Tencent Mobile Security product that serves millions of users worldwide.
The Knowref Coreference Corpus: Removing Gender and Number Cues for Difficult Pronominal Anaphora Resolution
Authors: Ali Emami, Paul Trichelair, Adam Trischler, Kaheer Suleman, Hannes Schulz, Jackie Chi Kit Cheung
Date: 2018-11-02 00:00:00
Proceeding: ACL 2019 7
Tasks: Common Sense Reasoning, coreference-resolution, Coreference Resolution, Data Augmentation, Diversity, World Knowledge

We introduce a new benchmark for coreference resolution and NLI, Knowref, that targets common-sense understanding and world knowledge. Previous coreference resolution tasks can largely be solved by exploiting the number and gender of the antecedents, or have been handcrafted and do not reflect the diversity of naturally occurring text. We present a corpus of over 8,000 annotated text passages with ambiguous pronominal anaphora. These instances are both challenging and realistic. We show that various coreference systems, whether rule-based, feature-rich, or neural, perform significantly worse on the task than humans, who display high inter-annotator agreement. To explain this performance gap, we show empirically that state-of-the art models often fail to capture context, instead relying on the gender or number of candidate antecedents to make a decision. We then use problem-specific insights to propose a data-augmentation trick called antecedent switching to alleviate this tendency in models. Finally, we show that antecedent switching yields promising results on other tasks as well: we use it to achieve state-of-the-art results on the GAP coreference task.
Transductive Learning with String Kernels for Cross-Domain Text Classification
Authors: Radu Tudor Ionescu, Andrei M. Butnaru
Date: 2018-11-02 00:00:00
Tasks: Classification, Cross-Domain Text Classification, General Classification, Language Identification, Native Language Identification, text-classification, Text Classification, Transductive Learning

For many text classification tasks, there is a major problem posed by the
lack of labeled data in a target domain. Although classifiers for a target
domain can be trained on labeled text data from a related source domain, the
accuracy of such classifiers is usually lower in the cross-domain setting.
Recently, string kernels have obtained state-of-the-art results in various text
classification tasks such as native language identification or automatic essay
scoring. Moreover, classifiers based on string kernels have been found to be
robust to the distribution gap between different domains. In this paper, we
formally describe an algorithm composed of two simple yet effective
transductive learning approaches to further improve the results of string
kernels in cross-domain settings. By adapting string kernels to the test set
without using the ground-truth test labels, we report significantly better
accuracy rates in cross-domain English polarity classification.
Cycle-consistency training for end-to-end speech recognition
Authors: Takaaki Hori, Ramon Astudillo, Tomoki Hayashi, Yu Zhang, Shinji Watanabe, Jonathan Le Roux
Date: 2018-11-02 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition, text-to-speech, Text to Speech

This paper presents a method to train end-to-end automatic speech recognition (ASR) models using unpaired data. Although the end-to-end approach can eliminate the need for expert knowledge such as pronunciation dictionaries to build ASR systems, it still requires a large amount of paired data, i.e., speech utterances and their transcriptions. Cycle-consistency losses have been recently proposed as a way to mitigate the problem of limited paired data. These approaches compose a reverse operation with a given transformation, e.g., text-to-speech (TTS) with ASR, to build a loss that only requires unsupervised data, speech in this example. Applying cycle consistency to ASR models is not trivial since fundamental information, such as speaker traits, are lost in the intermediate text bottleneck. To solve this problem, this work presents a loss that is based on the speech encoder state sequence instead of the raw speech signal. This is achieved by training a Text-To-Encoder model and defining a loss based on the encoder reconstruction error. Experimental results on the LibriSpeech corpus show that the proposed cycle-consistency training reduced the word error rate by 14.7% from an initial model trained with 100-hour paired data, using an additional 360 hours of audio data without transcriptions. We also investigate the use of text-only data mainly for language modeling to further improve the performance in the unpaired data training scenario.
META-DES.H: a dynamic ensemble selection technique using meta-learning and a dynamic weighting approach
Authors: Rafael M. O. Cruz, Robert Sabourin, George D. C. Cavalcanti
Date: 2018-11-01 00:00:00
Tasks: Meta-Learning

In Dynamic Ensemble Selection (DES) techniques, only the most competent
classifiers are selected to classify a given query sample. Hence, the key issue
in DES is how to estimate the competence of each classifier in a pool to select
the most competent ones. In order to deal with this issue, we proposed a novel
dynamic ensemble selection framework using meta-learning, called META-DES. The
framework is divided into three steps. In the first step, the pool of
classifiers is generated from the training data. In the second phase the
meta-features are computed using the training data and used to train a
meta-classifier that is able to predict whether or not a base classifier from
the pool is competent enough to classify an input instance. In this paper, we
propose improvements to the training and generalization phase of the META-DES
framework. In the training phase, we evaluate four different algorithms for the
training of the meta-classifier. For the generalization phase, three
combination approaches are evaluated: Dynamic selection, where only the
classifiers that attain a certain competence level are selected; Dynamic
weighting, where the meta-classifier estimates the competence of each
classifier in the pool, and the outputs of all classifiers in the pool are
weighted based on their level of competence; and a hybrid approach, in which
first an ensemble with the most competent classifiers is selected, after which
the weights of the selected classifiers are estimated in order to be used in a
weighted majority voting scheme. Experiments are carried out on 30
classification datasets. Experimental results demonstrate that the changes
proposed in this paper significantly improve the recognition accuracy of the
system in several datasets.
On Meta-Learning for Dynamic Ensemble Selection
Authors: Rafael M. O. Cruz, Robert Sabourin, George D. C. Cavalcanti
Date: 2018-11-01 00:00:00
Tasks: Meta-Learning

In this paper, we propose a novel dynamic ensemble selection framework using
meta-learning. The framework is divided into three steps. In the first step,
the pool of classifiers is generated from the training data. The second phase
is responsible to extract the meta-features and train the meta-classifier. Five
distinct sets of meta-features are proposed, each one corresponding to a
different criterion to measure the level of competence of a classifier for the
classification of a given query sample. The meta-features are computed using
the training data and used to train a meta-classifier that is able to predict
whether or not a base classifier from the pool is competent enough to classify
an input instance. Three different training scenarios for the training of the
meta-classifier are considered: problem-dependent, problem-independent and
hybrid. Experimental results show that the problem-dependent scenario provides
the best result. In addition, the performance of the problem-dependent scenario
is strongly correlated with the recognition rate of the system. A comparison
with state-of-the-art techniques shows that the proposed-dependent approach
outperforms current dynamic ensemble selection techniques.
Rethinking floating point for deep learning
Authors: Jeff Johnson
Date: 2018-11-01 00:00:00
Tasks: Deep Learning, Math, Quantization

Reducing hardware overhead of neural networks for faster or lower power
inference and training is an active area of research. Uniform quantization
using integer multiply-add has been thoroughly investigated, which requires
learning many quantization parameters, fine-tuning training or other
prerequisites. Little effort is made to improve floating point relative to this
baseline; it remains energy inefficient, and word size reduction yields drastic
loss in needed dynamic range. We improve floating point to be more energy
efficient than equivalent bit width integer hardware on a 28 nm ASIC process
while retaining accuracy in 8 bits with a novel hybrid log multiply/linear add,
Kulisch accumulation and tapered encodings from Gustafson's posit format. With
no network retraining, and drop-in replacement of all math and float32
parameters via round-to-nearest-even only, this open-sourced 8-bit log float is
within 0.9% top-1 and 0.2% top-5 accuracy of the original float32 ResNet-50 CNN
model on ImageNet. Unlike int8 quantization, it is still a general purpose
floating point arithmetic, interpretable out-of-the-box. Our 8/38-bit log float
multiply-add is synthesized and power profiled at 28 nm at 0.96x the power and
1.12x the area of 8/32-bit integer multiply-add. In 16 bits, our log float
multiply-add is 0.59x the power and 0.68x the area of IEEE 754 float16 fused
multiply-add, maintaining the same signficand precision and dynamic range,
proving useful for training ASICs as well.
A Recurrent Graph Neural Network for Multi-Relational Data
Authors: Vassilis N. Ioannidis, Antonio G. Marques, Georgios B. Giannakis
Date: 2018-11-05 00:00:00
Tasks: Graph Neural Network, Sociology

The era of data deluge has sparked the interest in graph-based learning
methods in a number of disciplines such as sociology, biology, neuroscience, or
engineering. In this paper, we introduce a graph recurrent neural network
(GRNN) for scalable semi-supervised learning from multi-relational data. Key
aspects of the novel GRNN architecture are the use of multi-relational graphs,
the dynamic adaptation to the different relations via learnable weights, and
the consideration of graph-based regularizers to promote smoothness and
alleviate over-parametrization. Our ultimate goal is to design a powerful
learning architecture able to: discover complex and highly non-linear data
associations, combine (and select) multiple types of relations, and scale
gracefully with respect to the size of the graph. Numerical tests with real
data sets corroborate the design goals and illustrate the performance gains
relative to competing alternatives.