pwc-archive/papers-with-abstracts huggingface dataset:
Train, Sort, Explain: Learning to Diagnose Translation Models
Authors: Robert Schwarzenberg, David Harbecke, Vivien Macketanz, Eleftherios Avramidis, Sebastian Möller
Date: 2019-03-28 00:00:00
Proceeding: NAACL 2019 6
Tasks: Translation

Evaluating translation models is a trade-off between effort and detail. On
the one end of the spectrum there are automatic count-based methods such as
BLEU, on the other end linguistic evaluations by humans, which arguably are
more informative but also require a disproportionately high effort. To narrow
the spectrum, we propose a general approach on how to automatically expose
systematic differences between human and machine translations to human experts.
Inspired by adversarial settings, we train a neural text classifier to
distinguish human from machine translations. A classifier that performs and
generalizes well after training should recognize systematic differences between
the two classes, which we uncover with neural explainability methods. Our
proof-of-concept implementation, DiaMaT, is open source. Applied to a dataset
translated by a state-of-the-art neural Transformer model, DiaMaT achieves a
classification accuracy of 75% and exposes meaningful differences between
humans and the Transformer, amidst the current discussion about human parity.
Handling Noisy Labels for Robustly Learning from Self-Training Data for Low-Resource Sequence Labeling
Authors: Debjit Paul, Mittul Singh, Michael A. Hedderich, Dietrich Klakow
Date: 2019-03-28 00:00:00
Proceeding: NAACL 2019 6
Tasks: Auxiliary Learning, Chunking, NER

In this paper, we address the problem of effectively self-training neural
networks in a low-resource setting. Self-training is frequently used to
automatically increase the amount of training data. However, in a low-resource
scenario, it is less effective due to unreliable annotations created using
self-labeling of unlabeled data. We propose to combine self-training with noise
handling on the self-labeled data. Directly estimating noise on the combined
clean training set and self-labeled data can lead to corruption of the clean
data and hence, performs worse. Thus, we propose the Clean and Noisy Label
Neural Network which trains on clean and noisy self-labeled data simultaneously
by explicitly modelling clean and noisy labels separately. In our experiments
on Chunking and NER, this approach performs more robustly than the baselines.
Complementary to this explicit approach, noise can also be handled implicitly
with the help of an auxiliary learning task. To such a complementary approach,
our method is more beneficial than other baseline methods and together provides
the best performance overall.
High Fidelity Face Manipulation with Extreme Poses and Expressions
Authors: Chaoyou Fu, Yibo Hu, Xiang Wu, Guoli Wang, Qian Zhang, Ran He
Date: 2019-03-28 00:00:00
Tasks: Face Generation, Face Recognition, Vocal Bursts Intensity Prediction

Face manipulation has shown remarkable advances with the flourish of Generative Adversarial Networks. However, due to the difficulties of controlling structures and textures, it is challenging to model poses and expressions simultaneously, especially for the extreme manipulation at high-resolution. In this paper, we propose a novel framework that simplifies face manipulation into two correlated stages: a boundary prediction stage and a disentangled face synthesis stage. The first stage models poses and expressions jointly via boundary images. Specifically, a conditional encoder-decoder network is employed to predict the boundary image of the target face in a semi-supervised way. Pose and expression estimators are introduced to improve the prediction performance. In the second stage, the predicted boundary image and the input face image are encoded into the structure and the texture latent space by two encoder networks, respectively. A proxy network and a feature threshold loss are further imposed to disentangle the latent space. Furthermore, due to the lack of high-resolution face manipulation databases to verify the effectiveness of our method, we collect a new high-quality Multi-View Face (MVF-HQ) database. It contains 120,283 images at 6000x4000 resolution from 479 identities with diverse poses, expressions, and illuminations. MVF-HQ is much larger in scale and much higher in resolution than publicly available high-resolution face manipulation databases. We will release MVF-HQ soon to push forward the advance of face manipulation. Qualitative and quantitative experiments on four databases show that our method dramatically improves the synthesis quality.
Parabolic Approximation Line Search for DNNs
Authors: Maximus Mutschler, Andreas Zell
Date: 2019-03-28 00:00:00
Proceeding: NeurIPS 2020 12

A major challenge in current optimization research for deep learning is to automatically find optimal step sizes for each update step. The optimal step size is closely related to the shape of the loss in the update step direction. However, this shape has not yet been examined in detail. This work shows empirically that the batch loss over lines in negative gradient direction is mostly convex locally and well suited for one-dimensional parabolic approximations. By exploiting this parabolic property we introduce a simple and robust line search approach, which performs loss-shape dependent update steps. Our approach combines well-known methods such as parabolic approximation, line search and conjugate gradient, to perform efficiently. It surpasses other step size estimating methods and competes with common optimization methods on a large variety of experiments without the need of hand-designed step size schedules. Thus, it is of interest for objectives where step-size schedules are unknown or do not perform well. Our extensive evaluation includes multiple comprehensive hyperparameter grid searches on several datasets and architectures. Finally, we provide a general investigation of exact line searches in the context of batch losses and exact losses, including their relation to our line search approach.
On the Stability and Generalization of Learning with Kernel Activation Functions
Authors: Michele Cirillo, Simone Scardapane, Steven Van Vaerenbergh, Aurelio Uncini
Date: 2019-03-28 00:00:00

In this brief we investigate the generalization properties of a
recently-proposed class of non-parametric activation functions, the kernel
activation functions (KAFs). KAFs introduce additional parameters in the
learning process in order to adapt nonlinearities individually on a per-neuron
basis, exploiting a cheap kernel expansion of every activation value. While
this increase in flexibility has been shown to provide significant improvements
in practice, a theoretical proof for its generalization capability has not been
addressed yet in the literature. Here, we leverage recent literature on the
stability properties of non-convex models trained via stochastic gradient
descent (SGD). By indirectly proving two key smoothness properties of the
models under consideration, we prove that neural networks endowed with KAFs
generalize well when trained with SGD for a finite number of steps.
Interestingly, our analysis provides a guideline for selecting one of the
hyper-parameters of the model, the bandwidth of the scalar Gaussian kernel. A
short experimental evaluation validates the proof.
Regularizing Trajectory Optimization with Denoising Autoencoders
Authors: Rinu Boney, Norman Di Palo, Mathias Berglund, Alexander Ilin, Juho Kannala, Antti Rasmus, Harri Valpola
Date: 2019-03-28 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Denoising, Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Trajectory optimization using a learned model of the environment is one of the core elements of model-based reinforcement learning. This procedure often suffers from exploiting inaccuracies of the learned model. We propose to regularize trajectory optimization by means of a denoising autoencoder that is trained on the same trajectories as the model of the environment. We show that the proposed regularization leads to improved planning with both gradient-based and gradient-free optimizers. We also demonstrate that using regularized trajectory optimization leads to rapid initial learning in a set of popular motor control tasks, which suggests that the proposed approach can be a useful tool for improving sample efficiency.
Learning Discrete Structures for Graph Neural Networks
Authors: Luca Franceschi, Mathias Niepert, Massimiliano Pontil, Xiao He
Date: 2019-03-28 00:00:00
Tasks: Music Genre Recognition, Node Classification

Graph neural networks (GNNs) are a popular class of machine learning models whose major advantage is their ability to incorporate a sparse and discrete dependency structure between data points. Unfortunately, GNNs can only be used when such a graph-structure is available. In practice, however, real-world graphs are often noisy and incomplete or might not be available at all. With this work, we propose to jointly learn the graph structure and the parameters of graph convolutional networks (GCNs) by approximately solving a bilevel program that learns a discrete probability distribution on the edges of the graph. This allows one to apply GCNs not only in scenarios where the given graph is incomplete or corrupted but also in those where a graph is not available. We conduct a series of experiments that analyze the behavior of the proposed method and demonstrate that it outperforms related methods by a significant margin.
Evolving Boolean Functions with Conjunctions and Disjunctions via Genetic Programming
Authors: Benjamin Doerr, Andrei Lissovoi, Pietro S. Oliveto
Date: 2019-03-28 00:00:00

Recently it has been proved that simple GP systems can efficiently evolve the
conjunction of $n$ variables if they are equipped with the minimal required
components. In this paper, we make a considerable step forward by analysing the
behaviour and performance of the GP system for evolving a Boolean function with
unknown components, i.e., the function may consist of both conjunctions and
disjunctions. We rigorously prove that if the target function is the
conjunction of $n$ variables, then the RLS-GP using the complete truth table to
evaluate program quality evolves the exact target function in $O(\ell n \log^2
n)$ iterations in expectation, where $\ell \geq n$ is a limit on the size of
any accepted tree. When, as in realistic applications, only a polynomial sample
of possible inputs is used to evaluate solution quality, we show how RLS-GP can
evolve a conjunction with any polynomially small generalisation error with
probability $1 - O(\log^2(n)/n)$. To produce our results we introduce a
super-multiplicative drift theorem that gives significantly stronger runtime
bounds when the expected progress is only slightly super-linear in the distance
from the optimum.
Imbalanced Sentiment Classification Enhanced with Discourse Marker
Authors: Tao Zhang, Xing Wu, Meng Lin, Jizhong Han, Songlin Hu
Date: 2019-03-28 00:00:00
Tasks: Classification, Data Augmentation, Diversity, General Classification, Sentiment Analysis, Sentiment Classification

Imbalanced data commonly exists in real world, espacially in
sentiment-related corpus, making it difficult to train a classifier to
distinguish latent sentiment in text data. We observe that humans often express
transitional emotion between two adjacent discourses with discourse markers
like "but", "though", "while", etc, and the head discourse and the tail
discourse 3 usually indicate opposite emotional tendencies. Based on this
observation, we propose a novel plug-and-play method, which first samples
discourses according to transitional discourse markers and then validates
sentimental polarities with the help of a pretrained attention-based model. Our
method increases sample diversity in the first place, can serve as a upstream
preprocessing part in data augmentation. We conduct experiments on three public
sentiment datasets, with several frequently used algorithms. Results show that
our method is found to be consistently effective, even in highly imbalanced
scenario, and easily be integrated with oversampling method to boost the
performance on imbalanced sentiment classification.
Intelligent Processing in Vehicular Ad hoc Networks: a Survey
Authors: Yang Liu
Date: 2019-03-28 00:00:00
Tasks: Survey

The intelligent Processing technique is more and more attractive to
researchers due to its ability to deal with key problems in Vehicular Ad hoc
networks. However, several problems in applying intelligent processing
technologies in VANETs remain open. The existing applications are
comprehensively reviewed and discussed, and classified into different
categories in this paper. Their strategies, advantages/disadvantages, and
performances are elaborated. By generalizing different tactics in various
applications related to different scenarios of VANETs and evaluating their
performances, several promising directions for future research have been
suggested.
Meta-Learning surrogate models for sequential decision making
Authors: Alexandre Galashov, Jonathan Schwarz, Hyunjik Kim, Marta Garnelo, David Saxton, Pushmeet Kohli, S. M. Ali Eslami, Yee Whye Teh
Date: 2019-03-28 00:00:00
Tasks: Bayesian Optimisation, Decision Making, Meta-Learning, Multi-Armed Bandits, Recommendation Systems, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

We introduce a unified probabilistic framework for solving sequential decision making problems ranging from Bayesian optimisation to contextual bandits and reinforcement learning. This is accomplished by a probabilistic model-based approach that explains observed data while capturing predictive uncertainty during the decision making process. Crucially, this probabilistic model is chosen to be a Meta-Learning system that allows learning from a distribution of related problems, allowing data efficient adaptation to a target task. As a suitable instantiation of this framework, we explore the use of Neural processes due to statistical and computational desiderata. We apply our framework to a broad range of problem domains, such as control problems, recommender systems and adversarial attacks on RL agents, demonstrating an efficient and general black-box learning approach.
Addressing Model Vulnerability to Distributional Shifts over Image Transformation Sets
Authors: Riccardo Volpi, Vittorio Murino
Date: 2019-03-28 00:00:00
Proceeding: ICCV 2019 10
Tasks: Combinatorial Optimization, Data Augmentation, General Classification, Semantic Segmentation

We are concerned with the vulnerability of computer vision models to distributional shifts. We formulate a combinatorial optimization problem that allows evaluating the regions in the image space where a given model is more vulnerable, in terms of image transformations applied to the input, and face it with standard search algorithms. We further embed this idea in a training procedure, where we define new data augmentation rules according to the image transformations that the current model is most vulnerable to, over iterations. An empirical evaluation on classification and semantic segmentation problems suggests that the devised algorithm allows to train models that are more robust against content-preserving image manipulations and, in general, against distributional shifts.
AED-Net: An Abnormal Event Detection Network
Authors: Tian Wang, Zichen Miao, Yuxin Chen, Yi Zhou, Guangcun Shan, Hichem Snoussi
Date: 2019-03-28 00:00:00
Tasks: Event Detection, One-class classifier

It is challenging to detect the anomaly in crowded scenes for quite a long
time. In this paper, a self-supervised framework, abnormal event detection
network (AED-Net), which is composed of PCAnet and kernel principal component
analysis (kPCA), is proposed to address this problem. Using surveillance video
sequences of different scenes as raw data, PCAnet is trained to extract
high-level semantics of crowd's situation. Next, kPCA,a one-class classifier,
is trained to determine anomaly of the scene. In contrast to some prevailing
deep learning methods,the framework is completely self-supervised because it
utilizes only video sequences in a normal situation. Experiments of global and
local abnormal event detection are carried out on UMN and UCSD datasets, and
competitive results with higher EER and AUC compared to other state-of-the-art
methods are observed. Furthermore, by adding local response normalization (LRN)
layer, we propose an improvement to original AED-Net. And it is proved to
perform better by promoting the framework's generalization capacity according
to the experiments.
Smooth Adversarial Examples
Authors: Hanwei Zhang, Yannis Avrithis, Teddy Furon, Laurent Amsaleg
Date: 2019-03-28 00:00:00

This paper investigates the visual quality of the adversarial examples.
Recent papers propose to smooth the perturbations to get rid of high frequency
artefacts. In this work, smoothing has a different meaning as it perceptually
shapes the perturbation according to the visual content of the image to be
attacked. The perturbation becomes locally smooth on the flat areas of the
input image, but it may be noisy on its textured areas and sharp across its
edges.
  This operation relies on Laplacian smoothing, well-known in graph signal
processing, which we integrate in the attack pipeline. We benchmark several
attacks with and without smoothing under a white-box scenario and evaluate
their transferability. Despite the additional constraint of smoothness, our
attack has the same probability of success at lower distortion.
Analysis and Extension of the Evidential Reasoning Algorithm for Multiple Attribute Decision Analysis with Uncertainty
Authors: Lianmeng Jiao, Xiaojiao Geng
Date: 2019-03-28 00:00:00
Tasks: Attribute, valid

In multiple attribute decision analysis (MADA) problems, one often needs to
deal with assessment information with uncertainty. The evidential reasoning
approach is one of the most effective methods to deal with such MADA problems.
As kernel of the evidential reasoning approach, an original evidential
reasoning (ER) algorithm was firstly proposed by Yang et al, and later they
modified the ER algorithm in order to satisfy the proposed four synthesis
axioms with which a rational aggregation process needs to satisfy. However, up
to present, the essential difference of the two ER algorithms as well as the
rationality of the synthesis axioms are still unclear. In this paper, we
analyze the ER algorithms in Dempster-Shafer theory (DST) framework and prove
that the original ER algorithm follows the reliability discounting and
combination scheme, while the modified one follows the importance discounting
and combination scheme. Further we reveal that the four synthesis axioms are
not valid criteria to check the rationality of one attribute aggregation
algorithm. Based on these new findings, an extended ER algorithm is proposed to
take into account both the reliability and importance of different attributes,
which provides a more general attribute aggregation scheme for MADA with
uncertainty. A motorcycle performance assessment problem is examined to
illustrate the proposed algorithm.
Feature Intertwiner for Object Detection
Authors: Hongyang Li, Bo Dai, Shaoshuai Shi, Wanli Ouyang, Xiaogang Wang
Date: 2019-03-28 00:00:00
Proceeding: ICLR 2019 5
Tasks: Object, object-detection, Object Detection

A well-trained model should classify objects with a unanimous score for every
category. This requires the high-level semantic features should be as much
alike as possible among samples. To achive this, previous works focus on
re-designing the loss or proposing new regularization constraints. In this
paper, we provide a new perspective. For each category, it is assumed that
there are two feature sets: one with reliable information and the other with
less reliable source. We argue that the reliable set could guide the feature
learning of the less reliable set during training - in spirit of student
mimicking teacher behavior and thus pushing towards a more compact class
centroid in the feature space. Such a scheme also benefits the reliable set
since samples become closer within the same category - implying that it is
easier for the classifier to identify. We refer to this mutual learning process
as feature intertwiner and embed it into object detection. It is well-known
that objects of low resolution are more difficult to detect due to the loss of
detailed information during network forward pass (e.g., RoI operation). We thus
regard objects of high resolution as the reliable set and objects of low
resolution as the less reliable set. Specifically, an intertwiner is designed
to minimize the distribution divergence between two sets. The choice of
generating an effective feature representation for the reliable set is further
investigated, where we introduce the optimal transport (OT) theory into the
framework. Samples in the less reliable set are better aligned with aid of OT
metric. Incorporated with such a plug-and-play intertwiner, we achieve an
evident improvement over previous state-of-the-arts.
Mining Discourse Markers for Unsupervised Sentence Representation Learning
Authors: Damien Sileo, Tim Van-De-Cruys, Camille Pradel, Philippe Muller
Date: 2019-03-28 00:00:00
Proceeding: NAACL 2019 6
Tasks: Relation Classification, Representation Learning, Sentence, Sentence Embeddings

Current state of the art systems in NLP heavily rely on manually annotated
datasets, which are expensive to construct. Very little work adequately
exploits unannotated data -- such as discourse markers between sentences --
mainly because of data sparseness and ineffective extraction methods. In the
present work, we propose a method to automatically discover sentence pairs with
relevant discourse markers, and apply it to massive amounts of data. Our
resulting dataset contains 174 discourse markers with at least 10k examples
each, even for rare markers such as coincidentally or amazingly We use the
resulting data as supervision for learning transferable sentence embeddings. In
addition, we show that even though sentence representation learning through
prediction of discourse markers yields state of the art results across
different transfer tasks, it is not clear that our models made use of the
semantic relation between sentences, thus leaving room for further
improvements. Our datasets are publicly available
(https://github.com/synapse-developpement/Discovery)
Sogou Machine Reading Comprehension Toolkit
Authors: Jindou Wu, Yunlun Yang, Chao Deng, Hongyi Tang, Bingning Wang, Haoze Sun, Ting Yao, Qi Zhang
Date: 2019-03-28 00:00:00
Tasks: Machine Reading Comprehension, Reading Comprehension

Machine reading comprehension have been intensively studied in recent years,
and neural network-based models have shown dominant performances. In this
paper, we present a Sogou Machine Reading Comprehension (SMRC) toolkit that can
be used to provide the fast and efficient development of modern machine
comprehension models, including both published models and original prototypes.
To achieve this goal, the toolkit provides dataset readers, a flexible
preprocessing pipeline, necessary neural network components, and built-in
models, which make the whole process of data preparation, model construction,
and training easier.