pwc-archive/papers-with-abstracts huggingface dataset:
APE at Scale and its Implications on MT Evaluation Biases
Authors: Markus Freitag, Isaac Caswell, Scott Roy
Date: 2019-04-09 00:00:00
Proceeding: WS 2019 8
Tasks: Automatic Post-Editing, Machine Translation, NMT, Translation

In this work, we train an Automatic Post-Editing (APE) model and use it to reveal biases in standard Machine Translation (MT) evaluation procedures. The goal of our APE model is to correct typical errors introduced by the translation process, and convert the "translationese" output into natural text. Our APE model is trained entirely on monolingual data that has been round-trip translated through English, to mimic errors that are similar to the ones introduced by NMT. We apply our model to the output of existing NMT systems, and demonstrate that, while the human-judged quality improves in all cases, BLEU scores drop with forward-translated test sets. We verify these results for the WMT18 English to German, WMT15 English to French, and WMT16 English to Romanian tasks. Furthermore, we selectively apply our APE model on the output of the top submissions of the most recent WMT evaluation campaigns. We see quality improvements on all tasks of up to 2.5 BLEU points.
Approximation in $L^p(μ)$ with deep ReLU neural networks
Authors: Felix Voigtlaender, Philipp Petersen
Date: 2019-04-09 00:00:00
Tasks: Learning Theory

We discuss the expressive power of neural networks which use the non-smooth
ReLU activation function $\varrho(x) = \max\{0,x\}$ by analyzing the
approximation theoretic properties of such networks. The existing results
mainly fall into two categories: approximation using ReLU networks with a fixed
depth, or using ReLU networks whose depth increases with the approximation
accuracy. After reviewing these findings, we show that the results concerning
networks with fixed depth--- which up to now only consider approximation in
$L^p(\lambda)$ for the Lebesgue measure $\lambda$--- can be generalized to
approximation in $L^p(\mu)$, for any finite Borel measure $\mu$. In particular,
the generalized results apply in the usual setting of statistical learning
theory, where one is interested in approximation in $L^2(\mathbb{P})$, with the
probability measure $\mathbb{P}$ describing the distribution of the data.
Time-Series Analysis via Low-Rank Matrix Factorization Applied to Infant-Sleep Data
Authors: Sheng Liu, Mark Cheng, Hayley Brooks, Wayne Mackey, David J. Heeger, Esteban G. Tabak, Carlos Fernandez-Granda
Date: 2019-04-09 00:00:00
Tasks: Time Series, Time Series Analysis

We propose a nonparametric model for time series with missing data based on low-rank matrix factorization. The model expresses each instance in a set of time series as a linear combination of a small number of shared basis functions. Constraining the functions and the corresponding coefficients to be nonnegative yields an interpretable low-dimensional representation of the data. A time-smoothing regularization term ensures that the model captures meaningful trends in the data, instead of overfitting short-term fluctuations. The low-dimensional representation makes it possible to detect outliers and cluster the time series according to the interpretable features extracted by the model, and also to perform forecasting via kernel regression. We apply our methodology to a large real-world dataset of infant-sleep data gathered by caregivers with a mobile-phone app. Our analysis automatically extracts daily-sleep patterns consistent with the existing literature. This allows us to compute sleep-development trends for the cohort, which characterize the emergence of circadian sleep and different napping habits. We apply our methodology to detect anomalous individuals, to cluster the cohort into groups with different sleeping tendencies, and to obtain improved predictions of future sleep behavior.
Multi-Agent Tensor Fusion for Contextual Trajectory Prediction
Authors: Tianyang Zhao, Yifei Xu, Mathew Monfort, Wongun Choi, Chris Baker, Yibiao Zhao, Yizhou Wang, Ying Nian Wu
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Autonomous Driving, Prediction, Trajectory Prediction

Accurate prediction of others' trajectories is essential for autonomous driving. Trajectory prediction is challenging because it requires reasoning about agents' past movements, social interactions among varying numbers and kinds of agents, constraints from the scene context, and the stochasticity of human behavior. Our approach models these interactions and constraints jointly within a novel Multi-Agent Tensor Fusion (MATF) network. Specifically, the model encodes multiple agents' past trajectories and the scene context into a Multi-Agent Tensor, then applies convolutional fusion to capture multiagent interactions while retaining the spatial structure of agents and the scene context. The model decodes recurrently to multiple agents' future trajectories, using adversarial loss to learn stochastic predictions. Experiments on both highway driving and pedestrian crowd datasets show that the model achieves state-of-the-art prediction accuracy.
A New GAN-based End-to-End TTS Training Algorithm
Authors: Haohan Guo, Frank K. Soong, Lei He, Lei Xie
Date: 2019-04-09 00:00:00
Tasks: Generative Adversarial Network, Sentence, Transfer Learning

End-to-end, autoregressive model-based TTS has shown significant performance
improvements over the conventional one. However, the autoregressive module
training is affected by the exposure bias, or the mismatch between the
different distributions of real and predicted data. While real data is
available in training, but in testing, only predicted data is available to feed
the autoregressive module. By introducing both real and generated data
sequences in training, we can alleviate the effects of the exposure bias. We
propose to use Generative Adversarial Network (GAN) along with the key idea of
Professor Forcing in training. A discriminator in GAN is jointly trained to
equalize the difference between real and predicted data. In AB subjective
listening test, the results show that the new approach is preferred over the
standard transfer learning with a CMOS improvement of 0.1. Sentence level
intelligibility tests show significant improvement in a pathological test set.
The GAN-trained new model is also more stable than the baseline to produce
better alignments for the Tacotron output.
Adversarial Learning of Disentangled and Generalizable Representations for Visual Attributes
Authors: James Oldfield, Yannis Panagakis, Mihalis A. Nicolaou
Date: 2019-04-09 00:00:00
Tasks: Attribute, Image-to-Image Translation, Translation

Recently, a multitude of methods for image-to-image translation have demonstrated impressive results on problems such as multi-domain or multi-attribute transfer. The vast majority of such works leverages the strengths of adversarial learning and deep convolutional autoencoders to achieve realistic results by well-capturing the target data distribution. Nevertheless, the most prominent representatives of this class of methods do not facilitate semantic structure in the latent space, and usually rely on binary domain labels for test-time transfer. This leads to rigid models, unable to capture the variance of each domain label. In this light, we propose a novel adversarial learning method that (i) facilitates the emergence of latent structure by semantically disentangling sources of variation, and (ii) encourages learning generalizable, continuous, and transferable latent codes that enable flexible attribute mixing. This is achieved by introducing a novel loss function that encourages representations to result in uniformly distributed class posteriors for disentangled attributes. In tandem with an algorithm for inducing generalizable properties, the resulting representations can be utilized for a variety of tasks such as intensity-preserving multi-attribute image translation and synthesis, without requiring labelled test data. We demonstrate the merits of the proposed method by a set of qualitative and quantitative experiments on popular databases such as MultiPIE, RaFD, and BU-3DFE, where our method outperforms other, state-of-the-art methods in tasks such as intensity-preserving multi-attribute transfer and synthesis.
Generic Variance Bounds on Estimation and Prediction Errors in Time Series Analysis: An Entropy Perspective
Authors: Song Fang, Mikael Skoglund, Karl Henrik Johansson, Hideaki Ishii, Quanyan Zhu
Date: 2019-04-09 00:00:00
Tasks: Gaussian Processes, Prediction, Time Series, Time Series Analysis

In this paper, we obtain generic bounds on the variances of estimation and prediction errors in time series analysis via an information-theoretic approach. It is seen in general that the error bounds are determined by the conditional entropy of the data point to be estimated or predicted given the side information or past observations. Additionally, we discover that in order to achieve the prediction error bounds asymptotically, the necessary and sufficient condition is that the "innovation" is asymptotically white Gaussian. When restricted to Gaussian processes and 1-step prediction, our bounds are shown to reduce to the Kolmogorov-Szeg\"o formula and Wiener-Masani formula known from linear prediction theory.
Exploiting Syntactic Features in a Parsed Tree to Improve End-to-End TTS
Authors: Haohan Guo, Frank K. Soong, Lei He, Lei Xie
Date: 2019-04-09 00:00:00
Tasks: Sentence

The end-to-end TTS, which can predict speech directly from a given sequence
of graphemes or phonemes, has shown improved performance over the conventional
TTS. However, its predicting capability is still limited by the
acoustic/phonetic coverage of the training data, usually constrained by the
training set size. To further improve the TTS quality in pronunciation, prosody
and perceived naturalness, we propose to exploit the information embedded in a
syntactically parsed tree where the inter-phrase/word information of a sentence
is organized in a multilevel tree structure. Specifically, two key features:
phrase structure and relations between adjacent words are investigated.
Experimental results in subjective listening, measured on three test sets, show
that the proposed approach is effective to improve the pronunciation clarity,
prosody and naturalness of the synthesized speech of the baseline system.
Hypothesis Set Stability and Generalization
Authors: Dylan J. Foster, Spencer Greenberg, Satyen Kale, Haipeng Luo, Mehryar Mohri, Karthik Sridharan
Date: 2019-04-09 00:00:00
Proceeding: NeurIPS 2019 12

We present a study of generalization for data-dependent hypothesis sets. We give a general learning guarantee for data-dependent hypothesis sets based on a notion of transductive Rademacher complexity. Our main result is a generalization bound for data-dependent hypothesis sets expressed in terms of a notion of hypothesis set stability and a notion of Rademacher complexity for data-dependent hypothesis sets that we introduce. This bound admits as special cases both standard Rademacher complexity bounds and algorithm-dependent uniform stability bounds. We also illustrate the use of these learning bounds in the analysis of several scenarios.
A Hybrid Evolutionary System for Automated Artificial Neural Networks Generation and Simplification in Biomedical Applications
Authors: Enrique Fernandez-Blanco, Daniel Rivero, Marcos Gestal, Carlos Fernandez-Lozano, Norberto Ezquerra, Cristian R. Munteanu, Julian Dorado
Date: 2019-04-09 00:00:00

Data mining and data classification over biomedical data are two of the most
important research fields in computer science. Among the great diversity of
techniques that can be used for this purpose, Artifical Neural Networks (ANNs)
is one of the most suited. One of the main problems in the development of this
technique is the slow performance of the full process. Traditionally, in this
development process, human experts are needed to experiment with different
architectural procedures until they find the one that presents the correct
results for solving a specific problem. However, many studies have emerged in
which different ANN developmental techniques, more or less automated, are
described. In this paper, the authors have focused on developing a new
technique to perform this process over biomedical data. The new technique is
described in which two Evolutionary Computation (EC) techniques are mixed to
automatically develop ANNs. These techniques are Genetic Algorithms and Genetic
Programming. The work goes further, and the system described here allows to
obtain simplified networks with a low number of neurons to resolve the
problems. The system is compared with the already existent system which also
uses EC over a set of well-known problems. The conclusions reached from these
comparisons indicate that this new system produces very good results, which in
the worst case are at least comparable to existing techniques and in many cases
are substantially better.
User-Controllable Multi-Texture Synthesis with Generative Adversarial Networks
Authors: Aibek Alanov, Max Kochurov, Denis Volkhonskiy, Daniil Yashkov, Evgeny Burnaev, Dmitry Vetrov
Date: 2019-04-09 00:00:00
Tasks: Descriptive, Texture Synthesis

We propose a novel multi-texture synthesis model based on generative
adversarial networks (GANs) with a user-controllable mechanism. The user
control ability allows to explicitly specify the texture which should be
generated by the model. This property follows from using an encoder part which
learns a latent representation for each texture from the dataset. To ensure a
dataset coverage, we use an adversarial loss function that penalizes for
incorrect reproductions of a given texture. In experiments, we show that our
model can learn descriptive texture manifolds for large datasets and from raw
data such as a collection of high-resolution photos. Moreover, we apply our
method to produce 3D textures and show that it outperforms existing baselines.
Segmentation of Skeletal Muscle in Thigh Dixon MRI Based on Texture Analysis
Authors: Rafael Rodrigues, Antonio M. G. Pinheiro
Date: 2019-04-09 00:00:00
Tasks: Segmentation, Texture Classification

Segmentation of skeletal muscles in Magnetic Resonance Images (MRI) is
essential for the study of muscle physiology and diagnosis of muscular
pathologies. However, manual segmentation of large MRI volumes is a
time-consuming task. The state-of-the-art on algorithms for muscle segmentation
in MRI is still not very extensive and is somewhat database-dependent. In this
paper, an automated segmentation method based on AdaBoost classification of
local texture features is presented. The texture descriptor consists of the
Histogram of Oriented Gradients (HOG), Wavelet-based features, and a set of
statistical measures computed from both the original and the Laplacian of
Gaussian filtering of the grayscale MRI. The classifier performance suggests
that texture analysis may be a helpful tool for designing a generalized and
automated MRI muscle segmentation framework. Furthermore, an atlas-based
approach to individual muscle segmentation is also described in this paper. The
atlas is obtained by overlaying the muscle segmentation ground truth, provided
by a radiologist, after image alignment using an appropriate affine
transformation. Then, it is used to define the muscle labels upon the AdaBoost
binary segmentation. The developed atlas method provides reasonable results
when an accurate muscle tissue segmentation was obtained.
Learning Across Tasks and Domains
Authors: Pierluigi Zama Ramirez, Alessio Tonioni, Samuele Salti, Luigi Di Stefano
Date: 2019-04-09 00:00:00
Proceeding: ICCV 2019 10
Tasks: Depth Estimation, Domain Adaptation, Monocular Depth Estimation, Semantic Segmentation

Recent works have proven that many relevant visual tasks are closely related one to another. Yet, this connection is seldom deployed in practice due to the lack of practical methodologies to transfer learned concepts across different training processes. In this work, we introduce a novel adaptation framework that can operate across both task and domains. Our framework learns to transfer knowledge across tasks in a fully supervised domain (e.g., synthetic data) and use this knowledge on a different domain where we have only partial supervision (e.g., real data). Our proposal is complementary to existing domain adaptation techniques and extends them to cross tasks scenarios providing additional performance gains. We prove the effectiveness of our framework across two challenging tasks (i.e., monocular depth estimation and semantic segmentation) and four different domains (Synthia, Carla, Kitti, and Cityscapes).
Bilingual-GAN: A Step Towards Parallel Text Generation
Authors: Ahmad Rashid, Alan Do-Omri, Md. Akmal Haidar, Qun Liu, Mehdi Rezagholizadeh
Date: 2019-04-09 00:00:00
Proceeding: WS 2019 6
Tasks: Decoder, Denoising, Machine Translation, Text Generation, Translation, Unsupervised Machine Translation

Latent space based GAN methods and attention based sequence to sequence models have achieved impressive results in text generation and unsupervised machine translation respectively. Leveraging the two domains, we propose an adversarial latent space based model capable of generating parallel sentences in two languages concurrently and translating bidirectionally. The bilingual generation goal is achieved by sampling from the latent space that is shared between both languages. First two denoising autoencoders are trained, with shared encoders and back-translation to enforce a shared latent state between the two languages. The decoder is shared for the two translation directions. Next, a GAN is trained to generate synthetic "code" mimicking the languages' shared latent space. This code is then fed into the decoder to generate text in either language. We perform our experiments on Europarl and Multi30k datasets, on the English-French language pair, and document our performance using both supervised and unsupervised machine translation.
Generative Models for Novelty Detection: Applications in abnormal event and situational change detection from data series
Authors: Mahdyar Ravanbakhsh
Date: 2019-04-09 00:00:00
Tasks: Change Detection, Novelty Detection, Outlier Detection

Novelty detection is a process for distinguishing the observations that
differ in some respect from the observations that the model is trained on.
Novelty detection is one of the fundamental requirements of a good
classification or identification system since sometimes the test data contains
observations that were not known at the training time. In other words, the
novelty class is often is not presented during the training phase or not well
defined.
  In light of the above, one-class classifiers and generative methods can
efficiently model such problems. However, due to the unavailability of data
from the novelty class, training an end-to-end model is a challenging task
itself. Therefore, detecting the Novel classes in unsupervised and
semi-supervised settings is a crucial step in such tasks.
  In this thesis, we propose several methods to model the novelty detection
problem in unsupervised and semi-supervised fashion. The proposed frameworks
applied to different related applications of anomaly and outlier detection
tasks. The results show the superior of our proposed methods in compare to the
baselines and state-of-the-art methods.
Software and application patterns for explanation methods
Authors: Maximilian Alber
Date: 2019-04-09 00:00:00
Tasks: Autonomous Driving

Deep neural networks successfully pervaded many applications domains and are
increasingly used in critical decision processes. Understanding their workings
is desirable or even required to further foster their potential as well as to
access sensitive domains like medical applications or autonomous driving. One
key to this broader usage of explaining frameworks is the accessibility and
understanding of respective software. In this work we introduce software and
application patterns for explanation techniques that aim to explain individual
predictions of neural networks. We discuss how to code well-known algorithms
efficiently within deep learning software frameworks and describe how to embed
algorithms in downstream implementations. Building on this we show how
explanation methods can be used in applications to understand predictions for
miss-classified samples, to compare algorithms or networks, and to examine the
focus of networks. Furthermore, we review available open-source packages and
discuss challenges posed by complex and evolving neural network structures to
explanation algorithm development and implementations.
Seq2Biseq: Bidirectional Output-wise Recurrent Neural Networks for Sequence Modelling
Authors: Marco Dinarelli, Loïc Grobol
Date: 2019-04-09 00:00:00

During the last couple of years, Recurrent Neural Networks (RNN) have reached
state-of-the-art performances on most of the sequence modelling problems. In
particular, the "sequence to sequence" model and the neural CRF have proved to
be very effective in this domain. In this article, we propose a new RNN
architecture for sequence labelling, leveraging gated recurrent layers to take
arbitrarily long contexts into account, and using two decoders operating
forward and backward. We compare several variants of the proposed solution and
their performances to the state-of-the-art. Most of our results are better than
the state-of-the-art or very close to it and thanks to the use of recent
technologies, our architecture can scale on corpora larger than those used in
this work.
A Note on the Equivalence of Upper Confidence Bounds and Gittins Indices for Patient Agents
Authors: Daniel Russo
Date: 2019-04-09 00:00:00

This note gives a short, self-contained, proof of a sharp connection between
Gittins indices and Bayesian upper confidence bound algorithms. I consider a
Gaussian multi-armed bandit problem with discount factor $\gamma$. The Gittins
index of an arm is shown to equal the $\gamma$-quantile of the posterior
distribution of the arm's mean plus an error term that vanishes as $\gamma\to
1$. In this sense, for sufficiently patient agents, a Gittins index measures
the highest plausible mean-reward of an arm in a manner equivalent to an upper
confidence bound.
Towards Safety Verification of Direct Perception Neural Networks
Authors: Chih-Hong Cheng, Chung-Hao Huang, Thomas Brunner, Vahid Hashemi
Date: 2019-04-09 00:00:00
Tasks: Autonomous Vehicles

We study the problem of safety verification of direct perception neural networks, where camera images are used as inputs to produce high-level features for autonomous vehicles to make control decisions. Formal verification of direct perception neural networks is extremely challenging, as it is difficult to formulate the specification that requires characterizing input as constraints, while the number of neurons in such a network can reach millions. We approach the specification problem by learning an input property characterizer which carefully extends a direct perception neural network at close-to-output layers, and address the scalability problem by a novel assume-guarantee based verification approach. The presented workflow is used to understand a direct perception neural network (developed by Audi) which computes the next waypoint and orientation for autonomous vehicles to follow.
Practical Open-Loop Optimistic Planning
Authors: Edouard Leurent, Odalric-Ambrym Maillard
Date: 2019-04-09 00:00:00

We consider the problem of online planning in a Markov Decision Process when
given only access to a generative model, restricted to open-loop policies -
i.e. sequences of actions - and under budget constraint. In this setting, the
Open-Loop Optimistic Planning (OLOP) algorithm enjoys good theoretical
guarantees but is overly conservative in practice, as we show in numerical
experiments. We propose a modified version of the algorithm with tighter
upper-confidence bounds, KLOLOP, that leads to better practical performances
while retaining the sample complexity bound. Finally, we propose an efficient
implementation that significantly improves the time complexity of both
algorithms.