pwc-archive/papers-with-abstracts huggingface dataset:
Character-level Chinese-English Translation through ASCII Encoding
Authors: Nikola I. Nikolov, Yuhuang Hu, Mi Xue Tan, Richard H. R. Hahnloser
Date: 2018-05-09 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, NMT, Translation

Character-level Neural Machine Translation (NMT) models have recently
achieved impressive results on many language pairs. They mainly do well for
Indo-European language pairs, where the languages share the same writing
system. However, for translating between Chinese and English, the gap between
the two different writing systems poses a major challenge because of a lack of
systematic correspondence between the individual linguistic units. In this
paper, we enable character-level NMT for Chinese, by breaking down Chinese
characters into linguistic units similar to that of Indo-European languages. We
use the Wubi encoding scheme, which preserves the original shape and semantic
information of the characters, while also being reversible. We show promising
results from training Wubi-based models on the character- and subword-level
with recurrent as well as convolutional models.
Privacy-preserving Prediction
Authors: Cynthia Dwork, Vitaly Feldman
Date: 2018-03-27 00:00:00
Tasks: PAC learning, Prediction, Privacy Preserving, regression

Ensuring differential privacy of models learned from sensitive user data is
an important goal that has been studied extensively in recent years. It is now
known that for some basic learning problems, especially those involving
high-dimensional data, producing an accurate private model requires much more
data than learning without privacy. At the same time, in many applications it
is not necessary to expose the model itself. Instead users may be allowed to
query the prediction model on their inputs only through an appropriate
interface. Here we formulate the problem of ensuring privacy of individual
predictions and investigate the overheads required to achieve it in several
standard models of classification and regression.
  We first describe a simple baseline approach based on training several models
on disjoint subsets of data and using standard private aggregation techniques
to predict. We show that this approach has nearly optimal sample complexity for
(realizable) PAC learning of any class of Boolean functions. At the same time,
without strong assumptions on the data distribution, the aggregation step
introduces a substantial overhead. We demonstrate that this overhead can be
avoided for the well-studied class of thresholds on a line and for a number of
standard settings of convex regression. The analysis of our algorithm for
learning thresholds relies crucially on strong generalization guarantees that
we establish for all differentially private prediction algorithms.
Transfer Learning from Adult to Children for Speech Recognition: Evaluation, Analysis and Recommendations
Authors: Prashanth Gurunath Shivakumar, Panayiotis Georgiou
Date: 2018-05-08 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition, Transfer Learning

Children speech recognition is challenging mainly due to the inherent high
variability in children's physical and articulatory characteristics and
expressions. This variability manifests in both acoustic constructs and
linguistic usage due to the rapidly changing developmental stage in children's
life. Part of the challenge is due to the lack of large amounts of available
children speech data for efficient modeling. This work attempts to address the
key challenges using transfer learning from adult's models to children's models
in a Deep Neural Network (DNN) framework for children's Automatic Speech
Recognition (ASR) task evaluating on multiple children's speech corpora with a
large vocabulary. The paper presents a systematic and an extensive analysis of
the proposed transfer learning technique considering the key factors affecting
children's speech recognition from prior literature. Evaluations are presented
on (i) comparisons of earlier GMM-HMM and the newer DNN Models, (ii)
effectiveness of standard adaptation techniques versus transfer learning, (iii)
various adaptation configurations in tackling the variabilities present in
children speech, in terms of (a) acoustic spectral variability, and (b)
pronunciation variability and linguistic constraints. Our Analysis spans over
(i) number of DNN model parameters (for adaptation), (ii) amount of adaptation
data, (iii) ages of children, (iv) age dependent-independent adaptation.
Finally, we provide Recommendations on (i) the favorable strategies over
various aforementioned - analyzed parameters, and (ii) potential future
research directions and relevant challenges/problems persisting in DNN based
ASR for children's speech.
Subsampling Sequential Monte Carlo for Static Bayesian Models
Authors: David Gunawan, Khue-Dung Dang, Matias Quiroz, Robert Kohn, Minh-Ngoc Tran
Date: 2018-05-08 00:00:00
Tasks: Bayesian Inference

We show how to speed up Sequential Monte Carlo (SMC) for Bayesian inference in large data problems by data subsampling. SMC sequentially updates a cloud of particles through a sequence of distributions, beginning with a distribution that is easy to sample from such as the prior and ending with the posterior distribution. Each update of the particle cloud consists of three steps: reweighting, resampling, and moving. In the move step, each particle is moved using a Markov kernel; this is typically the most computationally expensive part, particularly when the dataset is large. It is crucial to have an efficient move step to ensure particle diversity. Our article makes two important contributions. First, in order to speed up the SMC computation, we use an approximately unbiased and efficient annealed likelihood estimator based on data subsampling. The subsampling approach is more memory efficient than the corresponding full data SMC, which is an advantage for parallel computation. Second, we use a Metropolis within Gibbs kernel with two conditional updates. A Hamiltonian Monte Carlo update makes distant moves for the model parameters, and a block pseudo-marginal proposal is used for the particles corresponding to the auxiliary variables for the data subsampling. We demonstrate both the usefulness and limitations of the methodology for estimating four generalized linear models and a generalized additive model with large datasets.
The Three Pillars of Machine Programming
Authors: Justin Gottschlich, Armando Solar-Lezama, Nesime Tatbul, Michael Carbin, Martin Rinard, Regina Barzilay, Saman Amarasinghe, Joshua B. Tenenbaum, Tim Mattson
Date: 2018-03-20 00:00:00
Tasks: BIG-bench Machine Learning, Position

In this position paper, we describe our vision of the future of machine programming through a categorical examination of three pillars of research. Those pillars are: (i) intention, (ii) invention, and(iii) adaptation. Intention emphasizes advancements in the human-to-computer and computer-to-machine-learning interfaces. Invention emphasizes the creation or refinement of algorithms or core hardware and software building blocks through machine learning (ML). Adaptation emphasizes advances in the use of ML-based constructs to autonomously evolve software.
Investor Reaction to Financial Disclosures Across Topics: An Application of Latent Dirichlet Allocation
Authors: Stefan Feuerriegel, Nicolas Pröllochs
Date: 2018-05-08 00:00:00

This paper provides a holistic study of how stock prices vary in their
response to financial disclosures across different topics. Thereby, we
specifically shed light into the extensive amount of filings for which no a
priori categorization of their content exists. For this purpose, we utilize an
approach from data mining - namely, latent Dirichlet allocation - as a means of
topic modeling. This technique facilitates our task of automatically
categorizing, ex ante, the content of more than 70,000 regulatory 8-K filings
from U.S. companies. We then evaluate the subsequent stock market reaction. Our
empirical evidence suggests a considerable discrepancy among various types of
news stories in terms of their relevance and impact on financial markets. For
instance, we find a statistically significant abnormal return in response to
earnings results and credit rating, but also for disclosures regarding business
strategy, the health sector, as well as mergers and acquisitions. Our results
yield findings that benefit managers, investors and policy-makers by indicating
how regulatory filings should be structured and the topics most likely to
precede changes in stock valuations.
The Effectiveness of Instance Normalization: a Strong Baseline for Single Image Dehazing
Authors: Zheng Xu, Xitong Yang, Xue Li, Xiaoshuai Sun
Date: 2018-05-08 00:00:00
Tasks: Decoder, Image Dehazing, Single Image Dehazing

We propose a novel deep neural network architecture for the challenging
problem of single image dehazing, which aims to recover the clear image from a
degraded hazy image. Instead of relying on hand-crafted image priors or
explicitly estimating the components of the widely used atmospheric scattering
model, our end-to-end system directly generates the clear image from an input
hazy image. The proposed network has an encoder-decoder architecture with skip
connections and instance normalization. We adopt the convolutional layers of
the pre-trained VGG network as encoder to exploit the representation power of
deep features, and demonstrate the effectiveness of instance normalization for
image dehazing. Our simple yet effective network outperforms the
state-of-the-art methods by a large margin on the benchmark datasets.
Highly Scalable Image Reconstruction using Deep Neural Networks with Bandpass Filtering
Authors: Joseph Y. Cheng, Feiyu Chen, Marcus T. Alley, John M. Pauly, Shreyas S. Vasanawala
Date: 2018-05-08 00:00:00
Tasks: Image Reconstruction

To increase the flexibility and scalability of deep neural networks for image
reconstruction, a framework is proposed based on bandpass filtering. For many
applications, sensing measurements are performed indirectly. For example, in
magnetic resonance imaging, data are sampled in the frequency domain. The
introduction of bandpass filtering enables leveraging known imaging physics
while ensuring that the final reconstruction is consistent with actual
measurements to maintain reconstruction accuracy. We demonstrate this flexible
architecture for reconstructing subsampled datasets of MRI scans. The resulting
high subsampling rates increase the speed of MRI acquisitions and enable the
visualization rapid hemodynamics.
Improved training of end-to-end attention models for speech recognition
Authors: Albert Zeyer, Kazuki Irie, Ralf Schlüter, Hermann Ney
Date: 2018-05-08 00:00:00
Tasks: Language Modeling, Language Modelling, Speech Recognition

Sequence-to-sequence attention-based models on subword units allow simple
open-vocabulary end-to-end speech recognition. In this work, we show that such
models can achieve competitive results on the Switchboard 300h and LibriSpeech
1000h tasks. In particular, we report the state-of-the-art word error rates
(WER) of 3.54% on the dev-clean and 3.82% on the test-clean evaluation subsets
of LibriSpeech. We introduce a new pretraining scheme by starting with a high
time reduction factor and lowering it during training, which is crucial both
for convergence and final performance. In some experiments, we also use an
auxiliary CTC loss function to help the convergence. In addition, we train long
short-term memory (LSTM) language models on subword units. By shallow fusion,
we report up to 27% relative improvements in WER over the attention baseline
without a language model.
Fully Automated Segmentation of Hyperreflective Foci in Optical Coherence Tomography Images
Authors: Thomas Schlegl, Hrvoje Bogunovic, Sophie Klimscha, Philipp Seeböck, Amir Sadeghipour, Bianca Gerendas, Sebastian M. Waldstein, Georg Langs, Ursula Schmidt-Erfurth
Date: 2018-05-08 00:00:00

The automatic detection of disease related entities in retinal imaging data
is relevant for disease- and treatment monitoring. It enables the quantitative
assessment of large amounts of data and the corresponding study of disease
characteristics. The presence of hyperreflective foci (HRF) is related to
disease progression in various retinal diseases. Manual identification of HRF
in spectral-domain optical coherence tomography (SD-OCT) scans is error-prone
and tedious. We present a fully automated machine learning approach for
segmenting HRF in SD-OCT scans. Evaluation on annotated OCT images of the
retina demonstrates that a residual U-Net allows to segment HRF with high
accuracy. As our dataset comprised data from different retinal diseases
including age-related macular degeneration, diabetic macular edema and retinal
vein occlusion, the algorithm can safely be applied in all of them though
different pathophysiological origins are known.
Urban Land Cover Classification with Missing Data Modalities Using Deep Convolutional Neural Networks
Authors: Michael Kampffmeyer, Arnt-Børre Salberg, Robert Jenssen
Date: 2017-09-21 00:00:00
Tasks: Decision Making, General Classification, Hallucination, Land Cover Classification

Automatic urban land cover classification is a fundamental problem in remote
sensing, e.g. for environmental monitoring. The problem is highly challenging,
as classes generally have high inter-class and low intra-class variance.
Techniques to improve urban land cover classification performance in remote
sensing include fusion of data from different sensors with different data
modalities. However, such techniques require all modalities to be available to
the classifier in the decision-making process, i.e. at test time, as well as in
training. If a data modality is missing at test time, current state-of-the-art
approaches have in general no procedure available for exploiting information
from these modalities. This represents a waste of potentially useful
information. We propose as a remedy a convolutional neural network (CNN)
architecture for urban land cover classification which is able to embed all
available training modalities in a so-called hallucination network. The network
will in effect replace missing data modalities in the test phase, enabling
fusion capabilities even when data modalities are missing in testing. We
demonstrate the method using two datasets consisting of optical and digital
surface model (DSM) images. We simulate missing modalities by assuming that DSM
images are missing during testing. Our method outperforms both standard CNNs
trained only on optical images as well as an ensemble of two standard CNNs. We
further evaluate the potential of our method to handle situations where only
some DSM images are missing during testing. Overall, we show that we can
clearly exploit training time information of the missing modality during
testing.
Sparse Blind Deconvolution for Distributed Radar Autofocus Imaging
Authors: Hassan Mansour, Dehong Liu, Ulugbek S. Kamilov, Petros T. Boufounos
Date: 2018-05-08 00:00:00
Tasks: Position

A common problem that arises in radar imaging systems, especially those
mounted on mobile platforms, is antenna position ambiguity. Approaches to
resolve this ambiguity and correct position errors are generally known as radar
autofocus. Common techniques that attempt to resolve the antenna ambiguity
generally assume an unknown gain and phase error afflicting the radar
measurements. However, ensuring identifiability and tractability of the unknown
error imposes strict restrictions on the allowable antenna perturbations.
Furthermore, these techniques are often not applicable in near-field imaging,
where mapping the position ambiguity to phase errors breaks down.
  In this paper, we propose an alternate formulation where the position error
of each antenna is mapped to a spatial shift operator in the image-domain.
Thus, the radar autofocus problem becomes a multichannel blind deconvolution
problem, in which the radar measurements correspond to observations of a static
radar image that is convolved with the spatial shift kernel associated with
each antenna. To solve the reformulated problem, we also develop a block
coordinate descent framework that leverages the sparsity and piece-wise
smoothness of the radar scene, as well as the one-sparse property of the two
dimensional shift kernels. We evaluate the performance of our approach using
both simulated and experimental radar measurements, and demonstrate its
superior performance compared to state-of-the-art methods.
Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog
Authors: Jiaping Zhang, Tiancheng Zhao, Zhou Yu
Date: 2018-05-08 00:00:00
Proceeding: WS 2018 7
Tasks: Hierarchical Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Text Generation, Visual Dialog

Creating an intelligent conversational system that understands vision and
language is one of the ultimate goals in Artificial Intelligence
(AI)~\cite{winograd1972understanding}. Extensive research has focused on
vision-to-language generation, however, limited research has touched on
combining these two modalities in a goal-driven dialog context. We propose a
multimodal hierarchical reinforcement learning framework that dynamically
integrates vision and language for task-oriented visual dialog. The framework
jointly learns the multimodal dialog state representation and the hierarchical
dialog policy to improve both dialog task success and efficiency. We also
propose a new technique, state adaptation, to integrate context awareness in
the dialog state representation. We evaluate the proposed framework and the
state adaptation technique in an image guessing game and achieve promising
results.
Word2Vec applied to Recommendation: Hyperparameters Matter
Authors: Hugo Caselles-Dupré, Florian Lesaint, Jimena Royo-Letelier
Date: 2018-04-11 00:00:00
Tasks: Word Embeddings

Skip-gram with negative sampling, a popular variant of Word2vec originally
designed and tuned to create word embeddings for Natural Language Processing,
has been used to create item embeddings with successful applications in
recommendation. While these fields do not share the same type of data, neither
evaluate on the same tasks, recommendation applications tend to use the same
already tuned hyperparameters values, even if optimal hyperparameters values
are often known to be data and task dependent. We thus investigate the marginal
importance of each hyperparameter in a recommendation setting through large
hyperparameter grid searches on various datasets. Results reveal that
optimizing neglected hyperparameters, namely negative sampling distribution,
number of epochs, subsampling parameter and window-size, significantly improves
performance on a recommendation task, and can increase it by an order of
magnitude. Importantly, we find that optimal hyperparameters configurations for
Natural Language Processing tasks and Recommendation tasks are noticeably
different.
Post-Specialisation: Retrofitting Vectors of Words Unseen in Lexical Resources
Authors: Ivan Vulić, Goran Glavaš, Nikola Mrkšić, Anna Korhonen
Date: 2018-05-08 00:00:00
Proceeding: NAACL 2018 6
Tasks: Dialogue State Tracking, Text Simplification, Word Similarity

Word vector specialisation (also known as retrofitting) is a portable,
light-weight approach to fine-tuning arbitrary distributional word vector
spaces by injecting external knowledge from rich lexical resources such as
WordNet. By design, these post-processing methods only update the vectors of
words occurring in external lexicons, leaving the representations of all unseen
words intact. In this paper, we show that constraint-driven vector space
specialisation can be extended to unseen words. We propose a novel
post-specialisation method that: a) preserves the useful linguistic knowledge
for seen words; while b) propagating this external signal to unseen words in
order to improve their vector representations as well. Our post-specialisation
approach explicits a non-linear specialisation function in the form of a deep
neural network by learning to predict specialised vectors from their original
distributional counterparts. The learned function is then used to specialise
vectors of unseen words. This approach, applicable to any post-processing
model, yields considerable gains over the initial specialisation models both in
intrinsic word similarity tasks, and in two downstream tasks: dialogue state
tracking and lexical text simplification. The positive effects persist across
three languages, demonstrating the importance of specialising the full
vocabulary of distributional word vector spaces.
MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving
Authors: Marvin Teichmann, Michael Weber, Marius Zoellner, Roberto Cipolla, Raquel Urtasun
Date: 2016-12-22 00:00:00
Tasks: Autonomous Driving, General Classification, Road Segmentation, Segmentation, Semantic Segmentation

While most approaches to semantic reasoning have focused on improving
performance, in this paper we argue that computational times are very important
in order to enable real time applications such as autonomous driving. Towards
this goal, we present an approach to joint classification, detection and
semantic segmentation via a unified architecture where the encoder is shared
amongst the three tasks. Our approach is very simple, can be trained end-to-end
and performs extremely well in the challenging KITTI dataset, outperforming the
state-of-the-art in the road segmentation task. Our approach is also very
efficient, taking less than 100 ms to perform all tasks.
A Mixed Classification-Regression Framework for 3D Pose Estimation from 2D Images
Authors: Siddharth Mahendran, Haider Ali, Rene Vidal
Date: 2018-05-08 00:00:00
Tasks: 3D Pose Estimation, Autonomous Driving, Classification, General Classification, Pose Estimation, regression, Robot Manipulation

3D pose estimation from a single 2D image is an important and challenging
task in computer vision with applications in autonomous driving, robot
manipulation and augmented reality. Since 3D pose is a continuous quantity, a
natural formulation for this task is to solve a pose regression problem.
However, since pose regression methods return a single estimate of the pose,
they have difficulties handling multimodal pose distributions (e.g. in the case
of symmetric objects). An alternative formulation, which can capture multimodal
pose distributions, is to discretize the pose space into bins and solve a pose
classification problem. However, pose classification methods can give large
pose estimation errors depending on the coarseness of the discretization. In
this paper, we propose a mixed classification-regression framework that uses a
classification network to produce a discrete multimodal pose estimate and a
regression network to produce a continuous refinement of the discrete estimate.
The proposed framework can accommodate different architectures and loss
functions, leading to multiple classification-regression models, some of which
achieve state-of-the-art performance on the challenging Pascal3D+ dataset.