pwc-archive/papers-with-abstracts huggingface dataset:
The Fine Line between Linguistic Generalization and Failure in Seq2Seq-Attention Models
Authors: Noah Weber, Leena Shekhar, Niranjan Balasubramanian
Date: 2018-05-03 00:00:00
Proceeding: WS 2018 6

Seq2Seq based neural architectures have become the go-to architecture to
apply to sequence to sequence language tasks. Despite their excellent
performance on these tasks, recent work has noted that these models usually do
not fully capture the linguistic structure required to generalize beyond the
dense sections of the data distribution \cite{ettinger2017towards}, and as
such, are likely to fail on samples from the tail end of the distribution (such
as inputs that are noisy \citep{belkinovnmtbreak} or of different lengths
\citep{bentivoglinmtlength}). In this paper, we look at a model's ability to
generalize on a simple symbol rewriting task with a clearly defined structure.
We find that the model's ability to generalize this structure beyond the
training distribution depends greatly on the chosen random seed, even when
performance on the standard test set remains the same. This suggests that a
model's ability to capture generalizable structure is highly sensitive.
Moreover, this sensitivity may not be apparent when evaluating it on standard
test sets.
Learning image-to-image translation using paired and unpaired training samples
Authors: Soumya Tripathy, Juho Kannala, Esa Rahtu
Date: 2018-05-08 00:00:00
Tasks: Image-to-Image Translation, Translation

Image-to-image translation is a general name for a task where an image from
one domain is converted to a corresponding image in another domain, given
sufficient training data. Traditionally different approaches have been proposed
depending on whether aligned image pairs or two sets of (unaligned) examples
from both domains are available for training. While paired training samples
might be difficult to obtain, the unpaired setup leads to a highly
under-constrained problem and inferior results. In this paper, we propose a new
general purpose image-to-image translation model that is able to utilize both
paired and unpaired training data simultaneously. We compare our method with
two strong baselines and obtain both qualitatively and quantitatively improved
results. Our model outperforms the baselines also in the case of purely paired
and unpaired training data. To our knowledge, this is the first work to
consider such hybrid setup in image-to-image translation.
Superresolution method for data deconvolution by superposition of point sources
Authors: Sandra Martínez, Oscar E. Martínez
Date: 2018-05-08 00:00:00
Tasks: Retrieval, Super-Resolution

In this work we present a new algorithm for data deconvolution that allows
the retrieval of the target function with super-resolution with a simple
approach that after a precis e measurement of the instrument response function
(IRF), the measured data are fit by a superposition of point sources (SUPPOSe)
of equal intensity. In this manner only the positions of the sources need to be
determined by an algorithm that minimizes the norm of the difference between
the measured data and the convolution of the superposed point sources with the
IRF. An upper bound for the uncertainty in the position of the sources was
derived and two very different experimental situations were used for the test
(an optical spectrum and fluorescent microscopy images) showing excellent
reconstructions and agreement with the predicted uncertainties, achieving
{\lambda}/10 resolution for the microscope and a fivefold improvement in the
spectral resolution for the spectrometer. The method also provides a way to
determine the optimum number of sources to be used for the fit.
Superconducting Optoelectronic Neurons IV: Transmitter Circuits
Authors: Jeffrey M. Shainline, Adam N. McCaughan, Sonia M. Buckley, Richard P. Mirin, Sae Woo Nam
Date: 2018-05-04 00:00:00

A superconducting optoelectronic neuron will produce a small current pulse
upon reaching threshold. We present an amplifier chain that converts this small
current pulse to a voltage pulse sufficient to produce light from a
semiconductor diode. This light is the signal used to communicate between
neurons in the network. The amplifier chain comprises a thresholding Josephson
junction, a relaxation oscillator Josephson junction, a superconducting
thin-film current-gated current amplifier, and a superconducting thin-film
current-gated voltage amplifier. We analyze the performance of the elements in
the amplifier chain in the time domain to calculate the energy consumption per
photon created for several values of light-emitting diode capacitance and
efficiency. The speed of the amplification sequence allows neuronal firing up
to at least 20\,MHz with power density low enough to be cooled easily with
standard $^4$He cryogenic systems operating at 4.2\,K.
Polite Dialogue Generation Without Parallel Data
Authors: Tong Niu, Mohit Bansal
Date: 2018-05-08 00:00:00
Proceeding: TACL 2018 1
Tasks: Decoder, Dialogue Generation, Language Modeling, Language Modelling, Reinforcement Learning, Response Generation, Retrieval

Stylistic dialogue response generation, with valuable applications in
personality-based conversational agents, is a challenging task because the
response needs to be fluent, contextually-relevant, as well as
paralinguistically accurate. Moreover, parallel datasets for
regular-to-stylistic pairs are usually unavailable. We present three
weakly-supervised models that can generate diverse polite (or rude) dialogue
responses without parallel data. Our late fusion model (Fusion) merges the
decoder of an encoder-attention-decoder dialogue model with a language model
trained on stand-alone polite utterances. Our label-fine-tuning (LFT) model
prepends to each source sequence a politeness-score scaled label (predicted by
our state-of-the-art politeness classifier) during training, and at test time
is able to generate polite, neutral, and rude responses by simply scaling the
label embedding by the corresponding score. Our reinforcement learning model
(Polite-RL) encourages politeness generation by assigning rewards proportional
to the politeness classifier score of the sampled response. We also present two
retrieval-based polite dialogue model baselines. Human evaluation validates
that while the Fusion and the retrieval-based models achieve politeness with
poorer context-relevance, the LFT and Polite-RL models can produce
significantly more polite responses without sacrificing dialogue quality.
PAD-Net: A Perception-Aided Single Image Dehazing Network
Authors: Yu Liu, Guanlong Zhao
Date: 2018-05-08 00:00:00
Tasks: Image Dehazing, MS-SSIM, Single Image Dehazing, SSIM

In this work, we investigate the possibility of replacing the $\ell_2$ loss
with perceptually derived loss functions (SSIM, MS-SSIM, etc.) in training an
end-to-end dehazing neural network. Objective experimental results suggest that
by merely changing the loss function we can obtain significantly higher PSNR
and SSIM scores on the SOTS set in the RESIDE dataset, compared with a
state-of-the-art end-to-end dehazing neural network (AOD-Net) that uses the
$\ell_2$ loss. The best PSNR we obtained was 23.50 (4.2% relative improvement),
and the best SSIM we obtained was 0.8747 (2.3% relative improvement.)
Parallel Computation of PDFs on Big Spatial Data Using Spark
Authors: Ji Liu, Noel Moreno Lemus, Esther Pacitti, Fabio Porto, Patrick Valduriez
Date: 2018-05-08 00:00:00
Tasks: Seismic Interpretation

We consider big spatial data, which is typically produced in scientific areas
such as geological or seismic interpretation. The spatial data can be produced
by observation (e.g. using sensors or soil instrument) or numerical simulation
programs and correspond to points that represent a 3D soil cube area. However,
errors in signal processing and modeling create some uncertainty, and thus a
lack of accuracy in identifying geological or seismic phenomenons. Such
uncertainty must be carefully analyzed. To analyze uncertainty, the main
solution is to compute a Probability Density Function (PDF) of each point in
the spatial cube area. However, computing PDFs on big spatial data can be very
time consuming (from several hours to even months on a parallel computer). In
this paper, we propose a new solution to efficiently compute such PDFs in
parallel using Spark, with three methods: data grouping, machine learning
prediction and sampling. We evaluate our solution by extensive experiments on
different computer clusters using big data ranging from hundreds of GB to
several TB. The experimental results show that our solution scales up very well
and can reduce the execution time by a factor of 33 (in the order of seconds or
minutes) compared with a baseline method.
Image Retrieval with Mixed Initiative and Multimodal Feedback
Authors: Nils Murrugarra-Llerena, Adriana Kovashka
Date: 2018-05-08 00:00:00
Tasks: Attribute, Image Retrieval, Informativeness, Reinforcement Learning, Retrieval

How would you search for a unique, fashionable shoe that a friend wore and
you want to buy, but you didn't take a picture? Existing approaches propose
interactive image search as a promising venue. However, they either entrust the
user with taking the initiative to provide informative feedback, or give all
control to the system which determines informative questions to ask. Instead,
we propose a mixed-initiative framework where both the user and system can be
active participants, depending on whose initiative will be more beneficial for
obtaining high-quality search results. We develop a reinforcement learning
approach which dynamically decides which of three interaction opportunities to
give to the user: drawing a sketch, providing free-form attribute feedback, or
answering attribute-based questions. By allowing these three options, our
system optimizes both the informativeness and exploration capabilities allowing
faster image retrieval. We outperform three baselines on three datasets and
extensive experimental settings.
Robust Deep Reinforcement Learning for Security and Safety in Autonomous Vehicle Systems
Authors: Aidin Ferdowsi, Ursula Challita, Walid Saad, Narayan B. Mandayam
Date: 2018-05-02 00:00:00
Tasks: Autonomous Vehicles, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), State Estimation

To operate effectively in tomorrow's smart cities, autonomous vehicles (AVs)
must rely on intra-vehicle sensors such as camera and radar as well as
inter-vehicle communication. Such dependence on sensors and communication links
exposes AVs to cyber-physical (CP) attacks by adversaries that seek to take
control of the AVs by manipulating their data. Thus, to ensure safe and optimal
AV dynamics control, the data processing functions at AVs must be robust to
such CP attacks. To this end, in this paper, the state estimation process for
monitoring AV dynamics, in presence of CP attacks, is analyzed and a novel
adversarial deep reinforcement learning (RL) algorithm is proposed to maximize
the robustness of AV dynamics control to CP attacks. The attacker's action and
the AV's reaction to CP attacks are studied in a game-theoretic framework. In
the formulated game, the attacker seeks to inject faulty data to AV sensor
readings so as to manipulate the inter-vehicle optimal safe spacing and
potentially increase the risk of AV accidents or reduce the vehicle flow on the
roads. Meanwhile, the AV, acting as a defender, seeks to minimize the
deviations of spacing so as to ensure robustness to the attacker's actions.
Since the AV has no information about the attacker's action and due to the
infinite possibilities for data value manipulations, the outcome of the
players' past interactions are fed to long-short term memory (LSTM) blocks.
Each player's LSTM block learns the expected spacing deviation resulting from
its own action and feeds it to its RL algorithm. Then, the the attacker's RL
algorithm chooses the action which maximizes the spacing deviation, while the
AV's RL algorithm tries to find the optimal action that minimizes such
deviation.
Tensor Decomposition for Compressing Recurrent Neural Network
Authors: Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Date: 2018-02-28 00:00:00
Tasks: Tensor Decomposition

In the machine learning fields, Recurrent Neural Network (RNN) has become a
popular architecture for sequential data modeling. However, behind the
impressive performance, RNNs require a large number of parameters for both
training and inference. In this paper, we are trying to reduce the number of
parameters and maintain the expressive power from RNN simultaneously. We
utilize several tensor decompositions method including CANDECOMP/PARAFAC (CP),
Tucker decomposition and Tensor Train (TT) to re-parameterize the Gated
Recurrent Unit (GRU) RNN. We evaluate all tensor-based RNNs performance on
sequence modeling tasks with a various number of parameters. Based on our
experiment results, TT-GRU achieved the best results in a various number of
parameters compared to other decomposition methods.
Bleaching Text: Abstract Features for Cross-lingual Gender Prediction
Authors: Rob van der Goot, Nikola Ljubešić, Ian Matroos, Malvina Nissim, Barbara Plank
Date: 2018-05-08 00:00:00
Proceeding: ACL 2018 7
Tasks: Gender Prediction, Prediction

Gender prediction has typically focused on lexical and social network
features, yielding good performance, but making systems highly language-,
topic-, and platform-dependent. Cross-lingual embeddings circumvent some of
these limitations, but capture gender-specific style less. We propose an
alternative: bleaching text, i.e., transforming lexical strings into more
abstract features. This study provides evidence that such features allow for
better transfer across languages. Moreover, we present a first study on the
ability of humans to perform cross-lingual gender prediction. We find that
human predictive power proves similar to that of our bleached models, and both
perform better than lexical models.
Local, algebraic simplifications of Gaussian random fields
Authors: Theodor Bjorkmo, M. C. David Marsh
Date: 2018-05-08 00:00:00

Many applications of Gaussian random fields and Gaussian random processes are
limited by the computational complexity of evaluating the probability density
function, which involves inverting the relevant covariance matrix. In this
work, we show how that problem can be completely circumvented for the local
Taylor coefficients of a Gaussian random field with a Gaussian (or `square
exponential') covariance function. Our results hold for any dimension of the
field and to any order in the Taylor expansion. We present two applications.
First, we show that this method can be used to explicitly generate non-trivial
potential energy landscapes with many fields. This application is particularly
useful when one is concerned with the field locally around special points
(e.g.~maxima or minima), as we exemplify by the problem of cosmic `manyfield'
inflation in the early universe. Second, we show that this method has
applications in machine learning, and greatly simplifies the regression problem
of determining the hyperparameters of the covariance function given a training
data set consisting of local Taylor coefficients at single point. An
accompanying Mathematica notebook is available at
https://doi.org/10.17863/CAM.22859 .
Learning Data-Driven Objectives to Optimize Interactive Systems
Authors: Ziming Li, Julia Kiseleva, Alekh Agarwal, Maarten de Rijke
Date: 2018-02-17 00:00:00

Effective optimization is essential for interactive systems to provide a satisfactory user experience. However, it is often challenging to find an objective to optimize for. Generally, such objectives are manually crafted and rarely capture complex user needs in an accurate manner. We propose an approach that infers the objective directly from observed user interactions. These inferences can be made regardless of prior knowledge and across different types of user behavior. We introduce interactive system optimization, a novel algorithm that uses these inferred objectives for optimization. Our main contribution is a new general principled approach to optimizing interactive systems using data-driven objectives. We demonstrate the high effectiveness of interactive system optimization over several simulations.
Learning on the Edge: Explicit Boundary Handling in CNNs
Authors: Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra
Date: 2018-05-08 00:00:00
Tasks: Colorization, Disparity Estimation, Optical Flow Estimation

Convolutional neural networks (CNNs) handle the case where filters extend
beyond the image boundary using several heuristics, such as zero, repeat or
mean padding. These schemes are applied in an ad-hoc fashion and, being weakly
related to the image content and oblivious of the target task, result in low
output quality at the boundary. In this paper, we propose a simple and
effective improvement that learns the boundary handling itself. At
training-time, the network is provided with a separate set of explicit boundary
filters. At testing-time, we use these filters which have learned to
extrapolate features at the boundary in an optimal way for the specific task.
Our extensive evaluation, over a wide range of architectural changes
(variations of layers, feature channels, or both), shows how the explicit
filters result in improved boundary handling. Consequently, we demonstrate an
improvement of 5% to 20% across the board of typical CNN applications
(colorization, de-Bayering, optical flow, and disparity estimation).
Towards Learning Sparsely Used Dictionaries with Arbitrary Supports
Authors: Pranjal Awasthi, Aravindan Vijayaraghavan
Date: 2018-04-23 00:00:00
Tasks: Dictionary Learning

Dictionary learning is a popular approach for inferring a hidden basis or
dictionary in which data has a sparse representation. Data generated from the
dictionary A (an n by m matrix, with m > n in the over-complete setting) is
given by Y = AX where X is a matrix whose columns have supports chosen from a
distribution over k-sparse vectors, and the non-zero values chosen from a
symmetric distribution. Given Y, the goal is to recover A and X in polynomial
time. Existing algorithms give polytime guarantees for recovering incoherent
dictionaries, under strong distributional assumptions both on the supports of
the columns of X, and on the values of the non-zero entries. In this work, we
study the following question: Can we design efficient algorithms for recovering
dictionaries when the supports of the columns of X are arbitrary?
  To address this question while circumventing the issue of
non-identifiability, we study a natural semirandom model for dictionary
learning where there are a large number of samples $y=Ax$ with arbitrary
k-sparse supports for x, along with a few samples where the sparse supports are
chosen uniformly at random. While the few samples with random supports ensures
identifiability, the support distribution can look almost arbitrary in
aggregate. Hence existing algorithmic techniques seem to break down as they
make strong assumptions on the supports.
  Our main contribution is a new polynomial time algorithm for learning
incoherent over-complete dictionaries that works under the semirandom model.
Additionally the same algorithm provides polynomial time guarantees in new
parameter regimes when the supports are fully random. Finally using these
techniques, we also identify a minimal set of conditions on the supports under
which the dictionary can be (information theoretically) recovered from
polynomial samples for almost linear sparsity, i.e., $k=\tilde{O}(n)$.
Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than Character Level
Authors: Lifeng Han, Shaohui Kuang
Date: 2018-05-03 00:00:00
Tasks: Machine Translation, NMT, Translation

In neural machine translation (NMT), researchers face the challenge of un-seen (or out-of-vocabulary OOV) words translation. To solve this, some researchers propose the splitting of western languages such as English and German into sub-words or compounds. In this paper, we try to address this OOV issue and improve the NMT adequacy with a harder language Chinese whose characters are even more sophisticated in composition. We integrate the Chinese radicals into the NMT model with different settings to address the unseen words challenge in Chinese to English translation. On the other hand, this also can be considered as semantic part of the MT system since the Chinese radicals usually carry the essential meaning of the words they are constructed in. Meaningful radicals and new characters can be integrated into the NMT systems with our models. We use an attention-based NMT system as a strong baseline system. The experiments on standard Chinese-to-English NIST translation shared task data 2006 and 2008 show that our designed models outperform the baseline model in a wide range of state-of-the-art evaluation metrics including LEPOR, BEER, and CharacTER, in addition to BLEU and NIST scores, especially on the adequacy-level translation. We also have some interesting findings from the results of our various experiment settings about the performance of words and characters in Chinese NMT, which is different with other languages. For instance, the fully character level NMT may perform well or the state of the art in some other languages as researchers demonstrated recently, however, in the Chinese NMT model, word boundary knowledge is important for the model learning.
Deep Learning for Genomics: A Concise Overview
Authors: Tianwei Yue, Yuanxin Wang, Longxiang Zhang, Chunming Gu, Haoru Xue, Wenping Wang, Qi Lyu, Yujie Dun
Date: 2018-02-02 00:00:00
Tasks: Deep Learning

Advancements in genomic research such as high-throughput sequencing techniques have driven modern genomic studies into "big data" disciplines. This data explosion is constantly challenging conventional methods used in genomics. In parallel with the urgent demand for robust algorithms, deep learning has succeeded in a variety of fields such as vision, speech, and text processing. Yet genomics entails unique challenges to deep learning since we are expecting from deep learning a superhuman intelligence that explores beyond our knowledge to interpret the genome. A powerful deep learning model should rely on insightful utilization of task-specific knowledge. In this paper, we briefly discuss the strengths of different deep learning models from a genomic perspective so as to fit each particular task with a proper deep architecture, and remark on practical considerations of developing modern deep learning architectures for genomics. We also provide a concise review of deep learning applications in various aspects of genomic research, as well as pointing out potential opportunities and obstacles for future genomics applications.
Fast Feature Extraction with CNNs with Pooling Layers
Authors: Christian Bailer, Tewodros Habtegebrial, Kiran varanasi, Didier Stricker
Date: 2018-05-08 00:00:00
Tasks: Camera Calibration, object-detection, Object Detection, Optical Flow Estimation, Stereo Matching

In recent years, many publications showed that convolutional neural network
based features can have a superior performance to engineered features. However,
not much effort was taken so far to extract local features efficiently for a
whole image. In this paper, we present an approach to compute patch-based local
feature descriptors efficiently in presence of pooling and striding layers for
whole images at once. Our approach is generic and can be applied to nearly all
existing network architectures. This includes networks for all local feature
extraction tasks like camera calibration, Patchmatching, optical flow
estimation and stereo matching. In addition, our approach can be applied to
other patch-based approaches like sliding window object detection and
recognition. We complete our paper with a speed benchmark of popular CNN based
feature extraction approaches applied on a whole image, with and without our
speedup, and example code (for Torch) that shows how an arbitrary CNN
architecture can be easily converted by our approach.