pwc-archive/papers-with-abstracts huggingface dataset:
On The Alignment Problem In Multi-Head Attention-Based Neural Machine Translation
Authors: Tamer Alkhouli, Gabriel Bretschner, Hermann Ney
Date: 2018-09-11 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Translation

This work investigates the alignment problem in state-of-the-art multi-head
attention models based on the transformer architecture. We demonstrate that
alignment extraction in transformer models can be improved by augmenting an
additional alignment head to the multi-head source-to-target attention
component. This is used to compute sharper attention weights. We describe how
to use the alignment head to achieve competitive performance. To study the
effect of adding the alignment head, we simulate a dictionary-guided
translation task, where the user wants to guide translation using pre-defined
dictionary entries. Using the proposed approach, we achieve up to $3.8$ % BLEU
improvement when using the dictionary, in comparison to $2.4$ % BLEU in the
baseline case. We also propose alignment pruning to speed up decoding in
alignment-based neural machine translation (ANMT), which speeds up translation
by a factor of $1.8$ without loss in translation performance. We carry out
experiments on the shared WMT 2016 English$\to$Romanian news task and the BOLT
Chinese$\to$English discussion forum task.
Endowing Robots with Longer-term Autonomy by Recovering from External Disturbances in Manipulation through Grounded Anomaly Classification and Recovery Policies
Authors: Hongmin Wu, Shuangqi Luo, Longxin Chen, Shuangda Duan, Sakmongkon Chumkamon, Dong Liu, Yisheng Guan, Juan Rojas
Date: 2018-09-11 00:00:00
Tasks: Anomaly Classification, General Classification, Motion Generation, Robot Manipulation, Variational Inference

Robot manipulation is increasingly poised to interact with humans in
co-shared workspaces. Despite increasingly robust manipulation and control
algorithms, failure modes continue to exist whenever models do not capture the
dynamics of the unstructured environment. To obtain longer-term horizons in
robot automation, robots must develop introspection and recovery abilities. We
contribute a set of recovery policies to deal with anomalies produced by
external disturbances as well as anomaly classification through the use of
non-parametric statistics with memoized variational inference with scalable
adaptation. A recovery critic stands atop of a tightly-integrated, graph-based
online motion-generation and introspection system that resolves a wide range of
anomalous situations. Policies, skills, and introspection models are learned
incrementally and contextually in a task. Two task-level recovery policies:
re-enactment and adaptation resolve accidental and persistent anomalies
respectively. The introspection system uses non-parametric priors along with
Markov jump linear systems and memoized variational inference with scalable
adaptation to learn a model from the data. Extensive real-robot experimentation
with various strenuous anomalous conditions is induced and resolved at
different phases of a task and in different combinations. The system executes
around-the-clock introspection and recovery and even elicited self-recovery
when misclassifications occurred.
Deep Inferential Spatial-Temporal Network for Forecasting Air Pollution Concentrations
Authors: Hao Wang, Bojin Zhuang, Yang Chen, Ni Li, Dongxia Wei
Date: 2018-09-11 00:00:00

Air pollution poses a serious threat to human health as well as economic
development around the world. To meet the increasing demand for accurate
predictions for air pollutions, we proposed a Deep Inferential Spatial-Temporal
Network to deal with the complicated non-linear spatial and temporal
correlations. We forecast three air pollutants (i.e., PM2.5, PM10 and O3) of
monitoring stations over the next 48 hours, using a hybrid deep learning model
consists of inferential predictor (inference for regions without air pollution
readings), spatial predictor (capturing spatial correlations using CNN) and
temporal predictor (capturing temporal relationship using sequence-to-sequence
model with simplified attention mechanism). Our proposed model considers
historical air pollution records and historical meteorological data. We
evaluate our model on a large-scale dataset containing air pollution records of
35 monitoring stations and grid meteorological data in Beijing, China. Our
model outperforms other state-of-art methods in terms of SMAPE and RMSE.
Abstraction Learning
Authors: Fei Deng, Jinsheng Ren, Feng Chen
Date: 2018-09-11 00:00:00
Tasks: Lifelong learning

There has been a gap between artificial intelligence and human intelligence.
In this paper, we identify three key elements forming human intelligence, and
suggest that abstraction learning combines these elements and is thus a way to
bridge the gap. Prior researches in artificial intelligence either specify
abstraction by human experts, or take abstraction as a qualitative explanation
for the model. This paper aims to learn abstraction directly. We tackle three
main challenges: representation, objective function, and learning algorithm.
Specifically, we propose a partition structure that contains pre-allocated
abstraction neurons; we formulate abstraction learning as a constrained
optimization problem, which integrates abstraction properties; we develop a
network evolution algorithm to solve this problem. This complete framework is
named ONE (Optimization via Network Evolution). In our experiments on MNIST,
ONE shows elementary human-like intelligence, including low energy consumption,
knowledge sharing, and lifelong learning.
Hubless keypoint-based 3D deformable groupwise registration
Authors: Rémi Agier, Sébastien Valette, Razmig Kéchichian, Laurent Fanton, Rémy Prost
Date: 2018-09-11 00:00:00
Tasks: 3D Reconstruction, Anatomy, global-optimization, Semantic Segmentation

We present a novel algorithm for Fast Registration Of image Groups (FROG), applied to large 3D image groups. Our approach extracts 3D SURF keypoints from images, computes matched pairs of keypoints and registers the group by minimizing pair distances in a hubless way i.e. without computing any central mean image. Using keypoints significantly reduces the problem complexity compared to voxel-based approaches, and enables us to provide an in-core global optimization, similar to the Bundle Adjustment for 3D reconstruction. As we aim to register images of different patients, the matching step yields many outliers. Then we propose a new EM-weighting algorithm which efficiently discards outliers. Global optimization is carried out with a fast gradient descent algorithm. This allows our approach to robustly register large datasets. The result is a set of diffeomorphic half transforms which link the volumes together and can be subsequently exploited for computational anatomy and landmark detection. We show experimental results on whole-body CT scans, with groups of up to 103 volumes. On a benchmark based on anatomical landmarks, our algorithm compares favorably with the star-groupwise voxel-based ANTs and NiftyReg approaches while being much faster. We also discuss the limitations of our approach for lower resolution images such as brain MRI.
Multilingual Cross-domain Perspectives on Online Hate Speech
Authors: Tom De Smedt, Sylvia Jaki, Eduan Kotzé, Leïla Saoud, Maja Gwóźdź, Guy De Pauw, Walter Daelemans
Date: 2018-09-11 00:00:00
Tasks: General Classification, text-classification, Text Classification

In this report, we present a study of eight corpora of online hate speech, by
demonstrating the NLP techniques that we used to collect and analyze the
jihadist, extremist, racist, and sexist content. Analysis of the multilingual
corpora shows that the different contexts share certain characteristics in
their hateful rhetoric. To expose the main features, we have focused on text
classification, text profiling, keyword and collocation extraction, along with
manual annotation and qualitative study.
SAI, a Sensible Artificial Intelligence that plays Go
Authors: Francesco Morandin, Gianluca Amato, Rosa Gini, Carlo Metta, Maurizio Parton, Gian-Carlo Pascutto
Date: 2018-09-11 00:00:00
Tasks: Position, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a multiple-komi modification of the AlphaGo Zero/Leela Zero
paradigm. The winrate as a function of the komi is modeled with a
two-parameters sigmoid function, so that the neural network must predict just
one more variable to assess the winrate for all komi values. A second novel
feature is that training is based on self-play games that occasionally branch
-- with changed komi -- when the position is uneven. With this setting,
reinforcement learning is showed to work on 7x7 Go, obtaining very strong
playing agents. As a useful byproduct, the sigmoid parameters given by the
network allow to estimate the score difference on the board, and to evaluate
how much the game is decided.
A Detection and Segmentation Architecture for Skin Lesion Segmentation on Dermoscopy Images
Authors: Chengyao Qian, Ting Liu, Hao Jiang, Zhe Wang, Pengfei Wang, Mingxin Guan, Biao Sun
Date: 2018-09-11 00:00:00
Tasks: Lesion Segmentation, Segmentation, Skin Lesion Segmentation

This report summarises our method and validation results for the ISIC
Challenge 2018 - Skin Lesion Analysis Towards Melanoma Detection - Task 1:
Lesion Segmentation. We present a two-stage method for lesion segmentation with
optimised training method and ensemble post-process. Our method achieves
state-of-the-art performance on lesion segmentation and we win the first place
in ISIC 2018 task1.
Detecting Intentions of Vulnerable Road Users Based on Collective Intelligence
Authors: Maarten Bieshaar, Günther Reitberger, Stefan Zernetsch, Bernhard Sick, Erich Fuchs, Konrad Doll
Date: 2018-09-11 00:00:00
Tasks: Activity Recognition, Intent Detection

Vulnerable road users (VRUs, i.e. cyclists and pedestrians) will play an
important role in future traffic. To avoid accidents and achieve a highly
efficient traffic flow, it is important to detect VRUs and to predict their
intentions. In this article a holistic approach for detecting intentions of
VRUs by cooperative methods is presented. The intention detection consists of
basic movement primitive prediction, e.g. standing, moving, turning, and a
forecast of the future trajectory. Vehicles equipped with sensors, data
processing systems and communication abilities, referred to as intelligent
vehicles, acquire and maintain a local model of their surrounding traffic
environment, e.g. crossing cyclists. Heterogeneous, open sets of agents
(cooperating and interacting vehicles, infrastructure, e.g. cameras and laser
scanners, and VRUs equipped with smart devices and body-worn sensors) exchange
information forming a multi-modal sensor system with the goal to reliably and
robustly detect VRUs and their intentions under consideration of real time
requirements and uncertainties. The resulting model allows to extend the
perceptual horizon of the individual agent beyond their own sensory
capabilities, enabling a longer forecast horizon. Concealments,
implausibilities and inconsistencies are resolved by the collective
intelligence of cooperating agents. Novel techniques of signal processing and
modelling in combination with analytical and learning based approaches of
pattern and activity recognition are used for detection, as well as intention
prediction of VRUs. Cooperation, by means of probabilistic sensor and knowledge
fusion, takes place on the level of perception and intention recognition. Based
on the requirements of the cooperative approach for the communication a new
strategy for an ad hoc network is proposed.
Studying the History of the Arabic Language: Language Technology and a Large-Scale Historical Corpus
Authors: Yonatan Belinkov, Alexander Magidow, Alberto Barrón-Cedeño, Avi Shmidman, Maxim Romanov
Date: 2018-09-11 00:00:00

Arabic is a widely-spoken language with a long and rich history, but existing
corpora and language technology focus mostly on modern Arabic and its
varieties. Therefore, studying the history of the language has so far been
mostly limited to manual analyses on a small scale. In this work, we present a
large-scale historical corpus of the written Arabic language, spanning 1400
years. We describe our efforts to clean and process this corpus using Arabic
NLP tools, including the identification of reused text. We study the history of
the Arabic language using a novel automatic periodization algorithm, as well as
other techniques. Our findings confirm the established division of written
Arabic into Modern Standard and Classical Arabic, and confirm other established
periodizations, while suggesting that written Arabic may be divisible into
still further periods of development.
Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks
Authors: Ramin M. Hasani, Alexander Amini, Mathias Lechner, Felix Naser, Radu Grosu, Daniela Rus
Date: 2018-09-11 00:00:00

In this paper, we introduce a novel method to interpret recurrent neural
networks (RNNs), particularly long short-term memory networks (LSTMs) at the
cellular level. We propose a systematic pipeline for interpreting individual
hidden state dynamics within the network using response characterization
methods. The ranked contribution of individual cells to the network's output is
computed by analyzing a set of interpretable metrics of their decoupled step
and sinusoidal responses. As a result, our method is able to uniquely identify
neurons with insightful dynamics, quantify relationships between dynamical
properties and test accuracy through ablation analysis, and interpret the
impact of network capacity on a network's dynamical distribution. Finally, we
demonstrate generalizability and scalability of our method by evaluating a
series of different benchmark sequential datasets.
Visualizing Convolutional Neural Networks to Improve Decision Support for Skin Lesion Classification
Authors: Pieter Van Molle, Miguel De Strooper, Tim Verbelen, Bert Vankeirsbilck, Pieter Simoens, Bart Dhoedt
Date: 2018-09-11 00:00:00
Tasks: General Classification, Lesion Classification, Skin Lesion Classification

Because of their state-of-the-art performance in computer vision, CNNs are
becoming increasingly popular in a variety of fields, including medicine.
However, as neural networks are black box function approximators, it is
difficult, if not impossible, for a medical expert to reason about their
output. This could potentially result in the expert distrusting the network
when he or she does not agree with its output. In such a case, explaining why
the CNN makes a certain decision becomes valuable information. In this paper,
we try to open the black box of the CNN by inspecting and visualizing the
learned feature maps, in the field of dermatology. We show that, to some
extent, CNNs focus on features similar to those used by dermatologists to make
a diagnosis. However, more research is required for fully explaining their
output.
Unsupervised Domain Adaptation Based on Source-guided Discrepancy
Authors: Seiichi Kuroki, Nontawat Charoenphakdee, Han Bao, Junya Honda, Issei Sato, Masashi Sugiyama
Date: 2018-09-11 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation is the problem setting where data generating
distributions in the source and target domains are different, and labels in the
target domain are unavailable. One important question in unsupervised domain
adaptation is how to measure the difference between the source and target
domains. A previously proposed discrepancy that does not use the source domain
labels requires high computational cost to estimate and may lead to a loose
generalization error bound in the target domain. To mitigate these problems, we
propose a novel discrepancy called source-guided discrepancy (S-disc), which
exploits labels in the source domain. As a consequence, S-disc can be computed
efficiently with a finite sample convergence guarantee. In addition, we show
that S-disc can provide a tighter generalization error bound than the one based
on an existing discrepancy. Finally, we report experimental results that
demonstrate the advantages of S-disc over the existing discrepancies.
Learning Rate Adaptation for Federated and Differentially Private Learning
Authors: Antti Koskela, Antti Honkela
Date: 2018-09-11 00:00:00
Tasks: Federated Learning

We propose an algorithm for the adaptation of the learning rate for stochastic gradient descent (SGD) that avoids the need for validation set use. The idea for the adaptiveness comes from the technique of extrapolation: to get an estimate for the error against the gradient flow which underlies SGD, we compare the result obtained by one full step and two half-steps. The algorithm is applied in two separate frameworks: federated and differentially private learning. Using examples of deep neural networks we empirically show that the adaptive algorithm is competitive with manually tuned commonly used optimisation methods for differentially privately training. We also show that it works robustly in the case of federated learning unlike commonly used optimisation methods.
Predicting Blood Glucose with an LSTM and Bi-LSTM Based Deep Neural Network
Authors: Qingnan Sun, Marko V. Jankovic, Lia Bally, Stavroula G. Mougiakakou
Date: 2018-09-11 00:00:00

A deep learning network was used to predict future blood glucose levels, as
this can permit diabetes patients to take action before imminent hyperglycaemia
and hypoglycaemia. A sequential model with one long-short-term memory (LSTM)
layer, one bidirectional LSTM layer and several fully connected layers was used
to predict blood glucose levels for different prediction horizons. The method
was trained and tested on 26 datasets from 20 real patients. The proposed
network outperforms the baseline methods in terms of all evaluation criteria.
Convolutional Neural Networks for the segmentation of microcalcification in Mammography Imaging
Authors: Gabriele Valvano, Gianmarco Santini, Nicola Martini, Andrea Ripoli, Chiara Iacconi, Dante Chiappino, Daniele Della Latta
Date: 2018-09-11 00:00:00
Tasks: Segmentation

Cluster of microcalcifications can be an early sign of breast cancer. In this
paper we propose a novel approach based on convolutional neural networks for
the detection and segmentation of microcalcification clusters. In this work we
used 283 mammograms to train and validate our model, obtaining an accuracy of
98.22% in the detection of preliminary suspect regions and of 97.47% in the
segmentation task. Our results show how deep learning could be an effective
tool to effectively support radiologists during mammograms examination.
Normalization in Training U-Net for 2D Biomedical Semantic Segmentation
Authors: Xiao-Yun Zhou, Guang-Zhong Yang
Date: 2018-09-11 00:00:00
Tasks: 2D Semantic Segmentation, Segmentation, Semantic Segmentation

2D biomedical semantic segmentation is important for robotic vision in
surgery. Segmentation methods based on Deep Convolutional Neural Network (DCNN)
can out-perform conventional methods in terms of both accuracy and levels of
automation. One common issue in training a DCNN for biomedical semantic
segmentation is the internal covariate shift where the training of
convolutional kernels is encumbered by the distribution change of input
features, hence both the training speed and performance are decreased. Batch
Normalization (BN) is the first proposed method for addressing internal
covariate shift and is widely used. Instance Normalization (IN) and Layer
Normalization (LN) have also been proposed. Group Normalization (GN) is
proposed more recently and has not yet been applied to 2D biomedical semantic
segmentation, however, no specific validations on GN were given. Most DCNNs for
biomedical semantic segmentation adopt BN as the normalization method by
default, without reviewing its performance. In this paper, four normalization
methods - BN, IN, LN and GN are compared in details, specifically for 2D
biomedical semantic segmentation. U-Net is adopted as the basic DCNN structure.
Three datasets regarding the Right Ventricle (RV), aorta, and Left Ventricle
(LV) are used for the validation. The results show that detailed subdivision of
the feature map, i.e. GN with a large group number or IN, achieves higher
accuracy. This accuracy improvement mainly comes from better model
generalization. Codes are uploaded and maintained at Xiao-Yun Zhou's Github.
Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks
Authors: Marcel Schreiber, Stefan Hoermann, Klaus Dietmayer
Date: 2018-09-11 00:00:00
Tasks: Prediction

We tackle the long-term prediction of scene evolution in a complex downtown scenario for automated driving based on Lidar grid fusion and recurrent neural networks (RNNs). A bird's eye view of the scene, including occupancy and velocity, is fed as a sequence to a RNN which is trained to predict future occupancy. The nature of prediction allows generation of multiple hours of training data without the need of manual labeling. Thus, the training strategy and loss function is designed for long sequences of real-world data (unbalanced, continuously changing situations, false labels, etc.). The deep CNN architecture comprises convolutional long short-term memories (ConvLSTMs) to separate static from dynamic regions and to predict dynamic objects in future frames. Novel recurrent skip connections show the ability to predict small occluded objects, i.e. pedestrians, and occluded static regions. Spatio-temporal correlations between grid cells are exploited to predict multimodal future paths and interactions between objects. Experiments also quantify improvements to our previous network, a Monte Carlo approach, and literature.
Probabilistic approach to limited-data computed tomography reconstruction
Authors: Zenith Purisha, Carl Jidling, Niklas Wahlström, Simo Särkkä, Thomas B. Schön
Date: 2018-09-11 00:00:00
Tasks: Numerical Integration

In this work, we consider the inverse problem of reconstructing the internal structure of an object from limited x-ray projections. We use a Gaussian process prior to model the target function and estimate its (hyper)parameters from measured data. In contrast to other established methods, this comes with the advantage of not requiring any manual parameter tuning, which usually arises in classical regularization strategies. Our method uses a basis function expansion technique for the Gaussian process which significantly reduces the computational complexity and avoids the need for numerical integration. The approach also allows for reformulation of come classical regularization methods as Laplacian and Tikhonov regularization as Gaussian process regression, and hence provides an efficient algorithm and principled means for their parameter tuning. Results from simulated and real data indicate that this approach is less sensitive to streak artifacts as compared to the commonly used method of filtered backprojection.
Solving Non-identifiable Latent Feature Models
Authors: Ryota Suzuki, Shingo Takahashi, Murtuza Petladwala, Shigeru Kohmoto
Date: 2018-09-11 00:00:00
Tasks: parameter estimation

Latent feature models (LFM)s are widely employed for extracting latent
structures of data. While offering high, parameter estimation is difficult with
LFMs because of the combinational nature of latent features, and
non-identifiability is a particularly difficult problem when parameter
estimation is not unique and there exists equivalent solutions. In this paper,
a necessary and sufficient condition for non-identifiability is shown. The
condition is significantly related to dependency of features, and this implies
that non-identifiability may often occur in real-world applications. A novel
method for parameter estimation that solves the non-identifiability problem is
also proposed. This method can be combined as a post-process with existing
methods and can find an appropriate solution by hopping efficiently through
equivalent solutions. We have evaluated the effectiveness of the method on both
synthetic and real-world datasets.