pwc-archive/papers-with-abstracts huggingface dataset:
Capacity Control of ReLU Neural Networks by Basis-path Norm
Authors: Shuxin Zheng, Qi Meng, Huishuai Zhang, Wei Chen, Nenghai Yu, Tie-Yan Liu
Date: 2018-09-19 00:00:00

Recently, path norm was proposed as a new capacity measure for neural
networks with Rectified Linear Unit (ReLU) activation function, which takes the
rescaling-invariant property of ReLU into account. It has been shown that the
generalization error bound in terms of the path norm explains the empirical
generalization behaviors of the ReLU neural networks better than that of other
capacity measures. Moreover, optimization algorithms which take path norm as
the regularization term to the loss function, like Path-SGD, have been shown to
achieve better generalization performance. However, the path norm counts the
values of all paths, and hence the capacity measure based on path norm could be
improperly influenced by the dependency among different paths. It is also known
that each path of a ReLU network can be represented by a small group of
linearly independent basis paths with multiplication and division operation,
which indicates that the generalization behavior of the network only depends on
only a few basis paths. Motivated by this, we propose a new norm
\emph{Basis-path Norm} based on a group of linearly independent paths to
measure the capacity of neural networks more accurately. We establish a
generalization error bound based on this basis path norm, and show it explains
the generalization behaviors of ReLU networks more accurately than previous
capacity measures via extensive experiments. In addition, we develop
optimization algorithms which minimize the empirical risk regularized by the
basis-path norm. Our experiments on benchmark datasets demonstrate that the
proposed regularization method achieves clearly better performance on the test
set than the previous regularization approaches.
InfoSSM: Interpretable Unsupervised Learning of Nonparametric State-Space Model for Multi-modal Dynamics
Authors: Young-Jin Park, Han-Lim Choi
Date: 2018-09-19 00:00:00
Tasks: Time Series, Time Series Analysis

The goal of system identification is to learn about underlying physics
dynamics behind the time-series data. To model the probabilistic and
nonparametric dynamics model, Gaussian process (GP) have been widely used; GP
can estimate the uncertainty of prediction and avoid over-fitting. Traditional
GPSSMs, however, are based on Gaussian transition model, thus often have
difficulty in describing a more complex transition model, e.g. aircraft
motions. To resolve the challenge, this paper proposes a framework using
multiple GP transition models which is capable of describing multi-modal
dynamics. Furthermore, we extend the model to the information-theoretic
framework, the so-called InfoSSM, by introducing a mutual information
regularizer helping the model to learn interpretable and distinguishable
multiple dynamics models. Two illustrative numerical experiments in simple
Dubins vehicle and high-fidelity flight simulator are presented to demonstrate
the performance and interpretability of the proposed model. Finally, this paper
introduces a framework using InfoSSM with Bayesian filtering for air traffic
control tracking.
A unifying Bayesian approach for preterm brain-age prediction that models EEG sleep transitions over age
Authors: Kirubin Pillay, Maarten De Vos
Date: 2018-09-19 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Sleep Staging

Preterm newborns undergo various stresses that may materialize as learning
problems at school-age. Sleep staging of the Electroencephalogram (EEG),
followed by prediction of their brain-age from these sleep states can quantify
deviations from normal brain development early (when compared to the known
age). Current automation of this approach relies on explicit sleep state
classification, optimizing algorithms using clinician visually labelled sleep
stages, which remains a subjective gold-standard. Such models fail to perform
consistently over a wide age range and impacts the subsequent brain-age
estimates that could prevent identification of subtler developmental
deviations. We introduce a Bayesian Network utilizing multiple Gaussian Mixture
Models, as a novel, unified approach for directly estimating brain-age,
simultaneously modelling for both age and sleep dependencies on the EEG, to
improve the accuracy of prediction over a wider age range.
Dual Reconstruction Nets for Image Super-Resolution with Gradient Sensitive Loss
Authors: Yong Guo, Qi Chen, Jian Chen, Junzhou Huang, Yanwu Xu, JieZhang Cao, Peilin Zhao, Mingkui Tan
Date: 2018-09-19 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

Deep neural networks have exhibited promising performance in image
super-resolution (SR) due to the power in learning the non-linear mapping from
low-resolution (LR) images to high-resolution (HR) images. However, most deep
learning methods employ feed-forward architectures, and thus the dependencies
between LR and HR images are not fully exploited, leading to limited learning
performance. Moreover, most deep learning based SR methods apply the pixel-wise
reconstruction error as the loss, which, however, may fail to capture
high-frequency information and produce perceptually unsatisfying results,
whilst the recent perceptual loss relies on some pre-trained deep model and
they may not generalize well. In this paper, we introduce a mask to separate
the image into low- and high-frequency parts based on image gradient magnitude,
and then devise a gradient sensitive loss to well capture the structures in the
image without sacrificing the recovery of low-frequency content. Moreover, by
investigating the duality in SR, we develop a dual reconstruction network (DRN)
to improve the SR performance. We provide theoretical analysis on the
generalization performance of our method and demonstrate its effectiveness and
superiority with thorough experiments.
Novelty-organizing team of classifiers in noisy and dynamic environments
Authors: Danilo Vasconcellos Vargas, Hirotaka Takano, Junichi Murata
Date: 2018-09-19 00:00:00

In the real world, the environment is constantly changing with the input
variables under the effect of noise. However, few algorithms were shown to be
able to work under those circumstances. Here, Novelty-Organizing Team of
Classifiers (NOTC) is applied to the continuous action mountain car as well as
two variations of it: a noisy mountain car and an unstable weather mountain
car. These problems take respectively noise and change of problem dynamics into
account. Moreover, NOTC is compared with NeuroEvolution of Augmenting
Topologies (NEAT) in these problems, revealing a trade-off between the
approaches. While NOTC achieves the best performance in all of the problems,
NEAT needs less trials to converge. It is demonstrated that NOTC achieves
better performance because of its division of the input space (creating easier
problems). Unfortunately, this division of input space also requires a bit of
time to bootstrap.
Counting the uncountable: deep semantic density estimation from Space
Authors: Andres C. Rodriguez, Jan D. Wegner
Date: 2018-09-19 00:00:00
Tasks: Density Estimation, Object, Object Counting, object-detection, Object Detection, Semantic Segmentation

We propose a new method to count objects of specific categories that are
significantly smaller than the ground sampling distance of a satellite image.
This task is hard due to the cluttered nature of scenes where different object
categories occur. Target objects can be partially occluded, vary in appearance
within the same class and look alike to different categories. Since traditional
object detection is infeasible due to the small size of objects with respect to
the pixel size, we cast object counting as a density estimation problem. To
distinguish objects of different classes, our approach combines density
estimation with semantic segmentation in an end-to-end learnable convolutional
neural network (CNN). Experiments show that deep semantic density estimation
can robustly count objects of various classes in cluttered scenes. Experiments
also suggest that we need specific CNN architectures in remote sensing instead
of blindly applying existing ones from computer vision.
Deep Learning Based Rib Centerline Extraction and Labeling
Authors: Matthias Lenga, Tobias Klinder, Christian Bürger, Jens von Berg, Astrid Franz, Cristian Lorenz
Date: 2018-09-19 00:00:00
Tasks: Deep Learning

Automated extraction and labeling of rib centerlines is a typically needed
prerequisite for more advanced assisted reading tools that help the radiologist
to efficiently inspect all 24 ribs in a CT volume. In this paper, we combine a
deep learning-based rib detection with a dedicated centerline extraction
algorithm applied to the detection result for the purpose of fast, robust and
accurate rib centerline extraction and labeling from CT volumes. More
specifically, we first apply a fully convolutional neural network (FCNN) to
generate a probability map for detecting the first rib pair, the twelfth rib
pair, and the collection of all intermediate ribs. In a second stage, a newly
designed centerline extraction algorithm is applied to this multi-label
probability map. Finally, the distinct detection of first and twelfth rib
separately, allows to derive individual rib labels by simple sorting and
counting the detected centerlines. We applied our method to CT volumes from 116
patients which included a variety of different challenges and achieved a
centerline accuracy of 0.787 mm with respect to manual centerline annotations.
  This article is a preprint version of: Lenga M., Klinder T., B\"urger C., von
Berg J., Franz A., Lorenz C. (2019) Deep Learning Based Rib Centerline
Extraction and Labeling. In: Vrtovec T., Yao J., Zheng G., Pozo J. (eds)
Computational Methods and Clinical Applications in Musculoskeletal Imaging.
MSKI 2018. Lecture Notes in Computer Science, vol 11404. Springer, Cham
The Aqualoc Dataset: Towards Real-Time Underwater Localization from a Visual-Inertial-Pressure Acquisition System
Authors: Maxime Ferrera, Julien Moras, Pauline Trouvé-Peloux, Vincent Creuze, Denis Dégez
Date: 2018-09-19 00:00:00
Tasks: Visual Odometry

This paper presents a new underwater dataset acquired from a
visual-inertial-pressure acquisition system and meant to be used to benchmark
visual odometry, visual SLAM and multi-sensors SLAM solutions. The dataset is
publicly available and contains ground-truth trajectories for evaluation.
Detect, anticipate and generate: Semi-supervised recurrent latent variable models for human activity modeling
Authors: Judith Bütepage, Danica Kragic
Date: 2018-09-19 00:00:00
Tasks: Affordance Detection

Successful Human-Robot collaboration requires a predictive model of human
behavior. The robot needs to be able to recognize current goals and actions and
to predict future activities in a given context. However, the spatio-temporal
sequence of human actions is difficult to model since latent factors such as
intention, task, knowledge, intuition and preference determine the action
choices of each individual. In this work we introduce semi-supervised
variational recurrent neural networks which are able to a) model temporal
distributions over latent factors and the observable feature space, b)
incorporate discrete labels such as activity type when available, and c)
generate possible future action sequences on both feature and label level. We
evaluate our model on the Cornell Activity Dataset CAD-120 dataset. Our model
outperforms state-of-the-art approaches in both activity and affordance
detection and anticipation. Additionally, we show how samples of possible
future action sequences are in line with past observations.
Latent Topic Conversational Models
Authors: Tsung-Hsien Wen, Minh-Thang Luong
Date: 2018-09-19 00:00:00
Proceeding: ICLR 2018 1
Tasks: Response Generation, Sentence

Latent variable models have been a preferred choice in conversational
modeling compared to sequence-to-sequence (seq2seq) models which tend to
generate generic and repetitive responses. Despite so, training latent variable
models remains to be difficult. In this paper, we propose Latent Topic
Conversational Model (LTCM) which augments seq2seq with a neural latent topic
component to better guide response generation and make training easier. The
neural topic component encodes information from the source sentence to build a
global "topic" distribution over words, which is then consulted by the seq2seq
model at each generation step. We study in details how the latent
representation is learnt in both the vanilla model and LTCM. Our extensive
experiments contribute to better understanding and training of conditional
latent models for languages. Our results show that by sampling from the learnt
latent representations, LTCM can generate diverse and interesting responses. In
a subjective human evaluation, the judges also confirm that LTCM is the overall
preferred option.
Faster Training of Mask R-CNN by Focusing on Instance Boundaries
Authors: Roland S. Zimmermann, Julien N. Siems
Date: 2018-09-19 00:00:00
Tasks: Edge Detection, Instance Segmentation, Semantic Segmentation

We present an auxiliary task to Mask R-CNN, an instance segmentation network, which leads to faster training of the mask head. Our addition to Mask R-CNN is a new prediction head, the Edge Agreement Head, which is inspired by the way human annotators perform instance segmentation. Human annotators copy the contour of an object instance and only indirectly the occupied instance area. Hence, the edges of instance masks are particularly useful as they characterize the instance well. The Edge Agreement Head therefore encourages predicted masks to have similar image gradients to the ground-truth mask using edge detection filters. We provide a detailed survey of loss combinations and show improvements on the MS COCO Mask metrics compared to using no additional loss. Our approach marginally increases the model size and adds no additional trainable model variables. While the computational costs are increased slightly, the increment is negligible considering the high computational cost of the Mask R-CNN architecture. As the additional network head is only relevant during training, inference speed remains unchanged compared to Mask R-CNN. In a default Mask R-CNN setup, we achieve a training speed-up and a relative overall improvement of 8.1% on the MS COCO metrics compared to the baseline.
Prosocial or Selfish? Agents with different behaviors for Contract Negotiation using Reinforcement Learning
Authors: Vishal Sunder, Lovekesh Vig, Arnab Chatterjee, Gautam Shroff
Date: 2018-09-19 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We present an effective technique for training deep learning agents capable
of negotiating on a set of clauses in a contract agreement using a simple
communication protocol. We use Multi Agent Reinforcement Learning to train both
agents simultaneously as they negotiate with each other in the training
environment. We also model selfish and prosocial behavior to varying degrees in
these agents. Empirical evidence is provided showing consistency in agent
behaviors. We further train a meta agent with a mixture of behaviors by
learning an ensemble of different models using reinforcement learning. Finally,
to ascertain the deployability of the negotiating agents, we conducted
experiments pitting the trained agents against human players. Results
demonstrate that the agents are able to hold their own against human players,
often emerging as winners in the negotiation. Our experiments demonstrate that
the meta agent is able to reasonably emulate human behavior.
New approach for solar tracking systems based on computer vision, low cost hardware and deep learning
Authors: Jose A. Carballo, Javier Bonilla, Manuel Berenguel, Jesús Fernández-Reche, Ginés García
Date: 2018-09-19 00:00:00
Tasks: Shadow Detection

In this work, a new approach for Sun tracking systems is presented. Due to
the current system limitations regarding costs and operational problems, a new
approach based on low cost, computer vision open hardware and deep learning has
been developed. The preliminary tests carried out successfully in Plataforma
solar de Almeria (PSA), reveal the great potential and show the new approach as
a good alternative to traditional systems. The proposed approach can provide
key variables for the Sun tracking system control like cloud movements
prediction, block and shadow detection, atmospheric attenuation or measures of
concentrated solar radiation, which can improve the control strategies of the
system and therefore the system performance.
A Methodology for Search Space Reduction in QoS Aware Semantic Web Service Composition
Authors: Soumi Chattopadhyay, Ansuman Banerjee
Date: 2018-09-19 00:00:00
Tasks: Service Composition

The semantic information regulates the expressiveness of a web service.
State-of-the-art approaches in web services research have used the semantics of
a web service for different purposes, mainly for service discovery,
composition, execution etc. In this paper, our main focus is on semantic driven
Quality of Service (QoS) aware service composition. Most of the contemporary
approaches on service composition have used the semantic information to combine
the services appropriately to generate the composition solution. However, in
this paper, our intention is to use the semantic information to expedite the
service composition algorithm. Here, we present a service composition framework
that uses semantic information of a web service to generate different clusters,
where the services are semantically related within a cluster. Our final aim is
to construct a composition solution using these clusters that can efficiently
scale to large service spaces, while ensuring solution quality. Experimental
results show the efficiency of our proposed method.
NICT's Corpus Filtering Systems for the WMT18 Parallel Corpus Filtering Task
Authors: Rui Wang, Benjamin Marie, Masao Utiyama, Eiichiro Sumita
Date: 2018-09-19 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, NMT, Sentence, Translation

This paper presents the NICT's participation in the WMT18 shared parallel
corpus filtering task. The organizers provided 1 billion words German-English
corpus crawled from the web as part of the Paracrawl project. This corpus is
too noisy to build an acceptable neural machine translation (NMT) system. Using
the clean data of the WMT18 shared news translation task, we designed several
features and trained a classifier to score each sentence pairs in the noisy
data. Finally, we sampled 100 million and 10 million words and built
corresponding NMT systems. Empirical results show that our NMT systems trained
on sampled data achieve promising performance.
Exploring Visual Relationship for Image Captioning
Authors: Ting Yao, Yingwei Pan, Yehao Li, Tao Mei
Date: 2018-09-19 00:00:00
Proceeding: ECCV 2018 9
Tasks: Decoder, Image Captioning, Image Description, Sentence

It is always well believed that modeling relationships between objects would
be helpful for representing and eventually describing an image. Nevertheless,
there has not been evidence in support of the idea on image description
generation. In this paper, we introduce a new design to explore the connections
between objects for image captioning under the umbrella of attention-based
encoder-decoder framework. Specifically, we present Graph Convolutional
Networks plus Long Short-Term Memory (dubbed as GCN-LSTM) architecture that
novelly integrates both semantic and spatial object relationships into image
encoder. Technically, we build graphs over the detected objects in an image
based on their spatial and semantic connections. The representations of each
region proposed on objects are then refined by leveraging graph structure
through GCN. With the learnt region-level features, our GCN-LSTM capitalizes on
LSTM-based captioning framework with attention mechanism for sentence
generation. Extensive experiments are conducted on COCO image captioning
dataset, and superior results are reported when comparing to state-of-the-art
approaches. More remarkably, GCN-LSTM increases CIDEr-D performance from 120.1%
to 128.7% on COCO testing set.
NICT's Neural and Statistical Machine Translation Systems for the WMT18 News Translation Task
Authors: Benjamin Marie, Rui Wang, Atsushi Fujita, Masao Utiyama, Eiichiro Sumita
Date: 2018-09-19 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, NMT, Translation

This paper presents the NICT's participation to the WMT18 shared news
translation task. We participated in the eight translation directions of four
language pairs: Estonian-English, Finnish-English, Turkish-English and
Chinese-English. For each translation direction, we prepared state-of-the-art
statistical (SMT) and neural (NMT) machine translation systems. Our NMT systems
were trained with the transformer architecture using the provided parallel data
enlarged with a large quantity of back-translated monolingual data that we
generated with a new incremental training framework. Our primary submissions to
the task are the result of a simple combination of our SMT and NMT systems. Our
systems are ranked first for the Estonian-English and Finnish-English language
pairs (constraint) according to BLEU-cased.
The Key Concepts of Ethics of Artificial Intelligence - A Keyword based Systematic Mapping Study
Authors: Ville Vakkuri, Pekka Abrahamsson
Date: 2018-09-19 00:00:00
Tasks: Decision Making, Ethics

The growing influence and decision-making capacities of Autonomous systems
and Artificial Intelligence in our lives force us to consider the values
embedded in these systems. But how ethics should be implemented into these
systems? In this study, the solution is seen on philosophical conceptualization
as a framework to form practical implementation model for ethics of AI. To take
the first steps on conceptualization main concepts used on the field needs to
be identified. A keyword based Systematic Mapping Study (SMS) on the keywords
used in AI and ethics was conducted to help in identifying, defying and
comparing main concepts used in current AI ethics discourse. Out of 1062 papers
retrieved SMS discovered 37 re-occurring keywords in 83 academic papers. We
suggest that the focus on finding keywords is the first step in guiding and
providing direction for future research in the AI ethics field.
Removing the Feature Correlation Effect of Multiplicative Noise
Authors: Zijun Zhang, Yining Zhang, Zongpeng Li
Date: 2018-09-19 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Feature Correlation, image-classification, Image Classification

Multiplicative noise, including dropout, is widely used to regularize deep
neural networks (DNNs), and is shown to be effective in a wide range of
architectures and tasks. From an information perspective, we consider injecting
multiplicative noise into a DNN as training the network to solve the task with
noisy information pathways, which leads to the observation that multiplicative
noise tends to increase the correlation between features, so as to increase the
signal-to-noise ratio of information pathways. However, high feature
correlation is undesirable, as it increases redundancy in representations. In
this work, we propose non-correlating multiplicative noise (NCMN), which
exploits batch normalization to remove the correlation effect in a simple yet
effective way. We show that NCMN significantly improves the performance of
standard multiplicative noise on image classification tasks, providing a better
alternative to dropout for batch-normalized networks. Additionally, we present
a unified view of NCMN and shake-shake regularization, which explains the
performance gain of the latter.