pwc-archive/papers-with-abstracts huggingface dataset:
Efficient Reward-Based Structural Plasticity on a SpiNNaker 2 Prototype
Authors: Yexin Yan, David Kappel, Felix Neumaerker, Johannes Partzsch, Bernhard Vogginger, Sebastian Hoeppner, Steve Furber, Wolfgang Maass, Robert Legenstein, Christian Mayr
Date: 2019-03-20 00:00:00

Advances in neuroscience uncover the mechanisms employed by the brain to
efficiently solve complex learning tasks with very limited resources. However,
the efficiency is often lost when one tries to port these findings to a silicon
substrate, since brain-inspired algorithms often make extensive use of complex
functions such as random number generators, that are expensive to compute on
standard general purpose hardware. The prototype chip of the 2nd generation
SpiNNaker system is designed to overcome this problem. Low-power ARM processors
equipped with a random number generator and an exponential function accelerator
enable the efficient execution of brain-inspired algorithms. We implement the
recently introduced reward-based synaptic sampling model that employs
structural plasticity to learn a function or task. The numerical simulation of
the model requires to update the synapse variables in each time step including
an explorative random term. To the best of our knowledge, this is the most
complex synapse model implemented so far on the SpiNNaker system. By making
efficient use of the hardware accelerators and numerical optimizations the
computation time of one plasticity update is reduced by a factor of 2. This,
combined with fitting the model into to the local SRAM, leads to 62% energy
reduction compared to the case without accelerators and the use of external
DRAM. The model implementation is integrated into the SpiNNaker software
framework allowing for scalability onto larger systems. The hardware-software
system presented in this work paves the way for power-efficient mobile and
biomedical applications with biologically plausible brain-inspired algorithms.
Artificial Intelligence : from Research to Application ; the Upper-Rhine Artificial Intelligence Symposium (UR-AI 2019)
Authors: Andreas Christ, Franz Quint
Date: 2019-03-20 00:00:00
Tasks: Management

The TriRhenaTech alliance universities and their partners presented their
competences in the field of artificial intelligence and their cross-border
cooperations with the industry at the tri-national conference 'Artificial
Intelligence : from Research to Application' on March 13th, 2019 in Offenburg.
The TriRhenaTech alliance is a network of universities in the Upper Rhine
Trinational Metropolitan Region comprising of the German universities of
applied sciences in Furtwangen, Kaiserslautern, Karlsruhe, and Offenburg, the
Baden-Wuerttemberg Cooperative State University Loerrach, the French university
network Alsace Tech (comprised of 14 'grandes \'ecoles' in the fields of
engineering, architecture and management) and the University of Applied
Sciences and Arts Northwestern Switzerland. The alliance's common goal is to
reinforce the transfer of knowledge, research, and technology, as well as the
cross-border mobility of students.
Three-dimensional Segmentation of Trees Through a Flexible Multi-Class Graph Cut Algorithm (MCGC)
Authors: Jonathan Williams, Carola-Bibiane Schönlieb, Tom Swinfield, Juheon Lee, Xiaohao Cai, Lan Qie, David A. Coomes
Date: 2019-03-20 00:00:00

Developing a robust algorithm for automatic individual tree crown (ITC)
detection from laser scanning datasets is important for tracking the responses
of trees to anthropogenic change. Such approaches allow the size, growth and
mortality of individual trees to be measured, enabling forest carbon stocks and
dynamics to be tracked and understood. Many algorithms exist for structurally
simple forests including coniferous forests and plantations. Finding a robust
solution for structurally complex, species-rich tropical forests remains a
challenge; existing segmentation algorithms often perform less well than simple
area-based approaches when estimating plot-level biomass. Here we describe a
Multi-Class Graph Cut (MCGC) approach to tree crown delineation. This uses
local three-dimensional geometry and density information, alongside knowledge
of crown allometries, to segment individual tree crowns from LiDAR point
clouds. Our approach robustly identifies trees in the top and intermediate
layers of the canopy, but cannot recognise small trees. From these
three-dimensional crowns, we are able to measure individual tree biomass.
Comparing these estimates to those from permanent inventory plots, our
algorithm is able to produce robust estimates of hectare-scale carbon density,
demonstrating the power of ITC approaches in monitoring forests. The
flexibility of our method to add additional dimensions of information, such as
spectral reflectance, make this approach an obvious avenue for future
development and extension to other sources of three-dimensional data, such as
structure from motion datasets.
Deep Octonion Networks
Authors: Jiasong Wu, Ling Xu, Youyong Kong, Lotfi Senhadji, Huazhong Shu
Date: 2019-03-20 00:00:00
Tasks: General Classification, image-classification, Image Classification, Multi-Task Learning

Deep learning is a research hot topic in the field of machine learning.
Real-value neural networks (Real NNs), especially deep real networks (DRNs),
have been widely used in many research fields. In recent years, the deep
complex networks (DCNs) and the deep quaternion networks (DQNs) have attracted
more and more attentions. The octonion algebra, which is an extension of
complex algebra and quaternion algebra, can provide more efficient and compact
expression. This paper constructs a general framework of deep octonion networks
(DONs) and provides the main building blocks of DONs such as octonion
convolution, octonion batch normalization and octonion weight initialization;
DONs are then used in image classification tasks for CIFAR-10 and CIFAR-100
data sets. Compared with the DRNs, the DCNs, and the DQNs, the proposed DONs
have better convergence and higher classification accuracy. The success of DONs
is also explained by multi-task learning.
In Defense of Pre-trained ImageNet Architectures for Real-time Semantic Segmentation of Road-driving Images
Authors: Marin Oršić, Ivan Krešo, Petra Bevandić, Siniša Šegvić
Date: 2019-03-20 00:00:00
Tasks: Real-Time Semantic Segmentation, Semantic Segmentation

Recent success of semantic segmentation approaches on demanding road driving
datasets has spurred interest in many related application fields. Many of these
applications involve real-time prediction on mobile platforms such as cars,
drones and various kinds of robots. Real-time setup is challenging due to
extraordinary computational complexity involved. Many previous works address
the challenge with custom lightweight architectures which decrease
computational complexity by reducing depth, width and layer capacity with
respect to general purpose architectures. We propose an alternative approach
which achieves a significantly better performance across a wide range of
computing budgets. First, we rely on a light-weight general purpose
architecture as the main recognition engine. Then, we leverage light-weight
upsampling with lateral connections as the most cost-effective solution to
restore the prediction resolution. Finally, we propose to enlarge the receptive
field by fusing shared features at multiple resolutions in a novel fashion.
Experiments on several road driving datasets show a substantial advantage of
the proposed approach, either with ImageNet pre-trained parameters or when we
learn from scratch. Our Cityscapes test submission entitled SwiftNetRN-18
delivers 75.5% MIoU and achieves 39.9 Hz on 1024x2048 images on GTX1080Ti.
On Class Imbalance and Background Filtering in Visual Relationship Detection
Authors: Alessio Sarullo, Tingting Mu
Date: 2019-03-20 00:00:00
Tasks: Relationship Detection, Visual Relationship Detection

In this paper we investigate the problems of class imbalance and irrelevant
relationships in Visual Relationship Detection (VRD). State-of-the-art deep VRD
models still struggle to predict uncommon classes, limiting their
applicability. Moreover, many methods are incapable of properly filtering out
background relationships while predicting relevant ones. Although these
problems are very apparent, they have both been overlooked so far. We analyse
why this is the case and propose modifications to both model and training to
alleviate the aforementioned issues, as well as suggesting new measures to
complement existing ones and give a more holistic picture of the efficacy of a
model.
Extracting Frequent Gradual Patterns Using Constraints Modeling
Authors: Jerry Lonlac, Saïdd Jabbour, Engelbert Mephu Nguifo, Lakhdar Saïs, Badran Raddaoui
Date: 2019-03-20 00:00:00

In this paper, we propose a constraint-based modeling approach for the
problem of discovering frequent gradual patterns in a numerical dataset. This
SAT-based declarative approach offers an additional possibility to benefit from
the recent progress in satisfiability testing and to exploit the efficiency of
modern SAT solvers for enumerating all frequent gradual patterns in a numerical
dataset. Our approach can easily be extended with extra constraints, such as
temporal constraints in order to extract more specific patterns in a broad
range of gradual patterns mining applications. We show the practical
feasibility of our SAT model by running experiments on two real world datasets.
Decay-Function-Free Time-Aware Attention to Context and Speaker Indicator for Spoken Language Understanding
Authors: Jonggu Kim, Jong-Hyeok Lee
Date: 2019-03-20 00:00:00
Proceeding: NAACL 2019 6
Tasks: dialog state tracking, Spoken Language Understanding

To capture salient contextual information for spoken language understanding (SLU) of a dialogue, we propose time-aware models that automatically learn the latent time-decay function of the history without a manual time-decay function. We also propose a method to identify and label the current speaker to improve the SLU accuracy. In experiments on the benchmark dataset used in Dialog State Tracking Challenge 4, the proposed models achieved significantly higher F1 scores than the state-of-the-art contextual models. Finally, we analyze the effectiveness of the introduced models in detail. The analysis demonstrates that the proposed methods were effective to improve SLU accuracy individually.
Left-to-Right Dependency Parsing with Pointer Networks
Authors: Daniel Fernández-González, Carlos Gómez-Rodríguez
Date: 2019-03-20 00:00:00
Tasks: Dependency Parsing, Sentence

We propose a novel transition-based algorithm that straightforwardly parses
sentences from left to right by building $n$ attachments, with $n$ being the
length of the input sentence. Similarly to the recent stack-pointer parser by
Ma et al. (2018), we use the pointer network framework that, given a word, can
directly point to a position from the sentence. However, our left-to-right
approach is simpler than the original top-down stack-pointer parser (not
requiring a stack) and reduces transition sequence length in half, from 2$n$-1
actions to $n$. This results in a quadratic non-projective parser that runs
twice as fast as the original while achieving the best accuracy to date on the
English PTB dataset (96.04% UAS, 94.43% LAS) among fully-supervised
single-model dependency parsers, and improves over the former top-down
transition system in the majority of languages tested.
Designing nanophotonic structures using conditional-deep convolutional generative adversarial networks
Authors: Sunae So, Junsuk Rho
Date: 2019-03-20 00:00:00
Tasks: Form

Data-driven design approaches based on deep-learning have been introduced in
nanophotonics to reduce time-consuming iterative simulations which have been a
major challenge. Here, we report the first use of conditional deep
convolutional generative adversarial networks to design nanophotonic antennae
that are not constrained to a predefined shape. For given input reflection
spectra, the network generates desirable designs in the form of images; this
form allows suggestions of new structures that cannot be represented by
structural parameters. Simulation results obtained from the generated designs
agreed well with the input reflection spectrum. This method opens new avenues
towards the development of nanophotonics by providing a fast and convenient
approach to design complex nanophotonic structures that have desired optical
properties.
Counterexample-Guided Strategy Improvement for POMDPs Using Recurrent Neural Networks
Authors: Steven Carr, Nils Jansen, Ralf Wimmer, Alexandru C. Serban, Bernd Becker, Ufuk Topcu
Date: 2019-03-20 00:00:00
Tasks: Diagnostic

We study strategy synthesis for partially observable Markov decision
processes (POMDPs). The particular problem is to determine strategies that
provably adhere to (probabilistic) temporal logic constraints. This problem is
computationally intractable and theoretically hard. We propose a novel method
that combines techniques from machine learning and formal verification. First,
we train a recurrent neural network (RNN) to encode POMDP strategies. The RNN
accounts for memory-based decisions without the need to expand the full belief
space of a POMDP. Secondly, we restrict the RNN-based strategy to represent a
finite-memory strategy and implement it on a specific POMDP. For the resulting
finite Markov chain, efficient formal verification techniques provide provable
guarantees against temporal logic specifications. If the specification is not
satisfied, counterexamples supply diagnostic information. We use this
information to improve the strategy by iteratively training the RNN. Numerical
experiments show that the proposed method elevates the state of the art in
POMDP solving by up to three orders of magnitude in terms of solving times and
model sizes.
Modeling Intelligent Decision Making Command And Control Agents: An Application to Air Defense
Authors: Sumanta Kumar Das
Date: 2019-03-20 00:00:00
Tasks: Decision Making, Mathematical Reasoning

The paper is a half-way between the agent technology and the mathematical
reasoning to model tactical decision making tasks. These models are applied to
air defense (AD) domain for command and control (C2). It also addresses the
issues related to evaluation of agents. The agents are designed and implemented
using the agent-programming paradigm. The agents are deployed in an air combat
simulated environment for performing the tasks of C2 like electronic counter
counter measures, threat assessment, and weapon allocation. The simulated AD
system runs without any human intervention, and represents state-of-the-art
model for C2 autonomy. The use of agents as autonomous decision making entities
is particularly useful in view of futuristic network centric warfare.
Modelling Sequential Music Track Skips using a Multi-RNN Approach
Authors: Christian Hansen, Casper Hansen, Stephen Alstrup, Jakob Grue Simonsen, Christina Lioma
Date: 2019-03-20 00:00:00
Tasks: Sequential skip prediction

Modelling sequential music skips provides streaming companies the ability to
better understand the needs of the user base, resulting in a better user
experience by reducing the need to manually skip certain music tracks. This
paper describes the solution of the University of Copenhagen DIKU-IR team in
the 'Spotify Sequential Skip Prediction Challenge', where the task was to
predict the skip behaviour of the second half in a music listening session
conditioned on the first half. We model this task using a Multi-RNN approach
consisting of two distinct stacked recurrent neural networks, where one network
focuses on encoding the first half of the session and the other network focuses
on utilizing the encoding to make sequential skip predictions. The encoder
network is initialized by a learned session-wide music encoding, and both of
them utilize a learned track embedding. Our final model consists of a majority
voted ensemble of individually trained models, and ranked 2nd out of 45
participating teams in the competition with a mean average accuracy of 0.641
and an accuracy on the first skip prediction of 0.807. Our code is released at
https://github.com/Varyn/WSDM-challenge-2019-spotify.
Neural Check-Worthiness Ranking with Weak Supervision: Finding Sentences for Fact-Checking
Authors: Casper Hansen, Christian Hansen, Stephen Alstrup, Jakob Grue Simonsen, Christina Lioma
Date: 2019-03-20 00:00:00
Tasks: Fact Checking, Misinformation, Sentence

Automatic fact-checking systems detect misinformation, such as fake news, by
(i) selecting check-worthy sentences for fact-checking, (ii) gathering related
information to the sentences, and (iii) inferring the factuality of the
sentences. Most prior research on (i) uses hand-crafted features to select
check-worthy sentences, and does not explicitly account for the recent finding
that the top weighted terms in both check-worthy and non-check-worthy sentences
are actually overlapping [15]. Motivated by this, we present a neural
check-worthiness sentence ranking model that represents each word in a sentence
by \textit{both} its embedding (aiming to capture its semantics) and its
syntactic dependencies (aiming to capture its role in modifying the semantics
of other terms in the sentence). Our model is an end-to-end trainable neural
network for check-worthiness ranking, which is trained on large amounts of
unlabelled data through weak supervision. Thorough experimental evaluation
against state of the art baselines, with and without weak supervision, shows
our model to be superior at all times (+13% in MAP and +28% at various
Precision cut-offs from the best baseline with statistical significance).
Empirical analysis of the use of weak supervision, word embedding pretraining
on domain-specific data, and the use of syntactic dependencies of our model
reveals that check-worthy sentences contain notably more identical syntactic
dependencies than non-check-worthy sentences.
Contextual Compositionality Detection with External Knowledge Bases andWord Embeddings
Authors: Dongsheng Wang, Quichi Li, Lucas Chaves Lima, Jakob Grue Simonsen, Christina Lioma
Date: 2019-03-20 00:00:00

When the meaning of a phrase cannot be inferred from the individual meanings
of its words (e.g., hot dog), that phrase is said to be non-compositional.
Automatic compositionality detection in multi-word phrases is critical in any
application of semantic processing, such as search engines; failing to detect
non-compositional phrases can hurt system effectiveness notably. Existing
research treats phrases as either compositional or non-compositional in a
deterministic manner. In this paper, we operationalize the viewpoint that
compositionality is contextual rather than deterministic, i.e., that whether a
phrase is compositional or non-compositional depends on its context. For
example, the phrase `green card' is compositional when referring to a green
colored card, whereas it is non-compositional when meaning permanent residence
authorization. We address the challenge of detecting this type of contextual
compositionality as follows: given a multi-word phrase, we enrich the word
embedding representing its semantics with evidence about its global context
(terms it often collocates with) as well as its local context (narratives where
that phrase is used, which we call usage scenarios). We further extend this
representation with information extracted from external knowledge bases. The
resulting representation incorporates both localized context and more general
usage of the phrase and allows to detect its compositionality in a
non-deterministic and contextual way. Empirical evaluation of our model on a
dataset of phrase compositionality, manually collected by crowdsourcing
contextual compositionality assessments, shows that our model outperforms
state-of-the-art baselines notably on detecting phrase compositionality.
Convolution with even-sized kernels and symmetric padding
Authors: Shuang Wu, Guanrui Wang, Pei Tang, Feng Chen, Luping Shi
Date: 2019-03-20 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Continual Learning, image-classification, Image Classification

Compact convolutional neural networks gain efficiency mainly through depthwise convolutions, expanded channels and complex topologies, which contrarily aggravate the training process. Besides, 3x3 kernels dominate the spatial representation in these models, whereas even-sized kernels (2x2, 4x4) are rarely adopted. In this work, we quantify the shift problem occurs in even-sized kernel convolutions by an information erosion hypothesis, and eliminate it by proposing symmetric padding on four sides of the feature maps (C2sp, C4sp). Symmetric padding releases the generalization capabilities of even-sized kernels at little computational cost, making them outperform 3x3 kernels in image classification and generation tasks. Moreover, C2sp obtains comparable accuracy to emerging compact models with much less memory and time consumption during training. Symmetric padding coupled with even-sized convolutions can be neatly implemented into existing frameworks, providing effective elements for architecture designs, especially on online and continual learning occasions where training efforts are emphasized.
Uncertainty quantification of molecular property prediction with Bayesian neural networks
Authors: Seongok Ryu, Yongchan Kwon, Woo Youn Kim
Date: 2019-03-20 00:00:00
Tasks: Molecular Property Prediction, Property Prediction, Uncertainty Quantification

Deep neural networks have outperformed existing machine learning models in
various molecular applications. In practical applications, it is still
difficult to make confident decisions because of the uncertainty in predictions
arisen from insufficient quality and quantity of training data. Here, we show
that Bayesian neural networks are useful to quantify the uncertainty of
molecular property prediction with three numerical experiments. In particular,
it enables us to decompose the predictive variance into the model- and
data-driven uncertainties, which helps to elucidate the source of errors. In
the logP predictions, we show that data noise affected the data-driven
uncertainties more significantly than the model-driven ones. Based on this
analysis, we were able to find unexpected errors in the Harvard Clean Energy
Project dataset. Lastly, we show that the confidence of prediction is closely
related to the predictive uncertainty by performing on bio-activity and
toxicity classification problems.
Regularize, Expand and Compress: Multi-task based Lifelong Learning via NonExpansive AutoML
Authors: Jie Zhang, Junting Zhang, Shalini Ghosh, Dawei Li, Jingwen Zhu, Heming Zhang, Yalin Wang
Date: 2019-03-20 00:00:00
Tasks: AutoML, Continual Learning, Lifelong learning

Lifelong learning, the problem of continual learning where tasks arrive in
sequence, has been lately attracting more attention in the computer vision
community. The aim of lifelong learning is to develop a system that can learn
new tasks while maintaining the performance on the previously learned tasks.
However, there are two obstacles for lifelong learning of deep neural networks:
catastrophic forgetting and capacity limitation. To solve the above issues,
inspired by the recent breakthroughs in automatically learning good neural
network architectures, we develop a Multi-task based lifelong learning via
nonexpansive AutoML framework termed Regularize, Expand and Compress (REC). REC
is composed of three stages: 1) continually learns the sequential tasks without
the learned tasks' data via a newly proposed multi-task weight consolidation
(MWC) algorithm; 2) expands the network to help the lifelong learning with
potentially improved model capability and performance by network-transformation
based AutoML; 3) compresses the expanded model after learning every new task to
maintain model efficiency and performance. The proposed MWC and REC algorithms
achieve superior performance over other lifelong learning algorithms on four
different datasets.