pwc-archive/papers-with-abstracts huggingface dataset:
A Scalable Test Suite for Continuous Dynamic Multiobjective Optimisation
Authors: Shouyong Jiang, Marcus Kaiser, Shengxiang Yang, Stefanos Kollias, Natalio Krasnogor
Date: 2019-03-06 00:00:00

Dynamic multiobjective optimisation has gained increasing attention in recent
years. Test problems are of great importance in order to facilitate the
development of advanced algorithms that can handle dynamic environments well.
However, many of existing dynamic multiobjective test problems have not been
rigorously constructed and analysed, which may induce some unexpected bias when
they are used for algorithmic analysis. In this paper, some of these biases are
identified after a review of widely used test problems. These include poor
scalability of objectives and, more importantly, problematic overemphasis of
static properties rather than dynamics making it difficult to draw accurate
conclusion about the strengths and weaknesses of the algorithms studied. A
diverse set of dynamics and features is then highlighted that a good test suite
should have. We further develop a scalable continuous test suite, which
includes a number of dynamics or features that have been rarely considered in
literature but frequently occur in real life. It is demonstrated with empirical
studies that the proposed test suite is more challenging to the dynamic
multiobjective optimisation algorithms found in the literature. The test suite
can also test algorithms in ways that existing test suites can not.
Image captioning with weakly-supervised attention penalty
Authors: Jiayun Li, Mohammad K. Ebrahimpour, Azadeh Moghtaderi, Yen-Yun Yu
Date: 2019-03-06 00:00:00
Tasks: Decoder, Image Captioning

Stories are essential for genealogy research since they can help build
emotional connections with people. A lot of family stories are reserved in
historical photos and albums. Recent development on image captioning models
makes it feasible to "tell stories" for photos automatically. The attention
mechanism has been widely adopted in many state-of-the-art encoder-decoder
based image captioning models, since it can bridge the gap between the visual
part and the language part. Most existing captioning models implicitly trained
attention modules with word-likelihood loss. Meanwhile, lots of studies have
investigated intrinsic attentions for visual models using gradient-based
approaches. Ideally, attention maps predicted by captioning models should be
consistent with intrinsic attentions from visual models for any given visual
concept. However, no work has been done to align implicitly learned attention
maps with intrinsic visual attentions. In this paper, we proposed a novel model
that measured consistency between captioning predicted attentions and intrinsic
visual attentions. This alignment loss allows explicit attention correction
without using any expensive bounding box annotations. We developed and
evaluated our model on COCO dataset as well as a genealogical dataset from
Ancestry.com Operations Inc., which contains billions of historical photos. The
proposed model achieved better performances on all commonly used language
evaluation metrics for both datasets.
Spiking Neural Network on Neuromorphic Hardware for Energy-Efficient Unidimensional SLAM
Authors: Guangzhi Tang, Arpit Shah, Konstantinos P. Michmizos
Date: 2019-03-06 00:00:00
Tasks: Bayesian Inference, CPU, Simultaneous Localization and Mapping

Energy-efficient simultaneous localization and mapping (SLAM) is crucial for mobile robots exploring unknown environments. The mammalian brain solves SLAM via a network of specialized neurons, exhibiting asynchronous computations and event-based communications, with very low energy consumption. We propose a brain-inspired spiking neural network (SNN) architecture that solves the unidimensional SLAM by introducing spike-based reference frame transformation, visual likelihood computation, and Bayesian inference. We integrated our neuromorphic algorithm to Intel's Loihi neuromorphic processor, a non-Von Neumann hardware that mimics the brain's computing paradigms. We performed comparative analyses for accuracy and energy-efficiency between our neuromorphic approach and the GMapping algorithm, which is widely used in small environments. Our Loihi-based SNN architecture consumes 100 times less energy than GMapping run on a CPU while having comparable accuracy in head direction localization and map-generation. These results pave the way for scaling our approach towards active-SLAM alternative solutions for Loihi-controlled autonomous robots.
Prostate Segmentation from 3D MRI Using a Two-Stage Model and Variable-Input Based Uncertainty Measure
Authors: Huitong Pan, Yushan Feng, Quan Chen, Craig Meyer, Xue Feng
Date: 2019-03-06 00:00:00
Tasks: Data Augmentation, Segmentation

This paper proposes a two-stage segmentation model, variable-input based
uncertainty measures and an uncertainty-guided post-processing method for
prostate segmentation on 3D magnetic resonance images (MRI). The two-stage
model was based on 3D dilated U-Nets with the first stage to localize the
prostate and the second stage to obtain an accurate segmentation from cropped
images. For data augmentation, we proposed the variable-input method which
crops the region of interest with additional random variations. Similar to
other deep learning models, the proposed model also faced the challenge of
suboptimal performance in certain testing cases due to varied training and
testing image characteristics. Therefore, it is valuable to evaluate the
confidence and performance of the network using uncertainty measures, which are
often calculated from the probability maps or their standard deviations with
multiple model outputs for the same testing case. However, few studies have
quantitatively compared different methods of uncertainty calculation.
Furthermore, unlike the commonly used Bayesian dropout during testing, we
developed uncertainty measures based on the variable input images at the second
stage and evaluated its performance by calculating the correlation with
ground-truth-based performance metrics, such as Dice score. For performance
estimation, we predicted Dice scores and Hausdorff distance with the most
correlated uncertainty measure. For post-processing, we performed Gaussian
filter on the underperformed slices to improve segmentation quality. Using
PROMISE-12 data, we demonstrated the robustness of the two-stage model and
showed high correlation of the proposed variable-input based uncertainty
measures with GT-based performance. The uncertainty-guided post-processing
method significantly improved label smoothness.
Human Attention in Image Captioning: Dataset and Analysis
Authors: Sen He, Hamed R. -Tavakoli, Ali Borji, Nicolas Pugeault
Date: 2019-03-06 00:00:00
Proceeding: ICCV 2019 10
Tasks: Image Captioning, Image Description, Sentence, Task 2

In this work, we present a novel dataset consisting of eye movements and verbal descriptions recorded synchronously over images. Using this data, we study the differences in human attention during free-viewing and image captioning tasks. We look into the relationship between human attention and language constructs during perception and sentence articulation. We also analyse attention deployment mechanisms in the top-down soft attention approach that is argued to mimic human attention in captioning tasks, and investigate whether visual saliency can help image captioning. Our study reveals that (1) human attention behaviour differs in free-viewing and image description tasks. Humans tend to fixate on a greater variety of regions under the latter task, (2) there is a strong relationship between described objects and attended objects ($97\%$ of the described objects are being attended), (3) a convolutional neural network as feature encoder accounts for human-attended regions during image captioning to a great extent (around $78\%$), (4) soft-attention mechanism differs from human attention, both spatially and temporally, and there is low correlation between caption scores and attention consistency scores. These indicate a large gap between humans and machines in regards to top-down attention, and (5) by integrating the soft attention model with image saliency, we can significantly improve the model's performance on Flickr30k and MSCOCO benchmarks. The dataset can be found at: https://github.com/SenHe/Human-Attention-in-Image-Captioning.
Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries
Authors: Jawadul H. Bappy, Cody Simons, Lakshmanan Nataraj, B. S. Manjunath, Amit K. Roy-Chowdhury
Date: 2019-03-06 00:00:00
Tasks: Decoder

With advanced image journaling tools, one can easily alter the semantic
meaning of an image by exploiting certain manipulation techniques such as
copy-clone, object splicing, and removal, which mislead the viewers. In
contrast, the identification of these manipulations becomes a very challenging
task as manipulated regions are not visually apparent. This paper proposes a
high-confidence manipulation localization architecture which utilizes
resampling features, Long-Short Term Memory (LSTM) cells, and encoder-decoder
network to segment out manipulated regions from non-manipulated ones.
Resampling features are used to capture artifacts like JPEG quality loss,
upsampling, downsampling, rotation, and shearing. The proposed network exploits
larger receptive fields (spatial maps) and frequency domain correlation to
analyze the discriminative characteristics between manipulated and
non-manipulated regions by incorporating encoder and LSTM network. Finally,
decoder network learns the mapping from low-resolution feature maps to
pixel-wise predictions for image tamper localization. With predicted mask
provided by final layer (softmax) of the proposed architecture, end-to-end
training is performed to learn the network parameters through back-propagation
using ground-truth masks. Furthermore, a large image splicing dataset is
introduced to guide the training process. The proposed method is capable of
localizing image manipulations at pixel level with high precision, which is
demonstrated through rigorous experimentation on three diverse datasets.
GQ-STN: Optimizing One-Shot Grasp Detection based on Robustness Classifier
Authors: Alexandre Gariépy, Jean-Christophe Ruel, Brahim Chaib-Draa, Philippe Giguère
Date: 2019-03-06 00:00:00

Grasping is a fundamental robotic task needed for the deployment of household robots or furthering warehouse automation. However, few approaches are able to perform grasp detection in real time (frame rate). To this effect, we present Grasp Quality Spatial Transformer Network (GQ-STN), a one-shot grasp detection network. Being based on the Spatial Transformer Network (STN), it produces not only a grasp configuration, but also directly outputs a depth image centered at this configuration. By connecting our architecture to an externally-trained grasp robustness evaluation network, we can train efficiently to satisfy a robustness metric via the backpropagation of the gradient emanating from the evaluation network. This removes the difficulty of training detection networks on sparsely annotated databases, a common issue in grasping. We further propose to use this robustness classifier to compare approaches, being more reliable than the traditional rectangle metric. Our GQ-STN is able to detect robust grasps on the depth images of the Dex-Net 2.0 dataset with 92.4 % accuracy in a single pass of the network. We finally demonstrate in a physical benchmark that our method can propose robust grasps more often than previous sampling-based methods, while being more than 60 times faster.
LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models
Authors: Yuan Zhou, Bradley J. Gram-Hansen, Tobias Kohn, Tom Rainforth, Hongseok Yang, Frank Wood
Date: 2019-03-06 00:00:00
Tasks: Probabilistic Programming

We develop a new Low-level, First-order Probabilistic Programming Language
(LF-PPL) suited for models containing a mix of continuous, discrete, and/or
piecewise-continuous variables. The key success of this language and its
compilation scheme is in its ability to automatically distinguish parameters
the density function is discontinuous with respect to, while further providing
runtime checks for boundary crossings. This enables the introduction of new
inference engines that are able to exploit gradient information, while
remaining efficient for models which are not everywhere differentiable. We
demonstrate this ability by incorporating a discontinuous Hamiltonian Monte
Carlo (DHMC) inference engine that is able to deliver automated and efficient
inference for non-differentiable models. Our system is backed up by a
mathematical formalism that ensures that any model expressed in this language
has a density with measure zero discontinuities to maintain the validity of the
inference engine.
Orthogonal Structure Search for Efficient Causal Discovery from Observational Data
Authors: Anant Raj, Luigi Gresele, Michel Besserve, Bernhard Schölkopf, Stefan Bauer
Date: 2019-03-06 00:00:00
Tasks: Causal Discovery, regression

The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines. Recent work exploits stability of regression coefficients or invariance properties of models across different experimental conditions for reconstructing the full causal graph. These approaches generally do not scale well with the number of the explanatory variables and are difficult to extend to nonlinear relationships. Contrary to existing work, we propose an approach which even works for observational data alone, while still offering theoretical guarantees including the case of partially nonlinear relationships. Our algorithm requires only one estimation for each variable and in our experiments we apply our causal discovery algorithm even to large graphs, demonstrating significant improvements compared to well established approaches.
SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks with at most one Spike per Neuron
Authors: Milad Mozafari, Mohammad Ganjtabesh, Abbas Nowzari-Dalini, Timothée Masquelier
Date: 2019-03-06 00:00:00
Tasks: GPU

Application of deep convolutional spiking neural networks (SNNs) to artificial intelligence (AI) tasks has recently gained a lot of interest since SNNs are hardware-friendly and energy-efficient. Unlike the non-spiking counterparts, most of the existing SNN simulation frameworks are not practically efficient enough for large-scale AI tasks. In this paper, we introduce SpykeTorch, an open-source high-speed simulation framework based on PyTorch. This framework simulates convolutional SNNs with at most one spike per neuron and the rank-order encoding scheme. In terms of learning rules, both spike-timing-dependent plasticity (STDP) and reward-modulated STDP (R-STDP) are implemented, but other rules could be implemented easily. Apart from the aforementioned properties, SpykeTorch is highly generic and capable of reproducing the results of various studies. Computations in the proposed framework are tensor-based and totally done by PyTorch functions, which in turn brings the ability of just-in-time optimization for running on CPUs, GPUs, or Multi-GPU platforms.
Fast Graph Representation Learning with PyTorch Geometric
Authors: Matthias Fey, Jan Eric Lenssen
Date: 2019-03-06 00:00:00
Tasks: GPU, Graph Classification, Graph Representation Learning, Node Classification, Relational Reasoning, Representation Learning

We introduce PyTorch Geometric, a library for deep learning on irregularly
structured input data such as graphs, point clouds and manifolds, built upon
PyTorch. In addition to general graph data structures and processing methods,
it contains a variety of recently published methods from the domains of
relational learning and 3D data processing. PyTorch Geometric achieves high
data throughput by leveraging sparse GPU acceleration, by providing dedicated
CUDA kernels and by introducing efficient mini-batch handling for input
examples of different size. In this work, we present the library in detail and
perform a comprehensive comparative study of the implemented methods in
homogeneous evaluation scenarios.
KBQA: Learning Question Answering over QA Corpora and Knowledge Bases
Authors: Wanyun Cui, Yanghua Xiao, Haixun Wang, Yangqiu Song, Seung-won Hwang, Wei Wang
Date: 2019-03-06 00:00:00
Tasks: Question Answering

Question answering (QA) has become a popular way for humans to access
billion-scale knowledge bases. Unlike web search, QA over a knowledge base
gives out accurate and concise results, provided that natural language
questions can be understood and mapped precisely to structured queries over the
knowledge base. The challenge, however, is that a human can ask one question in
many different ways. Previous approaches have natural limits due to their
representations: rule based approaches only understand a small set of "canned"
questions, while keyword based or synonym based approaches cannot fully
understand the questions. In this paper, we design a new kind of question
representation: templates, over a billion scale knowledge base and a million
scale QA corpora. For example, for questions about a city's population, we
learn templates such as What's the population of $city?, How many people are
there in $city?. We learned 27 million templates for 2782 intents. Based on
these templates, our QA system KBQA effectively supports binary factoid
questions, as well as complex questions which are composed of a series of
binary factoid questions. Furthermore, we expand predicates in RDF knowledge
base, which boosts the coverage of knowledge base by 57 times. Our QA system
beats all other state-of-art works on both effectiveness and efficiency over
QALD benchmarks.
Explaining Anomalies Detected by Autoencoders Using SHAP
Authors: Liat Antwarg, Ronnie Mindlin Miller, Bracha Shapira, Lior Rokach
Date: 2019-03-06 00:00:00
Tasks: Anomaly Detection, Outlier Detection

Anomaly detection algorithms are often thought to be limited because they don't facilitate the process of validating results performed by domain experts. In Contrast, deep learning algorithms for anomaly detection, such as autoencoders, point out the outliers, saving experts the time-consuming task of examining normal cases in order to find anomalies. Most outlier detection algorithms output a score for each instance in the database. The top-k most intense outliers are returned to the user for further inspection; however the manual validation of results becomes challenging without additional clues. An explanation of why an instance is anomalous enables the experts to focus their investigation on most important anomalies and may increase their trust in the algorithm. Recently, a game theory-based framework known as SHapley Additive exPlanations (SHAP) has been shown to be effective in explaining various supervised learning models. In this research, we extend SHAP to explain anomalies detected by an autoencoder, an unsupervised model. The proposed method extracts and visually depicts both the features that most contributed to the anomaly and those that offset it. A preliminary experimental study using real world data demonstrates the usefulness of the proposed method in assisting the domain experts to understand the anomaly and filtering out the uninteresting anomalies, aiming at minimizing the false positive rate of detected anomalies.
Detecting Overfitting via Adversarial Examples
Authors: Roman Werpachowski, András György, Csaba Szepesvári
Date: 2019-03-06 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: General Classification, image-classification, Image Classification

The repeated community-wide reuse of test sets in popular benchmark problems raises doubts about the credibility of reported test-error rates. Verifying whether a learned model is overfitted to a test set is challenging as independent test sets drawn from the same data distribution are usually unavailable, while other test sets may introduce a distribution shift. We propose a new hypothesis test that uses only the original test data to detect overfitting. It utilizes a new unbiased error estimate that is based on adversarial examples generated from the test data and importance weighting. Overfitting is detected if this error estimate is sufficiently different from the original test error rate. We develop a specialized variant of our test for multiclass image classification, and apply it to testing overfitting of recent models to the popular ImageNet benchmark. Our method correctly indicates overfitting of the trained model to the training set, but is not able to detect any overfitting to the test set, in line with other recent work on this topic.
Compressing complex convolutional neural network based on an improved deep compression algorithm
Authors: Jiasong Wu, Hongshan Ren, Youyong Kong, Chunfeng Yang, Lotfi Senhadji, Huazhong Shu
Date: 2019-03-06 00:00:00

Although convolutional neural network (CNN) has made great progress, large
redundant parameters restrict its deployment on embedded devices, especially
mobile devices. The recent compression works are focused on real-value
convolutional neural network (Real CNN), however, to our knowledge, there is no
attempt for the compression of complex-value convolutional neural network
(Complex CNN). Compared with the real-valued network, the complex-value neural
network is easier to optimize, generalize, and has better learning potential.
This paper extends the commonly used deep compression algorithm from real
domain to complex domain and proposes an improved deep compression algorithm
for the compression of Complex CNN. The proposed algorithm compresses the
network about 8 times on CIFAR-10 dataset with less than 3% accuracy loss. On
the ImageNet dataset, our method compresses the model about 16 times and the
accuracy loss is about 2% without retraining.
Understanding the Artificial Intelligence Clinician and optimal treatment strategies for sepsis in intensive care
Authors: Matthieu Komorowski, Leo A. Celi, Omar Badawi, Anthony C. Gordon, A. Aldo Faisal
Date: 2019-03-06 00:00:00
Tasks: Reinforcement Learning

In this document, we explore in more detail our published work (Komorowski,
Celi, Badawi, Gordon, & Faisal, 2018) for the benefit of the AI in Healthcare
research community. In the above paper, we developed the AI Clinician system,
which demonstrated how reinforcement learning could be used to make useful
recommendations towards optimal treatment decisions from intensive care data.
Since publication a number of authors have reviewed our work (e.g. Abbasi,
2018; Bos, Azoulay, & Martin-Loeches, 2019; Saria, 2018). Given the difference
of our framework to previous work, the fact that we are bridging two very
different academic communities (intensive care and machine learning) and that
our work has impact on a number of other areas with more traditional
computer-based approaches (biosignal processing and control, biomedical
engineering), we are providing here additional details on our recent
publication.