pwc-archive/papers-with-abstracts huggingface dataset:
Combining Model and Parameter Uncertainty in Bayesian Neural Networks
Authors: Aliaksandr Hubin, Geir Storvik
Date: 2019-03-18 00:00:00
Tasks: Bayesian Inference, Model Selection, Variational Inference

Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using Bayesian approach: Parameter and prediction uncertainty become easily available, facilitating rigid statistical analysis. Furthermore, prior knowledge can be incorporated. However so far there have been no scalable techniques capable of combining both model (structural) and parameter uncertainty. In this paper we introduce the concept of model uncertainty in BNNs and hence make inference in the joint space of models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Finally, we show that incorporating model uncertainty via Bayesian model averaging and Bayesian model selection allows to drastically sparsify the structure of BNNs.
A Multilingual Encoding Method for Text Classification and Dialect Identification Using Convolutional Neural Network
Authors: Amr Adel Helmy
Date: 2019-03-18 00:00:00
Tasks: Dialect Identification, General Classification, text-classification, Text Classification

This thesis presents a language-independent text classification model by
introduced two new encoding methods "BUNOW" and "BUNOC" used for feeding the
raw text data into a new CNN spatial architecture with vertical and horizontal
convolutional process instead of commonly used methods like one hot vector or
word representation (i.e. word2vec) with temporal CNN architecture. The
proposed model can be classified as hybrid word-character model in its work
methodology because it consumes less memory space by using a fewer neural
network parameters as in character level representation, in addition to
providing much faster computations with fewer network layers depth, as in word
level representation. A promising result achieved compared to state of art
models in two different morphological benchmarked dataset one for Arabic
language and one for English language.
MediaRank: Computational Ranking of Online News Sources
Authors: Junting Ye, Steven Skiena
Date: 2019-03-18 00:00:00

In the recent political climate, the topic of news quality has drawn attention both from the public and the academic communities. The growing distrust of traditional news media makes it harder to find a common base of accepted truth. In this work, we design and build MediaRank (www.media-rank.com), a fully automated system to rank over 50,000 online news sources around the world. MediaRank collects and analyzes one million news webpages and two million related tweets everyday. We base our algorithmic analysis on four properties journalists have established to be associated with reporting quality: peer reputation, reporting bias / breadth, bottomline financial pressure, and popularity. Our major contributions of this paper include: (i) Open, interpretable quality rankings for over 50,000 of the world's major news sources. Our rankings are validated against 35 published news rankings, including French, German, Russian, and Spanish language sources. MediaRank scores correlate positively with 34 of 35 of these expert rankings. (ii) New computational methods for measuring influence and bottomline pressure. To the best of our knowledge, we are the first to study the large-scale news reporting citation graph in-depth. We also propose new ways to measure the aggressiveness of advertisements and identify social bots, establishing a connection between both types of bad behavior. (iii) Analyzing the effect of media source bias and significance. We prove that news sources cite others despite different political views in accord with quality measures. However, in four English-speaking countries (US, UK, Canada, and Australia), the highest ranking sources all disproportionately favor left-wing parties, even when the majority of news sources exhibited conservative slants.
Two models of double descent for weak features
Authors: Mikhail Belkin, Daniel Hsu, Ji Xu
Date: 2019-03-18 00:00:00
Tasks: BIG-bench Machine Learning, Vocal Bursts Valence Prediction

The "double descent" risk curve was proposed to qualitatively describe the out-of-sample prediction accuracy of variably-parameterized machine learning models. This article provides a precise mathematical analysis for the shape of this curve in two simple data models with the least squares/least norm predictor. Specifically, it is shown that the risk peaks when the number of features $p$ is close to the sample size $n$, but also that the risk decreases towards its minimum as $p$ increases beyond $n$. This behavior is contrasted with that of "prescient" models that select features in an a priori optimal order.
Human Activity Recognition for Edge Devices
Authors: Manjot Bilkhu, Hammababdullah Ayyubi
Date: 2019-03-18 00:00:00
Tasks: Activity Recognition, Human Activity Recognition

Video activity Recognition has recently gained a lot of momentum with the
release of massive Kinetics (400 and 600) data. Architectures such as I3D and
C3D networks have shown state-of-the-art performances for activity recognition.
The one major pitfall with these state-of-the-art networks is that they require
a lot of compute. In this paper we explore how we can achieve comparable
results to these state-of-the-art networks for devices-on-edge. We primarily
explore two architectures - I3D and Temporal Segment Network. We show that
comparable results can be achieved using one tenth the memory usage by changing
the testing procedure. We also report our results on Resnet architecture as our
backbone apart from the original Inception architecture. Specifically, we
achieve 84.54\% top-1 accuracy on UCF-101 dataset using only RGB frames.
Visual Cue Integration for Small Target Motion Detection in Natural Cluttered Backgrounds
Authors: Hongxin Wang, Jigen Peng, Qinbing Fu, Huatian Wang, Shigang Yue
Date: 2019-03-18 00:00:00
Tasks: Motion Detection

The robust detection of small targets against cluttered background is
important for future artificial visual systems in searching and tracking
applications. The insects' visual systems have demonstrated excellent ability
to avoid predators, find prey or identify conspecifics - which always appear as
small dim speckles in the visual field. Build a computational model of the
insects' visual pathways could provide effective solutions to detect small
moving targets. Although a few visual system models have been proposed, they
only make use of small-field visual features for motion detection and their
detection results often contain a number of false positives. To address this
issue, we develop a new visual system model for small target motion detection
against cluttered moving backgrounds. Compared to the existing models, the
small-field and wide-field visual features are separately extracted by two
motion-sensitive neurons to detect small target motion and background motion.
These two types of motion information are further integrated to filter out
false positives. Extensive experiments showed that the proposed model can
outperform the existing models in terms of detection rates.
LYRICS: a General Interface Layer to Integrate Logic Inference and Deep Learning
Authors: Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Marco Gori
Date: 2019-03-18 00:00:00

In spite of the amazing results obtained by deep learning in many applications, a real intelligent behavior of an agent acting in a complex environment is likely to require some kind of higher-level symbolic inference. Therefore, there is a clear need for the definition of a general and tight integration between low-level tasks, processing sensorial data that can be effectively elaborated using deep learning techniques, and the logic reasoning that allows humans to take decisions in complex environments. This paper presents LYRICS, a generic interface layer for AI, which is implemented in TersorFlow (TF). LYRICS provides an input language that allows to define arbitrary First Order Logic (FOL) background knowledge. The predicates and functions of the FOL knowledge can be bound to any TF computational graph, and the formulas are converted into a set of real-valued constraints, which participate to the overall optimization problem. This allows to learn the weights of the learners, under the constraints imposed by the prior knowledge. The framework is extremely general as it imposes no restrictions in terms of which models or knowledge can be integrated. In this paper, we show the generality of the approach showing some use cases of the presented language, including model checking, supervised learning and collective classification.
Computational Astrocyence: Astrocytes encode inhibitory activity into the frequency and spatial extent of their calcium elevations
Authors: Ioannis E. Polykretis, Vladimir A. Ivanov, Konstantinos P. Michmizos
Date: 2019-03-18 00:00:00

Deciphering the complex interactions between neurotransmission and astrocytic
$Ca^{2+}$ elevations is a target promising a comprehensive understanding of
brain function. While the astrocytic response to excitatory synaptic activity
has been extensively studied, how inhibitory activity results to intracellular
$Ca^{2+}$ waves remains elusive. In this study, we developed a compartmental
astrocytic model that exhibits distinct levels of responsiveness to inhibitory
activity. Our model suggested that the astrocytic coverage of inhibitory
terminals defines the spatial and temporal scale of their $Ca^{2+}$ elevations.
Understanding the interplay between the synaptic pathways and the astrocytic
responses will help us identify how astrocytes work independently and
cooperatively with neurons, in health and disease.
Subjectivity and complexity of facial attractiveness
Authors: Miguel Ibáñez-Berganza, Ambra Amico, Vittorio Loreto
Date: 2019-03-18 00:00:00

The origin and meaning of facial beauty represent a longstanding puzzle. Despite the profuse literature devoted to facial attractiveness, its very nature, its determinants and the nature of inter-person differences remain controversial issues. Here we tackle such questions proposing a novel experimental approach in which human subjects, instead of rating natural faces, are allowed to efficiently explore the face-space and 'sculpt' their favorite variation of a reference facial image. The results reveal that different subjects prefer distinguishable regions of the face-space, highlighting the essential subjectivity of the phenomenon.The different sculpted facial vectors exhibit strong correlations among pairs of facial distances, characterising the underlying universality and complexity of the cognitive processes, and the relative relevance and robustness of the different facial distances.
PZnet: Efficient 3D ConvNet Inference on Manycore CPUs
Authors: Sergiy Popovych, Davit Buniatyan, Aleksandar Zlateski, Kai Li, H. Sebastian Seung
Date: 2019-03-18 00:00:00
Tasks: CPU, GPU

Convolutional nets have been shown to achieve state-of-the-art accuracy in
many biomedical image analysis tasks. Many tasks within biomedical analysis
domain involve analyzing volumetric (3D) data acquired by CT, MRI and
Microscopy acquisition methods. To deploy convolutional nets in practical
working systems, it is important to solve the efficient inference problem.
Namely, one should be able to apply an already-trained convolutional network to
many large images using limited computational resources. In this paper we
present PZnet, a CPU-only engine that can be used to perform inference for a
variety of 3D convolutional net architectures. PZNet outperforms MKL-based CPU
implementations of PyTorch and Tensorflow by more than 3.5x for the popular
U-net architecture. Moreover, for 3D convolutions with low featuremap numbers,
cloud CPU inference with PZnet outperfroms cloud GPU inference in terms of cost
efficiency.
EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras
Authors: Anton Mitrokhin, Chengxi Ye, Cornelia Fermuller, Yiannis Aloimonos, Tobi Delbruck
Date: 2019-03-18 00:00:00
Tasks: Motion Segmentation, Object, Semantic Segmentation

We present the first event-based learning approach for motion segmentation in indoor scenes and the first event-based dataset - EV-IMO - which includes accurate pixel-wise motion masks, egomotion and ground truth depth. Our approach is based on an efficient implementation of the SfM learning pipeline using a low parameter neural network architecture on event data. In addition to camera egomotion and a dense depth map, the network estimates pixel-wise independently moving object segmentation and computes per-object 3D translational velocities for moving objects. We also train a shallow network with just 40k parameters, which is able to compute depth and egomotion. Our EV-IMO dataset features 32 minutes of indoor recording with up to 3 fast moving objects simultaneously in the camera field of view. The objects and the camera are tracked by the VICON motion capture system. By 3D scanning the room and the objects, accurate depth map ground truth and pixel-wise object masks are obtained, which are reliable even in poor lighting conditions and during fast motion. We then train and evaluate our learning pipeline on EV-IMO and demonstrate that our approach far surpasses its rivals and is well suited for scene constrained robotics applications.
Extrapolating paths with graph neural networks
Authors: Jean-Baptiste Cordonnier, Andreas Loukas
Date: 2019-03-18 00:00:00
Tasks: Graph Neural Network

We consider the problem of path inference: given a path prefix, i.e., a
partially observed sequence of nodes in a graph, we want to predict which nodes
are in the missing suffix. In particular, we focus on natural paths occurring
as a by-product of the interaction of an agent with a network---a driver on the
transportation network, an information seeker in Wikipedia, or a client in an
online shop. Our interest is sparked by the realization that, in contrast to
shortest-path problems, natural paths are usually not optimal in any
graph-theoretic sense, but might still follow predictable patterns.
  Our main contribution is a graph neural network called Gretel. Conditioned on
a path prefix, this network can efficiently extrapolate path suffixes, evaluate
path likelihood, and sample from the future path distribution. Our experiments
with GPS traces on a road network and user-navigation paths in Wikipedia
confirm that Gretel is able to adapt to graphs with very different properties,
while also comparing favorably to previous solutions.
Approximating exponential family models (not single distributions) with a two-network architecture
Authors: Sean R. Bittner, John P. Cunningham
Date: 2019-03-18 00:00:00
Tasks: General Classification, Variational Inference

Recently much attention has been paid to deep generative models, since they
have been used to great success for variational inference, generation of
complex data types, and more. In most all of these settings, the goal has been
to find a particular member of that model family: optimized parameters index a
distribution that is close (via a divergence or classification metric) to a
target distribution. Much less attention, however, has been paid to the problem
of learning a model itself. Here we introduce a two-network architecture and
optimization procedure for learning intractable exponential family models (not
a single distribution from those models). These exponential families are
learned accurately, allowing operations like posterior inference to be executed
directly and generically with an input choice of natural parameters, rather
than performing inference via optimization for each particular distribution
within that model.
A Comparison of Prediction Algorithms and Nexting for Short Term Weather Forecasts
Authors: Michael Koller, Johannes Feldmaier, Klaus Diepold
Date: 2019-03-18 00:00:00
Tasks: regression, Reinforcement Learning, Reinforcement Learning (RL)

This report first provides a brief overview of a number of supervised
learning algorithms for regression tasks. Among those are neural networks,
regression trees, and the recently introduced Nexting. Nexting has been
presented in the context of reinforcement learning where it was used to predict
a large number of signals at different timescales. In the second half of this
report, we apply the algorithms to historical weather data in order to evaluate
their suitability to forecast a local weather trend. Our experiments did not
identify one clearly preferable method, but rather show that choosing an
appropriate algorithm depends on the available side information. For slowly
varying signals and a proficient number of training samples, Nexting achieved
good results in the studied cases.
Forecasting the Progression of Alzheimer's Disease Using Neural Networks and a Novel Pre-Processing Algorithm
Authors: Jack Albright
Date: 2019-03-18 00:00:00
Tasks: BIG-bench Machine Learning

Alzheimer's disease (AD) is the most common neurodegenerative disease in
older people. Despite considerable efforts to find a cure for AD, there is a
99.6% failure rate of clinical trials for AD drugs, likely because AD patients
cannot easily be identified at early stages. This project investigated machine
learning approaches to predict the clinical state of patients in future years
to benefit AD research. Clinical data from 1737 patients was obtained from the
Alzheimer's Disease Neuroimaging Initiative (ADNI) database and was processed
using the "All-Pairs" technique, a novel methodology created for this project
involving the comparison of all possible pairs of temporal data points for each
patient. This data was then used to train various machine learning models.
Models were evaluated using 7-fold cross-validation on the training dataset and
confirmed using data from a separate testing dataset (110 patients). A neural
network model was effective (mAUC = 0.866) at predicting the progression of AD
on a month-by-month basis, both in patients who were initially cognitively
normal and in patients suffering from mild cognitive impairment. Such a model
could be used to identify patients at early stages of AD and who are therefore
good candidates for clinical trials for AD therapeutics.
An Effective Label Noise Model for DNN Text Classification
Authors: Ishan Jindal, Daniel Pressel, Brian Lester, Matthew Nokleby
Date: 2019-03-18 00:00:00
Proceeding: NAACL 2019 6
Tasks: Classification, General Classification, image-classification, Image Classification, Sentence, text-classification, Text Classification

Because large, human-annotated datasets suffer from labeling errors, it is
crucial to be able to train deep neural networks in the presence of label
noise. While training image classification models with label noise have
received much attention, training text classification models have not. In this
paper, we propose an approach to training deep networks that is robust to label
noise. This approach introduces a non-linear processing layer (noise model)
that models the statistics of the label noise into a convolutional neural
network (CNN) architecture. The noise model and the CNN weights are learned
jointly from noisy training data, which prevents the model from overfitting to
erroneous labels. Through extensive experiments on several text classification
datasets, we show that this approach enables the CNN to learn better sentence
representations and is robust even to extreme label noise. We find that proper
initialization and regularization of this noise model is critical. Further, by
contrast to results focusing on large batch sizes for mitigating label noise
for image classification, we find that altering the batch size does not have
much effect on classification performance.
Bilinear Representation for Language-based Image Editing Using Conditional Generative Adversarial Networks
Authors: Xiaofeng Mao, Yuefeng Chen, Yuhong Li, Tao Xiong, Yuan He, Hui Xue
Date: 2019-03-18 00:00:00
Tasks: Generative Adversarial Network

The task of Language-Based Image Editing (LBIE) aims at generating a target
image by editing the source image based on the given language description. The
main challenge of LBIE is to disentangle the semantics in image and text and
then combine them to generate realistic images. Therefore, the editing
performance is heavily dependent on the learned representation. In this work,
conditional generative adversarial network (cGAN) is utilized for LBIE. We find
that existing conditioning methods in cGAN lack of representation power as they
cannot learn the second-order correlation between two conditioning vectors. To
solve this problem, we propose an improved conditional layer named Bilinear
Residual Layer (BRL) to learning more powerful representations for LBIE task.
Qualitative and quantitative comparisons demonstrate that our method can
generate images with higher quality when compared to previous LBIE techniques.
Advanced Capsule Networks via Context Awareness
Authors: Nguyen Huu Phong, Bernardete Ribeiro
Date: 2019-03-18 00:00:00
Tasks: Image Restoration

Capsule Networks (CN) offer new architectures for Deep Learning (DL) community. Though its effectiveness has been demonstrated in MNIST and smallNORB datasets, the networks still face challenges in other datasets for images with distinct contexts. In this research, we improve the design of CN (Vector version) namely we expand more Pooling layers to filter image backgrounds and increase Reconstruction layers to make better image restoration. Additionally, we perform experiments to compare accuracy and speed of CN versus DL models. In DL models, we utilize Inception V3 and DenseNet V201 for powerful computers besides NASNet, MobileNet V1 and MobileNet V2 for small and embedded devices. We evaluate our models on a fingerspelling alphabet dataset from American Sign Language (ASL). The results show that CNs perform comparably to DL models while dramatically reducing training time. We also make a demonstration and give a link for the purpose of illustration.
Offline and Online Deep Learning for Image Recognition
Authors: Nguyen Huu Phong, Bernardete Ribeiro
Date: 2019-03-18 00:00:00
Tasks: Deep Learning

Image recognition using Deep Learning has been evolved for decades though
advances in the field through different settings is still a challenge. In this
paper, we present our findings in searching for better image classifiers in
offline and online environments. We resort to Convolutional Neural Network and
its variations of fully connected Multi-layer Perceptron. Though still
preliminary, these results are encouraging and may provide a better
understanding about the field and directions toward future works.
Exploiting Hierarchy for Learning and Transfer in KL-regularized RL
Authors: Dhruva Tirumala, Hyeonwoo Noh, Alexandre Galashov, Leonard Hasenclever, Arun Ahuja, Greg Wayne, Razvan Pascanu, Yee Whye Teh, Nicolas Heess
Date: 2019-03-18 00:00:00
Tasks: continuous-control, Continuous Control, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

As reinforcement learning agents are tasked with solving more challenging and diverse tasks, the ability to incorporate prior knowledge into the learning system and to exploit reusable structure in solution space is likely to become increasingly important. The KL-regularized expected reward objective constitutes one possible tool to this end. It introduces an additional component, a default or prior behavior, which can be learned alongside the policy and as such partially transforms the reinforcement learning problem into one of behavior modelling. In this work we consider the implications of this framework in cases where both the policy and default behavior are augmented with latent variables. We discuss how the resulting hierarchical structures can be used to implement different inductive biases and how their modularity can benefit transfer. Empirically we find that they can lead to faster learning and transfer on a range of continuous control tasks.