pwc-archive/papers-with-abstracts huggingface dataset:
Predictive Uncertainty Quantification with Compound Density Networks
Authors: Agustinus Kristiadi, Sina Däubener, Asja Fischer
Date: 2019-02-04 00:00:00
Tasks: Bayesian Inference, Uncertainty Quantification

Despite the huge success of deep neural networks (NNs), finding good mechanisms for quantifying their prediction uncertainty is still an open problem. Bayesian neural networks are one of the most popular approaches to uncertainty quantification. On the other hand, it was recently shown that ensembles of NNs, which belong to the class of mixture models, can be used to quantify prediction uncertainty. In this paper, we build upon these two approaches. First, we increase the mixture model's flexibility by replacing the fixed mixing weights by an adaptive, input-dependent distribution (specifying the probability of each component) represented by NNs, and by considering uncountably many mixture components. The resulting class of models can be seen as the continuous counterpart to mixture density networks and is therefore referred to as compound density networks (CDNs). We employ both maximum likelihood and variational Bayesian inference to train CDNs, and empirically show that they yield better uncertainty estimates on out-of-distribution data and are more robust to adversarial examples than the previous approaches.
Saliency Tubes: Visual Explanations for Spatio-Temporal Convolutions
Authors: Alexandros Stergiou, Georgios Kapidis, Grigorios Kalliatakis, Christos Chrysoulas, Remco Veltkamp, Ronald Poppe
Date: 2019-02-04 00:00:00
Tasks: Action Classification, General Classification, Video Classification

Deep learning approaches have been established as the main methodology for video classification and recognition. Recently, 3-dimensional convolutions have been used to achieve state-of-the-art performance in many challenging video datasets. Because of the high level of complexity of these methods, as the convolution operations are also extended to additional dimension in order to extract features from them as well, providing a visualization for the signals that the network interpret as informative, is a challenging task. An effective notion of understanding the network's inner-workings would be to isolate the spatio-temporal regions on the video that the network finds most informative. We propose a method called Saliency Tubes which demonstrate the foremost points and regions in both frame level and over time that are found to be the main focus points of the network. We demonstrate our findings on widely used datasets for third-person and egocentric action classification and enhance the set of methods and visualizations that improve 3D Convolutional Neural Networks (CNNs) intelligibility.
Jumping Manifolds: Geometry Aware Dense Non-Rigid Structure from Motion
Authors: Suryansh Kumar
Date: 2019-02-04 00:00:00
Proceeding: CVPR 2019 6
Tasks: Clustering

Given dense image feature correspondences of a non-rigidly moving object
across multiple frames, this paper proposes an algorithm to estimate its 3D
shape for each frame. To solve this problem accurately, the recent
state-of-the-art algorithm reduces this task to set of local linear subspace
reconstruction and clustering problem using Grassmann manifold representation
\cite{kumar2018scalable}. Unfortunately, their method missed on some of the
critical issues associated with the modeling of surface deformations, for e.g.,
the dependence of a local surface deformation on its neighbors. Furthermore,
their representation to group high dimensional data points inevitably introduce
the drawbacks of categorizing samples on the high-dimensional Grassmann
manifold \cite{huang2015projection, harandi2014manifold}. Hence, to deal with
such limitations with \cite{kumar2018scalable}, we propose an algorithm that
jointly exploits the benefit of high-dimensional Grassmann manifold to perform
reconstruction, and its equivalent lower-dimensional representation to infer
suitable clusters. To accomplish this, we project each Grassmannians onto a
lower-dimensional Grassmann manifold which preserves and respects the
deformation of the structure w.r.t its neighbors. These Grassmann points in the
lower-dimension then act as a representative for the selection of
high-dimensional Grassmann samples to perform each local reconstruction. In
practice, our algorithm provides a geometrically efficient way to solve dense
NRSfM by switching between manifolds based on its benefit and usage.
Experimental results show that the proposed algorithm is very effective in
handling noise with reconstruction accuracy as good as or better than the
competing methods.
Causal Effect Identification from Multiple Incomplete Data Sources: A General Search-based Approach
Authors: Santtu Tikka, Antti Hyttinen, Juha Karvanen
Date: 2019-02-04 00:00:00
Tasks: Causal Identification, Causal Inference, Selection bias

Causal effect identification considers whether an interventional probability distribution can be uniquely determined without parametric assumptions from measured source distributions and structural knowledge on the generating system. While complete graphical criteria and procedures exist for many identification problems, there are still challenging but important extensions that have not been considered in the literature. To tackle these new settings, we present a search algorithm directly over the rules of do-calculus. Due to generality of do-calculus, the search is capable of taking more advanced data-generating mechanisms into account along with an arbitrary type of both observational and experimental source distributions. The search is enhanced via a heuristic and search space reduction techniques. The approach, called do-search, is provably sound, and it is complete with respect to identifiability problems that have been shown to be completely characterized by do-calculus. When extended with additional rules, the search is capable of handling missing data problems as well. With the versatile search, we are able to approach new problems such as combined transportability and selection bias, or multiple sources of selection bias. We perform a systematic analysis of bivariate missing data problems and study causal inference under case-control design. We also present the R package dosearch that provides an interface for a C++ implementation of the search.
A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization
Authors: Wonseok Hwang, Jinyeong Yim, Seunghyun Park, Minjoon Seo
Date: 2019-02-04 00:00:00
Tasks: Decoder, Semantic Parsing

We present SQLova, the first Natural-language-to-SQL (NL2SQL) model to achieve human performance in WikiSQL dataset. We revisit and discuss diverse popular methods in NL2SQL literature, take a full advantage of BERT {Devlin et al., 2018) through an effective table contextualization method, and coherently combine them, outperforming the previous state of the art by 8.2% and 2.5% in logical form and execution accuracy, respectively. We particularly note that BERT with a seq2seq decoder leads to a poor performance in the task, indicating the importance of a careful design when using such large pretrained models. We also provide a comprehensive analysis on the dataset and our model, which can be helpful for designing future NL2SQL datsets and models. We especially show that our model's performance is near the upper bound in WikiSQL, where we observe that a large portion of the evaluation errors are due to wrong annotations, and our model is already exceeding human performance by 1.3% in execution accuracy.
Hop: Heterogeneity-Aware Decentralized Training
Authors: Qinyi Luo, JinKun Lin, Youwei Zhuo, Xuehai Qian
Date: 2019-02-04 00:00:00

Recent work has shown that decentralized algorithms can deliver superior
performance over centralized ones in the context of machine learning. The two
approaches, with the main difference residing in their distinct communication
patterns, are both susceptible to performance degradation in heterogeneous
environments. Although vigorous efforts have been devoted to supporting
centralized algorithms against heterogeneity, little has been explored in
decentralized algorithms regarding this problem.
  This paper proposes Hop, the first heterogeneity-aware decentralized training
protocol. Based on a unique characteristic of decentralized training that we
have identified, the iteration gap, we propose a queue-based synchronization
mechanism that can efficiently implement backup workers and bounded staleness
in the decentralized setting. To cope with deterministic slowdown, we propose
skipping iterations so that the effect of slower workers is further mitigated.
We build a prototype implementation of Hop on TensorFlow. The experiment
results on CNN and SVM show significant speedup over standard decentralized
training in heterogeneous settings.
3D point cloud registration with shape constraint
Authors: Swapna Agarwal, Brojeshwar Bhowmick
Date: 2019-02-04 00:00:00
Tasks: Change Detection, Point Cloud Registration, Translation

In this paper, a shape-constrained iterative algorithm is proposed to
register a rigid template point-cloud to a given reference point-cloud. The
algorithm embeds a shape-based similarity constraint into the principle of
gravitation. The shape-constrained gravitation, as induced by the reference,
controls the movement of the template such that at each iteration, the template
better aligns with the reference in terms of shape. This constraint enables the
alignment in difficult conditions indtroduced by change (presence of outliers
and/or missing parts), translation, rotation and scaling. We discuss efficient
implementation techniques with least manual intervention. The registration is
shown to be useful for change detection in the 3D point-cloud. The algorithm is
compared with three state-of-the-art registration approaches. The experiments
are done on both synthetic and real-world data. The proposed algorithm is shown
to perform better in the presence of big rotation, structured and unstructured
outliers and missing data.
End-to-end feature fusion siamese network for adaptive visual tracking
Authors: Dongyan Guo, Jun Wang, Weixuan Zhao, Ying Cui, Zhenhua Wang, Sheng-Yong Chen
Date: 2019-02-04 00:00:00
Tasks: Visual Tracking

According to observations, different visual objects have different salient
features in different scenarios. Even for the same object, its salient shape
and appearance features may change greatly from time to time in a long-term
tracking task. Motivated by them, we proposed an end-to-end feature fusion
framework based on Siamese network, named FF-Siam, which can effectively fuse
different features for adaptive visual tracking. The framework consists of four
layers. A feature extraction layer is designed to extract the different
features of the target region and search region. The extracted features are
then put into a weight generation layer to obtain the channel weights, which
indicate the importance of different feature channels. Both features and the
channel weights are utilized in a template generation layer to generate a
discriminative template. Finally, the corresponding response maps created by
the convolution of the search region features and the template are applied with
a fusion layer to obtain the final response map for locating the target.
Experimental results demonstrate that the proposed framework achieves
state-of-the-art performance on the popular Temple-Color, OTB50 and UAV123
benchmarks.
Online Multiclass Classification Based on Prediction Margin for Partial Feedback
Authors: Takuo Kaneko, Issei Sato, Masashi Sugiyama
Date: 2019-02-04 00:00:00
Tasks: Classification, General Classification

We consider the problem of online multiclass classification with partial
feedback, where an algorithm predicts a class for a new instance in each round
and only receives its correctness. Although several methods have been developed
for this problem, recent challenging real-world applications require further
performance improvement. In this paper, we propose a novel online learning
algorithm inspired by recent work on learning from complementary labels, where
a complementary label indicates a class to which an instance does not belong.
This allows us to handle partial feedback deterministically in a margin-based
way, where the prediction margin has been recognized as a key to superior
empirical performance. We provide a theoretical guarantee based on a cumulative
loss bound and experimentally demonstrate that our method outperforms existing
methods which are non-margin-based and stochastic.
Towards Federated Learning at Scale: System Design
Authors: Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečný, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, Jason Roselander
Date: 2019-02-04 00:00:00
Tasks: BIG-bench Machine Learning, Federated Learning

Federated Learning is a distributed machine learning approach which enables
model training on a large corpus of decentralized data. We have built a
scalable production system for Federated Learning in the domain of mobile
devices, based on TensorFlow. In this paper, we describe the resulting
high-level design, sketch some of the challenges and their solutions, and touch
upon the open problems and future directions.
Fully Convolutional Networks for Monocular Retinal Depth Estimation and Optic Disc-Cup Segmentation
Authors: Sharath M. Shankaranarayana, Keerthi Ram, Kaushik Mitra, Mohanasankar Sivaprakasam
Date: 2019-02-04 00:00:00
Tasks: Depth Estimation

Glaucoma is a serious ocular disorder for which the screening and diagnosis
are carried out by the examination of the optic nerve head (ONH). The color
fundus image (CFI) is the most common modality used for ocular screening. In
CFI, the central r
Prediction of ESG Compliance using a Heterogeneous Information Network
Authors: Ryohei Hisano, Didier Sornette, Takayuki Mizuno
Date: 2018-11-09 00:00:00
Tasks: Management, Prediction

Negative screening is one method to avoid interactions with inappropriate entities. For example, financial institutions keep investment exclusion lists of inappropriate firms that have environmental, social, and government (ESG) problems. They create their investment exclusion lists by gathering information from various news sources to keep their portfolios profitable as well as green. International organizations also maintain smart sanctions lists that are used to prohibit trade with entities that are involved in illegal activities. In the present paper, we focus on the prediction of investment exclusion lists in the finance domain. We construct a vast heterogeneous information network that covers the necessary information surrounding each firm, which is assembled using seven professionally curated datasets and two open datasets, which results in approximately 50 million nodes and 400 million edges in total. Exploiting these vast datasets and motivated by how professional investigators and journalists undertake their daily investigations, we propose a model that can learn to predict firms that are more likely to be added to an investment exclusion list in the near future. Our approach is tested using the negative news investment exclusion list data of more than 35,000 firms worldwide from January 2012 to May 2018. Comparing with the state-of-the-art methods with and without using the network, we show that the predictive accuracy is substantially improved when using the vast information stored in the heterogeneous information network. This work suggests new ways to consolidate the diffuse information contained in big data to monitor dominant firms on a global scale for better risk management and more socially responsible investment.
Towards Pedestrian Detection Using RetinaNet in ECCV 2018 Wider Pedestrian Detection Challenge
Authors: Md Ashraful Alam Milton
Date: 2019-02-04 00:00:00
Tasks: General Classification, image-classification, Image Classification, Object, object-detection, Object Detection, Pedestrian Detection

The main essence of this paper is to investigate the performance of RetinaNet
based object detectors on pedestrian detection. Pedestrian detection is an
important research topic as it provides a baseline for general object detection
and has a great number of practical applications like autonomous car, robotics
and Security camera. Though extensive research has made huge progress in
pedestrian detection, there are still many issues and open for more research
and improvement. Recent deep learning based methods have shown state-of-the-art
performance in computer vision tasks such as image classification, object
detection, and segmentation. Wider pedestrian detection challenge aims at
finding improve solutions for pedestrian detection problem. In this paper, We
propose a pedestrian detection system based on RetinaNet. Our solution has
scored 0.4061 mAP. The code is available at
https://github.com/miltonbd/ECCV_2018_pedestrian_detection_challenege.
Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers
Authors: Haoyu Wang, Ming Tan, Mo Yu, Shiyu Chang, Dakuo Wang, Kun Xu, Xiaoxiao Guo, Saloni Potdar
Date: 2019-02-04 00:00:00
Proceeding: ACL 2019 7
Tasks: Relation Extraction, Structured Prediction

Most approaches to extraction multiple relations from a paragraph require multiple passes over the paragraph. In practice, multiple passes are computationally expensive and this makes difficult to scale to longer paragraphs and larger text corpora. In this work, we focus on the task of multiple relation extraction by encoding the paragraph only once (one-pass). We build our solution on the pre-trained self-attentive (Transformer) models, where we first add a structured prediction layer to handle extraction between multiple entity pairs, then enhance the paragraph embedding to capture multiple relational information associated with each entity with an entity-aware attention technique. We show that our approach is not only scalable but can also perform state-of-the-art on the standard benchmark ACE 2005.
Can SGD Learn Recurrent Neural Networks with Provable Generalization?
Authors: Zeyuan Allen-Zhu, Yuanzhi Li
Date: 2019-02-04 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Generalization Bounds, PAC learning

Recurrent Neural Networks (RNNs) are among the most popular models in sequential data analysis. Yet, in the foundational PAC learning language, what concept class can it learn? Moreover, how can the same recurrent unit simultaneously learn functions from different input tokens to different output tokens, without affecting each other? Existing generalization bounds for RNN scale exponentially with the input length, significantly limiting their practical implications. In this paper, we show using the vanilla stochastic gradient descent (SGD), RNN can actually learn some notable concept class efficiently, meaning that both time and sample complexity scale polynomially in the input length (or almost polynomially, depending on the concept). This concept class at least includes functions where each output token is generated from inputs of earlier tokens using a smooth two-layer neural network.
Graph Warp Module: an Auxiliary Module for Boosting the Power of Graph Neural Networks in Molecular Graph Analysis
Authors: Katsuhiko Ishiguro, Shin-ichi Maeda, Masanori Koyama
Date: 2019-02-04 00:00:00
Tasks: Graph Neural Network

Graph Neural Network (GNN) is a popular architecture for the analysis of chemical molecules, and it has numerous applications in material and medicinal science. Current lines of GNNs developed for molecular analysis, however, do not fit well on the training set, and their performance does not scale well with the complexity of the network. In this paper, we propose an auxiliary module to be attached to a GNN that can boost the representation power of the model without hindering with the original GNN architecture. Our auxiliary module can be attached to a wide variety of GNNs, including those that are used commonly in biochemical applications. With our auxiliary architecture, the performances of many GNNs used in practice improve more consistently, achieving the state-of-the-art performance on popular molecular graph datasets.
Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network
Authors: Shervin Minaee, Amirali Abdolrashidi
Date: 2019-02-04 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER), Image-Variation

Facial expression recognition has been an active research area over the past
few decades, and it is still challenging due to the high intra-class variation.
  Traditional approaches for this problem rely on hand-crafted features such as
SIFT, HOG and LBP, followed by a classifier trained on a database of images or
videos.
  Most of these works perform reasonably well on datasets of images captured in
a controlled condition, but fail to perform as good on more challenging
datasets with more image variation and partial faces.
  In recent years, several works proposed an end-to-end framework for facial
expression recognition, using deep learning models.
  Despite the better performance of these works, there still seems to be a
great room for improvement.
  In this work, we propose a deep learning approach based on attentional
convolutional network, which is able to focus on important parts of the face,
and achieves significant improvement over previous models on multiple datasets,
including FER-2013, CK+, FERG, and JAFFE.
  We also use a visualization technique which is able to find important face
regions for detecting different emotions, based on the classifier's output.
  Through experimental results, we show that different emotions seems to be
sensitive to different parts of the face.
Seizure Type Classification using EEG signals and Machine Learning: Setting a benchmark
Authors: Subhrajit Roy, Umar Asif, Jianbin Tang, Stefan Harrer
Date: 2019-02-04 00:00:00
Tasks: BIG-bench Machine Learning, Classification, EEG, Electroencephalogram (EEG), General Classification, Management, Seizure Detection

Accurate classification of seizure types plays a crucial role in the treatment and disease management of epileptic patients. Epileptic seizure types not only impact the choice of drugs but also the range of activities a patient can safely engage in. With recent advances being made towards artificial intelligence enabled automatic seizure detection, the next frontier is the automatic classification of seizure types. On that note, in this paper, we explore the application of machine learning algorithms for multi-class seizure type classification. We used the recently released TUH EEG seizure corpus (V1.4.0 and V1.5.2) and conducted a thorough search space exploration to evaluate the performance of a combination of various pre-processing techniques, machine learning algorithms, and corresponding hyperparameters on this task. We show that our algorithms can reach a weighted $F1$ score of up to 0.901 for seizure-wise cross validation and 0.561 for patient-wise cross validation thereby setting a benchmark for scalp EEG based multi-class seizure type classification.