pwc-archive/papers-with-abstracts huggingface dataset:
Directing DNNs Attention for Facial Attribution Classification using Gradient-weighted Class Activation Mapping
Authors: Xi Yang, Bojian Wu, Issei Sato, Takeo Igarashi
Date: 2019-05-02 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

Deep neural networks (DNNs) have a high accuracy on image classification
tasks. However, DNNs trained by such dataset with co-occurrence bias may rely
on wrong features while making decisions for classification. It will greatly
affect the transferability of pre-trained DNNs. In this paper, we propose an
interactive method to direct classifiers paying attentions to the regions that
are manually specified by the users, in order to mitigate the influence of
co-occurrence bias. We test on CelebA dataset, the pre-trained AlexNet is
fine-tuned to focus on the specific facial attributes based on the results of
Grad-CAM.
Coordination and Trajectory Prediction for Vehicle Interactions via Bayesian Generative Modeling
Authors: Jiachen Li, Hengbo Ma, Wei Zhan, Masayoshi Tomizuka
Date: 2019-05-02 00:00:00
Tasks: Diversity, Prediction, Trajectory Prediction

Coordination recognition and subtle pattern prediction of future trajectories
play a significant role when modeling interactive behaviors of multiple agents.
Due to the essential property of uncertainty in the future evolution,
deterministic predictors are not sufficiently safe and robust. In order to
tackle the task of probabilistic prediction for multiple, interactive entities,
we propose a coordination and trajectory prediction system (CTPS), which has a
hierarchical structure including a macro-level coordination recognition module
and a micro-level subtle pattern prediction module which solves a probabilistic
generation task. We illustrate two types of representation of the coordination
variable: categorized and real-valued, and compare their effects and advantages
based on empirical studies. We also bring the ideas of Bayesian deep learning
into deep generative models to generate diversified prediction hypotheses. The
proposed system is tested on multiple driving datasets in various traffic
scenarios, which achieves better performance than baseline approaches in terms
of a set of evaluation metrics. The results also show that using categorized
coordination can better capture multi-modality and generate more diversified
samples than the real-valued coordination, while the latter can generate
prediction hypotheses with smaller errors with a sacrifice of sample diversity.
Moreover, employing neural networks with weight uncertainty is able to generate
samples with larger variance and diversity.
Estimating Kullback-Leibler Divergence Using Kernel Machines
Authors: Kartik Ahuja
Date: 2019-05-02 00:00:00

Recently, a method called the Mutual Information Neural Estimator (MINE) that uses neural networks has been proposed to estimate mutual information and more generally the Kullback-Leibler (KL) divergence between two distributions. The method uses the Donsker-Varadhan representation to arrive at the estimate of the KL divergence and is better than the existing estimators in terms of scalability and flexibility. The output of MINE algorithm is not guaranteed to be a consistent estimator. We propose a new estimator that instead of searching among functions characterized by neural networks searches the functions in a Reproducing Kernel Hilbert Space. We prove that the proposed estimator is consistent. We carry out simulations and show that when the datasets are small the proposed estimator is more reliable than the MINE estimator and when the datasets are large the performance of the two methods are close.
Recurrent Convolutional Strategies for Face Manipulation Detection in Videos
Authors: Ekraam Sabir, Jiaxin Cheng, Ayush Jaiswal, Wael Abd-Almageed, Iacopo Masi, Prem Natarajan
Date: 2019-05-02 00:00:00
Tasks: Face Swapping, Misinformation

The spread of misinformation through synthetically generated yet realistic images and videos has become a significant problem, calling for robust manipulation detection methods. Despite the predominant effort of detecting face manipulation in still images, less attention has been paid to the identification of tampered faces in videos by taking advantage of the temporal information present in the stream. Recurrent convolutional models are a class of deep learning models which have proven effective at exploiting the temporal information from image streams across domains. We thereby distill the best strategy for combining variations in these models along with domain specific face preprocessing techniques through extensive experimentation to obtain state-of-the-art performance on publicly available video-based facial manipulation benchmarks. Specifically, we attempt to detect Deepfake, Face2Face and FaceSwap tampered faces in video streams. Evaluation is performed on the recently introduced FaceForensics++ dataset, improving the previous state-of-the-art by up to 4.55% in accuracy.
Herding Effect based Attention for Personalized Time-Sync Video Recommendation
Authors: Wenmian Yang, Wenyuan Gao, Xiaojie Zhou, Weijia Jia, Shaohua Zhang, Yutao Luo
Date: 2019-05-02 00:00:00
Tasks: Collaborative Filtering

Time-sync comment (TSC) is a new form of user-interaction review associated
with real-time video contents, which contains a user's preferences for videos
and therefore well suited as the data source for video recommendations.
However, existing review-based recommendation methods ignore the
context-dependent (generated by user-interaction), real-time, and
time-sensitive properties of TSC data. To bridge the above gaps, in this paper,
we use video images and users' TSCs to design an Image-Text Fusion model with a
novel Herding Effect Attention mechanism (called ITF-HEA), which can predict
users' favorite videos with model-based collaborative filtering. Specifically,
in the HEA mechanism, we weight the context information based on the semantic
similarities and time intervals between each TSC and its context, thereby
considering influences of the herding effect in the model. Experiments show
that ITF-HEA is on average 3.78\% higher than the state-of-the-art method upon
F1-score in baselines.
Argument Identification in Public Comments from eRulemaking
Authors: Vlad Eidelman, Brian Grom
Date: 2019-05-02 00:00:00
Tasks: Diversity, Sentence

Administrative agencies in the United States receive millions of comments each year concerning proposed agency actions during the eRulemaking process. These comments represent a diversity of arguments in support and opposition of the proposals. While agencies are required to identify and respond to substantive comments, they have struggled to keep pace with the volume of information. In this work we address the tasks of identifying argumentative text, classifying the type of argument claims employed, and determining the stance of the comment. First, we propose a taxonomy of argument claims based on an analysis of thousands of rules and millions of comments. Second, we collect and semi-automatically bootstrap annotations to create a dataset of millions of sentences with argument claim type annotation at the sentence level. Third, we build a system for automatically determining argumentative spans and claim type using our proposed taxonomy in a hierarchical classification model.
Group Retention when Using Machine Learning in Sequential Decision Making: the Interplay between User Dynamics and Fairness
Authors: Xueru Zhang, Mohammad Mahdi Khalili, Cem Tekin, Mingyan Liu
Date: 2019-05-02 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Decision Making, Fairness, Sequential Decision Making

Machine Learning (ML) models trained on data from multiple demographic groups can inherit representation disparity (Hashimoto et al., 2018) that may exist in the data: the model may be less favorable to groups contributing less to the training process; this in turn can degrade population retention in these groups over time, and exacerbate representation disparity in the long run. In this study, we seek to understand the interplay between ML decisions and the underlying group representation, how they evolve in a sequential framework, and how the use of fairness criteria plays a role in this process. We show that the representation disparity can easily worsen over time under a natural user dynamics (arrival and departure) model when decisions are made based on a commonly used objective and fairness criteria, resulting in some groups diminishing entirely from the sample pool in the long run. It highlights the fact that fairness criteria have to be defined while taking into consideration the impact of decisions on user dynamics. Toward this end, we explain how a proper fairness criterion can be selected based on a general user dynamics model.
Weight Map Layer for Noise and Adversarial Attack Robustness
Authors: Mohammed Amer, Tomás Maul
Date: 2019-05-02 00:00:00
Tasks: Adversarial Attack

Convolutional neural networks (CNNs) are known for their good performance and generalization in vision-related tasks and have become state-of-the-art in both application and research-based domains. However, just like other neural network models, they suffer from a susceptibility to noise and adversarial attacks. An adversarial defence aims at reducing a neural network's susceptibility to adversarial attacks through learning or architectural modifications. We propose the weight map layer (WM) as a generic architectural addition to CNNs and show that it can increase their robustness to noise and adversarial attacks. We further explain that the enhanced robustness of the two WM variants results from the adaptive activation-variance amplification exhibited by the layer. We show that the WM layer can be integrated into scaled up models to increase their noise and adversarial attack robustness, while achieving comparable accuracy levels across different datasets.
Investigating Robustness and Interpretability of Link Prediction via Adversarial Modifications
Authors: Pouya Pezeshkpour, Yifan Tian, Sameer Singh
Date: 2019-05-02 00:00:00
Proceeding: NAACL 2019 6
Tasks: Link Prediction, Sensitivity

Representing entities and relations in an embedding space is a well-studied
approach for machine learning on relational data. Existing approaches, however,
primarily focus on improving accuracy and overlook other aspects such as
robustness and interpretability. In this paper, we propose adversarial
modifications for link prediction models: identifying the fact to add into or
remove from the knowledge graph that changes the prediction for a target fact
after the model is retrained. Using these single modifications of the graph, we
identify the most influential fact for a predicted link and evaluate the
sensitivity of the model to the addition of fake facts. We introduce an
efficient approach to estimate the effect of such modifications by
approximating the change in the embeddings when the knowledge graph changes. To
avoid the combinatorial search over all possible facts, we train a network to
decode embeddings to their corresponding graph components, allowing the use of
gradient-based optimization to identify the adversarial modification. We use
these techniques to evaluate the robustness of link prediction models (by
measuring sensitivity to additional facts), study interpretability through the
facts most responsible for predictions (by identifying the most influential
neighbors), and detect incorrect facts in the knowledge base.
Large-scale weakly-supervised pre-training for video action recognition
Authors: Deepti Ghadiyaram, Matt Feiszli, Du Tran, Xueting Yan, Heng Wang, Dhruv Mahajan
Date: 2019-05-02 00:00:00
Proceeding: CVPR 2019 6
Tasks: Action Classification, Action Recognition, Activity Recognition, Activity Recognition In Videos, Egocentric Activity Recognition, Temporal Action Localization, Transfer Learning

Current fully-supervised video datasets consist of only a few hundred
thousand videos and fewer than a thousand domain-specific labels. This hinders
the progress towards advanced video architectures. This paper presents an
in-depth study of using large volumes of web videos for pre-training video
models for the task of action recognition. Our primary empirical finding is
that pre-training at a very large scale (over 65 million videos), despite on
noisy social-media videos and hashtags, substantially improves the
state-of-the-art on three challenging public action recognition datasets.
Further, we examine three questions in the construction of weakly-supervised
video action datasets. First, given that actions involve interactions with
objects, how should one construct a verb-object pre-training label space to
benefit transfer learning the most? Second, frame-based models perform quite
well on action recognition; is pre-training for good image features sufficient
or is pre-training for spatio-temporal features valuable for optimal transfer
learning? Finally, actions are generally less well-localized in long videos vs.
short videos; since action labels are provided at a video level, how should one
choose video clips for best performance, given some fixed budget of number or
minutes of videos?
Adaptive Intelligent Secondary Control of Microgrids Using a Biologically-Inspired Reinforcement Learning
Authors: Mohammad Jafari, Vahid Sarfi, Amir Ghasemkhani, Hanif Livani, Lei Yang, Hao Xu
Date: 2019-05-02 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, a biologically-inspired adaptive intelligent secondary
controller is developed for microgrids to tackle system dynamics uncertainties,
faults, and/or disturbances. The developed adaptive biologically-inspired
controller adopts a novel computational model of emotional learning in
mammalian limbic system. The learning capability of the proposed
biologically-inspired intelligent controller makes it a promising approach to
deal with the power system non-linear and volatile dynamics without increasing
the controller complexity, and maintain the voltage and frequency stabilities
by using an efficient reference tracking mechanism. The performance of the
proposed intelligent secondary controller is validated in terms of the voltage
and frequency absolute errors in the simulated microgrid. Simulation results
highlight the efficiency and robustness of the proposed intelligent controller
under the fault conditions and different system uncertainties compared to other
benchmark controllers.
Billion-scale semi-supervised learning for image classification
Authors: I. Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, Dhruv Mahajan
Date: 2019-05-02 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Object Recognition, Video Classification

This paper presents a study of semi-supervised learning with large
convolutional networks. We propose a pipeline, based on a teacher/student
paradigm, that leverages a large collection of unlabelled images (up to 1
billion). Our main goal is to improve the performance for a given target
architecture, like ResNet-50 or ResNext. We provide an extensive analysis of
the success factors of our approach, which leads us to formulate some
recommendations to produce high-accuracy models for image classification with
semi-supervised learning. As a result, our approach brings important gains to
standard architectures for image, video and fine-grained classification. For
instance, by leveraging one billion unlabelled images, our learned vanilla
ResNet-50 achieves 81.2% top-1 accuracy on the ImageNet benchmark.
DPSNet: End-to-end Deep Plane Sweep Stereo
Authors: Sunghoon Im, Hae-Gon Jeon, Stephen Lin, In So Kweon
Date: 2019-05-02 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Learning, Optical Flow Estimation

Multiview stereo aims to reconstruct scene depth from images acquired by a
camera under arbitrary motion. Recent methods address this problem through deep
learning, which can utilize semantic cues to deal with challenges such as
textureless and reflective regions. In this paper, we present a convolutional
neural network called DPSNet (Deep Plane Sweep Network) whose design is
inspired by best practices of traditional geometry-based approaches for dense
depth reconstruction. Rather than directly estimating depth and/or optical flow
correspondence from image pairs as done in many previous deep learning methods,
DPSNet takes a plane sweep approach that involves building a cost volume from
deep features using the plane sweep algorithm, regularizing the cost volume via
a context-aware cost aggregation, and regressing the dense depth map from the
cost volume. The cost volume is constructed using a differentiable warping
process that allows for end-to-end training of the network. Through the
effective incorporation of conventional multiview stereo concepts within a deep
learning framework, DPSNet achieves state-of-the-art reconstruction results on
a variety of challenging datasets.
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Authors: Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, Samuel R. Bowman
Date: 2019-05-02 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Transfer Learning

In the last year, new models and methods for pretraining and transfer learning have driven striking performance improvements across a range of language understanding tasks. The GLUE benchmark, introduced a little over one year ago, offers a single-number metric that summarizes progress on a diverse set of such tasks, but performance on the benchmark has recently surpassed the level of non-expert humans, suggesting limited headroom for further research. In this paper we present SuperGLUE, a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, a software toolkit, and a public leaderboard. SuperGLUE is available at super.gluebenchmark.com.
Drug-Drug Adverse Effect Prediction with Graph Co-Attention
Authors: Andreea Deac, Yu-Hsiang Huang, Petar Veličković, Pietro Liò, Jian Tang
Date: 2019-05-02 00:00:00
Tasks: Drug–drug Interaction Extraction

Complex or co-existing diseases are commonly treated using drug combinations,
which can lead to higher risk of adverse side effects. The detection of
polypharmacy side effects is usually done in Phase IV clinical trials, but
there are still plenty which remain undiscovered when the drugs are put on the
market. Such accidents have been affecting an increasing proportion of the
population (15% in the US now) and it is thus of high interest to be able to
predict the potential side effects as early as possible. Systematic
combinatorial screening of possible drug-drug interactions (DDI) is challenging
and expensive. However, the recent significant increases in data availability
from pharmaceutical research and development efforts offer a novel paradigm for
recovering relevant insights for DDI prediction. Accordingly, several recent
approaches focus on curating massive DDI datasets (with millions of examples)
and training machine learning models on them. Here we propose a neural network
architecture able to set state-of-the-art results on this task---using the type
of the side-effect and the molecular structure of the drugs alone---by
leveraging a co-attentional mechanism. In particular, we show the importance of
integrating joint information from the drug pairs early on when learning each
drug's representation.
Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization
Authors: Rong Ge, Zhize Li, Wei-Yao Wang, Xiang Wang
Date: 2019-05-01 00:00:00

Variance reduction techniques like SVRG provide simple and fast algorithms
for optimizing a convex finite-sum objective. For nonconvex objectives, these
techniques can also find a first-order stationary point (with small gradient).
However, in nonconvex optimization it is often crucial to find a second-order
stationary point (with small gradient and almost PSD hessian). In this paper,
we show that Stabilized SVRG (a simple variant of SVRG) can find an
$\epsilon$-second-order stationary point using only
$\widetilde{O}(n^{2/3}/\epsilon^2+n/\epsilon^{1.5})$ stochastic gradients. To
our best knowledge, this is the first second-order guarantee for a simple
variant of SVRG. The running time almost matches the known guarantees for
finding $\epsilon$-first-order stationary points.
RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehicles
Authors: Ramin Nabati, Hairong Qi
Date: 2019-05-01 00:00:00
Tasks: Autonomous Driving, Autonomous Vehicles, Object, object-detection, Object Detection, Region Proposal

Region proposal algorithms play an important role in most state-of-the-art two-stage object detection networks by hypothesizing object locations in the image. Nonetheless, region proposal algorithms are known to be the bottleneck in most two-stage object detection networks, increasing the processing time for each image and resulting in slow networks not suitable for real-time applications such as autonomous driving vehicles. In this paper we introduce RRPN, a Radar-based real-time region proposal algorithm for object detection in autonomous driving vehicles. RRPN generates object proposals by mapping Radar detections to the image coordinate system and generating pre-defined anchor boxes for each mapped Radar detection point. These anchor boxes are then transformed and scaled based on the object's distance from the vehicle, to provide more accurate proposals for the detected objects. We evaluate our method on the newly released NuScenes dataset [1] using the Fast R-CNN object detection network [2]. Compared to the Selective Search object proposal algorithm [3], our model operates more than 100x faster while at the same time achieves higher detection precision and recall. Code has been made publicly available at https://github.com/mrnabati/RRPN .
3D BAT: A Semi-Automatic, Web-based 3D Annotation Toolbox for Full-Surround, Multi-Modal Data Streams
Authors: Walter Zimmer, Akshay Rangesh, Mohan Trivedi
Date: 2019-05-01 00:00:00
Tasks: Motion Planning, motion prediction

In this paper, we focus on obtaining 2D and 3D labels, as well as track IDs
for objects on the road with the help of a novel 3D Bounding Box Annotation
Toolbox (3D BAT). Our open source, web-based 3D BAT incorporates several smart
features to improve usability and efficiency. For instance, this annotation
toolbox supports semi-automatic labeling of tracks using interpolation, which
is vital for downstream tasks like tracking, motion planning and motion
prediction. Moreover, annotations for all camera images are automatically
obtained by projecting annotations from 3D space into the image domain. In
addition to the raw image and point cloud feeds, a Masterview consisting of the
top view (bird's-eye-view), side view and front views is made available to
observe objects of interest from different perspectives. Comparisons of our
method with other publicly available annotation tools reveal that 3D
annotations can be obtained faster and more efficiently by using our toolbox.
The Literary Theme Ontology for Media Annotation and Information Retrieval
Authors: Paul Sheridan, Mikael Onsjö, Janna Hastings
Date: 2019-05-01 00:00:00
Tasks: Information Retrieval, Retrieval

Literary theme identification and interpretation is a focal point of literary studies scholarship. Classical forms of literary scholarship, such as close reading, have flourished with scarcely any need for commonly defined literary themes. However, the rise in popularity of collaborative and algorithmic analyses of literary themes in works of fiction, together with a requirement for computational searching and indexing facilities for large corpora, creates the need for a collection of shared literary themes to ensure common terminology and definitions. To address this need, we here introduce a first draft of the Literary Theme Ontology. Inspired by a traditional framing from literary theory, the ontology comprises literary themes drawn from the authors own analyses, reference books, and online sources. The ontology is available at https://github.com/theme-ontology/lto under a Creative Commons Attribution 4.0 International license (CC BY 4.0).