pwc-archive/papers-with-abstracts huggingface dataset:
Efficient Video Understanding via Layered Multi Frame-Rate Analysis
Authors: Ziyao Tang, Yongxi Lu, Tara Javidi
Date: 2018-11-24 00:00:00
Tasks: Autonomous Driving, Video Understanding

One of the greatest challenges in the design of a real-time perception system
for autonomous driving vehicles and drones is the conflicting requirement of
safety (high prediction accuracy) and efficiency. Traditional approaches use a
single frame rate for the entire system. Motivated by the observation that the
lack of robustness against environmental factors is the major weakness of
compact ConvNet architectures, we propose a dual frame-rate system that brings
in the best of both worlds: A modulator stream that executes an expensive
models robust to environmental factors at a low frame rate to extract slowly
changing features describing the environment, and a prediction stream that
executes a light-weight model at real-time to extract transient signals that
describes particularities of the current frame. The advantage of our design is
validated by our extensive empirical study, showing that our solution leads to
consistent improvements using a variety of backbone architecture choice and
input resolutions. These findings suggest multiple frame-rate systems as a
promising direction in designing efficient perception for autonomous agents.
Attention, Please! Adversarial Defense via Activation Rectification and Preservation
Authors: Shangxi Wu, Jitao Sang, Kaiyuan Xu, Jiaming Zhang, Jian Yu
Date: 2018-11-24 00:00:00
Tasks: Adversarial Attack, Adversarial Defense

This study provides a new understanding of the adversarial attack problem by examining the correlation between adversarial attack and visual attention change. In particular, we observed that: (1) images with incomplete attention regions are more vulnerable to adversarial attacks; and (2) successful adversarial attacks lead to deviated and scattered attention map. Accordingly, an attention-based adversarial defense framework is designed to simultaneously rectify the attention map for prediction and preserve the attention area between adversarial and original images. The problem of adding iteratively attacked samples is also discussed in the context of visual attention change. We hope the attention-related data analysis and defense solution in this study will shed some light on the mechanism behind the adversarial attack and also facilitate future adversarial defense/attack model design.
Evolutionary-Neural Hybrid Agents for Architecture Search
Authors: Krzysztof Maziarz, Mingxing Tan, Andrey Khorlin, Marin Georgiev, Andrea Gesmundo
Date: 2018-11-24 00:00:00
Tasks: Deep Reinforcement Learning, Evolutionary Algorithms, General Classification, image-classification, Image Classification, Neural Architecture Search, Reinforcement Learning, Text Classification

Neural Architecture Search has shown potential to automate the design of neural networks. Deep Reinforcement Learning based agents can learn complex architectural patterns, as well as explore a vast and compositional search space. On the other hand, evolutionary algorithms offer higher sample efficiency, which is critical for such a resource intensive application. In order to capture the best of both worlds, we propose a class of Evolutionary-Neural hybrid agents (Evo-NAS). We show that the Evo-NAS agent outperforms both neural and evolutionary agents when applied to architecture search for a suite of text and image classification benchmarks. On a high-complexity architecture search space for image classification, the Evo-NAS agent surpasses the accuracy achieved by commonly used agents with only 1/3 of the search cost.
Object Detection based Deep Unsupervised Hashing
Authors: Rong-Cheng Tu, Xian-Ling Mao, Bo-Si Feng, Bing-Bing Bian, Yu-shu Ying
Date: 2018-11-24 00:00:00
Tasks: Image Retrieval, Novel Object Detection, Object, object-detection, Object Detection, Retrieval

Recently, similarity-preserving hashing methods have been extensively studied
for large-scale image retrieval. Compared with unsupervised hashing, supervised
hashing methods for labeled data have usually better performance by utilizing
semantic label information. Intuitively, for unlabeled data, it will improve
the performance of unsupervised hashing methods if we can first mine some
supervised semantic 'label information' from unlabeled data and then
incorporate the 'label information' into the training process. Thus, in this
paper, we propose a novel Object Detection based Deep Unsupervised Hashing
method (ODDUH). Specifically, a pre-trained object detection model is utilized
to mining supervised 'label information', which is used to guide the learning
process to generate high-quality hash codes.Extensive experiments on two public
datasets demonstrate that the proposed method outperforms the state-of-the-art
unsupervised hashing methods in the image retrieval task.
Streamlining Variational Inference for Constraint Satisfaction Problems
Authors: Aditya Grover, Tudor Achim, Stefano Ermon
Date: 2018-11-24 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Survey, Variational Inference

Several algorithms for solving constraint satisfaction problems are based on
survey propagation, a variational inference scheme used to obtain approximate
marginal probability estimates for variable assignments. These marginals
correspond to how frequently each variable is set to true among satisfying
assignments, and are used to inform branching decisions during search; however,
marginal estimates obtained via survey propagation are approximate and can be
self-contradictory. We introduce a more general branching strategy based on
streamlining constraints, which sidestep hard assignments to variables. We show
that streamlined solvers consistently outperform decimation-based solvers on
random k-SAT instances for several problem sizes, shrinking the gap between
empirical performance and theoretical limits of satisfiability by 16.3% on
average for k=3,4,5,6.
Polar Decoding on Sparse Graphs with Deep Learning
Authors: Weihong Xu, Xiaohu You, Chuan Zhang, Yair Be'ery
Date: 2018-11-24 00:00:00
Tasks: Decoder, Deep Learning, Single Particle Analysis

In this paper, we present a sparse neural network decoder (SNND) of polar
codes based on belief propagation (BP) and deep learning. At first, the
conventional factor graph of polar BP decoding is converted to the bipartite
Tanner graph similar to low-density parity-check (LDPC) codes. Then the Tanner
graph is unfolded and translated into the graphical representation of deep
neural network (DNN). The complex sum-product algorithm (SPA) is modified to
min-sum (MS) approximation with low complexity. We dramatically reduce the
number of weight by using single weight to parameterize the networks. Optimized
by the training techniques of deep learning, proposed SNND achieves comparative
decoding performance of SPA and obtains about $0.5$ dB gain over MS decoding on
($128,64$) and ($256,128$) codes. Moreover, $60 \%$ complexity reduction is
achieved and the decoding latency is significantly lower than the conventional
polar BP.
Bayesian QuickNAT: Model Uncertainty in Deep Whole-Brain Segmentation for Structure-wise Quality Control
Authors: Abhijit Guha Roy, Sailesh Conjeti, Nassir Navab, Christian Wachinger
Date: 2018-11-24 00:00:00
Tasks: Brain Segmentation, Segmentation

We introduce Bayesian QuickNAT for the automated quality control of
whole-brain segmentation on MRI T1 scans. Next to the Bayesian fully
convolutional neural network, we also present inherent measures of segmentation
uncertainty that allow for quality control per brain structure. For estimating
model uncertainty, we follow a Bayesian approach, wherein, Monte Carlo (MC)
samples from the posterior distribution are generated by keeping the dropout
layers active at test time. Entropy over the MC samples provides a voxel-wise
model uncertainty map, whereas expectation over the MC predictions provides the
final segmentation. Next to voxel-wise uncertainty, we introduce four metrics
to quantify structure-wise uncertainty in segmentation for quality control. We
report experiments on four out-of-sample datasets comprising of diverse age
range, pathology and imaging artifacts. The proposed structure-wise uncertainty
metrics are highly correlated with the Dice score estimated with manual
annotation and therefore present an inherent measure of segmentation quality.
In particular, the intersection over union over all the MC samples is a
suitable proxy for the Dice score. In addition to quality control at
scan-level, we propose to incorporate the structure-wise uncertainty as a
measure of confidence to do reliable group analysis on large data repositories.
We envisage that the introduced uncertainty metrics would help assess the
fidelity of automated deep learning based segmentation methods for large-scale
population studies, as they enable automated quality control and group analyses
in processing large data repositories.
A Novel Technique for Evidence based Conditional Inference in Deep Neural Networks via Latent Feature Perturbation
Authors: Dinesh Khandelwal, Suyash Agrawal, Parag Singla, Chetan Arora
Date: 2018-11-24 00:00:00
Tasks: Image Captioning, Instance Segmentation, Multi-Task Learning, Semantic Segmentation, Video Summarization

Auxiliary information can be exploited in machine learning models using the paradigm of evidence based conditional inference. Multi-modal techniques in Deep Neural Networks (DNNs) can be seen as perturbing the latent feature representation for incorporating evidence from the auxiliary modality. However, they require training a specialized network which can map sparse evidence to a high dimensional latent space vector. Designing such a network, as well as collecting jointly labeled data for training is a non-trivial task. In this paper, we present a novel multi-task learning (MTL) based framework to perform evidence based conditional inference in DNNs which can overcome both these shortcomings. Our framework incorporates evidence as the output of secondary task(s), while modeling the original problem as the primary task of interest. During inference, we employ a novel Bayesian formulation to change the joint latent feature representation so as to maximize the probability of the observed evidence. Since our approach models evidence as prediction from a DNN, this can often be achieved using standard pre-trained backbones for popular tasks, eliminating the need for training altogether. Even when training is required, our MTL architecture ensures the same can be done without any need for jointly labeled data. Exploiting evidence using our framework, we show an improvement of 3.9% over the state-of-the-art, for predicting semantic segmentation given the image tags, and 2.8% for predicting instance segmentation given image captions.
Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles
Authors: Dahun Kim, Donghyeon Cho, In So Kweon
Date: 2018-11-24 00:00:00
Tasks: Action Recognition, Colorization, Representation Learning, Self-Supervised Action Recognition, Temporal Action Localization

Self-supervised tasks such as colorization, inpainting and zigsaw puzzle have
been utilized for visual representation learning for still images, when the
number of labeled images is limited or absent at all. Recently, this worthwhile
stream of study extends to video domain where the cost of human labeling is
even more expensive. However, the most of existing methods are still based on
2D CNN architectures that can not directly capture spatio-temporal information
for video applications. In this paper, we introduce a new self-supervised task
called as \textit{Space-Time Cubic Puzzles} to train 3D CNNs using large scale
video dataset. This task requires a network to arrange permuted 3D
spatio-temporal crops. By completing \textit{Space-Time Cubic Puzzles}, the
network learns both spatial appearance and temporal relation of video frames,
which is our final goal. In experiments, we demonstrate that our learned 3D
representation is well transferred to action recognition tasks, and outperforms
state-of-the-art 2D CNN-based competitors on UCF101 and HMDB51 datasets.
Three-Dimensionally Embedded Graph Convolutional Network (3DGCN) for Molecule Interpretation
Authors: Hyeoncheol Cho, Insung S. Choi
Date: 2018-11-24 00:00:00
Tasks: Deep Learning, Molecule Interpretation

We present a three-dimensional graph convolutional network (3DGCN), which
predicts molecular properties and biochemical activities, based on 3D molecular
graph. In the 3DGCN, graph convolution is unified with learning operations on
the vector to handle the spatial information from molecular topology. The 3DGCN
model exhibits significantly higher performance on various tasks compared with
other deep-learning models, and has the ability of generalizing a given
conformer to targeted features regardless of its rotations in the 3D space.
More significantly, our model also can distinguish the 3D rotations of a
molecule and predict the target value, depending upon the rotation degree, in
the protein-ligand docking problem, when trained with orientation-dependent
datasets. The rotation distinguishability of 3DGCN, along with rotation
equivariance, provides a key milestone in the implementation of
three-dimensionality to the field of deep-learning chemistry that solves
challenging biochemical problems.
Discriminative Feature Learning for Unsupervised Video Summarization
Authors: Yunjae Jung, Donghyeon Cho, Dahun Kim, Sanghyun Woo, In So Kweon
Date: 2018-11-24 00:00:00
Tasks: Supervised Video Summarization, Unsupervised Video Summarization, Video Summarization

In this paper, we address the problem of unsupervised video summarization
that automatically extracts key-shots from an input video. Specifically, we
tackle two critical issues based on our empirical observations: (i) Ineffective
feature learning due to flat distributions of output importance scores for each
frame, and (ii) training difficulty when dealing with long-length video inputs.
To alleviate the first problem, we propose a simple yet effective
regularization loss term called variance loss. The proposed variance loss
allows a network to predict output scores for each frame with high discrepancy
which enables effective feature learning and significantly improves model
performance. For the second problem, we design a novel two-stream network named
Chunk and Stride Network (CSNet) that utilizes local (chunk) and global
(stride) temporal view on the video features. Our CSNet gives better
summarization results for long-length videos compared to the existing methods.
In addition, we introduce an attention mechanism to handle the dynamic
information in videos. We demonstrate the effectiveness of the proposed methods
by conducting extensive ablation studies and show that our final model achieves
new state-of-the-art results on two benchmark datasets.
Spatio-Temporal Road Scene Reconstruction using Superpixel Markov Random Field
Authors: Yaochen Li, Yuehu Liu, Jihua Zhu, Shiqi Ma, Zhenning Niu, Rui Guo
Date: 2018-11-24 00:00:00
Tasks: Superpixels

Scene model construction based on image rendering is an indispensable but challenging technique in computer vision and intelligent transportation systems. In this paper, we propose a framework for constructing 3D corridor-based road scene models. This consists of two successive stages: road detection and scene construction. The road detection is realized by a new superpixel Markov random field (MRF) algorithm. The data fidelity term in the MRF's energy function is jointly computed according to the superpixel features of color, texture and location. The smoothness term is established on the basis of the interaction of spatio-temporally adjacent superpixels. In the subsequent scene construction, the foreground and background regions are modeled independently. Experiments for road detection demonstrate the proposed method outperforms the state-of-the-art in both accuracy and speed. The scene construction experiments confirm that the proposed scene models show better correctness ratios, and have the potential to support a range of applications.
Senti-Attend: Image Captioning using Sentiment and Attention
Authors: Omid Mohamad Nezami, Mark Dras, Stephen Wan, Cecile Paris
Date: 2018-11-24 00:00:00
Tasks: Image Captioning

There has been much recent work on image captioning models that describe the
factual aspects of an image. Recently, some models have incorporated
non-factual aspects into the captions, such as sentiment or style. However,
such models typically have difficulty in balancing the semantic aspects of the
image and the non-factual dimensions of the caption; in addition, it can be
observed that humans may focus on different aspects of an image depending on
the chosen sentiment or style of the caption. To address this, we design an
attention-based model to better add sentiment to image captions. The model
embeds and learns sentiment with respect to image-caption data, and uses both
high-level and word-level sentiment information during the learning process.
The model outperforms the state-of-the-art work in image captioning with
sentiment using standard evaluation metrics. An analysis of generated captions
also shows that our model does this by a better selection of the
sentiment-bearing adjectives and adjective-noun pairs.
Recurrently Controlled Recurrent Networks
Authors: Yi Tay, Luu Anh Tuan, Siu Cheung Hui
Date: 2018-11-24 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Answer Selection, General Classification, Reading Comprehension, Sentiment Analysis

Recurrent neural networks (RNNs) such as long short-term memory and gated
recurrent units are pivotal building blocks across a broad spectrum of sequence
modeling problems. This paper proposes a recurrently controlled recurrent
network (RCRN) for expressive and powerful sequence encoding. More concretely,
the key idea behind our approach is to learn the recurrent gating functions
using recurrent networks. Our architecture is split into two components - a
controller cell and a listener cell whereby the recurrent controller actively
influences the compositionality of the listener cell. We conduct extensive
experiments on a myriad of tasks in the NLP domain such as sentiment analysis
(SST, IMDb, Amazon reviews, etc.), question classification (TREC), entailment
classification (SNLI, SciTail), answer selection (WikiQA, TrecQA) and reading
comprehension (NarrativeQA). Across all 26 datasets, our results demonstrate
that RCRN not only consistently outperforms BiLSTMs but also stacked BiLSTMs,
suggesting that our controller architecture might be a suitable replacement for
the widely adopted stacked architecture.
Strategy of the Negative Sampling for Training Retrieval-Based Dialogue Systems
Authors: Aigul Nugmanova, Andrei Smirnov, Galina Lavrentyeva, Irina Chernykh
Date: 2018-11-24 00:00:00
Tasks: Retrieval

The article describes the new approach for quality improvement of automated
dialogue systems for customer support service. Analysis produced in the paper
demonstrates the dependency of the quality of the retrieval-based dialogue
system quality on the choice of negative responses. The proposed approach
implies choosing the negative samples according to the distribution of
responses in the train set. In this implementation the negative samples are
randomly chosen from the original response distribution and from the
"artificial" distribution of negative responses, such as uniform distribution
or the distribution obtained by transformation of the original one. The results
obtained for the implemented systems and reported in this paper confirm the
significant improvement of automated dialogue systems quality in case of using
the negative responses from transformed distribution.
What and Where: A Context-based Recommendation System for Object Insertion
Authors: Song-Hai Zhang, Zhengping Zhou, Bin Liu, Xin Dong, Dun Liang, Peter Hall, Shi-Min Hu
Date: 2018-11-24 00:00:00
Tasks: Object

In this work, we propose a novel topic consisting of two dual tasks: 1) given
a scene, recommend objects to insert, 2) given an object category, retrieve
suitable background scenes. A bounding box for the inserted object is predicted
in both tasks, which helps downstream applications such as semi-automated
advertising and video composition. The major challenge lies in the fact that
the target object is neither present nor localized at test time, whereas
available datasets only provide scenes with existing objects. To tackle this
problem, we build an unsupervised algorithm based on object-level contexts,
which explicitly models the joint probability distribution of object categories
and bounding boxes with a Gaussian mixture model. Experiments on our newly
annotated test set demonstrate that our system outperforms existing baselines
on all subtasks, and do so under a unified framework. Our contribution promises
future extensions and applications.
Alternating Loss Correction for Preterm-Birth Prediction from EHR Data with Noisy Labels
Authors: Sabri Boughorbel, Fethi Jarray, Neethu Venugopal, Haithum Elhadi
Date: 2018-11-24 00:00:00
Tasks: De-identification

In this paper we are interested in the prediction of preterm birth based on
diagnosis codes from longitudinal EHR. We formulate the prediction problem as a
supervised classification with noisy labels. Our base classifier is a Recurrent
Neural Network with an attention mechanism. We assume the availability of a
data subset with both noisy and clean labels. For the cohort definition, most
of the diagnosis codes on mothers' records related to pregnancy are ambiguous
for the definition of full-term and preterm classes. On the other hand,
diagnosis codes on babies' records provide fine-grained information on
prematurity. Due to data de-identification, the links between mothers and
babies are not available. We developed a heuristic based on admission and
discharge times to match babies to their mothers and hence enrich mothers'
records with additional information on delivery status. The obtained additional
dataset from the matching heuristic has noisy labels and was used to leverage
the training of the deep learning model. We propose an Alternating Loss
Correction (ALC) method to train deep models with both clean and noisy labels.
First, the label corruption matrix is estimated using the data subset with both
noisy and clean labels. Then it is used in the model as a dense output layer to
correct for the label noise. The network is alternately trained on epochs with
the clean dataset with a simple cross-entropy loss and on next epoch with the
noisy dataset and a loss corrected with the estimated corruption matrix. The
experiments for the prediction of preterm birth at 90 days before delivery
showed an improvement in performance compared with baseline and state
of-the-art methods.
A^2Net: Adjacent Aggregation Networks for Image Raindrop Removal
Authors: Huangxing Lin, Xueyang Fu, Changxing Jing, Xinghao Ding, Yue Huang
Date: 2018-11-24 00:00:00
Tasks: Raindrop Removal, Rain Removal

Existing methods for single images raindrop removal either have poor
robustness or suffer from parameter burdens. In this paper, we propose a new
Adjacent Aggregation Network (A^2Net) with lightweight architectures to remove
raindrops from single images. Instead of directly cascading convolutional
layers, we design an adjacent aggregation architecture to better fuse features
for rich representations generation, which can lead to high quality images
reconstruction. To further simplify the learning process, we utilize a
problem-specific knowledge to force the network focus on the luminance channel
in the YUV color space instead of all RGB channels. By combining adjacent
aggregating operation with color space transformation, the proposed A^2Net can
achieve state-of-the-art performances on raindrop removal with significant
parameters reduction.