pwc-archive/papers-with-abstracts huggingface dataset:
Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition
Authors: Yu Pan, Jing Xu, Maolin Wang, Jinmian Ye, Fei Wang, Kun Bai, Zenglin Xu
Date: 2018-11-19 00:00:00
Proceeding: NIPS Workshop CDNNRIA 2018
Tasks: Action Recognition, Temporal Action Localization, Tensor Decomposition

Recurrent Neural Networks (RNNs) and their variants, such as Long-Short Term
Memory (LSTM) networks, and Gated Recurrent Unit (GRU) networks, have achieved
promising performance in sequential data modeling. The hidden layers in RNNs
can be regarded as the memory units, which are helpful in storing information
in sequential contexts. However, when dealing with high dimensional input data,
such as video and text, the input-to-hidden linear transformation in RNNs
brings high memory usage and huge computational cost. This makes the training
of RNNs unscalable and difficult. To address this challenge, we propose a novel
compact LSTM model, named as TR-LSTM, by utilizing the low-rank tensor ring
decomposition (TRD) to reformulate the input-to-hidden transformation. Compared
with other tensor decomposition methods, TR-LSTM is more stable. In addition,
TR-LSTM can complete an end-to-end training and also provide a fundamental
building block for RNNs in handling large input data. Experiments on real-world
action recognition datasets have demonstrated the promising performance of the
proposed TR-LSTM compared with the tensor train LSTM and other state-of-the-art
competitors.
Fast Efficient Object Detection Using Selective Attention
Authors: Shivanthan Yohanandan, Andy Song, Adrian G. Dyer, Angela Faragasso, Subhrajit Roy, DaCheng Tao
Date: 2018-11-19 00:00:00
Tasks: Object, object-detection, Object Detection

Retraction due to significant oversight
Robust Visual Tracking using Multi-Frame Multi-Feature Joint Modeling
Authors: Peng Zhang, Shujian Yu, Jiamiao Xu, Xinge You, Xiubao Jiang, Xiao-Yuan Jing, DaCheng Tao
Date: 2018-11-19 00:00:00
Tasks: Multi-Task Learning, MULTI-VIEW LEARNING, Object Tracking, Visual Object Tracking, Visual Tracking

It remains a huge challenge to design effective and efficient trackers under
complex scenarios, including occlusions, illumination changes and pose
variations. To cope with this problem, a promising solution is to integrate the
temporal consistency across consecutive frames and multiple feature cues in a
unified model. Motivated by this idea, we propose a novel correlation
filter-based tracker in this work, in which the temporal relatedness is
reconciled under a multi-task learning framework and the multiple feature cues
are modeled using a multi-view learning approach. We demonstrate the resulting
regression model can be efficiently learned by exploiting the structure of
blockwise diagonal matrix. A fast blockwise diagonal matrix inversion algorithm
is developed thereafter for efficient online tracking. Meanwhile, we
incorporate an adaptive scale estimation mechanism to strengthen the stability
of scale variation tracking. We implement our tracker using two types of
features and test it on two benchmark datasets. Experimental results
demonstrate the superiority of our proposed approach when compared with other
state-of-the-art trackers. project homepage
http://bmal.hust.edu.cn/project/KMF2JMTtracking.html
A Comparative Analysis of Content-based Geolocation in Blogs and Tweets
Authors: Konstantinos Pappas, Mahmoud Azab, Rada Mihalcea
Date: 2018-11-19 00:00:00

The geolocation of online information is an essential component in any
geospatial application. While most of the previous work on geolocation has
focused on Twitter, in this paper we quantify and compare the performance of
text-based geolocation methods on social media data drawn from both Blogger and
Twitter. We introduce a novel set of location specific features that are both
highly informative and easily interpretable, and show that we can achieve error
rate reductions of up to 12.5% with respect to the best previously proposed
geolocation features. We also show that despite posting longer text, Blogger
users are significantly harder to geolocate than Twitter users. Additionally,
we investigate the effect of training and testing on different media
(cross-media predictions), or combining multiple social media sources
(multi-media predictions). Finally, we explore the geolocability of social
media in relation to three user dimensions: state, gender, and industry.
FotonNet: A HW-Efficient Object Detection System Using 3D-Depth Segmentation and 2D-DNN Classifier
Authors: Gurjeet Singh, Sun Miao, Shi Shi, Patrick Chiang
Date: 2018-11-19 00:00:00
Tasks: Object, object-detection, Object Detection

Object detection and classification is one of the most important computer
vision problems. Ever since the introduction of deep learning
\cite{krizhevsky2012imagenet}, we have witnessed a dramatic increase in the
accuracy of this object detection problem. However, most of these improvements
have occurred using conventional 2D image processing. Recently, low-cost
3D-image sensors, such as the Microsoft Kinect (Time-of-Flight) or the Apple
FaceID (Structured-Light), can provide 3D-depth or point cloud data that can be
added to a convolutional neural network, acting as an extra set of dimensions.
In our proposed approach, we introduce a new 2D + 3D system that takes the
3D-data to determine the object region followed by any conventional 2D-DNN,
such as AlexNet. In this method, our approach can easily dissociate the
information collection from the Point Cloud and 2D-Image data and combine both
operations later. Hence, our system can use any existing trained 2D network on
a large image dataset, and does not require a large 3D-depth dataset for new
training. Experimental object detection results across 30 images show an
accuracy of 0.67, versus 0.54 and 0.51 for RCNN and YOLO, respectively.
DeepSeeNet: A deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs
Authors: Yifan Peng, Shazia Dharssi, Qingyu Chen, Tiarnan D. Keenan, Elvira Agrón, Wai T. Wong, Emily Y. Chew, Zhiyong Lu
Date: 2018-11-19 00:00:00
Tasks: Decision Making, General Classification

In assessing the severity of age-related macular degeneration (AMD), the
Age-Related Eye Disease Study (AREDS) Simplified Severity Scale predicts the
risk of progression to late AMD. However, its manual use requires the
time-consuming participation of expert practitioners. Although several
automated deep learning systems have been developed for classifying color
fundus photographs (CFP) of individual eyes by AREDS severity score, none to
date has used a patient-based scoring system that uses images from both eyes to
assign a severity score. DeepSeeNet, a deep learning model, was developed to
classify patients automatically by the AREDS Simplified Severity Scale (score
0-5) using bilateral CFP. DeepSeeNet was trained on 58,402 and tested on 900
images from the longitudinal follow-up of 4549 participants from AREDS. Gold
standard labels were obtained using reading center grades. DeepSeeNet simulates
the human grading process by first detecting individual AMD risk factors
(drusen size, pigmentary abnormalities) for each eye and then calculating a
patient-based AMD severity score using the AREDS Simplified Severity Scale.
DeepSeeNet performed better on patient-based classification (accuracy = 0.671;
kappa = 0.558) than retinal specialists (accuracy = 0.599; kappa = 0.467) with
high AUC in the detection of large drusen (0.94), pigmentary abnormalities
(0.93), and late AMD (0.97). DeepSeeNet demonstrated high accuracy with
increased transparency in the automated assignment of individual patients to
AMD risk categories based on the AREDS Simplified Severity Scale. These results
highlight the potential of deep learning to assist and enhance clinical
decision-making in patients with AMD, such as early AMD detection and risk
prediction for developing late AMD. DeepSeeNet is publicly available on
https://github.com/ncbi-nlp/DeepSeeNet.
A Self-Adaptive Network For Multiple Sclerosis Lesion Segmentation From Multi-Contrast MRI With Various Imaging Protocols
Authors: Yushan Feng, Huitong Pan, Craig Meyer, Xue Feng
Date: 2018-11-19 00:00:00
Tasks: Lesion Segmentation, Segmentation

Deep neural networks (DNN) have shown promises in the lesion segmentation of
multiple sclerosis (MS) from multicontrast MRI including T1, T2, proton density
(PD) and FLAIR sequences. However, one challenge in deploying such networks
into clinical practice is the variability of imaging protocols, which often
differ from the training dataset as certain MRI sequences may be unavailable or
unusable. Therefore, trained networks need to adapt to practical situations
when imaging protocols are different in deployment. In this paper, we propose a
DNN-based MS lesion segmentation framework with a novel technique called
sequence dropout which can adapt to various combinations of input MRI sequences
during deployment and achieve the maximal possible performance from the given
input. In addition, with this framework, we studied the quantitative impact of
each MRI sequence on the MS lesion segmentation task without training separate
networks. Experiments were performed using the IEEE ISBI 2015 Longitudinal MS
Lesion Challenge dataset and our method is currently ranked 2nd with a Dice
similarity coefficient of 0.684. Furthermore, we showed our network achieved
the maximal possible performance when one sequence is unavailable during
deployment by comparing with separate networks trained on the corresponding
input MRI sequences. In particular, we discovered T1 and PD have minor impact
on segmentation performance while FLAIR is the predominant sequence.
Experiments with multiple missing sequences were also performed and showed the
robustness of our network.
Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics
Authors: Yunbo Wang, Jianjin Zhang, Hongyu Zhu, Mingsheng Long, Jian-Min Wang, Philip S. Yu
Date: 2018-11-19 00:00:00
Proceeding: CVPR 2019 6
Tasks: Precipitation Forecasting, Time Series Analysis, Time Series Forecasting, Video Prediction

Natural spatiotemporal processes can be highly non-stationary in many ways,
e.g. the low-level non-stationarity such as spatial correlations or temporal
dependencies of local pixel values; and the high-level variations such as the
accumulation, deformation or dissipation of radar echoes in precipitation
forecasting. From Cramer's Decomposition, any non-stationary process can be
decomposed into deterministic, time-variant polynomials, plus a zero-mean
stochastic term. By applying differencing operations appropriately, we may turn
time-variant polynomials into a constant, making the deterministic component
predictable. However, most previous recurrent neural networks for
spatiotemporal prediction do not use the differential signals effectively, and
their relatively simple state transition functions prevent them from learning
too complicated variations in spacetime. We propose the Memory In Memory (MIM)
networks and corresponding recurrent blocks for this purpose. The MIM blocks
exploit the differential signals between adjacent recurrent states to model the
non-stationary and approximately stationary properties in spatiotemporal
dynamics with two cascaded, self-renewed memory modules. By stacking multiple
MIM blocks, we could potentially handle higher-order non-stationarity. The MIM
networks achieve the state-of-the-art results on four spatiotemporal prediction
tasks across both synthetic and real-world datasets. We believe that the
general idea of this work can be potentially applied to other time-series
forecasting tasks.
Quantifying Human Behavior on the Block Design Test Through Automated Multi-Level Analysis of Overhead Video
Authors: Seunghwan Cha, James Ainooson, Maithilee Kunda
Date: 2018-11-19 00:00:00

The block design test is a standardized, widely used neuropsychological
assessment of visuospatial reasoning that involves a person recreating a series
of given designs out of a set of colored blocks. In current testing procedures,
an expert neuropsychologist observes a person's accuracy and completion time as
well as overall impressions of the person's problem-solving procedures, errors,
etc., thus obtaining a holistic though subjective and often qualitative view of
the person's cognitive processes. We propose a new framework that combines room
sensors and AI techniques to augment the information available to
neuropsychologists from block design and similar tabletop assessments. In
particular, a ceiling-mounted camera captures an overhead view of the table
surface. From this video, we demonstrate how automated classification using
machine learning can produce a frame-level description of the state of the
block task and the person's actions over the course of each test problem. We
also show how a sequence-comparison algorithm can classify one individual's
problem-solving strategy relative to a database of simulated strategies, and
how these quantitative results can be visualized for use by neuropsychologists.
Re-Identification with Consistent Attentive Siamese Networks
Authors: Meng Zheng, Srikrishna Karanam, Ziyan Wu, Richard J. Radke
Date: 2018-11-19 00:00:00
Proceeding: CVPR 2019 6
Tasks: Person Re-Identification, Representation Learning

We propose a new deep architecture for person re-identification (re-id).
While re-id has seen much recent progress, spatial localization and
view-invariant representation learning for robust cross-view matching remain
key, unsolved problems. We address these questions by means of a new
attention-driven Siamese learning architecture, called the Consistent Attentive
Siamese Network. Our key innovations compared to existing, competing methods
include (a) a flexible framework design that produces attention with only
identity labels as supervision, (b) explicit mechanisms to enforce attention
consistency among images of the same person, and (c) a new Siamese framework
that integrates attention and attention consistency, producing principled
supervisory signals as well as the first mechanism that can explain the
reasoning behind the Siamese framework's predictions. We conduct extensive
evaluations on the CUHK03-NP, DukeMTMC-ReID, and Market-1501 datasets and
report competitive performance.
Visual-Texual Emotion Analysis with Deep Coupled Video and Danmu Neural Networks
Authors: Chenchen Li, Jialin Wang, Hongwei Wang, Miao Zhao, Wenjie Li, Xiaotie Deng
Date: 2018-11-19 00:00:00
Tasks: Emotion Recognition, MULTI-VIEW LEARNING

User emotion analysis toward videos is to automatically recognize the general
emotional status of viewers from the multimedia content embedded in the online
video stream. Existing works fall in two categories: 1) visual-based methods,
which focus on visual content and extract a specific set of features of videos.
However, it is generally hard to learn a mapping function from low-level video
pixels to high-level emotion space due to great intra-class variance. 2)
textual-based methods, which focus on the investigation of user-generated
comments associated with videos. The learned word representations by
traditional linguistic approaches typically lack emotion information and the
global comments usually reflect viewers' high-level understandings rather than
instantaneous emotions. To address these limitations, in this paper, we propose
to jointly utilize video content and user-generated texts simultaneously for
emotion analysis. In particular, we introduce exploiting a new type of
user-generated texts, i.e., "danmu", which are real-time comments floating on
the video and contain rich information to convey viewers' emotional opinions.
To enhance the emotion discriminativeness of words in textual feature
extraction, we propose Emotional Word Embedding (EWE) to learn text
representations by jointly considering their semantics and emotions.
Afterwards, we propose a novel visual-textual emotion analysis model with Deep
Coupled Video and Danmu Neural networks (DCVDN), in which visual and textual
features are synchronously extracted and fused to form a comprehensive
representation by deep-canonically-correlated-autoencoder-based multi-view
learning. Through extensive experiments on a self-crawled real-world
video-danmu dataset, we prove that DCVDN significantly outperforms the
state-of-the-art baselines.
Sharpen Focus: Learning with Attention Separability and Consistency
Authors: Lezi Wang, Ziyan Wu, Srikrishna Karanam, Kuan-Chuan Peng, Rajat Vikram Singh, Bo Liu, Dimitris N. Metaxas
Date: 2018-11-19 00:00:00
Proceeding: ICCV 2019 10
Tasks: General Classification, image-classification, Image Classification

Recent developments in gradient-based attention modeling have seen attention maps emerge as a powerful tool for interpreting convolutional neural networks. Despite good localization for an individual class of interest, these techniques produce attention maps with substantially overlapping responses among different classes, leading to the problem of visual confusion and the need for discriminative attention. In this paper, we address this problem by means of a new framework that makes class-discriminative attention a principled part of the learning process. Our key innovations include new learning objectives for attention separability and cross-layer consistency, which result in improved attention discriminability and reduced visual confusion. Extensive experiments on image classification benchmarks show the effectiveness of our approach in terms of improved classification accuracy, including CIFAR-100 (+3.33%), Caltech-256 (+1.64%), ILSVRC2012 (+0.92%), CUB-200-2011 (+4.8%) and PASCAL VOC2012 (+5.73%).
Show, Attend and Translate: Unpaired Multi-Domain Image-to-Image Translation with Visual Attention
Authors: Honglun Zhang, Wenqing Chen, Jidong Tian, Yongkun Wang, Yaohui Jin
Date: 2018-11-19 00:00:00
Tasks: Attribute, Generative Adversarial Network, Image-to-Image Translation, Translation

Recently unpaired multi-domain image-to-image translation has attracted great
interests and obtained remarkable progress, where a label vector is utilized to
indicate multi-domain information. In this paper, we propose SAT (Show, Attend
and Translate), an unified and explainable generative adversarial network
equipped with visual attention that can perform unpaired image-to-image
translation for multiple domains. By introducing an action vector, we treat the
original translation tasks as problems of arithmetic addition and subtraction.
Visual attention is applied to guarantee that only the regions relevant to the
target domains are translated. Extensive experiments on a facial attribute
dataset demonstrate the superiority of our approach and the generated attention
masks better explain what SAT attends when translating images.
Global and Local Sensitivity Guided Key Salient Object Re-augmentation for Video Saliency Detection
Authors: Ziqi Zhou, Zheng Wang, Huchuan Lu, Song Wang, Meijun Sun
Date: 2018-11-19 00:00:00
Tasks: Decision Making, feature selection, Object, Saliency Detection, Sensitivity, Video Saliency Detection

The existing still-static deep learning based saliency researches do not
consider the weighting and highlighting of extracted features from different
layers, all features contribute equally to the final saliency decision-making.
Such methods always evenly detect all "potentially significant regions" and
unable to highlight the key salient object, resulting in detection failure of
dynamic scenes. In this paper, based on the fact that salient areas in videos
are relatively small and concentrated, we propose a \textbf{key salient object
re-augmentation method (KSORA) using top-down semantic knowledge and bottom-up
feature guidance} to improve detection accuracy in video scenes. KSORA includes
two sub-modules (WFE and KOS): WFE processes local salient feature selection
using bottom-up strategy, while KOS ranks each object in global fashion by
top-down statistical knowledge, and chooses the most critical object area for
local enhancement. The proposed KSORA can not only strengthen the saliency
value of the local key salient object but also ensure global saliency
consistency. Results on three benchmark datasets suggest that our model has the
capability of improving the detection accuracy on complex scenes. The
significant performance of KSORA, with a speed of 17FPS on modern GPUs, has
been verified by comparisons with other ten state-of-the-art algorithms.
Best Arm Identification in Linked Bandits
Authors: Anant Gupta
Date: 2018-11-19 00:00:00
Tasks: Multi-Armed Bandits

We consider the problem of best arm identification in a variant of
multi-armed bandits called linked bandits. In a single interaction with linked
bandits, multiple arms are played sequentially until one of them receives a
positive reward. Since each interaction provides feedback about more than one
arm, the sample complexity can be much lower than in the regular bandit
setting. We propose an algorithm for linked bandits, that combines a novel
subroutine to perform uniform sampling with a known optimal algorithm for
regular bandits. We prove almost matching upper and lower bounds on the sample
complexity of best arm identification in linked bandits. These bounds have an
interesting structure, with an explicit dependence on the mean rewards of the
arms, not just the gaps. We also corroborate our theoretical results with
experiments.
Multi-scale 3D Convolution Network for Video Based Person Re-Identification
Authors: Jianing Li, Shiliang Zhang, Tiejun Huang
Date: 2018-11-19 00:00:00
Tasks: Person Re-Identification, Video-Based Person Re-Identification

This paper proposes a two-stream convolution network to extract spatial and
temporal cues for video based person Re-Identification (ReID). A temporal
stream in this network is constructed by inserting several Multi-scale 3D (M3D)
convolution layers into a 2D CNN network. The resulting M3D convolution network
introduces a fraction of parameters into the 2D CNN, but gains the ability of
multi-scale temporal feature learning. With this compact architecture, M3D
convolution network is also more efficient and easier to optimize than existing
3D convolution networks. The temporal stream further involves Residual
Attention Layers (RAL) to refine the temporal features. By jointly learning
spatial-temporal attention masks in a residual manner, RAL identifies the
discriminative spatial regions and temporal cues. The other stream in our
network is implemented with a 2D CNN for spatial feature extraction. The
spatial and temporal features from two streams are finally fused for the video
based person ReID. Evaluations on three widely used benchmarks datasets, i.e.,
MARS, PRID2011, and iLIDS-VID demonstrate the substantial advantages of our
method over existing 3D convolution networks and state-of-art methods.