pwc-archive/papers-with-abstracts huggingface dataset:
Recurrent Filter Learning for Visual Tracking
Authors: Tianyu Yang, Antoni B. Chan
Date: 2017-08-13 00:00:00
Tasks: Object, Visual Tracking

Recently using convolutional neural networks (CNNs) has gained popularity in
visual tracking, due to its robust feature representation of images. Recent
methods perform online tracking by fine-tuning a pre-trained CNN model to the
specific target object using stochastic gradient descent (SGD)
back-propagation, which is usually time-consuming. In this paper, we propose a
recurrent filter generation methods for visual tracking. We directly feed the
target's image patch to a recurrent neural network (RNN) to estimate an
object-specific filter for tracking. As the video sequence is a spatiotemporal
data, we extend the matrix multiplications of the fully-connected layers of the
RNN to a convolution operation on feature maps, which preserves the target's
spatial structure and also is memory-efficient. The tracked object in the
subsequent frames will be fed into the RNN to adapt the generated filters to
appearance variations of the target. Note that once the off-line training
process of our network is finished, there is no need to fine-tune the network
for specific objects, which makes our approach more efficient than methods that
use iterative fine-tuning to online learn the target. Extensive experiments
conducted on widely used benchmarks, OTB and VOT, demonstrate encouraging
results compared to other recent methods.
What your Facebook Profile Picture Reveals about your Personality
Authors: Cristina Segalin, Fabio Celli, Luca Polonio, Michal Kosinski, David Stillwell, Nicu Sebe, Marco Cristani, Bruno Lepri
Date: 2017-08-03 00:00:00
Tasks: General Classification

People spend considerable effort managing the impressions they give others.
Social psychologists have shown that people manage these impressions
differently depending upon their personality. Facebook and other social media
provide a new forum for this fundamental process; hence, understanding people's
behaviour on social media could provide interesting insights on their
personality. In this paper we investigate automatic personality recognition
from Facebook profile pictures. We analyze the effectiveness of four families
of visual features and we discuss some human interpretable patterns that
explain the personality traits of the individuals. For example, extroverts and
agreeable individuals tend to have warm colored pictures and to exhibit many
faces in their portraits, mirroring their inclination to socialize; while
neurotic ones have a prevalence of pictures of indoor places. Then, we propose
a classification approach to automatically recognize personality traits from
these visual features. Finally, we compare the performance of our
classification approach to the one obtained by human raters and we show that
computer-based classifications are significantly more accurate than averaged
human-based classifications for Extraversion and Neuroticism.
MarioQA: Answering Questions by Watching Gameplay Videos
Authors: Jonghwan Mun, Paul Hongsuck Seo, Ilchae Jung, Bohyung Han
Date: 2016-12-06 00:00:00
Proceeding: ICCV 2017 10
Tasks: Question Answering, Video Question Answering

We present a framework to analyze various aspects of models for video
question answering (VideoQA) using customizable synthetic datasets, which are
constructed automatically from gameplay videos. Our work is motivated by the
fact that existing models are often tested only on datasets that require
excessively high-level reasoning or mostly contain instances accessible through
single frame inferences. Hence, it is difficult to measure capacity and
flexibility of trained models, and existing techniques often rely on ad-hoc
implementations of deep neural networks without clear insight into datasets and
models. We are particularly interested in understanding temporal relationships
between video events to solve VideoQA problems; this is because reasoning
temporal dependency is one of the most distinct components in videos from
images. To address this objective, we automatically generate a customized
synthetic VideoQA dataset using {\em Super Mario Bros.} gameplay videos so that
it contains events with different levels of reasoning complexity. Using the
dataset, we show that properly constructed datasets with events in various
complexity levels are critical to learn effective models and improve overall
performance.
Automated Pulmonary Nodule Detection via 3D ConvNets with Online Sample Filtering and Hybrid-Loss Residual Learning
Authors: Qi Dou, Hao Chen, Yueming Jin, Huangjing Lin, Jing Qin, Pheng-Ann Heng
Date: 2017-08-13 00:00:00

In this paper, we propose a novel framework with 3D convolutional networks
(ConvNets) for automated detection of pulmonary nodules from low-dose CT scans,
which is a challenging yet crucial task for lung cancer early diagnosis and
treatment. Different from previous standard ConvNets, we try to tackle the
severe hard/easy sample imbalance problem in medical datasets and explore the
benefits of localized annotations to regularize the learning, and hence boost
the performance of ConvNets to achieve more accurate detections. Our proposed
framework consists of two stages: 1) candidate screening, and 2) false positive
reduction. In the first stage, we establish a 3D fully convolutional network,
effectively trained with an online sample filtering scheme, to sensitively and
rapidly screen the nodule candidates. In the second stage, we design a
hybrid-loss residual network which harnesses the location and size information
as important cues to guide the nodule recognition procedure. Experimental
results on the public large-scale LUNA16 dataset demonstrate superior
performance of our proposed method compared with state-of-the-art approaches
for the pulmonary nodule detection task.
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
Authors: Xinchen Yan, Jimei Yang, Ersin Yumer, Yijie Guo, Honglak Lee
Date: 2016-12-01 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: 3D Object Reconstruction, Decoder, Object, Object Reconstruction

Understanding the 3D world is a fundamental problem in computer vision.
However, learning a good representation of 3D objects is still an open problem
due to the high dimensionality of the data and many factors of variation
involved. In this work, we investigate the task of single-view 3D object
reconstruction from a learning agent's perspective. We formulate the learning
process as an interaction between 3D and 2D representations and propose an
encoder-decoder network with a novel projection loss defined by the perspective
transformation. More importantly, the projection loss enables the unsupervised
learning using 2D observation without explicit 3D supervision. We demonstrate
the ability of the model in generating 3D volume from a single 2D image with
three sets of experiments: (1) learning from single-class objects; (2) learning
from multi-class objects and (3) testing on novel object classes. Results show
superior performance and better generalization ability for 3D object
reconstruction when the projection loss is involved.
3D Convolutional Neural Networks for Cross Audio-Visual Matching Recognition
Authors: Amirsina Torfi, Seyed Mehdi Iranmanesh, Nasser M. Nasrabadi, Jeremy Dawson
Date: 2017-06-18 00:00:00
Tasks: Speaker Verification, speech-recognition, Speech Recognition

Audio-visual recognition (AVR) has been considered as a solution for speech
recognition tasks when the audio is corrupted, as well as a visual recognition
method used for speaker verification in multi-speaker scenarios. The approach
of AVR systems is to leverage the extracted information from one modality to
improve the recognition ability of the other modality by complementing the
missing information. The essential problem is to find the correspondence
between the audio and visual streams, which is the goal of this work. We
propose the use of a coupled 3D Convolutional Neural Network (3D-CNN)
architecture that can map both modalities into a representation space to
evaluate the correspondence of audio-visual streams using the learned
multimodal features. The proposed architecture will incorporate both spatial
and temporal information jointly to effectively find the correlation between
temporal information for different modalities. By using a relatively small
network architecture and much smaller dataset for training, our proposed method
surpasses the performance of the existing similar methods for audio-visual
matching which use 3D CNNs for feature representation. We also demonstrate that
an effective pair selection method can significantly increase the performance.
The proposed method achieves relative improvements over 20% on the Equal Error
Rate (EER) and over 7% on the Average Precision (AP) in comparison to the
state-of-the-art method.
Encoding Multi-Resolution Brain Networks Using Unsupervised Deep Learning
Authors: Arash Rahnama, Abdullah Alchihabi, Vijay Gupta, Panos Antsaklis, Fatos T. Yarman Vural
Date: 2017-08-13 00:00:00
Tasks: Deep Learning

The main goal of this study is to extract a set of brain networks in multiple
time-resolutions to analyze the connectivity patterns among the anatomic
regions for a given cognitive task. We suggest a deep architecture which learns
the natural groupings of the connectivity patterns of human brain in multiple
time-resolutions. The suggested architecture is tested on task data set of
Human Connectome Project (HCP) where we extract multi-resolution networks, each
of which corresponds to a cognitive task. At the first level of this
architecture, we decompose the fMRI signal into multiple sub-bands using
wavelet decompositions. At the second level, for each sub-band, we estimate a
brain network extracted from short time windows of the fMRI signal. At the
third level, we feed the adjacency matrices of each mesh network at each
time-resolution into an unsupervised deep learning algorithm, namely, a Stacked
De- noising Auto-Encoder (SDAE). The outputs of the SDAE provide a compact
connectivity representation for each time window at each sub-band of the fMRI
signal. We concatenate the learned representations of all sub-bands at each
window and cluster them by a hierarchical algorithm to find the natural
groupings among the windows. We observe that each cluster represents a
cognitive task with a performance of 93% Rand Index and 71% Adjusted Rand
Index. We visualize the mean values and the precisions of the networks at each
component of the cluster mixture. The mean brain networks at cluster centers
show the variations among cognitive tasks and the precision of each cluster
shows the within cluster variability of networks, across the subjects.
AI Challenges in Human-Robot Cognitive Teaming
Authors: Tathagata Chakraborti, Subbarao Kambhampati, Matthias Scheutz, Yu Zhang
Date: 2017-07-15 00:00:00

Among the many anticipated roles for robots in the future is that of being a
human teammate. Aside from all the technological hurdles that have to be
overcome with respect to hardware and control to make robots fit to work with
humans, the added complication here is that humans have many conscious and
subconscious expectations of their teammates - indeed, we argue that teaming is
mostly a cognitive rather than physical coordination activity. This introduces
new challenges for the AI and robotics community and requires fundamental
changes to the traditional approach to the design of autonomy. With this in
mind, we propose an update to the classical view of the intelligent agent
architecture, highlighting the requirements for mental modeling of the human in
the deliberative process of the autonomous agent. In this article, we outline
briefly the recent efforts of ours, and others in the community, towards
developing cognitive teammates along these guidelines.
The Devil is in the Decoder: Classification, Regression and GANs
Authors: Zbigniew Wojna, Vittorio Ferrari, Sergio Guadarrama, Nathan Silberman, Liang-Chieh Chen, Alireza Fathi, Jasper Uijlings
Date: 2017-07-18 00:00:00
Tasks: Boundary Detection, Decoder, Depth Estimation, Depth Prediction, General Classification, regression, Semantic Segmentation

Many machine vision applications, such as semantic segmentation and depth
prediction, require predictions for every pixel of the input image. Models for
such problems usually consist of encoders which decrease spatial resolution
while learning a high-dimensional representation, followed by decoders who
recover the original input resolution and result in low-dimensional
predictions. While encoders have been studied rigorously, relatively few
studies address the decoder side. This paper presents an extensive comparison
of a variety of decoders for a variety of pixel-wise tasks ranging from
classification, regression to synthesis. Our contributions are: (1) Decoders
matter: we observe significant variance in results between different types of
decoders on various problems. (2) We introduce new residual-like connections
for decoders. (3) We introduce a novel decoder: bilinear additive upsampling.
(4) We explore prediction artifacts.
Stochastic Generative Hashing
Authors: Bo Dai, Ruiqi Guo, Sanjiv Kumar, Niao He, Le Song
Date: 2017-01-11 00:00:00
Proceeding: ICML 2017 8
Tasks: Retrieval

Learning-based binary hashing has become a powerful paradigm for fast search
and retrieval in massive databases. However, due to the requirement of discrete
outputs for the hash functions, learning such functions is known to be very
challenging. In addition, the objective functions adopted by existing hashing
techniques are mostly chosen heuristically. In this paper, we propose a novel
generative approach to learn hash functions through Minimum Description Length
principle such that the learned hash codes maximally compress the dataset and
can also be used to regenerate the inputs. We also develop an efficient
learning algorithm based on the stochastic distributional gradient, which
avoids the notorious difficulty caused by binary output constraints, to jointly
optimize the parameters of the hash function and the associated generative
model. Extensive experiments on a variety of large-scale datasets show that the
proposed method achieves better retrieval results than the existing
state-of-the-art methods.
UbuntuWorld 1.0 LTS - A Platform for Automated Problem Solving & Troubleshooting in the Ubuntu OS
Authors: Tathagata Chakraborti, Kartik Talamadupula, Kshitij P. Fadnis, Murray Campbell, Subbarao Kambhampati
Date: 2016-09-27 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we present UbuntuWorld 1.0 LTS - a platform for developing
automated technical support agents in the Ubuntu operating system.
Specifically, we propose to use the Bash terminal as a simulator of the Ubuntu
environment for a learning-based agent and demonstrate the usefulness of
adopting reinforcement learning (RL) techniques for basic problem solving and
troubleshooting in this environment. We provide a plug-and-play interface to
the simulator as a python package where different types of agents can be
plugged in and evaluated, and provide pathways for integrating data from online
support forums like AskUbuntu into an automated agent's learning process.
Finally, we show that the use of this data significantly improves the agent's
learning efficiency. We believe that this platform can be adopted as a
real-world test bed for research on automated technical support.
Mass Displacement Networks
Authors: Natalia Neverova, Iasonas Kokkinos
Date: 2017-08-12 00:00:00
Tasks: Pose Estimation

Despite the large improvements in performance attained by using deep learning
in computer vision, one can often further improve results with some additional
post-processing that exploits the geometric nature of the underlying task. This
commonly involves displacing the posterior distribution of a CNN in a way that
makes it more appropriate for the task at hand, e.g. better aligned with local
image features, or more compact. In this work we integrate this geometric
post-processing within a deep architecture, introducing a differentiable and
probabilistically sound counterpart to the common geometric voting technique
used for evidence accumulation in vision. We refer to the resulting neural
models as Mass Displacement Networks (MDNs), and apply them to human pose
estimation in two distinct setups: (a) landmark localization, where we collapse
a distribution to a point, allowing for precise localization of body keypoints
and (b) communication across body parts, where we transfer evidence from one
part to the other, allowing for a globally consistent pose estimate. We
evaluate on large-scale pose estimation benchmarks, such as MPII Human Pose and
COCO datasets, and report systematic improvements when compared to strong
baselines.
On Clustering and Embedding Mixture Manifolds using a Low Rank Neighborhood Approach
Authors: Arun M. Saranathan, Mario Parente
Date: 2016-08-23 00:00:00
Tasks: Clustering

Samples from intimate (non-linear) mixtures are generally modeled as being
drawn from a smooth manifold. Scenarios where the data contains multiple
intimate mixtures with some constituent materials in common can be thought of
as manifolds which share a boundary. Two important steps in the processing of
such data are (i) to identify (cluster) the different mixture-manifolds present
in the data and (ii) to eliminate the non-linearities present the data by
mapping each mixture-manifold into some low-dimensional euclidean space
(embedding). Manifold clustering and embedding techniques appear to be an ideal
tool for this task, but the present state-of-the-art algorithms perform poorly
for hyperspectral data, particularly in the embedding task. We propose a novel
reconstruction-based algorithm for improved clustering and embedding of
mixture-manifolds. The algorithms attempts to reconstruct each target-point as
an affine combination of its nearest neighbors with an additional rank penalty
on the neighborhood to ensure that only neighbors on the same manifold as the
target-point are used in the reconstruction. The reconstruction matrix
generated by using this technique is block-diagonal and can be used for
clustering (using spectral clustering) and embedding. The improved performance
of the algorithms vis-a-vis its competitors is exhibited on a variety of
simulated and real mixture datasets.
Statistical Vs Rule Based Machine Translation; A Case Study on Indian Language Perspective
Authors: Sreelekha. S
Date: 2017-08-12 00:00:00
Tasks: Machine Translation, Translation

In this paper we present our work on a case study between Statistical Machien
Transaltion (SMT) and Rule-Based Machine Translation (RBMT) systems on
English-Indian langugae and Indian to Indian langugae perspective. Main
objective of our study is to make a five way performance compariosn; such as,
a) SMT and RBMT b) SMT on English-Indian langugae c) RBMT on English-Indian
langugae d) SMT on Indian to Indian langugae perspective e) RBMT on Indian to
Indian langugae perspective. Through a detailed analysis we describe the Rule
Based and the Statistical Machine Translation system developments and its
evaluations. Through a detailed error analysis, we point out the relative
strengths and weaknesses of both systems. The observations based on our study
are: a) SMT systems outperforms RBMT b) In the case of SMT, English to Indian
language MT systmes performs better than Indian to English langugae MT systems
c) In the case of RBMT, English to Indian langugae MT systems perofrms better
than Indian to Englsih Language MT systems d) SMT systems performs better for
Indian to Indian language MT systems compared to RBMT. Effectively, we shall
see that even with a small amount of training corpus a statistical machine
translation system has many advantages for high quality domain specific machine
translation over that of a rule-based counterpart.
Revisiting the Effectiveness of Off-the-shelf Temporal Modeling Approaches for Large-scale Video Classification
Authors: Yunlong Bian, Chuang Gan, Xiao Liu, Fu Li, Xiang Long, Yandong Li, Heng Qi, Jie zhou, Shilei Wen, Yuanqing Lin
Date: 2017-08-12 00:00:00
Tasks: Action Classification, General Classification, Video Classification, Video Recognition

This paper describes our solution for the video recognition task of
ActivityNet Kinetics challenge that ranked the 1st place. Most of existing
state-of-the-art video recognition approaches are in favor of an end-to-end
pipeline. One exception is the framework of DevNet. The merit of DevNet is that
they first use the video data to learn a network (i.e. fine-tuning or training
from scratch). Instead of directly using the end-to-end classification scores
(e.g. softmax scores), they extract the features from the learned network and
then fed them into the off-the-shelf machine learning models to conduct video
classification. However, the effectiveness of this line work has long-term been
ignored and underestimated. In this submission, we extensively use this
strategy. Particularly, we investigate four temporal modeling approaches using
the learned features: Multi-group Shifting Attention Network, Temporal Xception
Network, Multi-stream sequence Model and Fast-Forward Sequence Model.
Experiment results on the challenging Kinetics dataset demonstrate that our
proposed temporal modeling approaches can significantly improve existing
approaches in the large-scale video recognition tasks. Most remarkably, our
best single Multi-group Shifting Attention Network can achieve 77.7% in term of
top-1 accuracy and 93.2% in term of top-5 accuracy on the validation set.
An efficient genetic algorithm for large-scale planning of robust industrial wireless networks
Authors: Xu Gong, David Plets, Emmeric Tanghe, Toon De Pessemier, Luc Martens, Wout Joseph
Date: 2017-08-12 00:00:00

An industrial indoor environment is harsh for wireless communications
compared to an office environment, because the prevalent metal easily causes
shadowing effects and affects the availability of an industrial wireless local
area network (IWLAN). On the one hand, it is costly, time-consuming, and
ineffective to perform trial-and-error manual deployment of wireless nodes. On
the other hand, the existing wireless planning tools only focus on office
environments such that it is hard to plan IWLANs due to the larger problem size
and the deployed IWLANs are vulnerable to prevalent shadowing effects in harsh
industrial indoor environments. To fill this gap, this paper proposes an
overdimensioning model and a genetic algorithm based over-dimensioning (GAOD)
algorithm for deploying large-scale robust IWLANs. As a progress beyond the
state-of-the-art wireless planning, two full coverage layers are created. The
second coverage layer serves as redundancy in case of shadowing. Meanwhile, the
deployment cost is reduced by minimizing the number of access points (APs); the
hard constraint of minimal inter-AP spatial paration avoids multiple APs
covering the same area to be simultaneously shadowed by the same obstacle. The
computation time and occupied memory are dedicatedly considered in the design
of GAOD for large-scale optimization. A greedy heuristic based
over-dimensioning (GHOD) algorithm and a random OD algorithm are taken as
benchmarks. In two vehicle manufacturers with a small and large indoor
environment, GAOD outperformed GHOD with up to 20% less APs, while GHOD
outputted up to 25% less APs than a random OD algorithm. Furthermore, the
effectiveness of this model and GAOD was experimentally validated with a real
deployment system.