pwc-archive/papers-with-abstracts huggingface dataset:
Aggregated Residual Transformations for Deep Neural Networks
Authors: Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, Kaiming He
Date: 2016-11-16 00:00:00
Proceeding: CVPR 2017 7
Tasks: Domain Generalization, General Classification, image-classification, Image Classification

We present a simple, highly modularized network architecture for image
classification. Our network is constructed by repeating a building block that
aggregates a set of transformations with the same topology. Our simple design
results in a homogeneous, multi-branch architecture that has only a few
hyper-parameters to set. This strategy exposes a new dimension, which we call
"cardinality" (the size of the set of transformations), as an essential factor
in addition to the dimensions of depth and width. On the ImageNet-1K dataset,
we empirically show that even under the restricted condition of maintaining
complexity, increasing cardinality is able to improve classification accuracy.
Moreover, increasing cardinality is more effective than going deeper or wider
when we increase the capacity. Our models, named ResNeXt, are the foundations
of our entry to the ILSVRC 2016 classification task in which we secured 2nd
place. We further investigate ResNeXt on an ImageNet-5K set and the COCO
detection set, also showing better results than its ResNet counterpart. The
code and models are publicly available online.
Long-Term Identity-Aware Multi-Person Tracking for Surveillance Video Summarization
Authors: Shoou-I Yu, Yi Yang, Xuanchong Li, Alexander G. Hauptmann
Date: 2016-04-25 00:00:00
Tasks: Face Recognition, Video Summarization

Multi-person tracking plays a critical role in the analysis of surveillance
video. However, most existing work focus on shorter-term (e.g. minute-long or
hour-long) video sequences. Therefore, we propose a multi-person tracking
algorithm for very long-term (e.g. month-long) multi-camera surveillance
scenarios. Long-term tracking is challenging because 1) the apparel/appearance
of the same person will vary greatly over multiple days and 2) a person will
leave and re-enter the scene numerous times. To tackle these challenges, we
leverage face recognition information, which is robust to apparel change, to
automatically reinitialize our tracker over multiple days of recordings.
Unfortunately, recognized faces are unavailable oftentimes. Therefore, our
tracker propagates identity information to frames without recognized faces by
uncovering the appearance and spatial manifold formed by person detections. We
tested our algorithm on a 23-day 15-camera data set (4,935 hours total), and we
were able to localize a person 53.2% of the time with 69.8% precision. We
further performed video summarization experiments based on our tracking output.
Results on 116.25 hours of video showed that we were able to generate a
reasonable visual diary (i.e. a summary of what a person did) for different
people, thus potentially opening the door to automatic summarization of the
vast amount of surveillance video generated every day.
Deep Image Matting
Authors: Ning Xu, Brian Price, Scott Cohen, Thomas Huang
Date: 2017-03-10 00:00:00
Proceeding: CVPR 2017 7
Tasks: Decoder, Image Matting, Semantic Image Matting

Image matting is a fundamental computer vision problem and has many
applications. Previous algorithms have poor performance when an image has
similar foreground and background colors or complicated textures. The main
reasons are prior methods 1) only use low-level features and 2) lack high-level
context. In this paper, we propose a novel deep learning based algorithm that
can tackle both these problems. Our deep model has two parts. The first part is
a deep convolutional encoder-decoder network that takes an image and the
corresponding trimap as inputs and predict the alpha matte of the image. The
second part is a small convolutional network that refines the alpha matte
predictions of the first network to have more accurate alpha values and sharper
edges. In addition, we also create a large-scale image matting dataset
including 49300 training images and 1000 testing images. We evaluate our
algorithm on the image matting benchmark, our testing set, and a wide variety
of real images. Experimental results clearly demonstrate the superiority of our
algorithm over previous methods.
Minkowski Operations of Sets with Application to Robot Localization
Authors: Benoit Desrochers, Luc Jaulin
Date: 2017-04-11 00:00:00

This papers shows that using separators, which is a pair of two complementary
contractors, we can easily and efficiently solve the localization problem of a
robot with sonar measurements in an unstructured environment. We introduce
separators associated with the Minkowski sum and the Minkowski difference in
order to facilitate the resolution. A test-case is given in order to illustrate
the principle of the approach.
Manifold Matching using Shortest-Path Distance and Joint Neighborhood Selection
Authors: Cencheng Shen, Joshua T. Vogelstein, Carey E. Priebe
Date: 2014-12-12 00:00:00

Matching datasets of multiple modalities has become an important task in data
analysis. Existing methods often rely on the embedding and transformation of
each single modality without utilizing any correspondence information, which
often results in sub-optimal matching performance. In this paper, we propose a
nonlinear manifold matching algorithm using shortest-path distance and joint
neighborhood selection. Specifically, a joint nearest-neighbor graph is built
for all modalities. Then the shortest-path distance within each modality is
calculated from the joint neighborhood graph, followed by embedding into and
matching in a common low-dimensional Euclidean space. Compared to existing
algorithms, our approach exhibits superior performance for matching disparate
datasets of multiple modalities.
Fast-AT: Fast Automatic Thumbnail Generation using Deep Neural Networks
Authors: Seyed A. Esmaeili, Bharat Singh, Larry S. Davis
Date: 2016-12-14 00:00:00
Proceeding: CVPR 2017 7

Fast-AT is an automatic thumbnail generation system based on deep neural
networks. It is a fully-convolutional deep neural network, which learns
specific filters for thumbnails of different sizes and aspect ratios. During
inference, the appropriate filter is selected depending on the dimensions of
the target thumbnail. Unlike most previous work, Fast-AT does not utilize
saliency but addresses the problem directly. In addition, it eliminates the
need to conduct region search on the saliency map. The model generalizes to
thumbnails of different sizes including those with extreme aspect ratios and
can generate thumbnails in real time. A data set of more than 70,000 thumbnail
annotations was collected to train Fast-AT. We show competitive results in
comparison to existing techniques.
WRPN: Training and Inference using Wide Reduced-Precision Networks
Authors: Asit Mishra, Jeffrey J Cook, Eriko Nurvitadhi, Debbie Marr
Date: 2017-04-10 00:00:00
Tasks: Quantization

For computer vision applications, prior works have shown the efficacy of
reducing the numeric precision of model parameters (network weights) in deep
neural networks but also that reducing the precision of activations hurts model
accuracy much more than reducing the precision of model parameters. We study
schemes to train networks from scratch using reduced-precision activations
without hurting the model accuracy. We reduce the precision of activation maps
(along with model parameters) using a novel quantization scheme and increase
the number of filter maps in a layer, and find that this scheme compensates or
surpasses the accuracy of the baseline full-precision network. As a result, one
can significantly reduce the dynamic memory footprint, memory bandwidth,
computational energy and speed up the training and inference process with
appropriate hardware support. We call our scheme WRPN - wide reduced-precision
networks. We report results using our proposed schemes and show that our
results are better than previously reported accuracies on ILSVRC-12 dataset
while being computationally less expensive compared to previously reported
reduced-precision networks.
DeepCare: A Deep Dynamic Memory Model for Predictive Medicine
Authors: Trang Pham, Truyen Tran, Dinh Phung, Svetha Venkatesh
Date: 2016-02-01 00:00:00

Personalized predictive medicine necessitates the modeling of patient illness
and care processes, which inherently have long-term temporal dependencies.
Healthcare observations, recorded in electronic medical records, are episodic
and irregular in time. We introduce DeepCare, an end-to-end deep dynamic neural
network that reads medical records, stores previous illness history, infers
current illness states and predicts future medical outcomes. At the data level,
DeepCare represents care episodes as vectors in space, models patient health
state trajectories through explicit memory of historical records. Built on Long
Short-Term Memory (LSTM), DeepCare introduces time parameterizations to handle
irregular timed events by moderating the forgetting and consolidation of memory
cells. DeepCare also incorporates medical interventions that change the course
of illness and shape future medical risk. Moving up to the health state level,
historical and present health states are then aggregated through multiscale
temporal pooling, before passing through a neural network that estimates future
outcomes. We demonstrate the efficacy of DeepCare for disease progression
modeling, intervention recommendation, and future risk prediction. On two
important cohorts with heavy social and economic burden -- diabetes and mental
health -- the results show improved modeling and risk prediction accuracy.
Data-efficient Deep Reinforcement Learning for Dexterous Manipulation
Authors: Ivaylo Popov, Nicolas Heess, Timothy Lillicrap, Roland Hafner, Gabriel Barth-Maron, Matej Vecerik, Thomas Lampe, Yuval Tassa, Tom Erez, Martin Riedmiller
Date: 2017-04-10 00:00:00
Proceeding: ICLR 2018 1
Tasks: continuous-control, Continuous Control, Deep Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep learning and reinforcement learning methods have recently been used to
solve a variety of problems in continuous control domains. An obvious
application of these techniques is dexterous manipulation tasks in robotics
which are difficult to solve using traditional control theory or
hand-engineered approaches. One example of such a task is to grasp an object
and precisely stack it on another. Solving this difficult and practically
relevant problem in the real world is an important long-term goal for the field
of robotics. Here we take a step towards this goal by examining the problem in
simulation and providing models and techniques aimed at solving it. We
introduce two extensions to the Deep Deterministic Policy Gradient algorithm
(DDPG), a model-free Q-learning based method, which make it significantly more
data-efficient and scalable. Our results show that by making extensive use of
off-policy data and replay, it is possible to find control policies that
robustly grasp objects and stack them. Further, our results hint that it may
soon be feasible to train successful stacking policies by collecting
interactions on real robots.
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Authors: Charles R. Qi, Hao Su, Kaichun Mo, Leonidas J. Guibas
Date: 2016-12-02 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Face Reconstruction, 3D Semantic Segmentation, Few-Shot 3D Point Cloud Classification, Point Cloud Segmentation, Scene Segmentation, Semantic Parsing, Semantic Segmentation, Skeleton Based Action Recognition, Supervised Only 3D Point Cloud Classification

Point cloud is an important type of geometric data structure. Due to its
irregular format, most researchers transform such data to regular 3D voxel
grids or collections of images. This, however, renders data unnecessarily
voluminous and causes issues. In this paper, we design a novel type of neural
network that directly consumes point clouds and well respects the permutation
invariance of points in the input. Our network, named PointNet, provides a
unified architecture for applications ranging from object classification, part
segmentation, to scene semantic parsing. Though simple, PointNet is highly
efficient and effective. Empirically, it shows strong performance on par or
even better than state of the art. Theoretically, we provide analysis towards
understanding of what the network has learnt and why the network is robust with
respect to input perturbation and corruption.
NIPS 2016 Workshop on Representation Learning in Artificial and Biological Neural Networks (MLINI 2016)
Authors: Leila Wehbe, Anwar Nunez-Elizalde, Marcel van Gerven, Irina Rish, Brian Murphy, Moritz Grosse-Wentrup, Georg Langs, Guillermo Cecchi
Date: 2017-01-06 00:00:00
Tasks: BIG-bench Machine Learning, Representation Learning

This workshop explores the interface between cognitive neuroscience and
recent advances in AI fields that aim to reproduce human performance such as
natural language processing and computer vision, and specifically deep learning
approaches to such problems.
  When studying the cognitive capabilities of the brain, scientists follow a
system identification approach in which they present different stimuli to the
subjects and try to model the response that different brain areas have of that
stimulus. The goal is to understand the brain by trying to find the function
that expresses the activity of brain areas in terms of different properties of
the stimulus. Experimental stimuli are becoming increasingly complex with more
and more people being interested in studying real life phenomena such as the
perception of natural images or natural sentences. There is therefore a need
for a rich and adequate vector representation of the properties of the
stimulus, that we can obtain using advances in machine learning.
  In parallel, new ML approaches, many of which in deep learning, are inspired
to a certain extent by human behavior or biological principles. Neural networks
for example were originally inspired by biological neurons. More recently,
processes such as attention are being used which have are inspired by human
behavior. However, the large bulk of these methods are independent of findings
about brain function, and it is unclear whether it is at all beneficial for
machine learning to try to emulate brain function in order to achieve the same
tasks that the brain achieves.
Action Unit Detection with Region Adaptation, Multi-labeling Learning and Optimal Temporal Fusing
Authors: Wei Li, Farnaz Abitahi, Zhigang Zhu
Date: 2017-04-10 00:00:00
Proceeding: CVPR 2017 7
Tasks: Action Unit Detection, Multi-Label Learning

Action Unit (AU) detection becomes essential for facial analysis. Many
proposed approaches face challenging problems in dealing with the alignments of
different face regions, in the effective fusion of temporal information, and in
training a model for multiple AU labels. To better address these problems, we
propose a deep learning framework for AU detection with region of interest
(ROI) adaptation, integrated multi-label learning, and optimal LSTM-based
temporal fusing. First, ROI cropping nets (ROI Nets) are designed to make sure
specifically interested regions of faces are learned independently; each
sub-region has a local convolutional neural network (CNN) - an ROI Net, whose
convolutional filters will only be trained for the corresponding region.
Second, multi-label learning is employed to integrate the outputs of those
individual ROI cropping nets, which learns the inter-relationships of various
AUs and acquires global features across sub-regions for AU detection. Finally,
the optimal selection of multiple LSTM layers to form the best LSTM Net is
carried out to best fuse temporal features, in order to make the AU prediction
the most accurate. The proposed approach is evaluated on two popular AU
detection datasets, BP4D and DISFA, outperforming the state of the art
significantly, with an average improvement of around 13% on BP4D and 25% on
DISFA, respectively.
Quad-networks: unsupervised learning to rank for interest point detection
Authors: Nikolay Savinov, Akihito Seki, Lubor Ladicky, Torsten Sattler, Marc Pollefeys
Date: 2016-11-22 00:00:00
Proceeding: CVPR 2017 7
Tasks: Interest Point Detection, Learning-To-Rank

Several machine learning tasks require to represent the data using only a
sparse set of interest points. An ideal detector is able to find the
corresponding interest points even if the data undergo a transformation typical
for a given domain. Since the task is of high practical interest in computer
vision, many hand-crafted solutions were proposed. In this paper, we ask a
fundamental question: can we learn such detectors from scratch? Since it is
often unclear what points are "interesting", human labelling cannot be used to
find a truly unbiased solution. Therefore, the task requires an unsupervised
formulation. We are the first to propose such a formulation: training a neural
network to rank points in a transformation-invariant manner. Interest points
are then extracted from the top/bottom quantiles of this ranking. We validate
our approach on two tasks: standard RGB image interest point detection and
challenging cross-modal interest point detection between RGB and depth images.
We quantitatively show that our unsupervised method performs better or on-par
with baselines.
CERN: Confidence-Energy Recurrent Network for Group Activity Recognition
Authors: Tianmin Shu, Sinisa Todorovic, Song-Chun Zhu
Date: 2017-04-10 00:00:00
Proceeding: CVPR 2017 7
Tasks: Activity Recognition, Group Activity Recognition

This work is about recognizing human activities occurring in videos at
distinct semantic levels, including individual actions, interactions, and group
activities. The recognition is realized using a two-level hierarchy of Long
Short-Term Memory (LSTM) networks, forming a feed-forward deep architecture,
which can be trained end-to-end. In comparison with existing architectures of
LSTMs, we make two key contributions giving the name to our approach as
Confidence-Energy Recurrent Network -- CERN. First, instead of using the common
softmax layer for prediction, we specify a novel energy layer (EL) for
estimating the energy of our predictions. Second, rather than finding the
common minimum-energy class assignment, which may be numerically unstable under
uncertainty, we specify that the EL additionally computes the p-values of the
solutions, and in this way estimates the most confident energy minimum. The
evaluation on the Collective Activity and Volleyball datasets demonstrates: (i)
advantages of our two contributions relative to the common softmax and
energy-minimization formulations and (ii) a superior performance relative to
the state-of-the-art approaches.
DRAW: Deep networks for Recognizing styles of Artists Who illustrate children's books
Authors: Samet Hicsonmez, Nermin Samet, Fadime Sener, Pinar Duygulu
Date: 2017-04-10 00:00:00
Tasks: General Classification, valid

This paper is motivated from a young boy's capability to recognize an
illustrator's style in a totally different context. In the book "We are All
Born Free" [1], composed of selected rights from the Universal Declaration of
Human Rights interpreted by different illustrators, the boy was surprised to
see a picture similar to the ones in the "Winnie the Witch" series drawn by
Korky Paul (Figure 1). The style was noticeable in other characters of the same
illustrator in different books as well. The capability of a child to easily
spot the style was shown to be valid for other illustrators such as Axel
Scheffler and Debi Gliori. The boy's enthusiasm let us to start the journey to
explore the capabilities of machines to recognize the style of illustrators.
  We collected pages from children's books to construct a new illustrations
dataset consisting of about 6500 pages from 24 artists. We exploited deep
networks for categorizing illustrators and with around 94% classification
performance our method over-performed the traditional methods by more than 10%.
Going beyond categorization we explored transferring style. The classification
performance on the transferred images has shown the ability of our system to
capture the style. Furthermore, we discovered representative illustrations and
discriminative stylistic elements.
Deep Reinforcement Learning for Visual Object Tracking in Videos
Authors: Da Zhang, Hamid Maei, Xin Wang, Yuan-Fang Wang
Date: 2017-01-31 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, Object, Object Tracking, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making, Visual Object Tracking, Visual Tracking

In this paper we introduce a fully end-to-end approach for visual tracking in
videos that learns to predict the bounding box locations of a target object at
every frame. An important insight is that the tracking problem can be
considered as a sequential decision-making process and historical semantics
encode highly relevant information for future decisions. Based on this
intuition, we formulate our model as a recurrent convolutional neural network
agent that interacts with a video overtime, and our model can be trained with
reinforcement learning (RL) algorithms to learn good tracking policies that pay
attention to continuous, inter-frame correlation and maximize tracking
performance in the long run. The proposed tracking algorithm achieves
state-of-the-art performance in an existing tracking benchmark and operates at
frame-rates faster than real-time. To the best of our knowledge, our tracker is
the first neural-network tracker that combines convolutional and recurrent
networks with RL algorithms.
Matching Media Contents with User Profiles by means of the Dempster-Shafer Theory
Authors: Luigi Troiano, Irene Díaz, Ciro Gaglione
Date: 2017-04-10 00:00:00

The media industry is increasingly personalizing the offering of contents in
attempt to better target the audience. This requires to analyze the
relationships that goes established between users and content they enjoy,
looking at one side to the content characteristics and on the other to the user
profile, in order to find the best match between the two. In this paper we
suggest to build that relationship using the Dempster-Shafer's Theory of
Evidence, proposing a reference model and illustrating its properties by means
of a toy example. Finally we suggest possible applications of the model for
tasks that are common in the modern media industry.
Semantically Consistent Regularization for Zero-Shot Recognition
Authors: Pedro Morgado, Nuno Vasconcelos
Date: 2017-04-10 00:00:00
Proceeding: CVPR 2017 7
Tasks: General Classification, Zero-Shot Learning

The role of semantics in zero-shot learning is considered. The effectiveness
of previous approaches is analyzed according to the form of supervision
provided. While some learn semantics independently, others only supervise the
semantic subspace explained by training classes. Thus, the former is able to
constrain the whole space but lacks the ability to model semantic correlations.
The latter addresses this issue but leaves part of the semantic space
unsupervised. This complementarity is exploited in a new convolutional neural
network (CNN) framework, which proposes the use of semantics as constraints for
recognition.Although a CNN trained for classification has no transfer ability,
this can be encouraged by learning an hidden semantic layer together with a
semantic code for classification. Two forms of semantic constraints are then
introduced. The first is a loss-based regularizer that introduces a
generalization constraint on each semantic predictor. The second is a codeword
regularizer that favors semantic-to-class mappings consistent with prior
semantic knowledge while allowing these to be learned from data. Significant
improvements over the state-of-the-art are achieved on several datasets.