pwc-archive/papers-with-abstracts huggingface dataset:
Convolutional Models for Joint Object Categorization and Pose Estimation
Authors: Mohamed Elhoseiny, Tarek El-Gaaly, Amr Bakry, Ahmed Elgammal
Date: 2015-11-16 00:00:00
Tasks: Object, Object Categorization, Object Recognition, Pose Estimation

In the task of Object Recognition, there exists a dichotomy between the
categorization of objects and estimating object pose, where the former
necessitates a view-invariant representation, while the latter requires a
representation capable of capturing pose information over different categories
of objects. With the rise of deep architectures, the prime focus has been on
object category recognition. Deep learning methods have achieved wide success
in this task. In contrast, object pose regression using these approaches has
received relatively much less attention. In this paper we show how deep
architectures, specifically Convolutional Neural Networks (CNN), can be adapted
to the task of simultaneous categorization and pose estimation of objects. We
investigate and analyze the layers of various CNN models and extensively
compare between them with the goal of discovering how the layers of distributed
representations of CNNs represent object pose information and how this
contradicts with object category representations. We extensively experiment on
two recent large and challenging multi-view datasets. Our models achieve better
than state-of-the-art performance on both datasets.
Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd
Authors: Andreas Doumanoglou, Rigas Kouskouridas, Sotiris Malassiotis, Tae-Kyun Kim
Date: 2015-12-23 00:00:00
Proceeding: CVPR 2016 6
Tasks: 6D Pose Estimation, 6D Pose Estimation using RGB, Clustering, Object, object-detection, Object Detection, Pose Estimation

Object detection and 6D pose estimation in the crowd (scenes with multiple
object instances, severe foreground occlusions and background distractors), has
become an important problem in many rapidly evolving technological areas such
as robotics and augmented reality. Single shot-based 6D pose estimators with
manually designed features are still unable to tackle the above challenges,
motivating the research towards unsupervised feature learning and
next-best-view estimation. In this work, we present a complete framework for
both single shot-based 6D object pose estimation and next-best-view prediction
based on Hough Forests, the state of the art object pose estimator that
performs classification and regression jointly. Rather than using manually
designed features we a) propose an unsupervised feature learnt from
depth-invariant patches using a Sparse Autoencoder and b) offer an extensive
evaluation of various state of the art features. Furthermore, taking advantage
of the clustering performed in the leaf nodes of Hough Forests, we learn to
estimate the reduction of uncertainty in other views, formulating the problem
of selecting the next-best-view. To further improve pose estimation, we propose
an improved joint registration and hypotheses verification module as a final
refinement step to reject false detections. We provide two additional
challenging datasets inspired from realistic scenarios to extensively evaluate
the state of the art and our framework. One is related to domestic environments
and the other depicts a bin-picking scenario mostly found in industrial
settings. We show that our framework significantly outperforms state of the art
both on public and on our datasets.
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Authors: Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua Bengio
Date: 2015-02-10 00:00:00
Tasks: Caption Generation, Image Captioning, Translation

Inspired by recent work in machine translation and object detection, we
introduce an attention based model that automatically learns to describe the
content of images. We describe how we can train this model in a deterministic
manner using standard backpropagation techniques and stochastically by
maximizing a variational lower bound. We also show through visualization how
the model is able to automatically learn to fix its gaze on salient objects
while generating the corresponding words in the output sequence. We validate
the use of attention with state-of-the-art performance on three benchmark
datasets: Flickr8k, Flickr30k and MS COCO.
An Enhanced Method For Evaluating Automatic Video Summaries
Authors: Karim M. Mahmoud
Date: 2014-01-14 00:00:00
Tasks: Video Summarization

Evaluation of automatic video summaries is a challenging problem. In the past
years, some evaluation methods are presented that utilize only a single feature
like color feature to detect similarity between automatic video summaries and
ground-truth user summaries. One of the drawbacks of using a single feature is
that sometimes it gives a false similarity detection which makes the assessment
of the quality of the generated video summary less perceptual and not accurate.
In this paper, a novel method for evaluating automatic video summaries is
presented. This method is based on comparing automatic video summaries
generated by video summarization techniques with ground-truth user summaries.
The objective of this evaluation method is to quantify the quality of video
summaries, and allow comparing different video summarization techniques
utilizing both color and texture features of the video frames and using the
Bhattacharya distance as a dissimilarity measure due to its advantages. Our
Experiments show that the proposed evaluation method overcomes the drawbacks of
other methods and gives a more perceptual evaluation of the quality of the
automatic video summaries.
Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems
Authors: Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander Miller, Arthur Szlam, Jason Weston
Date: 2015-11-21 00:00:00

A long-term goal of machine learning is to build intelligent conversational
agents. One recent popular approach is to train end-to-end models on a large
amount of real dialog transcripts between humans (Sordoni et al., 2015; Vinyals
& Le, 2015; Shang et al., 2015). However, this approach leaves many questions
unanswered as an understanding of the precise successes and shortcomings of
each model is hard to assess. A contrasting recent proposal are the bAbI tasks
(Weston et al., 2015b) which are synthetic data that measure the ability of
learning machines at various reasoning tasks over toy language. Unfortunately,
those tests are very small and hence may encourage methods that do not scale.
In this work, we propose a suite of new tasks of a much larger scale that
attempt to bridge the gap between the two regimes. Choosing the domain of
movies, we provide tasks that test the ability of models to answer factual
questions (utilizing OMDB), provide personalization (utilizing MovieLens),
carry short conversations about the two, and finally to perform on natural
dialogs from Reddit. We provide a dataset covering 75k movie entities and with
3.5M training examples. We present results of various models on these tasks,
and evaluate their performance.
Right whale recognition using convolutional neural networks
Authors: Andrei Polzounov, Ilmira Terpugova, Deividas Skiparis, Andrei Mihai
Date: 2016-04-19 00:00:00
Tasks: Face Recognition, General Classification

We studied the feasibility of recognizing individual right whales (Eubalaena
glacialis) using convolutional neural networks. Prior studies have shown that
CNNs can be used in wide range of classification and categorization tasks such
as automated human face recognition. To test applicability of deep learning to
whale recognition we have developed several models based on best practices from
literature. Here, we describe the performance of the models. We conclude that
machine recognition of whales is feasible and comment on the difficulty of the
problem
Robust Scene Text Recognition with Automatic Rectification
Authors: Baoguang Shi, Xinggang Wang, Pengyuan Lyu, Cong Yao, Xiang Bai
Date: 2016-03-12 00:00:00
Proceeding: CVPR 2016 6
Tasks: Optical Character Recognition (OCR), Scene Text Detection, Scene Text Recognition

Recognizing text in natural images is a challenging task with many unsolved
problems. Different from those in documents, words in natural images often
possess irregular shapes, which are caused by perspective distortion, curved
character placement, etc. We propose RARE (Robust text recognizer with
Automatic REctification), a recognition model that is robust to irregular text.
RARE is a specially-designed deep neural network, which consists of a Spatial
Transformer Network (STN) and a Sequence Recognition Network (SRN). In testing,
an image is firstly rectified via a predicted Thin-Plate-Spline (TPS)
transformation, into a more "readable" image for the following SRN, which
recognizes text through a sequence recognition approach. We show that the model
is able to recognize several types of irregular text, including perspective
text and curved text. RARE is end-to-end trainable, requiring only images and
associated text labels, making it convenient to train and deploy the model in
practical systems. State-of-the-art or highly-competitive performance achieved
on several benchmarks well demonstrates the effectiveness of the proposed
model.
End-to-End Tracking and Semantic Segmentation Using Recurrent Neural Networks
Authors: Peter Ondruska, Julie Dequaire, Dominic Zeng Wang, Ingmar Posner
Date: 2016-04-18 00:00:00
Tasks: Classification, General Classification, Semantic Segmentation

In this work we present a novel end-to-end framework for tracking and
classifying a robot's surroundings in complex, dynamic and only partially
observable real-world environments. The approach deploys a recurrent neural
network to filter an input stream of raw laser measurements in order to
directly infer object locations, along with their identity in both visible and
occluded areas. To achieve this we first train the network using unsupervised
Deep Tracking, a recently proposed theoretical framework for end-to-end space
occupancy prediction. We show that by learning to track on a large amount of
unsupervised data, the network creates a rich internal representation of its
environment which we in turn exploit through the principle of inductive
transfer of knowledge to perform the task of it's semantic classification. As a
result, we show that only a small amount of labelled data suffices to steer the
network towards mastering this additional task. Furthermore we propose a novel
recurrent neural network architecture specifically tailored to tracking and
semantic classification in real-world robotics applications. We demonstrate the
tracking and classification performance of the method on real-world data
collected at a busy road junction. Our evaluation shows that the proposed
end-to-end framework compares favourably to a state-of-the-art, model-free
tracking solution and that it outperforms a conventional one-shot training
scheme for semantic classification.
Using Apache Lucene to Search Vector of Locally Aggregated Descriptors
Authors: Giuseppe Amato, Paolo Bolettieri, Fabrizio Falchi, Claudio Gennaro, Lucia Vadicamo
Date: 2016-04-19 00:00:00

Surrogate Text Representation (STR) is a profitable solution to efficient
similarity search on metric space using conventional text search engines, such
as Apache Lucene. This technique is based on comparing the permutations of some
reference objects in place of the original metric distance. However, the
Achilles heel of STR approach is the need to reorder the result set of the
search according to the metric distance. This forces to use a support database
to store the original objects, which requires efficient random I/O on a fast
secondary memory (such as flash-based storages). In this paper, we propose to
extend the Surrogate Text Representation to specifically address a class of
visual metric objects known as Vector of Locally Aggregated Descriptors (VLAD).
This approach is based on representing the individual sub-vectors forming the
VLAD vector with the STR, providing a finer representation of the vector and
enabling us to get rid of the reordering phase. The experiments on a publicly
available dataset show that the extended STR outperforms the baseline STR
achieving satisfactory performance near to the one obtained with the original
VLAD vectors.
Automatic Content-aware Non-Photorealistic Rendering of Images
Authors: Akshay Gadi Patil, Shanmuganathan Raman
Date: 2016-04-07 00:00:00

Non-photorealistic rendering techniques work on image features and often
manipulate a set of characteristics such as edges and texture to achieve a
desired depiction of the scene. Most computational photography methods
decompose an image using edge preserving filters and work on the resulting base
and detail layers independently to achieve desired visual effects. We propose a
new approach for content-aware non-photorealistic rendering of images where we
manipulate the visually salient and the non-salient regions separately. We
propose a novel content-aware framework in order to render an image for
applications such as detail exaggeration, artificial blurring and image
abstraction. The processed regions of the image are blended seamlessly for all
these applications. We demonstrate that content awareness of the proposed
method leads to automatic generation of non-photorealistic rendering of the
same image for the different applications mentioned above.
An Attentive Neural Architecture for Fine-grained Entity Type Classification
Authors: Sonse Shimaoka, Pontus Stenetorp, Kentaro Inui, Sebastian Riedel
Date: 2016-04-19 00:00:00
Proceeding: WS 2016 6
Tasks: Classification, General Classification, Vocal Bursts Type Prediction

In this work we propose a novel attention-based neural network model for the
task of fine-grained entity type classification that unlike previously proposed
models recursively composes representations of entity mention contexts. Our
model achieves state-of-the-art performance with 74.94% loose micro F1-score on
the well-established FIGER dataset, a relative improvement of 2.59%. We also
investigate the behavior of the attention mechanism of our model and observe
that it can learn contextual linguistic expressions that indicate the
fine-grained category memberships of an entity.
Exploring Segment Representations for Neural Segmentation Models
Authors: Yijia Liu, Wanxiang Che, Jiang Guo, Bing Qin, Ting Liu
Date: 2016-04-19 00:00:00
Tasks: Chinese Word Segmentation, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Segmentation

Many natural language processing (NLP) tasks can be generalized into
segmentation problem. In this paper, we combine semi-CRF with neural network to
solve NLP segmentation tasks. Our model represents a segment both by composing
the input units and embedding the entire segment. We thoroughly study different
composition functions and different segment embeddings. We conduct extensive
experiments on two typical segmentation tasks: named entity recognition (NER)
and Chinese word segmentation (CWS). Experimental results show that our neural
semi-CRF model benefits from representing the entire segment and achieves the
state-of-the-art performance on CWS benchmark dataset and competitive results
on the CoNLL03 dataset.
Deep Saliency with Encoded Low level Distance Map and High Level Features
Authors: Gayoung Lee, Yu-Wing Tai, Junmo Kim
Date: 2016-04-19 00:00:00
Proceeding: CVPR 2016 6
Tasks: Deep Learning, Saliency Detection

Recent advances in saliency detection have utilized deep learning to obtain
high level features to detect salient regions in a scene. These advances have
demonstrated superior results over previous works that utilize hand-crafted low
level features for saliency detection. In this paper, we demonstrate that
hand-crafted features can provide complementary information to enhance
performance of saliency detection that utilizes only high level features. Our
method utilizes both high level and low level features for saliency detection
under a unified deep learning framework. The high level features are extracted
using the VGG-net, and the low level features are compared with other parts of
an image to form a low level distance map. The low level distance map is then
encoded using a convolutional neural network(CNN) with multiple 1X1
convolutional and ReLU layers. We concatenate the encoded low level distance
map and the high level features, and connect them to a fully connected neural
network classifier to evaluate the saliency of a query region. Our experiments
show that our method can further improve the performance of state-of-the-art
deep learning-based saliency detection methods.
Understanding Rating Behaviour and Predicting Ratings by Identifying Representative Users
Authors: Rahul Kamath, Masanao Ochi, Yutaka Matsuo
Date: 2016-04-19 00:00:00
Proceeding: PACLIC 2015 10

Online user reviews describing various products and services are now abundant
on the web. While the information conveyed through review texts and ratings is
easily comprehensible, there is a wealth of hidden information in them that is
not immediately obvious. In this study, we unlock this hidden value behind user
reviews to understand the various dimensions along which users rate products.
We learn a set of users that represent each of these dimensions and use their
ratings to predict product ratings. Specifically, we work with restaurant
reviews to identify users whose ratings are influenced by dimensions like
'Service', 'Atmosphere' etc. in order to predict restaurant ratings and
understand the variation in rating behaviour across different cuisines. While
previous approaches to obtaining product ratings require either a large number
of user ratings or a few review texts, we show that it is possible to predict
ratings with few user ratings and no review text. Our experiments show that our
approach outperforms other conventional methods by 16-27% in terms of RMSE.
Estimation of Passenger Route Choice Pattern Using Smart Card Data for Complex Metro Systems
Authors: Juanjuan Zhao, Fan Zhang, Lai Tu, Cheng-Zhong Xu, Dayong Shen, Chen Tian, Xiang-Yang Li, Zhengxi Li
Date: 2016-04-19 00:00:00
Tasks: Management

Nowadays, metro systems play an important role in meeting the urban
transportation demand in large cities. The understanding of passenger route
choice is critical for public transit management. The wide deployment of
Automated Fare Collection(AFC) systems opens up a new opportunity. However,
only each trip's tap-in and tap-out timestamp and stations can be directly
obtained from AFC system records; the train and route chosen by a passenger are
unknown, which are necessary to solve our problem. While existing methods work
well in some specific situations, they don't work for complicated situations.
In this paper, we propose a solution that needs no additional equipment or
human involvement than the AFC systems. We develop a probabilistic model that
can estimate from empirical analysis how the passenger flows are dispatched to
different routes and trains. We validate our approach using a large scale data
set collected from the Shenzhen metro system. The measured results provide us
with useful inputs when building the passenger path choice model.
An Online Structural Plasticity Rule for Generating Better Reservoirs
Authors: Subhrajit Roy, Arindam Basu
Date: 2016-04-19 00:00:00

In this article, a novel neuro-inspired low-resolution online unsupervised
learning rule is proposed to train the reservoir or liquid of Liquid State
Machine. The liquid is a sparsely interconnected huge recurrent network of
spiking neurons. The proposed learning rule is inspired from structural
plasticity and trains the liquid through formation and elimination of synaptic
connections. Hence, the learning involves rewiring of the reservoir connections
similar to structural plasticity observed in biological neural networks. The
network connections can be stored as a connection matrix and updated in memory
by using Address Event Representation (AER) protocols which are generally
employed in neuromorphic systems. On investigating the 'pairwise separation
property' we find that trained liquids provide 1.36 $\pm$ 0.18 times more
inter-class separation while retaining similar intra-class separation as
compared to random liquids. Moreover, analysis of the 'linear separation
property' reveals that trained liquids are 2.05 $\pm$ 0.27 times better than
random liquids. Furthermore, we show that our liquids are able to retain the
'generalization' ability and 'generality' of random liquids. A memory analysis
shows that trained liquids have 83.67 $\pm$ 5.79 ms longer fading memory than
random liquids which have shown 92.8 $\pm$ 5.03 ms fading memory for a
particular type of spike train inputs. We also throw some light on the dynamics
of the evolution of recurrent connections within the liquid. Moreover, compared
to 'Separation Driven Synaptic Modification' - a recently proposed algorithm
for iteratively refining reservoirs, our learning rule provides 9.30%, 15.21%
and 12.52% more liquid separations and 2.8%, 9.1% and 7.9% better
classification accuracies for four, eight and twelve class pattern recognition
tasks respectively.
Parts for the Whole: The DCT Norm for Extreme Visual Recovery
Authors: Yunhe Wang, Chang Xu, Shan You, DaCheng Tao, Chao Xu
Date: 2016-04-19 00:00:00
Tasks: Missing Values

Here we study the extreme visual recovery problem, in which over 90\% of
pixel values in a given image are missing. Existing low rank-based algorithms
are only effective for recovering data with at most 90\% missing values. Thus,
we exploit visual data's smoothness property to help solve this challenging
extreme visual recovery problem. Based on the Discrete Cosine Transformation
(DCT), we propose a novel DCT norm that involves all pixels and produces smooth
estimations in any view. Our theoretical analysis shows that the total
variation (TV) norm, which only achieves local smoothness, is a special case of
the proposed DCT norm. We also develop a new visual recovery algorithm by
minimizing the DCT and nuclear norms to achieve a more visually pleasing
estimation. Experimental results on a benchmark image dataset demonstrate that
the proposed approach is superior to state-of-the-art methods in terms of peak
signal-to-noise ratio and structural similarity.
Streaming Label Learning for Modeling Labels on the Fly
Authors: Shan You, Chang Xu, Yunhe Wang, Chao Xu, DaCheng Tao
Date: 2016-04-19 00:00:00
Tasks: Multi-Label Learning

It is challenging to handle a large volume of labels in multi-label learning.
However, existing approaches explicitly or implicitly assume that all the
labels in the learning process are given, which could be easily violated in
changing environments. In this paper, we define and study streaming label
learning (SLL), i.e., labels are arrived on the fly, to model newly arrived
labels with the help of the knowledge learned from past labels. The core of SLL
is to explore and exploit the relationships between new labels and past labels
and then inherit the relationship into hypotheses of labels to boost the
performance of new classifiers. In specific, we use the label
self-representation to model the label relationship, and SLL will be divided
into two steps: a regression problem and a empirical risk minimization (ERM)
problem. Both problems are simple and can be efficiently solved. We further
show that SLL can generate a tighter generalization error bound for new labels
than the general ERM framework with trace norm or Frobenius norm
regularization. Finally, we implement extensive experiments on various
benchmark datasets to validate the new setting. And results show that SLL can
effectively handle the constantly emerging new labels and provides excellent
classification performance.