pwc-archive/papers-with-abstracts huggingface dataset:
Traffic scene recognition based on deep cnn and vlad spatial pyramids
Authors: Fang-Yu Wu, Shi-Yang Yan, Jeremy S. Smith, Bai-Ling Zhang
Date: 2017-07-24 00:00:00
Tasks: Region Proposal, Scene Classification, Scene Recognition

Traffic scene recognition is an important and challenging issue in
Intelligent Transportation Systems (ITS). Recently, Convolutional Neural
Network (CNN) models have achieved great success in many applications,
including scene classification. The remarkable representational learning
capability of CNN remains to be further explored for solving real-world
problems. Vector of Locally Aggregated Descriptors (VLAD) encoding has also
proved to be a powerful method in catching global contextual information. In
this paper, we attempted to solve the traffic scene recognition problem by
combining the features representational capabilities of CNN with the VLAD
encoding scheme. More specifically, the CNN features of image patches generated
by a region proposal algorithm are encoded by applying VLAD, which subsequently
represent an image in a compact representation. To catch the spatial
information, spatial pyramids are exploited to encode CNN features. We
experimented with a dataset of 10 categories of traffic scenes, with
satisfactory categorization performances.
Toward Geometric Deep SLAM
Authors: Daniel DeTone, Tomasz Malisiewicz, Andrew Rabinovich
Date: 2017-07-24 00:00:00
Tasks: CPU, Point Tracking

We present a point tracking system powered by two deep convolutional neural
networks. The first network, MagicPoint, operates on single images and extracts
salient 2D points. The extracted points are "SLAM-ready" because they are by
design isolated and well-distributed throughout the image. We compare this
network against classical point detectors and discover a significant
performance gap in the presence of image noise. As transformation estimation is
more simple when the detected points are geometrically stable, we designed a
second network, MagicWarp, which operates on pairs of point images (outputs of
MagicPoint), and estimates the homography that relates the inputs. This
transformation engine differs from traditional approaches because it does not
use local point descriptors, only point locations. Both networks are trained
with simple synthetic data, alleviating the requirement of expensive external
camera ground truthing and advanced graphics rendering pipelines. The system is
fast and lean, easily running 30+ FPS on a single CPU.
Wavelet Convolutional Neural Networks for Texture Classification
Authors: Shin Fujieda, Kohei Takayama, Toshiya Hachisuka
Date: 2017-07-24 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Texture Classification

Texture classification is an important and challenging problem in many image
processing applications. While convolutional neural networks (CNNs) achieved
significant successes for image classification, texture classification remains
a difficult problem since textures usually do not contain enough information
regarding the shape of object. In image processing, texture classification has
been traditionally studied well with spectral analyses which exploit repeated
structures in many textures. Since CNNs process images as-is in the spatial
domain whereas spectral analyses process images in the frequency domain, these
models have different characteristics in terms of performance. We propose a
novel CNN architecture, wavelet CNNs, which integrates a spectral analysis into
CNNs. Our insight is that the pooling layer and the convolution layer can be
viewed as a limited form of a spectral analysis. Based on this insight, we
generalize both layers to perform a spectral analysis with wavelet transform.
Wavelet CNNs allow us to utilize spectral information which is lost in
conventional CNNs but useful in texture classification. The experiments
demonstrate that our model achieves better accuracy in texture classification
than existing models. We also show that our model has significantly fewer
parameters than CNNs, making our model easier to train with less memory.
Semantic 3D Occupancy Mapping through Efficient High Order CRFs
Authors: Shichao Yang, Yulan Huang, Sebastian Scherer
Date: 2017-07-24 00:00:00
Tasks: Robot Navigation, Segmentation, Semantic Segmentation, Superpixels, Vocal Bursts Intensity Prediction

Semantic 3D mapping can be used for many applications such as robot
navigation and virtual interaction. In recent years, there has been great
progress in semantic segmentation and geometric 3D mapping. However, it is
still challenging to combine these two tasks for accurate and large-scale
semantic mapping from images. In the paper, we propose an incremental and
(near) real-time semantic mapping system. A 3D scrolling occupancy grid map is
built to represent the world, which is memory and computationally efficient and
bounded for large scale environments. We utilize the CNN segmentation as prior
prediction and further optimize 3D grid labels through a novel CRF model.
Superpixels are utilized to enforce smoothness and form robust P N high order
potential. An efficient mean field inference is developed for the graph
optimization. We evaluate our system on the KITTI dataset and improve the
segmentation accuracy by 10% over existing systems.
A spatiotemporal model with visual attention for video classification
Authors: Mo Shan, Nikolay Atanasov
Date: 2017-07-07 00:00:00
Tasks: Classification, General Classification, Object, object-detection, Object Detection, Video Classification

High level understanding of sequential visual input is important for safe and
stable autonomy, especially in localization and object detection. While
traditional object classification and tracking approaches are specifically
designed to handle variations in rotation and scale, current state-of-the-art
approaches based on deep learning achieve better performance. This paper
focuses on developing a spatiotemporal model to handle videos containing moving
objects with rotation and scale changes. Built on models that combine
Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to
classify sequential data, this work investigates the effectiveness of
incorporating attention modules in the CNN stage for video classification. The
superiority of the proposed spatiotemporal model is demonstrated on the Moving
MNIST dataset augmented with rotation and scaling.
Super-Trajectory for Video Segmentation
Authors: Wenguan Wang, Jianbing Shen, Jianwen Xie, Fatih Porikli
Date: 2017-02-28 00:00:00
Proceeding: ICCV 2017 10
Tasks: Clustering, Segmentation, Video Segmentation, Video Semantic Segmentation

We introduce a novel semi-supervised video segmentation approach based on an
efficient video representation, called as "super-trajectory". Each
super-trajectory corresponds to a group of compact trajectories that exhibit
consistent motion patterns, similar appearance and close spatiotemporal
relationships. We generate trajectories using a probabilistic model, which
handles occlusions and drifts in a robust and natural way. To reliably group
trajectories, we adopt a modified version of the density peaks based clustering
algorithm that allows capturing rich spatiotemporal relations among
trajectories in the clustering process. The presented video representation is
discriminative enough to accurately propagate the initial annotations in the
first frame onto the remaining video frames. Extensive experimental analysis on
challenging benchmarks demonstrate our method is capable of distinguishing the
target objects from complex backgrounds and even reidentifying them after
long-term occlusions.
Compact Model Representation for 3D Reconstruction
Authors: Jhony K. Pontes, Chen Kong, Anders Eriksson, Clinton Fookes, Sridha Sridharan, Simon Lucey
Date: 2017-07-23 00:00:00
Tasks: 3D Reconstruction, model

3D reconstruction from 2D images is a central problem in computer vision.
Recent works have been focusing on reconstruction directly from a single image.
It is well known however that only one image cannot provide enough information
for such a reconstruction. A prior knowledge that has been entertained are 3D
CAD models due to its online ubiquity. A fundamental question is how to
compactly represent millions of CAD models while allowing generalization to new
unseen objects with fine-scaled geometry. We introduce an approach to compactly
represent a 3D mesh. Our method first selects a 3D model from a graph structure
by using a novel free-form deformation FFD 3D-2D registration, and then the
selected 3D model is refined to best fit the image silhouette. We perform a
comprehensive quantitative and qualitative analysis that demonstrates
impressive dense and realistic 3D reconstruction from single images.
Event Coreference Resolution by Iteratively Unfolding Inter-dependencies among Events
Authors: Prafulla Kumar Choubey, Ruihong Huang
Date: 2017-07-23 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Clustering, coreference-resolution, Coreference Resolution, Event Coreference Resolution

We introduce a novel iterative approach for event coreference resolution that
gradually builds event clusters by exploiting inter-dependencies among event
mentions within the same chain as well as across event chains. Among event
mentions in the same chain, we distinguish within- and cross-document event
coreference links by using two distinct pairwise classifiers, trained
separately to capture differences in feature distributions of within- and
cross-document event clusters. Our event coreference approach alternates
between WD and CD clustering and combines arguments from both event clusters
after every merge, continuing till no more merge can be made. And then it
performs further merging between event chains that are both closely related to
a set of other chains of events. Experiments on the ECB+ corpus show that our
model outperforms state-of-the-art methods in joint task of WD and CD event
coreference resolution.
A Sequential Model for Classifying Temporal Relations between Intra-Sentence Events
Authors: Prafulla Kumar Choubey, Ruihong Huang
Date: 2017-07-23 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Relation, Relation Classification, Sentence, TAG, Temporal Relation Classification

We present a sequential model for temporal relation classification between
intra-sentence events. The key observation is that the overall syntactic
structure and compositional meanings of the multi-word context between events
are important for distinguishing among fine-grained temporal relations.
Specifically, our approach first extracts a sequence of context words that
indicates the temporal relation between two events, which well align with the
dependency path between two event mentions. The context word sequence, together
with a parts-of-speech tag sequence and a dependency relation sequence that are
generated corresponding to the word sequence, are then provided as input to
bidirectional recurrent neural network (LSTM) models. The neural nets learn
compositional syntactic and semantic representations of contexts surrounding
the two events and predict the temporal relation between them. Evaluation of
the proposed approach on TimeBank corpus shows that sequential modeling is
capable of accurately recognizing temporal relations between events, which
outperforms a neural net model using various discrete features as input that
imitates previous feature based models.
Prediction-Constrained Training for Semi-Supervised Mixture and Topic Models
Authors: Michael C. Hughes, Leah Weiner, Gabriel Hope, Thomas H. McCoy Jr., Roy H. Perlis, Erik B. Sudderth, Finale Doshi-Velez
Date: 2017-07-23 00:00:00
Tasks: Prediction, Sentiment Analysis, Topic Models

Supervisory signals have the potential to make low-dimensional data
representations, like those learned by mixture and topic models, more
interpretable and useful. We propose a framework for training latent variable
models that explicitly balances two goals: recovery of faithful generative
explanations of high-dimensional data, and accurate prediction of associated
semantic labels. Existing approaches fail to achieve these goals due to an
incomplete treatment of a fundamental asymmetry: the intended application is
always predicting labels from data, not data from labels. Our
prediction-constrained objective for training generative models coherently
integrates loss-based supervisory signals while enabling effective
semi-supervised learning from partially labeled data. We derive learning
algorithms for semi-supervised mixture and topic models using stochastic
gradient descent with automatic differentiation. We demonstrate improved
prediction quality compared to several previous supervised topic models,
achieving predictions competitive with high-dimensional logistic regression on
text sentiment analysis and electronic health records tasks while
simultaneously learning interpretable topics.
Deep Multi-camera People Detection
Authors: Tatjana Chavdarova, François Fleuret
Date: 2017-02-15 00:00:00

This paper addresses the problem of multi-view people occupancy map
estimation. Existing solutions for this problem either operate per-view, or
rely on a background subtraction pre-processing. Both approaches lessen the
detection performance as scenes become more crowded. The former does not
exploit joint information, whereas the latter deals with ambiguous input due to
the foreground blobs becoming more and more interconnected as the number of
targets increases.
  Although deep learning algorithms have proven to excel on remarkably numerous
computer vision tasks, such a method has not been applied yet to this problem.
In large part this is due to the lack of large-scale multi-camera data-set.
  The core of our method is an architecture which makes use of monocular
pedestrian data-set, available at larger scale then the multi-view ones,
applies parallel processing to the multiple video streams, and jointly utilises
it. Our end-to-end deep learning method outperforms existing methods by large
margins on the commonly used PETS 2009 data-set. Furthermore, we make publicly
available a new three-camera HD data-set. Our source code and trained models
will be made available under an open-source license.
Rule-Based Spanish Morphological Analyzer Built From Spell Checking Lexicon
Authors: Natalie Ahn
Date: 2017-07-23 00:00:00
Tasks: Morphological Analysis, valid

Preprocessing tools for automated text analysis have become more widely
available in major languages, but non-English tools are often still limited in
their functionality. When working with Spanish-language text, researchers can
easily find tools for tokenization and stemming, but may not have the means to
extract more complex word features like verb tense or mood. Yet Spanish is a
morphologically rich language in which such features are often identifiable
from word form. Conjugation rules are consistent, but many special verbs and
nouns take on different rules. While building a complete dictionary of known
words and their morphological rules would be labor intensive, resources to do
so already exist, in spell checkers designed to generate valid forms of known
words. This paper introduces a set of tools for Spanish-language morphological
analysis, built using the COES spell checking tools, to label person, mood,
tense, gender and number, derive a word's root noun or verb infinitive, and
convert verbs to their nominal form.
Adversarial Examples for Evaluating Reading Comprehension Systems
Authors: Robin Jia, Percy Liang
Date: 2017-07-23 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Question Answering, Reading Comprehension

Standard accuracy metrics indicate that reading comprehension systems are
making rapid progress, but the extent to which these systems truly understand
language remains unclear. To reward systems with real language understanding
abilities, we propose an adversarial evaluation scheme for the Stanford
Question Answering Dataset (SQuAD). Our method tests whether systems can answer
questions about paragraphs that contain adversarially inserted sentences, which
are automatically generated to distract computer systems without changing the
correct answer or misleading humans. In this adversarial setting, the accuracy
of sixteen published models drops from an average of $75\%$ F1 score to $36\%$;
when the adversary is allowed to add ungrammatical sequences of words, average
accuracy on four models decreases further to $7\%$. We hope our insights will
motivate the development of new models that understand language more precisely.
Count-ception: Counting by Fully Convolutional Redundant Counting
Authors: Joseph Paul Cohen, Genevieve Boucher, Craig A. Glastonbury, Henry Z. Lo, Yoshua Bengio
Date: 2017-03-25 00:00:00
Tasks: Object Localization, regression

Counting objects in digital images is a process that should be replaced by
machines. This tedious task is time consuming and prone to errors due to
fatigue of human annotators. The goal is to have a system that takes as input
an image and returns a count of the objects inside and justification for the
prediction in the form of object localization. We repose a problem, originally
posed by Lempitsky and Zisserman, to instead predict a count map which contains
redundant counts based on the receptive field of a smaller regression network.
The regression network predicts a count of the objects that exist inside this
frame. By processing the image in a fully convolutional way each pixel is going
to be accounted for some number of times, the number of windows which include
it, which is the size of each window, (i.e., 32x32 = 1024). To recover the true
count we take the average over the redundant predictions. Our contribution is
redundant counting instead of predicting a density map in order to average over
errors. We also propose a novel deep neural network architecture adapted from
the Inception family of networks called the Count-ception network. Together our
approach results in a 20% relative improvement (2.9 to 2.3 MAE) over the state
of the art method by Xie, Noble, and Zisserman in 2016.
Exploiting Deep Features for Remote Sensing Image Retrieval: A Systematic Investigation
Authors: Xin-Yi Tong, Gui-Song Xia, Fan Hu, Yanfei Zhong, Mihai Datcu, Liangpei Zhang
Date: 2017-07-23 00:00:00
Tasks: Image Retrieval, Retrieval

Remote sensing (RS) image retrieval is of great significant for geological information mining. Over the past two decades, a large amount of research on this task has been carried out, which mainly focuses on the following three core issues: feature extraction, similarity metric and relevance feedback. Due to the complexity and multiformity of ground objects in high-resolution remote sensing (HRRS) images, there is still room for improvement in the current retrieval approaches. In this paper, we analyze the three core issues of RS image retrieval and provide a comprehensive review on existing methods. Furthermore, for the goal to advance the state-of-the-art in HRRS image retrieval, we focus on the feature extraction issue and delve how to use powerful deep representations to address this task. We conduct systematic investigation on evaluating correlative factors that may affect the performance of deep features. By optimizing each factor, we acquire remarkable retrieval results on publicly available HRRS datasets. Finally, we explain the experimental phenomenon in detail and draw conclusions according to our analysis. Our work can serve as a guiding role for the research of content-based RS image retrieval.
Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments
Authors: Jingwei Zhang, Jost Tobias Springenberg, Joschka Boedecker, Wolfram Burgard
Date: 2016-12-16 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Robot Navigation

In this paper we consider the problem of robot navigation in simple maze-like
environments where the robot has to rely on its onboard sensors to perform the
navigation task. In particular, we are interested in solutions to this problem
that do not require localization, mapping or planning. Additionally, we require
that our solution can quickly adapt to new situations (e.g., changing
navigation goals and environments). To meet these criteria we frame this
problem as a sequence of related reinforcement learning tasks. We propose a
successor feature based deep reinforcement learning algorithm that can learn to
transfer knowledge from previously mastered navigation tasks to new problem
instances. Our algorithm substantially decreases the required learning time
after the first task instance has been solved, which makes it easily adaptable
to changing environments. We validate our method in both simulated and real
robot experiments with a Robotino and compare it to a set of baseline methods
including classical planning-based navigation.
Zero-resource Machine Translation by Multimodal Encoder-decoder Network with Multimedia Pivot
Authors: Hideki Nakayama, Noriki Nishida
Date: 2016-11-14 00:00:00
Tasks: Decoder, Machine Translation, Translation

We propose an approach to build a neural machine translation system with no
supervised resources (i.e., no parallel corpora) using multimodal embedded
representation over texts and images. Based on the assumption that text
documents are often likely to be described with other multimedia information
(e.g., images) somewhat related to the content, we try to indirectly estimate
the relevance between two languages. Using multimedia as the "pivot", we
project all modalities into one common hidden space where samples belonging to
similar semantic concepts should come close to each other, whatever the
observed space of each sample is. This modality-agnostic representation is the
key to bridging the gap between different modalities. Putting a decoder on top
of it, our network can flexibly draw the outputs from any input modality.
Notably, in the testing phase, we need only source language texts as the input
for translation. In experiments, we tested our method on two benchmarks to show
that it can achieve reasonable translation performance. We compared and
investigated several possible implementations and found that an end-to-end
model that simultaneously optimized both rank loss in multimodal encoders and
cross-entropy loss in decoders performed the best.