pwc-archive/papers-with-abstracts huggingface dataset:
Residual Dense Network for Image Super-Resolution
Authors: Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, Yun Fu
Date: 2018-02-24 00:00:00
Proceeding: CVPR 2018 6
Tasks: Color Image Denoising, Image Super-Resolution, Super-Resolution

A very deep convolutional neural network (CNN) has recently achieved great
success for image super-resolution (SR) and offered hierarchical features as
well. However, most deep CNN based SR models do not make full use of the
hierarchical features from the original low-resolution (LR) images, thereby
achieving relatively-low performance. In this paper, we propose a novel
residual dense network (RDN) to address this problem in image SR. We fully
exploit the hierarchical features from all the convolutional layers.
Specifically, we propose residual dense block (RDB) to extract abundant local
features via dense connected convolutional layers. RDB further allows direct
connections from the state of preceding RDB to all the layers of current RDB,
leading to a contiguous memory (CM) mechanism. Local feature fusion in RDB is
then used to adaptively learn more effective features from preceding and
current local features and stabilizes the training of wider network. After
fully obtaining dense local features, we use global feature fusion to jointly
and adaptively learn global hierarchical features in a holistic way. Extensive
experiments on benchmark datasets with different degradation models show that
our RDN achieves favorable performance against state-of-the-art methods.
MAttNet: Modular Attention Network for Referring Expression Comprehension
Authors: Licheng Yu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, Mohit Bansal, Tamara L. Berg
Date: 2018-01-24 00:00:00
Proceeding: CVPR 2018 6
Tasks: Generalized Referring Expression Segmentation, Referring Expression, Referring Expression Comprehension, Referring Expression Segmentation

In this paper, we address referring expression comprehension: localizing an
image region described by a natural language expression. While most recent work
treats expressions as a single unit, we propose to decompose them into three
modular components related to subject appearance, location, and relationship to
other objects. This allows us to flexibly adapt to expressions containing
different types of information in an end-to-end framework. In our model, which
we call the Modular Attention Network (MAttNet), two types of attention are
utilized: language-based attention that learns the module weights as well as
the word/phrase attention that each module should focus on; and visual
attention that allows the subject and relationship modules to focus on relevant
image components. Module weights combine scores from all three modules
dynamically to output an overall score. Experiments show that MAttNet
outperforms previous state-of-art methods by a large margin on both
bounding-box-level and pixel-level comprehension tasks. Demo and code are
provided.
Pseudo Mask Augmented Object Detection
Authors: Xiangyun Zhao, Shuang Liang, Yichen Wei
Date: 2018-03-15 00:00:00
Proceeding: CVPR 2018 6
Tasks: Instance Segmentation, Object, object-detection, Object Detection, Segmentation, Semantic Segmentation, Weakly supervised segmentation

In this work, we present a novel and effective framework to facilitate object
detection with the instance-level segmentation information that is only
supervised by bounding box annotation. Starting from the joint object detection
and instance segmentation network, we propose to recursively estimate the
pseudo ground-truth object masks from the instance-level object segmentation
network training, and then enhance the detection network with top-down
segmentation feedbacks. The pseudo ground truth mask and network parameters are
optimized alternatively to mutually benefit each other. To obtain the promising
pseudo masks in each iteration, we embed a graphical inference that
incorporates the low-level image appearance consistency and the bounding box
annotations to refine the segmentation masks predicted by the segmentation
network. Our approach progressively improves the object detection performance
by incorporating the detailed pixel-wise information learned from the
weakly-supervised segmentation network. Extensive evaluation on the detection
task in PASCAL VOC 2007 and 2012 [12] verifies that the proposed approach is
effective.
Generative Design in Minecraft (GDMC), Settlement Generation Competition
Authors: Christoph Salge, Michael Cerny Green, Rodrigo Canaan, Julian Togelius
Date: 2018-03-27 00:00:00
Tasks: Minecraft

This paper introduces the settlement generation competition for Minecraft,
the first part of the Generative Design in Minecraft challenge. The settlement
generation competition is about creating Artificial Intelligence (AI) agents
that can produce functional, aesthetically appealing and believable settlements
adapted to a given Minecraft map - ideally at a level that can compete with
human created designs. The aim of the competition is to advance procedural
content generation for games, especially in overcoming the challenges of
adaptive and holistic PCG. The paper introduces the technical details of the
challenge, but mostly focuses on what challenges this competition provides and
why they are scientifically relevant.
Attributes as Operators: Factorizing Unseen Attribute-Object Compositions
Authors: Tushar Nagarajan, Kristen Grauman
Date: 2018-03-27 00:00:00
Proceeding: ECCV 2018 9
Tasks: Attribute, Compositional Zero-Shot Learning, Image Retrieval with Multi-Modal Query, Object

We present a new approach to modeling visual attributes. Prior work casts
attributes in a similar role as objects, learning a latent representation where
properties (e.g., sliced) are recognized by classifiers much in the way objects
(e.g., apple) are. However, this common approach fails to separate the
attributes observed during training from the objects with which they are
composed, making it ineffectual when encountering new attribute-object
compositions. Instead, we propose to model attributes as operators. Our
approach learns a semantic embedding that explicitly factors out attributes
from their accompanying objects, and also benefits from novel regularizers
expressing attribute operators' effects (e.g., blunt should undo the effects of
sharp). Not only does our approach align conceptually with the linguistic role
of attributes as modifiers, but it also generalizes to recognize unseen
compositions of objects and attributes. We validate our approach on two
challenging datasets and demonstrate significant improvements over the
state-of-the-art. In addition, we show that not only can our model recognize
unseen compositions robustly in an open-world setting, it can also generalize
to compositions where objects themselves were unseen during training.
Hierarchical Metric Learning and Matching for 2D and 3D Geometric Correspondences
Authors: Mohammed E. Fathy, Quoc-Huy Tran, M. Zeeshan Zia, Paul Vernaza, Manmohan Chandraker
Date: 2018-03-20 00:00:00
Proceeding: ECCV 2018 9
Tasks: Geometric Matching, Metric Learning, Optical Flow Estimation

Interest point descriptors have fueled progress on almost every problem in
computer vision. Recent advances in deep neural networks have enabled
task-specific learned descriptors that outperform hand-crafted descriptors on
many problems. We demonstrate that commonly used metric learning approaches do
not optimally leverage the feature hierarchies learned in a Convolutional
Neural Network (CNN), especially when applied to the task of geometric feature
matching. While a metric loss applied to the deepest layer of a CNN, is often
expected to yield ideal features irrespective of the task, in fact the growing
receptive field as well as striding effects cause shallower features to be
better at high precision matching tasks. We leverage this insight together with
explicit supervision at multiple levels of the feature hierarchy for better
regularization, to learn more effective descriptors in the context of geometric
matching tasks. Further, we propose to use activation maps at different layers
of a CNN, as an effective and principled replacement for the multi-resolution
image pyramids often used for matching tasks. We propose concrete CNN
architectures employing these ideas, and evaluate them on multiple datasets for
2D and 3D geometric matching as well as optical flow, demonstrating
state-of-the-art results and generalization across datasets.
Neural Baby Talk
Authors: Jiasen Lu, Jianwei Yang, Dhruv Batra, Devi Parikh
Date: 2018-03-27 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Captioning, Object, Sentence, slot-filling, Slot Filling

We introduce a novel framework for image captioning that can produce natural
language explicitly grounded in entities that object detectors find in the
image. Our approach reconciles classical slot filling approaches (that are
generally better grounded in images) with modern neural captioning approaches
(that are generally more natural sounding and accurate). Our approach first
generates a sentence `template' with slot locations explicitly tied to specific
image regions. These slots are then filled in by visual concepts identified in
the regions by object detectors. The entire architecture (sentence template
generation and slot filling with object detectors) is end-to-end
differentiable. We verify the effectiveness of our proposed model on different
image captioning tasks. On standard image captioning and novel object
captioning, our model reaches state-of-the-art on both COCO and Flickr30k
datasets. We also demonstrate that our model has unique advantages when the
train and test distributions of scene compositions -- and hence language priors
of associated captions -- are different. Code has been made available at:
https://github.com/jiasenlu/NeuralBabyTalk
Heat Kernel analysis of Syntactic Structures
Authors: Andrew Ortegaray, Robert C. Berwick, Matilde Marcolli
Date: 2018-03-26 00:00:00
Tasks: Clustering

We consider two different data sets of syntactic parameters and we discuss
how to detect relations between parameters through a heat kernel method
developed by Belkin-Niyogi, which produces low dimensional representations of
the data, based on Laplace eigenfunctions, that preserve neighborhood
information. We analyze the different connectivity and clustering structures
that arise in the two datasets, and the regions of maximal variance in the
two-parameter space of the Belkin-Niyogi construction, which identify
preferable choices of independent variables. We compute clustering coefficients
and their variance.
Rogue Signs: Deceiving Traffic Sign Recognition with Malicious Ads and Logos
Authors: Chawin Sitawarin, Arjun Nitin Bhagoji, Arsalan Mosenia, Prateek Mittal, Mung Chiang
Date: 2018-01-09 00:00:00
Tasks: Autonomous Vehicles, Traffic Sign Recognition

We propose a new real-world attack against the computer vision based systems
of autonomous vehicles (AVs). Our novel Sign Embedding attack exploits the
concept of adversarial examples to modify innocuous signs and advertisements in
the environment such that they are classified as the adversary's desired
traffic sign with high confidence. Our attack greatly expands the scope of the
threat posed to AVs since adversaries are no longer restricted to just
modifying existing traffic signs as in previous work. Our attack pipeline
generates adversarial samples which are robust to the environmental conditions
and noisy image transformations present in the physical world. We ensure this
by including a variety of possible image transformations in the optimization
problem used to generate adversarial samples. We verify the robustness of the
adversarial samples by printing them out and carrying out drive-by tests
simulating the conditions under which image capture would occur in a real-world
scenario. We experimented with physical attack samples for different distances,
lighting conditions and camera angles. In addition, extensive evaluations were
carried out in the virtual setting for a variety of image transformations. The
adversarial samples generated using our method have adversarial success rates
in excess of 95% in the physical as well as virtual settings.
Low-Shot Learning for the Semantic Segmentation of Remote Sensing Imagery
Authors: Ronald Kemker, Ryan Luu, Christopher Kanan
Date: 2018-03-26 00:00:00
Tasks: Few-Shot Image Classification, Few-Shot Learning, General Classification, Object Recognition, Segmentation, Segmentation Of Remote Sensing Imagery, Semantic Segmentation, The Semantic Segmentation Of Remote Sensing Imagery

Recent advances in computer vision using deep learning with RGB imagery
(e.g., object recognition and detection) have been made possible thanks to the
development of large annotated RGB image datasets. In contrast, multispectral
image (MSI) and hyperspectral image (HSI) datasets contain far fewer labeled
images, in part due to the wide variety of sensors used. These annotations are
especially limited for semantic segmentation, or pixel-wise classification, of
remote sensing imagery because it is labor intensive to generate image
annotations. Low-shot learning algorithms can make effective inferences despite
smaller amounts of annotated data. In this paper, we study low-shot learning
using self-taught feature learning for semantic segmentation. We introduce 1)
an improved self-taught feature learning framework for HSI and MSI data and 2)
a semi-supervised classification algorithm. When these are combined, they
achieve state-of-the-art performance on remote sensing datasets that have
little annotated training data available. These low-shot learning frameworks
will reduce the manual image annotation burden and improve semantic
segmentation performance for remote sensing imagery.
Spectral feature mapping with mimic loss for robust speech recognition
Authors: Deblin Bagchi, Peter Plantinga, Adam Stiff, Eric Fosler-Lussier
Date: 2018-03-26 00:00:00
Tasks: Robust Speech Recognition, Speech Enhancement, speech-recognition, Speech Recognition

For the task of speech enhancement, local learning objectives are agnostic to
phonetic structures helpful for speech recognition. We propose to add a global
criterion to ensure de-noised speech is useful for downstream tasks like ASR.
We first train a spectral classifier on clean speech to predict senone labels.
Then, the spectral classifier is joined with our speech enhancer as a noisy
speech recognizer. This model is taught to imitate the output of the spectral
classifier alone on clean speech. This \textit{mimic loss} is combined with the
traditional local criterion to train the speech enhancer to produce de-noised
speech. Feeding the de-noised speech to an off-the-shelf Kaldi training recipe
for the CHiME-2 corpus shows significant improvements in WER.
Detecting truth, just on parts
Authors: Zoltán Kovács, Tomás Recio, M. Pilar Vélez
Date: 2018-02-16 00:00:00

We introduce and discuss, through a computational algebraic geometry
approach, the automatic reasoning handling of propositions that are
simultaneously true and false over some relevant collections of instances. A
rigorous, algorithmic criterion is presented for detecting such cases, and its
performance is exemplified through the implementation of this test on the
dynamic geometry program GeoGebra.
Deep learning as a tool for neural data analysis: speech classification and cross-frequency coupling in human sensorimotor cortex
Authors: Jesse A. Livezey, Kristofer E. Bouchard, Edward F. Chang
Date: 2018-03-26 00:00:00
Tasks: General Classification

A fundamental challenge in neuroscience is to understand what structure in
the world is represented in spatially distributed patterns of neural activity
from multiple single-trial measurements. This is often accomplished by learning
a simple, linear transformations between neural features and features of the
sensory stimuli or motor task. While successful in some early sensory
processing areas, linear mappings are unlikely to be ideal tools for
elucidating nonlinear, hierarchical representations of higher-order brain areas
during complex tasks, such as the production of speech by humans. Here, we
apply deep networks to predict produced speech syllables from cortical surface
electric potentials recorded from human sensorimotor cortex. We found that deep
networks had higher decoding prediction accuracy compared to baseline models,
and also exhibited greater improvements in accuracy with increasing dataset
size. We further demonstrate that deep network's confusions revealed
hierarchical latent structure in the neural data, which recapitulated the
underlying articulatory nature of speech motor control. Finally, we used deep
networks to compare task-relevant information in different neural frequency
bands, and found that the high-gamma band contains the vast majority of
information relevant for the speech prediction task, with little-to-no
additional contribution from lower-frequencies. Together, these results
demonstrate the utility of deep networks as a data analysis tool for
neuroscience.
Failing to Learn: Autonomously Identifying Perception Failures for Self-driving Cars
Authors: Manikandasriram Srinivasan Ramanagopal, Cyrus Anderson, Ram Vasudevan, Matthew Johnson-Roberson
Date: 2017-06-30 00:00:00
Tasks: Autonomous Driving, Autonomous Vehicles, Navigate, Object, Object Detection, Self-Driving Cars, valid

One of the major open challenges in self-driving cars is the ability to
detect cars and pedestrians to safely navigate in the world. Deep
learning-based object detector approaches have enabled great advances in using
camera imagery to detect and classify objects. But for a safety critical
application, such as autonomous driving, the error rates of the current state
of the art are still too high to enable safe operation. Moreover, the
characterization of object detector performance is primarily limited to testing
on prerecorded datasets. Errors that occur on novel data go undetected without
additional human labels. In this letter, we propose an automated method to
identify mistakes made by object detectors without ground truth labels. We show
that inconsistencies in the object detector output between a pair of similar
images can be used as hypotheses for false negatives (e.g., missed detections)
and using a novel set of features for each hypothesis, an off-the-shelf binary
classifier can be used to find valid errors. In particular, we study two
distinct cues - temporal and stereo inconsistencies - using data that are
readily available on most autonomous vehicles. Our method can be used with any
camera-based object detector and we illustrate the technique on several sets of
real world data. We show that a state-of-the-art detector, tracker, and our
classifier trained only on synthetic data can identify valid errors on KITTI
tracking dataset with an average precision of 0.94. We also release a new
tracking dataset with 104 sequences totaling 80,655 labeled pairs of stereo
images along with ground truth disparity from a game engine to facilitate
further research. The dataset and code are available at
https://fcav.engin.umich.edu/research/failing-to-learn
Enhancing Evolutionary Conversion Rate Optimization via Multi-armed Bandit Algorithms
Authors: Xin Qiu, Risto Miikkulainen
Date: 2018-03-10 00:00:00
Tasks: Evolutionary Algorithms

Conversion rate optimization means designing web interfaces such that more
visitors perform a desired action (such as register or purchase) on the site.
One promising approach, implemented in Sentient Ascend, is to optimize the
design using evolutionary algorithms, evaluating each candidate design online
with actual visitors. Because such evaluations are costly and noisy, several
challenges emerge: How can available visitor traffic be used most efficiently?
How can good solutions be identified most reliably? How can a high conversion
rate be maintained during optimization? This paper proposes a new technique to
address these issues. Traffic is allocated to candidate solutions using a
multi-armed bandit algorithm, using more traffic on those evaluations that are
most useful. In a best-arm identification mode, the best candidate can be
identified reliably at the end of evolution, and in a campaign mode, the
overall conversion rate can be optimized throughout the entire evolution
process. Multi-armed bandit algorithms thus improve performance and reliability
of machine discovery in noisy real-world environments.
On Chatbots Exhibiting Goal-Directed Autonomy in Dynamic Environments
Authors: Biplav Srivastava
Date: 2018-03-26 00:00:00
Tasks: Position

Conversation interfaces (CIs), or chatbots, are a popular form of intelligent
agents that engage humans in task-oriented or informal conversation. In this
position paper and demonstration, we argue that chatbots working in dynamic
environments, like with sensor data, can not only serve as a promising platform
to research issues at the intersection of learning, reasoning, representation
and execution for goal-directed autonomy; but also handle non-trivial business
applications. We explore the underlying issues in the context of Water Advisor,
a preliminary multi-modal conversation system that can access and explain water
quality data.
Transferable Joint Attribute-Identity Deep Learning for Unsupervised Person Re-Identification
Authors: Jingya Wang, Xiatian Zhu, Shaogang Gong, Wei Li
Date: 2018-03-26 00:00:00
Proceeding: CVPR 2018 6
Tasks: Attribute, Deep Learning, Person Re-Identification, Unsupervised Domain Adaptation, Unsupervised Person Re-Identification

Most existing person re-identification (re-id) methods require supervised
model learning from a separate large set of pairwise labelled training data for
every single camera pair. This significantly limits their scalability and
usability in real-world large scale deployments with the need for performing
re-id across many camera views. To address this scalability problem, we develop
a novel deep learning method for transferring the labelled information of an
existing dataset to a new unseen (unlabelled) target domain for person re-id
without any supervised learning in the target domain. Specifically, we
introduce an Transferable Joint Attribute-Identity Deep Learning (TJ-AIDL) for
simultaneously learning an attribute-semantic and identitydiscriminative
feature representation space transferrable to any new (unseen) target domain
for re-id tasks without the need for collecting new labelled training data from
the target domain (i.e. unsupervised learning in the target domain). Extensive
comparative evaluations validate the superiority of this new TJ-AIDL model for
unsupervised person re-id over a wide range of state-of-the-art methods on four
challenging benchmarks including VIPeR, PRID, Market-1501, and DukeMTMC-ReID.
Algorithm Configuration: Learning policies for the quick termination of poor performers
Authors: Daniel Karapetyan, Andrew J. Parkes, Thomas Stützle
Date: 2018-03-26 00:00:00

One way to speed up the algorithm configuration task is to use short runs
instead of long runs as much as possible, but without discarding the
configurations that eventually do well on the long runs. We consider the
problem of selecting the top performing configurations of the Conditional
Markov Chain Search (CMCS), a general algorithm schema that includes, for
examples, VNS. We investigate how the structure of performance on short tests
links with those on long tests, showing that significant differences arise
between test domains. We propose a "performance envelope" method to exploit the
links; that learns when runs should be terminated, but that automatically
adapts to the domain.