pwc-archive/papers-with-abstracts huggingface dataset:
Wasserstein CNN: Learning Invariant Features for NIR-VIS Face Recognition
Authors: Ran He, Xiang Wu, Zhenan Sun, Tieniu Tan
Date: 2017-08-08 00:00:00
Tasks: Face Recognition, Heterogeneous Face Recognition

Heterogeneous face recognition (HFR) aims to match facial images acquired
from different sensing modalities with mission-critical applications in
forensics, security and commercial sectors. However, HFR is a much more
challenging problem than traditional face recognition because of large
intra-class variations of heterogeneous face images and limited training
samples of cross-modality face image pairs. This paper proposes a novel
approach namely Wasserstein CNN (convolutional neural networks, or WCNN for
short) to learn invariant features between near-infrared and visual face images
(i.e. NIR-VIS face recognition). The low-level layers of WCNN are trained with
widely available face images in visual spectrum. The high-level layer is
divided into three parts, i.e., NIR layer, VIS layer and NIR-VIS shared layer.
The first two layers aims to learn modality-specific features and NIR-VIS
shared layer is designed to learn modality-invariant feature subspace.
Wasserstein distance is introduced into NIR-VIS shared layer to measure the
dissimilarity between heterogeneous feature distributions. So W-CNN learning
aims to achieve the minimization of Wasserstein distance between NIR
distribution and VIS distribution for invariant deep feature representation of
heterogeneous face images. To avoid the over-fitting problem on small-scale
heterogeneous face data, a correlation prior is introduced on the
fully-connected layers of WCNN network to reduce parameter space. This prior is
implemented by a low-rank constraint in an end-to-end network. The joint
formulation leads to an alternating minimization for deep feature
representation at training stage and an efficient computation for heterogeneous
data at testing stage. Extensive experiments on three challenging NIR-VIS face
recognition databases demonstrate the significant superiority of Wasserstein
CNN over state-of-the-art methods.
Robust Conditional Probabilities
Authors: Yoav Wald, Amir Globerson
Date: 2017-08-08 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Structured Prediction

Conditional probabilities are a core concept in machine learning. For
example, optimal prediction of a label $Y$ given an input $X$ corresponds to
maximizing the conditional probability of $Y$ given $X$. A common approach to
inference tasks is learning a model of conditional probabilities. However,
these models are often based on strong assumptions (e.g., log-linear models),
and hence their estimate of conditional probabilities is not robust and is
highly dependent on the validity of their assumptions.
  Here we propose a framework for reasoning about conditional probabilities
without assuming anything about the underlying distributions, except knowledge
of their second order marginals, which can be estimated from data. We show how
this setting leads to guaranteed bounds on conditional probabilities, which can
be calculated efficiently in a variety of settings, including
structured-prediction. Finally, we apply them to semi-supervised deep learning,
obtaining results competitive with variational autoencoders.
Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs
Authors: Michael Gygli, Mohammad Norouzi, Anelia Angelova
Date: 2017-03-13 00:00:00
Proceeding: ICML 2017 8
Tasks: General Classification, Image Segmentation, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Segmentation, Semantic Segmentation

We approach structured output prediction by optimizing a deep value network
(DVN) to precisely estimate the task loss on different output configurations
for a given input. Once the model is trained, we perform inference by gradient
descent on the continuous relaxations of the output variables to find outputs
with promising scores from the value network. When applied to image
segmentation, the value network takes an image and a segmentation mask as
inputs and predicts a scalar estimating the intersection over union between the
input and ground truth masks. For multi-label classification, the DVN's
objective is to correctly predict the F1 score for any potential label
configuration. The DVN framework achieves the state-of-the-art results on
multi-label prediction and image segmentation benchmarks.
MHTN: Modal-adversarial Hybrid Transfer Network for Cross-modal Retrieval
Authors: Xin Huang, Yuxin Peng, Mingkuan Yuan
Date: 2017-08-08 00:00:00
Tasks: Cross-Modal Retrieval, Representation Learning, Retrieval, Transfer Learning

Cross-modal retrieval has drawn wide interest for retrieval across different
modalities of data. However, existing methods based on DNN face the challenge
of insufficient cross-modal training data, which limits the training
effectiveness and easily leads to overfitting. Transfer learning is for
relieving the problem of insufficient training data, but it mainly focuses on
knowledge transfer only from large-scale datasets as single-modal source domain
to single-modal target domain. Such large-scale single-modal datasets also
contain rich modal-independent semantic knowledge that can be shared across
different modalities. Besides, large-scale cross-modal datasets are very
labor-consuming to collect and label, so it is significant to fully exploit the
knowledge in single-modal datasets for boosting cross-modal retrieval. This
paper proposes modal-adversarial hybrid transfer network (MHTN), which to the
best of our knowledge is the first work to realize knowledge transfer from
single-modal source domain to cross-modal target domain, and learn cross-modal
common representation. It is an end-to-end architecture with two subnetworks:
(1) Modal-sharing knowledge transfer subnetwork is proposed to jointly transfer
knowledge from a large-scale single-modal dataset in source domain to all
modalities in target domain with a star network structure, which distills
modal-independent supplementary knowledge for promoting cross-modal common
representation learning. (2) Modal-adversarial semantic learning subnetwork is
proposed to construct an adversarial training mechanism between common
representation generator and modality discriminator, making the common
representation discriminative for semantics but indiscriminative for modalities
to enhance cross-modal semantic consistency during transfer process.
Comprehensive experiments on 4 widely-used datasets show its effectiveness and
generality.
Learning a Repression Network for Precise Vehicle Search
Authors: Qiantong Xu, Ke Yan, Yonghong Tian
Date: 2017-08-08 00:00:00
Tasks: Attribute, Multi-Task Learning, Retrieval

The growing explosion in the use of surveillance cameras in public security
highlights the importance of vehicle search from large-scale image databases.
Precise vehicle search, aiming at finding out all instances for a given query
vehicle image, is a challenging task as different vehicles will look very
similar to each other if they share same visual attributes. To address this
problem, we propose the Repression Network (RepNet), a novel multi-task
learning framework, to learn discriminative features for each vehicle image
from both coarse-grained and detailed level simultaneously. Besides, benefited
from the satisfactory accuracy of attribute classification, a bucket search
method is proposed to reduce the retrieval time while still maintaining
competitive performance. We conduct extensive experiments on the revised
VehcileID dataset. Experimental results show that our RepNet achieves the
state-of-the-art performance and the bucket search method can reduce the
retrieval time by about 24 times.
Learning how to Active Learn: A Deep Reinforcement Learning Approach
Authors: Meng Fang, Yuan Li, Trevor Cohn
Date: 2017-08-08 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Active Learning, Deep Reinforcement Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Active learning aims to select a small subset of data for annotation such
that a classifier learned on the data is highly accurate. This is usually done
using heuristic selection methods, however the effectiveness of such methods is
limited and moreover, the performance of heuristics varies between datasets. To
address these shortcomings, we introduce a novel formulation by reframing the
active learning as a reinforcement learning problem and explicitly learning a
data selection policy, where the policy takes the role of the active learning
heuristic. Importantly, our method allows the selection policy learned using
simulation on one language to be transferred to other languages. We demonstrate
our method using cross-lingual named entity recognition, observing uniform
improvements over traditional active learning.
FVQA: Fact-based Visual Question Answering
Authors: Peng Wang, Qi Wu, Chunhua Shen, Anton Van Den Hengel, Anthony Dick
Date: 2016-06-17 00:00:00
Tasks: Common Sense Reasoning, Question Answering, Triplet, Visual Question Answering, Visual Question Answering (VQA)

Visual Question Answering (VQA) has attracted a lot of attention in both
Computer Vision and Natural Language Processing communities, not least because
it offers insight into the relationships between two important sources of
information. Current datasets, and the models built upon them, have focused on
questions which are answerable by direct analysis of the question and image
alone. The set of such questions that require no external information to answer
is interesting, but very limited. It excludes questions which require common
sense, or basic factual knowledge to answer, for example. Here we introduce
FVQA, a VQA dataset which requires, and supports, much deeper reasoning. FVQA
only contains questions which require external information to answer.
  We thus extend a conventional visual question answering dataset, which
contains image-question-answerg triplets, through additional
image-question-answer-supporting fact tuples. The supporting fact is
represented as a structural triplet, such as <Cat,CapableOf,ClimbingTrees>.
  We evaluate several baseline models on the FVQA dataset, and describe a novel
model which is capable of reasoning about an image on the basis of supporting
facts.
Beyond the technical challenges for deploying Machine Learning solutions in a software company
Authors: Ilias Flaounas
Date: 2017-08-08 00:00:00
Tasks: BIG-bench Machine Learning

Recently software development companies started to embrace Machine Learning
(ML) techniques for introducing a series of advanced functionality in their
products such as personalisation of the user experience, improved search,
content recommendation and automation. The technical challenges for tackling
these problems are heavily researched in literature. A less studied area is a
pragmatic approach to the role of humans in a complex modern industrial
environment where ML based systems are developed. Key stakeholders affect the
system from inception and up to operation and maintenance. Product managers
want to embed "smart" experiences for their users and drive the decisions on
what should be built next; software engineers are challenged to build or
utilise ML software tools that require skills that are well outside of their
comfort zone; legal and risk departments may influence design choices and data
access; operations teams are requested to maintain ML systems which are
non-stationary in their nature and change behaviour over time; and finally ML
practitioners should communicate with all these stakeholders to successfully
build a reliable system. This paper discusses some of the challenges we faced
in Atlassian as we started investing more in the ML space.
Verification & Validation of Agent Based Simulations using the VOMAS (Virtual Overlay Multi-agent System) approach
Authors: Muaz A. Niazi, Amir Hussain, Mario Kolberg
Date: 2017-08-08 00:00:00

Agent Based Models are very popular in a number of different areas. For example, they have been used in a range of domains ranging from modeling of tumor growth, immune systems, molecules to models of social networks, crowds and computer and mobile self-organizing networks. One reason for their success is their intuitiveness and similarity to human cognition. However, with this power of abstraction, in spite of being easily applicable to such a wide number of domains, it is hard to validate agent-based models. In addition, building valid and credible simulations is not just a challenging task but also a crucial exercise to ensure that what we are modeling is, at some level of abstraction, a model of our conceptual system; the system that we have in mind. In this paper, we address this important area of validation of agent based models by presenting a novel technique which has broad applicability and can be applied to all kinds of agent-based models. We present a framework, where a virtual overlay multi-agent system can be used to validate simulation models. In addition, since agent-based models have been typically growing, in parallel, in multiple domains, to cater for all of these, we present a new single validation technique applicable to all agent based models. Our technique, which allows for the validation of agent based simulations uses VOMAS: a Virtual Overlay Multi-agent System. This overlay multi-agent system can comprise various types of agents, which form an overlay on top of the agent based simulation model that needs to be validated. Other than being able to watch and log, each of these agents contains clearly defined constraints, which, if violated, can be logged in real time. To demonstrate its effectiveness, we show its broad applicability in a wide variety of simulation models ranging from social sciences to computer networks in spatial and non-spatial conceptual models.
Towards A Novel Unified Framework for Developing Formal, Network and Validated Agent-Based Simulation Models of Complex Adaptive Systems
Authors: Muaz A. Niazi
Date: 2017-08-08 00:00:00

Literature on the modeling and simulation of complex adaptive systems (cas)
has primarily advanced vertically in different scientific domains with
scientists developing a variety of domain-specific approaches and applications.
However, while cas researchers are inher-ently interested in an
interdisciplinary comparison of models, to the best of our knowledge, there is
currently no single unified framework for facilitating the development,
comparison, communication and validation of models across different scientific
domains. In this thesis, we propose first steps towards such a unified
framework using a combination of agent-based and complex network-based modeling
approaches and guidelines formulated in the form of a set of four levels of
usage, which allow multidisciplinary researchers to adopt a suitable framework
level on the basis of available data types, their research study objectives and
expected outcomes, thus allowing them to better plan and conduct their
respective re-search case studies.
Temporal Context Network for Activity Localization in Videos
Authors: Xiyang Dai, Bharat Singh, Guyue Zhang, Larry S. Davis, Yan Qiu Chen
Date: 2017-08-08 00:00:00
Proceeding: ICCV 2017 10
Tasks: General Classification, Temporal Localization

We present a Temporal Context Network (TCN) for precise temporal localization
of human activities. Similar to the Faster-RCNN architecture, proposals are
placed at equal intervals in a video which span multiple temporal scales. We
propose a novel representation for ranking these proposals. Since pooling
features only inside a segment is not sufficient to predict activity
boundaries, we construct a representation which explicitly captures context
around a proposal for ranking it. For each temporal segment inside a proposal,
features are uniformly sampled at a pair of scales and are input to a temporal
convolutional neural network for classification. After ranking proposals,
non-maximum suppression is applied and classification is performed to obtain
final detections. TCN outperforms state-of-the-art methods on the ActivityNet
dataset and the THUMOS14 dataset.
Ensemble Distillation for Neural Machine Translation
Authors: Markus Freitag, Yaser Al-Onaizan, Baskaran Sankaran
Date: 2017-02-06 00:00:00
Tasks: Knowledge Distillation, Machine Translation, NMT, Sentence, Translation

Knowledge distillation describes a method for training a student network to
perform better by learning from a stronger teacher network. Translating a
sentence with an Neural Machine Translation (NMT) engine is time expensive and
having a smaller model speeds up this process. We demonstrate how to transfer
the translation quality of an ensemble and an oracle BLEU teacher network into
a single NMT system. Further, we present translation improvements from a
teacher network that has the same architecture and dimensions of the student
network. As the training of the student model is still expensive, we introduce
a data filtering method based on the knowledge of the teacher model that not
only speeds up the training, but also leads to better translation quality. Our
techniques need no code change and can be easily reproduced with any NMT
architecture to speed up the decoding process.
Identifying 3 moss species by deep learning, using the "chopped picture" method
Authors: Takeshi Ise, Mari Minagawa, Masanori Onishi
Date: 2017-08-07 00:00:00

In general, object identification tends not to work well on ambiguous,
amorphous objects such as vegetation. In this study, we developed a simple but
effective approach to identify ambiguous objects and applied the method to
several moss species. As a result, the model correctly classified test images
with accuracy more than 90%. Using this approach will help progress in computer
vision studies.
Dense Captioning with Joint Inference and Visual Context
Authors: Linjie Yang, Kevin Tang, Jianchao Yang, Li-Jia Li
Date: 2016-11-21 00:00:00
Proceeding: CVPR 2017 7
Tasks: Dense Captioning, Descriptive

Dense captioning is a newly emerging computer vision topic for understanding
images with dense language descriptions. The goal is to densely detect visual
concepts (e.g., objects, object parts, and interactions between them) from
images, labeling each with a short descriptive phrase. We identify two key
challenges of dense captioning that need to be properly addressed when tackling
the problem. First, dense visual concept annotations in each image are
associated with highly overlapping target regions, making accurate localization
of each visual concept challenging. Second, the large amount of visual concepts
makes it hard to recognize each of them by appearance alone. We propose a new
model pipeline based on two novel ideas, joint inference and context fusion, to
alleviate these two challenges. We design our model architecture in a
methodical manner and thoroughly evaluate the variations in architecture. Our
final model, compact and efficient, achieves state-of-the-art accuracy on
Visual Genome for dense captioning with a relative gain of 73\% compared to the
previous best algorithm. Qualitative experiments also reveal the semantic
capabilities of our model in dense captioning.
What Makes a Place? Building Bespoke Place Dependent Object Detectors for Robotics
Authors: Jeffrey Hawke, Alex Bewley, Ingmar Posner
Date: 2017-08-07 00:00:00
Tasks: Autonomous Driving

This paper is about enabling robots to improve their perceptual performance
through repeated use in their operating environment, creating local expert
detectors fitted to the places through which a robot moves. We leverage the
concept of 'experiences' in visual perception for robotics, accounting for bias
in the data a robot sees by fitting object detector models to a particular
place. The key question we seek to answer in this paper is simply: how do we
define a place? We build bespoke pedestrian detector models for autonomous
driving, highlighting the necessary trade off between generalisation and model
capacity as we vary the extent of the place we fit to. We demonstrate a
sizeable performance gain over a current state-of-the-art detector when using
computationally lightweight bespoke place-fitted detector models.
AirCode: Unobtrusive Physical Tags for Digital Fabrication
Authors: Dingzeyu Li, Avinash S. Nair, Shree K. Nayar, Changxi Zheng
Date: 2017-07-18 00:00:00
Tasks: Object, Robotic Grasping, TAG

We present AirCode, a technique that allows the user to tag physically
fabricated objects with given information. An AirCode tag consists of a group
of carefully designed air pockets placed beneath the object surface. These air
pockets are easily produced during the fabrication process of the object,
without any additional material or postprocessing. Meanwhile, the air pockets
affect only the scattering light transport under the surface, and thus are hard
to notice to our naked eyes. But, by using a computational imaging method, the
tags become detectable. We present a tool that automates the design of air
pockets for the user to encode information. AirCode system also allows the user
to retrieve the information from captured images via a robust decoding
algorithm. We demonstrate our tagging technique with applications for metadata
embedding, robotic grasping, as well as conveying object affordances.
Multibiometric Secure System Based on Deep Learning
Authors: Veeru Talreja, Matthew C. Valenti, Nasser M. Nasrabadi
Date: 2017-08-07 00:00:00
Tasks: Decoder, Deep Learning

In this paper, we propose a secure multibiometric system that uses deep
neural networks and error-correction coding. We present a feature-level fusion
framework to generate a secure multibiometric template from each user's
multiple biometrics. Two fusion architectures, fully connected architecture and
bilinear architecture, are implemented to develop a robust multibiometric
shared representation. The shared representation is used to generate a
cancelable biometric template that involves the selection of a different set of
reliable and discriminative features for each user. This cancelable template is
a binary vector and is passed through an appropriate error-correcting decoder
to find a closest codeword and this codeword is hashed to generate the final
secure template. The efficacy of the proposed approach is shown using a
multimodal database where we achieve state-of-the-art matching performance,
along with cancelability and security.
GPLAC: Generalizing Vision-Based Robotic Skills using Weakly Labeled Images
Authors: Avi Singh, Larry Yang, Sergey Levine
Date: 2017-08-07 00:00:00
Proceeding: ICCV 2017 10
Tasks: Binary Classification, Diversity, Domain Adaptation, Reinforcement Learning

We tackle the problem of learning robotic sensorimotor control policies that
can generalize to visually diverse and unseen environments. Achieving broad
generalization typically requires large datasets, which are difficult to obtain
for task-specific interactive processes such as reinforcement learning or
learning from demonstration. However, much of the visual diversity in the world
can be captured through passively collected datasets of images or videos. In
our method, which we refer to as GPLAC (Generalized Policy Learning with
Attentional Classifier), we use both interaction data and weakly labeled image
data to augment the generalization capacity of sensorimotor policies. Our
method combines multitask learning on action selection and an auxiliary binary
classification objective, together with a convolutional neural network
architecture that uses an attentional mechanism to avoid distractors. We show
that pairing interaction data from just a single environment with a diverse
dataset of weakly labeled data results in greatly improved generalization to
unseen environments, and show that this generalization depends on both the
auxiliary objective and the attentional architecture that we propose. We
demonstrate our results in both simulation and on a real robotic manipulator,
and demonstrate substantial improvement over standard convolutional
architectures and domain adaptation methods.