pwc-archive/papers-with-abstracts huggingface dataset:
Efficient Decision-based Black-box Adversarial Attacks on Face Recognition
Authors: Yinpeng Dong, Hang Su, Baoyuan Wu, Zhifeng Li, Wei Liu, Tong Zhang, Jun Zhu
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Face Recognition

Face recognition has obtained remarkable progress in recent years due to the
great improvement of deep convolutional neural networks (CNNs). However, deep
CNNs are vulnerable to adversarial examples, which can cause fateful
consequences in real-world face recognition applications with
security-sensitive purposes. Adversarial attacks are widely studied as they can
identify the vulnerability of the models before they are deployed. In this
paper, we evaluate the robustness of state-of-the-art face recognition models
in the decision-based black-box attack setting, where the attackers have no
access to the model parameters and gradients, but can only acquire hard-label
predictions by sending queries to the target model. This attack setting is more
practical in real-world face recognition systems. To improve the efficiency of
previous methods, we propose an evolutionary attack algorithm, which can model
the local geometries of the search directions and reduce the dimension of the
search space. Extensive experiments demonstrate the effectiveness of the
proposed method that induces a minimum perturbation to an input face image with
fewer queries. We also apply the proposed method to attack a real-world face
recognition system successfully.
Reliable and Efficient Image Cropping: A Grid Anchor based Approach
Authors: Hui Zeng, Lida Li, Zisheng Cao, Lei Zhang
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Cropping

Image cropping aims to improve the composition as well as aesthetic quality
of an image by removing extraneous content from it. Existing image cropping
databases provide only one or several human-annotated bounding boxes as the
groundtruth, which cannot reflect the non-uniqueness and flexibility of image
cropping in practice. The employed evaluation metrics such as
intersection-over-union cannot reliably reflect the real performance of
cropping models, either. This work revisits the problem of image cropping, and
presents a grid anchor based formulation by considering the special properties
and requirements (e.g., local redundancy, content preservation, aspect ratio)
of image cropping. Our formulation reduces the searching space of candidate
crops from millions to less than one hundred. Consequently, a grid anchor based
cropping benchmark is constructed, where all crops of each image are annotated
and more reliable evaluation metrics are defined. We also design an effective
and lightweight network module, which simultaneously considers the region of
interest and region of discard for more accurate image cropping. Our model can
stably output visually pleasing crops for images of different scenes and run at
a speed of 125 FPS. Code and dataset are available at:
https://github.com/HuiZeng/Grid-Anchor-based-Image-Cropping.
Graphonomy: Universal Human Parsing via Graph Transfer Learning
Authors: Ke Gong, Yiming Gao, Xiaodan Liang, Xiaohui Shen, Meng Wang, Liang Lin
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Human Parsing, Transfer Learning

Prior highly-tuned human parsing models tend to fit towards each dataset in a
specific domain or with discrepant label granularity, and can hardly be adapted
to other human parsing tasks without extensive re-training. In this paper, we
aim to learn a single universal human parsing model that can tackle all kinds
of human parsing needs by unifying label annotations from different domains or
at various levels of granularity. This poses many fundamental learning
challenges, e.g. discovering underlying semantic structures among different
label granularity, performing proper transfer learning across different image
domains, and identifying and utilizing label redundancies across related tasks.
  To address these challenges, we propose a new universal human parsing agent,
named "Graphonomy", which incorporates hierarchical graph transfer learning
upon the conventional parsing network to encode the underlying label semantic
structures and propagate relevant semantic information. In particular,
Graphonomy first learns and propagates compact high-level graph representation
among the labels within one dataset via Intra-Graph Reasoning, and then
transfers semantic information across multiple datasets via Inter-Graph
Transfer. Various graph transfer dependencies (\eg, similarity, linguistic
knowledge) between different datasets are analyzed and encoded to enhance graph
transfer capability. By distilling universal semantic graph representation to
each specific task, Graphonomy is able to predict all levels of parsing labels
in one system without piling up the complexity. Experimental results show
Graphonomy effectively achieves the state-of-the-art results on three human
parsing benchmarks as well as advantageous universal human parsing performance.
Deep Virtual Networks for Memory Efficient Inference of Multiple Tasks
Authors: Eunwoo Kim, Chanho Ahn, Philip H. S. Torr, Songhwai Oh
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2019 6

Deep networks consume a large amount of memory by their nature. A natural
question arises can we reduce that memory requirement whilst maintaining
performance. In particular, in this work we address the problem of memory
efficient learning for multiple tasks. To this end, we propose a novel network
architecture producing multiple networks of different configurations, termed
deep virtual networks (DVNs), for different tasks. Each DVN is specialized for
a single task and structured hierarchically. The hierarchical structure, which
contains multiple levels of hierarchy corresponding to different numbers of
parameters, enables multiple inference for different memory budgets. The
building block of a deep virtual network is based on a disjoint collection of
parameters of a network, which we call a unit. The lowest level of hierarchy in
a deep virtual network is a unit, and higher levels of hierarchy contain lower
levels' units and other additional units. Given a budget on the number of
parameters, a different level of a deep virtual network can be chosen to
perform the task. A unit can be shared by different DVNs, allowing multiple
DVNs in a single network. In addition, shared units provide assistance to the
target task with additional knowledge learned from another tasks. This
cooperative configuration of DVNs makes it possible to handle different tasks
in a memory-aware manner. Our experiments show that the proposed method
outperforms existing approaches for multiple tasks. Notably, ours is more
efficient than others as it allows memory-aware inference for all tasks.
Holistic and Comprehensive Annotation of Clinically Significant Findings on Diverse CT Images: Learning from Radiology Reports and Label Ontology
Authors: Ke Yan, Yifan Peng, Veit Sandfort, Mohammadhadi Bagheri, Zhiyong Lu, Ronald M. Summers
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Metric Learning

In radiologists' routine work, one major task is to read a medical image,
e.g., a CT scan, find significant lesions, and describe them in the radiology
report. In this paper, we study the lesion description or annotation problem.
Given a lesion image, our aim is to predict a comprehensive set of relevant
labels, such as the lesion's body part, type, and attributes, which may assist
downstream fine-grained diagnosis. To address this task, we first design a deep
learning module to extract relevant semantic labels from the radiology reports
associated with the lesion images. With the images and text-mined labels, we
propose a lesion annotation network (LesaNet) based on a multilabel
convolutional neural network (CNN) to learn all labels holistically.
Hierarchical relations and mutually exclusive relations between the labels are
leveraged to improve the label prediction accuracy. The relations are utilized
in a label expansion strategy and a relational hard example mining algorithm.
We also attach a simple score propagation layer on LesaNet to enhance recall
and explore implicit relation between labels. Multilabel metric learning is
combined with classification to enable interpretable prediction. We evaluated
LesaNet on the public DeepLesion dataset, which contains over 32K diverse
lesion images. Experiments show that LesaNet can precisely annotate the lesions
using an ontology of 171 fine-grained labels with an average AUC of 0.9344.
Domain-Symmetric Networks for Adversarial Domain Adaptation
Authors: Yabin Zhang, Hui Tang, Kui Jia, Mingkui Tan
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation aims to learn a model of classifier for unlabeled samples on the target domain, given training data of labeled samples on the source domain. Impressive progress is made recently by learning invariant features via domain-adversarial training of deep networks. In spite of the recent progress, domain adaptation is still limited in achieving the invariance of feature distributions at a finer category level. To this end, we propose in this paper a new domain adaptation method called Domain-Symmetric Networks (SymNets). The proposed SymNet is based on a symmetric design of source and target task classifiers, based on which we also construct an additional classifier that shares with them its layer neurons. To train the SymNet, we propose a novel adversarial learning objective whose key design is based on a two-level domain confusion scheme, where the category-level confusion loss improves over the domain-level one by driving the learning of intermediate network features to be invariant at the corresponding categories of the two domains. Both domain discrimination and domain confusion are implemented based on the constructed additional classifier. Since target samples are unlabeled, we also propose a scheme of cross-domain training to help learn the target classifier. Careful ablation studies show the efficacy of our proposed method. In particular, based on commonly used base networks, our SymNets achieve the new state of the art on three benchmark domain adaptation datasets.
Action Recognition from Single Timestamp Supervision in Untrimmed Videos
Authors: Davide Moltisanti, Sanja Fidler, Dima Damen
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Action Recognition, Temporal Action Localization

Recognising actions in videos relies on labelled supervision during training,
typically the start and end times of each action instance. This supervision is
not only subjective, but also expensive to acquire. Weak video-level
supervision has been successfully exploited for recognition in untrimmed
videos, however it is challenged when the number of different actions in
training videos increases. We propose a method that is supervised by single
timestamps located around each action instance, in untrimmed videos. We replace
expensive action bounds with sampling distributions initialised from these
timestamps. We then use the classifier's response to iteratively update the
sampling distributions. We demonstrate that these distributions converge to the
location and extent of discriminative action segments. We evaluate our method
on three datasets for fine-grained recognition, with increasing number of
different actions per video, and show that single timestamps offer a reasonable
compromise between recognition performance and labelling effort, performing
comparably to full temporal supervision. Our update method improves top-1 test
accuracy by up to 5.4%. across the evaluated datasets.
Label Propagation for Deep Semi-supervised Learning
Authors: Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, Ondrej Chum
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Transductive Learning

Semi-supervised learning is becoming increasingly important because it can
combine data carefully labeled by humans with abundant unlabeled data to train
deep neural networks. Classic methods on semi-supervised learning that have
focused on transductive learning have not been fully exploited in the inductive
framework followed by modern deep learning. The same holds for the manifold
assumption---that similar examples should get the same prediction. In this
work, we employ a transductive label propagation method that is based on the
manifold assumption to make predictions on the entire dataset and use these
predictions to generate pseudo-labels for the unlabeled data and train a deep
neural network. At the core of the transductive method lies a nearest neighbor
graph of the dataset that we create based on the embeddings of the same
network.Therefore our learning process iterates between these two steps. We
improve performance on several datasets especially in the few labels regime and
show that our work is complementary to current state of the art.
Prime Sample Attention in Object Detection
Authors: Yuhang Cao, Kai Chen, Chen Change Loy, Dahua Lin
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2020 6
Tasks: Object, object-detection, Object Detection

It is a common paradigm in object detection frameworks to treat all samples equally and target at maximizing the performance on average. In this work, we revisit this paradigm through a careful study on how different samples contribute to the overall performance measured in terms of mAP. Our study suggests that the samples in each mini-batch are neither independent nor equally important, and therefore a better classifier on average does not necessarily mean higher mAP. Motivated by this study, we propose the notion of Prime Samples, those that play a key role in driving the detection performance. We further develop a simple yet effective sampling and learning strategy called PrIme Sample Attention (PISA) that directs the focus of the training process towards such samples. Our experiments demonstrate that it is often more effective to focus on prime samples than hard samples when training a detector. Particularly, On the MSCOCO dataset, PISA outperforms the random sampling baseline and hard mining schemes, e.g., OHEM and Focal Loss, consistently by around 2% on both single-stage and two-stage detectors, even with a strong backbone ResNeXt-101.
Learning from Videos with Deep Convolutional LSTM Networks
Authors: Logan Courtney, Ramavarapu Sreenivas
Date: 2019-04-09 00:00:00
Tasks: Lipreading, Lip Reading

This paper explores the use of convolution LSTMs to simultaneously learn
spatial- and temporal-information in videos. A deep network of convolutional
LSTMs allows the model to access the entire range of temporal information at
all spatial scales of the data. We describe our experiments involving
convolution LSTMs for lipreading that demonstrate the model is capable of
selectively choosing which spatiotemporal scales are most relevant for a
particular dataset. The proposed deep architecture also holds promise in other
applications where spatiotemporal features play a vital role without having to
specifically cater the design of the network for the particular spatiotemporal
features existent within the problem. For the Lip Reading in the Wild (LRW)
dataset, our model slightly outperforms the previous state of the art (83.4%
vs. 83.0%) and sets the new state of the art at 85.2% when the model is
pretrained on the Lip Reading Sentences (LRS2) dataset.
Unsupervised 3D Pose Estimation with Geometric Self-Supervision
Authors: Ching-Hang Chen, Ambrish Tyagi, Amit Agrawal, Dylan Drover, Rohith MV, Stefan Stojanov, James M. Rehg
Date: 2019-04-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Human Pose Estimation, 3D Pose Estimation, Pose Estimation, valid

We present an unsupervised learning approach to recover 3D human pose from 2D
skeletal joints extracted from a single image. Our method does not require any
multi-view image data, 3D skeletons, correspondences between 2D-3D points, or
use previously learned 3D priors during training. A lifting network accepts 2D
landmarks as inputs and generates a corresponding 3D skeleton estimate. During
training, the recovered 3D skeleton is reprojected on random camera viewpoints
to generate new "synthetic" 2D poses. By lifting the synthetic 2D poses back to
3D and re-projecting them in the original camera view, we can define
self-consistency loss both in 3D and in 2D. The training can thus be self
supervised by exploiting the geometric self-consistency of the
lift-reproject-lift process. We show that self-consistency alone is not
sufficient to generate realistic skeletons, however adding a 2D pose
discriminator enables the lifter to output valid 3D poses. Additionally, to
learn from 2D poses "in the wild", we train an unsupervised 2D domain adapter
network to allow for an expansion of 2D data. This improves results and
demonstrates the usefulness of 2D pose data for unsupervised 3D lifting.
Results on Human3.6M dataset for 3D human pose estimation demonstrate that our
approach improves upon the previous unsupervised methods by 30% and outperforms
many weakly supervised approaches that explicitly use 3D data.
Embodied Neuromorphic Vision with Event-Driven Random Backpropagation
Authors: Jacques Kaiser, Alexander Friedrich, J. Camilo Vasquez Tieck, Daniel Reichard, Arne Roennau, Emre Neftci, Rüdiger Dillmann
Date: 2019-04-09 00:00:00

Spike-based communication between biological neurons is sparse and unreliable. This enables the brain to process visual information from the eyes efficiently. Taking inspiration from biology, artificial spiking neural networks coupled with silicon retinas attempt to model these computations. Recent findings in machine learning allowed the derivation of a family of powerful synaptic plasticity rules approximating backpropagation for spiking networks. Are these rules capable of processing real-world visual sensory data? In this paper, we evaluate the performance of Event-Driven Random Back-Propagation (eRBP) at learning representations from event streams provided by a Dynamic Vision Sensor (DVS). First, we show that eRBP matches state-of-the-art performance on the DvsGesture dataset with the addition of a simple covert attention mechanism. By remapping visual receptive fields relatively to the center of the motion, this attention mechanism provides translation invariance at low computational cost compared to convolutions. Second, we successfully integrate eRBP in a real robotic setup, where a robotic arm grasps objects according to detected visual affordances. In this setup, visual information is actively sensed by a DVS mounted on a robotic head performing microsaccadic eye movements. We show that our method classifies affordances within 100ms after microsaccade onset, which is comparable to human performance reported in behavioral study. Our results suggest that advances in neuromorphic technology and plasticity rules enable the development of autonomous robots operating at high speed and low energy consumption.
Generation & Evaluation of Adversarial Examples for Malware Obfuscation
Authors: Daniel Park, Haidar Khan, Bülent Yener
Date: 2019-04-09 00:00:00
Tasks: General Classification, Malware Classification

There has been an increased interest in the application of convolutional neural networks for image based malware classification, but the susceptibility of neural networks to adversarial examples allows malicious actors to evade classifiers. Adversarial examples are usually generated by adding small perturbations to the input that are unrecognizable to humans, but the same approach is not effective with malware. In general, these perturbations cause changes in the byte sequences that change the initial functionality or result in un-executable binaries. We present a generative model for executable adversarial malware examples using obfuscation that achieves a high misclassification rate, up to 100% and 98% in white-box and black-box settings respectively, and demonstrates transferability. We further evaluate the effectiveness of the proposed method by reporting insignificant change in the evasion rate of our adversarial examples against popular defense strategies.
CMIR-NET : A Deep Learning Based Model For Cross-Modal Retrieval In Remote Sensing
Authors: Ushasi Chaudhuri, Biplab Banerjee, Avik Bhattacharya, Mihai Datcu
Date: 2019-04-09 00:00:00
Tasks: Cross-Modal Information Retrieval, Cross-Modal Retrieval, Image Retrieval, Information Retrieval, Multi-Label Image Retrieval, Retrieval

We address the problem of cross-modal information retrieval in the domain of remote sensing. In particular, we are interested in two application scenarios: i) cross-modal retrieval between panchromatic (PAN) and multi-spectral imagery, and ii) multi-label image retrieval between very high resolution (VHR) images and speech based label annotations. Notice that these multi-modal retrieval scenarios are more challenging than the traditional uni-modal retrieval approaches given the inherent differences in distributions between the modalities. However, with the growing availability of multi-source remote sensing data and the scarcity of enough semantic annotations, the task of multi-modal retrieval has recently become extremely important. In this regard, we propose a novel deep neural network based architecture which is considered to learn a discriminative shared feature space for all the input modalities, suitable for semantically coherent information retrieval. Extensive experiments are carried out on the benchmark large-scale PAN - multi-spectral DSRSID dataset and the multi-label UC-Merced dataset. Together with the Merced dataset, we generate a corpus of speech signals corresponding to the labels. Superior performance with respect to the current state-of-the-art is observed in all the cases.
Quizbowl: The Case for Incremental Question Answering
Authors: Pedro Rodriguez, Shi Feng, Mohit Iyyer, He He, Jordan Boyd-Graber
Date: 2019-04-09 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Question Answering, Sequential Decision Making

Scholastic trivia competitions test knowledge and intelligence through mastery of question answering. Modern question answering benchmarks are one variant of the Turing test. Specifically, answering a set of questions as well as a human is a minimum bar towards demonstrating human-like intelligence. This paper makes the case that the format of one competition -- where participants can answer in the middle of hearing a question (incremental) -- better differentiates the skill between (human or machine) players. Additionally, merging a sequential decision-making sub-task with question answering (QA) provides a good setting for research in model calibration and opponent modeling. Thus, embedded in this task are three machine learning challenges: (1) factoid QA over thousands of Wikipedia-like answers, (2) calibration of the QA model's confidence scores, and (3) sequential decision-making that incorporates knowledge of the QA model, its calibration, and what the opponent may do. We make two contributions: (1) collecting and curating a large factoid QA dataset and an accompanying gameplay dataset, and (2) developing a model that addresses these three machine learning challenges. In addition to offline evaluation, we pitted our model against some of the most accomplished trivia players in the world in a series of exhibition matches spanning several years. Throughout this paper, we show that collaborations with the vibrant trivia community have contributed to the quality of our dataset, spawned new research directions, and doubled as an exciting way to engage the public with research in machine learning and natural language processing.