pwc-archive/papers-with-abstracts huggingface dataset:
Few-shot Object Detection via Feature Reweighting
Authors: Bingyi Kang, Zhuang Liu, Xin Wang, Fisher Yu, Jiashi Feng, Trevor Darrell
Date: 2018-12-05 00:00:00
Proceeding: ICCV 2019 10
Tasks: Few-Shot Learning, Few-Shot Object Detection, Image Classification, Meta-Learning, Object, object-detection, Object Detection

Conventional training of a deep CNN based object detector demands a large number of bounding box annotations, which may be unavailable for rare categories. In this work we develop a few-shot object detector that can learn to detect novel objects from only a few annotated examples. Our proposed model leverages fully labeled base classes and quickly adapts to novel classes, using a meta feature learner and a reweighting module within a one-stage detection architecture. The feature learner extracts meta features that are generalizable to detect novel object classes, using training data from base classes with sufficient samples. The reweighting module transforms a few support examples from the novel classes to a global vector that indicates the importance or relevance of meta features for detecting the corresponding objects. These two modules, together with a detection prediction module, are trained end-to-end based on an episodic few-shot learning scheme and a carefully designed loss function. Through extensive experiments we demonstrate that our model outperforms well-established baselines by a large margin for few-shot object detection, on multiple datasets and settings. We also present analysis on various aspects of our proposed model, aiming to provide some inspiration for future few-shot detection works.
Enhancing Label-Driven Deep Deformable Image Registration with Local Distance Metrics for State-of-the-Art Cardiac Motion Tracking
Authors: Alessa Hering, Sven Kuckertz, Stefan Heldmann, Mattias Heinrich
Date: 2018-12-05 00:00:00
Tasks: Deep Learning, Image Registration, Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

While deep learning has achieved significant advances in accuracy for medical
image segmentation, its benefits for deformable image registration have so far
remained limited to reduced computation times. Previous work has either focused
on replacing the iterative optimization of distance and smoothness terms with
CNN-layers or using supervised approaches driven by labels. Our method is the
first to combine the complementary strengths of global semantic information
(represented by segmentation labels) and local distance metrics that help align
surrounding structures. We demonstrate significant higher Dice scores (of
86.5\%) for deformable cardiac image registration compared to classic
registration (79.0\%) as well as label-driven deep learning frameworks
(83.4\%).
Chore division on a graph
Authors: Sylvain Bouveret, Katarína Cechlárová, Julien Lesca
Date: 2018-12-05 00:00:00

The paper considers fair allocation of indivisible nondisposable items that
generate disutility (chores). We assume that these items are placed in the
vertices of a graph and each agent's share has to form a connected subgraph of
this graph. Although a similar model has been investigated before for goods, we
show that the goods and chores settings are inherently different. In
particular, it is impossible to derive the solution of the chores instance from
the solution of its naturally associated fair division instance. We consider
three common fair division solution concepts, namely proportionality,
envy-freeness and equitability, and two individual disutility aggregation
functions: additive and maximum based. We show that deciding the existence of a
fair allocation is hard even if the underlying graph is a path or a star. We
also present some efficiently solvable special cases for these graph
topologies.
Explainable and Explicit Visual Reasoning over Scene Graphs
Authors: Jiaxin Shi, Hanwang Zhang, Juanzi Li
Date: 2018-12-05 00:00:00
Proceeding: CVPR 2019 6
Tasks: Inductive Bias, Visual Question Answering (VQA), Visual Reasoning

We aim to dismantle the prevalent black-box neural architectures used in
complex visual reasoning tasks, into the proposed eXplainable and eXplicit
Neural Modules (XNMs), which advance beyond existing neural module networks
towards using scene graphs --- objects as nodes and the pairwise relationships
as edges --- for explainable and explicit reasoning with structured knowledge.
XNMs allow us to pay more attention to teach machines how to "think",
regardless of what they "look". As we will show in the paper, by using scene
graphs as an inductive bias, 1) we can design XNMs in a concise and flexible
fashion, i.e., XNMs merely consist of 4 meta-types, which significantly reduce
the number of parameters by 10 to 100 times, and 2) we can explicitly trace the
reasoning-flow in terms of graph attentions. XNMs are so generic that they
support a wide range of scene graph implementations with various qualities. For
example, when the graphs are detected perfectly, XNMs achieve 100% accuracy on
both CLEVR and CLEVR CoGenT, establishing an empirical performance upper-bound
for visual reasoning; when the graphs are noisily detected from real-world
images, XNMs are still robust to achieve a competitive 67.5% accuracy on
VQAv2.0, surpassing the popular bag-of-objects attention models without graph
structures.
Improving Similarity Search with High-dimensional Locality-sensitive Hashing
Authors: Jaiyam Sharma, Saket Navlakha
Date: 2018-12-05 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We propose a new class of data-independent locality-sensitive hashing (LSH)
algorithms based on the fruit fly olfactory circuit. The fundamental difference
of this approach is that, instead of assigning hashes as dense points in a low
dimensional space, hashes are assigned in a high dimensional space, which
enhances their separability. We show theoretically and empirically that this
new family of hash functions is locality-sensitive and preserves rank
similarity for inputs in any `p space. We then analyze different variations on
this strategy and show empirically that they outperform existing LSH methods
for nearest-neighbors search on six benchmark datasets. Finally, we propose a
multi-probe version of our algorithm that achieves higher performance for the
same query time, or conversely, that maintains performance of prior approaches
while taking significantly less indexing time and memory. Overall, our approach
leverages the advantages of separability provided by high-dimensional spaces,
while still remaining computationally efficient
MLIC: A MaxSAT-Based framework for learning interpretable classification rules
Authors: Dmitry Malioutov, Kuldeep S. Meel
Date: 2018-12-05 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, Interpretable Machine Learning

The wide adoption of machine learning approaches in the industry, government,
medicine and science has renewed the interest in interpretable machine
learning: many decisions are too important to be delegated to black-box
techniques such as deep neural networks or kernel SVMs. Historically, problems
of learning interpretable classifiers, including classification rules or
decision trees, have been approached by greedy heuristic methods as essentially
all the exact optimization formulations are NP-hard. Our primary contribution
is a MaxSAT-based framework, called MLIC, which allows principled search for
interpretable classification rules expressible in propositional logic. Our
approach benefits from the revolutionary advances in the constraint
satisfaction community to solve large-scale instances of such problems. In
experimental evaluations over a collection of benchmarks arising from practical
scenarios, we demonstrate its effectiveness: we show that the formulation can
solve large classification problems with tens or hundreds of thousands of
examples and thousands of features, and to provide a tunable balance of
accuracy vs. interpretability. Furthermore, we show that in many problems
interpretability can be obtained at only a minor cost in accuracy. The primary
objective of the paper is to show that recent advances in the MaxSAT literature
make it realistic to find optimal (or very high quality near-optimal) solutions
to large-scale classification problems. The key goal of the paper is to excite
researchers in both interpretable classification and in the CP community to
take it further and propose richer formulations, and to develop bespoke solvers
attuned to the problem of interpretable ML.
Attention Boosted Sequential Inference Model
Authors: Guanyu Li, Pengfei Zhang, Caiyan Jia
Date: 2018-12-05 00:00:00
Tasks: model, Natural Language Inference

Attention mechanism has been proven effective on natural language processing.
This paper proposes an attention boosted natural language inference model named
aESIM by adding word attention and adaptive direction-oriented attention
mechanisms to the traditional Bi-LSTM layer of natural language inference
models, e.g. ESIM. This makes the inference model aESIM has the ability to
effectively learn the representation of words and model the local subsentential
inference between pairs of premise and hypothesis. The empirical studies on the
SNLI, MultiNLI and Quora benchmarks manifest that aESIM is superior to the
original ESIM model.
Few Sample Knowledge Distillation for Efficient Network Compression
Authors: Tianhong Li, Jianguo Li, Zhuang Liu, Chang-Shui Zhang
Date: 2018-12-05 00:00:00
Proceeding: CVPR 2020 6
Tasks: Knowledge Distillation, Network Pruning, Neural Network Compression, Tensor Decomposition

Deep neural network compression techniques such as pruning and weight tensor decomposition usually require fine-tuning to recover the prediction accuracy when the compression ratio is high. However, conventional fine-tuning suffers from the requirement of a large training set and the time-consuming training procedure. This paper proposes a novel solution for knowledge distillation from label-free few samples to realize both data efficiency and training/processing efficiency. We treat the original network as "teacher-net" and the compressed network as "student-net". A 1x1 convolution layer is added at the end of each layer block of the student-net, and we fit the block-level outputs of the student-net to the teacher-net by estimating the parameters of the added layers. We prove that the added layer can be merged without adding extra parameters and computation cost during inference. Experiments on multiple datasets and network architectures verify the method's effectiveness on student-nets obtained by various network pruning and weight decomposition methods. Our method can recover student-net's accuracy to the same level as conventional fine-tuning methods in minutes while using only 1% label-free data of the full training data.
ADARES: Adaptive Resource Management for Virtual Machines
Authors: Ignacio Cano, Lequn Chen, Pedro Fonseca, Tianqi Chen, Chern Cheah, Karan Gupta, Ramesh Chandra, Arvind Krishnamurthy
Date: 2018-12-05 00:00:00
Tasks: Management, Multi-Armed Bandits, Transfer Learning

Virtual execution environments allow for consolidation of multiple
applications onto the same physical server, thereby enabling more efficient use
of server resources. However, users often statically configure the resources of
virtual machines through guesswork, resulting in either insufficient resource
allocations that hinder VM performance, or excessive allocations that waste
precious data center resources. In this paper, we first characterize real-world
resource allocation and utilization of VMs through the analysis of an extensive
dataset, consisting of more than 250k VMs from over 3.6k private enterprise
clusters. Our large-scale analysis confirms that VMs are often misconfigured,
either overprovisioned or underprovisioned, and that this problem is pervasive
across a wide range of private clusters. We then propose ADARES, an adaptive
system that dynamically adjusts VM resources using machine learning techniques.
In particular, ADARES leverages the contextual bandits framework to effectively
manage the adaptations. Our system exploits easily collectible data, at the
cluster, node, and VM levels, to make more sensible allocation decisions, and
uses transfer learning to safely explore the configurations space and speed up
training. Our empirical evaluation shows that ADARES can significantly improve
system utilization without sacrificing performance. For instance, when compared
to threshold and prediction-based baselines, it achieves more predictable
VM-level performance and also reduces the amount of virtual CPUs and memory
provisioned by up to 35% and 60% respectively for synthetic workloads on real
clusters.
Graph based Question Answering System
Authors: Piyush Mital, Saurabh Agarwal, Bhargavi Neti, Yashodhara Haribhakta, Vibhavari Kamble, Krishnanjan Bhattacharjee, Debashri Das, Swati Mehta, Ajai Kumar
Date: 2018-12-05 00:00:00
Tasks: Question Answering, Retrieval

In today's digital age in the dawning era of big data analytics it is not the
information but the linking of information through entities and actions which
defines the discourse. Any textual data either available on the Internet off
off-line (like newspaper data, Wikipedia dump, etc) is basically connect
information which cannot be treated isolated for its wholesome semantics. There
is a need for an automated retrieval process with proper information extraction
to structure the data for relevant and fast text analytics. The first big
challenge is the conversion of unstructured textual data to structured data.
Unlike other databases, graph databases handle relationships and connections
elegantly. Our project aims at developing a graph-based information extraction
and retrieval system.
Cooperative Multi-Agent Policy Gradients with Sub-optimal Demonstration
Authors: Peixi Peng, Junliang Xing
Date: 2018-12-05 00:00:00

Many reality tasks such as robot coordination can be naturally modelled as multi-agent cooperative system where the rewards are sparse. This paper focuses on learning decentralized policies for such tasks using sub-optimal demonstration. To learn the multi-agent cooperation effectively and tackle the sub-optimality of demonstration, a self-improving learning method is proposed: On the one hand, the centralized state-action values are initialized by the demonstration and updated by the learned decentralized policy to improve the sub-optimality. On the other hand, the Nash Equilibrium are found by the current state-action value and are used as a guide to learn the policy. The proposed method is evaluated on the combat RTS games which requires a high level of multi-agent cooperation. Extensive experimental results on various combat scenarios demonstrate that the proposed method can learn multi-agent cooperation effectively. It significantly outperforms many state-of-the-art demonstration based approaches.
Regularized Ensembles and Transferability in Adversarial Learning
Authors: Yifan Chen, Yevgeniy Vorobeychik
Date: 2018-12-05 00:00:00

Despite the considerable success of convolutional neural networks in a broad
array of domains, recent research has shown these to be vulnerable to small
adversarial perturbations, commonly known as adversarial examples. Moreover,
such examples have shown to be remarkably portable, or transferable, from one
model to another, enabling highly successful black-box attacks. We explore this
issue of transferability and robustness from two dimensions: first, considering
the impact of conventional $l_p$ regularization as well as replacing the top
layer with a linear support vector machine (SVM), and second, the value of
combining regularized models into an ensemble. We show that models trained with
different regularizers present barriers to transferability, as does partial
information about the models comprising the ensemble.
An Embarrassingly Simple Approach for Knowledge Distillation
Authors: Mengya Gao, Yujun Shen, Quanquan Li, Junjie Yan, Liang Wan, Dahua Lin, Chen Change Loy, Xiaoou Tang
Date: 2018-12-05 00:00:00
Tasks: Face Recognition, Knowledge Distillation, object-detection, Object Detection, Transfer Learning

Knowledge Distillation (KD) aims at improving the performance of a low-capacity student model by inheriting knowledge from a high-capacity teacher model. Previous KD methods typically train a student by minimizing a task-related loss and the KD loss simultaneously, using a pre-defined loss weight to balance these two terms. In this work, we propose to first transfer the backbone knowledge from a teacher to the student, and then only learn the task-head of the student network. Such a decomposition of the training process circumvents the need of choosing an appropriate loss weight, which is often difficult in practice, and thus makes it easier to apply to different datasets and tasks. Importantly, the decomposition permits the core of our method, Stage-by-Stage Knowledge Distillation (SSKD), which facilitates progressive feature mimicking from teacher to student. Extensive experiments on CIFAR-100 and ImageNet suggest that SSKD significantly narrows down the performance gap between student and teacher, outperforming state-of-the-art approaches. We also demonstrate the generalization ability of SSKD on other challenging benchmarks, including face recognition on IJB-A dataset as well as object detection on COCO dataset.
Photo-Realistic Blocksworld Dataset
Authors: Masataro Asai
Date: 2018-12-05 00:00:00
Tasks: Task Planning

In this report, we introduce an artificial dataset generator for
Photo-realistic Blocksworld domain. Blocksworld is one of the oldest high-level
task planning domain that is well defined but contains sufficient complexity,
e.g., the conflicting subgoals and the decomposability into subproblems. We aim
to make this dataset a benchmark for Neural-Symbolic integrated systems and
accelerate the research in this area. The key advantage of such systems is the
ability to obtain a symbolic model from the real-world input and perform a
fast, systematic, complete algorithm for symbolic reasoning, without any
supervision and the reward signal from the environment.
Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on Zero-One Loss
Authors: Stephen Mussmann, Percy Liang
Date: 2018-12-05 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Active Learning

Uncertainty sampling, a popular active learning algorithm, is used to reduce
the amount of data required to learn a classifier, but it has been observed in
practice to converge to different parameters depending on the initialization
and sometimes to even better parameters than standard training on all the data.
In this work, we give a theoretical explanation of this phenomenon, showing
that uncertainty sampling on a convex loss can be interpreted as performing a
preconditioned stochastic gradient step on a smoothed version of the population
zero-one loss that converges to the population zero-one loss. Furthermore,
uncertainty sampling moves in a descent direction and converges to stationary
points of the smoothed population zero-one loss. Experiments on synthetic and
real datasets support this connection.
Machine-learned epidemiology: real-time detection of foodborne illness at scale
Authors: Adam Sadilek, Stephanie Caty, Lauren DiPrete, Raed Mansour, Tom Schenk Jr, Mark Bergtholdt, Ashish Jha, Prem Ramaswami, Evgeniy Gabrilovich
Date: 2018-12-05 00:00:00
Tasks: Epidemiology

Machine learning has become an increasingly powerful tool for solving complex
problems, and its application in public health has been underutilized. The
objective of this study is to test the efficacy of a machine-learned model of
foodborne illness detection in a real-world setting. To this end, we built
FINDER, a machine-learned model for real-time detection of foodborne illness
using anonymous and aggregated web search and location data. We computed the
fraction of people who visited a particular restaurant and later searched for
terms indicative of food poisoning to identify potentially unsafe restaurants.
We used this information to focus restaurant inspections in two cities and
demonstrated that FINDER improves the accuracy of health inspections;
restaurants identified by FINDER are 3.1 times as likely to be deemed unsafe
during the inspection as restaurants identified by existing methods.
Additionally, FINDER enables us to ascertain previously intractable
epidemiological information, for example, in 38% of cases the restaurant
potentially causing food poisoning was not the last one visited, which may
explain the lower precision of complaint-based inspections. We found that
FINDER is able to reliably identify restaurants that have an active lapse in
food safety, allowing for implementation of corrective actions that would
prevent the potential spread of foodborne illness.
Random Spiking and Systematic Evaluation of Defenses Against Adversarial Examples
Authors: Huangyi Ge, Sze Yiu Chau, Bruno Ribeiro, Ninghui Li
Date: 2018-12-05 00:00:00

Image classifiers often suffer from adversarial examples, which are generated by strategically adding a small amount of noise to input images to trick classifiers into misclassification. Over the years, many defense mechanisms have been proposed, and different researchers have made seemingly contradictory claims on their effectiveness. We present an analysis of possible adversarial models, and propose an evaluation framework for comparing different defense mechanisms. As part of the framework, we introduce a more powerful and realistic adversary strategy. Furthermore, we propose a new defense mechanism called Random Spiking (RS), which generalizes dropout and introduces random noises in the training process in a controlled manner. Evaluations under our proposed framework suggest RS delivers better protection against adversarial examples than many existing schemes.
ECC: Platform-Independent Energy-Constrained Deep Neural Network Compression via a Bilinear Regression Model
Authors: Haichuan Yang, Yuhao Zhu, Ji Liu
Date: 2018-12-05 00:00:00
Proceeding: CVPR 2019 6
Tasks: Neural Network Compression, regression

Many DNN-enabled vision applications constantly operate under severe energy
constraints such as unmanned aerial vehicles, Augmented Reality headsets, and
smartphones. Designing DNNs that can meet a stringent energy budget is becoming
increasingly important. This paper proposes ECC, a framework that compresses
DNNs to meet a given energy constraint while minimizing accuracy loss. The key
idea of ECC is to model the DNN energy consumption via a novel bilinear
regression function. The energy estimate model allows us to formulate DNN
compression as a constrained optimization that minimizes the DNN loss function
over the energy constraint. The optimization problem, however, has nontrivial
constraints. Therefore, existing deep learning solvers do not apply directly.
We propose an optimization algorithm that combines the essence of the
Alternating Direction Method of Multipliers (ADMM) framework with
gradient-based learning algorithms. The algorithm decomposes the original
constrained optimization into several subproblems that are solved iteratively
and efficiently. ECC is also portable across different hardware platforms
without requiring hardware knowledge. Experiments show that ECC achieves higher
accuracy under the same or lower energy budget compared to state-of-the-art
resource-constrained DNN compression techniques.
Visual Attention for Behavioral Cloning in Autonomous Driving
Authors: Sourav Pal, Tharun Mohandoss, Pabitra Mitra
Date: 2018-12-05 00:00:00
Tasks: Autonomous Driving

The goal of our work is to use visual attention to enhance autonomous driving
performance. We present two methods of predicting visual attention maps. The
first method is a supervised learning approach in which we collect eye-gaze
data for the task of driving and use this to train a model for predicting the
attention map. The second method is a novel unsupervised approach where we
train a model to learn to predict attention as it learns to drive a car.
Finally, we present a comparative study of our results and show that the
supervised approach for predicting attention when incorporated performs better
than other approaches.