pwc-archive/papers-with-abstracts huggingface dataset:
Channel selection using Gumbel Softmax
Authors: Charles Herrmann, Richard Strong Bowen, Ramin Zabih
Date: 2018-12-11 00:00:00
Proceeding: ECCV 2020 8
Tasks: channel selection, Classification, General Classification

Important applications such as mobile computing require reducing the computational costs of neural network inference. Ideally, applications would specify their preferred tradeoff between accuracy and speed, and the network would optimize this end-to-end, using classification error to remove parts of the network. Increasing speed can be done either during training - e.g., pruning filters - or during inference - e.g., conditionally executing a subset of the layers. We propose a single end-to-end framework that can improve inference efficiency in both settings. We use a combination of batch activation loss and classification loss, and Gumbel reparameterization to learn network structure. We train end-to-end, and the same technique supports pruning as well as conditional computation. We obtain promising experimental results for ImageNet classification with ResNet (45-52% less computation).
Material Based Object Tracking in Hyperspectral Videos: Benchmark and Algorithms
Authors: Fengchao Xiong, Jun Zhou, Yuntao Qian
Date: 2018-12-11 00:00:00
Tasks: Object, Object Tracking

Traditional color images only depict color intensities in red, green and blue channels, often making object trackers fail in challenging scenarios, e.g., background clutter and rapid changes of target appearance. Alternatively, material information of targets contained in a large amount of bands of hyperspectral images (HSI) is more robust to these difficult conditions. In this paper, we conduct a comprehensive study on how material information can be utilized to boost object tracking from three aspects: benchmark dataset, material feature representation and material based tracking. In terms of benchmark, we construct a dataset of fully-annotated videos, which contain both hyperspectral and color sequences of the same scene. Material information is represented by spectral-spatial histogram of multidimensional gradient, which describes the 3D local spectral-spatial structure in an HSI, and fractional abundances of constituted material components which encode the underlying material distribution. These two types of features are embedded into correlation filters, yielding material based tracking. Experimental results on the collected benchmark dataset show the potentials and advantages of material based object tracking.
Learning Discriminative Motion Features Through Detection
Authors: Gedas Bertasius, Christoph Feichtenhofer, Du Tran, Jianbo Shi, Lorenzo Torresani
Date: 2018-12-11 00:00:00
Tasks: Action Localization, Action Recognition, Fine-grained Action Recognition, Pose Estimation, Temporal Action Localization

Despite huge success in the image domain, modern detection models such as
Faster R-CNN have not been used nearly as much for video analysis. This is
arguably due to the fact that detection models are designed to operate on
single frames and as a result do not have a mechanism for learning motion
representations directly from video. We propose a learning procedure that
allows detection models such as Faster R-CNN to learn motion features directly
from the RGB video data while being optimized with respect to a pose estimation
task. Given a pair of video frames---Frame A and Frame B---we force our model
to predict human pose in Frame A using the features from Frame B. We do so by
leveraging deformable convolutions across space and time. Our network learns to
spatially sample features from Frame B in order to maximize pose detection
accuracy in Frame A. This naturally encourages our network to learn motion
offsets encoding the spatial correspondences between the two frames. We refer
to these motion offsets as DiMoFs (Discriminative Motion Features).
  In our experiments we show that our training scheme helps learn effective
motion cues, which can be used to estimate and localize salient human motion.
Furthermore, we demonstrate that as a byproduct, our model also learns features
that lead to improved pose detection in still-images, and better keypoint
tracking. Finally, we show how to leverage our learned model for the tasks of
spatiotemporal action localization and fine-grained action recognition.
Delta Embedding Learning
Authors: Xiao Zhang, Ji Wu, Dejing Dou
Date: 2018-12-11 00:00:00
Proceeding: ACL 2019 7
Tasks: Reading Comprehension, Word Embeddings

Unsupervised word embeddings have become a popular approach of word representation in NLP tasks. However there are limitations to the semantics represented by unsupervised embeddings, and inadequate fine-tuning of embeddings can lead to suboptimal performance. We propose a novel learning technique called Delta Embedding Learning, which can be applied to general NLP tasks to improve performance by optimized tuning of the word embeddings. A structured regularization is applied to the embeddings to ensure they are tuned in an incremental way. As a result, the tuned word embeddings become better word representations by absorbing semantic information from supervision without "forgetting." We apply the method to various NLP tasks and see a consistent improvement in performance. Evaluation also confirms the tuned word embeddings have better semantic properties.
Deep Learning for Robotic Mass Transport Cloaking
Authors: Reza Khodayi-mehr, Michael M. Zavlanos
Date: 2018-12-11 00:00:00
Tasks: Deep Learning

We consider the problem of mass transport cloaking using mobile robots. The robots move along a predefined curve that encloses a safe zone and carry sources that collectively counteract a chemical agent released in the environment. The goal is to steer the mass flux around a desired region so that it remains unaffected by the external concentration. We formulate the problem of controlling the robot positions and release rates as a PDE-constrained optimization, where the propagation of the chemical is modeled by the advection-diffusion (AD) PDE. We use a neural network (NN) to approximate the solution of the PDE. Particularly, we propose a novel loss function for the NN that utilizes the variational form of the AD-PDE and allows us to reformulate the planning problem as an unsupervised model-based learning problem. Our loss function is discretization-free and highly parallelizable. Unlike passive cloaking methods that use metamaterials to steer the mass flux, our method is the first to use mobile robots to actively control the concentration levels and create safe zones independent of environmental conditions. We demonstrate the performance of our method in simulations.
Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention
Authors: Khanh Nguyen, Debadeepta Dey, Chris Brockett, Bill Dolan
Date: 2018-12-10 00:00:00
Proceeding: CVPR 2019 6
Tasks: Imitation Learning, Navigate, Vision-based navigation with language-based assistance, VNLA

We present Vision-based Navigation with Language-based Assistance (VNLA), a
grounded vision-language task where an agent with visual perception is guided
via language to find objects in photorealistic indoor environments. The task
emulates a real-world scenario in that (a) the requester may not know how to
navigate to the target objects and thus makes requests by only specifying
high-level end-goals, and (b) the agent is capable of sensing when it is lost
and querying an advisor, who is more qualified at the task, to obtain language
subgoals to make progress. To model language-based assistance, we develop a
general framework termed Imitation Learning with Indirect Intervention (I3L),
and propose a solution that is effective on the VNLA task. Empirical results
show that this approach significantly improves the success rate of the learning
agent over other baselines in both seen and unseen environments. Our code and
data are publicly available at https://github.com/debadeepta/vnla .
Duelling Bandits with Weak Regret in Adversarial Environments
Authors: Lennard Hilgendorf
Date: 2018-12-10 00:00:00

Research on the multi-armed bandit problem has studied the trade-off of
exploration and exploitation in depth. However, there are numerous applications
where the cardinal absolute-valued feedback model (e.g. ratings from one to
five) is not suitable. This has motivated the formulation of the duelling
bandits problem, where the learner picks a pair of actions and observes a noisy
binary feedback, indicating a relative preference between the two. There exist
a multitude of different settings and interpretations of the problem for two
reasons. First, due to the absence of a total order of actions, there is no
natural definition of the best action. Existing work either explicitly assumes
the existence of a linear order, or uses a custom definition for the winner.
Second, there are multiple reasonable notions of regret to measure the
learner's performance. Most prior work has been focussing on the
$\textit{strong regret}$, which averages the quality of the two actions picked.
This work focusses on the $\textit{weak regret}$, which is based on the quality
of the better of the two actions selected. Weak regret is the more appropriate
performance measure when the pair's inferior action has no significant
detrimental effect on the pair's quality.
  We study the duelling bandits problem in the adversarial setting. We provide
an algorithm which has theoretical guarantees in both the utility-based
setting, which implies a total order, and the unrestricted setting. For the
latter, we work with the $\textit{Borda winner}$, finding the action maximising
the probability of winning against an action sampled uniformly at random. The
thesis concludes with experimental results based on both real-world data and
synthetic data, showing the algorithm's performance and limitations.
Learning Sharing Behaviors with Arbitrary Numbers of Agents
Authors: Katherine Metcalf, Barry-John Theobald, Nicholas Apostoloff
Date: 2018-12-10 00:00:00
Tasks: Q-Learning

We propose a method for modeling and learning turn-taking behaviors for
accessing a shared resource. We model the individual behavior for each agent in
an interaction and then use a multi-agent fusion model to generate a summary
over the expected actions of the group to render the model independent of the
number of agents. The individual behavior models are weighted finite state
transducers (WFSTs) with weights dynamically updated during interactions, and
the multi-agent fusion model is a logistic regression classifier.
  We test our models in a multi-agent tower-building environment, where a
Q-learning agent learns to interact with rule-based agents. Our approach
accurately models the underlying behavior patterns of the rule-based agents
with accuracy ranging between 0.63 and 1.0 depending on the stochasticity of
the other agent behaviors. In addition we show using KL-divergence that the
model accurately captures the distribution of next actions when interacting
with both a single agent (KL-divergence < 0.1) and with multiple agents
(KL-divergence < 0.37). Finally, we demonstrate that our behavior model can be
used by a Q-learning agent to take turns in an interactive turn-taking
environment.
Probabilistic Model Checking of Robots Deployed in Extreme Environments
Authors: Xingyu Zhao, Valentin Robu, David Flynn, Fateme Dinmohammadi, Michael Fisher, Matt Webster
Date: 2018-12-10 00:00:00
Tasks: Bayesian Inference

Robots are increasingly used to carry out critical missions in extreme
environments that are hazardous for humans. This requires a high degree of
operational autonomy under uncertain conditions, and poses new challenges for
assuring the robot's safety and reliability. In this paper, we develop a
framework for probabilistic model checking on a layered Markov model to verify
the safety and reliability requirements of such robots, both at pre-mission
stage and during runtime. Two novel estimators based on conservative Bayesian
inference and imprecise probability model with sets of priors are introduced to
learn the unknown transition parameters from operational data. We demonstrate
our approach using data from a real-world deployment of unmanned underwater
vehicles in extreme environments.
Montage based 3D Medical Image Retrieval from Traumatic Brain Injury Cohort using Deep Convolutional Neural Network
Authors: Cailey I. Kerley, Yuankai Huo, Shikha Chaganti, Shunxing Bao, Mayur B. Patel, Bennett A. Landman
Date: 2018-12-10 00:00:00
Tasks: Computed Tomography (CT), Image Retrieval, Medical Image Retrieval, Retrieval

Brain imaging analysis on clinically acquired computed tomography (CT) is
essential for the diagnosis, risk prediction of progression, and treatment of
the structural phenotypes of traumatic brain injury (TBI). However, in real
clinical imaging scenarios, entire body CT images (e.g., neck, abdomen, chest,
pelvis) are typically captured along with whole brain CT scans. For instance,
in a typical sample of clinical TBI imaging cohort, only ~15% of CT scans
actually contain whole brain CT images suitable for volumetric brain analyses;
the remaining are partial brain or non-brain images. Therefore, a manual image
retrieval process is typically required to isolate the whole brain CT scans
from the entire cohort. However, the manual image retrieval is time and
resource consuming and even more difficult for the larger cohorts. To alleviate
the manual efforts, in this paper we propose an automated 3D medical image
retrieval pipeline, called deep montage-based image retrieval (dMIR), which
performs classification on 2D montage images via a deep convolutional neural
network. The novelty of the proposed method for image processing is to
characterize the medical image retrieval task based on the montage images. In a
cohort of 2000 clinically acquired TBI scans, 794 scans were used as training
data, 206 scans were used as validation data, and the remaining 1000 scans were
used as testing data. The proposed achieved accuracy=1.0, recall=1.0,
precision=1.0, f1=1.0 for validation data, while achieved accuracy=0.988,
recall=0.962, precision=0.962, f1=0.962 for testing data. Thus, the proposed
dMIR is able to perform accurate CT whole brain image retrieval from
large-scale clinical cohorts.
Top-N-Rank: A Scalable List-wise Ranking Method for Recommender Systems
Authors: Junjie Liang, Jinlong Hu, Shoubin Dong, Vasant Honavar
Date: 2018-12-10 00:00:00
Tasks: Learning-To-Rank, Recommendation Systems

We propose Top-N-Rank, a novel family of list-wise Learning-to-Rank models
for reliably recommending the N top-ranked items. The proposed models optimize
a variant of the widely used discounted cumulative gain (DCG) objective
function which differs from DCG in two important aspects: (i) It limits the
evaluation of DCG only on the top N items in the ranked lists, thereby
eliminating the impact of low-ranked items on the learned ranking function; and
(ii) it incorporates weights that allow the model to leverage multiple types of
implicit feedback with differing levels of reliability or trustworthiness.
Because the resulting objective function is non-smooth and hence challenging to
optimize, we consider two smooth approximations of the objective function,
using the traditional sigmoid function and the rectified linear unit (ReLU). We
propose a family of learning-to-rank algorithms (Top-N-Rank) that work with any
smooth objective function. Then, a more efficient variant, Top-N-Rank.ReLU, is
introduced, which effectively exploits the properties of ReLU function to
reduce the computational complexity of Top-N-Rank from quadratic to linear in
the average number of items rated by users. The results of our experiments
using two widely used benchmarks, namely, the MovieLens data set and the Amazon
Video Games data set demonstrate that: (i) The `top-N truncation' of the
objective function substantially improves the ranking quality of the top N
recommendations; (ii) using the ReLU for smoothing the objective function
yields significant improvement in both ranking quality as well as runtime as
compared to using the sigmoid; and (iii) Top-N-Rank.ReLU substantially
outperforms the well-performing list-wise ranking methods in terms of ranking
quality.
Non-local U-Net for Biomedical Image Segmentation
Authors: Zhengyang Wang, Na Zou, Dinggang Shen, Shuiwang Ji
Date: 2018-12-10 00:00:00
Tasks: Brain Image Segmentation, Decoder, Image Segmentation, Segmentation, Semantic Segmentation

Deep learning has shown its great promise in various biomedical image segmentation tasks. Existing models are typically based on U-Net and rely on an encoder-decoder architecture with stacked local operators to aggregate long-range information gradually. However, only using the local operators limits the efficiency and effectiveness. In this work, we propose the non-local U-Nets, which are equipped with flexible global aggregation blocks, for biomedical image segmentation. These blocks can be inserted into U-Net as size-preserving processes, as well as down-sampling and up-sampling layers. We perform thorough experiments on the 3D multimodality isointense infant brain MR image segmentation task to evaluate the non-local U-Nets. Results show that our proposed models achieve top performances with fewer parameters and faster computation.
The Effects of Super-Resolution on Object Detection Performance in Satellite Imagery
Authors: Jacob Shermeyer, Adam Van Etten
Date: 2018-12-10 00:00:00
Tasks: Object, object-detection, Object Detection, satellite image super-resolution, Super-Resolution

We explore the application of super-resolution techniques to satellite
imagery, and the effects of these techniques on object detection algorithm
performance. Specifically, we enhance satellite imagery beyond its native
resolution, and test if we can identify various types of vehicles, planes, and
boats with greater accuracy than native resolution. Using the Very Deep
Super-Resolution (VDSR) framework and a custom Random Forest Super-Resolution
(RFSR) framework we generate enhancement levels of 2x, 4x, and 8x over five
distinct resolutions ranging from 30 cm to 4.8 meters. Using both native and
super-resolved data, we then train several custom detection models using the
SIMRDWN object detection framework. SIMRDWN combines a number of popular object
detection algorithms (e.g. SSD, YOLO) into a unified framework that is designed
to rapidly detect objects in large satellite images. This approach allows us to
quantify the effects of super-resolution techniques on object detection
performance across multiple classes and resolutions. We also quantify the
performance of object detection as a function of native resolution and object
pixel size. For our test set we note that performance degrades from mean
average precision (mAP) = 0.53 at 30 cm resolution, down to mAP = 0.11 at 4.8 m
resolution. Super-resolving native 30 cm imagery to 15 cm yields the greatest
benefit; a 13-36% improvement in mAP. Super-resolution is less beneficial at
coarser resolutions, though still provides a small improvement in performance.
Visual Depth Mapping from Monocular Images using Recurrent Convolutional Neural Networks
Authors: John Mern, Kyle Julian, Rachael E. Tompa, Mykel J. Kochenderfer
Date: 2018-12-10 00:00:00
Tasks: Collision Avoidance, Depth And Camera Motion

A reliable sense-and-avoid system is critical to enabling safe autonomous
operation of unmanned aircraft. Existing sense-and-avoid methods often require
specialized sensors that are too large or power intensive for use on small
unmanned vehicles. This paper presents a method to estimate object distances
based on visual image sequences, allowing for the use of low-cost, on-board
monocular cameras as simple collision avoidance sensors. We present a deep
recurrent convolutional neural network and training method to generate depth
maps from video sequences. Our network is trained using simulated camera and
depth data generated with Microsoft's AirSim simulator. Empirically, we show
that our model achieves superior performance compared to models generated using
prior methods.We further demonstrate that the method can be used for
sense-and-avoid of obstacles in simulation.
Chat-crowd: A Dialog-based Platform for Visual Layout Composition
Authors: Paola Cascante-Bonilla, Xuwang Yin, Vicente Ordonez, Song Feng
Date: 2018-12-10 00:00:00
Proceeding: NAACL 2019 6
Tasks: Goal-Oriented Dialog

In this paper we introduce Chat-crowd, an interactive environment for visual
layout composition via conversational interactions. Chat-crowd supports
multiple agents with two conversational roles: agents who play the role of a
designer are in charge of placing objects in an editable canvas according to
instructions or commands issued by agents with a director role. The system can
be integrated with crowdsourcing platforms for both synchronous and
asynchronous data collection and is equipped with comprehensive quality
controls on the performance of both types of agents. We expect that this system
will be useful to build multimodal goal-oriented dialog tasks that require
spatial and geometric reasoning.
PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image
Authors: Chen Liu, Kihwan Kim, Jinwei Gu, Yasutaka Furukawa, Jan Kautz
Date: 2018-12-10 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Plane Detection, 3D Reconstruction, Segmentation

This paper proposes a deep neural architecture, PlaneRCNN, that detects and
reconstructs piecewise planar surfaces from a single RGB image. PlaneRCNN
employs a variant of Mask R-CNN to detect planes with their plane parameters
and segmentation masks. PlaneRCNN then jointly refines all the segmentation
masks with a novel loss enforcing the consistency with a nearby view during
training. The paper also presents a new benchmark with more fine-grained plane
segmentations in the ground-truth, in which, PlaneRCNN outperforms existing
state-of-the-art methods with significant margins in the plane detection,
segmentation, and reconstruction metrics. PlaneRCNN makes an important step
towards robust plane extraction, which would have an immediate impact on a wide
range of applications including Robotics, Augmented Reality, and Virtual
Reality.
Individual Fairness in Hindsight
Authors: Swati Gupta, Vijay Kamble
Date: 2018-12-10 00:00:00
Tasks: Decision Making, Fairness

Since many critical decisions impacting human lives are increasingly being
made by algorithms, it is important to ensure that the treatment of individuals
under such algorithms is demonstrably fair under reasonable notions of
fairness. One compelling notion proposed in the literature is that of
individual fairness (IF), which advocates that similar individuals should be
treated similarly (Dwork et al. 2012). Originally proposed for offline
decisions, this notion does not, however, account for temporal considerations
relevant for online decision-making. In this paper, we extend the notion of IF
to account for the time at which a decision is made, in settings where there
exists a notion of conduciveness of decisions as perceived by the affected
individuals. We introduce two definitions: (i) fairness-across-time (FT) and
(ii) fairness-in-hindsight (FH). FT is the simplest temporal extension of IF
where treatment of individuals is required to be individually fair relative to
the past as well as future, while in FH, we require a one-sided notion of
individual fairness that is defined relative to only the past decisions. We
show that these two definitions can have drastically different implications in
the setting where the principal needs to learn the utility model. Linear regret
relative to optimal individually fair decisions is inevitable under FT for
non-trivial examples. On the other hand, we design a new algorithm: Cautious
Fair Exploration (CaFE), which satisfies FH and achieves sub-linear regret
guarantees for a broad range of settings. We characterize lower bounds showing
that these guarantees are order-optimal in the worst case. FH can thus be
embedded as a primary safeguard against unfair discrimination in algorithmic
deployments, without hindering the ability to take good decisions in the
long-run.