pwc-archive/papers-with-abstracts huggingface dataset:
Appearance-based Gesture recognition in the compressed domain
Authors: Shaojie Xu, Anvesha Amaravati, Justin Romberg, Arijit Raychowdhury
Date: 2019-02-19 00:00:00
Tasks: Computational Efficiency, Gesture Recognition

We propose a novel appearance-based gesture recognition algorithm using
compressed domain signal processing techniques. Gesture features are extracted
directly from the compressed measurements, which are the block averages and the
coded linear combinations of the image sensor's pixel values. We also improve
both the computational efficiency and the memory requirement of the previous
DTW-based K-NN gesture classifiers. Both simulation testing and hardware
implementation strongly support the proposed algorithm.
Quantifying contribution and propagation of error from computational steps, algorithms and hyperparameter choices in image classification pipelines
Authors: Aritra Chowdhury, Malik Magdon-Ismail, Bulent Yener
Date: 2019-02-21 00:00:00
Tasks: Bayesian Optimization, BIG-bench Machine Learning, General Classification, Hyperparameter Optimization, image-classification, Image Classification

Data science relies on pipelines that are organized in the form of
interdependent computational steps. Each step consists of various candidate
algorithms that maybe used for performing a particular function. Each algorithm
consists of several hyperparameters. Algorithms and hyperparameters must be
optimized as a whole to produce the best performance. Typical machine learning
pipelines consist of complex algorithms in each of the steps. Not only is the
selection process combinatorial, but it is also important to interpret and
understand the pipelines. We propose a method to quantify the importance of
different components in the pipeline, by computing an error contribution
relative to an agnostic choice of computational steps, algorithms and
hyperparameters. We also propose a methodology to quantify the propagation of
error from individual components of the pipeline with the help of a naive set
of benchmark algorithms not involved in the pipeline. We demonstrate our
methodology on image classification pipelines. The agnostic and naive
methodologies quantify the error contribution and propagation respectively from
the computational steps, algorithms and hyperparameters in the image
classification pipeline. We show that algorithm selection and hyperparameter
optimization methods like grid search, random search and Bayesian optimization
can be used to quantify the error contribution and propagation, and that random
search is able to quantify them more accurately than Bayesian optimization.
This methodology can be used by domain experts to understand machine learning
and data analysis pipelines in terms of their individual components, which can
help in prioritizing different components of the pipeline.
Real-world multiobject, multigrasp detection
Authors: Fu-Jen Chu, Ruinian Xu and Patricio A. Vela
Date: 2018-10-01 00:00:00
Proceeding: IEEE ROBOTICS AND AUTOMATION LETTERS 2018 10
Tasks: Object, Robotic Grasping

A deep learning architecture is proposed to predict graspable locations for robotic manipulation. It considers situations where no, one, or multiple object(s) are seen. By defining the learning problem to be classified with null hypothesis competition instead of regression, the deep neural network with red, green, blue and depth (RGB-D) image input predicts multiple grasp candidates for a single object or multiple objects, in a single shot. The method outperforms state-of-the-art approaches on the Cornell dataset with 96.0% and 96.1% accuracy on imagewise and object-wise splits, respectively. Evaluation on a multiobject dataset illustrates the generalization capability of the architecture. Grasping experiments achieve 96.0% grasp localization and 89.0% grasping success rates on a test set of household objects. The real-time process takes less than 0.25 s from image to plan.
RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion
Authors: Jie Li, Yu Liu, Dong Gong, Qinfeng Shi, Xia Yuan, Chunxia Zhao, Ian Reid
Date: 2019-03-02 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Semantic Scene Completion, Scene Labeling

RGB images differentiate from depth images as they carry more details about
the color and texture information, which can be utilized as a vital
complementary to depth for boosting the performance of 3D semantic scene
completion (SSC). SSC is composed of 3D shape completion (SC) and semantic
scene labeling while most of the existing methods use depth as the sole input
which causes the performance bottleneck. Moreover, the state-of-the-art methods
employ 3D CNNs which have cumbersome networks and tremendous parameters. We
introduce a light-weight Dimensional Decomposition Residual network (DDR) for
3D dense prediction tasks. The novel factorized convolution layer is effective
for reducing the network parameters, and the proposed multi-scale fusion
mechanism for depth and color image can improve the completion and segmentation
accuracy simultaneously. Our method demonstrates excellent performance on two
public datasets. Compared with the latest method SSCNet, we achieve 5.9% gains
in SC-IoU and 5.7% gains in SSC-IOU, albeit with only 21% network parameters
and 16.6% FLOPs employed compared with that of SSCNet.
Feature Selective Anchor-Free Module for Single-Shot Object Detection
Authors: Chenchen Zhu, Yihui He, Marios Savvides
Date: 2019-03-02 00:00:00
Proceeding: CVPR 2019 6
Tasks: feature selection, object-detection, Object Detection

We motivate and present feature selective anchor-free (FSAF) module, a simple
and effective building block for single-shot object detectors. It can be
plugged into single-shot detectors with feature pyramid structure. The FSAF
module addresses two limitations brought up by the conventional anchor-based
detection: 1) heuristic-guided feature selection; 2) overlap-based anchor
sampling. The general concept of the FSAF module is online feature selection
applied to the training of multi-level anchor-free branches. Specifically, an
anchor-free branch is attached to each level of the feature pyramid, allowing
box encoding and decoding in the anchor-free manner at an arbitrary level.
During training, we dynamically assign each instance to the most suitable
feature level. At the time of inference, the FSAF module can work jointly with
anchor-based branches by outputting predictions in parallel. We instantiate
this concept with simple implementations of anchor-free branches and online
feature selection strategy. Experimental results on the COCO detection track
show that our FSAF module performs better than anchor-based counterparts while
being faster. When working jointly with anchor-based branches, the FSAF module
robustly improves the baseline RetinaNet by a large margin under various
settings, while introducing nearly free inference overhead. And the resulting
best model can achieve a state-of-the-art 44.6% mAP, outperforming all existing
single-shot detectors on COCO.
PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation
Authors: Fenggen Yu, Kun Liu, Yan Zhang, Chenyang Zhu, Kai Xu
Date: 2019-03-02 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Instance Segmentation, 3D Part Segmentation, Segmentation, Semantic Segmentation

Deep learning approaches to 3D shape segmentation are typically formulated as a multi-class labeling problem. Existing models are trained for a fixed set of labels, which greatly limits their flexibility and adaptivity. We opt for top-down recursive decomposition and develop the first deep learning model for hierarchical segmentation of 3D shapes, based on recursive neural networks. Starting from a full shape represented as a point cloud, our model performs recursive binary decomposition, where the decomposition network at all nodes in the hierarchy share weights. At each node, a node classifier is trained to determine the type (adjacency or symmetry) and stopping criteria of its decomposition. The features extracted in higher level nodes are recursively propagated to lower level ones. Thus, the meaningful decompositions in higher levels provide strong contextual cues constraining the segmentations in lower levels. Meanwhile, to increase the segmentation accuracy at each node, we enhance the recursive contextual feature with the shape feature extracted for the corresponding part. Our method segments a 3D shape in point cloud into an unfixed number of parts, depending on the shape complexity, showing strong generality and flexibility. It achieves the state-of-the-art performance, both for fine-grained and semantic segmentation, on the public benchmark and a new benchmark of fine-grained segmentation proposed in this work. We also demonstrate its application for fine-grained part refinements in image-to-shape reconstruction.
AIRD: Adversarial Learning Framework for Image Repurposing Detection
Authors: Ayush Jaiswal, Yue Wu, Wael Abd-Almageed, Iacopo Masi, Premkumar Natarajan
Date: 2019-03-02 00:00:00
Proceeding: CVPR 2019 6
Tasks: Misinformation

Image repurposing is a commonly used method for spreading misinformation on
social media and online forums, which involves publishing untampered images
with modified metadata to create rumors and further propaganda. While manual
verification is possible, given vast amounts of verified knowledge available on
the internet, the increasing prevalence and ease of this form of semantic
manipulation call for the development of robust automatic ways of assessing the
semantic integrity of multimedia data. In this paper, we present a novel method
for image repurposing detection that is based on the real-world adversarial
interplay between a bad actor who repurposes images with counterfeit metadata
and a watchdog who verifies the semantic consistency between images and their
accompanying metadata, where both players have access to a reference dataset of
verified content, which they can use to achieve their goals. The proposed
method exhibits state-of-the-art performance on location-identity,
subject-identity and painting-artist verification, showing its efficacy across
a diverse set of scenarios.
3D Hand Shape and Pose Estimation from a Single RGB Image
Authors: Liuhao Ge, Zhou Ren, Yuncheng Li, Zehao Xue, Yingying Wang, Jianfei Cai, Junsong Yuan
Date: 2019-03-03 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Hand Pose Estimation, Hand Pose Estimation, Pose Estimation

This work addresses a novel and challenging problem of estimating the full 3D
hand shape and pose from a single RGB image. Most current methods in 3D hand
analysis from monocular RGB images only focus on estimating the 3D locations of
hand keypoints, which cannot fully express the 3D shape of hand. In contrast,
we propose a Graph Convolutional Neural Network (Graph CNN) based method to
reconstruct a full 3D mesh of hand surface that contains richer information of
both 3D hand shape and pose. To train networks with full supervision, we create
a large-scale synthetic dataset containing both ground truth 3D meshes and 3D
poses. When fine-tuning the networks on real-world datasets without 3D ground
truth, we propose a weakly-supervised approach by leveraging the depth map as a
weak supervision in training. Through extensive evaluations on our proposed new
datasets and two public datasets, we show that our proposed method can produce
accurate and reasonable 3D hand mesh, and can achieve superior 3D hand pose
estimation accuracy when compared with state-of-the-art methods.
Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing
Authors: Xihui Liu, ZiHao Wang, Jing Shao, Xiaogang Wang, Hongsheng Li
Date: 2019-03-03 00:00:00
Proceeding: CVPR 2019 6
Tasks: Referring Expression

Referring expression grounding aims at locating certain objects or persons in
an image with a referring expression, where the key challenge is to comprehend
and align various types of information from visual and textual domain, such as
visual attributes, location and interactions with surrounding regions. Although
the attention mechanism has been successfully applied for cross-modal
alignments, previous attention models focus on only the most dominant features
of both modalities, and neglect the fact that there could be multiple
comprehensive textual-visual correspondences between images and referring
expressions. To tackle this issue, we design a novel cross-modal
attention-guided erasing approach, where we discard the most dominant
information from either textual or visual domains to generate difficult
training samples online, and to drive the model to discover complementary
textual-visual correspondences. Extensive experiments demonstrate the
effectiveness of our proposed method, which achieves state-of-the-art
performance on three referring expression grounding datasets.
Less is More: Learning Highlight Detection from Video Duration
Authors: Bo Xiong, Yannis Kalantidis, Deepti Ghadiyaram, Kristen Grauman
Date: 2019-03-03 00:00:00
Proceeding: CVPR 2019 6
Tasks: Highlight Detection

Highlight detection has the potential to significantly ease video browsing,
but existing methods often suffer from expensive supervision requirements,
where human viewers must manually identify highlights in training videos. We
propose a scalable unsupervised solution that exploits video duration as an
implicit supervision signal. Our key insight is that video segments from
shorter user-generated videos are more likely to be highlights than those from
longer videos, since users tend to be more selective about the content when
capturing shorter videos. Leveraging this insight, we introduce a novel ranking
framework that prefers segments from shorter videos, while properly accounting
for the inherent noise in the (unlabeled) training data. We use it to train a
highlight detector with 10M hashtagged Instagram videos. In experiments on two
challenging public video highlight detection benchmarks, our method
substantially improves the state-of-the-art for unsupervised highlight
detection.
A Kernelized Manifold Mapping to Diminish the Effect of Adversarial Perturbations
Authors: Saeid Asgari Taghanaki, Kumar Abhishek, Shekoofeh Azizi, Ghassan Hamarneh
Date: 2019-03-03 00:00:00
Proceeding: CVPR 2019 6
Tasks: image-classification, Image Classification, Lesion Segmentation, Segmentation, Skin Lesion Segmentation

The linear and non-flexible nature of deep convolutional models makes them vulnerable to carefully crafted adversarial perturbations. To tackle this problem, we propose a non-linear radial basis convolutional feature mapping by learning a Mahalanobis-like distance function. Our method then maps the convolutional features onto a linearly well-separated manifold, which prevents small adversarial perturbations from forcing a sample to cross the decision boundary. We test the proposed method on three publicly available image classification and segmentation datasets namely, MNIST, ISBI ISIC 2017 skin lesion segmentation, and NIH Chest X-Ray-14. We evaluate the robustness of our method to different gradient (targeted and untargeted) and non-gradient based attacks and compare it to several non-gradient masking defense strategies. Our results demonstrate that the proposed method can increase the resilience of deep convolutional neural networks to adversarial perturbations without accuracy drop on clean data.
Collaborative Spatio-temporal Feature Learning for Video Action Recognition
Authors: Chao Li, Qiaoyong Zhong, Di Xie, ShiLiang Pu
Date: 2019-03-04 00:00:00
Tasks: Action Recognition, Action Recognition In Videos, Temporal Action Localization, Video Recognition

Spatio-temporal feature learning is of central importance for action
recognition in videos. Existing deep neural network models either learn spatial
and temporal features independently (C2D) or jointly with unconstrained
parameters (C3D). In this paper, we propose a novel neural operation which
encodes spatio-temporal features collaboratively by imposing a weight-sharing
constraint on the learnable parameters. In particular, we perform 2D
convolution along three orthogonal views of volumetric video data,which learns
spatial appearance and temporal motion cues respectively. By sharing the
convolution kernels of different views, spatial and temporal features are
collaboratively learned and thus benefit from each other. The complementary
features are subsequently fused by a weighted summation whose coefficients are
learned end-to-end. Our approach achieves state-of-the-art performance on
large-scale benchmarks and won the 1st place in the Moments in Time Challenge
2018. Moreover, based on the learned coefficients of different views, we are
able to quantify the contributions of spatial and temporal features. This
analysis sheds light on interpretability of the model and may also guide the
future design of algorithm for video recognition.
Sequential Design and Spatial Modeling for Portfolio Tail Risk Measurement
Authors: Michael Ludkovski, James Risk
Date: 2017-10-14 00:00:00
Tasks: Uncertainty Quantification

We consider calculation of capital requirements when the underlying economic
scenarios are determined by simulatable risk factors. In the respective nested
simulation framework, the goal is to estimate portfolio tail risk, quantified
via VaR or TVaR of a given collection of future economic scenarios representing
factor levels at the risk horizon. Traditionally, evaluating portfolio losses
of an outer scenario is done by computing a conditional expectation via
inner-level Monte Carlo and is computationally expensive. We introduce several
inter-related machine learning techniques to speed up this computation, in
particular by properly accounting for the simulation noise. Our main workhorse
is an advanced Gaussian Process (GP) regression approach which uses
nonparametric spatial modeling to efficiently learn the relationship between
the stochastic factors defining scenarios and corresponding portfolio value.
Leveraging this emulator, we develop sequential algorithms that adaptively
allocate inner simulation budgets to target the quantile region. The GP
framework also yields better uncertainty quantification for the resulting
VaR/TVaR estimators that reduces bias and variance compared to existing
methods. We illustrate the proposed strategies with two case-studies in two and
six dimensions.
Crowd Counting and Density Estimation by Trellis Encoder-Decoder Network
Authors: Xiaolong Jiang, Zehao Xiao, Baochang Zhang, Xian-Tong Zhen, Xian-Bin Cao, David Doermann, Ling Shao
Date: 2019-03-03 00:00:00
Tasks: Crowd Counting, Decoder, Density Estimation

Crowd counting has recently attracted increasing interest in computer vision
but remains a challenging problem. In this paper, we propose a trellis
encoder-decoder network (TEDnet) for crowd counting, which focuses on
generating high-quality density estimation maps. The major contributions are
four-fold. First, we develop a new trellis architecture that incorporates
multiple decoding paths to hierarchically aggregate features at different
encoding stages, which can handle large variations of objects. Second, we
design dense skip connections interleaved across paths to facilitate sufficient
multi-scale feature fusions and to absorb the supervision information. Third,
we propose a new combinatorial loss to enforce local coherence and spatial
correlation in density maps. By distributedly imposing this combinatorial loss
on intermediate outputs, gradient vanishing can be largely alleviated for
better back-propagation and faster convergence. Finally, our TEDnet achieves
new state-of-the art performance on four benchmarks, with an improvement up to
14% in terms of MAE.
An Optimistic Acceleration of AMSGrad for Nonconvex Optimization
Authors: Jun-Kun Wang, Xiaoyun Li, Belhal Karimi, Ping Li
Date: 2019-03-04 00:00:00
Proceeding: ICLR 2020 1

We propose a new variant of AMSGrad, a popular adaptive gradient based optimization algorithm widely used for training deep neural networks. Our algorithm adds prior knowledge about the sequence of consecutive mini-batch gradients and leverages its underlying structure making the gradients sequentially predictable. By exploiting the predictability and ideas from optimistic online learning, the proposed algorithm can accelerate the convergence and increase sample efficiency. After establishing a tighter upper bound under some convexity conditions on the regret, we offer a complimentary view of our algorithm which generalizes the offline and stochastic version of nonconvex optimization. In the nonconvex case, we establish a non-asymptotic convergence bound independently of the initialization. We illustrate the practical speedup on several deep learning models via numerical experiments.
VideoFlow: A Conditional Flow-Based Model for Stochastic Video Generation
Authors: Manoj Kumar, Mohammad Babaeizadeh, Dumitru Erhan, Chelsea Finn, Sergey Levine, Laurent Dinh, Durk Kingma
Date: 2019-03-04 00:00:00
Proceeding: ICLR 2020 1
Tasks: Predict Future Video Frames, Video Generation, Video Prediction

Generative models that can model and predict sequences of future events can, in principle, learn to capture complex real-world phenomena, such as physical interactions. However, a central challenge in video prediction is that the future is highly uncertain: a sequence of past observations of events can imply many possible futures. Although a number of recent works have studied probabilistic models that can represent uncertain futures, such models are either extremely expensive computationally as in the case of pixel-level autoregressive models, or do not directly optimize the likelihood of the data. To our knowledge, our work is the first to propose multi-frame video prediction with normalizing flows, which allows for direct optimization of the data likelihood, and produces high-quality stochastic predictions. We describe an approach for modeling the latent space dynamics, and demonstrate that flow-based generative models offer a viable and competitive approach to generative modelling of video.
Data Amplification: Instance-Optimal Property Estimation
Authors: Yi Hao, Alon Orlitsky
Date: 2019-03-04 00:00:00
Proceeding: ICML 2020 1

The best-known and most commonly used distribution-property estimation
technique uses a plug-in estimator, with empirical frequency replacing the
underlying distribution. We present novel linear-time-computable estimators
that significantly "amplify" the effective amount of data available. For a
large variety of distribution properties including four of the most popular
ones and for every underlying distribution, they achieve the accuracy that the
empirical-frequency plug-in estimators would attain using a logarithmic-factor
more samples.
  Specifically, for Shannon entropy and a very broad class of properties
including $\ell_1$-distance, the new estimators use $n$ samples to achieve the
accuracy attained by the empirical estimators with $n\log n$ samples. For
support-size and coverage, the new estimators use $n$ samples to achieve the
performance of empirical frequency with sample size $n$ times the logarithm of
the property value. Significantly strengthening the traditional min-max
formulation, these results hold not only for the worst distributions, but for
each and every underlying distribution. Furthermore, the logarithmic
amplification factors are optimal. Experiments on a wide variety of
distributions show that the new estimators outperform the previous
state-of-the-art estimators designed for each specific property.
Database Alignment with Gaussian Features
Authors: Osman Emre Dai, Daniel Cullina, Negar Kiyavash
Date: 2019-03-04 00:00:00

We consider the problem of aligning a pair of databases with jointly Gaussian features. We consider two algorithms, complete database alignment via MAP estimation among all possible database alignments, and partial alignment via a thresholding approach of log likelihood ratios. We derive conditions on mutual information between feature pairs, identifying the regimes where the algorithms are guaranteed to perform reliably and those where they cannot be expected to succeed.