pwc-archive/papers-with-abstracts huggingface dataset:
Keyframing the Future: Keyframe Discovery for Visual Prediction and Planning
Authors: Karl Pertsch, Oleh Rybkin, Jingyun Yang, Shenghao Zhou, Konstantinos G. Derpanis, Kostas Daniilidis, Joseph Lim, Andrew Jaegle
Date: 2019-04-11 00:00:00
Proceeding: L4DC 2020 6
Tasks: Video Prediction

Temporal observations such as videos contain essential information about the dynamics of the underlying scene, but they are often interleaved with inessential, predictable details. One way of dealing with this problem is by focusing on the most informative moments in a sequence. We propose a model that learns to discover these important events and the times when they occur and uses them to represent the full sequence. We do so using a hierarchical Keyframe-Inpainter (KeyIn) model that first generates a video's keyframes and then inpaints the rest by generating the frames at the intervening times. We propose a fully differentiable formulation to efficiently learn this procedure. We show that KeyIn finds informative keyframes in several datasets with different dynamics and visual properties. KeyIn outperforms other recent hierarchical predictive models for planning. For more details, please see the project website at \url{https://sites.google.com/view/keyin}.
Improved training of binary networks for human pose estimation and image recognition
Authors: Adrian Bulat, Georgios Tzimiropoulos, Jean Kossaifi, Maja Pantic
Date: 2019-04-11 00:00:00
Tasks: Binarization, Classification with Binary Neural Network, Knowledge Distillation, Object Recognition, Pose Estimation, Quantization

Big neural networks trained on large datasets have advanced the
state-of-the-art for a large variety of challenging problems, improving
performance by a large margin. However, under low memory and limited
computational power constraints, the accuracy on the same problems drops
considerable. In this paper, we propose a series of techniques that
significantly improve the accuracy of binarized neural networks (i.e networks
where both the features and the weights are binary). We evaluate the proposed
improvements on two diverse tasks: fine-grained recognition (human pose
estimation) and large-scale image recognition (ImageNet classification).
Specifically, we introduce a series of novel methodological changes including:
(a) more appropriate activation functions, (b) reverse-order initialization,
(c) progressive quantization, and (d) network stacking and show that these
additions improve existing state-of-the-art network binarization techniques,
significantly. Additionally, for the first time, we also investigate the extent
to which network binarization and knowledge distillation can be combined. When
tested on the challenging MPII dataset, our method shows a performance
improvement of more than 4% in absolute terms. Finally, we further validate our
findings by applying the proposed techniques for large-scale object recognition
on the Imagenet dataset, on which we report a reduction of error rate by 4%.
Expressive Body Capture: 3D Hands, Face, and Body from a Single Image
Authors: Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, Michael J. Black
Date: 2019-04-11 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Human Pose Estimation, 3D Human Reconstruction, 3D Multi-Person Mesh Recovery, 3D Reconstruction

To facilitate the analysis of human actions, interactions and emotions, we
compute a 3D model of human body pose, hand pose, and facial expression from a
single monocular image. To achieve this, we use thousands of 3D scans to train
a new, unified, 3D model of the human body, SMPL-X, that extends SMPL with
fully articulated hands and an expressive face. Learning to regress the
parameters of SMPL-X directly from images is challenging without paired images
and 3D ground truth. Consequently, we follow the approach of SMPLify, which
estimates 2D features and then optimizes model parameters to fit the features.
We improve on SMPLify in several significant ways: (1) we detect 2D features
corresponding to the face, hands, and feet and fit the full SMPL-X model to
these; (2) we train a new neural network pose prior using a large MoCap
dataset; (3) we define a new interpenetration penalty that is both fast and
accurate; (4) we automatically detect gender and the appropriate body models
(male, female, or neutral); (5) our PyTorch implementation achieves a speedup
of more than 8x over Chumpy. We use the new method, SMPLify-X, to fit SMPL-X to
both controlled images and images in the wild. We evaluate 3D accuracy on a new
curated dataset comprising 100 images with pseudo ground-truth. This is a step
towards automatic expressive human capture from monocular RGB data. The models,
code, and data are available for research purposes at
https://smpl-x.is.tue.mpg.de.
wav2vec: Unsupervised Pre-training for Speech Recognition
Authors: Steffen Schneider, Alexei Baevski, Ronan Collobert, Michael Auli
Date: 2019-04-11 00:00:00
Tasks: Binary Classification, General Classification, Speech Recognition, Unsupervised Pre-training

We explore unsupervised pre-training for speech recognition by learning representations of raw audio. wav2vec is trained on large amounts of unlabeled audio data and the resulting representations are then used to improve acoustic model training. We pre-train a simple multi-layer convolutional neural network optimized via a noise contrastive binary classification task. Our experiments on WSJ reduce WER of a strong character-based log-mel filterbank baseline by up to 36% when only a few hours of transcribed data is available. Our approach achieves 2.43% WER on the nov92 test set. This outperforms Deep Speech 2, the best reported character-based system in the literature while using two orders of magnitude less labeled training data.
Connections Between Adaptive Control and Optimization in Machine Learning
Authors: Joseph E. Gaudio, Travis E. Gibson, Anuradha M. Annaswamy, Michael A. Bolender, Eugene Lavretsky
Date: 2019-04-11 00:00:00
Tasks: BIG-bench Machine Learning

This paper demonstrates many immediate connections between adaptive control
and optimization methods commonly employed in machine learning. Starting from
common output error formulations, similarities in update law modifications are
examined. Concepts in stability, performance, and learning, common to both
fields are then discussed. Building on the similarities in update laws and
common concepts, new intersections and opportunities for improved algorithm
analysis are provided. In particular, a specific problem related to higher
order learning is solved through insights obtained from these intersections.
MAIN: Multi-Attention Instance Network for Video Segmentation
Authors: Juan Leon Alcazar, Maria A. Bravo, Ali K. Thabet, Guillaume Jeanneret, Thomas Brox, Pablo Arbelaez, Bernard Ghanem
Date: 2019-04-11 00:00:00
Tasks: One-shot visual object segmentation, Segmentation, Video Segmentation, Video Semantic Segmentation

Instance-level video segmentation requires a solid integration of spatial and
temporal information. However, current methods rely mostly on domain-specific
information (online learning) to produce accurate instance-level segmentations.
We propose a novel approach that relies exclusively on the integration of
generic spatio-temporal attention cues. Our strategy, named Multi-Attention
Instance Network (MAIN), overcomes challenging segmentation scenarios over
arbitrary videos without modelling sequence- or instance-specific knowledge. We
design MAIN to segment multiple instances in a single forward pass, and
optimize it with a novel loss function that favors class agnostic predictions
and assigns instance-specific penalties. We achieve state-of-the-art
performance on the challenging Youtube-VOS dataset and benchmark, improving the
unseen Jaccard and F-Metric by 6.8% and 12.7% respectively, while operating at
real-time (30.3 FPS).
Variational Information Distillation for Knowledge Transfer
Authors: Sungsoo Ahn, Shell Xu Hu, Andreas Damianou, Neil D. Lawrence, Zhenwen Dai
Date: 2019-04-11 00:00:00
Proceeding: CVPR 2019 6
Tasks: Knowledge Distillation, Transfer Learning

Transferring knowledge from a teacher neural network pretrained on the same
or a similar task to a student neural network can significantly improve the
performance of the student neural network. Existing knowledge transfer
approaches match the activations or the corresponding hand-crafted features of
the teacher and the student networks. We propose an information-theoretic
framework for knowledge transfer which formulates knowledge transfer as
maximizing the mutual information between the teacher and the student networks.
We compare our method with existing knowledge transfer methods on both
knowledge distillation and transfer learning tasks and show that our method
consistently outperforms existing methods. We further demonstrate the strength
of our method on knowledge transfer across heterogeneous network architectures
by transferring knowledge from a convolutional neural network (CNN) to a
multi-layer perceptron (MLP) on CIFAR-10. The resulting MLP significantly
outperforms the-state-of-the-art methods and it achieves similar performance to
the CNN with a single convolutional layer.
Adapting RNN Sequence Prediction Model to Multi-label Set Prediction
Authors: Kechen Qin, Cheng Li, Virgil Pavlu, Javed A. Aslam
Date: 2019-04-11 00:00:00
Proceeding: NAACL 2019 6
Tasks: Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Prediction

We present an adaptation of RNN sequence models to the problem of multi-label
classification for text, where the target is a set of labels, not a sequence.
Previous such RNN models define probabilities for sequences but not for sets;
attempts to obtain a set probability are after-thoughts of the network design,
including pre-specifying the label order, or relating the sequence probability
to the set probability in ad hoc ways.
  Our formulation is derived from a principled notion of set probability, as
the sum of probabilities of corresponding permutation sequences for the set. We
provide a new training objective that maximizes this set probability, and a new
prediction objective that finds the most probable set on a test document. These
new objectives are theoretically appealing because they give the RNN model
freedom to discover the best label order, which often is the natural one (but
different among documents).
  We develop efficient procedures to tackle the computation difficulties
involved in training and prediction. Experiments on benchmark datasets
demonstrate that we outperform state-of-the-art methods for this task.
Learning Single Camera Depth Estimation using Dual-Pixels
Authors: Rahul Garg, Neal Wadhwa, Sameer Ansari, Jonathan T. Barron
Date: 2019-04-11 00:00:00
Proceeding: ICCV 2019 10
Tasks: Depth Estimation, Monocular Depth Estimation

Deep learning techniques have enabled rapid progress in monocular depth estimation, but their quality is limited by the ill-posed nature of the problem and the scarcity of high quality datasets. We estimate depth from a single camera by leveraging the dual-pixel auto-focus hardware that is increasingly common on modern camera sensors. Classic stereo algorithms and prior learning-based depth estimation techniques under-perform when applied on this dual-pixel data, the former due to too-strong assumptions about RGB image matching, and the latter due to not leveraging the understanding of optics of dual-pixel image formation. To allow learning based methods to work well on dual-pixel imagery, we identify an inherent ambiguity in the depth estimated from dual-pixel cues, and develop an approach to estimate depth up to this ambiguity. Using our approach, existing monocular depth estimation techniques can be effectively applied to dual-pixel data, and much smaller models can be constructed that still infer high quality depth. To demonstrate this, we capture a large dataset of in-the-wild 5-viewpoint RGB images paired with corresponding dual-pixel data, and show how view supervision with this data can be used to learn depth up to the unknown ambiguities. On our new task, our model is 30% more accurate than any prior work on learning-based monocular or stereoscopic depth estimation.
GP-HD: Using Genetic Programming to Generate Dynamical Systems Models for Health Care
Authors: Mark Hoogendoorn, Ward van Breda, Jeroen Ruwaard
Date: 2019-04-11 00:00:00

The huge wealth of data in the health domain can be exploited to create
models that predict development of health states over time. Temporal learning
algorithms are well suited to learn relationships between health states and
make predictions about their future developments. However, these algorithms:
(1) either focus on learning one generic model for all patients, providing
general insights but often with limited predictive performance, or (2) learn
individualized models from which it is hard to derive generic concepts. In this
paper, we present a middle ground, namely parameterized dynamical systems
models that are generated from data using a Genetic Programming (GP) framework.
A fitness function suitable for the health domain is exploited. An evaluation
of the approach in the mental health domain shows that performance of the model
generated by the GP is on par with a dynamical systems model developed based on
domain knowledge, significantly outperforms a generic Long Term Short Term
Memory (LSTM) model and in some cases also outperforms an individualized LSTM
model.
Probabilistic Permutation Synchronization using the Riemannian Structure of the Birkhoff Polytope
Authors: Tolga Birdal, Umut Şimşekli
Date: 2019-04-11 00:00:00
Proceeding: CVPR 2019 6
Tasks: Graph Matching

We present an entirely new geometric and probabilistic approach to
synchronization of correspondences across multiple sets of objects or images.
In particular, we present two algorithms: (1) Birkhoff-Riemannian L-BFGS for
optimizing the relaxed version of the combinatorially intractable cycle
consistency loss in a principled manner, (2) Birkhoff-Riemannian Langevin Monte
Carlo for generating samples on the Birkhoff Polytope and estimating the
confidence of the found solutions. To this end, we first introduce the very
recently developed Riemannian geometry of the Birkhoff Polytope. Next, we
introduce a new probabilistic synchronization model in the form of a Markov
Random Field (MRF). Finally, based on the first order retraction operators, we
formulate our problem as simulating a stochastic differential equation and
devise new integrators. We show on both synthetic and real datasets that we
achieve high quality multi-graph matching results with faster convergence and
reliable confidence/uncertainty estimates.
Relational Graph Attention Networks
Authors: Dan Busbridge, Dane Sherburn, Pietro Cavallo, Nils Y. Hammerla
Date: 2019-04-11 00:00:00
Proceeding: ICLR 2019 5
Tasks: Graph Attention

We investigate Relational Graph Attention Networks, a class of models that
extends non-relational graph attention mechanisms to incorporate relational
information, opening up these methods to a wider variety of problems. A
thorough evaluation of these models is performed, and comparisons are made
against established benchmarks. To provide a meaningful comparison, we retrain
Relational Graph Convolutional Networks, the spectral counterpart of Relational
Graph Attention Networks, and evaluate them under the same conditions. We find
that Relational Graph Attention Networks perform worse than anticipated,
although some configurations are marginally beneficial for modelling molecular
properties. We provide insights as to why this may be, and suggest both
modifications to evaluation strategies, as well as directions to investigate
for future work.
Difficulty-aware Image Super Resolution via Deep Adaptive Dual-Network
Authors: Jinghui Qin, Ziwei Xie, Yukai Shi, Wushao Wen
Date: 2019-04-11 00:00:00
Tasks: Diversity, Image Super-Resolution, Super-Resolution

Recently, deep learning based single image super-resolution(SR) approaches
have achieved great development. The state-of-the-art SR methods usually adopt
a feed-forward pipeline to establish a non-linear mapping between low-res(LR)
and high-res(HR) images. However, due to treating all image regions equally
without considering the difficulty diversity, these approaches meet an upper
bound for optimization. To address this issue, we propose a novel SR approach
that discriminately processes each image region within an image by its
difficulty. Specifically, we propose a dual-way SR network that one way is
trained to focus on easy image regions and another is trained to handle hard
image regions. To identify whether a region is easy or hard, we propose a novel
image difficulty recognition network based on PSNR prior. Our SR approach that
uses the region mask to adaptively enforce the dual-way SR network yields
superior results. Extensive experiments on several standard benchmarks (e.g.,
Set5, Set14, BSD100, and Urban100) show that our approach achieves
state-of-the-art performance.
Bridging Theory and Algorithm for Domain Adaptation
Authors: Yuchen Zhang, Tianle Liu, Mingsheng Long, Michael. I. Jordan
Date: 2019-04-11 00:00:00
Tasks: Domain Adaptation, Generalization Bounds

This paper addresses the problem of unsupervised domain adaption from theoretical and algorithmic perspectives. Existing domain adaptation theories naturally imply minimax optimization algorithms, which connect well with the domain adaptation methods based on adversarial learning. However, several disconnections still exist and form the gap between theory and algorithm. We extend previous theories (Mansour et al., 2009c; Ben-David et al., 2010) to multiclass classification in domain adaptation, where classifiers based on the scoring functions and margin loss are standard choices in algorithm design. We introduce Margin Disparity Discrepancy, a novel measurement with rigorous generalization bounds, tailored to the distribution comparison with the asymmetric margin loss, and to the minimax optimization for easier training. Our theory can be seamlessly transformed into an adversarial learning algorithm for domain adaptation, successfully bridging the gap between theory and algorithm. A series of empirical studies show that our algorithm achieves the state of the art accuracies on challenging domain adaptation tasks.
Learning joint reconstruction of hands and manipulated objects
Authors: Yana Hasson, Gül Varol, Dimitrios Tzionas, Igor Kalevatykh, Michael J. Black, Ivan Laptev, Cordelia Schmid
Date: 2019-04-11 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Hand Pose Estimation, Hand Joint Reconstruction, hand-object pose, Object, valid

Estimating hand-object manipulations is essential for interpreting and
imitating human actions. Previous work has made significant progress towards
reconstruction of hand poses and object shapes in isolation. Yet,
reconstructing hands and objects during manipulation is a more challenging task
due to significant occlusions of both the hand and object. While presenting
challenges, manipulations may also simplify the problem since the physics of
contact restricts the space of valid hand-object configurations. For example,
during manipulation, the hand and object should be in contact but not
interpenetrate. In this work, we regularize the joint reconstruction of hands
and objects with manipulation constraints. We present an end-to-end learnable
model that exploits a novel contact loss that favors physically plausible
hand-object constellations. Our approach improves grasp quality metrics over
baselines, using RGB images as input. To train and evaluate the model, we also
propose a new large-scale synthetic dataset, ObMan, with hand-object
manipulations. We demonstrate the transferability of ObMan-trained models to
real data.
Scalarizing Functions in Bayesian Multiobjective Optimization
Authors: Tinkle Chugh
Date: 2019-04-11 00:00:00
Tasks: Multiobjective Optimization

Scalarizing functions have been widely used to convert a multiobjective
optimization problem into a single objective optimization problem. However,
their use in solving (computationally) expensive multi- and many-objective
optimization problems in Bayesian multiobjective optimization is scarce.
Scalarizing functions can play a crucial role on the quality and number of
evaluations required when doing the optimization. In this article, we study and
review 15 different scalarizing functions in the framework of Bayesian
multiobjective optimization and build Gaussian process models (as surrogates,
metamodels or emulators) on them. We use expected improvement as infill
criterion (or acquisition function) to update the models. In particular, we
compare different scalarizing functions and analyze their performance on
several benchmark problems with different number of objectives to be optimized.
The review and experiments on different functions provide useful insights when
using and selecting a scalarizing function when using a Bayesian multiobjective
optimization method.
Investigating Retrieval Method Selection with Axiomatic Features
Authors: Siddhant Arora, Andrew Yates
Date: 2019-04-11 00:00:00
Tasks: Ad-Hoc Information Retrieval, Information Retrieval, Retrieval

We consider algorithm selection in the context of ad-hoc information
retrieval. Given a query and a pair of retrieval methods, we propose a
meta-learner that predicts how to combine the methods' relevance scores into an
overall relevance score. Inspired by neural models' different properties with
regard to IR axioms, these predictions are based on features that quantify
axiom-related properties of the query and its top ranked documents. We conduct
an evaluation on TREC Web Track data and find that the meta-learner often
significantly improves over the individual methods. Finally, we conduct feature
and query weight analyses to investigate the meta-learner's behavior.
A Kaczmarz Algorithm for Solving Tree Based Distributed Systems of Equations
Authors: Chinmay Hegde, Fritz Keinert, Eric S. Weber
Date: 2019-04-11 00:00:00

The Kaczmarz algorithm is an iterative method for solving systems of linear
equations. We introduce a modified Kaczmarz algorithm for solving systems of
linear equations in a distributed environment, i.e. the equations within the
system are distributed over multiple nodes within a network. The modification
we introduce is designed for a network with a tree structure that allows for
passage of solution estimates between the nodes in the network. We prove that
the modified algorithm converges under no additional assumptions on the
equations. We demonstrate that the algorithm converges to the solution, or the
solution of minimal norm, when the system is consistent. We also demonstrate
that in the case of an inconsistent system of equations, the modified relaxed
Kaczmarz algorithm converges to a weighted least squares solution as the
relaxation parameter approaches $0$.