pwc-archive/papers-with-abstracts huggingface dataset:
Learning Transformation Synchronization
Authors: Xiangru Huang, Zhenxiao Liang, Xiaowei Zhou, Yao Xie, Leonidas Guibas, Qi-Xing Huang
Date: 2019-01-27 00:00:00
Proceeding: CVPR 2019 6

Reconstructing the 3D model of a physical object typically requires us to align the depth scans obtained from different camera poses into the same coordinate system. Solutions to this global alignment problem usually proceed in two steps. The first step estimates relative transformations between pairs of scans using an off-the-shelf technique. Due to limited information presented between pairs of scans, the resulting relative transformations are generally noisy. The second step then jointly optimizes the relative transformations among all input depth scans. A natural constraint used in this step is the cycle-consistency constraint, which allows us to prune incorrect relative transformations by detecting inconsistent cycles. The performance of such approaches, however, heavily relies on the quality of the input relative transformations. Instead of merely using the relative transformations as the input to perform transformation synchronization, we propose to use a neural network to learn the weights associated with each relative transformation. Our approach alternates between transformation synchronization using weighted relative transformations and predicting new weights of the input relative transformations using a neural network. We demonstrate the usefulness of this approach across a wide range of datasets.
Off-Policy Deep Reinforcement Learning by Bootstrapping the Covariate Shift
Authors: Carles Gelada, Marc G. Bellemare
Date: 2019-01-27 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper we revisit the method of off-policy corrections for
reinforcement learning (COP-TD) pioneered by Hallak et al. (2017). Under this
method, online updates to the value function are reweighted to avoid divergence
issues typical of off-policy learning. While Hallak et al.'s solution is
appealing, it cannot easily be transferred to nonlinear function approximation.
First, it requires a projection step onto the probability simplex; second, even
though the operator describing the expected behavior of the off-policy learning
algorithm is convergent, it is not known to be a contraction mapping, and
hence, may be more unstable in practice. We address these two issues by
introducing a discount factor into COP-TD. We analyze the behavior of
discounted COP-TD and find it better behaved from a theoretical perspective. We
also propose an alternative soft normalization penalty that can be minimized
online and obviates the need for an explicit projection step. We complement our
analysis with an empirical evaluation of the two techniques in an off-policy
setting on the game Pong from the Atari domain where we find discounted COP-TD
to be better behaved in practice than the soft normalization penalty. Finally,
we perform a more extensive evaluation of discounted COP-TD in 5 games of the
Atari domain, where we find performance gains for our approach.
On Learning Invariant Representation for Domain Adaptation
Authors: Han Zhao, Remi Tachet des Combes, Kun Zhang, Geoffrey J. Gordon
Date: 2019-01-27 00:00:00
Tasks: Domain Adaptation, Representation Learning, Unsupervised Domain Adaptation

Due to the ability of deep neural nets to learn rich representations, recent advances in unsupervised domain adaptation have focused on learning domain-invariant features that achieve a small error on the source domain. The hope is that the learnt representation, together with the hypothesis learnt from the source domain, can generalize to the target domain. In this paper, we first construct a simple counterexample showing that, contrary to common belief, the above conditions are not sufficient to guarantee successful domain adaptation. In particular, the counterexample exhibits \emph{conditional shift}: the class-conditional distributions of input features change between source and target domains. To give a sufficient condition for domain adaptation, we propose a natural and interpretable generalization upper bound that explicitly takes into account the aforementioned shift. Moreover, we shed new light on the problem by proving an information-theoretic lower bound on the joint error of \emph{any} domain adaptation method that attempts to learn invariant representations. Our result characterizes a fundamental tradeoff between learning invariant representations and achieving small joint error on both domains when the marginal label distributions differ from source to target. Finally, we conduct experiments on real-world datasets that corroborate our theoretical findings. We believe these insights are helpful in guiding the future design of domain adaptation and representation learning algorithms.
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
Authors: Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Adam Tauman Kalai
Date: 2019-01-27 00:00:00
Tasks: Classification, General Classification

We present a large-scale study of gender bias in occupation classification, a
task where the use of machine learning may lead to negative outcomes on
peoples' lives. We analyze the potential allocation harms that can result from
semantic representation bias. To do so, we study the impact on occupation
classification of including explicit gender indicators---such as first names
and pronouns---in different semantic representations of online biographies.
Additionally, we quantify the bias that remains when these indicators are
"scrubbed," and describe proxy behavior that occurs in the absence of explicit
gender indicators. As we demonstrate, differences in true positive rates
between genders are correlated with existing gender imbalances in occupations,
which may compound these imbalances.
ADMM-SOFTMAX : An ADMM Approach for Multinomial Logistic Regression
Authors: Samy Wu Fung, Sanna Tyrväinen, Lars Ruthotto, Eldad Haber
Date: 2019-01-27 00:00:00
Tasks: General Classification, image-classification, Image Classification, regression, Transfer Learning

We present ADMM-Softmax, an alternating direction method of multipliers (ADMM) for solving multinomial logistic regression (MLR) problems. Our method is geared toward supervised classification tasks with many examples and features. It decouples the nonlinear optimization problem in MLR into three steps that can be solved efficiently. In particular, each iteration of ADMM-Softmax consists of a linear least-squares problem, a set of independent small-scale smooth, convex problems, and a trivial dual variable update. Solution of the least-squares problem can be be accelerated by pre-computing a factorization or preconditioner, and the separability in the smooth, convex problem can be easily parallelized across examples. For two image classification problems, we demonstrate that ADMM-Softmax leads to improved generalization compared to a Newton-Krylov, a quasi Newton, and a stochastic gradient descent method.
Open Source Face Recognition Performance Evaluation Package
Authors: Xiang Xu, Ioannis A. Kakadiaris
Date: 2019-01-27 00:00:00
Tasks: Deep Learning, Face Recognition

Biometrics-related research has been accelerated significantly by deep
learning technology. However, there are limited open-source resources to help
researchers evaluate their deep learning-based biometrics algorithms
efficiently, especially for the face recognition tasks. In this work, we design
and implement a light-weight, maintainable, scalable, generalizable, and
extendable face recognition evaluation toolbox named FaRE that supports both
online and offline evaluation to provide feedback to algorithm development and
accelerate biometrics-related research. FaRE consists of a set of evaluation
metric functions and provides various APIs for commonly-used face recognition
datasets including LFW, CFP, UHDB31, and IJB-series datasets, which can be
easily extended to include other customized datasets. The package and the
pre-trained baseline models will be released for public academic research use
after obtaining university approval.
Promoting Diversity for End-to-End Conversation Response Generation
Authors: Yu-Ping Ruan, Zhen-Hua Ling, Quan Liu, Jia-Chen Gu, Xiaodan Zhu
Date: 2019-01-27 00:00:00
Tasks: Diversity, Response Generation, Retrieval

We present our work on Track 2 in the Dialog System Technology Challenges 7
(DSTC7). The DSTC7-Track 2 aims to evaluate the response generation of fully
data-driven conversation models in knowledge-grounded settings, which provides
the contextual-relevant factual texts. The Sequenceto-Sequence models have been
widely used for end-to-end generative conversation modelling and achieved
impressive results. However, they tend to output dull and repeated responses in
previous studies. Our work aims to promote the diversity for end-to-end
conversation response generation, which follows a two-stage pipeline: 1)
Generate multiple responses. At this stage, two different models are proposed,
i.e., a variational generative (VariGen) model and a retrieval based
(Retrieval) model. 2) Rank and return the most related response by training a
topic coherence discrimination (TCD) model for the ranking process. According
to the official evaluation results, our proposed Retrieval and VariGen systems
ranked first and second respectively on objective diversity metrics, i.e.,
Entropy, among all participant systems. And the VariGen system ranked second on
NIST and METEOR metrics.
99% of Distributed Optimization is a Waste of Time: The Issue and How to Fix it
Authors: Konstantin Mishchenko, Filip Hanzely, Peter Richtárik
Date: 2019-01-27 00:00:00
Tasks: Distributed Optimization

Many popular distributed optimization methods for training machine learning models fit the following template: a local gradient estimate is computed independently by each worker, then communicated to a master, which subsequently performs averaging. The average is broadcast back to the workers, which use it to perform a gradient-type step to update the local version of the model. It is also well known that many such methods, including SGD, SAGA, and accelerated SGD for over-parameterized models, do not scale well with the number of parallel workers. In this paper we observe that the above template is fundamentally inefficient in that too much data is unnecessarily communicated by the workers, which slows down the overall system. We propose a fix based on a new update-sparsification method we develop in this work, which we suggest be used on top of existing methods. Namely, we develop a new variant of parallel block coordinate descent based on independent sparsification of the local gradient estimates before communication. We demonstrate that with only $m/n$ blocks sent by each of $n$ workers, where $m$ is the total number of parameter blocks, the theoretical iteration complexity of the underlying distributed methods is essentially unaffected. As an illustration, this means that when $n=100$ parallel workers are used, the communication of $99\%$ blocks is redundant, and hence a waste of time. Our theoretical claims are supported through extensive numerical experiments which demonstrate an almost perfect match with our theory on a number of synthetic and real datasets.
Degraded Historical Documents Images Binarization Using a Combination of Enhanced Techniques
Authors: Omar Boudraa, Walid Khaled Hidouci, Dominique Michelucci
Date: 2019-01-27 00:00:00
Tasks: Binarization

Document image binarization is the initial step and a crucial in many
document analysis and recognition scheme. In fact, it is still a relevant
research subject and a fundamental challenge due to its importance and
influence. This paper provides an original multi-phases system that hybridizes
various efficient image thresholding methods in order to get the best
binarization output. First, to improve contrast in particularly defective
images, the application of CLAHE algorithm is suggested and justified. We then
use a cooperative technique to segment image into two separated classes. At the
end, a special transformation is applied for the purpose of removing scattered
noise and of correcting characters forms. Experimentations demonstrate the
precision and the robustness of our framework applied on historical degraded
documents images within three benchmarks compared to other noted methods.
Information-Theoretic Understanding of Population Risk Improvement with Model Compression
Authors: Yuheng Bu, Weihao Gao, Shaofeng Zou, Venugopal V. Veeravalli
Date: 2019-01-27 00:00:00
Tasks: Clustering, Model Compression

We show that model compression can improve the population risk of a
pre-trained model, by studying the tradeoff between the decrease in the
generalization error and the increase in the empirical risk with model
compression. We first prove that model compression reduces an
information-theoretic bound on the generalization error; this allows for an
interpretation of model compression as a regularization technique to avoid
overfitting. We then characterize the increase in empirical risk with model
compression using rate distortion theory. These results imply that the
population risk could be improved by model compression if the decrease in
generalization error exceeds the increase in empirical risk. We show through a
linear regression example that such a decrease in population risk due to model
compression is indeed possible. Our theoretical results further suggest that
the Hessian-weighted $K$-means clustering compression approach can be improved
by regularizing the distance between the clustering centers. We provide
experiments with neural networks to support our theoretical assertions.
Disentangling and Learning Robust Representations with Natural Clustering
Authors: Javier Antoran, Antonio Miguel
Date: 2019-01-27 00:00:00
Tasks: Clustering

Learning representations that disentangle the underlying factors of variability in data is an intuitive way to achieve generalization in deep models. In this work, we address the scenario where generative factors present a multimodal distribution due to the existence of class distinction in the data. We propose N-VAE, a model which is capable of separating factors of variation which are exclusive to certain classes from factors that are shared among classes. This model implements an explicitly compositional latent variable structure by defining a class-conditioned latent space and a shared latent space. We show its usefulness for detecting and disentangling class-dependent generative factors as well as its capacity to generate artificial samples which contain characteristics unseen in the training data.
An Information-Theoretic Explanation for the Adversarial Fragility of AI Classifiers
Authors: Hui Xie, Jirong Yi, Weiyu Xu, Raghu Mudumbai
Date: 2019-01-27 00:00:00

We present a simple hypothesis about a compression property of artificial
intelligence (AI) classifiers and present theoretical arguments to show that
this hypothesis successfully accounts for the observed fragility of AI
classifiers to small adversarial perturbations. We also propose a new method
for detecting when small input perturbations cause classifier errors, and show
theoretical guarantees for the performance of this detection method. We present
experimental results with a voice recognition system to demonstrate this
method. The ideas in this paper are motivated by a simple analogy between AI
classifiers and the standard Shannon model of a communication system.
3D Contouring for Breast Tumor in Sonography
Authors: Dar-Ren Chen, Yu-Chih Lin, Yu-Len Huang
Date: 2019-01-27 00:00:00

Malignant and benign breast tumors present differently in their shape and size on sonography. Morphological information provided by tumor contours are important in clinical diagnosis. However, ultrasound images contain noises and tissue texture; clinical diagnosis thus highly depends on the experience of physicians. The manual way to sketch three-dimensional (3D) contours of breast tumor is a time-consuming and complicate task. If automatic contouring could provide a precise breast tumor contour that might assist physicians in making an accurate diagnosis. This study presents an efficient method for automatically contouring breast tumors in 3D sonography. The proposed method utilizes an efficient segmentation procedure, i.e. level-set method (LSM), to automatic detect contours of breast tumors. This study evaluates 20 cases comprising ten benign and ten malignant tumors. The results of computer simulation reveal that the proposed 3D segmentation method provides robust contouring for breast tumor on ultrasound images. This approach consistently obtains contours similar to those obtained by manual contouring of the breast tumor and can save much of the time required to sketch precise contours.
Spatio-temporal Action Recognition: A Survey
Authors: Amlaan Bhoi
Date: 2019-01-27 00:00:00
Tasks: Action Detection, Action Localization, Action Recognition, Spatio-temporal Action Recognition, Survey, Temporal Action Localization, Video Editing

The task of action recognition or action detection involves analyzing videos
and determining what action or motion is being performed. The primary subject
of these videos are predominantly humans performing some action. However, this
requirement can be relaxed to generalize over other subjects such as animals or
robots. The applications can range from anywhere between human-computer
inter-action to automated video editing proposals. When we consider
spatiotemporal action recognition, we deal with action localization. This task
not only involves determining what action is being performed but also when and
where itis being performed in said video. This paper aims to survey the
plethora of approaches and algorithms attempted to solve this task, give a
comprehensive comparison between them, explore various datasets available for
the problem, and determine the most promising approaches.
Monocular Depth Estimation: A Survey
Authors: Amlaan Bhoi
Date: 2019-01-27 00:00:00
Tasks: Depth Estimation, Monocular Depth Estimation, Survey

Monocular depth estimation is often described as an ill-posed and inherently
ambiguous problem. Estimating depth from 2D images is a crucial step in scene
reconstruction, 3Dobject recognition, segmentation, and detection. The problem
can be framed as: given a single RGB image as input, predict a dense depth map
for each pixel. This problem is worsened by the fact that most scenes have
large texture and structural variations, object occlusions, and rich geometric
detailing. All these factors contribute to difficulty in accurate depth
estimation. In this paper, we review five papers that attempt to solve the
depth estimation problem with various techniques including supervised,
weakly-supervised, and unsupervised learning techniques. We then compare these
papers and understand the improvements made over one another. Finally, we
explore potential improvements that can aid to better solve this problem.
SGD: General Analysis and Improved Rates
Authors: Robert Mansel Gower, Nicolas Loizou, Xun Qian, Alibek Sailanbayev, Egor Shulgin, Peter Richtarik
Date: 2019-01-27 00:00:00

We propose a general yet simple theorem describing the convergence of SGD
under the arbitrary sampling paradigm. Our theorem describes the convergence of
an infinite array of variants of SGD, each of which is associated with a
specific probability law governing the data selection rule used to form
mini-batches. This is the first time such an analysis is performed, and most of
our variants of SGD were never explicitly considered in the literature before.
Our analysis relies on the recently introduced notion of expected smoothness
and does not rely on a uniform bound on the variance of the stochastic
gradients. By specializing our theorem to different mini-batching strategies,
such as sampling with replacement and independent sampling, we derive exact
expressions for the stepsize as a function of the mini-batch size. With this we
can also determine the mini-batch size that optimizes the total complexity, and
show explicitly that as the variance of the stochastic gradient evaluated at
the minimum grows, so does the optimal mini-batch size. For zero variance, the
optimal mini-batch size is one. Moreover, we prove insightful
stepsize-switching rules which describe when one should switch from a constant
to a decreasing stepsize regime.
NeuralSampler: Euclidean Point Cloud Auto-Encoder and Sampler
Authors: Edoardo Remelli, Pierre Baque, Pascal Fua
Date: 2019-01-27 00:00:00

Most algorithms that rely on deep learning-based approaches to generate 3D
point sets can only produce clouds containing fixed number of points.
Furthermore, they typically require large networks parameterized by many
weights, which makes them hard to train. In this paper, we propose an
auto-encoder architecture that can both encode and decode clouds of arbitrary
size and demonstrate its effectiveness at upsampling sparse point clouds.
Interestingly, we can do so using less than half as many parameters as
state-of-the-art architectures while still delivering better performance. We
will make our code base fully available.
On the (In)fidelity and Sensitivity for Explanations
Authors: Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Sai Suggala, David I. Inouye, Pradeep Ravikumar
Date: 2019-01-27 00:00:00
Tasks: Sensitivity

We consider objective evaluation measures of saliency explanations for complex black-box machine learning models. We propose simple robust variants of two notions that have been considered in recent literature: (in)fidelity, and sensitivity. We analyze optimal explanations with respect to both these measures, and while the optimal explanation for sensitivity is a vacuous constant explanation, the optimal explanation for infidelity is a novel combination of two popular explanation methods. By varying the perturbation distribution that defines infidelity, we obtain novel explanations by optimizing infidelity, which we show to out-perform existing explanations in both quantitative and qualitative measurements. Another salient question given these measures is how to modify any given explanation to have better values with respect to these measures. We propose a simple modification based on lowering sensitivity, and moreover show that when done appropriately, we could simultaneously improve both sensitivity as well as fidelity.
Imitation Learning from Imperfect Demonstration
Authors: Yueh-Hua Wu, Nontawat Charoenphakdee, Han Bao, Voot Tangkaratt, Masashi Sugiyama
Date: 2019-01-27 00:00:00
Tasks: Imitation Learning

Imitation learning (IL) aims to learn an optimal policy from demonstrations.
However, such demonstrations are often imperfect since collecting optimal ones
is costly. To effectively learn from imperfect demonstrations, we propose a
novel approach that utilizes confidence scores, which describe the quality of
demonstrations. More specifically, we propose two confidence-based IL methods,
namely two-step importance weighting IL (2IWIL) and generative adversarial IL
with imperfect demonstration and confidence (IC-GAIL). We show that confidence
scores given only to a small portion of sub-optimal demonstrations
significantly improve the performance of IL both theoretically and empirically.