pwc-archive/papers-with-abstracts huggingface dataset:
Approximate Leave-One-Out for High-Dimensional Non-Differentiable Learning Problems
Authors: Shuaiwen Wang, Wenda Zhou, Arian Maleki, Haihao Lu, Vahab Mirrokni
Date: 2018-10-04 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Consider the following class of learning schemes: \begin{equation}
\label{eq:main-problem1}
  \hat{\boldsymbol{\beta}} := \underset{\boldsymbol{\beta} \in
\mathcal{C}}{\arg\min} \;\sum_{j=1}^n
\ell(\boldsymbol{x}_j^\top\boldsymbol{\beta}; y_j) + \lambda
R(\boldsymbol{\beta}), \qquad \qquad \qquad (1) \end{equation} where
$\boldsymbol{x}_i \in \mathbb{R}^p$ and $y_i \in \mathbb{R}$ denote the $i^{\rm
th}$ feature and response variable respectively. Let $\ell$ and $R$ be the
convex loss function and regularizer, $\boldsymbol{\beta}$ denote the unknown
weights, and $\lambda$ be a regularization parameter. $\mathcal{C} \subset
\mathbb{R}^{p}$ is a closed convex set. Finding the optimal choice of $\lambda$
is a challenging problem in high-dimensional regimes where both $n$ and $p$ are
large. We propose three frameworks to obtain a computationally efficient
approximation of the leave-one-out cross validation (LOOCV) risk for nonsmooth
losses and regularizers. Our three frameworks are based on the primal, dual,
and proximal formulations of (1). Each framework shows its strength in certain
types of problems. We prove the equivalence of the three approaches under
smoothness conditions. This equivalence enables us to justify the accuracy of
the three methods under such conditions. We use our approaches to obtain a risk
estimate for several standard problems, including generalized LASSO, nuclear
norm regularization, and support vector machines. We empirically demonstrate
the effectiveness of our results for non-differentiable cases.
Recurrent Transition Networks for Character Locomotion
Authors: Félix G. Harvey, Christopher Pal
Date: 2018-10-04 00:00:00
Tasks: Super-Resolution

Manually authoring transition animations for a complete locomotion system can be a tedious and time-consuming task, especially for large games that allow complex and constrained locomotion movements, where the number of transitions grows exponentially with the number of states. In this paper, we present a novel approach, based on deep recurrent neural networks, to automatically generate such transitions given a past context of a few frames and a target character state to reach. We present the Recurrent Transition Network (RTN), based on a modified version of the Long-Short-Term-Memory (LSTM) network, designed specifically for transition generation and trained without any gait, phase, contact or action labels. We further propose a simple yet principled way to initialize the hidden states of the LSTM layer for a given sequence which improves the performance and generalization to new motions. We both quantitatively and qualitatively evaluate our system and show that making the network terrain-aware by adding a local terrain representation to the input yields better performance for rough-terrain navigation on long transitions. Our system produces realistic and fluid transitions that rival the quality of Motion Capture-based ground-truth motions, even before applying any inverse-kinematics postprocess. Direct benefits of our approach could be to accelerate the creation of transition variations for large coverage, or even to entirely replace transition nodes in an animation graph. We further explore applications of this model in a animation super-resolution setting where we temporally decompress animations saved at 1 frame per second and show that the network is able to reconstruct motions that are hard to distinguish from un-compressed locomotion sequences.
Convex Clustering: Model, Theoretical Guarantee and Efficient Algorithm
Authors: Defeng Sun, Kim-Chuan Toh, Yancheng Yuan
Date: 2018-10-04 00:00:00
Tasks: Clustering

Clustering is a fundamental problem in unsupervised learning. Popular methods
like K-means, may suffer from poor performance as they are prone to get stuck
in its local minima. Recently, the sum-of-norms (SON) model (also known as the
clustering path) has been proposed in Pelckmans et al. (2005), Lindsten et al.
(2011) and Hocking et al. (2011). The perfect recovery properties of the convex
clustering model with uniformly weighted all pairwise-differences
regularization have been proved by Zhu et al. (2014) and Panahi et al. (2017).
However, no theoretical guarantee has been established for the general weighted
convex clustering model, where better empirical results have been observed. In
the numerical optimization aspect, although algorithms like the alternating
direction method of multipliers (ADMM) and the alternating minimization
algorithm (AMA) have been proposed to solve the convex clustering model (Chi
and Lange, 2015), it still remains very challenging to solve large-scale
problems. In this paper, we establish sufficient conditions for the perfect
recovery guarantee of the general weighted convex clustering model, which
include and improve existing theoretical results as special cases. In addition,
we develop a semismooth Newton based augmented Lagrangian method for solving
large-scale convex clustering problems. Extensive numerical experiments on both
simulated and real data demonstrate that our algorithm is highly efficient and
robust for solving large-scale problems. Moreover, the numerical results also
show the superior performance and scalability of our algorithm comparing to the
existing first-order methods. In particular, our algorithm is able to solve a
convex clustering problem with 200,000 points in $\mathbb{R}^3$ in about 6
minutes.
Learning Depth with Convolutional Spatial Propagation Network
Authors: Xinjing Cheng, Peng Wang, Ruigang Yang
Date: 2018-10-04 00:00:00
Tasks: Depth Completion, Depth Estimation, Depth Prediction, Stereo Matching, Stereo Matching Hand

Depth prediction is one of the fundamental problems in computer vision. In this paper, we propose a simple yet effective convolutional spatial propagation network (CSPN) to learn the affinity matrix for various depth estimation tasks. Specifically, it is an efficient linear propagation model, in which the propagation is performed with a manner of recurrent convolutional operation, and the affinity among neighboring pixels is learned through a deep convolutional neural network (CNN). We can append this module to any output from a state-of-the-art (SOTA) depth estimation networks to improve their performances. In practice, we further extend CSPN in two aspects: 1) take sparse depth map as additional input, which is useful for the task of depth completion; 2) similar to commonly used 3D convolution operation in CNNs, we propose 3D CSPN to handle features with one additional dimension, which is effective in the task of stereo matching using 3D cost volume. For the tasks of sparse to dense, a.k.a depth completion. We experimented the proposed CPSN conjunct algorithms over the popular NYU v2 and KITTI datasets, where we show that our proposed algorithms not only produce high quality (e.g., 30% more reduction in depth error), but also run faster (e.g., 2 to 5x faster) than previous SOTA spatial propagation network. We also evaluated our stereo matching algorithm on the Scene Flow and KITTI Stereo datasets, and rank 1st on both the KITTI Stereo 2012 and 2015 benchmarks, which demonstrates the effectiveness of the proposed module. The code of CSPN proposed in this work will be released at https://github.com/XinJCheng/CSPN.
Leave-one-out least squares Monte Carlo algorithm for pricing Bermudan options
Authors: Jeechul Woo, Chenru Liu, Jaehyuk Choi
Date: 2018-10-04 00:00:00
Tasks: regression

The least squares Monte Carlo (LSM) algorithm proposed by Longstaff and Schwartz (2001) is widely used for pricing Bermudan options. The LSM estimator contains undesirable look-ahead bias, and the conventional technique of avoiding it requires additional simulation paths. We present the leave-one-out LSM (LOOLSM) algorithm to eliminate look-ahead bias without doubling simulations. We also show that look-ahead bias is asymptotically proportional to the regressors-to-paths ratio. Our findings are demonstrated with several option examples in which the LSM algorithm overvalues the options. The LOOLSM method can be extended to other regression-based algorithms that improve the LSM method.
Transfer Learning via Unsupervised Task Discovery for Visual Question Answering
Authors: Hyeonwoo Noh, Tae-hoon Kim, Jonghwan Mun, Bohyung Han
Date: 2018-10-03 00:00:00
Proceeding: CVPR 2019 6
Tasks: Question Answering, Transfer Learning, Visual Question Answering, Visual Question Answering (VQA)

We study how to leverage off-the-shelf visual and linguistic data to cope
with out-of-vocabulary answers in visual question answering task. Existing
large-scale visual datasets with annotations such as image class labels,
bounding boxes and region descriptions are good sources for learning rich and
diverse visual concepts. However, it is not straightforward how the visual
concepts can be captured and transferred to visual question answering models
due to missing link between question dependent answering models and visual data
without question. We tackle this problem in two steps: 1) learning a task
conditional visual classifier, which is capable of solving diverse
question-specific visual recognition tasks, based on unsupervised task
discovery and 2) transferring the task conditional visual classifier to visual
question answering models. Specifically, we employ linguistic knowledge sources
such as structured lexical database (e.g. WordNet) and visual descriptions for
unsupervised task discovery, and transfer a learned task conditional visual
classifier as an answering unit in a visual question answering model. We
empirically show that the proposed algorithm generalizes to out-of-vocabulary
answers successfully using the knowledge transferred from the visual dataset.
Relative Saliency and Ranking: Models, Metrics, Data, and Benchmarks
Authors: Mahmoud Kalash, Md Amirul Islam, Neil D. B. Bruce
Date: 2018-10-03 00:00:00
Tasks: Object, object-detection, Object Detection, RGB Salient Object Detection, Salient Object Detection

Salient object detection is a problem that has been considered in detail and \textcolor{black}{many solutions have been proposed}. In this paper, we argue that work to date has addressed a problem that is relatively ill-posed. Specifically, there is not universal agreement about what constitutes a salient object when multiple observers are queried. This implies that some objects are more likely to be judged salient than others, and implies a relative rank exists on salient objects. Initially, we present a novel deep learning solution based on a hierarchical representation of relative saliency and stage-wise refinement. Further to this, we present data, analysis and baseline benchmark results towards addressing the problem of salient object ranking. Methods for deriving suitable ranked salient object instances are presented, along with metrics suitable to measuring algorithm performance. In addition, we show how a derived dataset can be successively refined to provide cleaned results that correlate well with pristine ground truth in its characteristics and value for training and testing models. Finally, we provide a comparison among prevailing algorithms that address salient object ranking or detection to establish initial baselines providing a basis for comparison with future efforts addressing this problem. \textcolor{black}{The source code and data are publicly available via our project page:} \textrm{\href{https://ryersonvisionlab.github.io/cocosalrank.html}{ryersonvisionlab.github.io/cocosalrank}}
Human Indignity: From Legal AI Personhood to Selfish Memes
Authors: Roman V. Yampolskiy
Date: 2018-10-02 00:00:00

It is possible to rely on current corporate law to grant legal personhood to Artificially Intelligent (AI) agents. In this paper, after introducing pathways to AI personhood, we analyze consequences of such AI empowerment on human dignity, human safety and AI rights. We emphasize possibility of creating selfish memes and legal system hacking in the context of artificial entities. Finally, we consider some potential solutions for addressing described problems.
Deep learning cardiac motion analysis for human survival prediction
Authors: Ghalib A. Bello, Timothy J. W. Dawes, Jinming Duan, Carlo Biffi, Antonio de Marvao, Luke S. G. E. Howard, J. Simon R. Gibbs, Martin R. Wilkins, Stuart A. Cook, Daniel Rueckert, Declan P. O'Regan
Date: 2018-10-08 00:00:00
Tasks: Deep Learning, Denoising, Survival Prediction

Motion analysis is used in computer vision to understand the behaviour of
moving objects in sequences of images. Optimising the interpretation of dynamic
biological systems requires accurate and precise motion tracking as well as
efficient representations of high-dimensional motion trajectories so that these
can be used for prediction tasks. Here we use image sequences of the heart,
acquired using cardiac magnetic resonance imaging, to create time-resolved
three-dimensional segmentations using a fully convolutional network trained on
anatomical shape priors. This dense motion model formed the input to a
supervised denoising autoencoder (4Dsurvival), which is a hybrid network
consisting of an autoencoder that learns a task-specific latent code
representation trained on observed outcome data, yielding a latent
representation optimised for survival prediction. To handle right-censored
survival outcomes, our network used a Cox partial likelihood loss function. In
a study of 302 patients the predictive accuracy (quantified by Harrell's
C-index) was significantly higher (p < .0001) for our model C=0.73 (95$\%$ CI:
0.68 - 0.78) than the human benchmark of C=0.59 (95$\%$ CI: 0.53 - 0.65). This
work demonstrates how a complex computer vision task using high-dimensional
medical image data can efficiently predict human survival.
SFV: Reinforcement Learning of Physical Skills from Videos
Authors: Xue Bin Peng, Angjoo Kanazawa, Jitendra Malik, Pieter Abbeel, Sergey Levine
Date: 2018-10-08 00:00:00
Tasks: Deep Reinforcement Learning, Pose Estimation, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Data-driven character animation based on motion capture can produce highly
naturalistic behaviors and, when combined with physics simulation, can provide
for natural procedural responses to physical perturbations, environmental
changes, and morphological discrepancies. Motion capture remains the most
popular source of motion data, but collecting mocap data typically requires
heavily instrumented environments and actors. In this paper, we propose a
method that enables physically simulated characters to learn skills from videos
(SFV). Our approach, based on deep pose estimation and deep reinforcement
learning, allows data-driven animation to leverage the abundance of publicly
available video clips from the web, such as those from YouTube. This has the
potential to enable fast and easy design of character controllers simply by
querying for video recordings of the desired behavior. The resulting
controllers are robust to perturbations, can be adapted to new settings, can
perform basic object interactions, and can be retargeted to new morphologies
via reinforcement learning. We further demonstrate that our method can predict
potential human motions from still images, by forward simulation of learned
controllers initialized from the observed pose. Our framework is able to learn
a broad range of dynamic skills, including locomotion, acrobatics, and martial
arts.
End-to-End Text Classification via Image-based Embedding using Character-level Networks
Authors: Shunsuke Kitada, Ryunosuke Kotani, Hitoshi Iyatomi
Date: 2018-10-08 00:00:00
Tasks: Document Classification, General Classification, Morphological Analysis, text-classification, Text Classification, Word Embeddings

For analysing and/or understanding languages having no word boundaries based
on morphological analysis such as Japanese, Chinese, and Thai, it is desirable
to perform appropriate word segmentation before word embeddings. But it is
inherently difficult in these languages. In recent years, various language
models based on deep learning have made remarkable progress, and some of these
methodologies utilizing character-level features have successfully avoided such
a difficult problem. However, when a model is fed character-level features of
the above languages, it often causes overfitting due to a large number of
character types. In this paper, we propose a CE-CLCNN, character-level
convolutional neural networks using a character encoder to tackle these
problems. The proposed CE-CLCNN is an end-to-end learning model and has an
image-based character encoder, i.e. the CE-CLCNN handles each character in the
target document as an image. Through various experiments, we found and
confirmed that our CE-CLCNN captured closely embedded features for visually and
semantically similar characters and achieves state-of-the-art results on
several open document classification tasks. In this paper we report the
performance of our CE-CLCNN with the Wikipedia title estimation task and
analyse the internal behaviour.
Proximal Online Gradient is Optimum for Dynamic Regret
Authors: Yawei Zhao, Shuang Qiu, Ji Liu
Date: 2018-10-08 00:00:00

In online learning, the dynamic regret metric chooses the reference (optimal) solution that may change over time, while the typical (static) regret metric assumes the reference solution to be constant over the whole time horizon. The dynamic regret metric is particularly interesting for applications such as online recommendation (since the customers' preference always evolves over time). While the online gradient method has been shown to be optimal for the static regret metric, the optimal algorithm for the dynamic regret remains unknown. In this paper, we show that proximal online gradient (a general version of online gradient) is optimum to the dynamic regret by showing that the proved lower bound matches the upper bound that slightly improves existing upper bound.
Algorithmic Aspects of Inverse Problems Using Generative Models
Authors: Chinmay Hegde
Date: 2018-10-08 00:00:00

The traditional approach of hand-crafting priors (such as sparsity) for
solving inverse problems is slowly being replaced by the use of richer learned
priors (such as those modeled by generative adversarial networks, or GANs). In
this work, we study the algorithmic aspects of such a learning-based approach
from a theoretical perspective. For certain generative network architectures,
we establish a simple non-convex algorithmic approach that (a) theoretically
enjoys linear convergence guarantees for certain inverse problems, and (b)
empirically improves upon conventional techniques such as back-propagation. We
also propose an extension of our approach that can handle model mismatch (i.e.,
situations where the generative network prior is not exactly applicable.)
Together, our contributions serve as building blocks towards a more complete
algorithmic understanding of generative models in inverse problems.
Towards Robot-Centric Conceptual Knowledge Acquisition
Authors: Georg Jäger, Christian A. Mueller, Madhura Thosar, Sebastian Zug, Andreas Birk
Date: 2018-10-08 00:00:00
Tasks: Common Sense Reasoning

Robots require knowledge about objects in order to efficiently perform
various household tasks involving objects. The existing knowledge bases for
robots acquire symbolic knowledge about objects from manually-coded external
common sense knowledge bases such as ConceptNet, Word-Net etc. The problem with
such approaches is the discrepancy between human-centric symbolic knowledge and
robot-centric object perception due to its limited perception capabilities.
Ultimately, significant portion of knowledge in the knowledge base remains
ungrounded into robot's perception. To overcome this discrepancy, we propose an
approach to enable robots to generate robot-centric symbolic knowledge about
objects from their own sensory data, thus, allowing them to assemble their own
conceptual understanding of objects. With this goal in mind, the presented
paper elaborates on the work-in-progress of the proposed approach followed by
the preliminary results.
Improving the Transformer Translation Model with Document-Level Context
Authors: Jiacheng Zhang, Huanbo Luan, Maosong Sun, FeiFei Zhai, Jingfang Xu, Min Zhang, Yang Liu
Date: 2018-10-08 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Decoder, Sentence, Translation

Although the Transformer translation model (Vaswani et al., 2017) has
achieved state-of-the-art performance in a variety of translation tasks, how to
use document-level context to deal with discourse phenomena problematic for
Transformer still remains a challenge. In this work, we extend the Transformer
model with a new context encoder to represent document-level context, which is
then incorporated into the original encoder and decoder. As large-scale
document-level parallel corpora are usually not available, we introduce a
two-step training method to take full advantage of abundant sentence-level
parallel corpora and limited document-level parallel corpora. Experiments on
the NIST Chinese-English datasets and the IWSLT French-English datasets show
that our approach improves over Transformer significantly.
Bootstrapped CNNs for Building Segmentation on RGB-D Aerial Imagery
Authors: Clint Sebastian, Bas Boom, Thijs van Lankveld, Egor Bondarev, Peter H. N. de With
Date: 2018-10-08 00:00:00

Detection of buildings and other objects from aerial images has various
applications in urban planning and map making. Automated building detection
from aerial imagery is a challenging task, as it is prone to varying lighting
conditions, shadows and occlusions. Convolutional Neural Networks (CNNs) are
robust against some of these variations, although they fail to distinguish easy
and difficult examples. We train a detection algorithm from RGB-D images to
obtain a segmented mask by using the CNN architecture DenseNet.First, we
improve the performance of the model by applying a statistical re-sampling
technique called Bootstrapping and demonstrate that more informative examples
are retained. Second, the proposed method outperforms the non-bootstrapped
version by utilizing only one-sixth of the original training data and it
obtains a precision-recall break-even of 95.10% on our aerial imagery dataset.
Multi-Source Cross-Lingual Model Transfer: Learning What to Share
Authors: Xilun Chen, Ahmed Hassan Awadallah, Hany Hassan, Wei Wang, Claire Cardie
Date: 2018-10-08 00:00:00
Proceeding: ACL 2019 7
Tasks: Cross-Lingual NER, Cross-Lingual Transfer, Mixture-of-Experts, text-classification, Text Classification, Transfer Learning

Modern NLP applications have enjoyed a great boost utilizing neural networks models. Such deep neural models, however, are not applicable to most human languages due to the lack of annotated training data for various NLP tasks. Cross-lingual transfer learning (CLTL) is a viable method for building NLP models for a low-resource target language by leveraging labeled data from other (source) languages. In this work, we focus on the multilingual transfer setting where training data in multiple source languages is leveraged to further boost target language performance. Unlike most existing methods that rely only on language-invariant features for CLTL, our approach coherently utilizes both language-invariant and language-specific features at instance level. Our model leverages adversarial networks to learn language-invariant features, and mixture-of-experts models to dynamically exploit the similarity between the target language and each individual source language. This enables our model to learn effectively what to share between various languages in the multilingual setup. Moreover, when coupled with unsupervised multilingual embeddings, our model can operate in a zero-resource setting where neither target language training data nor cross-lingual resources are available. Our model achieves significant performance gains over prior art, as shown in an extensive set of experiments over multiple text classification and sequence tagging tasks including a large-scale industry dataset.