pwc-archive/papers-with-abstracts huggingface dataset:
Potential-Function Proofs for First-Order Methods
Authors: Nikhil Bansal, Anupam Gupta
Date: 2017-12-13 00:00:00

This note discusses proofs for convergence of first-order methods based on simple potential-function arguments. We cover methods like gradient descent (for both smooth and non-smooth settings), mirror descent, and some accelerated variants.
Over the Air Deep Learning Based Radio Signal Classification
Authors: Timothy J. O'Shea, Tamoghna Roy, T. Charles Clancy
Date: 2017-12-13 00:00:00
Tasks: Deep Learning, General Classification

We conduct an in depth study on the performance of deep learning based radio
signal classification for radio communications signals. We consider a rigorous
baseline method using higher order moments and strong boosted gradient tree
classification and compare performance between the two approaches across a
range of configurations and channel impairments. We consider the effects of
carrier frequency offset, symbol rate, and multi-path fading in simulation and
conduct over-the-air measurement of radio classification performance in the lab
using software radios and compare performance and training strategies for both.
Finally we conclude with a discussion of remaining problems, and design
considerations for using such techniques.
Low-dose spectral CT reconstruction using L0 image gradient and tensor dictionary
Authors: Weiwen Wu, Yanbo Zhang, Qian Wang, Fenglin Liu, Peijun Chen, Hengyong Yu
Date: 2017-12-13 00:00:00
Tasks: Computed Tomography (CT), CT Reconstruction, Dictionary Learning, Lesion Detection

Spectral computed tomography (CT) has a great superiority in lesion
detection, tissue characterization and material decomposition. To further
extend its potential clinical applications, in this work, we propose an
improved tensor dictionary learning method for low-dose spectral CT
reconstruction with a constraint of image gradient L0-norm, which is named as
L0TDL. The L0TDL method inherits the advantages of tensor dictionary learning
(TDL) by employing the similarity of spectral CT images. On the other hand, by
introducing the L0-norm constraint in gradient image domain, the proposed
method emphasizes the spatial sparsity to overcome the weakness of TDL on
preserving edge information. The alternative direction minimization method
(ADMM) is employed to solve the proposed method. Both numerical simulations and
real mouse studies are perform to evaluate the proposed method. The results
show that the proposed L0TDL method outperforms other competing methods, such
as total variation (TV) minimization, TV with low rank (TV+LR), and TDL
methods.
Backprop as Functor: A compositional perspective on supervised learning
Authors: Brendan Fong, David I. Spivak, Rémy Tuyéras
Date: 2017-11-28 00:00:00

A supervised learning algorithm searches over a set of functions $A \to B$
parametrised by a space $P$ to find the best approximation to some ideal
function $f\colon A \to B$. It does this by taking examples $(a,f(a)) \in
A\times B$, and updating the parameter according to some rule. We define a
category where these update rules may be composed, and show that gradient
descent---with respect to a fixed step size and an error function satisfying a
certain property---defines a monoidal functor from a category of parametrised
functions to this category of update rules. This provides a structural
perspective on backpropagation, as well as a broad generalisation of neural
networks.
Logical Induction
Authors: Scott Garrabrant, Tsvi Benson-Tilsen, Andrew Critch, Nate Soares, Jessica Taylor
Date: 2016-09-12 00:00:00
Tasks: Sentence

We present a computable algorithm that assigns probabilities to every logical statement in a given formal language, and refines those probabilities over time. For instance, if the language is Peano arithmetic, it assigns probabilities to all arithmetical statements, including claims about the twin prime conjecture, the outputs of long-running computations, and its own probabilities. We show that our algorithm, an instance of what we call a logical inductor, satisfies a number of intuitive desiderata, including: (1) it learns to predict patterns of truth and falsehood in logical statements, often long before having the resources to evaluate the statements, so long as the patterns can be written down in polynomial time; (2) it learns to use appropriate statistical summaries to predict sequences of statements whose truth values appear pseudorandom; and (3) it learns to have accurate beliefs about its own current beliefs, in a manner that avoids the standard paradoxes of self-reference. For example, if a given computer program only ever produces outputs in a certain range, a logical inductor learns this fact in a timely manner; and if late digits in the decimal expansion of $\pi$ are difficult to predict, then a logical inductor learns to assign $\approx 10\%$ probability to "the $n$th digit of $\pi$ is a 7" for large $n$. Logical inductors also learn to trust their future beliefs more than their current beliefs, and their beliefs are coherent in the limit (whenever $\phi \implies \psi$, $\mathbb{P}_\infty(\phi) \le \mathbb{P}_\infty(\psi)$, and so on); and logical inductors strictly dominate the universal semimeasure in the limit. These properties and many others all follow from a single logical induction criterion, which is motivated by a series of stock trading analogies. Roughly speaking, each logical sentence $\phi$ is associated with a stock that is worth \$1 per share if [...]
Fusing Multiple Multiband Images
Authors: Reza Arablouei
Date: 2017-12-13 00:00:00
Tasks: Infrared And Visible Image Fusion

We consider the problem of fusing an arbitrary number of multiband, i.e.,
panchromatic, multispectral, or hyperspectral, images belonging to the same
scene. We use the well-known forward observation and linear mixture models with
Gaussian perturbations to formulate the maximum-likelihood estimator of the
endmember abundance matrix of the fused image. We calculate the Fisher
information matrix for this estimator and examine the conditions for the
uniqueness of the estimator. We use a vector total-variation penalty term
together with nonnegativity and sum-to-one constraints on the endmember
abundances to regularize the derived maximum-likelihood estimation problem. The
regularization facilitates exploiting the prior knowledge that natural images
are mostly composed of piecewise smooth regions with limited abrupt changes,
i.e., edges, as well as coping with potential ill-posedness of the fusion
problem. We solve the resultant convex optimization problem using the
alternating direction method of multipliers. We utilize the circular
convolution theorem in conjunction with the fast Fourier transform to alleviate
the computational complexity of the proposed algorithm. Experiments with
multiband images constructed from real hyperspectral datasets reveal the
superior performance of the proposed algorithm in comparison with the
state-of-the-art algorithms, which need to be used in tandem to fuse more than
two multiband images.
Im2Pano3D: Extrapolating 360 Structure and Semantics Beyond the Field of View
Authors: Shuran Song, Andy Zeng, Angel X. Chang, Manolis Savva, Silvio Savarese, Thomas Funkhouser
Date: 2017-12-12 00:00:00

We present Im2Pano3D, a convolutional neural network that generates a dense
prediction of 3D structure and a probability distribution of semantic labels
for a full 360 panoramic view of an indoor scene when given only a partial
observation (<= 50%) in the form of an RGB-D image. To make this possible,
Im2Pano3D leverages strong contextual priors learned from large-scale synthetic
and real-world indoor scenes. To ease the prediction of 3D structure, we
propose to parameterize 3D surfaces with their plane equations and train the
model to predict these parameters directly. To provide meaningful training
supervision, we use multiple loss functions that consider both pixel level
accuracy and global context consistency. Experiments demon- strate that
Im2Pano3D is able to predict the semantics and 3D structure of the unobserved
scene with more than 56% pixel accuracy and less than 0.52m average distance
error, which is significantly better than alternative approaches.
Practical Bayesian optimization in the presence of outliers
Authors: Ruben Martinez-Cantin, Kevin Tee, Michael McCourt
Date: 2017-12-12 00:00:00
Tasks: Bayesian Optimization, regression

Inference in the presence of outliers is an important field of research as
outliers are ubiquitous and may arise across a variety of problems and domains.
Bayesian optimization is method that heavily relies on probabilistic inference.
This allows outstanding sample efficiency because the probabilistic machinery
provides a memory of the whole optimization process. However, that virtue
becomes a disadvantage when the memory is populated with outliers, inducing
bias in the estimation. In this paper, we present an empirical evaluation of
Bayesian optimization methods in the presence of outliers. The empirical
evidence shows that Bayesian optimization with robust regression often produces
suboptimal results. We then propose a new algorithm which combines robust
regression (a Gaussian process with Student-t likelihood) with outlier
diagnostics to classify data points as outliers or inliers. By using an
scheduler for the classification of outliers, our method is more efficient and
has better convergence over the standard robust regression. Furthermore, we
show that even in controlled situations with no expected outliers, our method
is able to produce better results.
LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation
Authors: Gabriele Costante, Thomas A. Ciarfuglia
Date: 2017-09-18 00:00:00
Tasks: Motion Estimation, Optical Flow Estimation, Visual Odometry

This work proposes a novel deep network architecture to solve the camera
Ego-Motion estimation problem. A motion estimation network generally learns
features similar to Optical Flow (OF) fields starting from sequences of images.
This OF can be described by a lower dimensional latent space. Previous research
has shown how to find linear approximations of this space. We propose to use an
Auto-Encoder network to find a non-linear representation of the OF manifold. In
addition, we propose to learn the latent space jointly with the estimation
task, so that the learned OF features become a more robust description of the
OF input. We call this novel architecture LS-VO.
  The experiments show that LS-VO achieves a considerable increase in
performances in respect to baselines, while the number of parameters of the
estimation network only slightly increases.
Provably Accurate Double-Sparse Coding
Authors: Thanh V. Nguyen, Raymond K. W. Wong, Chinmay Hegde
Date: 2017-11-09 00:00:00

Sparse coding is a crucial subroutine in algorithms for various signal
processing, deep learning, and other machine learning applications. The central
goal is to learn an overcomplete dictionary that can sparsely represent a given
input dataset. However, a key challenge is that storage, transmission, and
processing of the learned dictionary can be untenably high if the data
dimension is high. In this paper, we consider the double-sparsity model
introduced by Rubinstein et al. (2010b) where the dictionary itself is the
product of a fixed, known basis and a data-adaptive sparse component. First, we
introduce a simple algorithm for double-sparse coding that can be amenable to
efficient implementation via neural architectures. Second, we theoretically
analyze its performance and demonstrate asymptotic sample complexity and
running time benefits over existing (provable) approaches for sparse coding. To
our knowledge, our work introduces the first computationally efficient
algorithm for double-sparse coding that enjoys rigorous statistical guarantees.
Finally, we support our analysis via several numerical experiments on simulated
data, confirming that our method can indeed be useful in problem sizes
encountered in practical applications.
POSEidon: Face-from-Depth for Driver Pose Estimation
Authors: Guido Borghi, Marco Venturelli, Roberto Vezzani, Rita Cucchiara
Date: 2016-11-30 00:00:00
Proceeding: CVPR 2017 7
Tasks: Face Reconstruction, Head Pose Estimation, Pose Estimation

Fast and accurate upper-body and head pose estimation is a key task for
automatic monitoring of driver attention, a challenging context characterized
by severe illumination changes, occlusions and extreme poses. In this work, we
present a new deep learning framework for head localization and pose estimation
on depth images. The core of the proposal is a regression neural network,
called POSEidon, which is composed of three independent convolutional nets
followed by a fusion layer, specially conceived for understanding the pose by
depth. In addition, to recover the intrinsic value of face appearance for
understanding head position and orientation, we propose a new Face-from-Depth
approach for learning image faces from depth. Results in face reconstruction
are qualitatively impressive. We test the proposed framework on two public
datasets, namely Biwi Kinect Head Pose and ICT-3DHP, and on Pandora, a new
challenging dataset mainly inspired by the automotive setup. Results show that
our method overcomes all recent state-of-art works, running in real time at
more than 30 frames per second.
A Mathematical Programming Approach for Integrated Multiple Linear Regression Subset Selection and Validation
Authors: Seokhyun Chung, Young Woong Park, Taesu Cheong
Date: 2017-12-12 00:00:00
Tasks: regression

Subset selection for multiple linear regression aims to construct a regression model that minimizes errors by selecting a small number of explanatory variables. Once a model is built, various statistical tests and diagnostics are conducted to validate the model and to determine whether the regression assumptions are met. Most traditional approaches require human decisions at this step. For example, the user adding or removing a variable until a satisfactory model is obtained. However, this trial-and-error strategy cannot guarantee that a subset that minimizes the errors while satisfying all regression assumptions will be found. In this paper, we propose a fully automated model building procedure for multiple linear regression subset selection that integrates model building and validation based on mathematical programming. The proposed model minimizes mean squared errors while ensuring that the majority of the important regression assumptions are met. We also propose an efficient constraint to approximate the constraint for the coefficient t-test. When no subset satisfies all of the considered regression assumptions, our model provides an alternative subset that satisfies most of these assumptions. Computational results show that our model yields better solutions (i.e., satisfying more regression assumptions) compared to the state-of-the-art benchmark models while maintaining similar explanatory power.
Learning Sparse Graphs for Prediction and Filtering of Multivariate Data Processes
Authors: Arun Venkitaraman, Dave Zachariah
Date: 2017-12-12 00:00:00
Tasks: Prediction

We address the problem of prediction of multivariate data process using an
underlying graph model. We develop a method that learns a sparse partial
correlation graph in a tuning-free and computationally efficient manner.
Specifically, the graph structure is learned recursively without the need for
cross-validation or parameter tuning by building upon a hyperparameter-free
framework. Our approach does not require the graph to be undirected and also
accommodates varying noise levels across different nodes.Experiments using
real-world datasets show that the proposed method offers significant
performance gains in prediction, in comparison with the graphs frequently
associated with these datasets.
Face-from-Depth for Head Pose Estimation on Depth Images
Authors: Guido Borghi, Matteo Fabbri, Roberto Vezzani, Simone Calderara, Rita Cucchiara
Date: 2017-12-12 00:00:00
Tasks: Head Detection, Head Pose Estimation, Pose Estimation

Depth cameras allow to set up reliable solutions for people monitoring and
behavior understanding, especially when unstable or poor illumination
conditions make unusable common RGB sensors. Therefore, we propose a complete
framework for the estimation of the head and shoulder pose based on depth
images only. A head detection and localization module is also included, in
order to develop a complete end-to-end system. The core element of the
framework is a Convolutional Neural Network, called POSEidon+, that receives as
input three types of images and provides the 3D angles of the pose as output.
Moreover, a Face-from-Depth component based on a Deterministic Conditional GAN
model is able to hallucinate a face from the corresponding depth image. We
empirically demonstrate that this positively impacts the system performances.
We test the proposed framework on two public datasets, namely Biwi Kinect Head
Pose and ICT-3DHP, and on Pandora, a new challenging dataset mainly inspired by
the automotive setup. Experimental results show that our method overcomes
several recent state-of-art works based on both intensity and depth input data,
running in real-time at more than 30 frames per second.
Camera Calibration for Daylight Specular-Point Locus
Authors: Mark S. Drew, Hamid Reza Vaezi Joze, Graham D. Finlayson
Date: 2017-12-12 00:00:00
Tasks: Camera Calibration

In this paper we present a new camera calibration method aimed at finding a
straight-line locus, in a special colour feature space, that is traversed by
daylights and as well also approximately followed by specular points. The aim
of the calibration is to enable recovering the colour of the illuminant in a
scene, using the calibrated camera. First we prove theoretically that any
candidate specular points, for an image that is generated by a specific camera
and taken under a daylight, must lie on a straight line in log-chromaticity
space, for a chromaticity that is generated using a geometric-mean denominator.
Use is made of the assumptions that daylight illuminants can be approximated
using Planckians and that camera sensors are narrowband or can be made so by
spectral sharpening. Then we show how a particular camera can be calibrated so
as to discover this locus. As applications we use this curve for illuminant
detection, and also for re-lighting of images to show they would appear under
lighting having a different colour temperature.
ANI-1: A data set of 20M off-equilibrium DFT calculations for organic molecules
Authors: Justin S. Smith, Olexandr Isayev, Adrian E. Roitberg
Date: 2017-08-16 00:00:00

One of the grand challenges in modern theoretical chemistry is designing and
implementing approximations that expedite ab initio methods without loss of
accuracy. Machine learning (ML), in particular neural networks, are emerging as
a powerful approach to constructing various forms of transferable atomistic
potentials. They have been successfully applied in a variety of applications in
chemistry, biology, catalysis, and solid-state physics. However, these models
are heavily dependent on the quality and quantity of data used in their
fitting. Fitting highly flexible ML potentials comes at a cost: a vast amount
of reference data is required to properly train these models. We address this
need by providing access to a large computational DFT database, which consists
of 20M conformations for 57,454 small organic molecules. We believe it will
become a new standard benchmark for comparison of current and future methods in
the ML potential community.
A Data-Driven Sparse-Learning Approach to Model Reduction in Chemical Reaction Networks
Authors: Farshad Harirchi, Omar A. Khalil, Sijia Liu, Paolo Elvati, Angela Violi, Alfred O. Hero
Date: 2017-12-12 00:00:00
Tasks: Sparse Learning

In this paper, we propose an optimization-based sparse learning approach to
identify the set of most influential reactions in a chemical reaction network.
This reduced set of reactions is then employed to construct a reduced chemical
reaction mechanism, which is relevant to chemical interaction network modeling.
The problem of identifying influential reactions is first formulated as a
mixed-integer quadratic program, and then a relaxation method is leveraged to
reduce the computational complexity of our approach. Qualitative and
quantitative validation of the sparse encoding approach demonstrates that the
model captures important network structural properties with moderate
computational load.
Double/Debiased Machine Learning for Treatment and Causal Parameters
Authors: Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, James Robins
Date: 2016-07-30 00:00:00
Tasks: BIG-bench Machine Learning, Causal Inference, valid

Most modern supervised statistical/machine learning (ML) methods are explicitly designed to solve prediction problems very well. Achieving this goal does not imply that these methods automatically deliver good estimators of causal parameters. Examples of such parameters include individual regression coefficients, average treatment effects, average lifts, and demand or supply elasticities. In fact, estimates of such causal parameters obtained via naively plugging ML estimators into estimating equations for such parameters can behave very poorly due to the regularization bias. Fortunately, this regularization bias can be removed by solving auxiliary prediction problems via ML tools. Specifically, we can form an orthogonal score for the target low-dimensional parameter by combining auxiliary and main ML predictions. The score is then used to build a de-biased estimator of the target parameter which typically will converge at the fastest possible 1/root(n) rate and be approximately unbiased and normal, and from which valid confidence intervals for these parameters of interest may be constructed. The resulting method thus could be called a "double ML" method because it relies on estimating primary and auxiliary predictive models. In order to avoid overfitting, our construction also makes use of the K-fold sample splitting, which we call cross-fitting. This allows us to use a very broad set of ML predictive methods in solving the auxiliary and main prediction problems, such as random forest, lasso, ridge, deep neural nets, boosted trees, as well as various hybrids and aggregators of these methods.
Scalable Co-Optimization of Morphology and Control in Embodied Machines
Authors: Nick Cheney, Josh Bongard, Vytas SunSpiral, Hod Lipson
Date: 2017-06-19 00:00:00

Evolution sculpts both the body plans and nervous systems of agents together
over time. In contrast, in AI and robotics, a robot's body plan is usually
designed by hand, and control policies are then optimized for that fixed
design. The task of simultaneously co-optimizing the morphology and controller
of an embodied robot has remained a challenge. In psychology, the theory of
embodied cognition posits that behavior arises from a close coupling between
body plan and sensorimotor control, which suggests why co-optimizing these two
subsystems is so difficult: most evolutionary changes to morphology tend to
adversely impact sensorimotor control, leading to an overall decrease in
behavioral performance. Here, we further examine this hypothesis and
demonstrate a technique for "morphological innovation protection", which
temporarily reduces selection pressure on recently morphologically-changed
individuals, thus enabling evolution some time to "readapt" to the new
morphology with subsequent control policy mutations. We show the potential for
this method to avoid local optima and converge to similar highly fit
morphologies across widely varying initial conditions, while sustaining fitness
improvements further into optimization. While this technique is admittedly only
the first of many steps that must be taken to achieve scalable optimization of
embodied machines, we hope that theoretical insight into the cause of
evolutionary stagnation in current methods will help to enable the automation
of robot design and behavioral training -- while simultaneously providing a
testbed to investigate the theory of embodied cognition.