pwc-archive/papers-with-abstracts huggingface dataset:
Hyperparameter Optimization: A Spectral Approach
Authors: Elad Hazan, Adam Klivans, Yang Yuan
Date: 2017-06-02 00:00:00
Proceeding: ICLR 2018 1
Tasks: Bayesian Optimization, compressed sensing, Hyperparameter Optimization

We give a simple, fast algorithm for hyperparameter optimization inspired by
techniques from the analysis of Boolean functions. We focus on the
high-dimensional regime where the canonical example is training a neural
network with a large number of hyperparameters. The algorithm --- an iterative
application of compressed sensing techniques for orthogonal polynomials ---
requires only uniform sampling of the hyperparameters and is thus easily
parallelizable.
  Experiments for training deep neural networks on Cifar-10 show that compared
to state-of-the-art tools (e.g., Hyperband and Spearmint), our algorithm finds
significantly improved solutions, in some cases better than what is attainable
by hand-tuning. In terms of overall running time (i.e., time required to sample
various settings of hyperparameters plus additional computation time), we are
at least an order of magnitude faster than Hyperband and Bayesian Optimization.
We also outperform Random Search 8x.
  Additionally, our method comes with provable guarantees and yields the first
improvements on the sample complexity of learning decision trees in over two
decades. In particular, we obtain the first quasi-polynomial time algorithm for
learning noisy decision trees with polynomial sample complexity.
Unsupervised End-to-end Learning for Deformable Medical Image Registration
Authors: Siyuan Shan, Wen Yan, Xiaoqing Guo, Eric I-Chao Chang, Yubo Fan, Yan Xu
Date: 2017-11-23 00:00:00
Tasks: Deformable Medical Image Registration, Image Registration, Medical Image Registration

We propose a registration algorithm for 2D CT/MRI medical images with a new
unsupervised end-to-end strategy using convolutional neural networks. The
contributions of our algorithm are threefold: (1) We transplant traditional
image registration algorithms to an end-to-end convolutional neural network
framework, while maintaining the unsupervised nature of image registration
problems. The image-to-image integrated framework can simultaneously learn both
image features and transformation matrix for registration. (2) Training with
additional data without any label can further improve the registration
performance by approximately 10 %. (3) The registration speed is 100x faster
than traditional methods. The proposed network is easy to implement and can be
trained efficiently. Experiments demonstrate that our system achieves
state-of-the-art results on 2D brain registration and achieves comparable
results on 2D liver registration. It can be extended to register other organs
beyond liver and brain such as kidney, lung, and heart.
Symmetry, Saddle Points, and Global Optimization Landscape of Nonconvex Matrix Factorization
Authors: Xingguo Li, Junwei Lu, Raman Arora, Jarvis Haupt, Han Liu, Zhaoran Wang, Tuo Zhao
Date: 2016-12-29 00:00:00
Tasks: global-optimization, Retrieval

We propose a general theory for studying the \xl{landscape} of nonconvex
\xl{optimization} with underlying symmetric structures \tz{for a class of
machine learning problems (e.g., low-rank matrix factorization, phase
retrieval, and deep linear neural networks)}. In specific, we characterize the
locations of stationary points and the null space of Hessian matrices \xl{of
the objective function} via the lens of invariant groups\removed{for associated
optimization problems, including low-rank matrix factorization, phase
retrieval, and deep linear neural networks}. As a major motivating example, we
apply the proposed general theory to characterize the global \xl{landscape} of
the \xl{nonconvex optimization in} low-rank matrix factorization problem. In
particular, we illustrate how the rotational symmetry group gives rise to
infinitely many nonisolated strict saddle points and equivalent global minima
of the objective function. By explicitly identifying all stationary points, we
divide the entire parameter space into three regions: ($\cR_1$) the region
containing the neighborhoods of all strict saddle points, where the objective
has negative curvatures; ($\cR_2$) the region containing neighborhoods of all
global minima, where the objective enjoys strong convexity along certain
directions; and ($\cR_3$) the complement of the above regions, where the
gradient has sufficiently large magnitudes. We further extend our result to the
matrix sensing problem. Such global landscape implies strong global convergence
guarantees for popular iterative algorithms with arbitrary initial solutions.
The game theoretic p-Laplacian and semi-supervised learning with few labels
Authors: Jeff Calder
Date: 2017-11-28 00:00:00

We study the game theoretic p-Laplacian for semi-supervised learning on
graphs, and show that it is well-posed in the limit of finite labeled data and
infinite unlabeled data. In particular, we show that the continuum limit of
graph-based semi-supervised learning with the game theoretic p-Laplacian is a
weighted version of the continuous p-Laplace equation. We also prove that
solutions to the graph p-Laplace equation are approximately Holder continuous
with high probability. Our proof uses the viscosity solution machinery and the
maximum principle on a graph.
Side Information for Face Completion: a Robust PCA Approach
Authors: Niannan Xue, Jiankang Deng, Shiyang Cheng, Yannis Panagakis, Stefanos Zafeiriou
Date: 2018-01-20 00:00:00
Tasks: Face Recognition, Facial Inpainting, Generative Adversarial Network, Missing Values, Robust Face Recognition

Robust principal component analysis (RPCA) is a powerful method for learning
low-rank feature representation of various visual data. However, for certain
types as well as significant amount of error corruption, it fails to yield
satisfactory results; a drawback that can be alleviated by exploiting
domain-dependent prior knowledge or information. In this paper, we propose two
models for the RPCA that take into account such side information, even in the
presence of missing values. We apply this framework to the task of UV
completion which is widely used in pose-invariant face recognition. Moreover,
we construct a generative adversarial network (GAN) to extract side information
as well as subspaces. These subspaces not only assist in the recovery but also
speed up the process in case of large-scale data. We quantitatively and
qualitatively evaluate the proposed approaches through both synthetic data and
five real-world datasets to verify their effectiveness.
Multiple Description Convolutional Neural Networks for Image Compression
Authors: Lijun Zhao, Huihui Bai, Anhong Wang, Yao Zhao
Date: 2018-01-20 00:00:00
Tasks: Image Compression

Multiple description coding (MDC) is able to stably transmit the signal in
the un-reliable and non-prioritized networks, which has been broadly studied
for several decades. However, the traditional MDC doesn't well leverage image's
context features to generate multiple descriptions. In this paper, we propose a
novel standard-compliant convolutional neural network-based MDC framework in
term of image's context features. Firstly, multiple description generator
network (MDGN) is designed to produce appearance-similar yet feature-different
multiple descriptions automatically according to image's content, which are
compressed by standard codec. Secondly, we present multiple description
reconstruction network (MDRN) including side reconstruction network (SRN) and
central reconstruction network (CRN). When any one of two lossy descriptions is
received at the decoder, SRN network is used to improve the quality of this
decoded lossy description by removing the compression artifact and up-sampling
simultaneously. Meanwhile, we utilize CRN network with two decoded descriptions
as inputs for better reconstruction, if both of lossy descriptions are
available. Thirdly, multiple description virtual codec network (MDVCN) is
proposed to bridge the gap between MDGN network and MDRN network in order to
train an end-to-end MDC framework. Here, two learning algorithms are provided
to train our whole framework. In addition to structural similarity loss
function, the produced descriptions are used as opposing labels with multiple
description distance loss function to regularize the training of MDGN network.
These losses guarantee that the generated description images are structurally
similar yet finely diverse. Experimental results show a great deal of objective
and subjective quality measurements to validate the efficiency of the proposed
method.
A Work Zone Simulation Model for Travel Time Prediction in a Connected Vehicle Environment
Authors: Xuejin Wen
Date: 2018-01-20 00:00:00
Tasks: Model Selection, regression, Travel Time Estimation

A work zone bottleneck in a roadway network can cause traffic delays,
emissions and safety issues. Accurate measurement and prediction of work zone
travel time can help travelers make better routing decisions and therefore
mitigate its impact. Historically, data used for travel time analyses comes
from fixed loop detectors, which are expensive to install and maintain. With
connected vehicle technology, such as Vehicle-to-Infrastructure, portable
roadside unit (RSU) can be located in and around a work zone segment to
communicate with the vehicles and collect traffic data. A PARAMICS simulation
model for a prototypical freeway work zone in a connected vehicle environment
was built to test this idea using traffic demand data from NY State Route 104.
For the simulation, twelve RSUs were placed along the work zone segment and
sixteen variables were extracted from the simulation results to explore travel
time estimation and prediction. For the travel time analysis, four types of
models were constructed, including linear regression, multivariate adaptive
regression splines (MARS), stepwise regression and elastic net. The results
show that the modeling approaches under consideration have similar performance
in terms of the Root of Mean Square Error (RMSE), which provides an opportunity
for model selection based on additional factors including the number and
locations of the RSUs according to the significant variables identified in the
various models. Among the four approaches, the stepwise regression model only
needs variables from two RSUs: one placed sufficiently upstream of the work
zone and one at the end of the work zone.
CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
Authors: Liangzhen Lai, Naveen Suda, Vikas Chandra
Date: 2018-01-19 00:00:00
Tasks: Efficient Neural Network

Deep Neural Networks are becoming increasingly popular in always-on IoT edge
devices performing data analytics right at the source, reducing latency as well
as energy consumption for data communication. This paper presents CMSIS-NN,
efficient kernels developed to maximize the performance and minimize the memory
footprint of neural network (NN) applications on Arm Cortex-M processors
targeted for intelligent IoT edge devices. Neural network inference based on
CMSIS-NN kernels achieves 4.6X improvement in runtime/throughput and 4.9X
improvement in energy efficiency.
mvn2vec: Preservation and Collaboration in Multi-View Network Embedding
Authors: Yu Shi, Fangqiu Han, Xinwei He, Xinran He, Carl Yang, Jie Luo, Jiawei Han
Date: 2018-01-19 00:00:00
Tasks: Network Embedding, Representation Learning

Multi-view networks are broadly present in real-world applications. In the meantime, network embedding has emerged as an effective representation learning approach for networked data. Therefore, we are motivated to study the problem of multi-view network embedding with a focus on the optimization objectives that are specific and important in embedding this type of network. In our practice of embedding real-world multi-view networks, we explicitly identify two such objectives, which we refer to as preservation and collaboration. The in-depth analysis of these two objectives is discussed throughout this paper. In addition, the mvn2vec algorithms are proposed to (i) study how varied extent of preservation and collaboration can impact embedding learning and (ii) explore the feasibility of achieving better embedding quality by modeling them simultaneously. With experiments on a series of synthetic datasets, a large-scale internal Snapchat dataset, and two public datasets, we confirm the validity and importance of preservation and collaboration as two objectives for multi-view network embedding. These experiments further demonstrate that better embedding can be obtained by simultaneously modeling the two objectives, while not over-complicating the model or requiring additional supervision. The code and the processed datasets are available at http://yushi2.web.engr.illinois.edu/.
Machine Learning Topological Invariants with Neural Networks
Authors: Pengfei Zhang, Huitao Shen, Hui Zhai
Date: 2017-08-30 00:00:00
Tasks: BIG-bench Machine Learning

In this Letter we supervisedly train neural networks to distinguish different
topological phases in the context of topological band insulators. After
training with Hamiltonians of one-dimensional insulators with chiral symmetry,
the neural network can predict their topological winding numbers with nearly
100% accuracy, even for Hamiltonians with larger winding numbers that are not
included in the training data. These results show a remarkable success that the
neural network can capture the global and nonlinear topological features of
quantum phases from local inputs. By opening up the neural network, we confirm
that the network does learn the discrete version of the winding number formula.
We also make a couple of remarks regarding the role of the symmetry and the
opposite effect of regularization techniques when applying machine learning to
physical systems.
A Foreground Inference Network for Video Surveillance Using Multi-View Receptive Field
Authors: Thangarajah Akilan
Date: 2018-01-19 00:00:00
Tasks: image-classification, Image Classification, Transfer Learning

Foreground (FG) pixel labelling plays a vital role in video surveillance.
Recent engineering solutions have attempted to exploit the efficacy of deep
learning (DL) models initially targeted for image classification to deal with
FG pixel labelling. One major drawback of such strategy is the lacking
delineation of visual objects when training samples are limited. To grapple
with this issue, we introduce a multi-view receptive field fully convolutional
neural network (MV-FCN) that harness recent seminal ideas, such as, fully
convolutional structure, inception modules, and residual networking. Therefrom,
we implement a system in an encoder-decoder fashion that subsumes a core and
two complementary feature flow paths. The model exploits inception modules at
early and late stages with three different sizes of receptive fields to capture
invariance at various scales. The features learned in the encoding phase are
fused with appropriate feature maps in the decoding phase through residual
connections for achieving enhanced spatial representation. These multi-view
receptive fields and residual feature connections are expected to yield highly
generalized features for an accurate pixel-wise FG region identification. It
is, then, trained with database specific exemplary segmentations to predict
desired FG objects.
  The comparative experimental results on eleven benchmark datasets validate
that the proposed model achieves very competitive performance with the prior-
and state-of-the-art algorithms. We also report that how well a transfer
learning approach can be useful to enhance the performance of our proposed
MV-FCN.
A machine learning approach to reconstruction of heart surface potentials from body surface potentials
Authors: Avinash Malik, Tommy Peng, Mark Trew
Date: 2018-01-19 00:00:00
Tasks: BIG-bench Machine Learning

Invasive cardiac catheterisation is a common procedure that is carried out
before surgical intervention. Yet, invasive cardiac diagnostics are full of
risks, especially for young children. Decades of research has been conducted on
the so called inverse problem of electrocardiography, which can be used to
reconstruct Heart Surface Potentials (HSPs) from Body Surface Potentials
(BSPs), for non-invasive diagnostics. State of the art solutions to the inverse
problem are unsatisfactory, since the inverse problem is known to be ill-posed.
In this paper we propose a novel approach to reconstructing HSPs from BSPs
using a Time-Delay Artificial Neural Network (TDANN). We first design the TDANN
architecture, and then develop an iterative search space algorithm to find the
parameters of the TDANN, which results in the best overall HSP prediction. We
use real-world recorded BSPs and HSPs from individuals suffering from serious
cardiac conditions to validate our TDANN. The results are encouraging, in that
coefficients obtained by correlating the predicted HSP with the recorded
patient' HSP approach ideal values.
Network Structure Inference, A Survey: Motivations, Methods, and Applications
Authors: Ivan Brugere, Brian Gallagher, Tanya Y. Berger-Wolf
Date: 2016-10-03 00:00:00
Tasks: Survey

Networks represent relationships between entities in many complex systems,
spanning from online social interactions to biological cell development and
brain connectivity. In many cases, relationships between entities are
unambiguously known: are two users 'friends' in a social network? Do two
researchers collaborate on a published paper? Do two road segments in a
transportation system intersect? These are directly observable in the system in
question. In most cases, relationship between nodes are not directly observable
and must be inferred: does one gene regulate the expression of another? Do two
animals who physically co-locate have a social bond? Who infected whom in a
disease outbreak in a population?
  Existing approaches for inferring networks from data are found across many
application domains and use specialized knowledge to infer and measure the
quality of inferred network for a specific task or hypothesis. However, current
research lacks a rigorous methodology which employs standard statistical
validation on inferred models. In this survey, we examine (1) how network
representations are constructed from underlying data, (2) the variety of
questions and tasks on these representations over several domains, and (3)
validation strategies for measuring the inferred network's capability of
answering questions on the system of interest.
Stochastic Gradient Descent as Approximate Bayesian Inference
Authors: Stephan Mandt, Matthew D. Hoffman, David M. Blei
Date: 2017-04-13 00:00:00
Tasks: Bayesian Inference

Stochastic Gradient Descent with a constant learning rate (constant SGD)
simulates a Markov chain with a stationary distribution. With this perspective,
we derive several new results. (1) We show that constant SGD can be used as an
approximate Bayesian posterior inference algorithm. Specifically, we show how
to adjust the tuning parameters of constant SGD to best match the stationary
distribution to a posterior, minimizing the Kullback-Leibler divergence between
these two distributions. (2) We demonstrate that constant SGD gives rise to a
new variational EM algorithm that optimizes hyperparameters in complex
probabilistic models. (3) We also propose SGD with momentum for sampling and
show how to adjust the damping coefficient accordingly. (4) We analyze MCMC
algorithms. For Langevin Dynamics and Stochastic Gradient Fisher Scoring, we
quantify the approximation errors due to finite learning rates. Finally (5), we
use the stochastic process perspective to give a short proof of why Polyak
averaging is optimal. Based on this idea, we propose a scalable approximate
MCMC algorithm, the Averaged Stochastic Gradient Sampler.
Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces
Authors: Garrett Warnell, Nicholas Waytowich, Vernon Lawhern, Peter Stone
Date: 2017-09-28 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Vocal Bursts Intensity Prediction

While recent advances in deep reinforcement learning have allowed autonomous
learning agents to succeed at a variety of complex tasks, existing algorithms
generally require a lot of training data. One way to increase the speed at
which agents are able to learn to perform tasks is by leveraging the input of
human trainers. Although such input can take many forms, real-time,
scalar-valued feedback is especially useful in situations where it proves
difficult or impossible for humans to provide expert demonstrations. Previous
approaches have shown the usefulness of human input provided in this fashion
(e.g., the TAMER framework), but they have thus far not considered
high-dimensional state spaces or employed the use of deep learning. In this
paper, we do both: we propose Deep TAMER, an extension of the TAMER framework
that leverages the representational power of deep neural networks in order to
learn complex tasks in just a short amount of time with a human trainer. We
demonstrate Deep TAMER's success by using it and just 15 minutes of
human-provided feedback to train an agent that performs better than humans on
the Atari game of Bowling - a task that has proven difficult for even
state-of-the-art reinforcement learning methods.
Supervised Learning of Labeled Pointcloud Differences via Cover-Tree Entropy Reduction
Authors: Abraham Smith, Paul Bendich, John Harer, Alex Pieloch, Jay Hineman
Date: 2017-02-26 00:00:00
Tasks: BIG-bench Machine Learning

We introduce a new algorithm, called CDER, for supervised machine learning
that merges the multi-scale geometric properties of Cover Trees with the
information-theoretic properties of entropy. CDER applies to a training set of
labeled pointclouds embedded in a common Euclidean space. If typical
pointclouds corresponding to distinct labels tend to differ at any scale in any
sub-region, CDER can identify these differences in (typically) linear time,
creating a set of distributional coordinates which act as a feature extraction
mechanism for supervised learning. We describe theoretical properties and
implementation details of CDER, and illustrate its benefits on several
synthetic examples.
How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Prediction
Authors: Nachiket Deo, Akshay Rangesh, Mohan M. Trivedi
Date: 2018-01-19 00:00:00
Tasks: Autonomous Vehicles, General Classification, motion prediction, Prediction, Trajectory Prediction

Reliable prediction of surround vehicle motion is a critical requirement for
path planning for autonomous vehicles. In this paper we propose a unified
framework for surround vehicle maneuver classification and motion prediction
that exploits multiple cues, namely, the estimated motion of vehicles, an
understanding of typical motion patterns of freeway traffic and inter-vehicle
interaction. We report our results in terms of maneuver classification accuracy
and mean and median absolute error of predicted trajectories against the ground
truth for real traffic data collected using vehicle mounted sensors on
freeways. An ablative analysis is performed to analyze the relative importance
of each cue for trajectory prediction. Additionally, an analysis of execution
time for the components of the framework is presented. Finally, we present
multiple case studies analyzing the outputs of our model for complex traffic
scenarios
A generalized parsing framework for Abstract Grammars
Authors: Daniel Harasim, Chris Bruno, Eva Portelance, Martin Rohrmeier, Timothy J. O'Donnell
Date: 2017-10-31 00:00:00

This technical report presents a general framework for parsing a variety of
grammar formalisms. We develop a grammar formalism, called an Abstract Grammar,
which is general enough to represent grammars at many levels of the hierarchy,
including Context Free Grammars, Minimalist Grammars, and Generalized
Context-free Grammars. We then develop a single parsing framework which is
capable of parsing grammars which are at least up to GCFGs on the hierarchy.
Our parsing framework exposes a grammar interface, so that it can parse any
particular grammar formalism that can be reduced to an Abstract Grammar.