pwc-archive/papers-with-abstracts huggingface dataset:
Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields
Authors: Seungryong Kim, Kihong Park, Kwanghoon Sohn, Stephen Lin
Date: 2016-03-21 00:00:00
Tasks: Depth Estimation, Depth Prediction, Intrinsic Image Decomposition

We present a method for jointly predicting a depth map and intrinsic images
from single-image input. The two tasks are formulated in a synergistic manner
through a joint conditional random field (CRF) that is solved using a novel
convolutional neural network (CNN) architecture, called the joint convolutional
neural field (JCNF) model. Tailored to our joint estimation problem, JCNF
differs from previous CNNs in its sharing of convolutional activations and
layers between networks for each task, its inference in the gradient domain
where there exists greater correlation between depth and intrinsic images, and
the incorporation of a gradient scale network that learns the confidence of
estimated gradients in order to effectively balance them in the solution. This
approach is shown to surpass state-of-the-art methods both on single-image
depth estimation and on intrinsic image decomposition.
A Discontinuous Neural Network for Non-Negative Sparse Approximation
Authors: Martijn Arts, Marius Cordts, Monika Gorin, Marc Spehr, Rudolf Mathar
Date: 2016-03-21 00:00:00
Tasks: Denoising

This paper investigates a discontinuous neural network which is used as a
model of the mammalian olfactory system and can more generally be applied to
solve non-negative sparse approximation problems. By inherently limiting the
systems integrators to having non-negative outputs, the system function becomes
discontinuous since the integrators switch between being inactive and being
active. It is shown that the presented network converges to equilibrium points
which are solutions to general non-negative least squares optimization
problems. We specify a Caratheodory solution and prove that the network is
stable, provided that the system matrix has full column-rank. Under a mild
condition on the equilibrium point, we show that the network converges to its
equilibrium within a finite number of switches. Two applications of the neural
network are shown. Firstly, we apply the network as a model of the olfactory
system and show that in principle it may be capable of performing complex
sparse signal recovery tasks. Secondly, we generalize the application to
include non-negative sparse approximation problems and compare the recovery
performance to a classical non-negative basis pursuit denoising algorithm. We
conclude that the recovery performance differs only marginally from the
classical algorithm, while the neural network has the advantage that no
performance critical regularization parameter has to be chosen prior to
recovery.
Data Augmentation via Levy Processes
Authors: Stefan Wager, William Fithian, Percy Liang
Date: 2016-03-21 00:00:00
Tasks: Data Augmentation, Image Augmentation

If a document is about travel, we may expect that short snippets of the
document should also be about travel. We introduce a general framework for
incorporating these types of invariances into a discriminative classifier. The
framework imagines data as being drawn from a slice of a Levy process. If we
slice the Levy process at an earlier point in time, we obtain additional
pseudo-examples, which can be used to train the classifier. We show that this
scheme has two desirable properties: it preserves the Bayes decision boundary,
and it is equivalent to fitting a generative model in the limit where we rewind
time back to 0. Our construction captures popular schemes such as Gaussian
feature noising and dropout training, as well as admitting new generalizations.
Deep Self-Convolutional Activations Descriptor for Dense Cross-Modal Correspondence
Authors: Seungryong Kim, Dongbo Min, Stephen Lin, Kwanghoon Sohn
Date: 2016-03-21 00:00:00

We present a novel descriptor, called deep self-convolutional activations
(DeSCA), designed for establishing dense correspondences between images taken
under different imaging modalities, such as different spectral ranges or
lighting conditions. Motivated by descriptors based on local self-similarity
(LSS), we formulate a novel descriptor by leveraging LSS in a deep
architecture, leading to better discriminative power and greater robustness to
non-rigid image deformations than state-of-the-art cross-modality descriptors.
The DeSCA first computes self-convolutions over a local support window for
randomly sampled patches, and then builds self-convolution activations by
performing an average pooling through a hierarchical formulation within a deep
convolutional architecture. Finally, the feature responses on the
self-convolution activations are encoded through a spatial pyramid pooling in a
circular configuration. In contrast to existing convolutional neural networks
(CNNs) based descriptors, the DeSCA is training-free (i.e., randomly sampled
patches are utilized as the convolution kernels), is robust to cross-modal
imaging, and can be densely computed in an efficient manner that significantly
reduces computational redundancy. The state-of-the-art performance of DeSCA on
challenging cases of cross-modal image pairs is demonstrated through extensive
experiments.
High-dimensional Black-box Optimization via Divide and Approximate Conquer
Authors: Peng Yang, Ke Tang, Xin Yao
Date: 2016-03-11 00:00:00
Tasks: Vocal Bursts Intensity Prediction

Divide and Conquer (DC) is conceptually well suited to high-dimensional
optimization by decomposing a problem into multiple small-scale sub-problems.
However, appealing performance can be seldom observed when the sub-problems are
interdependent. This paper suggests that the major difficulty of tackling
interdependent sub-problems lies in the precise evaluation of a partial
solution (to a sub-problem), which can be overwhelmingly costly and thus makes
sub-problems non-trivial to conquer. Thus, we propose an approximation
approach, named Divide and Approximate Conquer (DAC), which reduces the cost of
partial solution evaluation from exponential time to polynomial time.
Meanwhile, the convergence to the global optimum (of the original problem) is
still guaranteed. The effectiveness of DAC is demonstrated empirically on two
sets of non-separable high-dimensional problems.
Flow of Information in Feed-Forward Deep Neural Networks
Authors: Pejman Khadivi, Ravi Tandon, Naren Ramakrishnan
Date: 2016-03-20 00:00:00

Feed-forward deep neural networks have been used extensively in various
machine learning applications. Developing a precise understanding of the
underling behavior of neural networks is crucial for their efficient
deployment. In this paper, we use an information theoretic approach to study
the flow of information in a neural network and to determine how entropy of
information changes between consecutive layers. Moreover, using the Information
Bottleneck principle, we develop a constrained optimization problem that can be
used in the training process of a deep neural network. Furthermore, we
determine a lower bound for the level of data representation that can be
achieved in a deep neural network with an acceptable level of distortion.
An Approximation Approach for Solving the Subpath Planning Problem
Authors: Masoud Safilian, S. Mehdi Tashakkori, Sepehr Eghbali, Aliakbar Safilian
Date: 2016-03-20 00:00:00
Tasks: Robot Navigation

The subpath planning problem is a branch of the path planning problem, which
has widespread applications in automated manufacturing process as well as
vehicle and robot navigation. This problem is to find the shortest path or tour
subject for travelling a set of given subpaths. The current approaches for
dealing with the subpath planning problem are all based on meta-heuristic
approaches. It is well-known that meta-heuristic based approaches have several
deficiencies. To address them, we propose a novel approximation algorithm in
the O(n^3) time complexity class, which guarantees to solve any subpath
planning problem instance with the fixed ratio bound of 2. Also, the formal
proofs of the claims, our empirical evaluation shows that our approximation
method acts much better than a state-of-the-art method, both in result and
execution time.
Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science
Authors: Randal S. Olson, Nathan Bartley, Ryan J. Urbanowicz, Jason H. Moore
Date: 2016-03-20 00:00:00
Tasks: Automated Feature Engineering, AutoML, BIG-bench Machine Learning, Hyperparameter Optimization, Neural Architecture Search

As the field of data science continues to grow, there will be an
ever-increasing demand for tools that make machine learning accessible to
non-experts. In this paper, we introduce the concept of tree-based pipeline
optimization for automating one of the most tedious parts of machine
learning---pipeline design. We implement an open source Tree-based Pipeline
Optimization Tool (TPOT) in Python and demonstrate its effectiveness on a
series of simulated and real-world benchmark data sets. In particular, we show
that TPOT can design machine learning pipelines that provide a significant
improvement over a basic machine learning analysis while requiring little to no
input nor prior knowledge from the user. We also address the tendency for TPOT
to design overly complex pipelines by integrating Pareto optimization, which
produces compact pipelines without sacrificing classification accuracy. As
such, this work represents an important step toward fully automating machine
learning pipeline design.
Extracting Predictive Information from Heterogeneous Data Streams using Gaussian Processes
Authors: Sid Ghoshal, Stephen Roberts
Date: 2016-03-20 00:00:00
Tasks: Gaussian Processes, Sentiment Analysis, Time Series, Time Series Analysis

Financial markets are notoriously complex environments, presenting vast
amounts of noisy, yet potentially informative data. We consider the problem of
forecasting financial time series from a wide range of information sources
using online Gaussian Processes with Automatic Relevance Determination (ARD)
kernels. We measure the performance gain, quantified in terms of Normalised
Root Mean Square Error (NRMSE), Median Absolute Deviation (MAD) and Pearson
correlation, from fusing each of four separate data domains: time series
technicals, sentiment analysis, options market data and broker recommendations.
We show evidence that ARD kernels produce meaningful feature rankings that help
retain salient inputs and reduce input dimensionality, providing a framework
for sifting through financial complexity. We measure the performance gain from
fusing each domain's heterogeneous data streams into a single probabilistic
model. In particular our findings highlight the critical value of options data
in mapping out the curvature of price space and inspire an intuitive, novel
direction for research in financial prediction.
Stochastic model for phonemes uncovers an author-dependency of their usage
Authors: Weibing Deng, Armen E. Allahverdyan
Date: 2015-10-05 00:00:00

We study rank-frequency relations for phonemes, the minimal units that still
relate to linguistic meaning. We show that these relations can be described by
the Dirichlet distribution, a direct analogue of the ideal-gas model in
statistical mechanics. This description allows us to demonstrate that the
rank-frequency relations for phonemes of a text do depend on its author. The
author-dependency effect is not caused by the author's vocabulary (common words
used in different texts), and is confirmed by several alternative means. This
suggests that it can be directly related to phonemes. These features contrast
to rank-frequency relations for words, which are both author and text
independent and are governed by the Zipf's law.
Segmentation from Natural Language Expressions
Authors: Ronghang Hu, Marcus Rohrbach, Trevor Darrell
Date: 2016-03-20 00:00:00
Tasks: Referring Expression Segmentation, Segmentation, Semantic Segmentation

In this paper we approach the novel problem of segmenting an image based on a
natural language expression. This is different from traditional semantic
segmentation over a predefined set of semantic classes, as e.g., the phrase
"two men sitting on the right bench" requires segmenting only the two people on
the right bench and no one standing or sitting on another bench. Previous
approaches suitable for this task were limited to a fixed set of categories
and/or rectangular regions. To produce pixelwise segmentation for the language
expression, we propose an end-to-end trainable recurrent and convolutional
network model that jointly learns to process visual and linguistic information.
In our model, a recurrent LSTM network is used to encode the referential
expression into a vector representation, and a fully convolutional network is
used to a extract a spatial feature map from the image and output a spatial
response map for the target object. We demonstrate on a benchmark dataset that
our model can produce quality segmentation output from the natural language
expression, and outperforms baseline methods by a large margin.
Joint Stochastic Approximation learning of Helmholtz Machines
Authors: Haotian Xu, Zhijian Ou
Date: 2016-03-20 00:00:00

Though with progress, model learning and performing posterior inference still
remains a common challenge for using deep generative models, especially for
handling discrete hidden variables. This paper is mainly concerned with
algorithms for learning Helmholz machines, which is characterized by pairing
the generative model with an auxiliary inference model. A common drawback of
previous learning algorithms is that they indirectly optimize some bounds of
the targeted marginal log-likelihood. In contrast, we successfully develop a
new class of algorithms, based on stochastic approximation (SA) theory of the
Robbins-Monro type, to directly optimize the marginal log-likelihood and
simultaneously minimize the inclusive KL-divergence. The resulting learning
algorithm is thus called joint SA (JSA). Moreover, we construct an effective
MCMC operator for JSA. Our results on the MNIST datasets demonstrate that the
JSA's performance is consistently superior to that of competing algorithms like
RWS, for learning a range of difficult models.
ACDC: A Structured Efficient Linear Layer
Authors: Marcin Moczulski, Misha Denil, Jeremy Appleyard, Nando de Freitas
Date: 2015-11-18 00:00:00

The linear layer is one of the most pervasive modules in deep learning
representations. However, it requires $O(N^2)$ parameters and $O(N^2)$
operations. These costs can be prohibitive in mobile applications or prevent
scaling in many domains. Here, we introduce a deep, differentiable,
fully-connected neural network module composed of diagonal matrices of
parameters, $\mathbf{A}$ and $\mathbf{D}$, and the discrete cosine transform
$\mathbf{C}$. The core module, structured as $\mathbf{ACDC^{-1}}$, has $O(N)$
parameters and incurs $O(N log N )$ operations. We present theoretical results
showing how deep cascades of ACDC layers approximate linear layers. ACDC is,
however, a stand-alone module and can be used in combination with any other
types of module. In our experiments, we show that it can indeed be successfully
interleaved with ReLU modules in convolutional neural networks for image
recognition. Our experiments also study critical factors in the training of
these structured modules, including initialization and depth. Finally, this
paper also provides a connection between structured linear transforms used in
deep learning and the field of Fourier optics, illustrating how ACDC could in
principle be implemented with lenses and diffractive elements.
Fast Incremental Method for Nonconvex Optimization
Authors: Sashank J. Reddi, Suvrit Sra, Barnabas Poczos, Alex Smola
Date: 2016-03-19 00:00:00

We analyze a fast incremental aggregated gradient method for optimizing
nonconvex problems of the form $\min_x \sum_i f_i(x)$. Specifically, we analyze
the SAGA algorithm within an Incremental First-order Oracle framework, and show
that it converges to a stationary point provably faster than both gradient
descent and stochastic gradient descent. We also discuss a Polyak's special
class of nonconvex problems for which SAGA converges at a linear rate to the
global optimum. Finally, we analyze the practically valuable regularized and
minibatch variants of SAGA. To our knowledge, this paper presents the first
analysis of fast convergence for an incremental aggregated gradient method for
nonconvex problems.
Evolving Shepherding Behavior with Genetic Programming Algorithms
Authors: Joshua Brulé, Kevin Engel, Nick Fung, Isaac Julien
Date: 2016-03-19 00:00:00

We apply genetic programming techniques to the `shepherding' problem, in
which a group of one type of animal (sheep dogs) attempts to control the
movements of a second group of animals (sheep) obeying flocking behavior. Our
genetic programming algorithm evolves an expression tree that governs the
movements of each dog. The operands of the tree are hand-selected features of
the simulation environment that may allow the dogs to herd the sheep
effectively. The algorithm uses tournament-style selection, crossover
reproduction, and a point mutation. We find that the evolved solutions
generalize well and outperform a (naive) human-designed algorithm.
Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural Networks
Authors: Sahil Bhatia, Rishabh Singh
Date: 2016-03-19 00:00:00
Tasks: valid

We present a method for automatically generating repair feedback for syntax
errors for introductory programming problems. Syntax errors constitute one of
the largest classes of errors (34%) in our dataset of student submissions
obtained from a MOOC course on edX. The previous techniques for generating
automated feed- back on programming assignments have focused on functional
correctness and style considerations of student programs. These techniques
analyze the program AST of the program and then perform some dynamic and
symbolic analyses to compute repair feedback. Unfortunately, it is not possible
to generate ASTs for student pro- grams with syntax errors and therefore the
previous feedback techniques are not applicable in repairing syntax errors.
  We present a technique for providing feedback on syntax errors that uses
Recurrent neural networks (RNNs) to model syntactically valid token sequences.
Our approach is inspired from the recent work on learning language models from
Big Code (large code corpus). For a given programming assignment, we first
learn an RNN to model all valid token sequences using the set of syntactically
correct student submissions. Then, for a student submission with syntax errors,
we query the learnt RNN model with the prefix to- ken sequence to predict token
sequences that can fix the error by either replacing or inserting the predicted
token sequence at the error location. We evaluate our technique on over 14, 000
student submissions with syntax errors. Our technique can completely re- pair
31.69% (4501/14203) of submissions with syntax errors and in addition partially
correct 6.39% (908/14203) of the submissions.
The Computational Power of Dynamic Bayesian Networks
Authors: Joshua Brulé
Date: 2016-03-19 00:00:00

This paper considers the computational power of constant size, dynamic
Bayesian networks. Although discrete dynamic Bayesian networks are no more
powerful than hidden Markov models, dynamic Bayesian networks with continuous
random variables and discrete children of continuous parents are capable of
performing Turing-complete computation. With modified versions of existing
algorithms for belief propagation, such a simulation can be carried out in real
time. This result suggests that dynamic Bayesian networks may be more powerful
than previously considered. Relationships to causal models and recurrent neural
networks are also discussed.
Buried object detection using handheld WEMI with task-driven extended functions of multiple instances
Authors: Matthew Cook, Alina Zare, Dominic Ho
Date: 2016-03-19 00:00:00
Tasks: Dictionary Learning, object-detection, Object Detection

Many effective supervised discriminative dictionary learning methods have
been developed in the literature. However, when training these algorithms,
precise ground-truth of the training data is required to provide very accurate
point-wise labels. Yet, in many applications, accurate labels are not always
feasible. This is especially true in the case of buried object detection in
which the size of the objects are not consistent. In this paper, a new multiple
instance dictionary learning algorithm for detecting buried objects using a
handheld WEMI sensor is detailed. The new algorithm, Task Driven Extended
Functions of Multiple Instances, can overcome data that does not have very
precise point-wise labels and still learn a highly discriminative dictionary.
Results are presented and discussed on measured WEMI data.
You-Do, I-Learn: Unsupervised Multi-User egocentric Approach Towards Video-Based Guidance
Authors: Dima Damen, Teesid Leelasawassuk, Walterio Mayol-Cuevas
Date: 2015-10-16 00:00:00
Tasks: Object

This paper presents an unsupervised approach towards automatically extracting
video-based guidance on object usage, from egocentric video and wearable gaze
tracking, collected from multiple users while performing tasks. The approach i)
discovers task relevant objects, ii) builds a model for each, iii)
distinguishes different ways in which each discovered object has been used and
iv) discovers the dependencies between object interactions. The work
investigates using appearance, position, motion and attention, and presents
results using each and a combination of relevant features. Moreover, an online
scalable approach is presented and is compared to offline results. The paper
proposes a method for selecting a suitable video guide to be displayed to a
novice user indicating how to use an object, purely triggered by the user's
gaze. The potential assistive mode can also recommend an object to be used next
based on the learnt sequence of object interactions. The approach was tested on
a variety of daily tasks such as initialising a printer, preparing a coffee and
setting up a gym machine.
Large scale near-duplicate image retrieval using Triples of Adjacent Ranked Features (TARF) with embedded geometric information
Authors: Sergei Fedorov, Olga Kacher
Date: 2016-03-19 00:00:00
Tasks: Image Retrieval, Retrieval

Most approaches to large-scale image retrieval are based on the construction
of the inverted index of local image descriptors or visual words. A search in
such an index usually results in a large number of candidates. This list of
candidates is then re-ranked with the help of a geometric verification, using a
RANSAC algorithm, for example. In this paper we propose a feature
representation, which is built as a combination of three local descriptors. It
allows one to significantly decrease the number of false matches and to shorten
the list of candidates after the initial search in the inverted index. This
combination of local descriptors is both reproducible and highly
discriminative, and thus can be efficiently used for large-scale near-duplicate
image retrieval.