pwc-archive/papers-with-abstracts huggingface dataset:
Learning Physical Intuition of Block Towers by Example
Authors: Adam Lerer, Sam Gross, Rob Fergus
Date: 2016-03-03 00:00:00
Tasks: Physical Intuition

Wooden blocks are a common toy for infants, allowing them to develop motor
skills and gain intuition about the physical behavior of the world. In this
paper, we explore the ability of deep feed-forward models to learn such
intuitive physics. Using a 3D game engine, we create small towers of wooden
blocks whose stability is randomized and render them collapsing (or remaining
upright). This data allows us to train large convolutional network models which
can accurately predict the outcome, as well as estimating the block
trajectories. The models are also able to generalize in two important ways: (i)
to new physical scenarios, e.g. towers with an additional block and (ii) to
images of real wooden blocks, where it obtains a performance comparable to
human subjects.
Integrated Inference and Learning of Neural Factors in Structural Support Vector Machines
Authors: Rein Houthooft, Filip De Turck
Date: 2015-08-03 00:00:00
Tasks: Image Segmentation, Semantic Segmentation, Structured Prediction

Tackling pattern recognition problems in areas such as computer vision,
bioinformatics, speech or text recognition is often done best by taking into
account task-specific statistical relations between output variables. In
structured prediction, this internal structure is used to predict multiple
outputs simultaneously, leading to more accurate and coherent predictions.
Structural support vector machines (SSVMs) are nonprobabilistic models that
optimize a joint input-output function through margin-based learning. Because
SSVMs generally disregard the interplay between unary and interaction factors
during the training phase, final parameters are suboptimal. Moreover, its
factors are often restricted to linear combinations of input features, limiting
its generalization power. To improve prediction accuracy, this paper proposes:
(i) Joint inference and learning by integration of back-propagation and
loss-augmented inference in SSVM subgradient descent; (ii) Extending SSVM
factors to neural networks that form highly nonlinear functions of input
features. Image segmentation benchmark results demonstrate improvements over
conventional SSVM training methods in terms of accuracy, highlighting the
feasibility of end-to-end SSVM training with neural factors.
Decision Forests, Convolutional Networks and the Models in-Between
Authors: Yani Ioannou, Duncan Robertson, Darko Zikic, Peter Kontschieder, Jamie Shotton, Matthew Brown, Antonio Criminisi
Date: 2016-03-03 00:00:00
Tasks: image-classification, Image Classification, Representation Learning

This paper investigates the connections between two state of the art
classifiers: decision forests (DFs, including decision jungles) and
convolutional neural networks (CNNs). Decision forests are computationally
efficient thanks to their conditional computation property (computation is
confined to only a small region of the tree, the nodes along a single branch).
CNNs achieve state of the art accuracy, thanks to their representation learning
capabilities. We present a systematic analysis of how to fuse conditional
computation with representation learning and achieve a continuum of hybrid
models with different ratios of accuracy vs. efficiency. We call this new
family of hybrid models conditional networks. Conditional networks can be
thought of as: i) decision trees augmented with data transformation operators,
or ii) CNNs, with block-diagonal sparse weight matrices, and explicit data
routing functions. Experimental validation is performed on the common task of
image classification on both the CIFAR and Imagenet datasets. Compared to state
of the art CNNs, our hybrid models yield the same accuracy with a fraction of
the compute cost and much smaller number of parameters.
Multilingual Relation Extraction using Compositional Universal Schema
Authors: Patrick Verga, David Belanger, Emma Strubell, Benjamin Roth, Andrew McCallum
Date: 2015-11-19 00:00:00
Proceeding: NAACL 2016 6
Tasks: Knowledge Base Construction, Relation, Relation Extraction, slot-filling, Slot Filling, Transfer Learning, Word Embeddings

Universal schema builds a knowledge base (KB) of entities and relations by
jointly embedding all relation types from input KBs as well as textual patterns
expressing relations from raw text. In most previous applications of universal
schema, each textual pattern is represented as a single embedding, preventing
generalization to unseen patterns. Recent work employs a neural network to
capture patterns' compositional semantics, providing generalization to all
possible input text. In response, this paper introduces significant further
improvements to the coverage and flexibility of universal schema relation
extraction: predictions for entities unseen in training and multilingual
transfer learning to domains with no annotation. We evaluate our model through
extensive experiments on the English and Spanish TAC KBP benchmark,
outperforming the top system from TAC 2013 slot-filling using no handwritten
patterns or additional annotation. We also consider a multilingual setting in
which English training data entities overlap with the seed KB, but Spanish text
does not. Despite having no annotation for Spanish data, we train an accurate
predictor, with additional improvements obtained by tying word embeddings
across languages. Furthermore, we find that multilingual training improves
English relation extraction accuracy. Our approach is thus suited to
broad-coverage automated knowledge base construction in a variety of languages
and domains.
Convex Relaxation Regression: Black-Box Optimization of Smooth Functions by Learning Their Convex Envelopes
Authors: Mohammad Gheshlaghi Azar, Eva Dyer, Konrad Kording
Date: 2016-02-05 00:00:00
Tasks: regression

Finding efficient and provable methods to solve non-convex optimization
problems is an outstanding challenge in machine learning and optimization
theory. A popular approach used to tackle non-convex problems is to use convex
relaxation techniques to find a convex surrogate for the problem.
Unfortunately, convex relaxations typically must be found on a
problem-by-problem basis. Thus, providing a general-purpose strategy to
estimate a convex relaxation would have a wide reaching impact. Here, we
introduce Convex Relaxation Regression (CoRR), an approach for learning convex
relaxations for a class of smooth functions. The main idea behind our approach
is to estimate the convex envelope of a function $f$ by evaluating $f$ at a set
of $T$ random points and then fitting a convex function to these function
evaluations. We prove that with probability greater than $1-\delta$, the
solution of our algorithm converges to the global optimizer of $f$ with error
$\mathcal{O} \Big( \big(\frac{\log(1/\delta) }{T} \big)^{\alpha} \Big)$ for
some $\alpha> 0$. Our approach enables the use of convex optimization tools to
solve a class of non-convex optimization problems.
Automated quantification of one-dimensional nanostructure alignment on surfaces
Authors: Jianjin Dong, Irene A. Goldthorpe, Nasser Mohieddin Abukhdeir
Date: 2016-03-03 00:00:00

A method for automated quantification of the alignment of one-dimensional
nanostructures from microscopy imaging is presented. Nanostructure alignment
metrics are formulated and shown to able to rigorously quantify the
orientational order of nanostructures within a two-dimensional domain
(surface). A complementary image processing method is also presented which
enables robust processing of microscopy images where overlapping nanostructures
might be present. Scanning electron microscopy (SEM) images of nanowire-covered
surfaces are analyzed using the presented methods and it is shown that past
single parameter alignment metrics are insufficient for highly aligned domains.
Through the use of multiple parameter alignment metrics, automated quantitative
analysis of SEM images is shown to be possible and the alignment
characteristics of different samples are able to be rigorously compared using a
similarity metric. The results of this work provide researchers in nanoscience
and nanotechnology with a rigorous method for the determination of
structure/property relationships where alignment of one-dimensional
nanostructures is significant.
Multi-domain Neural Network Language Generation for Spoken Dialogue Systems
Authors: Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Lina M. Rojas-Barahona, Pei-Hao Su, David Vandyke, Steve Young
Date: 2016-03-03 00:00:00
Proceeding: NAACL 2016 6
Tasks: Domain Adaptation, Spoken Dialogue Systems, Text Generation

Moving from limited-domain natural language generation (NLG) to open domain
is difficult because the number of semantic input combinations grows
exponentially with the number of domains. Therefore, it is important to
leverage existing resources and exploit similarities between domains to
facilitate domain adaptation. In this paper, we propose a procedure to train
multi-domain, Recurrent Neural Network-based (RNN) language generators via
multiple adaptation steps. In this procedure, a model is first trained on
counterfeited data synthesised from an out-of-domain dataset, and then fine
tuned on a small set of in-domain utterances with a discriminative objective
function. Corpus-based evaluation results show that the proposed procedure can
achieve competitive performance in terms of BLEU score and slot error rate
while significantly reducing the data needed to train generators in new, unseen
domains. In subjective testing, human judges confirm that the procedure greatly
improves generator performance when only a small amount of data is available in
the domain.
GeoGebra Tools with Proof Capabilities
Authors: Zoltán Kovács, Csilla Sólyom-Gecse
Date: 2016-03-03 00:00:00
Tasks: Automated Theorem Proving, Benchmarking

We report about significant enhancements of the complex algebraic geometry
theorem proving subsystem in GeoGebra for automated proofs in Euclidean
geometry, concerning the extension of numerous GeoGebra tools with proof
capabilities. As a result, a number of elementary theorems can be proven by
using GeoGebra's intuitive user interface on various computer architectures
including native Java and web based systems with JavaScript. We also provide a
test suite for benchmarking our results with 200 test cases.
Comparative Design Space Exploration of Dense and Semi-Dense SLAM
Authors: M. Zeeshan Zia, Luigi Nardi, Andrew Jack, Emanuele Vespa, Bruno Bodin, Paul H. J. Kelly, Andrew J. Davison
Date: 2015-09-15 00:00:00
Tasks: Benchmarking

SLAM has matured significantly over the past few years, and is beginning to
appear in serious commercial products. While new SLAM systems are being
proposed at every conference, evaluation is often restricted to qualitative
visualizations or accuracy estimation against a ground truth. This is due to
the lack of benchmarking methodologies which can holistically and
quantitatively evaluate these systems. Further investigation at the level of
individual kernels and parameter spaces of SLAM pipelines is non-existent,
which is absolutely essential for systems research and integration. We extend
the recently introduced SLAMBench framework to allow comparing two
state-of-the-art SLAM pipelines, namely KinectFusion and LSD-SLAM, along the
metrics of accuracy, energy consumption, and processing frame rate on two
different hardware platforms, namely a desktop and an embedded device. We also
analyze the pipelines at the level of individual kernels and explore their
algorithmic and hardware design spaces for the first time, yielding valuable
insights.
Rank Aggregation for Course Sequence Discovery
Authors: Mihai Cucuringu, Charlie Marshak, Dillon Montag, Puck Rombach
Date: 2016-03-03 00:00:00

In this work, we adapt the rank aggregation framework for the discovery of
optimal course sequences at the university level. Each student provides a
partial ranking of the courses taken throughout his or her undergraduate
career. We compute pairwise rank comparisons between courses based on the order
students typically take them, aggregate the results over the entire student
population, and then obtain a proxy for the rank offset between pairs of
courses. We extract a global ranking of the courses via several state-of-the
art algorithms for ranking with pairwise noisy information, including
SerialRank, Rank Centrality, and the recent SyncRank based on the group
synchronization problem. We test this application of rank aggregation on 15
years of student data from the Department of Mathematics at the University of
California, Los Angeles (UCLA). Furthermore, we experiment with the above
approach on different subsets of the student population conditioned on final
GPA, and highlight several differences in the obtained rankings that uncover
hidden pre-requisites in the Mathematics curriculum.
Overdispersed Black-Box Variational Inference
Authors: Francisco J. R. Ruiz, Michalis K. Titsias, David M. Blei
Date: 2016-03-03 00:00:00
Tasks: Variational Inference

We introduce overdispersed black-box variational inference, a method to
reduce the variance of the Monte Carlo estimator of the gradient in black-box
variational inference. Instead of taking samples from the variational
distribution, we use importance sampling to take samples from an overdispersed
distribution in the same exponential family as the variational approximation.
Our approach is general since it can be readily applied to any exponential
family distribution, which is the typical choice for the variational
approximation. We run experiments on two non-conjugate probabilistic models to
show that our method effectively reduces the variance, and the overhead
introduced by the computation of the proposal parameters and the importance
weights is negligible. We find that our overdispersed importance sampling
scheme provides lower variance than black-box variational inference, even when
the latter uses twice the number of samples. This results in faster convergence
of the black-box inference procedure.
Elastic Net Hypergraph Learning for Image Clustering and Semi-supervised Classification
Authors: Qingshan Liu, Yubao Sun, Cantian Wang, Tongliang Liu, DaCheng Tao
Date: 2016-03-03 00:00:00
Tasks: Clustering, General Classification, graph construction, Image Clustering, Variable Selection

Graph model is emerging as a very effective tool for learning the complex
structures and relationships hidden in data. Generally, the critical purpose of
graph-oriented learning algorithms is to construct an informative graph for
image clustering and classification tasks. In addition to the classical
$K$-nearest-neighbor and $r$-neighborhood methods for graph construction,
$l_1$-graph and its variants are emerging methods for finding the neighboring
samples of a center datum, where the corresponding ingoing edge weights are
simultaneously derived by the sparse reconstruction coefficients of the
remaining samples. However, the pair-wise links of $l_1$-graph are not capable
of capturing the high order relationships between the center datum and its
prominent data in sparse reconstruction. Meanwhile, from the perspective of
variable selection, the $l_1$ norm sparse constraint, regarded as a LASSO
model, tends to select only one datum from a group of data that are highly
correlated and ignore the others. To simultaneously cope with these drawbacks,
we propose a new elastic net hypergraph learning model, which consists of two
steps. In the first step, the Robust Matrix Elastic Net model is constructed to
find the canonically related samples in a somewhat greedy way, achieving the
grouping effect by adding the $l_2$ penalty to the $l_1$ constraint. In the
second step, hypergraph is used to represent the high order relationships
between each datum and its prominent samples by regarding them as a hyperedge.
Subsequently, hypergraph Laplacian matrix is constructed for further analysis.
New hypergraph learning algorithms, including unsupervised clustering and
multi-class semi-supervised classification, are then derived. Extensive
experiments on face and handwriting databases demonstrate the effectiveness of
the proposed method.
Modeling the Sequence of Brain Volumes by Local Mesh Models for Brain Decoding
Authors: Itir Onal, Mete Ozay, Eda Mizrak, Ilke Oztekin, Fatos T. Yarman Vural
Date: 2016-03-03 00:00:00
Tasks: Brain Decoding, Object Recognition, Retrieval, Time Series, Time Series Analysis

We represent the sequence of fMRI (Functional Magnetic Resonance Imaging)
brain volumes recorded during a cognitive stimulus by a graph which consists of
a set of local meshes. The corresponding cognitive process, encoded in the
brain, is then represented by these meshes each of which is estimated assuming
a linear relationship among the voxel time series in a predefined locality.
First, we define the concept of locality in two neighborhood systems, namely,
the spatial and functional neighborhoods. Then, we construct spatially and
functionally local meshes around each voxel, called seed voxel, by connecting
it either to its spatial or functional p-nearest neighbors. The mesh formed
around a voxel is a directed sub-graph with a star topology, where the
direction of the edges is taken towards the seed voxel at the center of the
mesh. We represent the time series recorded at each seed voxel in terms of
linear combination of the time series of its p-nearest neighbors in the mesh.
The relationships between a seed voxel and its neighbors are represented by the
edge weights of each mesh, and are estimated by solving a linear regression
equation. The estimated mesh edge weights lead to a better representation of
information in the brain for encoding and decoding of the cognitive tasks. We
test our model on a visual object recognition and emotional memory retrieval
experiments using Support Vector Machines that are trained using the mesh edge
weights as features. In the experimental analysis, we observe that the edge
weights of the spatial and functional meshes perform better than the
state-of-the-art brain decoding models.
Knowledge Base Population using Semantic Label Propagation
Authors: Lucas Sterckx, Thomas Demeester, Johannes Deleu, Chris Develder
Date: 2015-11-19 00:00:00
Tasks: Knowledge Base Population, Relation

A crucial aspect of a knowledge base population system that extracts new
facts from text corpora, is the generation of training data for its relation
extractors. In this paper, we present a method that maximizes the effectiveness
of newly trained relation extractors at a minimal annotation cost. Manual
labeling can be significantly reduced by Distant Supervision, which is a method
to construct training data automatically by aligning a large text corpus with
an existing knowledge base of known facts. For example, all sentences
mentioning both 'Barack Obama' and 'US' may serve as positive training
instances for the relation born_in(subject,object). However, distant
supervision typically results in a highly noisy training set: many training
sentences do not really express the intended relation. We propose to combine
distant supervision with minimal manual supervision in a technique called
feature labeling, to eliminate noise from the large and noisy initial training
set, resulting in a significant increase of precision. We further improve on
this approach by introducing the Semantic Label Propagation method, which uses
the similarity between low-dimensional representations of candidate training
instances, to extend the training set in order to increase recall while
maintaining high precision. Our proposed strategy for generating training data
is studied and evaluated on an established test collection designed for
knowledge base population tasks. The experimental results show that the
Semantic Label Propagation strategy leads to substantial performance gains when
compared to existing approaches, while requiring an almost negligible manual
annotation effort.
A novel and automatic pectoral muscle identification algorithm for mediolateral oblique (MLO) view mammograms using ImageJ
Authors: Chao Wang
Date: 2016-03-03 00:00:00

Pectoral muscle identification is often required for breast cancer risk
analysis, such as estimating breast density. Traditional methods are
overwhelmingly based on manual visual assessment or straight line fitting for
the pectoral muscle boundary, which are inefficient and inaccurate since
pectoral muscle in mammograms can have curved boundaries.
  This paper proposes a novel and automatic pectoral muscle identification
algorithm for MLO view mammograms. It is suitable for both scanned film and
full field digital mammograms. This algorithm is demonstrated using a public
domain software ImageJ. A validation of this algorithm has been performed using
real-world data and it shows promising result.
Photographic dataset: random peppercorns
Authors: Teemu Helenius, Samuli Siltanen
Date: 2016-03-03 00:00:00

This is a photographic dataset collected for testing image processing
algorithms. The idea is to have sets of different but statistically similar
images. In this work the images show randomly distributed peppercorns. The
dataset is made available at www.fips.fi/photographic_dataset.php .
Self-localization from Images with Small Overlap
Authors: Tanaka Kanji
Date: 2016-03-03 00:00:00
Tasks: Retrieval

With the recent success of visual features from deep convolutional neural
networks (DCNN) in visual robot self-localization, it has become important and
practical to address more general self-localization scenarios. In this paper,
we address the scenario of self-localization from images with small overlap. We
explicitly introduce a localization difficulty index as a decreasing function
of view overlap between query and relevant database images and investigate
performance versus difficulty for challenging cross-view self-localization
tasks. We then reformulate the self-localization as a scalable
bag-of-visual-features (BoVF) scene retrieval and present an efficient solution
called PCA-NBNN, aiming to facilitate fast and yet discriminative
correspondence between partially overlapping images. The proposed approach
adopts recent findings in discriminativity preserving encoding of DCNN features
using principal component analysis (PCA) and cross-domain scene matching using
naive Bayes nearest neighbor distance metric (NBNN). We experimentally
demonstrate that the proposed PCA-NBNN framework frequently achieves comparable
results to previous DCNN features and that the BoVF model is significantly more
efficient. We further address an important alternative scenario of
"self-localization from images with NO overlap" and report the result.