pwc-archive/papers-with-abstracts huggingface dataset:
Joint Training of Generic CNN-CRF Models with Stochastic Optimization
Authors: Alexander Kirillov, Dmitrij Schlesinger, Shuai Zheng, Bogdan Savchynskyy, Philip H. S. Torr, Carsten Rother
Date: 2015-11-16 00:00:00
Tasks: Stochastic Optimization

We propose a new CNN-CRF end-to-end learning framework, which is based on
joint stochastic optimization with respect to both Convolutional Neural Network
(CNN) and Conditional Random Field (CRF) parameters. While stochastic gradient
descent is a standard technique for CNN training, it was not used for joint
models so far. We show that our learning method is (i) general, i.e. it applies
to arbitrary CNN and CRF architectures and potential functions; (ii) scalable,
i.e. it has a low memory footprint and straightforwardly parallelizes on GPUs;
(iii) easy in implementation. Additionally, the unified CNN-CRF optimization
approach simplifies a potential hardware implementation. We empirically
evaluate our method on the task of semantic labeling of body parts in depth
images and show that it compares favorably to competing techniques.
Fully-Convolutional Siamese Networks for Object Tracking
Authors: Luca Bertinetto, Jack Valmadre, João F. Henriques, Andrea Vedaldi, Philip H. S. Torr
Date: 2016-06-30 00:00:00
Tasks: Object, object-detection, Object Detection, Object Tracking

The problem of arbitrary object tracking has traditionally been tackled by learning a model of the object's appearance exclusively online, using as sole training data the video itself. Despite the success of these methods, their online-only approach inherently limits the richness of the model they can learn. Recently, several attempts have been made to exploit the expressive power of deep convolutional networks. However, when the object to track is not known beforehand, it is necessary to perform Stochastic Gradient Descent online to adapt the weights of the network, severely compromising the speed of the system. In this paper we equip a basic tracking algorithm with a novel fully-convolutional Siamese network trained end-to-end on the ILSVRC15 dataset for object detection in video. Our tracker operates at frame-rates beyond real-time and, despite its extreme simplicity, achieves state-of-the-art performance in multiple benchmarks.
Distributed Estimation of the Operating State of a Single-Bus DC MicroGrid without an External Communication Interface
Authors: Marko Angjelichinoski, Anna Scaglione, Petar Popovski, Cedomir Stefanovic
Date: 2016-09-14 00:00:00

We propose a decentralized Maximum Likelihood solution for estimating the
stochastic renewable power generation and demand in single bus Direct Current
(DC) MicroGrids (MGs), with high penetration of droop controlled power
electronic converters. The solution relies on the fact that the primary control
parameters are set in accordance with the local power generation status of the
generators. Therefore, the steady state voltage is inherently dependent on the
generation capacities and the load, through a non-linear parametric model,
which can be estimated. To have a well conditioned estimation problem, our
solution avoids the use of an external communication interface and utilizes
controlled voltage disturbances to perform distributed training. Using this
tool, we develop an efficient, decentralized Maximum Likelihood Estimator (MLE)
and formulate the sufficient condition for the existence of the globally
optimal solution. The numerical results illustrate the promising performance of
our MLE algorithm.
Sequencing Chess
Authors: A. Atashpendar, T. Schilling, Th. Voigtmann
Date: 2016-09-14 00:00:00

We analyze the structure of the state space of chess by means of transition
path sampling Monte Carlo simulation. Based on the typical number of moves
required to transpose a given configuration of chess pieces into another, we
conclude that the state space consists of several pockets between which
transitions are rare. Skilled players explore an even smaller subset of
positions that populate some of these pockets only very sparsely. These results
suggest that the usual measures to estimate both, the size of the state space
and the size of the tree of legal moves, are not unique indicators of the
complexity of the game, but that topological considerations are equally
important.
Neural Machine Translation with Supervised Attention
Authors: Lemao Liu, Masao Utiyama, Andrew Finch, Eiichiro Sumita
Date: 2016-09-14 00:00:00
Proceeding: COLING 2016 12
Tasks: Machine Translation, NMT, Sentence, Translation

The attention mechanisim is appealing for neural machine translation, since
it is able to dynam- ically encode a source sentence by generating a alignment
between a target word and source words. Unfortunately, it has been proved to be
worse than conventional alignment models in aligment accuracy. In this paper,
we analyze and explain this issue from the point view of re- ordering, and
propose a supervised attention which is learned with guidance from conventional
alignment models. Experiments on two Chinese-to-English translation tasks show
that the super- vised attention mechanism yields better alignments leading to
substantial gains over the standard attention based NMT.
Proceedings of the third "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'16)
Authors: V. Abrol, O. Absil, P. -A. Absil, S. Anthoine, P. Antoine, T. Arildsen, N. Bertin, F. Bleichrodt, J. Bobin, A. Bol, A. Bonnefoy, F. Caltagirone, V. Cambareri, C. Chenot, V. Crnojević, M. Daňková, K. Degraux, J. Eisert, J. M. Fadili, M. Gabrié, N. Gac, D. Giacobello, A. Gonzalez, C. A. Gomez Gonzalez, A. González, P. -Y. Gousenbourger, M. Græsbøll Christensen, R. Gribonval, S. Guérit, S. Huang, P. Irofti, L. Jacques, U. S. Kamilov, S. Kiticć, M. Kliesch, F. Krzakala, J. A. Lee, W. Liao, T. Lindstrøm Jensen, A. Manoel, H. Mansour, A. Mohammad-Djafari, A. Moshtaghpour, F. Ngolè, B. Pairet, M. Panić, G. Peyré, A. Pižurica, P. Rajmic, M. Roblin, I. Roth, A. K. Sao, P. Sharma, J. -L. Starck, E. W. Tramel, T. van Waterschoot, D. Vukobratovic, L. Wang, B. Wirth, G. Wunder, H. Zhang
Date: 2016-09-14 00:00:00
Tasks: Astronomy, Dictionary Learning, Retrieval

The third edition of the "international - Traveling Workshop on Interactions
between Sparse models and Technology" (iTWIST) took place in Aalborg, the 4th
largest city in Denmark situated beautifully in the northern part of the
country, from the 24th to 26th of August 2016. The workshop venue was at the
Aalborg University campus. One implicit objective of this biennial workshop is
to foster collaboration between international scientific teams by disseminating
ideas through both specific oral/poster presentations and free discussions. For
this third edition, iTWIST'16 gathered about 50 international participants and
features 8 invited talks, 12 oral presentations, and 12 posters on the
following themes, all related to the theory, application and generalization of
the "sparsity paradigm": Sparsity-driven data sensing and processing (e.g.,
optics, computer vision, genomics, biomedical, digital communication, channel
estimation, astronomy); Application of sparse models in non-convex/non-linear
inverse problems (e.g., phase retrieval, blind deconvolution, self
calibration); Approximate probabilistic inference for sparse problems; Sparse
machine learning and inference; "Blind" inverse problems and dictionary
learning; Optimization for sparse modelling; Information theory, geometry and
randomness; Sparsity? What's next? (Discrete-valued signals; Union of
low-dimensional spaces, Cosparsity, mixed/group norm, model-based,
low-complexity models, ...); Matrix/manifold sensing/processing (graph,
low-rank approximation, ...); Complexity/accuracy tradeoffs in numerical
methods/optimization; Electronic/optical compressive sensors (hardware).
The Structured Weighted Violations Perceptron Algorithm
Authors: Rotem Dror, Roi Reichart
Date: 2016-02-09 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Dependency Parsing, Generalization Bounds, Structured Prediction

We present the Structured Weighted Violations Perceptron (SWVP) algorithm, a
new structured prediction algorithm that generalizes the Collins Structured
Perceptron (CSP). Unlike CSP, the update rule of SWVP explicitly exploits the
internal structure of the predicted labels. We prove the convergence of SWVP
for linearly separable training sets, provide mistake and generalization
bounds, and show that in the general case these bounds are tighter than those
of the CSP special case. In synthetic data experiments with data drawn from an
HMM, various variants of SWVP substantially outperform its CSP special case.
SWVP also provides encouraging initial dependency parsing results.
Joint Gender Classification and Age Estimation by Nearly Orthogonalizing Their Semantic Spaces
Authors: Qing Tian, Songcan Chen
Date: 2016-09-14 00:00:00
Tasks: Age Estimation, Gender Classification, General Classification, MORPH, regression

In human face-based biometrics, gender classification and age estimation are
two typical learning tasks. Although a variety of approaches have been proposed
to handle them, just a few of them are solved jointly, even so, these joint
methods do not yet specifically concern the semantic difference between human
gender and age, which is intuitively helpful for joint learning, consequently
leaving us a room of further improving the performance. To this end, in this
work we firstly propose a general learning framework for jointly estimating
human gender and age by specially attempting to formulate such semantic
relationships as a form of near-orthogonality regularization and then
incorporate it into the objective of the joint learning framework. In order to
evaluate the effectiveness of the proposed framework, we exemplify it by
respectively taking the widely used binary-class SVM for gender classification,
and two threshold-based ordinal regression methods (i.e., the discriminant
learning for ordinal regression and support vector ordinal regression) for age
estimation, and crucially coupling both through the proposed semantic
formulation. Moreover, we develop its kernelized nonlinear counterpart by
deriving a representer theorem for the joint learning strategy. Finally,
through extensive experiments on three aging datasets FG-NET, Morph Album I and
Morph Album II, we demonstrate the effectiveness and superiority of the
proposed joint learning strategy.
Tracking Tensor Subspaces with Informative Random Sampling for Real-Time MR Imaging
Authors: Morteza Mardani, Georgios B. Giannakis, Kamil Ugurbil
Date: 2016-09-14 00:00:00
Tasks: Diagnostic, GPU

Magnetic resonance imaging (MRI) nowadays serves as an important modality for
diagnostic and therapeutic guidance in clinics. However, the {\it slow
acquisition} process, the dynamic deformation of organs, as well as the need
for {\it real-time} reconstruction, pose major challenges toward obtaining
artifact-free images. To cope with these challenges, the present paper
advocates a novel subspace learning framework that permeates benefits from
parallel factor (PARAFAC) decomposition of tensors (multiway data) to low-rank
modeling of temporal sequence of images. Treating images as multiway data
arrays, the novel method preserves spatial structures and unravels the latent
correlations across various dimensions by means of the tensor subspace.
Leveraging the spatio-temporal correlation of images, Tykhonov regularization
is adopted as a rank surrogate for a least-squares optimization program.
Alteranating majorization minimization is adopted to develop online algorithms
that recursively procure the reconstruction upon arrival of a new undersampled
$k$-space frame. The developed algorithms are {\it provably convergent} and
highly {\it parallelizable} with lightweight FFT tasks per iteration. To
further accelerate the acquisition process, randomized subsampling policies are
devised that leverage intermediate estimates of the tensor subspace, offered by
the online scheme, to {\it randomly} acquire {\it informative} $k$-space
samples. In a nutshell, the novel approach enables tracking motion dynamics
under low acquisition rates `on the fly.' GPU-based tests with real {\it in
vivo} MRI datasets of cardiac cine images corroborate the merits of the novel
approach relative to state-of-the-art alternatives.
The CUDA LATCH Binary Descriptor: Because Sometimes Faster Means Better
Authors: Christopher Parker, Matthew Daiter, Kareem Omar, Gil Levi, Tal Hassner
Date: 2016-09-13 00:00:00
Tasks: GPU

Accuracy, descriptor size, and the time required for extraction and matching
are all important factors when selecting local image descriptors. To optimize
over all these requirements, this paper presents a CUDA port for the recent
Learned Arrangement of Three Patches (LATCH) binary descriptors to the GPU
platform. The design of LATCH makes it well suited for GPU processing. Owing to
its small size and binary nature, the GPU can further be used to efficiently
match LATCH features. Taken together, this leads to breakneck descriptor
extraction and matching speeds. We evaluate the trade off between these speeds
and the quality of results in a feature matching intensive application. To this
end, we use our proposed CUDA LATCH (CLATCH) to recover structure from motion
(SfM), comparing 3D reconstructions and speed using different representations.
Our results show that CLATCH provides high quality 3D reconstructions at
fractions of the time required by other representations, with little, if any,
loss of reconstruction quality.
Single-image RGB Photometric Stereo With Spatially-varying Albedo
Authors: Ayan Chakrabarti, Kalyan Sunkavalli
Date: 2016-09-14 00:00:00

We present a single-shot system to recover surface geometry of objects with
spatially-varying albedos, from images captured under a calibrated RGB
photometric stereo setup---with three light directions multiplexed across
different color channels in the observed RGB image. Since the problem is
ill-posed point-wise, we assume that the albedo map can be modeled as
piece-wise constant with a restricted number of distinct albedo values. We show
that under ideal conditions, the shape of a non-degenerate local constant
albedo surface patch can theoretically be recovered exactly. Moreover, we
present a practical and efficient algorithm that uses this model to robustly
recover shape from real images. Our method first reasons about shape locally in
a dense set of patches in the observed image, producing shape distributions for
every patch. These local distributions are then combined to produce a single
consistent surface normal map. We demonstrate the efficacy of the approach
through experiments on both synthetic renderings as well as real captured
images.
A Generic Bet-and-run Strategy for Speeding Up Traveling Salesperson and Minimum Vertex Cover
Authors: Tobias Friedrich, Timo Kötzing, Markus Wagner
Date: 2016-09-13 00:00:00
Tasks: Combinatorial Optimization

A common strategy for improving optimization algorithms is to restart the
algorithm when it is believed to be trapped in an inferior part of the search
space. However, while specific restart strategies have been developed for
specific problems (and specific algorithms), restarts are typically not
regarded as a general tool to speed up an optimization algorithm. In fact, many
optimization algorithms do not employ restarts at all.
  Recently, "bet-and-run" was introduced in the context of mixed-integer
programming, where first a number of short runs with randomized initial
conditions is made, and then the most promising run of these is continued. In
this article, we consider two classical NP-complete combinatorial optimization
problems, traveling salesperson and minimum vertex cover, and study the
effectiveness of different bet-and-run strategies. In particular, our restart
strategies do not take any problem knowledge into account, nor are tailored to
the optimization algorithm. Therefore, they can be used off-the-shelf. We
observe that state-of-the-art solvers for these problems can benefit
significantly from restarts on standard benchmark instances.
Multimodal Attention for Neural Machine Translation
Authors: Ozan Caglayan, Loïc Barrault, Fethi Bougares
Date: 2016-09-13 00:00:00
Tasks: Image Captioning, Machine Translation, NMT, Translation

The attention mechanism is an important part of the neural machine
translation (NMT) where it was reported to produce richer source representation
compared to fixed-length encoding sequence-to-sequence models. Recently, the
effectiveness of attention has also been explored in the context of image
captioning. In this work, we assess the feasibility of a multimodal attention
mechanism that simultaneously focus over an image and its natural language
description for generating a description in another language. We train several
variants of our proposed attention mechanism on the Multi30k multilingual image
captioning dataset. We show that a dedicated attention for each modality
achieves up to 1.6 points in BLEU and METEOR compared to a textual NMT
baseline.
Feynman Machine: The Universal Dynamical Systems Computer
Authors: Eric Laukien, Richard Crowder, Fergal Byrne
Date: 2016-09-13 00:00:00

Efforts at understanding the computational processes in the brain have met
with limited success, despite their importance and potential uses in building
intelligent machines. We propose a simple new model which draws on recent
findings in Neuroscience and the Applied Mathematics of interacting Dynamical
Systems. The Feynman Machine is a Universal Computer for Dynamical Systems,
analogous to the Turing Machine for symbolic computing, but with several
important differences. We demonstrate that networks and hierarchies of simple
interacting Dynamical Systems, each adaptively learning to forecast its
evolution, are capable of automatically building sensorimotor models of the
external and internal world. We identify such networks in mammalian neocortex,
and show how existing theories of cortical computation combine with our model
to explain the power and flexibility of mammalian intelligence. These findings
lead directly to new architectures for machine intelligence. A suite of
software implementations has been built based on these principles, and applied
to a number of spatiotemporal learning tasks.
Low-Rank Approximation and Completion of Positive Tensors
Authors: Anil Aswani
Date: 2014-12-01 00:00:00
Tasks: Tensor Decomposition

Unlike the matrix case, computing low-rank approximations of tensors is
NP-hard and numerically ill-posed in general. Even the best rank-1
approximation of a tensor is NP-hard. In this paper, we use convex optimization
to develop polynomial-time algorithms for low-rank approximation and completion
of positive tensors. Our approach is to use algebraic topology to define a new
(numerically well-posed) decomposition for positive tensors, which we show is
equivalent to the standard tensor decomposition in important cases. Though
computing this decomposition is a nonconvex optimization problem, we prove it
can be exactly reformulated as a convex optimization problem. This allows us to
construct polynomial-time randomized algorithms for computing this
decomposition and for solving low-rank tensor approximation problems. Among the
consequences is that best rank-1 approximations of positive tensors can be
computed in polynomial time. Our framework is next extended to the tensor
completion problem, where noisy entries of a tensor are observed and then used
to estimate missing entries. We provide a polynomial-time algorithm that for
specific cases requires a polynomial (in tensor order) number of measurements,
in contrast to existing approaches that require an exponential number of
measurements. These algorithms are extended to exploit sparsity in the tensor
to reduce the number of measurements needed. We conclude by providing a novel
interpretation of statistical regression problems with categorical variables as
tensor completion problems, and numerical examples with synthetic data and data
from a bioengineered metabolic network show the improved performance of our
approach on this problem.
Self-Sustaining Iterated Learning
Authors: Bernard Chazelle, Chu Wang
Date: 2016-09-13 00:00:00
Tasks: regression

An important result from psycholinguistics (Griffiths & Kalish, 2005) states
that no language can be learned iteratively by rational agents in a
self-sustaining manner. We show how to modify the learning process slightly in
order to achieve self-sustainability. Our work is in two parts. First, we
characterize iterated learnability in geometric terms and show how a slight,
steady increase in the lengths of the training sessions ensures
self-sustainability for any discrete language class. In the second part, we
tackle the nondiscrete case and investigate self-sustainability for iterated
linear regression. We discuss the implications of our findings to issues of
non-equilibrium dynamics in natural algorithms.
Noisy Inductive Matrix Completion Under Sparse Factor Models
Authors: Akshay Soni, Troy Chevalier, Swayambhoo Jain
Date: 2016-09-13 00:00:00
Tasks: Dictionary Learning, Matrix Completion, Recommendation Systems

Inductive Matrix Completion (IMC) is an important class of matrix completion
problems that allows direct inclusion of available features to enhance
estimation capabilities. These models have found applications in personalized
recommendation systems, multilabel learning, dictionary learning, etc. This
paper examines a general class of noisy matrix completion tasks where the
underlying matrix is following an IMC model i.e., it is formed by a mixing
matrix (a priori unknown) sandwiched between two known feature matrices. The
mixing matrix here is assumed to be well approximated by the product of two
sparse matrices---referred here to as "sparse factor models." We leverage the
main theorem of Soni:2016:NMC and extend it to provide theoretical error bounds
for the sparsity-regularized maximum likelihood estimators for the class of
problems discussed in this paper. The main result is general in the sense that
it can be used to derive error bounds for various noise models. In this paper,
we instantiate our main result for the case of Gaussian noise and provide
corresponding error bounds in terms of squared loss.
On Lipschitz Continuity and Smoothness of Loss Functions in Learning to Rank
Authors: Ambuj Tewari, Sougata Chaudhuri
Date: 2014-05-03 00:00:00
Tasks: Binary Classification, Learning-To-Rank

In binary classification and regression problems, it is well understood that
Lipschitz continuity and smoothness of the loss function play key roles in
governing generalization error bounds for empirical risk minimization
algorithms. In this paper, we show how these two properties affect
generalization error bounds in the learning to rank problem. The learning to
rank problem involves vector valued predictions and therefore the choice of the
norm with respect to which Lipschitz continuity and smoothness are defined
becomes crucial. Choosing the $\ell_\infty$ norm in our definition of Lipschitz
continuity allows us to improve existing bounds. Furthermore, under smoothness
assumptions, our choice enables us to prove rates that interpolate between
$1/\sqrt{n}$ and $1/n$ rates. Application of our results to ListNet, a popular
learning to rank method, gives state-of-the-art performance guarantees.