pwc-archive/papers-with-abstracts huggingface dataset:
Joint Calibration for Semantic Segmentation
Authors: Holger Caesar, Jasper Uijlings, Vittorio Ferrari
Date: 2015-07-06 00:00:00
Tasks: Segmentation, Semantic Segmentation

Semantic segmentation is the task of assigning a class-label to each pixel in
an image. We propose a region-based semantic segmentation framework which
handles both full and weak supervision, and addresses three common problems:
(1) Objects occur at multiple scales and therefore we should use regions at
multiple scales. However, these regions are overlapping which creates
conflicting class predictions at the pixel-level. (2) Class frequencies are
highly imbalanced in realistic datasets. (3) Each pixel can only be assigned to
a single class, which creates competition between classes. We address all three
problems with a joint calibration method which optimizes a multi-class loss
defined over the final pixel-level output labeling, as opposed to simply region
classification. Our method outperforms the state-of-the-art on the popular SIFT
Flow [18] dataset in both the fully and weakly supervised setting by a
considerably margin (+6% and +10%, respectively).
RCR: Robust Compound Regression for Robust Estimation of Errors-in-Variables Model
Authors: Hao Han, Wei Zhu
Date: 2015-08-12 00:00:00
Tasks: regression

The errors-in-variables (EIV) regression model, being more realistic by
accounting for measurement errors in both the dependent and the independent
variables, is widely adopted in applied sciences. The traditional EIV model
estimators, however, can be highly biased by outliers and other departures from
the underlying assumptions. In this paper, we develop a novel nonparametric
regression approach - the robust compound regression (RCR) analysis method for
the robust estimation of EIV models. We first introduce a robust and efficient
estimator called least sine squares (LSS). Taking full advantage of both the
new LSS method and the compound regression analysis method developed in our own
group, we subsequently propose the RCR approach as a generalization of those
two, which provides a robust counterpart of the entire class of the maximum
likelihood estimation (MLE) solutions of the EIV model, in a 1-1 mapping.
Technically, our approach gives users the flexibility to select from a class of
RCR estimates the optimal one with a predefined regression efficiency criterion
satisfied. Simulation studies and real-life examples are provided to illustrate
the effectiveness of the RCR approach.
Bayesian Dropout
Authors: Tue Herlau, Morten Mørup, Mikkel N. Schmidt
Date: 2015-08-12 00:00:00
Tasks: regression

Dropout has recently emerged as a powerful and simple method for training neural networks preventing co-adaptation by stochastically omitting neurons. Dropout is currently not grounded in explicit modelling assumptions which so far has precluded its adoption in Bayesian modelling. Using Bayesian entropic reasoning we show that dropout can be interpreted as optimal inference under constraints. We demonstrate this on an analytically tractable regression model providing a Bayesian interpretation of its mechanism for regularizing and preventing co-adaptation as well as its connection to other Bayesian techniques. We also discuss two general approximate techniques for applying Bayesian dropout for general models, one based on an analytical approximation and the other on stochastic variational techniques. These techniques are then applied to a Baysian logistic regression problem and are shown to improve performance as the model become more misspecified. Our framework roots dropout as a theoretically justified and practical tool for statistical modelling allowing Bayesians to tap into the benefits of dropout training.
Manifold regularization in structured output space for semi-supervised structured output prediction
Authors: Fei Jiang, Lili Jia, Xiaobao Sheng, Riley LeMieux
Date: 2015-08-12 00:00:00

Structured output prediction aims to learn a predictor to predict a
structured output from a input data vector. The structured outputs include
vector, tree, sequence, etc. We usually assume that we have a training set of
input-output pairs to train the predictor. However, in many real-world appli-
cations, it is difficult to obtain the output for a input, thus for many
training input data points, the structured outputs are missing. In this paper,
we dis- cuss how to learn from a training set composed of some input-output
pairs, and some input data points without outputs. This problem is called semi-
supervised structured output prediction. We propose a novel method for this
problem by constructing a nearest neighbor graph from the input space to
present the manifold structure, and using it to regularize the structured out-
put space directly. We define a slack structured output for each training data
point, and proposed to predict it by learning a structured output predictor.
The learning of both slack structured outputs and the predictor are unified
within one single minimization problem. In this problem, we propose to mini-
mize the structured loss between the slack structured outputs of neighboring
data points, and the prediction error measured by the structured loss. The
problem is optimized by an iterative algorithm. Experiment results over three
benchmark data sets show its advantage.
Massively-concurrent Agent-based Evolutionary Computing
Authors: D. Krzywicki, W. Turek, A. Byrski, M. Kisiel-Dorohinicki
Date: 2015-01-27 00:00:00
Tasks: Evolutionary Algorithms

The fusion of the multi-agent paradigm with evolutionary computation yielded
promising results in many optimization problems. Evolutionary multi-agent
system (EMAS) are more similar to biological evolution than classical
evolutionary algorithms. However, technological limitations prevented the use
of fully asynchronous agents in previous EMAS implementations. In this paper we
present a new algorithm for agent-based evolutionary computations. The
individuals are represented as fully autonomous and asynchronous agents. An
efficient implementation of this algorithm was possible through the use of
modern technologies based on functional languages (namely Erlang and Scala),
which natively support lightweight processes and asynchronous communication.
Our experiments show that such an asynchronous approach is both faster and more
efficient in solving common optimization problems.
Learning to Hire Teams
Authors: Adish Singla, Eric Horvitz, Pushmeet Kohli, Andreas Krause
Date: 2015-08-12 00:00:00

Crowdsourcing and human computation has been employed in increasingly
sophisticated projects that require the solution of a heterogeneous set of
tasks. We explore the challenge of building or hiring an effective team, for
performing tasks required for such projects on an ongoing basis, from an
available pool of applicants or workers who have bid for the tasks. The
recruiter needs to learn workers' skills and expertise by performing online
tests and interviews, and would like to minimize the amount of budget or time
spent in this process before committing to hiring the team. How can one
optimally spend budget to learn the expertise of workers as part of recruiting
a team? How can one exploit the similarities among tasks as well as underlying
social ties or commonalities among the workers for faster learning? We tackle
these decision-theoretic challenges by casting them as an instance of online
learning for best action selection. We present algorithms with PAC bounds on
the required budget to hire a near-optimal team with high confidence.
Furthermore, we consider an embedding of the tasks and workers in an underlying
graph that may arise from task similarities or social ties, and that can
provide additional side-observations for faster learning. We then quantify the
improvement in the bounds that we can achieve depending on the characteristic
properties of this graph structure. We evaluate our methodology on simulated
problem instances as well as on real-world crowdsourcing data collected from
the oDesk platform. Our methodology and results present an interesting
direction of research to tackle the challenges faced by a recruiter for
contract-based crowdsourcing.
Maximum a Posteriori Adaptation of Network Parameters in Deep Models
Authors: Zhen Huang, Sabato Marco Siniscalchi, I-Fan Chen, Jiadong Wu, Chin-Hui Lee
Date: 2015-03-06 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

We present a Bayesian approach to adapting parameters of a well-trained
context-dependent, deep-neural-network, hidden Markov model (CD-DNN-HMM) to
improve automatic speech recognition performance. Given an abundance of DNN
parameters but with only a limited amount of data, the effectiveness of the
adapted DNN model can often be compromised. We formulate maximum a posteriori
(MAP) adaptation of parameters of a specially designed CD-DNN-HMM with an
augmented linear hidden networks connected to the output tied states, or
senones, and compare it to feature space MAP linear regression previously
proposed. Experimental evidences on the 20,000-word open vocabulary Wall Street
Journal task demonstrate the feasibility of the proposed framework. In
supervised adaptation, the proposed MAP adaptation approach provides more than
10% relative error reduction and consistently outperforms the conventional
transformation based methods. Furthermore, we present an initial attempt to
generate hierarchical priors to improve adaptation efficiency and effectiveness
with limited adaptation data by exploiting similarities among senones.
Identifying manifolds underlying group motion in Vicsek agents
Authors: Kelum Gajamannage, Sachit Butail, Maurizio Porfiri, Erik M. Bollt
Date: 2015-08-12 00:00:00
Tasks: Dimensionality Reduction

Collective motion of animal groups often undergoes changes due to
perturbations. In a topological sense, we describe these changes as switching
between low-dimensional embedding manifolds underlying a group of evolving
agents. To characterize such manifolds, first we introduce a simple mapping of
agents between time-steps. Then, we construct a novel metric which is
susceptible to variations in the collective motion, thus revealing distinct
underlying manifolds. The method is validated through three sample scenarios
simulated using a Vicsek model, namely switching of speed, coordination, and
structure of a group. Combined with a dimensionality reduction technique that
is used to infer the dimensionality of the embedding manifold, this approach
provides an effective model-free framework for the analysis of collective
behavior across animal species.
Possible Mechanisms for Neural Reconfigurability and their Implications
Authors: Thomas M. Breuel
Date: 2015-08-12 00:00:00

The paper introduces a biologically and evolutionarily plausible neural
architecture that allows a single group of neurons, or an entire cortical
pathway, to be dynamically reconfigured to perform multiple, potentially very
different computations. The paper shows that reconfigurability can account for
the observed stochastic and distributed coding behavior of neurons and provides
a parsimonious explanation for timing phenomena in psychophysical experiments.
It also shows that reconfigurable pathways correspond to classes of statistical
classifiers that include decision lists, decision trees, and hierarchical
Bayesian methods. Implications for the interpretation of neurophysiological and
psychophysical results are discussed, and future experiments for testing the
reconfigurability hypothesis are explored.
On the Convergence of SGD Training of Neural Networks
Authors: Thomas M. Breuel
Date: 2015-08-12 00:00:00

Neural networks are usually trained by some form of stochastic gradient
descent (SGD)). A number of strategies are in common use intended to improve
SGD optimization, such as learning rate schedules, momentum, and batching.
These are motivated by ideas about the occurrence of local minima at different
scales, valleys, and other phenomena in the objective function. Empirical
results presented here suggest that these phenomena are not significant factors
in SGD optimization of MLP-related objective functions, and that the behavior
of stochastic gradient descent in these problems is better described as the
simultaneous convergence at different rates of many, largely non-interacting
subproblems
The Effects of Hyperparameters on SGD Training of Neural Networks
Authors: Thomas M. Breuel
Date: 2015-08-12 00:00:00

The performance of neural network classifiers is determined by a number of
hyperparameters, including learning rate, batch size, and depth. A number of
attempts have been made to explore these parameters in the literature, and at
times, to develop methods for optimizing them. However, exploration of
parameter spaces has often been limited. In this note, I report the results of
large scale experiments exploring these different parameters and their
interactions.
Benchmarking of LSTM Networks
Authors: Thomas M. Breuel
Date: 2015-08-11 00:00:00
Tasks: Benchmarking

LSTM (Long Short-Term Memory) recurrent neural networks have been highly
successful in a number of application areas. This technical report describes
the use of the MNIST and UW3 databases for benchmarking LSTM networks and
explores the effect of different architectural and hyperparameter choices on
performance. Significant findings include: (1) LSTM performance depends
smoothly on learning rates, (2) batching and momentum has no significant effect
on performance, (3) softmax training outperforms least square training, (4)
peephole units are not useful, (5) the standard non-linearities (tanh and
sigmoid) perform best, (6) bidirectional training combined with CTC performs
better than other methods.
Maintaining prediction quality under the condition of a growing knowledge space
Authors: Christoph Jahnz
Date: 2015-08-03 00:00:00

Intelligence can be understood as an agent's ability to predict its
environment's dynamic by a level of precision which allows it to effectively
foresee opportunities and threats. Under the assumption that such intelligence
relies on a knowledge space any effective reasoning would benefit from a
maximum portion of useful and a minimum portion of misleading knowledge
fragments. It begs the question of how the quality of such knowledge space can
be kept high as the amount of knowledge keeps growing. This article proposes a
mathematical model to describe general principles of how quality of a growing
knowledge space evolves depending on error rate, error propagation and
countermeasures. There is also shown to which extend the quality of a knowledge
space collapses as removal of low quality knowledge fragments occurs too slowly
for a given knowledge space's growth rate.
Artificial Prediction Markets for Online Prediction of Continuous Variables-A Preliminary Report
Authors: Fatemeh Jahedpari, Marina De Vos, Sattar Hashemi, Benjamin Hirsch, Julian Padget
Date: 2015-08-11 00:00:00
Tasks: Decision Making, Prediction, Q-Learning

We propose the Artificial Continuous Prediction Market (ACPM) as a means to
predict a continuous real value, by integrating a range of data sources and
aggregating the results of different machine learning (ML) algorithms. ACPM
adapts the concept of the (physical) prediction market to address the
prediction of real values instead of discrete events. Each ACPM participant has
a data source, a ML algorithm and a local decision-making procedure that
determines what to bid on what value. The contributions of ACPM are: (i)
adaptation to changes in data quality by the use of learning in: (a) the
market, which weights each market participant to adjust the influence of each
on the market prediction and (b) the participants, which use a Q-learning based
trading strategy to incorporate the market prediction into their subsequent
predictions, (ii) resilience to a changing population of low- and
high-performing participants. We demonstrate the effectiveness of ACPM by
application to an influenza-like illnesses data set, showing ACPM out-performs
a range of well-known regression models and is resilient to variation in data
source quality.
Integrating K-means with Quadratic Programming Feature Selection
Authors: Yamuna Prasad, K. K. Biswas
Date: 2015-05-07 00:00:00
Tasks: Clustering, feature selection

Several data mining problems are characterized by data in high dimensions.
One of the popular ways to reduce the dimensionality of the data is to perform
feature selection, i.e, select a subset of relevant and non-redundant features.
Recently, Quadratic Programming Feature Selection (QPFS) has been proposed
which formulates the feature selection problem as a quadratic program. It has
been shown to outperform many of the existing feature selection methods for a
variety of applications. Though, better than many existing approaches, the
running time complexity of QPFS is cubic in the number of features, which can
be quite computationally expensive even for moderately sized datasets. In this
paper we propose a novel method for feature selection by integrating k-means
clustering with QPFS. The basic variant of our approach runs k-means to bring
down the number of features which need to be passed on to QPFS. We then enhance
this idea, wherein we gradually refine the feature space from a very coarse
clustering to a fine-grained one, by interleaving steps of QPFS with k-means
clustering. Every step of QPFS helps in identifying the clusters of irrelevant
features (which can then be thrown away), whereas every step of k-means further
refines the clusters which are potentially relevant. We show that our iterative
refinement of clusters is guaranteed to converge. We provide bounds on the
number of distance computations involved in the k-means algorithm. Further,
each QPFS run is now cubic in number of clusters, which can be much smaller
than actual number of features. Experiments on eight publicly available
datasets show that our approach gives significant computational gains (both in
time and memory), over standard QPFS as well as other state of the art feature
selection methods, even while improving the overall accuracy.
Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs
Authors: Miguel Ballesteros, Chris Dyer, Noah A. Smith
Date: 2015-08-04 00:00:00
Proceeding: EMNLP 2015 9
Tasks: Dependency Parsing

We present extensions to a continuous-state dependency parsing method that makes it applicable to morphologically rich languages. Starting with a high-performance transition-based parser that uses long short-term memory (LSTM) recurrent neural networks to learn representations of the parser state, we replace lookup-based word representations with representations constructed from the orthographic representations of the words, also using LSTMs. This allows statistical sharing across word forms that are similar on the surface. Experiments for morphologically rich languages show that the parsing model benefits from incorporating the character-based encodings of words.
Communication-efficient sparse regression: a one-shot approach
Authors: Jason D. Lee, Yuekai Sun, Qiang Liu, Jonathan E. Taylor
Date: 2015-03-14 00:00:00
Tasks: regression

We devise a one-shot approach to distributed sparse regression in the
high-dimensional setting. The key idea is to average "debiased" or
"desparsified" lasso estimators. We show the approach converges at the same
rate as the lasso as long as the dataset is not split across too many machines.
We also extend the approach to generalized linear models.
Alternating Minimization Algorithm with Automatic Relevance Determination for Transmission Tomography under Poisson Noise
Authors: Yan Kaganovsky, Shaobo Han, Soysal Degirmenci, David G. Politte, David J. Brady, Joseph A. O'Sullivan, Lawrence Carin
Date: 2014-12-29 00:00:00
Tasks: Image Reconstruction

We propose a globally convergent alternating minimization (AM) algorithm for
image reconstruction in transmission tomography, which extends automatic
relevance determination (ARD) to Poisson noise models with Beer's law. The
algorithm promotes solutions that are sparse in the pixel/voxel-differences
domain by introducing additional latent variables, one for each pixel/voxel,
and then learning these variables from the data using a hierarchical Bayesian
model. Importantly, the proposed AM algorithm is free of any tuning parameters
with image quality comparable to standard penalized likelihood methods. Our
algorithm exploits optimization transfer principles which reduce the problem
into parallel 1D optimization tasks (one for each pixel/voxel), making the
algorithm feasible for large-scale problems. This approach considerably reduces
the computational bottleneck of ARD associated with the posterior variances.
Positivity constraints inherent in transmission tomography problems are also
enforced. We demonstrate the performance of the proposed algorithm for x-ray
computed tomography using synthetic and real-world datasets. The algorithm is
shown to have much better performance than prior ARD algorithms based on
approximate Gaussian noise models, even for high photon flux.
Handling oversampling in dynamic networks using link prediction
Authors: Benjamin Fish, Rajmonda S. Caceres
Date: 2015-04-24 00:00:00
Tasks: Link Prediction, Prediction

Oversampling is a common characteristic of data representing dynamic
networks. It introduces noise into representations of dynamic networks, but
there has been little work so far to compensate for it. Oversampling can affect
the quality of many important algorithmic problems on dynamic networks,
including link prediction. Link prediction seeks to predict edges that will be
added to the network given previous snapshots. We show that not only does
oversampling affect the quality of link prediction, but that we can use link
prediction to recover from the effects of oversampling. We also introduce a
novel generative model of noise in dynamic networks that represents
oversampling. We demonstrate the results of our approach on both synthetic and
real-world data.
InAR:Inverse Augmented Reality
Authors: Hao Hu, Hainan Cui
Date: 2015-08-11 00:00:00
Tasks: 3D Reconstruction

Augmented reality is the art to seamlessly fuse virtual objects into real
ones. In this short note, we address the opposite problem, the inverse
augmented reality, that is, given a perfectly augmented reality scene where
human is unable to distinguish real objects from virtual ones, how the machine
could help do the job. We show by structure from motion (SFM), a simple 3D
reconstruction technique from images in computer vision, the real and virtual
objects can be easily separated in the reconstructed 3D scene.
Learning Deep Representation for Face Alignment with Auxiliary Attributes
Authors: Zhanpeng Zhang, Ping Luo, Chen Change Loy, Xiaoou Tang
Date: 2014-08-18 00:00:00
Tasks: Attribute, Face Alignment

In this study, we show that landmark detection or face alignment task is not a single and independent problem. Instead, its robustness can be greatly improved with auxiliary information. Specifically, we jointly optimize landmark detection together with the recognition of heterogeneous but subtly correlated facial attributes, such as gender, expression, and appearance attributes. This is non-trivial since different attribute inference tasks have different learning difficulties and convergence rates. To address this problem, we formulate a novel tasks-constrained deep model, which not only learns the inter-task correlation but also employs dynamic task coefficients to facilitate the optimization convergence when learning multiple complex tasks. Extensive evaluations show that the proposed task-constrained learning (i) outperforms existing face alignment methods, especially in dealing with faces with severe occlusion and pose variation, and (ii) reduces model complexity drastically compared to the state-of-the-art methods based on cascaded deep model.