pwc-archive/papers-with-abstracts huggingface dataset:
Conclusions from a NAIVE Bayes Operator Predicting the Medicare 2011 Transaction Data Set
Authors: Nick Williams
Date: 2014-02-20 00:00:00

Introduction: The United States Federal Government operates one of the worlds
largest medical insurance programs, Medicare, to ensure payment for clinical
services for the elderly, illegal aliens and those without the ability to pay
for their care directly. This paper evaluates the Medicare 2011 Transaction
Data Set which details the transfer of funds from Medicare to private and
public clinical care facilities for specific clinical services for the
operational year 2011. Methods: Data mining was conducted to establish the
relationships between reported and computed transaction values in the data set
to better understand the drivers of Medicare transactions at a programmatic
level. Results: The models averaged 88 for average model accuracy and 38 for
average Kappa during training. Some reported classes are highly independent
from the available data as their predictability remains stable regardless of
redaction of supporting and contradictory evidence. DRG or procedure type
appears to be unpredictable from the available financial transaction values.
Conclusions: Overlay hypotheses such as charges being driven by the volume
served or DRG being related to charges or payments is readily false in this
analysis despite 28 million Americans being billed through Medicare in 2011 and
the program distributing over 70 billion in this transaction set alone. It may
be impossible to predict the dependencies and data structures the payer of last
resort without data from payers of first and second resort. Political concerns
about Medicare would be better served focusing on these first and second order
payer systems as what Medicare costs is not dependent on Medicare itself.
Convergence Analysis and Parallel Computing Implementation for the Multiagent Coordination Optimization Algorithm
Authors: Qing Hui, Haopeng Zhang
Date: 2013-06-02 00:00:00

In this report, a novel variation of Particle Swarm Optimization (PSO)
algorithm, called Multiagent Coordination Optimization (MCO), is implemented in
a parallel computing way for practical use by introducing MATLAB built-in
function "parfor" into MCO. Then we rigorously analyze the global convergence
of MCO by means of semistability theory. Besides sharing global optimal
solutions with the PSO algorithm, the MCO algorithm integrates cooperative
swarm behavior of multiple agents into the update formula by sharing velocity
and position information between neighbors to improve its performance.
Numerical evaluation of the parallel MCO algorithm is provided in the report by
running the proposed algorithm on supercomputers in the High Performance
Computing Center at Texas Tech University. In particular, the optimal value and
consuming time are compared with PSO and serial MCO by solving several
benchmark functions in the literature, respectively. Based on the simulation
results, the performance of the parallel MCO is not only superb compared with
PSO for solving many nonlinear, noncovex optimization problems, but also is of
high efficiency by saving the computational time.
Learning the Parameters of Determinantal Point Process Kernels
Authors: Raja Hafiz Affandi, Emily B. Fox, Ryan P. Adams, Ben Taskar
Date: 2014-02-20 00:00:00
Tasks: Diversity, Point Processes

Determinantal point processes (DPPs) are well-suited for modeling repulsion
and have proven useful in many applications where diversity is desired. While
DPPs have many appealing properties, such as efficient sampling, learning the
parameters of a DPP is still considered a difficult problem due to the
non-convex nature of the likelihood function. In this paper, we propose using
Bayesian methods to learn the DPP kernel parameters. These methods are
applicable in large-scale and continuous DPP settings even when the exact form
of the eigendecomposition is unknown. We demonstrate the utility of our DPP
learning methods in studying the progression of diabetic neuropathy based on
spatial distribution of nerve fibers, and in studying human perception of
diversity in images.
A Quasi-Newton Method for Large Scale Support Vector Machines
Authors: Aryan Mokhtari, Alejandro Ribeiro
Date: 2014-02-20 00:00:00
Tasks: General Classification

This paper adapts a recently developed regularized stochastic version of the
Broyden, Fletcher, Goldfarb, and Shanno (BFGS) quasi-Newton method for the
solution of support vector machine classification problems. The proposed method
is shown to converge almost surely to the optimal classifier at a rate that is
linear in expectation. Numerical results show that the proposed method exhibits
a convergence rate that degrades smoothly with the dimensionality of the
feature vectors.
Asymptotic Accuracy of Distribution-Based Estimation for Latent Variables
Authors: Keisuke Yamazaki
Date: 2012-04-10 00:00:00
Tasks: Active Learning, Model Selection

Hierarchical statistical models are widely employed in information science
and data engineering. The models consist of two types of variables: observable
variables that represent the given data and latent variables for the
unobservable labels. An asymptotic analysis of the models plays an important
role in evaluating the learning process; the result of the analysis is applied
not only to theoretical but also to practical situations, such as optimal model
selection and active learning. There are many studies of generalization errors,
which measure the prediction accuracy of the observable variables. However, the
accuracy of estimating the latent variables has not yet been elucidated. For a
quantitative evaluation of this, the present paper formulates
distribution-based functions for the errors in the estimation of the latent
variables. The asymptotic behavior is analyzed for both the maximum likelihood
and the Bayes methods.
Diffusion Least Mean Square: Simulations
Authors: Jonathan Gelati, Sithan Kanna
Date: 2014-02-19 00:00:00
Tasks: Diversity

In this technical report we analyse the performance of diffusion strategies
applied to the Least-Mean-Square adaptive filter. We configure a network of
cooperative agents running adaptive filters and discuss their behaviour when
compared with a non-cooperative agent which represents the average of the
network. The analysis provides conditions under which diversity in the filter
parameters is beneficial in terms of convergence and stability. Simulations
drive and support the analysis.
The Application of Imperialist Competitive Algorithm for Fuzzy Random Portfolio Selection Problem
Authors: Mir Ehsan Hesam Sadati, Jamshid Bagherzadeh Mohasefi
Date: 2014-02-19 00:00:00
Tasks: Portfolio Optimization

This paper presents an implementation of the Imperialist Competitive
Algorithm (ICA) for solving the fuzzy random portfolio selection problem where
the asset returns are represented by fuzzy random variables. Portfolio
Optimization is an important research field in modern finance. By using the
necessity-based model, fuzzy random variables reformulate to the linear
programming and ICA will be designed to find the optimum solution. To show the
efficiency of the proposed method, a numerical example illustrates the whole
idea on implementation of ICA for fuzzy random portfolio selection problem.
Nonparametric Weight Initialization of Neural Networks via Integral Representation
Authors: Sho Sonoda, Noboru Murata
Date: 2013-12-23 00:00:00
Tasks: regression

A new initialization method for hidden parameters in a neural network is
proposed. Derived from the integral representation of the neural network, a
nonparametric probability distribution of hidden parameters is introduced. In
this proposal, hidden parameters are initialized by samples drawn from this
distribution, and output parameters are fitted by ordinary linear regression.
Numerical experiments show that backpropagation with proposed initialization
converges faster than uniformly random initialization. Also it is shown that
the proposed method achieves enough accuracy by itself without backpropagation
in some cases.
Near-optimal-sample estimators for spherical Gaussian mixtures
Authors: Jayadev Acharya, Ashkan Jafarpour, Alon Orlitsky, Ananda Theertha Suresh
Date: 2014-02-19 00:00:00
Proceeding: NeurIPS 2014 12

Statistical and machine-learning algorithms are frequently applied to
high-dimensional data. In many of these applications data is scarce, and often
much more costly than computation time. We provide the first sample-efficient
polynomial-time estimator for high-dimensional spherical Gaussian mixtures.
  For mixtures of any $k$ $d$-dimensional spherical Gaussians, we derive an
intuitive spectral-estimator that uses
$\mathcal{O}_k\bigl(\frac{d\log^2d}{\epsilon^4}\bigr)$ samples and runs in time
$\mathcal{O}_{k,\epsilon}(d^3\log^5 d)$, both significantly lower than
previously known. The constant factor $\mathcal{O}_k$ is polynomial for sample
complexity and is exponential for the time complexity, again much smaller than
what was previously known. We also show that
$\Omega_k\bigl(\frac{d}{\epsilon^2}\bigr)$ samples are needed for any
algorithm. Hence the sample complexity is near-optimal in the number of
dimensions.
  We also derive a simple estimator for one-dimensional mixtures that uses
$\mathcal{O}\bigl(\frac{k \log \frac{k}{\epsilon} }{\epsilon^2} \bigr)$ samples
and runs in time
$\widetilde{\mathcal{O}}\left(\bigl(\frac{k}{\epsilon}\bigr)^{3k+1}\right)$.
Our other technical contributions include a faster algorithm for choosing a
density estimate from a set of distributions, that minimizes the $\ell_1$
distance to an unknown underlying distribution.
Understanding Deep Architectures using a Recursive Convolutional Network
Authors: David Eigen, Jason Rolfe, Rob Fergus, Yann Lecun
Date: 2013-12-06 00:00:00

A key challenge in designing convolutional network models is sizing them
appropriately. Many factors are involved in these decisions, including number
of layers, feature maps, kernel sizes, etc. Complicating this further is the
fact that each of these influence not only the numbers and dimensions of the
activation units, but also the total number of parameters. In this paper we
focus on assessing the independent contributions of three of these linked
variables: The numbers of layers, feature maps, and parameters. To accomplish
this, we employ a recursive convolutional network whose weights are tied
between layers; this allows us to vary each of the three factors in a
controlled setting. We find that while increasing the numbers of layers and
parameters each have clear benefit, the number of feature maps (and hence
dimensionality of the representation) appears ancillary, and finds most of its
benefit through the introduction of more weights. Our results (i) empirically
confirm the notion that adding layers alone increases computational power,
within the context of convolutional layers, and (ii) suggest that precise
sizing of convolutional feature map dimensions is itself of little concern;
more attention should be paid to the number of parameters in these layers
instead.
A normative account of defeasible and probabilistic inference
Authors: Julio Lemos
Date: 2014-02-19 00:00:00

In this paper, we provide more evidence for the contention that logical
consequence should be understood in normative terms. Hartry Field and John
MacFarlane covered the classical case. We extend their work, examining what it
means for an agent to be obliged to infer a conclusion when faced with
uncertain information or reasoning within a non-monotonic, defeasible, logical
framework (which allows e. g. for inference to be drawn from premises
considered true unless evidence to the contrary is presented).
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Authors: Andrew M. Saxe, James L. McClelland, Surya Ganguli
Date: 2013-12-20 00:00:00
Tasks: Unsupervised Pre-training

Despite the widespread practical success of deep learning methods, our
theoretical understanding of the dynamics of learning in deep neural networks
remains quite sparse. We attempt to bridge the gap between the theory and
practice of deep learning by systematically analyzing learning dynamics for the
restricted case of deep linear neural networks. Despite the linearity of their
input-output map, such networks have nonlinear gradient descent dynamics on
weights that change with the addition of each new hidden layer. We show that
deep linear networks exhibit nonlinear learning phenomena similar to those seen
in simulations of nonlinear networks, including long plateaus followed by rapid
transitions to lower error solutions, and faster convergence from greedy
unsupervised pretraining initial conditions than from random initial
conditions. We provide an analytical description of these phenomena by finding
new exact solutions to the nonlinear dynamics of deep learning. Our theoretical
analysis also reveals the surprising finding that as the depth of a network
approaches infinity, learning speed can nevertheless remain finite: for a
special class of initial conditions on the weights, very deep networks incur
only a finite, depth independent, delay in learning speed relative to shallow
networks. We show that, under certain conditions on the training data,
unsupervised pretraining can find this special class of initial conditions,
while scaled random Gaussian initializations cannot. We further exhibit a new
class of random orthogonal initial conditions on weights that, like
unsupervised pre-training, enjoys depth independent learning times. We further
show that these initial conditions also lead to faithful propagation of
gradients even in deep nonlinear networks, as long as they operate in a special
regime known as the edge of chaos.
Efficient Inference of Gaussian Process Modulated Renewal Processes with Application to Medical Event Data
Authors: Thomas A. Lasko
Date: 2014-02-19 00:00:00
Tasks: Gaussian Processes

The episodic, irregular and asynchronous nature of medical data render them
difficult substrates for standard machine learning algorithms. We would like to
abstract away this difficulty for the class of time-stamped categorical
variables (or events) by modeling them as a renewal process and inferring a
probability density over continuous, longitudinal, nonparametric intensity
functions modulating that process. Several methods exist for inferring such a
density over intensity functions, but either their constraints and assumptions
prevent their use with our potentially bursty event streams, or their time
complexity renders their use intractable on our long-duration observations of
high-resolution events, or both. In this paper we present a new and efficient
method for inferring a distribution over intensity functions that uses direct
numeric integration and smooth interpolation over Gaussian processes. We
demonstrate that our direct method is up to twice as accurate and two orders of
magnitude more efficient than the best existing method (thinning). Importantly,
the direct method can infer intensity functions over the full range of bursty
to memoryless to regular events, which thinning and many other methods cannot.
Finally, we apply the method to clinical event data and demonstrate the
face-validity of the abstraction, which is now amenable to standard learning
algorithms.
Intriguing properties of neural networks
Authors: Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus
Date: 2013-12-21 00:00:00

Deep neural networks are highly expressive models that have recently achieved
state of the art performance on speech and visual recognition tasks. While
their expressiveness is the reason they succeed, it also causes them to learn
uninterpretable solutions that could have counter-intuitive properties. In this
paper we report two such properties.
  First, we find that there is no distinction between individual high level
units and random linear combinations of high level units, according to various
methods of unit analysis. It suggests that it is the space, rather than the
individual units, that contains of the semantic information in the high layers
of neural networks.
  Second, we find that deep neural networks learn input-output mappings that
are fairly discontinuous to a significant extend. We can cause the network to
misclassify an image by applying a certain imperceptible perturbation, which is
found by maximizing the network's prediction error. In addition, the specific
nature of these perturbations is not a random artifact of learning: the same
perturbation can cause a different network, that was trained on a different
subset of the dataset, to misclassify the same input.
Deep learning for neuroimaging: a validation study
Authors: Sergey M. Plis, Devon R. Hjelm, Ruslan Salakhutdinov, Vince D. Calhoun
Date: 2013-12-20 00:00:00
Tasks: Deep Learning, Representation Learning

Deep learning methods have recently made notable advances in the tasks of
classification and representation learning. These tasks are important for brain
imaging and neuroscience discovery, making the methods attractive for porting
to a neuroimager's toolbox. Success of these methods is, in part, explained by
the flexibility of deep learning models. However, this flexibility makes the
process of porting to new areas a difficult parameter optimization problem. In
this work we demonstrate our results (and feasible parameter ranges) in
application of deep learning methods to structural and functional brain imaging
data. We also describe a novel constraint-based approach to visualizing high
dimensional data. We use it to analyze the effect of parameter choices on data
transformations. Our results show that deep learning methods are able to learn
physiologically important representations and detect latent relations in
neuroimaging data.
A Powerful Genetic Algorithm for Traveling Salesman Problem
Authors: Shujia Liu
Date: 2014-02-19 00:00:00
Tasks: Traveling Salesman Problem

This paper presents a powerful genetic algorithm(GA) to solve the traveling
salesman problem (TSP). To construct a powerful GA, I use edge swapping(ES)
with a local search procedure to determine good combinations of building blocks
of parent solutions for generating even better offspring solutions.
Experimental results on well studied TSP benchmarks demonstrate that the
proposed GA is competitive in finding very high quality solutions on instances
with up to 16,862 cities.
Retrieval of Experiments by Efficient Estimation of Marginal Likelihood
Authors: Sohan Seth, John Shawe-Taylor, Samuel Kaski
Date: 2014-02-19 00:00:00
Tasks: Retrieval

We study the task of retrieving relevant experiments given a query
experiment. By experiment, we mean a collection of measurements from a set of
`covariates' and the associated `outcomes'. While similar experiments can be
retrieved by comparing available `annotations', this approach ignores the
valuable information available in the measurements themselves. To incorporate
this information in the retrieval task, we suggest employing a retrieval metric
that utilizes probabilistic models learned from the measurements. We argue that
such a metric is a sensible measure of similarity between two experiments since
it permits inclusion of experiment-specific prior knowledge. However, accurate
models are often not analytical, and one must resort to storing posterior
samples which demands considerable resources. Therefore, we study strategies to
select informative posterior samples to reduce the computational load while
maintaining the retrieval performance. We demonstrate the efficacy of our
approach on simulated data with simple linear regression as the models, and
real world datasets.
A Survey on Semi-Supervised Learning Techniques
Authors: V. Jothi Prakash, Dr. L. M. Nithya
Date: 2014-02-19 00:00:00
Tasks: Survey

Semisupervised learning is a learning standard which deals with the study of
how computers and natural systems such as human beings acquire knowledge in the
presence of both labeled and unlabeled data. Semisupervised learning based
methods are preferred when compared to the supervised and unsupervised learning
because of the improved performance shown by the semisupervised approaches in
the presence of large volumes of data. Labels are very hard to attain while
unlabeled data are surplus, therefore semisupervised learning is a noble
indication to shrink human labor and improve accuracy. There has been a large
spectrum of ideas on semisupervised learning. In this paper we bring out some
of the key approaches for semisupervised learning.
Sparse Quantile Huber Regression for Efficient and Robust Estimation
Authors: Aleksandr Y. Aravkin, Anju Kambadur, Aurelie C. Lozano, Ronny Luss
Date: 2014-02-19 00:00:00
Tasks: quantile regression, regression, Variable Selection

We consider new formulations and methods for sparse quantile regression in
the high-dimensional setting. Quantile regression plays an important role in
many applications, including outlier-robust exploratory analysis in gene
selection. In addition, the sparsity consideration in quantile regression
enables the exploration of the entire conditional distribution of the response
variable given the predictors and therefore yields a more comprehensive view of
the important predictors. We propose a generalized OMP algorithm for variable
selection, taking the misfit loss to be either the traditional quantile loss or
a smooth version we call quantile Huber, and compare the resulting greedy
approaches with convex sparsity-regularized formulations. We apply a recently
proposed interior point methodology to efficiently solve all convex
formulations as well as convex subproblems in the generalized OMP setting, pro-
vide theoretical guarantees of consistent estimation, and demonstrate the
performance of our approach using empirical studies of simulated and genomic
datasets.
Image Representation Learning Using Graph Regularized Auto-Encoders
Authors: Yiyi Liao, Yue Wang, Yong liu
Date: 2013-12-03 00:00:00
Tasks: Clustering, Image Clustering, Representation Learning

We consider the problem of image representation for the tasks of unsupervised
learning and semi-supervised learning. In those learning tasks, the raw image
vectors may not provide enough representation for their intrinsic structures
due to their highly dense feature space. To overcome this problem, the raw
image vectors should be mapped to a proper representation space which can
capture the latent structure of the original data and represent the data
explicitly for further learning tasks such as clustering.
  Inspired by the recent research works on deep neural network and
representation learning, in this paper, we introduce the multiple-layer
auto-encoder into image representation, we also apply the locally invariant
ideal to our image representation with auto-encoders and propose a novel
method, called Graph regularized Auto-Encoder (GAE). GAE can provide a compact
representation which uncovers the hidden semantics and simultaneously respects
the intrinsic geometric structure.
  Extensive experiments on image clustering show encouraging results of the
proposed algorithm in comparison to the state-of-the-art algorithms on
real-word cases.
Improving Deep Neural Networks with Probabilistic Maxout Units
Authors: Jost Tobias Springenberg, Martin Riedmiller
Date: 2013-12-20 00:00:00
Tasks: General Classification, image-classification, Image Classification

We present a probabilistic variant of the recently introduced maxout unit.
The success of deep neural networks utilizing maxout can partly be attributed
to favorable performance under dropout, when compared to rectified linear
units. It however also depends on the fact that each maxout unit performs a
pooling operation over a group of linear transformations and is thus partially
invariant to changes in its input. Starting from this observation we ask the
question: Can the desirable properties of maxout units be preserved while
improving their invariance properties ? We argue that our probabilistic maxout
(probout) units successfully achieve this balance. We quantitatively verify
this claim and report classification performance matching or exceeding the
current state of the art on three challenging image classification benchmarks
(CIFAR-10, CIFAR-100 and SVHN).