Miracle-dz/UniFAR-FTUs huggingface dataset:
INLA or MCMC? A tutorial and comparative evaluation for spatial prediction in log-Gaussian Cox processes

We investigate two options for performing Bayesian inference on spatial log-Gaussian Cox processes assuming a spatially continuous latent field: Markov chain Monte Carlo (MCMC) and the integrated nested Laplace approximation (INLA). We first describe the device of approximating a spatially continuous Gaussian field by a Gaussian Markov random field on a discrete lattice, and present a simulation study showing that, with careful choice of parameter values, small neighbourhood sizes can give excellent approximations. We then introduce the spatial log-Gaussian Cox process and describe MCMC and INLA methods for spatial prediction within this model class. We report the results of a simulation study in which we compare the Metropolis-adjusted Langevin Algorithm (MALA) and the technique of approximating the continuous latent field by a discrete one, followed by approximate Bayesian inference via INLA over a selection of 18 simulated scenarios. The results question the notion that the latter technique is both significantly faster and more robust than MCMC in this setting; 100,000 iterations of the MALA algorithm running in 20 min on a desktop PC delivered greater predictive accuracy than the default INLA strategy, which ran in 4 min and gave comparative performance to the full Laplace approximation which ran in 39 min.
Sparse approximations of protein structure from noisy random projections

Single-particle electron microscopy is a modern technique that biophysicists employ to learn the structure of proteins. It yields data that consist of noisy random projections of the protein structure in random directions, with the added complication that the projection angles cannot be observed. In order to reconstruct a three-dimensional model, the projection directions need to be estimated by use of an ad-hoc starting estimate of the unknown particle. In this paper we propose a methodology that does not rely on knowledge of the projection angles, to construct an objective data-dependent low-resolution approximation of the unknown structure that can serve as such a starting estimate. The approach assumes that the protein admits a suitable sparse representation, and employs discrete L-1-regularization (LASSO) as well as notions from shape theory to tackle the peculiar challenges involved in the associated inverse problem. We illustrate the approach by application to the reconstruction of an E. coli protein component called the Klenow fragment.
Further results on the H-Test of Durbin for stable autoregressive processes

The purpose of this paper is to investigate the asymptotic behavior of the Durbin-Watson statistic for the stable p−order autoregressive process when the driven noise is given by a first-order autoregressive process. It is an extension of the previous work of Bercu and Pröıa devoted to the particular case p = 1. We establish the almost sure convergence and the asymptotic normality for both the least squares estimator of the unknown vector parameter of the autoregressive process as well as for the serial correlation estimator associated with the driven noise. In addition, the almost sure rates of convergence of our estimates are also provided. Then, we prove the almost sure convergence and the asymptotic normality for the Durbin-Watson statistic and we derive a two-sided statistical procedure for testing the presence of a significant first-order residual autocorrelation that appears to clarify and to improve the well-known h-test suggested by Durbin. Finally, we briefly summarize our observations on simulated samples.
Parametric estimation. Finite sample theory

The paper aims at reconsidering the famous Le Cam LAN theory. The main features of the approach which make it different from the classical one are: (1) the study is non-asymptotic, that is, the sample size is xed and does not tend to infinity; (2) the parametric assumption is possibly misspecified and the underlying data distribution can lie beyond the given parametric family. The main results include a large deviation bounds for the (quasi) maximum likelihood and the local quadratic majorization of the log-likelihood process. The latter yields a number of important corollaries for statistical inference - concentration, confidence and risk bounds, expansion of the maximum likelihood estimate, etc. All these corollaries are stated in a non-classical way admitting a model misspecification and finite samples. However, the classical asymptotic results including the efficiency bounds can be easily derived as corollaries of the obtained non-asymptotic statements. The general results are illustrated for the i.i. d. set-up as well as for generalized linear and median estimation. The results apply for any dimension of the parameter space and provide a quantitative lower bound on the sample size yielding the root-n accuracy. We also discuss the procedures which allows to recover the structure when its e ective dimension is unknown.
Elicitation of Weibull priors

Based on expert opinions, informative prior elicitation for the common Weibull lifetime distribution usually presents some difficulties since it requires to elicit a two-dimensional joint prior. We consider here a reliability framework where the available expert information states directly in terms of prior predictive values (lifetimes) and not parameter values, which are less intuitive. The novelty of our procedure is to weigh the expert information by the size m of a virtual sample yielding a similar information, the prior being seen as a reference posterior. Thus, the prior calibration by the Bayesian analyst, who has to moderate the subjective information with respect to the data information, is made simple. A main result is the full tractability of the prior under mild conditions, despite the conjugation issues encountered with the Weibull distribution. Besides, m is a practical focus point for discussion between analysts and experts, and a helpful parameter for leading sensitivity studies and reducing the potential imbalance in posterior selection between Bayesian Weibull models, which can be due to favoring arbitrarily a prior. The calibration of m is discussed and a real example is treated along the paper.
Comment: The Need for Syncretism in Applied Statistics

tribution to the merging of frequentisi and Bayesian thinking into a harmonious (even if not strictly coherent) statistical viewpoint. I will review my thinking along those lines and some inspirations for it. I agree with most of Dr. Efron's views expressed here and in Efron (2005), with these important exceptions: First, I disagree that frequentism has supplied a good set of working rules. Instead, I argue that frequentism has been a prime source of reckless overconfidence in many fields (especially but not only in the form of 0.05-level testing; see Rothman, Greenland and Lash, 2008, Chapter 10 for examples and further citations). I also disagree that Bayesians are more aggressive than frequentists in modeling. The most aggressive modeling is that which fixes unknown parameters at some known constant like zero (whence they disappear from the model and are forgotten), thus generating overconfident inferences and an illusion of simplicity; such practice is a hallmark of conventional frequentist applications in observational studies. As working rules, the problem with conventional methods lies not so much with frequentism, but rather with frequentist tools for designed experiments being misapplied to observational data (Greenland, 2005a). Bayesians can and do misapply their methods similarly; they just haven't been given as much opportunity to do so. Conversely, many frequentist as well as Bayesian tools for observational studies have been developed, especially for sensitivity analysis. But the overconfidence problem has been perpetuated by the ongoing concealment of unbelievable point-mass priors within models in order to maintain frequentist identification of target parameters. The problem can be addressed by sacrificing identification and replacing bad modeling assumptions with explicit and reasonable priors (Gustaf son, 2005;
Regularized Least-Mean-Square Algorithms

We consider adaptive system identification problems with convex constraints and propose a family of regularized Least-Mean-Square (LMS) algorithms. We show that with a properly selected regularization parameter the regularized LMS provably dominates its conventional counterpart in terms of mean square deviations. We establish simple and closed-form expressions for choosing this regularization parameter. For identifying an unknown sparse system we propose sparse and group-sparse LMS algorithms, which are special examples of the regularized LMS family. Simulation results demonstrate the advantages of the proposed filters in both convergence rate and steady-state error under sparsity assumptions on the true coefficient vector.
STATISTICAL ANALYSIS OF FACTOR MODELS OF HIGH DIMENSION

This paper considers the maximum likelihood estimation of factor models of high dimension, where the number of variables (N) is comparable with or even greater than the number of observations (T). An inferential theory is developed. We establish not only consistency but also the rate of convergence and the limiting distributions. Five different sets of identification conditions are considered. We show that the distributions of the MLE estimators depend on the identification restrictions. Unlike the principal components approach, the maximum likelihood estimator explicitly allows heteroskedasticities, which are jointly estimated with other parameters. Efficiency of MLE relative to the principal components method is also considered.
A FUNCTIONAL VERSION OF THE ARCH MODEL

Improvements in data acquisition and processing techniques have led to an almost continuous flow of information for financial data. High-resolution tick data are available and can be quite conveniently described by a continuous-time process. It is therefore natural to ask for possible extensions of financial time series models to a functional setup. In this paper we propose a functional version of the popular autoregressive conditional heteroskedasticity model. We will establish conditions for the existence of a strictly stationary solution, derive weak dependence and moment conditions, show consistency of the estimators, and perform a small empirical study demonstrating how our model matches with real data.
Positive definite matrices and the S-divergence

Positive definite matrices abound in a dazzling variety of applications. This ubiquity can be in part attributed to their rich geometric structure: positive definite matrices form a self-dual convex cone whose strict interior is a Riemannian manifold. The manifold view is endowed with a "natural" distance function while the conic view is not. Nevertheless, drawing motivation from the conic view, we introduce the S-Divergence as a "natural" distance-like function on the open cone of positive definite matrices. We motivate the S-divergence via a sequence of results that connect it to the Riemannian distance. In particular, we show that (a) this divergence is the square of a distance; and (b) that it has several geometric properties similar to those of the Riemannian distance, though without being computationally as demanding. The S-divergence is even more intriguing: although nonconvex, we can still compute matrix means and medians using it to global optimality. We complement our results with some numerical experiments illustrating our theorems and our optimization algorithm for computing matrix medians.
Conjugate Projective Limits

We characterize conjugate nonparametric Bayesian models as projective limits of conjugate, finite-dimensional Bayesian models. In particular, we identify a large class of nonparametric models representable as infinite-dimensional analogues of exponential family distributions and their canonical conjugate priors. This class contains most models studied in the literature, including Dirichlet processes and Gaussian process regression models. To derive these results, we introduce a representation of infinite-dimensional Bayesian models by projective limits of regular conditional probabilities. We show under which conditions the nonparametric model itself, its sufficient statistics, and -- if they exist -- conjugate updates of the posterior are projective limits of their respective finite-dimensional counterparts. We illustrate our results both by application to existing nonparametric models and by construction of a model on infinite permutations.
MedLDA: A General Framework of Maximum Margin Supervised Topic Models

Supervised topic models utilize document's side information for discovering predictive low dimensional representations of documents. Existing models apply the likelihood-based estimation. In this paper, we present a general framework of max-margin supervised topic models for both continuous and categorical response variables. Our approach, the maximum entropy discrimination latent Dirichlet allocation (MedLDA), utilizes the max-margin principle to train supervised topic models and estimate predictive topic representations that are arguably more suitable for prediction tasks. The general principle of MedLDA can be applied to perform joint max-margin learning and maximum likelihood estimation for arbitrary topic models, directed or undirected, and supervised or unsupervised, when the supervised side information is available. We develop efficient variational methods for posterior inference and parameter estimation, and demonstrate qualitatively and quantitatively the advantages of MedLDA over likelihood-based topic models on movie review and 20 Newsgroups data sets.
Discussion of "Riemann manifold Langevin and Hamiltonian Monte Carlo methods'' by M. Girolami and B. Calderhead

This technical report is the union of two contributions to the discussion of the Read Paper "Riemann manifold Langevin and Hamiltonian Monte Carlo methods" by B. Calderhead and M. Girolami, presented in front of the Royal Statistical Society on October 13th 2010 and to appear in the Journal of the Royal Statistical Society Series B. The first comment establishes a parallel and possible interactions with Adaptive Monte Carlo methods. The second comment exposes a detailed study of Riemannian Manifold Hamiltonian Monte Carlo (RMHMC) for a weakly identifiable model presenting a strong ridge in its geometry.
An Algorithm for Learning the Essential Graph

This article presents an algorithm for learning the essential graph of a Bayesian network. The basis of the algorithm is the Maximum Minimum Parents and Children algorithm developed by previous authors, with three substantial modifications. The MMPC algorithm is the first stage of the Maximum Minimum Hill Climbing algorithm for learning the directed acyclic graph of a Bayesian network, introduced by previous authors. The MMHC algorithm runs in two phases; firstly, the MMPC algorithm to locate the skeleton and secondly an edge orientation phase. The computationally expensive part is the edge orientation phase. The first modification introduced to the MMPC algorithm, which requires little additional computational cost, is to obtain the immoralities and hence the essential graph. This renders the edge orientation phase, the computationally expensive part, unnecessary, since the entire Markov structure that can be derived from data is present in the essential graph. Secondly, the MMPC algorithm can accept independence statements that are logically inconsistent with those rejected, since with tests for independence, a `do not reject' conclusion for a particular independence statement is taken as `accept' independence. An example is given to illustrate this and a modification is suggested to ensure that the conditional independence statements are logically consistent. Thirdly, the MMHC algorithm makes an assumption of faithfulness. An example of a data set is given that does not satisfy this assumption and a modification is suggested to deal with some situations where the assumption is not satisfied. The example in question also illustrates problems with the `faithfulness' assumption that cannot be tackled by this modification.
Feature selection when there are many influential features

Recent discussion of the success of feature selection methods has argued that focusing on a relatively small number of features has been counterproductive. Instead, it is suggested, the number of significant features can be in the thousands or tens of thousands, rather than (as is commonly supposed at present) approximately in the range from five to fifty. This change, in orders of magnitude, in the number of influential features, necessitates alterations to the way in which we choose features and to the manner in which the success of feature selection is assessed. In this paper, we suggest a general approach that is suited to cases where the number of relevant features is very large, and we consider particular versions of the approach in detail. We propose ways of measuring performance, and we study both theoretical and numerical properties of the proposed methodology.
Efficient Marginal Likelihood Computation for Gaussian Process Regression

In a Bayesian learning setting, the posterior distribution of a predictive model arises from a trade-off between its prior distribution and the conditional likelihood of observed data. Such distribution functions usually rely on additional hyperparameters which need to be tuned in order to achieve optimum predictive performance; this operation can be efficiently performed in an Empirical Bayes fashion by maximizing the posterior marginal likelihood of the observed data. Since the score function of this optimization problem is in general characterized by the presence of local optima, it is necessary to resort to global optimization strategies, which require a large number of function evaluations. Given that the evaluation is usually computationally intensive and badly scaled with respect to the dataset size, the maximum number of observations that can be treated simultaneously is quite limited. In this paper, we consider the case of hyperparameter tuning in Gaussian process regression. A straightforward implementation of the posterior log-likelihood for this model requires O(N^3) operations for every iteration of the optimization procedure, where N is the number of examples in the input dataset. We derive a novel set of identities that allow, after an initial overhead of O(N^3), the evaluation of the score function, as well as the Jacobian and Hessian matrices, in O(N) operations. We prove how the proposed identities, that follow from the eigendecomposition of the kernel matrix, yield a reduction of several orders of magnitude in the computation time for the hyperparameter optimization problem. Notably, the proposed solution provides computational advantages even with respect to state of the art approximations that rely on sparse kernel matrices.
Structure Variability in Bayesian Networks.

The structure of a Bayesian network encodes most of the information about the probability distribution of the data, which is uniquely identified given some general distributional assumptions. Therefore it’s important to study the variability of its network structure, which can be used to compare the performance of different learning algorithms and to measure the strength of any arbitrary subset of arcs. In this paper we will introduce some descriptive statistics and the corresponding parametric and Monte Carlo tests on the undirected graph underlying the structure of a Bayesian network, modeled as a multivariate Bernoulli random variable.
Parallel marginalization Monte Carlo with applications to conditional path sampling

Monte Carlo sampling methods often suffer from long correlation times. Consequently, these methods must be run for many steps to generate an independent sample. In this paper a method is proposed to overcome this difficulty. The method utilizes information from rapidly equilibrating coarse Markov chains that sample marginal distributions of the full system. This is accomplished through exchanges between the full chain and the auxiliary coarse chains. Results of numerical tests on the bridge sampling and filtering/smoothing problems for a stochastic differential equation are presented.
On the asymptotic normality of kernel density estimators for linear random fields

We establish sufficient conditions for the asymptotic normality of kernel density estimators, applied to causal linear random fields. Our conditions on the coefficients of linear random fields are weaker than known results, although our assumption on the bandwidth is not minimal. The proof is based on the $m$-approximation method. As a key step, we prove a central limit theorem for triangular arrays of stationary $m$-dependent random fields with unbounded $m$. We also apply a moment inequality recently established for stationary random fields.
On the spatial correlation between areas of high coseismic slip and aftershock clusters of the Maule earthquake Mw=8.8

We study the spatial distribution of clusters associated to the aftershocks of the megathrust Maule earthquake MW 8.8 of 27 February 2010. We used a recent clustering method which hinges on a nonparametric estimation of the underlying probability density function to detect subsets of points forming clusters associated with high density areas. In addition, we estimate the probability density function using a nonparametric kernel method for each of these clusters. This allows us to identify a set of regions where there is an association between frequency of events and coseismic slip. Our results suggest that high coseismic slip spatially correlates with high aftershock frequency.
The Role of Family-Based Designs in Genome-Wide Association Studies

Genome-Wide Association Studies (GWAS) offer an exciting and promising new research avenue for finding genes for complex diseases. Traditional case-control and cohort studies offer many advantages for such designs. Family-based association designs have long been attractive for their robustness properties, but robustness can mean a loss of power. In this paper we discuss some of the special features of family designs and their relevance in the era of GWAS.
Inference for SDE Models via Approximate Bayesian Computation

Models defined by stochastic differential equations (SDEs) allow for the representation of random variability in dynamical systems. The relevance of this class of models is growing in many applied research areas and is already a standard tool to model, for example, financial, neuronal, and population growth dynamics. However, inference for multidimensional SDE models is still very challenging, both computationally and theoretically. Approximate Bayesian computation (ABC) allows to perform Bayesian inference for models which are sufficiently complex that the likelihood function is either analytically unavailable or computationally prohibitive to evaluate. A computationally efficient ABC-MCMC algorithm is proposed, halving the running time in our simulations. Focus here is on the case where the SDE describes latent dynamics in state-space models; however, the methodology is not limited to the state-space framework. We consider simulation studies for a pharmacokinetics/pharmacodynamics model and for stochastic chemical reactions and we provide a Matlab package that implements our ABC-MCMC algorithm.