Miracle-dz/UniFAR-FTUs huggingface dataset:
Exact Bayesian Analysis of Mixtures

In this paper, we show how a complete and exact Bayesian analysis of a parametric mixture model is possible in some cases when components of the mixture are taken from exponential families and when conjugate priors are used. This restricted set-up allows us to show the relevance of the Bayesian approach as well as to exhibit the limitations of a complete analysis, namely that it is impossible to conduct this analysis when the sample size is too large, when the data are not from an exponential family, or when priors that are more complex than conjugate priors are used.
Summarization and Classification of Non-Poisson Point Processes

Fitting models for non-Poisson point processes is complicated by the lack of tractable models for much of the data. By using large samples of independent and identically distributed realizations and statistical learning, it is possible to identify absence of fit through finding a classification rule that can efficiently identify single realizations of each type. The method requires a much wider range of descriptive statistics than are currently in use, and a new concept of model fitting which is derive from how physical laws are judged to fit data.
Parameter Estimation from Censored Samples using the Expectation-Maximization Algorithm

This paper deals with parameter estimation when the data are randomly right censored. The maximum likelihood estimates from censored samples are obtained by using the expectation-maximization (EM) and Monte Carlo EM (MCEM) algorithms. We introduce the concept of the EM and MCEM algorithms and develop parameter estimation methods for a variety of distributions such as normal, Laplace and Rayleigh distributions. These proposed methods are illustrated with three examples.
Robust model-based clustering with gene ranking

Cluster analysis of biological samples using gene expression measurements is a common task which aids the discovery of heterogeneous biological sub-populations having distinct mRNA profiles. Several model-based clustering algorithms have been proposed in which the distribution of gene expression values within each sub-group is assumed to be Gaussian. In the presence of noise and extreme observations, a mixture of Gaussian densities may over-fit and overestimate the true number of clusters. Moreover, commonly used model-based clustering algorithms do not generally provide a mechanism to quantify the relative contribution of each gene to the final partitioning of the data. We propose a penalised mixture of Student's t distributions for model-based clustering and gene ranking. Together with a bootstrap procedure, the proposed approach provides a means for ranking genes according to their contributions to the clustering process. Experimental results show that the algorithm performs well comparably to traditional Gaussian mixtures in the presence of outliers and longer tailed distributions. The algorithm also identifies the true informative genes with high sensitivity, and achieves improved model selection. An illustrative application to breast cancer data is also presented which confirms established tumor subclasses.
General Purpose Convolution Algorithm in S4 Classes by Means of FFT

Object orientation provides a flexible framework for the implementation of the convolution of arbitrary distributions of real-valued random variables. We discuss an algorithm which is based on the fast Fourier transform. It directly applies to lattice-supported distributions. In the case of continuous distributions an additional discretization to a linear lattice is necessary and the resulting lattice-supported distributions are suitably smoothed after convolution. We compare our algorithm to other approaches aiming at a similar generality as to accuracy and speed. In situations where the exact results are known, several checks confirm a high accuracy of the proposed algorithm which is also illustrated for approximations of non-central χ 2 distributions. By means of object orientation this default algorithm is overloaded by more specific algorithms where possible, in particular where explicit convolution formulae are available. Our focus is on R package distr which implements this approach, overloading operator + for convolution; based on this convolution, we define a whole arithmetics of mathematical operations acting on distribution objects, comprising operators +, -, *, /, and ^.
Grouping Priors and the Bayesian Elastic Net

In the literature surrounding Bayesian penalized regression, the two primary choices of prior distribution on the regression coe! cients are zero-mean Gaussian and Laplace. While both have been compared numerically and theoretically, there remains little guidance on which to use in real-life situations. We propose two viable solutions to this problem in the form of prior distributions which combine and compromise between Laplace and Gaussian priors, respectively. Through cross-validation the prior which optimizes prediction performance is automatically selected. We then demonstrate the improved performance of these new prior distributions relative to Laplace and Gaussian priors in both a simulated and experimental environment.
One-sided L\'{e}vy stable distributions

In this paper, we show new representations of one-sided L\'{e}vy stable distributions for irrational L\'{e}vy indices of the type $\left(\frac{p}{q}\right)^{\frac{l_{2}}{l_{1}}}$ which are not covered in \cite{pg1} : for rational L\'{e}vy indices. Furthermore, other equivalent representations for a distribution of a rational L\'{e}vy index is described. We also give a simplest proof for the formulae which cover the cases for rational L\'{e}vy indices. Finally we introduce the concepts of L\'{e}vy smashing and L\'{e}vy-smashed gamma stochastic processes.
Convergence of adaptive and interacting Markov chain Monte Carlo algorithms

Adaptive and interacting Markov chain Monte Carlo algorithms (MCMC) have been recently introduced in the literature. These novel simulation algorithms are designed to increase the simulation efficiency to sample complex distributions. Motivated by some recently introduced algorithms (such as the adaptive Metropolis algorithm and the interacting tempering algorithm), we develop a general methodological and theoretical framework to establish both the convergence of the marginal distribution and a strong law of large numbers. This framework weakens the conditions introduced in the pioneering paper by Roberts and Rosenthal [J. Appl. Probab. 44 (2007) 458--475]. It also covers the case when the target distribution $\pi$ is sampled by using Markov transition kernels with a stationary distribution that differs from $\pi$.
Asymptotic equivalence for inference on the volatility from noisy observations

We consider discrete-time observations of a continuous martingale under measurement error. This serves as a fundamental model for high-frequency data in finance, where an efficient price process is observed under microstructure noise. It is shown that this nonparametric model is in Le Cam's sense asymptotically equivalent to a Gaussian shift experiment in terms of the square root of the volatility function σ and a nonstandard noise level. As an application, new rate-optimal estimators of the volatility function and simple efficient estimators of the integrated volatility are constructed.
Estimation of safety areas for epidemic spread

In this work we study safety areas in epidemic spred. The aim of this work is, given the evolution of epidemic at time $t$, find a safety set at time $t+h$. This is, a random set $K_{t+h}$ such that the probability that infection reaches $K_{t+h}$ at time $t+h$ is small. More precisely, inspired on the study of epidemic spread, we consider a model in which the measure $\mu_n(A)$ is the incidence -density of infectives individuals- in the set $A$, at time $n$ and $$\mu_{n+1}(A)(\omega)=\int_S{\pi_{n+1}(A;s)(\omega)\mu_n(ds)(\omega)}, {for any Borel set} A, $$ with random transition kernels of the form $$\pi_n(.;.)(\omega)=\Pi(.;.)(\xi_n(\omega),Y_n(\omega)),$$ where $\xi$, $Y$ satisfy some ergodic conditions. The support of $\mu_n$ is called $S_n$. We also assume that $S_0$ is compact with regular border and that for any $x,y$ the kernel $\Pi(.;.)(x,y)$ has compact support. A random set $K_{n+1}$ is a safety area of level $\alpha$ if: [{$i$)}] $K_{n+1}$ {\rm is a function of} $S_0, S_1, . ,S_n. $ [{$ii$)}] $P(K_{n+1} \cap S_{n+1} \neq \emptyset)\leq \alpha. $ We present a method to find these safety areas and some related results.
On dimension folding of matrix- or array-valued statistical objects

We consider dimension reduction for regression or classification in which the predictors are matrix- or array-valued. This type of predictor arises when measurements are obtained for each combination of two or more underlying variables--for example, the voltage measured at different channels and times in electroencephalography data. For these applications, it is desirable to preserve the array structure of the reduced predictor (e.g., time versus channel), but this cannot be achieved within the conventional dimension reduction formulation. In this paper, we introduce a dimension reduction method, to be called dimension folding, for matrix- and array-valued predictors that preserves the array structure. In an application of dimension folding to an electroencephalography data set, we correctly classify 97 out of 122 subjects as alcoholic or nonalcoholic based on their electroencephalography in a cross-validation sample.
Discriminant Analysis with Adaptively Pooled Covariance

Linear and Quadratic Discriminant analysis (LDA/QDA) are common tools for classification problems. For these methods we assume observations are normally distributed within group. We estimate a mean and covariance matrix for each group and classify using Bayes theorem. With LDA, we estimate a single, pooled covariance matrix, while for QDA we estimate a separate covariance matrix for each group. Rarely do we believe in a homogeneous covariance structure between groups, but often there is insufficient data to separately estimate covariance matrices. We propose L1- PDA, a regularized model which adaptively pools elements of the precision matrices. Adaptively pooling these matrices decreases the variance of our estimates (as in LDA), without overly biasing them. In this paper, we propose and discuss this method, give an efficient algorithm to fit it for moderate sized problems, and show its efficacy on real and simulated datasets.
Probabilistic Recovery of Multiple Subspaces in Point Clouds by Geometric lp Minimization

We assume data independently sampled from a mixture distribution on the unit ball of the D-dimensional Euclidean space with K+1 components: the first component is a uniform distribution on that ball representing outliers and the other K components are uniform distributions along K d-dimensional linear subspaces restricted to that ball. We study both the simultaneous recovery of all K underlying subspaces and the recovery of the best l0 subspace (i.e., with largest number of points) by minimizing the lp-averaged distances of data points from d-dimensional subspaces of the D-dimensional space. Unlike other lp minimization problems, this minimization is non-convex for all p>0 and thus requires different methods for its analysis. We show that if 0 1 and p>1, then we show that both all underlying subspaces and the best l0 subspace cannot be recovered and even nearly recovered. Further relaxations are also discussed. We use the results of this paper for partially justifying recent effective algorithms for modeling data by mixtures of multiple subspaces as well as for discussing the effect of using variants of lp minimizations in RANSAC-type strategies for single subspace recovery.
Semi-parametric dynamic time series modelling with applications to detecting neural dynamics

This paper illustrates novel methods for nonstationary time series modeling along with their applications to selected problems in neuroscience. These methods are semi-parametric in that inferences are derived by combining sequential Bayesian updating with a non-parametric change-point test. As a test statistic, we propose a Kullback-Leibler (KL) divergence between posterior distributions arising from different sets of data. A closed form expression of this statistic is derived for exponential family models, whereas standard Markov chain Monte Carlo output is used to approximate its value and its critical region for more general models. The behavior of one-step ahead predictive distributions under our semi-parametric framework is described analytically for a dynamic linear time series model. Conditions under which our approach reduces to fully parametric state-space modeling are also illustrated. We apply our methods to estimating the functional dynamics of a wide range of neural data, including multi-channel electroencephalogram recordings, longitudinal behavioral experiments and in-vivo multiple spike trains recordings. The estimated dynamics are related to the presentation of visual stimuli, to the evaluation of a learning performance and to changes in the functional connections between neurons over a sequence of experiments.
Accelerated Monte Carlo estimation of failure probabilities in output of monotone computer codes

The problem of estimating the probability p=P(g(X<0) is considered when X represents a multivariate stochastic input of a monotone function g. First, a heuristic method to bound p is formally described, involving a specialized design of numerical experiments. Then a statistical estimation of p is considered based on a sequential stochastic exploration of the input space. A maximum likelihood estimator of p based on successive dependent Bernoulli data is defined and its theoretical convergence properties are studied. Under intuitive or mild conditions, the estimation is faster and more robust than the traditional Monte Carlo approach, therefore adapted to time-consuming computer codes g. The main result of the paper is related to the variance of the estimator. It appears as a new baseline measure of efficiency under monotone constraints, which could play a similar role to the usual Monte Carlo estimator variance in unconstrained frameworks. Furthermore the bias of the estimator is shown to be corrigible via bootstrap heuristics. The behavior of the method is illustrated by numerical tests led on a class of toy examples and a more realistic hydraulic case-study. Keywords : monotone function, deterministic computer codes, Monte Carlo acceleration, failure probability
Confounding of three binary-variables counterfactual model

Confounding of three binary-variables counterfactual model is discussed in this paper. According to the effect between the control variable and the covariate variable, we investigate three counterfactual models: the control variable is independent of the covariate variable, the control variable has the effect on the covariate variable and the covariate variable affects the control variable. Using the ancillary information based on conditional independence hypotheses, the sufficient conditions to determine whether the covariate variable is an irrelevant factor or a confounder in each counterfactual model are obtained.
Twisted particle filters

We investigate sampling laws for particle algorithms and the influence of these laws on the efficiency of particle approximations of marginal likelihoods in hidden Markov models. Among a broad class of candidates we characterize the essentially unique family of particle system transition kernels which is optimal with respect to an asymptotic-in-time variance growth rate criterion. The sampling structure of the algorithm defined by these optimal transitions turns out to be only subtly different from standard algorithms and yet the fluctuation properties of the estimates it provides can be dramatically different. The structure of the optimal transition suggests a new class of algorithms, which we term "twisted" particle filters and which we validate with asymptotic analysis of a more traditional nature, in the regime where the number of particles tends to infinity.
Discussion of “Impact of Frequentist and Bayesian Methods on Survey Sampling Practice: A Selective Appraisal” by J. N. K. Rao

I would like to congratulate Professor Rao on having produced an overview of survey methodology which is at the same time a broad-ranging prospectus of cur rent research and also an impressive retrospective from a modern viewpoint of the early historical develop ments. He shows us in broad terms where the various approaches to survey methodology have been success ful and where they cannot quite be relied upon without further development. Most of the paper is not specifically directed at contrasting the Bayesian and frequentist viewpoints. The most important distinctions for Rao seem to be between model-dependent and design-based methods, and Bayes methods are faulted in Rao's chosen ter rain of "the large-scale production of official statis tics from complex surveys" primarily for using models where models are not absolutely necessary. He takes for granted that models will be used in adjusting for nonresponse, in his formulation largely through cal ibration, and in small area estimation. The faults he finds with unnecessarily model-dependent survey es timation methods are:
Error bounds for Metropolis–Hastings algorithms applied to perturbations of Gaussian measures in high dimensions

The Metropolis-adjusted Langevin algorithm (MALA) is a Metropolis-Hastings method for approximate sampling from continuous distributions. We derive upper bounds for the contraction rate in Kantorovich-Rubinstein-Wasserstein distance of the MALA chain with semi-implicit Euler proposals applied to log-concave probability measures that have a density w.r. t. a Gaussian reference measure. For sufficiently "regular" densities, the estimates are dimension-independent, and they hold for sufficiently small step sizes $h$ that do not depend on the dimension either. In the limit $h\downarrow0$, the bounds approach the known optimal contraction rates for overdamped Langevin diffusions in a convex potential. A similar approach also applies to Metropolis-Hastings chains with Ornstein-Uhlenbeck proposals. In this case, the resulting estimates are still independent of the dimension but less optimal, reflecting the fact that MALA is a higher order approximation of the diffusion limit than Metropolis-Hastings with Ornstein-Uhlenbeck proposals.
Sequential Markov Chain Monte Carlo

We propose a sequential Markov chain Monte Carlo (SMCMC) algorithm to sample from a sequence of probability distributions, corresponding to posterior distributions at different times in on-line applications. SMCMC proceeds as in usual MCMC but with the stationary distribution updated appropriately each time new data arrive. SMCMC has advantages over sequential Monte Carlo (SMC) in avoiding particle degeneracy issues. We provide theoretical guarantees for the marginal convergence of SMCMC under various settings, including parametric and nonparametric models. The proposed approach is compared to competitors in a simulation study. We also consider an application to on-line nonparametric regression.
The Gamma Lasso

The statistics literature of the past 15 years has established many favorable properties for sparse diminishing-bias regularization: techniques which can roughly be understood as providing estimation under penalty functions spanning the range of concavity between $L_0$ and $L_1$ norms. However, lasso $L_1$-regularized estimation remains the standard tool for industrial `Big Data' applications because of its minimal computational cost and the presence of easy-to-apply rules for penalty selection. In response, this article proposes a simple new algorithm framework that requires no more computation than a lasso path: the path of one-step estimators (POSE) does $L_1$ penalized regression estimation on a grid of decreasing penalties, but adapts coefficient-specific weights to decrease as a function of the coefficient estimated in the previous path step. This provides sparse diminishing-bias regularization at no extra cost over the fastest lasso algorithms. Moreover, our `gamma lasso' implementation of POSE is accompanied by a reliable heuristic for the fit degrees of freedom, so that standard information criteria can be applied in penalty selection. We also provide novel results on the distance between weighted-$L_1$ and $L_0$ penalized predictors; this allows us to build intuition about POSE and other diminishing-bias regularization schemes. The methods and results are illustrated in extensive simulations and in application of logistic regression to evaluating the performance of hockey players.
False Variable Selection Rates in Regression

There has been recent interest in extending the ideas of False Discovery Rates (FDR) to variable selection in regression settings. Traditionally the FDR in these settings has been defined in terms of the coefficients of the full regression model. Recent papers have struggled with controlling this quantity when the predictors are correlated. This paper shows that this full model definition of FDR suffers from unintuitive and potentially undesirable behavior in the presence of correlated predictors. We propose a new false selection error criterion, the False Variable Rate (FVR), that avoids these problems and behaves in a more intuitive manner. We discuss the behavior of this criterion and how it compares with the traditional FDR, as well as presenting guidelines for determining which is appropriate in a particular setting. Finally, we present a simple estimation procedure for FVR in stepwise variable selection. We analyze the performance of this estimator and draw connections to recent estimators in the literature.
Detecting gradual changes in locally stationary processes

In a wide range of applications, the stochastic properties of the observed time series change over time. The changes often occur gradually rather than abruptly: the properties are (approximately) constant for some time and then slowly start to change. In such situations, it is frequently of interest to locate the time point where the properties start to vary. In contrast to the analysis of abrupt changes, methods for detecting smooth or gradual change points are less developed and often require strong parametric assumptions. In this paper, we develop a fully nonparametric method to estimate a smooth change point in a locally stationary framework. We set up a general procedure which allows to deal with a wide variety of stochastic properties including the mean, (auto)covariances and higher-order moments. The theoretical part of the paper establishes the convergence rate of the new estimator. In addition, we examine its finite sample performance by means of a simulation study and illustrate the methodology by applications to temperature and financial return data.
Generalized Negative Binomial Processes and the Representation of Cluster Structures

The paper introduces the concept of a cluster structure to define a joint distribution of the sample size and its exchangeable random partitions. The cluster structure allows the probability distribution of the random partitions of a subset of the sample to be dependent on the sample size, a feature not presented in a partition structure. A generalized negative binomial process count-mixture model is proposed to generate a cluster structure, where in the prior the number of clusters is finite and Poisson distributed and the cluster sizes follow a truncated negative binomial distribution. The number and sizes of clusters can be controlled to exhibit distinct asymptotic behaviors. Unique model properties are illustrated with example clustering results using a generalized Polya urn sampling scheme. The paper provides new methods to generate exchangeable random partitions and to control both the cluster-number and cluster-size distributions.