Miracle-dz/UniFAR-FTUs huggingface dataset:
High-Dimensional Mixed Graphical Models

ABSTRACT While graphical models for continuous data (Gaussian graphical models) and discrete data (Ising models) have been extensively studied, there is little work on graphical models for datasets with both continuous and discrete variables (mixed data), which are common in many scientific applications. We propose a novel graphical model for mixed data, which is simple enough to be suitable for high-dimensional data, yet flexible enough to represent all possible graph structures. We develop a computationally efficient regression-based algorithm for fitting the model by focusing on the conditional log-likelihood of each variable given the rest. The parameters have a natural group structure, and sparsity in the fitted graph is attained by incorporating a group lasso penalty, approximated by a weighted lasso penalty for computational efficiency. We demonstrate the effectiveness of our method through an extensive simulation study and apply it to a music annotation dataset (CAL500), obtaining a sparse and interpretable graphical model relating the continuous features of the audio signal to binary variables such as genre, emotions, and usage associated with particular songs. While we focus on binary discrete variables for the main presentation, we also show that the proposed methodology can be easily extended to general discrete variables.
Adaptive Bayesian Estimation in Indirect Gaussian Sequence Space
 Models

In an indirect Gaussian sequence space model lower and upper bounds are derived for the concentration rate of the posterior distribution of the parameter of interest shrinking to the parameter value $\theta^\circ$ that generates the data. While this establishes posterior consistency, however, the concentration rate depends on both $\theta^\circ$ and a tuning parameter which enters the prior distribution. We first provide an oracle optimal choice of the tuning parameter, i.e., optimized for each $\theta^\circ$ separately. The optimal choice of the prior distribution allows us to derive an oracle optimal concentration rate of the associated posterior distribution. Moreover, for a given class of parameters and a suitable choice of the tuning parameter, we show that the resulting uniform concentration rate over the given class is optimal in a minimax sense. Finally, we construct a hierarchical prior that is adaptive. This means that, given a parameter $\theta^\circ$ or a class of parameters, respectively, the posterior distribution contracts at the oracle rate or at the minimax rate over the class. Notably, the hierarchical prior does not depend neither on $\theta^\circ$ nor on the given class. Moreover, convergence of the fully data-driven Bayes estimator at the oracle or at the minimax rate is established.
{Sparsistency of \ell_1-Regularized M-Estimators}

We consider the model selection consistency or sparsistency of a broad set of $\ell_1$-regularized $M$-estimators for linear and non-linear statistical models in a unified fashion. For this purpose, we propose the local structured smoothness condition (LSSC) on the loss function. We provide a general result giving deterministic sufficient conditions for sparsistency in terms of the regularization parameter, ambient dimension, sparsity level, and number of measurements. We show that several important statistical models have $M$-estimators that indeed satisfy the LSSC, and as a result, the sparsistency guarantees for the corresponding $\ell_1$-regularized $M$-estimators can be derived as simple applications of our main theorem.
On adaptive posterior concentration rates

tion of Holder balls and that moreover achieve our lower bound. We analyse the consequences in terms of asymptotic behaviour of poste- rior credible balls as well as frequentist minimax adaptive estimation. Our results are appended with an upper bound for the contraction rate under an arbitrary loss in a generic regular experiment. The up- per bound is attained for certain sieve priors and enables to extend our results to density estimation.
A Linear-Time Particle Gibbs Sampler for Infinite Hidden Markov Models

Infinite Hidden Markov Models (iHMM's) are an attractive, nonparametric generalization of the classical Hidden Markov Model which can automatically infer the number of hidden states in the system. However, due to the infinite-dimensional nature of transition dynamics performing inference in the iHMM is difficult. In this paper, we present an infinite-state Particle Gibbs (PG) algorithm to resample state trajectories for the iHMM. The proposed algorithm uses an efficient proposal optimized for iHMMs and leverages ancestor sampling to suppress degeneracy of the standard PG algorithm. Our algorithm demonstrates significant convergence improvements on synthetic and real world data sets. Additionally, the infinite-state PG algorithm has linear-time complexity in the number of states in the sampler, while competing methods scale quadratically.
Modeling Covariate Effects in Group Independent Component Analysis with Applications to Functional Magnetic Resonance Imaging

Independent component analysis (ICA) is a powerful computational tool for separating independent source signals from their linear mixtures. ICA has been widely applied in neuroimaging studies to identify and characterize underlying brain functional networks. An important goal in such studies is to assess the effects of subjects' clinical and demographic covariates on the spatial distributions of the functional networks. Currently, covariate effects are not incorporated in existing group ICA decomposition methods. Hence, they can only be evaluated through ad-hoc approaches which may not be accurate in many cases. In this paper, we propose a hierarchical covariate ICA model that provides a formal statistical framework for estimating and testing covariate effects in ICA decomposition. A maximum likelihood method is proposed for estimating the covariate ICA model. We develop two expectation-maximization (EM) algorithms to obtain maximum likelihood estimates. The first is an exact EM algorithm, which has analytically tractable E-step and M-step. Additionally, we propose a subspace-based approximate EM, which can significantly reduce computational time while still retain high model-fitting accuracy. Furthermore, to test covariate effects on the functional networks, we develop a voxel-wise approximate inference procedure which eliminates the needs of computationally expensive covariance estimation. The performance of the proposed methods is evaluated via simulation studies. The application is illustrated through an fMRI study of Zen meditation.
Estimating Net Effects of Treatments in Treatment Sequence without the Assumption of Strongly Ignorable Treatment Assignment

In sequential causal inference, one estimates the causal net effect of treatment in treatment sequence on an outcome after last treatment in the presence of time-dependent covariates between treatments, improves the estimation by the untestable assumption of strongly ignorable treatment assignment, and obtains consistent but non-genuine likelihood-based estimate. In this article, we introduce the net effect of treatment as parameter for the conditional distribution of outcome given all treatments and time-dependent covariates and show that it is equal to the causal net effect of treatment under the assumption of strongly ignorable treatment assignment. As a result, we can estimate the net effect of treatment and evaluate its causal interpretation in two separate steps. The first step is fucus of this article while the second step can be accomplished by usual sensitivity analyses. We construct point parametrization for the conditional outcome distribution in which the parameters of interest are the point effects of single-point treatments. With point parametrization and without the untestable assumption, we estimate the net effect of treatment by maximum likelihood, improve the estimation by testable pattern of the net effect of treatment, and obtain unbiased consistent maximum-likelihood estimate for the net effect of treatment with finite-dimensional pattern.
Compatible weighted proper scoring rules

Many proper scoring rules such as the Brier and log scoring rules implicitly reward a probability forecaster relative to a uniform baseline distribution. Recent work has motivated weighted proper scoring rules, which have an additional baseline parameter. To date two families of weighted proper scoring rules have been introduced, the weighted power and pseudospherical scoring families. These families are compatible with the log scoring rule: when the baseline maximizes the log scoring rule over some set of distributions, the baseline also maximizes the weighted power and pseudospherical scoring rules over the same set. We characterize all weighted proper scoring families and prove a general property: every proper scoring rule is compatible with some weighted scoring family, and every weighted scoring family is compatible with some proper scoring rule. Copyright 2012, Oxford University Press.
Confidence intervals for average success probabilities

We provide Buehler-optimal one-sided and some valid two-sided confidence intervals for the average success probability of a possibly inhomogeneous fixed length Bernoulli chain, based on the number of observed successes. Contrary to some claims in the literature, the one-sided Clopper-Pearson intervals for the homogeneous case are not completely robust here, not even if applied to hypergeometric estimation problems.
Inhomogeneous Wright-Fisher construction of two-parameter Poisson-Dirichlet diffusions

The recently introduced two-parameter Poisson-Dirichlet diffusion extends the infinitely-many-neutral-alleles model, related to Kingman's distribution and to Fleming-Viot processes. The role of the additional parameter has been shown to regulate the clustering structure of the population, but is yet to be fully understood in the way it governs the reproductive process. Here we shed some light on these dynamics by providing a finite-population construction, with finitely-many species, of the two-parameter infinite-dimensional diffusion. The costruction is obtained in terms of Wright-Fisher chains that feature a classical symmetric mutation mechanism and a frequency-dependent immigration, whose inhomogeneity is investigated in detail. The local immigration dynamics are built upon an underlying array of Bernoulli trials and can be described by means of a dartboard experiment and a rank-dependent type distribution. These involve a delicate balance between reinforcement and redistributive effects, among the current species abundances, for the convergence to hold.
Parametric estimation of a one-dimensional ballistic random walk in a Markov environment

We focus on the parametric estimation of the distribution of a Markov environment from the observation of a single trajectory of a one-dimensional nearest-neighbor path evolving in this random environment. In the ballistic case, as the length of the path increases, we prove consistency, asymptotic normality and efficiency of the maximum likelihood estimator. Our contribution is two-fold: we cast the problem into the one of parameter estimation in a hidden Markov model (HMM) and establish that the bivariate Markov chain underlying this HMM is positive Harris recurrent. We provide different examples of setups in which our results apply, in particular that of DNA unzipping model, and we give a simple synthetic experiment to illustrate those results.
Convergence Theorems for the Non-Local Means Filter

In this paper, we establish convergence theorems for the Non-Local Means Filter in removing the additive Gaussian noise. We employ the techniques of "Oracle" estimation to determine the order of the widths of the similarity patches and search windows in the aforementioned filter. We propose a practical choice of these parameters which improve the restoration quality of the filter compared with the usual choice of parameters.
Multiscale change point inference

We introduce a new estimator, the simultaneous multiscale change point estimator SMUCE, for the change point problem in exponential family regression. An unknown step function is estimated by minimizing the number of change points over the acceptance region of a multiscale test at a level α. The probability of overestimating the true number of change points K is controlled by the asymptotic null distribution of the multiscale test statistic. Further, we derive exponential bounds for the probability of underestimating K. By balancing these quantities, α will be chosen such that the probability of correctly estimating K is maximized. All results are even non‐asymptotic for the normal case. On the basis of these bounds, we construct (asymptotically) honest confidence sets for the unknown step function and its change points. At the same time, we obtain exponential bounds for estimating the change point locations which for example yield the minimax rate O(n−1) up to a log‐term. Finally, the simultaneous multiscale change point estimator achieves the optimal detection rate of vanishing signals as n→∞, even for an unbounded number of change points. We illustrate how dynamic programming techniques can be employed for efficient computation of estimators and confidence regions. The performance of the multiscale approach proposed is illustrated by simulations and in two cutting edge applications from genetic engineering and photoemission spectroscopy.
Inference for Sparse Conditional Precision Matrices

Given $n$ i.i. d. observations of a random vector $(X,Z)$, where $X$ is a high-dimensional vector and $Z$ is a low-dimensional index variable, we study the problem of estimating the conditional inverse covariance matrix $\Omega(z) = (E[(X-E[X \mid Z])(X-E[X \mid Z])^T \mid Z=z])^{-1}$ under the assumption that the set of non-zero elements is small and does not depend on the index variable. We develop a novel procedure that combines the ideas of the local constant smoothing and the group Lasso for estimating the conditional inverse covariance matrix. A proximal iterative smoothing algorithm is used to solve the corresponding convex optimization problems. We prove that our procedure recovers the conditional independence assumptions of the distribution $X \mid Z$ with high probability. This result is established by developing a uniform deviation bound for the high-dimensional conditional covariance matrix from its population counterpart, which may be of independent interest. Furthermore, we develop point-wise confidence intervals for individual elements of the conditional inverse covariance matrix. We perform extensive simulation studies, in which we demonstrate that our proposed procedure outperforms sensible competitors. We illustrate our proposal on a S&P 500 stock price data set.
Maximum likelihood estimation for single particle, passive microrheology data with drift

Volume limitations and low yield thresholds of biological fluids have led to widespread use of passive microparticle rheology. The mean-squared-displacement (MSD) statistics of bead position time series (bead paths) are either applied directly to determine the creep compliance [Xu et al., Rheol. Acta 37, 387–398 (1998)] or transformed to determine dynamic storage and loss moduli [Mason and Weitz, Phys. Rev. Lett. 74, 1250–1253 (1995)]. A prevalent hurdle arises when there is a nondiffusive experimental drift in the data. Commensurate with the magnitude of drift relative to diffusive mobility, quantified by a Peclet number, the MSD statistics are distorted, and thus the path data must be “corrected” for drift. The standard approach is to estimate and subtract the drift from particle paths, and then calculate MSD statistics. We present an alternative, parametric approach using maximum likelihood estimation that simultaneously fits drift and diffusive model parameters from the path data; the MSD statistics (and consequently the compliance and dynamic moduli) then follow directly from the best-fit model. We illustrate and compare both methods on simulated path data over a range of Peclet numbers, where exact answers are known. We choose fractional Brownian motion as the numerical model, because it affords tunable, subdiffusive MSD statistics consistent with typical 30 s long, experimental observations of microbeads in several biological fluids. Finally, we apply and compare both methods on data from human bronchial epithelial cell culture mucus.
Modeling Population Structure Under Hierarchical Dirichlet Processes

We propose a Bayesian nonparametric model to infer population admixture, extending the Hierarchical Dirichlet Process to allow for correlation between loci due to Linkage Disequilibrium. Given multilocus genotype data from a sample of individuals, the model allows inferring classifying individuals as unadmixed or admixed, inferring the number of subpopulations ancestral to an admixed population and the population of origin of chromosomal regions. Our model does not assume any specific mutation process and can be applied to most of the commonly used genetic markers. We present a MCMC algorithm to perform posterior inference from the model and discuss methods to summarise the MCMC output for the analysis of population admixture. We demonstrate the performance of the proposed model in simulations and in a real application, using genetic data from the EDAR gene, which is considered to be ancestry-informative due to well-known variations in allele frequency as well as phenotypic effects across ancestry. The structure analysis of this dataset leads to the identification of a rare haplotype in Europeans.
Distribution and Symmetric Distribution Regression Model for Interval-Valued Variables

Symbolic Data Analysis works with variables for which each unit or class of units takes a finite set of values/categories, an interval or a distribution (an histogram, for instance). When to each observation corresponds an empirical distribution, we have a histogram-valued variable; it reduces to the case of an interval-valued variable if each unit takes values on only one interval with probability equal to one. Distribution and Symmetric Distribution is a linear regression model proposed for histogram-valued variables that may be particularized to interval-valued variables. This model is defined for n explicative variables and is based on the distributions considered within the intervals. In this paper we study the special case where the Uniform distribution is assumed in each observed interval. As in the classical case, a goodness-of-fit measure is deduced from the model. Some illustrative examples are presented. A simulation study allows discussing interpretations of the behavior of the model for this variable type.
The Law of Total Odds

The law of total probability may be deployed in binary classication exercises to estimate the unconditional class probabilities if the class proportions in the training set are not representative of the population class proportions. We argue that this is not a conceptually good approach and suggest an alternative based on the new law of total odds. We quantify the bias of the total probability estimator of the unconditional class probabilities and show that the total odds estimator is unbiased. The law of total odds can also be used for transforming the conditional class probabilities if exogenous estimates of the unconditional class probabilities of the population are
Efficiency and computability of MCMC with Langevin, Hamiltonian, and other matrix-splitting proposals

We analyse computational efficiency of Metropolis-Hastings algorithms with AR(1) process proposals. These proposals include, as a subclass, discretized Langevin diffusion (e.g. MALA) and discretized Hamiltonian dynamics (e.g. HMC). By including the effect of Metropolis-Hastings we extend earlier work by Fox and Parker, who used matrix splitting techniques to analyse the performance and improve efficiency of AR(1) processes for targeting Gaussian distributions. Our research enables analysis of MCMC methods that draw samples from non-Gaussian target distributions by using AR(1) process proposals in Metropolis-Hastings algorithms, by analysing the matrix splitting of the precision matrix for a local Gaussian approximation of the non-Gaussian target.
Algorithm for the maximum likelihood estimation of the parameters of the truncated normal and lognormal distributions

This paper describes a simple procedure to estimate the parameters of the univariate truncated normal and lognormal distributions by maximum likelihood. It starts from a reparameterization of the lognormal that was previously introduced by the author and is especially useful when the lognormal is close to a power law, which is a limiting case of the first distribution. One of the new parameters quantifies the distance from the power law, and vanishes when the power law gives a sufficient description of the data. At this point, the other parameter equals the exponent of the power law. In contrast, when using the standard parameterization, the parameters of the lognormal diverge in the neighborhood of the power law. Whether or not we are in this neighborhood, the new parameters have properties that ease the process of estimation.
FIXED POINTS OF THE EM ALGORITHM AND NONNEGATIVE RANK BOUNDARIES

Mixtures of $r$ independent distributions for two discrete random variables can be represented by matrices of nonnegative rank $r$. Likelihood inference for the model of such joint distributions leads to problems in real algebraic geometry that are addressed here for the first time. We characterize the set of fixed points of the Expectation-Maximization algorithm, and we study the boundary of the space of matrices with nonnegative rank at most $3$. Both of these sets correspond to algebraic varieties with many irreducible components.
Identifying the number of clusters in discrete mixture models

Research on cluster analysis for categorical data continues to develop, with new clustering algorithms being proposed. However, in this context, the determination of the number of clusters is rarely addressed. In this paper, we propose a new approach in which clustering of categorical data and the estimation of the number of clusters is carried out simultaneously. Assuming that the data originate from a finite mixture of multinomial distributions, we develop a method to select the number of mixture components based on a minimum message length (MML) criterion and implement a new expectationmaximization (EM) algorithm to estimate all the model parameters. The proposed EM-MML approach, rather than selecting one among a set of pre-estimated candidate models (which requires running EM several times), seamlessly integrates estimation and model selection in a single algorithm. The performance of the proposed approach is compared with other well-known criteria (such as the Bayesian information criterion–BIC), resorting to synthetic data and to two real applications from the European Social Survey. The EM-MML computation time is a clear advantage of the proposed method. Also, the real data solutions are much more parsimonious than the solutions provided by competing methods, which reduces the risk of model order overestimation and increases interpretability.