Miracle-dz/UniFAR-FTUs huggingface dataset:
Variable Importance Assessments and Backward Variable Selection for High-Dimensional Data

Variable selection in high-dimensional scenarios is of great interested in statistics. One application involves identifying differentially expressed genes in genomic analysis. Existing methods for addressing this problem have some limits or disadvantages. In this paper, we propose distance based variable importance measures to deal with these problems, which is inspired by the Multi-Response Permutation Procedure (MRPP). The proposed variable importance assessments can effectively measure the importance of an individual dimension by quantifying its influence on the differences between multivariate distributions. A backward selection algorithm is developed that can be used in high-dimensional variable selection to discover important variables. Both simulations and real data applications demonstrate that our proposed method enjoys good properties and has advantages over other methods.
Contrast function estimation for the drift parameter of ergodic jump diffusion process

In this paper, we consider an ergodic diffusion process with jumps whose drift coefficient depends on an unknown parameter. We suppose that the process is discretely observed. We introduce an estimator based on a contrast function, which is efficient without requiring any conditions on the rate at which the step discretization goes to zero, and where we allow the observed process to have nonsummable jumps. This extends earlier results where the condition on the step discretization was needed and where the process was supposed to have summable jumps. In general situations, our contrast function is not explicit and one has to resort to some approximation. In the case of a finite jump activity, we propose explicit approximations of the contrast function such that the efficient estimation of the drift parameter is feasible. This extends the results obtained by Kessler in the case of continuous processes.
Learning out of leaders

Summary. The paper investigates the estimation problem in a regression‐type model. To be able to deal with potential high dimensions, we provide a procedure called LOL—for learning out of leaders—with no optimization step. LOL is an autodriven algorithm with two thresholding steps. A first adaptive thresholding helps to select leaders among the initial regressors to obtain a first reduction of dimensionality. Then a second thresholding is performed on the linear regression on the leaders. The consistency of the procedure is investigated. Exponential bounds are obtained, leading to minimax and adaptive results for a wide class of sparse parameters, with (quasi) no restriction on the number p of possible regressors. An extensive computational experiment is conducted to emphasize the practical good performances of LOL.
Multinomial Inverse Regression for Text Analysis

Text data, including speeches, stories, and other document forms, are often connected to sentiment variables that are of interest for research in marketing, economics, and elsewhere. It is also very high dimensional and difficult to incorporate into statistical analyses. This article introduces a straightforward framework of sentiment-sufficient dimension reduction for text data. Multinomial inverse regression is introduced as a general tool for simplifying predictor sets that can be represented as draws from a multinomial distribution, and we show that logistic regression of phrase counts onto document annotations can be used to obtain low-dimensional document representations that are rich in sentiment information. To facilitate this modeling, a novel estimation technique is developed for multinomial logistic regression with very high-dimensional response. In particular, independent Laplace priors with unknown variance are assigned to each regression coefficient, and we detail an efficient routine for maximization of the joint posterior over coefficients and their prior scale. This “gamma-lasso” scheme yields stable and effective estimation for general high-dimensional logistic regression, and we argue that it will be superior to current methods in many settings. Guidelines for prior specification are provided, algorithm convergence is detailed, and estimator properties are outlined from the perspective of the literature on nonconcave likelihood penalization. Related work on sentiment analysis from statistics, econometrics, and machine learning is surveyed and connected. Finally, the methods are applied in two detailed examples and we provide out-of-sample prediction studies to illustrate their effectiveness.
Functional CLT for sample covariance matrices

Using Bernstein polynomial approximations, we prove the central limit theorem for linear spectral statistics of sample covariance matrices, indexed by a set of functions with continuous fourth order derivatives on an open interval including $[(1-\sqrt{y})^2,(1+\sqrt{y})^2]$, the support of the Mar\u{c}enko--Pastur law. We also derive the explicit expressions for asymptotic mean and covariance functions.
The Loss Rank Criterion for Variable Selection in Linear Regression Analysis

Abstract. Lasso and other regularization procedures are attractive methods for variable selection, subject to a proper choice of shrinkage parameter. Given a set of potential subsets produced by a regularization algorithm, a consistent model selection criterion is proposed to select the best one among this preselected set. The approach leads to a fast and efficient procedure for variable selection, especially in high‐dimensional settings. Model selection consistency of the suggested criterion is proven when the number of covariates d is fixed. Simulation studies suggest that the criterion still enjoys model selection consistency when d is much larger than the sample size. The simulations also show that our approach for variable selection works surprisingly well in comparison with existing competitors. The method is also applied to a real data set.
Local Linear Regression on Manifolds and Its Geometric Interpretation

High-dimensional data analysis has been an active area, and the main focus areas have been variable selection and dimension reduction. In practice, it occurs often that the variables are located on an unknown, lower-dimensional nonlinear manifold. Under this manifold assumption, one purpose of this article is regression and gradient estimation on the manifold, and another is developing a new tool for manifold learning. As regards the first aim, we suggest directly reducing the dimensionality to the intrinsic dimension d of the manifold, and performing the popular local linear regression (LLR) on a tangent plane estimate. An immediate consequence is a dramatic reduction in the computational time when the ambient space dimension p ≫ d. We provide rigorous theoretical justification of the convergence of the proposed regression and gradient estimators by carefully analyzing the curvature, boundary, and nonuniform sampling effects. We propose a bandwidth selector that can handle heteroscedastic errors. With reference to the second aim, we analyze carefully the asymptotic behavior of our regression estimator both in the interior and near the boundary of the manifold, and make explicit its relationship with manifold learning, in particular estimating the Laplace–Beltrami operator of the manifold. In this context, we also make clear that it is important to use a smaller bandwidth in the tangent plane estimation than in the LLR. A simulation study and applications to the Isomap face data and a clinically computed tomography scan dataset are used to illustrate the computational speed and estimation accuracy of our methods. Supplementary materials for this article are available online.
An adaptively weighted statistic for detecting differential gene expression when combining multiple transcriptomic studies

Global expression analyses using microarray technologies are becoming more common in genomic research, therefore, new statistical challenges associated with combining information from multiple studies must be addressed. In this paper we will describe our proposal for an adaptively weighted (AW) statistic to combine multiple genomic studies for detecting differentially expressed genes. We will also present our results from comparisons of our proposed AW statistic to Fisher's equally weighted (EW), Tippett's minimum p-value (minP) and Pearson's (PR) statistics. Due to the absence of a uniformly powerful test, we used a simplified Gaussian scenario to compare the four methods. Our AW statistic consistently produced the best or near-best power for a range of alternative hypotheses. AW-obtained weights also have the additional advantage of filtering discordant biomarkers and providing natural detected gene categories for further biological investigation. Here we will demonstrate the superior performance of our proposed AW statistic based on a mix of power analyses, simulations and applications using data sets for multi-tissue energy metabolism mouse, multi-lab prostate cancer and lung cancer.
Kernel deconvolution estimation for random fields

In this work, we establish the asymptotic normality of the deconvolution kernel density estimator in the context of strongly mixing random fields. Only minimal conditions on the bandwidth parameter are required and a simple criterion on the strong mixing coefficients is provided. Our approach is based on the Lindeberg's method rather than on Bernstein's technique and coupling arguments widely used in previous works on nonparametric estimation for spatial processes. We deal also with nonmixing random fields which can be written as a (nonlinear) functional of i.i. d. random fields by considering the physical dependence measure coefficients introduced by Wu (2005).
A multiple comparison procedure for dose‐finding trials with subpopulations

Identifying subgroups of patients with an enhanced response to a new treatment has become an area of increased interest in the last few years. When there is knowledge about possible subpopulations with an enhanced treatment effect before the start of a trial it might be beneficial to set up a testing strategy, which tests for a significant treatment effect not only in the full population, but also in these prespecified subpopulations. In this paper, we present a parametric multiple testing approach for tests in multiple populations for dose‐finding trials. Our approach is based on the MCP‐Mod methodology, which uses multiple comparison procedures (MCPs) to test for a dose–response signal, while considering multiple possible candidate dose–response shapes. Our proposed methods allow for heteroscedastic error variances between populations and control the family‐wise error rate over tests in multiple populations and for multiple candidate models. We show in simulations that the proposed multipopulation testing approaches can increase the power to detect a significant dose–response signal over the standard single‐population MCP‐Mod, when the specified subpopulation has an enhanced treatment effect.
A Sparse SVD Method for High-dimensional Data

We present a new computational approach to approximating a large, noisy data table by a low-rank matrix with sparse singular vectors. The approximation is obtained from thresholded subspace iterations that produce the singular vectors simultaneously, rather than successively as in competing proposals. We introduce novel ways to estimate thresholding parameters which obviate the need for computationally expensive cross-validation. We also introduce a way to sparsely initialize the algorithm for computational savings that allow our algorithm to outperform the vanilla SVD on the full data table when the signal is sparse. A comparison with two existing sparse SVD methods suggests that our algorithm is computationally always faster and statistically always at least comparable to the better of the two competing algorithms.
Non‐Parametric Estimation of the Conditional Distribution of the Interjumping Times for Piecewise‐Deterministic Markov Processes

This paper presents a non‐parametric method for estimating the conditional density associated to the jump rate of a piecewise‐deterministic Markov process. In our framework, the estimation needs only one observation of the process within a long time interval. Our method relies on a generalization of Aalen's multiplicative intensity model. We prove the uniform consistency of our estimator, under some reasonable assumptions related to the primitive characteristics of the process. A simulation study illustrates the behaviour of our estimator.
An essay on copula modelling for discrete random vectors; or how to pour new wine into old bottles

Copulas have now become ubiquitous statistical tools for describing, analysing and modelling dependence between random variables. Sklar's theorem, "the fundamental theorem of copulas", makes a clear distinction between the continuous case and the discrete case, though. In particular, the copula of a discrete random vector is not identifiable, which causes serious inconsistencies. In spite of this, downplaying statements are widespread in the related literature, and copula methods are used for modelling dependence between discrete variables. This paper calls to reconsidering the soundness of copula modelling for discrete data. It suggests a more fundamental construction which allows copula ideas to smoothly carry over to the discrete case. Actually it is an attempt at rejuvenating some century-old ideas of Udny Yule, who mentioned a similar construction a long time before copulas got in fashion.
Mean Field Analysis of Deep Neural Networks

We analyze multilayer neural networks in the asymptotic regime of simultaneously (a) large network sizes and (b) large numbers of stochastic gradient descent training iterations. We rigorously establish the limiting behavior of the multilayer neural network output. The limit procedure is valid for any number of hidden layers, and it naturally also describes the limiting behavior of the training loss. The ideas that we explore are to (a) take the limits of each hidden layer sequentially and (b) characterize the evolution of parameters in terms of their initialization. The limit satisfies a system of deterministic integro-differential equations. The proof uses methods from weak convergence and stochastic analysis. We show that, under suitable assumptions on the activation functions and the behavior for large times, the limit neural network recovers a global minimum (with zero loss for the objective function).
A General Theory of Concave Regularization for High-Dimensional Sparse Estimation Problems

Concave regularization methods provide natural procedures for sparse recovery. However, they are difficult to analyze in the high dimensional setting. Only recently a few sparse recovery results have been established for some specific local solutions obtained via specialized numerical procedures. Still, the fundamental relationship between these solutions such as whether they are identical or their relationship to the global minimizer of the underlying nonconvex formulation is unknown. The current paper fills this conceptual gap by presenting a general theoretical framework showing that under appropriate conditions, the global solution of nonconvex regularization leads to desirable recovery performance; moreover, under suitable conditions, the global solution corresponds to the unique sparse local solution, which can be obtained via different numerical procedures. Under this unified framework, we present an overview of existing results and discuss their connections. The unified view of this work leads to a more satisfactory treatment of concave high dimensional sparse estimation procedures, and serves as guideline for developing further numerical procedures for concave regularization.
Critical Properties of $S^{4}$ System Restudied via Generalized Migdal-Kadanoff Bond-moving Renormalization

We study the critical properties of the spin-continuous $S^{4}$ system on the typical translational invariant triangular lattices by combining the recently-developed generalized Migdal-Kadanoff bond-moving recursion procedures with the cumulative expansion technique. In three different cases of nearest-neighbor, next nearest neighbor and external field we obtain the critical points and further calculate the critical exponents according to the scaling theory. In all case it is found that there exists three fixed points. The correlation length critical exponents obtained near the Wilson-Fisher fixed points are found getting smaller and smaller with the increasing of the system complexity. Others are found similar to the results of the classical Gaussian model and different from those of the Ising system.
Goodness of fit tests for a class of Markov random field models

This paper develops goodness of fit statistics that can be used to formally assess Markov random field models for spatial data, when the model distributions are discrete or continuous and potentially parametric. Test statistics are formed from generalized spatial residuals which are collected over groups of nonneighboring spatial observations, called concliques. Under a hypothesized Markov model structure, spatial residuals within each conclique are shown to be independent and identically distributed as uniform variables. The information from a series of concliques can be then pooled into goodness of fit statistics. Under some conditions, large sample distributions of these statistics are explicitly derived for testing both simple and composite hypotheses, where the latter involves additional parametric estimation steps. The distributional results are verified through simulation, and a data example illustrates the method for model assessment.
De-biasing the lasso with degrees-of-freedom adjustment

This paper studies schemes to de-bias the Lasso in sparse linear regression where the goal is to estimate and construct confidence intervals for a low-dimensional projection of the unknown coefficient vector in a preconceived direction $a_0$. We assume that the design matrix has iid Gaussian rows with known covariance matrix $\Sigma$. Our analysis reveals that previous propositions to de-bias the Lasso require a modification in order to enjoy asymptotic efficiency in a full range of the level of sparsity. This modification takes the form of a degrees-of-freedom adjustment that accounts for the dimension of the model selected by the Lasso. Let $s_0$ denote the number of nonzero coefficients of the true coefficient vector. The unadjusted de-biasing schemes proposed in previous studies enjoys efficiency if $s_0\lll n^{2/3}$, up to logarithmic factors. However, if $s_0\ggg n^{2/3}$, the unadjusted scheme cannot be efficient in certain directions $a_0$. In the latter regime, it it necessary to modify existing procedures by an adjustment that accounts for the degrees-of-freedom of the Lasso. The proposed degrees-of-freedom adjustment grants asymptotic efficiency for any direction $a_0$. This holds under a Sparse Riecz Condition on the covariance matrix $\Sigma$ and the sample size requirement $s_0/p\to0$ and $s_0\log(p/s_0)/n\to0$. Our analysis also highlights that the degrees-of-freedom adjustment is not necessary when the initial bias of the Lasso in the direction $a_0$ is small, which is granted under more stringent conditions on $\Sigma^{-1}$. This explains why the necessity of degrees-of-freedom adjustment did not appear in some previous studies. The main proof argument involves a Gaussian interpolation path similar to that used to derive Slepian's lemma. It yields a sharp $\ell_\infty$ error bound for the Lasso under Gaussian design which is of independent interest.
Structured, Sparse Aggregation

This article introduces a method for aggregating many least-squares estimators so that the resulting estimate has two properties: sparsity and structure. That is, only a few candidate covariates are used in the resulting model, and the selected covariates follow some structure over the candidate covariates that is assumed to be known a priori. Although sparsity is well studied in many settings, including aggregation, structured sparse methods are still emerging. We demonstrate a general framework for structured sparse aggregation that allows for a wide variety of structures, including overlapping grouped structures and general structural penalties defined as set functions on the set of covariates. We show that such estimators satisfy structured sparse oracle inequalities—their finite sample risk adapts to the structured sparsity of the target. These inequalities reveal that under suitable settings, the structured sparse estimator performs at least as well as, and potentially much better than, a sparse aggregation estimator. We empirically establish the effectiveness of the method using simulation and an application to HIV drug resistance.
Quarticity and other functionals of volatility: Efficient estimation

We consider a multidimensional Ito semimartingale regularly sampled on [0,t] at high frequency $1/\Delta_n$, with $\Delta_n$ going to zero. The goal of this paper is to provide an estimator for the integral over [0,t] of a given function of the volatility matrix. To approximate the integral, we simply use a Riemann sum based on local estimators of the pointwise volatility. We show that although the accuracy of the pointwise estimation is at most $\Delta_n^{1/4}$, this procedure reaches the parametric rate $\Delta_n^{1/2}$, as it is usually the case in integrated functionals estimation. After a suitable bias correction, we obtain an unbiased central limit theorem for our estimator and show that it is asymptotically efficient within some classes of sub models.
Markovian acyclic directed mixed graphs for discrete data

Acyclic directed mixed graphs (ADMGs) are graphs that contain directed ($\rightarrow$) and bidirected ($\leftrightarrow$) edges, subject to the constraint that there are no cycles of directed edges. Such graphs may be used to represent the conditional independence structure induced by a DAG model containing hidden variables on its observed margin. The Markovian model associated with an ADMG is simply the set of distributions obeying the global Markov property, given via a simple path criterion (m-separation). We first present a factorization criterion characterizing the Markovian model that generalizes the well-known recursive factorization for DAGs. For the case of finite discrete random variables, we also provide a parameterization of the model in terms of simple conditional probabilities, and characterize its variation dependence. We show that the induced models are smooth. Consequently, Markovian ADMG models for discrete variables are curved exponential families of distributions.
Multi-stage Convex Relaxation for Feature Selection

A number of recent work studied the effectiveness of feature selection using Lasso. It is known that under the restricted isometry properties (RIP), Lasso does not generally lead to the exact recovery of the set of nonzero coefficients, due to the looseness of convex relaxation. This paper considers the feature selection property of nonconvex regularization, where the solution is given by a multi-stage convex relaxation scheme. Under appropriate conditions, we show that the local solution obtained by this procedure recovers the set of nonzero coefficients without suffering from the bias of Lasso relaxation, which complements parameter estimation results of this procedure.