Miracle-dz/UniFAR-FTUs huggingface dataset:
Algorithms for Envelope Estimation II

We propose a new algorithm for envelope estimation, along with a new root n consistent method for computing starting values. The new algorithm, which does not require optimization over a Grassmannian, is shown by simulation to be much faster and typically more accurate that the best existing algorithm proposed by Cook and Zhang (2015c).
$L^p$-Wasserstein distance for stochastic differential equations driven by L\'{e}vy processes

Coupling by reflection mixed with synchronous coupling is constructed for a class of stochastic differential equations (SDEs) driven by L\'{e}vy noises. As an application, we establish the exponential contractivity of the associated semigroups $(P_t)_{t\ge0}$ with respect to the standard $L^p$-Wasserstein distance for all $p\in[1,\infty)$. In particular, consider the following SDE: \[\mathrm{d}X_t=\mathrm{d}Z_t+b(X_t)\,\mathrm{d}t,\] where $(Z_t)_{t\ge0}$ is a symmetric $\alpha$-stable process on $\mathbb{R}^d$ with $\alpha\in(1,2)$. We show that if the drift term $b$ satisfies that for any $x,y\in\mathbb{R}^d$, \[\bigl\langle b(x)-b(y),x-y\bigr\rangle\le\cases{K_1|x-y|^2,\qquad |x-y|\le L_0;\cr -K_2|x-y|^{\theta},\qquad |x-y|>L_0}\] holds with some positive constants $K_1$, $K_2$, $L_0>0$ and $\theta\ge2$, then there is a constant $\lambda:=\lambda(\theta,K_1,K_2,L_0)>0$ such that for all $p\in[1,\infty)$, $t>0$ and $x,y\in\mathbb{R}^d$, \[W_p(\delta_xP_t,\delta_yP_t)\le C(p,\theta,K_1,K_2,L_0)\mathrm{e}^{-\lambda t/p}\biggl[\frac{|x-y|^{1/p}\vee|x-y|}{1+|x-y|{\mathbf{1}}_{(1,\infty )\times (2,\infty)}(t,\theta)}\biggr]. \]
Always Valid Inference: Bringing Sequential Analysis to A/B Testing

A/B tests are typically analyzed via frequentist p-values and confidence intervals; but these inferences are wholly unreliable if users endogenously choose samples sizes by *continuously monitoring* their tests. We define *always valid* p-values and confidence intervals that let users try to take advantage of data as fast as it becomes available, providing valid statistical inference whenever they make their decision. Always valid inference can be interpreted as a natural interface for a sequential hypothesis test, which empowers users to implement a modified test tailored to them. In particular, we show in an appropriate sense that the measures we develop tradeoff sample size and power efficiently, despite a lack of prior knowledge of the user's relative preference between these two goals. We also use always valid p-values to obtain multiple hypothesis testing control in the sequential context. Our methodology has been implemented in a large scale commercial A/B testing platform to analyze hundreds of thousands of experiments to date.
On Some New Modifications of Ridge Estimators

Ridge estimator is an alternative to ordinary least square estimator when there is multicollinearity problem. There are many proposed estimators in literature. In this paper, we propose new estimators which are modifications of the estimator suggested by Lawless and Wang (1976). A Monte Carlo experiment has been conducted for the comparison of the performances of the estimators. Mean squared error (MSE) is used as a performance criterion. The benefits of new estimators are illustrated using two real datasets. According to both simulation results and applications, our new estimators have better performances in the sense of MSE in most of the situations.
Exact p-Values for Network Interference

ABSTRACT We study the calculation of exact p-values for a large class of nonsharp null hypotheses about treatment effects in a setting with data from experiments involving members of a single connected network. The class includes null hypotheses that limit the effect of one unit’s treatment status on another according to the distance between units, for example, the hypothesis might specify that the treatment status of immediate neighbors has no effect, or that units more than two edges away have no effect. We also consider hypotheses concerning the validity of sparsification of a network (e.g., based on the strength of ties) and hypotheses restricting heterogeneity in peer effects (so that, e.g., only the number or fraction treated among neighboring units matters). Our general approach is to define an artificial experiment, such that the null hypothesis that was not sharp for the original experiment is sharp for the artificial experiment, and such that the randomization analysis for the artificial experiment is validated by the design of the original experiment.
Approximation method for discrete Markov decision models with a large state space

To solve discrete Markov decision models with a large number of dimensions is always difficult (and at times, impossible), because size of state space and computation cost increases exponentially with the number of dimensions. This phenomenon is called "The Curse of Dimensionality," and it prevents us from using models with many state variables. To overcome this problem, we propose a new approximation method, named statistical least square temporal difference (SLSTD) method, that can solve discrete Markov decision models with large state space. SLSTD method approximate the value function on the whole state space at once, and obtain optimal approximation weight by minimizing temporal residuals of the Bellman equation. Furthermore, a stochastic approximation method enables us to optimize the problem with low computational cost. SLSTD method can solve DMD models with large state space more easily than other existing methods, and in some cases, reduces the computation time by over 99 percent. We also show the parameter estimated by SLSTD is consistent and asymptotically normally distributed.
Tracking rapid intracellular movements: A Bayesian random set approach

We focus on the biological problem of tracking organelles as they move through cells. In the past, most intracellular movements were recorded manually, however, the results are too incomplete to capture the full complexity of organelle motions. An automated tracking algorithm promises to provide a complete analysis of noisy microscopy data. In this paper, we adopt statistical techniques from a Bayesian random set point of view. Instead of considering each individual organelle, we examine a random set whose members are the organelle states and we establish a Bayesian filtering algorithm involving such set states. The propagated multi-object densities are approximated using a Gaussian mixture scheme. Our algorithm is applied to synthetic and experimental data.
A Nonparametric Maximum Likelihood Approach for Partially Observed Cured Data with Left Truncation and Right-Censoring

Partially observed cured data occur in the analysis of spontaneous abortion (SAB) in observational studies in pregnancy. In contrast to the traditional cured data, such data has an observable `cured' portion as women who do not abort spontaneously. The data is also subject to left truncate in addition to right-censoring because women may enter or withdraw from a study any time during their pregnancy. Left truncation in particular causes unique bias in the presence of a cured portion. In this paper, we study a cure rate model and develop a conditional nonparametric maximum likelihood approach. To tackle the computational challenge we adopt an EM algorithm making use of "ghost copies" of the data, and a closed form variance estimator is derived. Under suitable assumptions, we prove the consistency of the resulting estimator involving an unbounded cumulative baseline hazard function, as well as the asymptotic normality. Simulation results are carried out to evaluate the finite sample performance. We present the analysis of the motivating SAB study to illustrate the power of our model addressing both occurrence and timing of SAB, as compared to existing approaches in practice.
Estimating average treatment effects with a double‐index propensity score

We consider estimating average treatment effects (ATE) of a binary treatment in observational data when data‐driven variable selection is needed to select relevant covariates from a moderately large number of available covariates X . To leverage covariates among X predictive of the outcome for efficiency gain while using regularization to fit a parametric propensity score (PS) model, we consider a dimension reduction of X based on fitting both working PS and outcome models using adaptive LASSO. A novel PS estimator, the Double‐index Propensity Score (DiPS), is proposed, in which the treatment status is smoothed over the linear predictors for X from both the initial working models. The ATE is estimated by using the DiPS in a normalized inverse probability weighting estimator, which is found to maintain double robustness and also local semiparametric efficiency with a fixed number of covariates p. Under misspecification of working models, the smoothing step leads to gains in efficiency and robustness over traditional doubly robust estimators. These results are extended to the case where p diverges with sample size and working models are sparse. Simulations show the benefits of the approach in finite samples. We illustrate the method by estimating the ATE of statins on colorectal cancer risk in an electronic medical record study and the effect of smoking on C‐reactive protein in the Framingham Offspring Study.
Nearly Semiparametric Efficient Estimation of Quantile Regression

As a competitive alternative to least squares regression, quantile regression is popular in analyzing heterogenous data. For quantile regression model specified for one single quantile level $\tau$, major difficulties of semiparametric efficient estimation are the unavailability of a parametric efficient score and the conditional density estimation. In this paper, with the help of the least favorable submodel technique, we first derive the semiparametric efficient scores for linear quantile regression models that are assumed for a single quantile level, multiple quantile levels and all the quantile levels in $(0,1)$ respectively. Our main discovery is a one-step (nearly) semiparametric efficient estimation for the regression coefficients of the quantile regression models assumed for multiple quantile levels, which has several advantages: it could be regarded as an optimal way to pool information across multiple/other quantiles for efficiency gain; it is computationally feasible and easy to implement, as the initial estimator is easily available; due to the nature of quantile regression models under investigation, the conditional density estimation is straightforward by plugging in an initial estimator. The resulting estimator is proved to achieve the corresponding semiparametric efficiency lower bound under regularity conditions. Numerical studies including simulations and an example of birth weight of children confirms that the proposed estimator leads to higher efficiency compared with the Koenker-Bassett quantile regression estimator for all quantiles of interest.
Statistical Guarantees for Estimating the Centers of a Two-component Gaussian Mixture by EM

Recently, a general method for analyzing the statistical accuracy of the EM algorithm has been developed and applied to some simple latent variable models [Balakrishnan et al. 2016]. In that method, the basin of attraction for valid initialization is required to be a ball around the truth. Using Stein's Lemma, we extend these results in the case of estimating the centers of a two-component Gaussian mixture in $d$ dimensions. In particular, we significantly expand the basin of attraction to be the intersection of a half space and a ball around the origin. If the signal-to-noise ratio is at least a constant multiple of $ \sqrt{d\log d} $, we show that a random initialization strategy is feasible.
Gradient-based Regularization Parameter Selection for Problems With Nonsmooth Penalty Functions

ABSTRACT In high-dimensional and/or nonparametric regression problems , regularization (or penalization) is used to control model complexity and induce desired structure. Each penalty has a weight parameter that indicates how strongly the structure corresponding to that penalty should be enforced. Typically, the parameters are chosen to minimize the error on a separate validation set using a simple grid search or a gradient-free optimization method. It is more efficient to tune parameters if the gradient can be determined, but this is often difficult for problems with nonsmooth penalty functions. Here, we show that for many penalized regression problems, the validation loss is actually smooth almost-everywhere with respect to the penalty parameters. We can, therefore, apply a modified gradient descent algorithm to tune parameters. Through simulation studies on example regression problems, we find that increasing the number of penalty parameters and tuning them using our method can decrease the generalization error.
Sacrificing information for the greater good: how to select photometric bands for optimal accuracy

Large-scale surveys make huge amounts of photometric data available. Because of the sheer amount of objects, spectral data cannot be obtained for all of them. Therefore it is important to devise techniques for reliably estimating physical properties of objects from photometric information alone. These estimates are needed to automatically identify interesting objects worth a follow-up investigation as well as to produce the required data for a statistical analysis of the space covered by a survey. We argue that machine learning techniques are suitable to compute these estimates accurately and efficiently. This study promotes a feature selection algorithm, which selects the most informative magnitudes and colours for a given task of estimating physical quantities from photometric data alone. Using k nearest neighbours regression, a well-known non-parametric machine learning method, we show that using the found features significantly increases the accuracy of the estimations compared to using standard features and standard methods. We illustrate the usefulness of the approach by estimating specific star formation rates (sSFRs) and redshifts (photo-z's) using only the broad-band photometry from the Sloan Digital Sky Survey (SDSS). For estimating sSFRs, we demonstrate that our method produces better estimates than traditional spectral energy distribution (SED) fitting. For estimating photo-z's, we show that our method produces more accurate photo-z's than the method employed by SDSS. The study highlights the general importance of performing proper model selection to improve the results of machine learning systems and how feature selection can provide insights into the predictive relevance of particular input features.
An oracle property of the Nadaraya–Watson kernel estimator for high‐dimensional nonparametric regression

The Nadaraya–Watson estimator is among the most studied nonparametric regression methods. A classical result is that its convergence rate depends on the number of covariates and deteriorates quickly as the dimension grows. This underscores the “curse of dimensionality” and has limited its use in high‐dimensional settings. In this paper, however, we show that the Nadaraya–Watson estimator has an oracle property such that when the true regression function is single‐ or multi‐index, it discovers the low‐rank dependence structure between the response and the covariates, mitigating the curse of dimensionality. Specifically, we prove that, using K‐fold cross‐validation and a positive‐semidefinite bandwidth matrix, the Nadaraya–Watson estimator has a convergence rate that depends on the number of indices rather than on the number of covariates. This result follows by allowing the bandwidths to diverge to infinity rather than restricting them all to converge to zero at certain rates, as in previous theoretical studies.
Tractable Post-Selection Maximum Likelihood Inference for the Lasso

Applying standard statistical methods after model selection may yield inefficient estimators and hypothesis tests that fail to achieve nominal type-I error rates. The main issue is the fact that the post-selection distribution of the data differs from the original distribution. In particular, the observed data is constrained to lie in a subset of the original sample space that is determined by the selected model. This often makes the post-selection likelihood of the observed data intractable and maximum likelihood inference difficult. In this work, we get around the intractable likelihood by generating noisy unbiased estimates of the post-selection score function and using them in a stochastic ascent algorithm that yields correct post-selection maximum likelihood estimates. We apply the proposed technique to the problem of estimating linear models selected by the lasso. In an asymptotic analysis the resulting estimates are shown to be consistent for the selected parameters and to have a limiting truncated normal distribution. Confidence intervals constructed based on the asymptotic distribution obtain close to nominal coverage rates in all simulation settings considered, and the point estimates are shown to be superior to the lasso estimates when the true model is sparse.
Maximum Likelihood for Gaussian Process Classification and Generalized Linear Mixed Models under Case-Control Sampling

Modern data sets in various domains often include units that were sampled non-randomly from the population and have a latent correlation structure. Here we investigate a common form of this setting, where every unit is associated with a latent variable, all latent variables are correlated, and the probability of sampling a unit depends on its response. Such settings often arise in case-control studies, where the sampled units are correlated due to spatial proximity, family relations, or other sources of relatedness. Maximum likelihood estimation in such settings is challenging from both a computational and statistical perspective, necessitating approximations that take the sampling scheme into account. We propose a family of approximate likelihood approaches which combine composite likelihood and expectation propagation. We demonstrate the efficacy of our solutions via extensive simulations. We utilize them to investigate the genetic architecture of several complex disorders collected in case-control genetic association studies, where hundreds of thousands of genetic variants are measured for every individual, and the underlying disease liabilities of individuals are correlated due to genetic similarity. Our work is the first to provide a tractable likelihood-based solution for case-control data with complex dependency structures.
Generalized and hybrid Metropolis-Hastings overdamped Langevin algorithms

It has been shown that the nonreversible overdamped Langevin dynamics enjoy better convergence properties in terms of spectral gap and asymptotic variance than the reversible one. In this article we propose a variance reduction method for the Metropolis-Hastings Adjusted Langevin Algorithm (MALA) that makes use of the good behaviour of the these nonreversible dynamics. It consists in constructing a nonreversible Markov chain (with respect to the target invariant measure) by using a Generalized Metropolis-Hastings adjustment on a lifted state space. We present two variations of this method and we discuss the importance of a well-chosen proposal distribution in terms of average rejection probability. We conclude with numerical experimentations to compare our algorithms with the MALA, and show variance reduction of several order of magnitude in some favourable toy cases.
Non-Parametric Estimation of Conditional Densities: A New Method

Let X = (X1,.,Xp) be a stochastic vector having joint density function fX(x) with partitions X1 = (X1,.,Xk) and X2 = (Xk+1,.,Xp). A new method for estimating the conditional density function of X1 given X2 is presented. It is based on locally Gaussian approximations, but simplified in order to tackle the curse of dimensionality in multivariate applications, where both response and explanatory variables can be vectors. We compare our method to some available competitors, and the error of approximation is shown to be small in a series of examples using real and simulated data, and the estimator is shown to be particularly robust against noise caused by independent variables. We also present examples of practical applications of our conditional density estimator in the analysis of time series. Typical values for k in our examples are 1 and 2, and we include simulation experiments with values of p up to 6. Large sample theory is established under a strong mixing condition.
On spectral properties of high-dimensional spatial-sign covariance matrices in elliptical distributions with applications

Spatial-sign covariance matrix (SSCM) is an important substitute of sample covariance matrix (SCM) in robust statistics. This paper investigates the SSCM on its asymptotic spectral behaviors under high-dimensional elliptical populations, where both the dimension $p$ of observations and the sample size $n$ tend to infinity with their ratio $p/n\to c\in (0, \infty)$. The empirical spectral distribution of this nonparametric scatter matrix is shown to converge in distribution to a generalized Marcenko-Pastur law. Beyond this, a new central limit theorem (CLT) for general linear spectral statistics of the SSCM is also established. For polynomial spectral statistics, explicit formulae of the limiting mean and covarance functions in the CLT are provided. The derived results are then applied to an estimation procedure and a test procedure for the spectrum of the shape component of population covariance matrices.
Empirical Bayes analysis of spike and slab posterior distributions

In the sparse normal means model, convergence of the Bayesian posterior distribution associated to spike and slab prior distributions is considered. The key sparsity hyperparameter is calibrated via marginal maximum likelihood empirical Bayes. The plug-in posterior squared-$L^2$ norm is shown to converge at the minimax rate for the euclidean norm for appropriate choices of spike and slab distributions. Possible choices include standard spike and slab with heavy tailed slab, and the spike and slab LASSO of Rockova and George with heavy tailed slab. Surprisingly, the popular Laplace slab is shown to lead to a suboptimal rate for the full empirical Bayes posterior. This provides a striking example where convergence of aspects of the empirical Bayes posterior does not entail convergence of the full empirical Bayes posterior itself.
A new randomness test solving problems of Discrete Fourier Transform Test

Discrete Fourier Transform Test (DFTT), which is a randomness test included in NIST SP800-22, has a problem. It is that theoretical reference distribution of the test statistic has not been derived. In this paper, we propose a new test using variance of power spectrum as the test statistic, whose reference distribution can be theoretically derived. The purpose of DFTT is to detect periodic features and that of the proposed test is the same. We make some experiments and show that the proposed test has stronger detection power than DFTT.