Miracle-dz/UniFAR-FTUs huggingface dataset:
Asymptotic distribution-free change-point detection for multivariate and non-Euclidean data

We consider the testing and estimation of change-points, locations where the distribution abruptly changes, in a sequence of multivariate or non-Euclidean observations. We study a nonparametric framework that utilizes similarity information among observations, which can be applied to various data types as long as an informative similarity measure on the sample space can be defined. The existing approach along this line has low power and/or biased estimates for change-points under some common scenarios. We address these problems by considering new tests based on similarity information. Simulation studies show that the new approaches exhibit substantial improvements in detecting and estimating change-points. In addition, under some mild conditions, the new test statistics are asymptotically distribution free under the null hypothesis of no change. Analytic p-value approximations to the significance of the new test statistics for the single change-point alternative and changed interval alternative are derived, making the new approaches easy off-the-shelf tools for large datasets. The new approaches are illustrated in an analysis of New York taxi data.
The asymptotic distribution of the isotonic regression estimator over a countable pre-ordered set

We study the isotonic regression estimator over a general countable pre-ordered set. We obtain the limiting distribution of the estimator and study its properties: It is proved that, under some general assumptions, the limiting distribution of the isotonised estimator is given by the concatenation of the separate isotonic regressions of the restrictions of an underlying estimator's asymptotic distribution to the comparable level sets of the underlying estimator's probability limit. Also, we show that the isotonisation preserves the rate of convergence of the underlying estimator. We apply these results to the problems of estimation of a bimonotone regression function and estimation of a bimonotone probability mass function.
Simultaneous inference for Best Linear Predictor of the Conditional Average Treatment Effect and other structural functions

This paper provides estimation and inference methods for a structural function, such as Conditional Average Treatment Effect (CATE), based on modern machine learning (ML) tools. We assume that such function can be represented as a conditional expectation = of a signal , where is the unknown nuisance function. In addition to CATE, examples of such functions include regression function with Partially Missing Outcome and Conditional Average Partial Derivative. We approximate by a linear form , where is a vector of the approximating functions and is the Best Linear Predictor. Plugging in the fi rst-stage estimate into the signal , we estimate via ordinary least squares of on . We deliver a high-quality estimate of the pseudo-target function , that features (a) a pointwise Gaussian approximation of at a point , (b) a simultaneous Gaussian approximation of uniformly over x, and (c) optimal rate of convergence of to uniformly over x. In the case the misspeci cation error of the linear form decays sufficiently fast, these approximations automatically hold for the target function instead of a pseudo-target . The fi rst stage nuisance parameter is allowed to be high-dimensional and is estimated by modern ML tools, such as neural networks, -shrinkage estimators, and random forest. Using our method, we estimate the average price elasticity conditional on income using Yatchew and No (2001) data and provide uniform con fidence bands for the target regression function.
Bootstrap tuning in ordered model selection

In the problem of model selection for a given family of linear estimators, ordered by their variance, we offer a new "smallest accepted" approach motivated by Lepski's method and multiple testing theory. The procedure selects the smallest model which satisfies an acceptance rule based on comparison with all larger models. The method is completely data-driven and does not use any prior information about the variance structure of the noise: its parameters are adjusted to the underlying possibly heterogeneous noise by the so-called "propagation condition" using a wild bootstrap method. The validity of the bootstrap calibration is proved for finite samples with an explicit error bound. We provide a comprehensive theoretical study of the method and describe in detail the set of possible values of the selector \( \hat{m} \). We also establish some precise oracle error bounds for the corresponding estimator \( \hat{\theta} = \tilde{\theta}_{\hat{m}} \) which equally applies to estimation of the whole parameter vectors, some subvector or linear mapping, as well as the estimation of a linear functional.
A Causal Inference Approach to Network Meta-Analysis

Abstract: While standard meta-analysis pools the results from randomized trials that compare two treatments, network meta-analysis aggregates the results of randomized trials comparing a wider variety of treatment options. However, it is unclear whether the aggregation of effect estimates across heterogeneous populations will be consistent for a meaningful parameter when not all treatments are evaluated on each population. Drawing from counterfactual theory and the causal inference framework, we define the population of interest in a network meta-analysis and define the target parameter under a series of nonparametric structural assumptions. This allows us to determine the requirements for identifiability of this parameter, enabling a description of the conditions under which network meta-analysis is appropriate and when it might mislead decision making. We then adapt several modeling strategies from the causal inference literature to obtain consistent estimation of the intervention-specific mean outcome and model-independent contrasts between treatments. Finally, we perform a reanalysis of a systematic review to compare the efficacy of antibiotics on suspected or confirmed methicillin-resistant Staphylococcus aureus in hospitalized patients.
Structure-aware error bounds for linear classification with the zero-one loss

We prove risk bounds for binary classification in high-dimensional settings when the sample size is allowed to be smaller than the dimensionality of the training set observations. In particular, we prove upper bounds for both 'compressive learning' by empirical risk minimization (ERM) (that is when the ERM classifier is learned from data that have been projected from high-dimensions onto a randomly selected low-dimensional subspace) as well as uniform upper bounds in the full high-dimensional space. A novel tool we employ in both settings is the 'flipping probability' of Durrant and Kaban (ICML 2013) which we use to capture benign geometric structures that make a classification problem 'easy' in the sense of demanding a relatively low sample size for guarantees of good generalization. Furthermore our bounds also enable us to explain or draw connections between several existing successful classification algorithms. Finally we show empirically that our bounds are informative enough in practice to serve as the objective function for learning a classifier (by using them to do so).
Orthogonal Gaussian process models

Gaussian processes models are widely adopted for nonparameteric/semi-parametric modeling. Identifiability issues occur when the mean model contains polynomials with unknown coefficients. Though resulting prediction is unaffected, this leads to poor estimation of the coefficients in the mean model, and thus the estimated mean model loses interpretability. This paper introduces a new Gaussian process model whose stochastic part is orthogonal to the mean part to address this issue. This paper also discusses applications to multi-fidelity simulations using data examples.
Probability, propensity and probability of propensities (and of probabilities)

The process of doing Science in condition of uncertainty is illustrated with a toy experiment in which the inferential and the forecasting aspects are both present. The fundamental aspects of probabilistic reasoning, also relevant in real life applications, arise quite naturally and the resulting discussion among non-ideologized, free-minded people offers an opportunity for clarifications.
Approximating Likelihood Ratios with Calibrated Discriminative Classifiers

In many fields of science, generalized likelihood ratio tests are established tools for statistical inference. At the same time, it has become increasingly common that a simulator (or generative model) is used to describe complex processes that tie parameters $\theta$ of an underlying theory and measurement apparatus to high-dimensional observations $\mathbf{x}\in \mathbb{R}^p$. However, simulator often do not provide a way to evaluate the likelihood function for a given observation $\mathbf{x}$, which motivates a new class of likelihood-free inference algorithms. In this paper, we show that likelihood ratios are invariant under a specific class of dimensionality reduction maps $\mathbb{R}^p \mapsto \mathbb{R}$. As a direct consequence, we show that discriminative classifiers can be used to approximate the generalized likelihood ratio statistic when only a generative model for the data is available. This leads to a new machine learning-based approach to likelihood-free inference that is complementary to Approximate Bayesian Computation, and which does not require a prior on the model parameters. Experimental results on artificial problems with known exact likelihoods illustrate the potential of the proposed method.
An analysis of penalized interaction models

An important consideration for variable selection in interaction models is to design an appropriate penalty that respects hierarchy of the importance of the variables. A common theme is to include an interaction term only after the corresponding main effects are present. In this paper, we study several recently proposed approaches and present a unified analysis on the convergence rate for a class of estimators, when the design satisfies the restricted eigenvalue condition. In particular, we show that with probability tending to one, the resulting estimates have a rate of convergence $s\sqrt{\log p_1/n}$ in the $\ell_1$ error, where $p_1$ is the ambient dimension, $s$ is the true dimension and $n$ is the sample size. We give a new proof that the restricted eigenvalue condition holds with high probability, when the variables in the main effects and the errors follow sub-Gaussian distributions. Under this setup, the interactions no longer follow Gaussian or sub-Gaussian distributions even if the main effects follow Gaussian, and thus existing works are not applicable. This result is of independent interest.
On the Optimality of Kernel-Embedding Based Goodness-of-Fit Tests

The reproducing kernel Hilbert space (RKHS) embedding of distributions offers a general and flexible framework for testing problems in arbitrary domains and has attracted considerable amount of attention in recent years. To gain insights into their operating characteristics, we study here the statistical performance of such approaches within a minimax framework. Focusing on the case of goodness-of-fit tests, our analyses show that a vanilla version of the kernel-embedding based test could be suboptimal, and suggest a simple remedy by moderating the embedding. We prove that the moderated approach provides optimal tests for a wide range of deviations from the null and can also be made adaptive over a large collection of interpolation spaces. Numerical experiments are presented to further demonstrate the merits of our approach.
Uniform convergence over time of a nested particle filtering scheme for recursive parameter estimation in state-space Markov models

Abstract We analyse the performance of a recursive Monte Carlo method for the Bayesian estimation of the static parameters of a discrete-time state-space Markov model. The algorithm employs two layers of particle filters to approximate the posterior probability distribution of the model parameters. In particular, the first layer yields an empirical distribution of samples on the parameter space, while the filters in the second layer are auxiliary devices to approximate the (analytically intractable) likelihood of the parameters. This approach relates the novel algorithm to the recent sequential Monte Carlo square method, which provides a nonrecursive solution to the same problem. In this paper we investigate the approximation of integrals of real bounded functions with respect to the posterior distribution of the system parameters. Under assumptions related to the compactness of the parameter support and the stability and continuity of the sequence of posterior distributions for the state-space model, we prove that the L p norms of the approximation errors vanish asymptotically (as the number of Monte Carlo samples generated by the algorithm increases) and uniformly over time. We also prove that, under the same assumptions, the proposed scheme can asymptotically identify the parameter values for a class of models. We conclude the paper with a numerical example that illustrates the uniform convergence results by exploring the accuracy and stability of the proposed algorithm operating with long sequences of observations.
Sample Out-of-Sample Inference Based on Wasserstein Distance

Financial institutions make decisions according to a model of uncertainty. At the same time, regulators often evaluate the risk exposure of these institutions using a model of uncertainty, which is often different from the one used by the institutions. How can one incorporate both views into a single framework? This paper provides such a framework. It quantifies the impact of the misspecification inherent to the financial institution data-driven model via the introduction of an adversarial player. The adversary replaces the institution's generated scenarios by the regulator's scenarios subject to a budget constraint and a cost that measures the distance between the two sets of scenarios (using what in statistics is known as the Wasserstein distance). This paper also harnesses statistical theory to make inference about the size of the estimated error when the sample sizes (both of the institution and the regulator) are large. The framework is explained more broadly in the context of distributionally robust optimization (a class of perfect information games, in which decisions are taken against an adversary that perturbs a baseline distribution).
The Kato–Temple inequality and eigenvalue concentration with applications to graph inference

We present an adaptation of the Kato--Temple inequality for bounding perturbations of eigenvalues with applications to statistical inference for random graphs, specifically hypothesis testing and change-point detection. We obtain explicit high-probability bounds for the individual distances between certain signal eigenvalues of a graph's adjacency matrix and the corresponding eigenvalues of the model's edge probability matrix, even when the latter eigenvalues have multiplicity. Our results extend more broadly to the perturbation of singular values in the presence of quite general random matrix noise.
Shape constrained kernel-weighted least squares: Application to production function estimation for Chilean manufacturing industries

Two approaches to nonparametric regression include local averaging and shape constrained regression. In this paper we examine a novel way to impose shape constraints on a local linear kernel estimator. The proposed approach is referred to as Shape Constrained Kernel-weighted Least Squares (SCKLS). We prove consistency of SCKLS estimator and show that SCKLS is a generalization of Convex Nonparametric Least Squares (CNLS). We compare the performance of three estimators, SCKLS, CNLS, and Constrained Weighted Bootstrap, via Monte Carlo simulations. From the simulation results, we conclude that SCKLS performance is very competitive and provides better out-of-sample prediction than existing estimators. We analyze Chilean manufacturing data using the SCKLS estimator and quantify production in the plastics and wood industries. Importantly we find firms that export have significantly higher productivity.
Testing the Sphericity of a covariance matrix when the dimension is much larger than the sample size

This paper focuses on the prominent sphericity test when the dimension $p$ is much lager than sample size $n$. The classical likelihood ratio test(LRT) is no longer applicable when $p\gg n$. Therefore a Quasi-LRT is proposed and asymptotic distribution of the test statistic under the null when $p/n\rightarrow\infty, n\rightarrow\infty$ is well established in this paper. Meanwhile, John's test has been found to possess the powerful {\it dimension-proof} property, which keeps exactly the same limiting distribution under the null with any $(n,p)$-asymptotic, i.e. $p/n\rightarrow[0,\infty]$, $n\rightarrow\infty$. All asymptotic results are derived for general population with finite fourth order moment. Numerical experiments are implemented for comparison.
Relative Density and Exact Recovery in Heterogeneous Stochastic Block Models

The Stochastic Block Model (SBM) is a widely used random graph model for networks with communities. Despite the recent burst of interest in recovering communities in the SBM from statistical and computational points of view, there are still gaps in understanding the fundamental information theoretic and computational limits of recovery. In this paper, we consider the SBM in its full generality, where there is no restriction on the number and sizes of communities or how they grow with the number of nodes, as well as on the connection probabilities inside or across communities. This generality allows us to move past the artifacts of homogenous SBM, and understand the right parameters (such as the relative densities of communities) that define the various recovery thresholds. We outline the implications of our generalizations via a set of illustrative examples. For instance, $\log n$ is considered to be the standard lower bound on the cluster size for exact recovery via convex methods, for homogenous SBM. We show that it is possible, in the right circumstances (when sizes are spread and the smaller the cluster, the denser), to recover very small clusters (up to $\sqrt{\log n}$ size), if there are just a few of them (at most polylogarithmic in $n$).
CHARACTERIZATIONS OF MULTINORMALITY AND CORRESPONDING TESTS OF FIT, INCLUDING FOR GARCH MODELS

We provide novel characterizations of multivariate normality that incorporate both the characteristic function and the moment generating function, and we employ these results to construct a class of affine invariant, consistent and easy-to-use goodness-of-fit tests for normality. The test statistics are suitably weighted L2-statistics, and we provide their asymptotic behavior both for i.i. d. observations as well as in the context of testing that the innovation distribution of a multivariate GARCH model is Gaussian. We also study the finite-sample behavior of the new tests and compare the new criteria with alternative existing tests.
Rates of convergence of rho-estimators for sets of densities satisfying shape constraints

The purpose of this paper is to pursue our study of rho-estimators built from i.i. d. observations that we defined in Baraud et al. (2014). For a \rho-estimator based on some model S (which means that the estimator belongs to S) and a true distribution of the observations that also belongs to S, the risk (with squared Hellinger loss) is bounded by a quantity which can be viewed as a dimension function of the model and is often related to the "metric dimension" of this model, as defined in Birg\'e (2006). This is a minimax point of view and it is well-known that it is pessimistic. Typically, the bound is accurate for most points in the model but may be very pessimistic when the true distribution belongs to some specific part of it. This is the situation that we want to investigate here. For some models, like the set of decreasing densities on [0,1], there exist specific points in the model that we shall call "extremal" and for which the risk is substantially smaller than the typical risk. Moreover, the risk at a non-extremal point of the model can be bounded by the sum of the risk bound at a well-chosen extremal point plus the square of its distance to this point. This implies that if the true density is close enough to an extremal point, the risk at this point may be smaller than the minimax risk on the model and this actually remains true even if the true density does not belong to the model. The result is based on some refined bounds on the suprema of empirical processes that are established in Baraud (2016).
A General Decision Theory for Huber's $\epsilon$-Contamination Model

Today's data pose unprecedented challenges to statisticians. It may be incomplete, corrupted or exposed to some unknown source of contamination. We need new methods and theories to grapple with these challenges. Robust estimation is one of the revived fields with potential to accommodate such complexity and glean useful information from modern datasets. Following our recent work on high dimensional robust covariance matrix estimation, we establish a general decision theory for robust statistics under Huber's $\epsilon$-contamination model. We propose a solution using Scheff{\'e} estimate to a robust two-point testing problem that leads to the construction of robust estimators adaptive to the proportion of contamination. Applying the general theory, we construct robust estimators for nonparametric density estimation, sparse linear regression and low-rank trace regression. We show that these new estimators achieve the minimax rate with optimal dependence on the contamination proportion. This testing procedure, Scheff{\'e} estimate, also enjoys an optimal rate in the exponent of the testing error, which may be of independent interest.
Bayesian Latent Pattern Mixture Models for Handling Attrition in Panel Studies With Refreshment Samples

Many panel studies collect refreshment samples---new, randomly sampled respondents who complete the questionnaire at the same time as a subsequent wave of the panel. With appropriate modeling, these samples can be leveraged to correct inferences for biases caused by non-ignorable attrition. We present such a model when the panel includes many categorical survey variables. The model relies on a Bayesian latent pattern mixture model, in which an indicator for attrition and the survey variables are modeled jointly via a latent class model. We allow the multinomial probabilities within classes to depend on the attrition indicator, which offers additional flexibility over standard applications of latent class models. We present results of simulation studies that illustrate the benefits of this flexibility. We apply the model to correct attrition bias in an analysis of data from the 2007-2008 Associated Press/Yahoo News election panel study.
Detecting Confounding in Multivariate Linear Models via Spectral Analysis

Abstract We study a model where one target variable Y$Y$ is correlated with a vector X:=(X1,…,Xd)$\textbf{X}:=(X_1,\dots,X_d)$ of predictor variables being potential causes of Y$Y$. We describe a method that infers to what extent the statistical dependences between X$\textbf{X}$ and Y$Y$ are due to the influence of X$\textbf{X}$ on Y$Y$ and to what extent due to a hidden common cause (confounder) of X$\textbf{X}$ and Y$Y$. The method relies on concentration of measure results for large dimensions d$d$ and an independence assumption stating that, in the absence of confounding, the vector of regression coefficients describing the influence of each X$\textbf{X}$ on Y$Y$ typically has ‘generic orientation’ relative to the eigenspaces of the covariance matrix of X$\textbf{X}$. For the special case of a scalar confounder we show that confounding typically spoils this generic orientation in a characteristic way that can be used to quantitatively estimate the amount of confounding (subject to our idealized model assumptions).