Miracle-dz/UniFAR-FTUs huggingface dataset:
Generic Conditions for Forecast Dominance

Abstract Recent studies have analyzed whether one forecast method dominates another under a class of consistent scoring functions. While the existing literature focuses on empirical tests of forecast dominance, little is known about the theoretical conditions under which one forecast dominates another. To address this question, we derive a new characterization of dominance among forecasts of the mean functional. We present various scenarios under which dominance occurs. Unlike existing results, our results allow for the case that the forecasts’ underlying information sets are not nested, and allow for uncalibrated forecasts that suffer, for example, from model misspecification or parameter estimation error. We illustrate the empirical relevance of our results via data examples from finance and economics.
High-Dimensional Learning under Approximate Sparsity: A Unifying Framework for Nonsmooth Learning and Regularized Neural Networks

High-dimensional statistical learning (HDSL) has been widely applied in data analysis, operations research, and stochastic optimization. Despite the availability of multiple theoretical frameworks, most HDSL theories stipulate the following two conditions, which are sometimes overly critical: (a) the sparsity, and (b) the restricted strong convexity (RSC). This paper generalizes both conditions via the use of the folded concave penalty (FCP); we show that, for an M-estimation problem where (i) the (conventional) sparsity is relaxed into the approximate sparsity and (ii) the RSC is completely absent, the FCP-based regularization leads to poly-logarithmic sample complexity: the size of the training data is only required to be poly-logarithmic in the problem dimensionality. This finding allows us to further understand two important paradigms much less discussed formerly: the high-dimensional nonsmooth learning and the (deep) neural networks (NN). For both problems, we show that the poly-logarithmic sample complexity can be maintained. Furthermore, via integrating the NN with the FCP, the excess risk of a stationary point to the training formulation for the NN is strictly monotonic with respect to the solution's suboptimality gap, providing the first theoretical evidence for the empirically observed consistency between the generalization performance and the optimization quality in training an NN.
Minimax signal detection in ill-posed inverse problems

Ill-posed inverse problems arise in various scientific fields. We consider the signal detection problem for mildly, severely and extremely ill-posed inverse problems with $l^q$-ellipsoids (bodies), $q\in(0,2]$, for Sobolev, analytic and generalized analytic classes of functions under the Gaussian white noise model. We study both rate and sharp asymptotics for the error probabilities in the minimax setup. By construction, the derived tests are, often, nonadaptive. Minimax rate-optimal adaptive tests of rather simple structure are also constructed.
Adaptive goodness-of-fit tests in a density model

Given an i.i. d. sample drawn from a density f, we propose to test that f equals some prescribed density f 0 or that f belongs to some translation/scale family. We introduce a multiple testing procedure based on an estimation of the L 2 -distance between f and f 0 or between f and the parametric family that we consider. For each sample size n, our test has level of significance a. In the case of simple hypotheses, we prove that our test is adaptive: it achieves the optimal rates of testing established by Ingster [J. Math. Sci. 99 (2000) 1110-1119] over various classes of smooth functions simultaneously. As for composite hypotheses, we obtain similar results up to a logarithmic factor. We carry out a simulation study to compare our procedures with the Kolmogorov-Smimov tests, or with goodness-of-fit tests proposed by Bickel and Ritov [in Nonparametric Statistics and Related Topics (1992) 51-57] and by Kallenberg and Ledwina [Ann. Statist. 23 (1995) 1594-1608].
Asymptotics of maximum likelihood estimation for stable law with continuous parameterization

Abstract Asymptotics of maximum likelihood estimation for α-stable law are analytically investigated with a continuous parameterization. The consistency and asymptotic normality are shown on the interior of the whole parameter space. Although these asymptotics have been provided with Zolotarev’s (B) parameterization, there are several gaps between. Especially in the latter, the density is discontinuous at α = 1 for and usual asymptotics are impossible. This is a considerable inconvenience for applications. By showing that these quantities are smooth in the continuous form, we fill the gaps between and provide a convenient theory. We numerically approximate the Fisher information matrix around the Cauchy law The results exhibit continuity at and this secures the accuracy of our calculations.
Consistency of spectral clustering

Consistency is a key property of all statistical procedures analyzing randomly sampled data. Surprisingly, despite decades of work, little is known about consistency of most clustering algorithms. In this paper we investigate consistency of the popular family of spectral clustering algorithms, which clusters the data with the help of eigenvectors of graph Laplacian matrices. We develop new methods to establish that, for increasing sample size, those eigenvectors converge to the eigenvectors of certain limit operators. As a result, we can prove that one of the two major classes of spectral clustering (normalized clustering) converges under very general conditions, while the other (unnormalized clustering) is only consistent under strong additional assumptions, which are not always satisfied in real data. We conclude that our analysis provides strong evidence for the superiority of normalized spectral clustering.
Spherical Regression Under Mismatch Corruption With Application to Automated Knowledge Translation

Abstract Motivated by a series of applications in data integration, language translation, bioinformatics, and computer vision, we consider spherical regression with two sets of unit-length vectors when the data are corrupted by a small fraction of mismatch in the response-predictor pairs. We propose a three-step algorithm in which we initialize the parameters by solving an orthogonal Procrustes problem to estimate a translation matrix ignoring the mismatch. We then estimate a mapping matrix aiming to correct the mismatch using hard-thresholding to induce sparsity, while incorporating potential group information. We eventually obtain a refined estimate for by removing the estimated mismatched pairs. We derive the error bound for the initial estimate of in both fixed and high-dimensional setting. We demonstrate that the refined estimate of achieves an error rate that is as good as if no mismatch is present. We show that our mapping recovery method not only correctly distinguishes one-to-one and one-to-many correspondences, but also consistently identifies the matched pairs and estimates the weight vector for combined correspondence. We examine the finite sample performance of the proposed method via extensive simulation studies, and with application to the unsupervised translation of medical codes using electronic health records data. Supplementary materials for this article are available online.
On Nonlinear Dimensionality Reduction, Linear Smoothing and Autoencoding

We develop theory for nonlinear dimensionality reduction (NLDR). A number of NLDR methods have been developed, but there is limited understanding of how these methods work and the relationships between them. There is limited basis for using existing NLDR theory for deriving new algorithms. We provide a novel framework for analysis of NLDR via a connection to the statistical theory of linear smoothers. This allows us to both understand existing methods and derive new ones. We use this connection to smoothing to show that asymptotically, existing NLDR methods correspond to discrete approximations of the solutions of sets of differential equations given a boundary condition. In particular, we can characterize many existing methods in terms of just three limiting differential operators and boundary conditions. Our theory also provides a way to assert that one method is preferable to another; indeed, we show Local Tangent Space Alignment is superior within a class of methods that assume a global coordinate chart defines an isometric embedding of the manifold.
Deep neural networks algorithms for stochastic control problems on finite horizon, part I: convergence analysis

This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming (DP). Differently from the classical approximate DP approach, we first approximate the optimal policy by means of neural networks in the spirit of deep reinforcement learning, and then the value function by Monte Carlo regression. This is achieved in the DP recursion by performance or hybrid iteration, and regress now or later/quantization methods from numerical probabilities. We provide a theoretical justification of these algorithms. Consistency and rate of convergence for the control and value function estimates are analyzed and expressed in terms of the universal approximation error of the neural networks. Numerical results on various applications are presented in a companion paper [2] and illustrate the performance of our algorithms.
PCABM: Pairwise Covariates-Adjusted Block Model for Community Detection*

One of the most fundamental problems in network study is community detection. The stochastic block model (SBM) is one widely used model for network data with different estimation methods developed with their community detection consistency results unveiled. However, the SBM is restricted by the strong assumption that all nodes in the same community are stochastically equivalent, which may not be suitable for practical applications. We introduce pairwise covariates-adjusted stochastic block model (PCABM), a generalization of SBM that incorporates pairwise covariate information. We study the maximum likelihood estimates of the coefficients for the covariates as well as the community assignments. It is shown that both the coefficient estimates of the covariates and the community assignments are consistent under suitable sparsity conditions. Spectral clustering with adjustment (SCWA) is introduced to efficiently solve PCABM. Under certain conditions, we derive the error bound of community estimation under SCWA and show that it is community detection consistent. PCABM compares favorably with the SBM or degree-corrected stochastic block model (DCBM) under a wide range of simulated and real networks when covariate information is accessible.
Aggregation for Gaussian regression

This paper studies statistical aggregation procedures in the regression setting. A motivating factor is the existence of many different methods of estimation, leading to possibly competing estimators. We consider here three different types of aggregation: model selection (MS) aggregation, convex (C) aggregation and linear (L) aggregation. The objective of (MS) is to select the optimal single estimator from the list; that of (C) is to select the optimal convex combination of the given estimators; and that of (L) is to select the optimal linear combination of the given estimators. We are interested in evaluating the rates of convergence of the excess risks of the estimators obtained by these procedures. Our approach is motivated by recently published minimax results [Nemirovski, A. (2000). Topics in non-parametric statistics. Lectures on Probability Theory and Statistics (Saint-Flour, 1998). Lecture Notes in Math. 1738 85-277. Springer, Berlin; Tsybakov, A. B. (2003). Optimal rates of aggregation. Learning Theory and Kernel Machines. Lecture Notes in Artificial Intelligence 2777 303-313. Springer, Heidelberg]. There exist competing aggregation procedures achieving optimal convergence rates for each of the (MS), (C) and (L) cases separately. Since these procedures are not directly comparable with each other, we suggest an alternative solution. We prove that all three optimal rates, as well as those for the newly introduced (S) aggregation (subset selection), are nearly achieved via a single "universal" aggregation procedure. The procedure consists of mixing the initial estimators with weights obtained by penalized least squares. Two different penalties are considered: one of them is of the BIC type, the second one is a data-dependent l 1 -type penalty.
FREE EXPONENTIAL FAMILIES AS KERNEL FAMILIES

Free exponential families have been previously introduced as a special case of the q-exponential family. We show that free exponential families arise also from a procedure analogous to the definition of exponential families by using the Cauchy-Stieltjes kernel instead of the exponential kernel. We use this approach to re-derive several known results and to study further similarities with exponential families and reproductive exponential models.
A Statistical Theory for the Analysis of Uncertain Systems

This paper addresses the issues of conservativeness and computational complexity of probabilistic robustness analysis. We solve both issues by defining a new sampling strategy and robustness measure. The new measure is shown to be much less conservative than the existing one. The new sampling strategy enables the definition of efficient hierarchical sample reuse algorithms that reduce significantly the computational complexity and make it independent of the dimension of the uncertainty space. Moreover, we show that there exists a one to one correspondence between the new and the existing robustness measures and provide a computationally simple algorithm to derive one from the other.
SpaceNet MVOI: A Multi-View Overhead Imagery Dataset

Detection and segmentation of objects in overheard imagery is a challenging task. The variable density, random orientation, small size, and instance-to-instance heterogeneity of objects in overhead imagery calls for approaches distinct from existing models designed for natural scene datasets. Though new overhead imagery datasets are being developed, they almost universally comprise a single view taken from directly overhead ("at nadir"), failing to address a critical variable: look angle. By contrast, views vary in real-world overhead imagery, particularly in dynamic scenarios such as natural disasters where first looks are often over 40 degrees off-nadir. This represents an important challenge to computer vision methods, as changing view angle adds distortions, alters resolution, and changes lighting. At present, the impact of these perturbations for algorithmic detection and segmentation of objects is untested. To address this problem, we present an open source Multi-View Overhead Imagery dataset, termed SpaceNet MVOI, with 27 unique looks from a broad range of viewing angles (-32.5 degrees to 54.0 degrees). Each of these images cover the same 665 square km geographic extent and are annotated with 126,747 building footprint labels, enabling direct assessment of the impact of viewpoint perturbation on model performance. We benchmark multiple leading segmentation and object detection models on: (1) building detection, (2) generalization to unseen viewing angles and resolutions, and (3) sensitivity of building footprint extraction to changes in resolution. We find that state of the art segmentation and object detection models struggle to identify buildings in off-nadir imagery and generalize poorly to unseen views, presenting an important benchmark to explore the broadly relevant challenge of detecting small, heterogeneous target objects in visually dynamic contexts.
Moderate-Dimensional Inferences on Quadratic Functionals in Ordinary Least Squares

Abstract Statistical inferences for quadratic functionals of linear regression parameter have found wide applications including signal detection, global testing, inferences of error variance and fraction of variance explained. Classical theory based on ordinary least squares estimator works perfectly in the low-dimensional regime, but fails when the parameter dimension pn grows proportionally to the sample size n. In some cases, its performance is not satisfactory even when . The main contribution of this article is to develop dimension-adaptive inferences for quadratic functionals when . We propose a bias-and-variance-corrected test statistic and demonstrate that its theoretical validity (such as consistency and asymptotic normality) is adaptive to both low dimension with τ = 0 and moderate dimension with . Our general theory holds, in particular, without Gaussian design/error or structural parameter assumption, and applies to a broad class of quadratic functionals covering all aforementioned applications. As a by-product, we find that the classical fixed-dimensional results continue to hold if and only if the signal-to-noise ratio is large enough, say when pn diverges but slower than n. Extensive numerical results demonstrate the satisfactory performance of the proposed methodology even when in some extreme cases. The mathematical arguments are based on the random matrix theory and leave-one-observation-out method.
Edgeworth expansions for studentized statistics under weak dependence

In this paper, we derive valid Edgeworth expansions for studentized versions of a large class of statistics when the data are generated by a strongly mixing process. Under dependence, the asymptotic variance of such a statistic is given by an infinite series oflag-covariances, and therefore, studentizing factors (i.e., estimators of the asymptotic standard error) typically involve an increasing number, say, l of lag-covariance estimators, which are themselves quadratic functions of the observations. The unboundedness of the dimension l of these quadratic functions makes the derivation and the form of the expansions nonstandard. It is shown that in contrast to the case of the studentized means under independence, the derived Edgeworth expansion is a superposition of three distinct series, respectively, given by one in powers of n -1/2 , one in powers of [n/l] -1/2 (resulting from the standard error of the studentizing factor) and one in powers of the bias of the studentizing factor, where n denotes the sample size.
Lower bounds for posterior rates with Gaussian process priors

Upper bounds for rates of convergence of posterior distributions associated to Gaussian process priors are obtained by van der Vaart and van Zanten in [14] and expressed in terms of a concentration function involving the Reproducing Kernel Hilbert Space of the Gaussian prior. Here lower-bound counterparts are obtained. As a corollary, we obtain the precise rate of convergence of posteriors for Gaussian priors in various settings. Additionally, we extend the upper-bound results of [14] about Riemann-Liouville priors to a continuous family of parameters.
The Multiple Random Dot Product Graph Model

Data in the form of graphs, or networks, arise naturally in a number of contexts; examples include social networks and biological networks. We are often faced with the availability of multiple graphs on a single set of nodes. In this article, we propose the multiple random dot product graph model for this setting. Our proposed model leads naturally to an optimization problem, which we solve using an efficient alternating minimization approach. We further use this model as the basis for a new test for the hypothesis that the graphs come from a single distribution, versus the alternative that they are drawn from different distributions. We evaluate the performance of both the fitting algorithm and the hypothesis test in several simulation settings, and demonstrate empirical improvement over existing approaches. We apply these new approaches to a Wikipedia data set and a C. elegans data set.
Convergence rates for the generalized Fréchet mean via the quadruple inequality

For sets $\mathcal Q$ and $\mathcal Y$, the generalized Frechet mean $m \in \mathcal Q$ of a random variable $Y$, which has values in $\mathcal Y$, is any minimizer of $q\mapsto \mathbb E[\mathfrak c(q,Y)]$, where $\mathfrak c \colon \mathcal Q \times \mathcal Y \to \mathbb R$ is a cost function. There are little restrictions to $\mathcal Q$ and $\mathcal Y$. In particular, $\mathcal Q$ can be a non-Euclidean metric space. We provide convergence rates for the empirical generalized Frechet mean. Conditions for rates in probability and rates in expectation are given. In contrast to previous results on Frechet means, we do not require a finite diameter of the $\mathcal Q$ or $\mathcal Y$. Instead, we assume an inequality, which we call quadruple inequality. It generalizes an otherwise common Lipschitz condition on the cost function. This quadruple inequality is known to hold in Hadamard spaces. We show that it also holds in a suitable way for certain powers of a Hadamard-metric.
Asymptotic expansions for sums of block-variables under weak dependence

Let {X,} ∞ i=-∞ be a sequence of random vectors and Y in = f in (X i,l ) be zero mean block-variables where X i,l = (X i X i+l-1 ), i > 1, are overlapping blocks of length l and where f in are Borel measurable functions. This paper establishes valid joint asymptotic expansions of general orders for the joint distribution of the sums Σ n i=1 X i and Σ n i=1 Y in under weak dependence conditions on the sequence {X i } ∞ i=-∞ when the block length t grows to infinity. In contrast to the classical Edgeworth expansion results where the terms in the expansions are given by powers of n -1/2 , the expansions derived here are mixtures of two series, one in powers of n -1/2 and the other in powers of [n l] -1/2 . Applications of the main results to (i) expansions for Studentized statistics of time series data and (ii) second order correctness of the blocks of blocks bootstrap method are given.
Differences of Type I error rates for ANOVA and Multilevel-Linear-Models using SAS and SPSS for repeated measures designs

To derive recommendations on how to analyze longitudinal data, we examined Type I error rates of Multilevel Linear Models (MLM) and repeated measures Analysis of Variance (rANOVA) using SAS and SPSS. We performed a simulation with the following specifications: To explore the effects of high numbers of measurement occasions and small sample sizes on Type I error, measurement occasions of m = 9 and 12 were investigated as well as sample sizes of n = 15, 20, 25 and 30. Effects of non-sphericity in the population on Type I error were also inspected: 5,000 random samples were drawn from two populations containing neither a within-subject nor a between-group effect. They were analyzed including the most common options to correct rANOVA and MLM-results: The Huynh-Feldt-correction for rANOVA (rANOVA-HF) and the Kenward-Roger-correction for MLM (MLM-KR), which could help to correct progressive bias of MLM with an unstructured covariance matrix (MLM-UN). Moreover, uncorrected rANOVA and MLM assuming a compound symmetry covariance structure (MLM-CS) were also taken into account. The results showed a progressive bias for MLM-UN for small samples which was stronger in SPSS than in SAS. Moreover, an appropriate bias correction for Type I error via rANOVA-HF and an insufficient correction by MLM-UN-KR for n < 30 were found. These findings suggest MLM-CS or rANOVA if sphericity holds and a correction of a violation via rANOVA-HF. If an analysis requires MLM, SPSS yields more accurate Type I error rates for MLM-CS and SAS yields more accurate Type I error rates for MLM-UN.
Estimation of the Hurst parameter from discrete noisy data

We estimate the Hurst parameter $H$ of a fractional Brownian motion from discrete noisy data observed along a high frequency sampling scheme. The presence of systematic experimental noise makes recovery of $H$ more difficult since relevant information is mostly contained in the high frequencies of the signal. We quantify the difficulty of the statistical problem in a min-max sense: we prove that the rate $n^{-1/(4H+2)}$ is optimal for estimating $H$ and propose rate optimal estimators based on adaptive estimation of quadratic functionals.