Miracle-dz/UniFAR-FTUs huggingface dataset:
Ranking soccer teams on basis of their current strength: a comparison of maximum likelihood approaches

We present ten different strength-based statistical models that we use to model soccer match outcomes with the aim of producing a new ranking. The models are of four main types: Thurstone-Mosteller, Bradley-Terry, Independent Poisson and Bivariate Poisson, and their common aspect is that the parameters are estimated via weighted maximum likelihood, the weights being a match importance factor and a time depreciation factor giving less weight to matches that are played a long time ago. Since our goal is to build a ranking reflecting the teams' current strengths, we compare the 10 models on basis of their predictive performance via the Rank Probability Score at the level of both domestic leagues and national teams. We find that the best models are the Bivariate and Independent Poisson models. We then illustrate the versatility and usefulness of our new rankings by means of three examples where the existing rankings fail to provide enough information or lead to peculiar results.
Paired-move multiple-try stochastic search for Bayesian variable selection

Variable selection is a key issue when analyzing high-dimensional data. The explosion of data with large sample sizes and dimensionality brings new challenges to this problem in both inference accuracy and computational complexity. To alleviate these problems, we propose a new scalable Markov chain Monte Carlo (MCMC) sampling algorithm for "large $p$ small $n$" scenarios by generalizing multiple-try Metropolis to discrete model spaces and further incorporating neighborhood-based stochastic search. The proof of reversibility of the proposed MCMC algorithm is provided. Extensive simulation studies are performed to examine the efficiency of the new algorithm compared with existing methods. A real data example is provided to illustrate the prediction performances of the new algorithm.
Functional Generalized Autoregressive Conditional Heteroskedasticity

Heteroskedasticity is a common feature of financial time series and is commonly addressed in the model building process through the use of autoregressive conditional heteroskedastic and generalized autoregressive conditional heteroskedastic (GARCH) processes. More recently, multivariate variants of these processes have been the focus of research with attention given to methods seeking an efficient and economic estimation of a large number of model parameters. Because of the need for estimation of many parameters, however, these models may not be suitable for modelling now prevalent high‐frequency volatility data. One potentially useful way to bypass these issues is to take a functional approach. In this article, theory is developed for a new functional version of the GARCH process, termed fGARCH. The main results are concerned with the structure of the fGARCH(1,1) process, providing criteria for the existence of strictly stationary solutions both in the space of square‐integrable and continuous functions. An estimation procedure is introduced, and its consistency and asymptotic normality are verified. A small empirical study highlights potential applications to intraday volatility estimation.
Multivariate convex regression: global risk bounds and adaptation

We study the problem of estimating a multivariate convex function defined on a convex body in a regression setting with random design. We are interested in optimal rates of convergence under a squared global continuous $l_2$ loss in the multivariate setting $(d\geq 2)$. One crucial fact is that the minimax risks depend heavily on the shape of the support of the regression function. It is shown that the global minimax risk is on the order of $n^{-2/(d+1)}$ when the support is sufficiently smooth, but that the rate $n^{-4/(d+4)}$ is when the support is a polytope. Such differences in rates are due to difficulties in estimating the regression function near the boundary of smooth regions. We then study the natural bounded least squares estimators (BLSE): we show that the BLSE nearly attains the optimal rates of convergence in low dimensions, while suffering rate-inefficiency in high dimensions. We show that the BLSE adapts nearly parametrically to polyhedral functions when the support is polyhedral in low dimensions by a local entropy method. We also show that the boundedness constraint cannot be dropped when risk is assessed via continuous $l_2$ loss. Given rate sub-optimality of the BLSE in higher dimensions, we further study rate-efficient adaptive estimation procedures. Two general model selection methods are developed to provide sieved adaptive estimators (SAE) that achieve nearly optimal rates of convergence for particular "regular" classes of convex functions, while maintaining nearly parametric rate-adaptivity to polyhedral functions in arbitrary dimensions. Interestingly, the uniform boundedness constraint is unnecessary when risks are measured in discrete $l_2$ norms.
Conditional predictive inference for high-dimensional stable algorithms

We investigate generically applicable and intuitively appealing prediction intervals based on leave-one-out residuals. The conditional coverage probability of the proposed interval, given the observations in the training sample, is close to the nominal level, provided that the underlying algorithm used for computing point predictions is sufficiently stable under the omission of single feature-response pairs. Our results are based on a finite sample analysis of the empirical distribution function of the leaveone- out residuals and hold in a non-parametric setting with only minimal assumptions on the error distribution. To illustrate our results, we also apply them to high-dimensional linear predictors, where we obtain uniform asymptotic conditional validity as both sample size and dimension tend to infinity at the same rate. These results show that despite the serious problems of resampling procedures for inference on the unknown parameters (cf. Bickel and Freedman, 1983; El Karoui and Purdom, 2015; Mammen, 1996), leave-one-out methods can be successfully applied to obtain reliable predictive inference even in high dimensions.
Uncertainty Quantification for High-Dimensional Sparse Nonparametric Additive Models

Abstract Statistical inference in high-dimensional settings has recently attracted enormous attention within the literature. However, most published work focuses on the parametric linear regression problem. This article considers an important extension of this problem: statistical inference for high-dimensional sparse nonparametric additive models. To be more precise, this article develops a methodology for constructing a probability density function on the set of all candidate models. This methodology can also be applied to construct confidence intervals for various quantities of interest (such as noise variance) and confidence bands for the additive functions. This methodology is derived using a generalized fiducial inference framework. It is shown that results produced by the proposed methodology enjoy correct asymptotic frequentist properties. Empirical results obtained from numerical experimentation verify this theoretical claim. Lastly, the methodology is applied to a gene expression dataset and discovers new findings for which most existing methods based on parametric linear modeling failed to observe.
Median bias reduction of maximum likelihood estimates

For regular parametric problems, we show how median centering of the maximum likelihood estimate can be achieved by a simple modification of the score equation. For a scalar parameter of interest, the estimator is equivariant under interest respecting parameterizations and third-order median unbiased. With a vector parameter of interest, componentwise equivariance and third-order median centering are obtained. Like Firth's (1993, Biometrika) implicit method for bias reduction, the new method does not require finiteness of the maximum likelihood estimate and is effective in preventing infinite estimates. Simulation results for continuous and discrete models, including binary and beta regression, confirm that the method succeeds in achieving componentwise median centering and in solving the infinite estimate problem, while keeping comparable dispersion and the same approximate distribution as its main competitors.
Large Scale Optimal Transport and Mapping Estimation

This paper presents a novel two-step approach for the fundamental problem of learning an optimal map from one distribution to another. First, we learn an optimal transport (OT) plan, which can be thought as a one-to-many map between the two distributions. To that end, we propose a stochastic dual approach of regularized OT, and show empirically that it scales better than a recent related approach when the amount of samples is very large. Second, we estimate a \textit{Monge map} as a deep neural network learned by approximating the barycentric projection of the previously-obtained OT plan. This parameterization allows generalization of the mapping outside the support of the input measure. We prove two theoretical stability results of regularized OT which show that our estimations converge to the OT plan and Monge map between the underlying continuous measures. We showcase our proposed approach on two applications: domain adaptation and generative modeling.
Semiparametric theory

In this paper we give a brief review of semiparametric theory, using as a running example the common problem of estimating an average causal effect. Semiparametric models allow at least part of the data-generating process to be unspecified and unrestricted, and can often yield robust estimators that nonetheless behave similarly to those based on parametric likelihood assumptions, e.g., fast rates of convergence to normal limiting distributions. We discuss the basics of semiparametric theory, focusing on influence functions.
On Affine and Conjugate Nonparametric Regression

Suppose the nonparametric regression function of a response variable $Y$ on covariates $X$ and $Z$ is an affine function of $X$ such that the slope $\beta$ and the intercept $\alpha$ are real valued measurable functions on the range of the completely arbitrary random element $Z$. Assume that $X$ has a finite moment of order greater than or equal to $2$, $Y$ has a finite moment of conjugate order, and $\alpha\left(Z\right)$ and $\alpha\left(Z\right)X$ have finite first moments. Then, the nonparametric regression function equals the least squares linear regression function of $Y$ on $X$ with all the moments that appear in the expression of the linear regression function calculated conditional on $Z$. Consequently, conditional mean independence implies zero conditional covariance and a degenerate version of the aforesaid affine form for the nonparametric regression function, whereas the aforesaid affine form and zero conditional covariance imply conditional mean independence. Further, it turns out that the nonparametric regression function has the aforesaid affine form if $X$ is Bernoulli, and since $1$ is the conjugate exponent of $\infty$, the least squares linear regression formula for the nonparametric regression function holds when $Y$ has only a finite first moment and $Z$ is completely arbitrary.
Guide Actor-Critic for Continuous Control

Actor-critic methods solve reinforcement learning problems by updating a parameterized policy known as an actor in a direction that increases an estimate of the expected return known as a critic. However, existing actor-critic methods only use values or gradients of the critic to update the policy parameter. In this paper, we propose a novel actor-critic method called the guide actor-critic (GAC). GAC firstly learns a guide actor that locally maximizes the critic and then it updates the policy parameter based on the guide actor by supervised learning. Our main theoretical contributions are two folds. First, we show that GAC updates the guide actor by performing second-order optimization in the action space where the curvature matrix is based on the Hessians of the critic. Second, we show that the deterministic policy gradient method is a special case of GAC when the Hessians are ignored. Through experiments, we show that our method is a promising reinforcement learning method for continuous controls.
A design criterion for symmetric model discrimination based on flexible nominal sets

Experimental design applications for discriminating between models have been hampered by the assumption to know beforehand which model is the true one, which is counter to the very aim of the experiment. Previous approaches to alleviate this requirement were either symmetrizations of asymmetric techniques, or Bayesian, minimax, and sequential approaches. Here we present a genuinely symmetric criterion based on a linearized distance between mean‐value surfaces and the newly introduced tool of flexible nominal sets. We demonstrate the computational efficiency of the approach using the proposed criterion and provide a Monte‐Carlo evaluation of its discrimination performance on the basis of the likelihood ratio. An application for a pair of competing models in enzyme kinetics is given.
Nonparametric modal regression in the presence of measurement error

In the context of regressing a response $Y$ on a predictor $X$, we consider estimating the local modes of the distribution of $Y$ given $X=x$ when $X$ is prone to measurement error. We propose two nonparametric estimation methods, with one based on estimating the joint density of $(X, Y)$ in the presence of measurement error, and the other built upon estimating the conditional density of $Y$ given $X=x$ using error-prone data. We study the asymptotic properties of each proposed mode estimator, and provide implementation details including the mean-shift algorithm for mode seeking and bandwidth selection. Numerical studies are presented to compare the proposed methods with an existing mode estimation method developed for error-free data naively applied to error-prone data.
An Efficient and Distribution-Free Two-Sample Test Based on Energy Statistics and Random Projections

A common disadvantage in existing distribution-free two-sample testing approaches is that the computational complexity could be high. Specifically, if the sample size is $N$, the computational complexity of those two-sample tests is at least $O(N^2)$. In this paper, we develop an efficient algorithm with complexity $O(N \log N)$ for computing energy statistics in univariate cases. For multivariate cases, we introduce a two-sample test based on energy statistics and random projections, which enjoys the $O(K N \log N)$ computational complexity, where $K$ is the number of random projections. We name our method for multivariate cases as Randomly Projected Energy Statistics (RPES). We can show RPES achieves nearly the same test power with energy statistics both theoretically and empirically. Numerical experiments also demonstrate the efficiency of the proposed method over the competitors.
Communication Optimality Trade-offs For Distributed Estimation

This paper proposes $\mathbf{C}$ommunication efficient $\mathbf{RE}$cursive $\mathbf{D}$istributed estimati$\mathbf{O}$n algorithm, $\mathcal{CREDO}$, for networked multi-worker setups without a central master node. $\mathcal{CREDO}$ is designed for scenarios in which the worker nodes aim to collaboratively estimate a vector parameter of interest using distributed online time-series data at the individual worker nodes. The individual worker nodes iteratively update their estimate of the parameter by assimilating latest locally sensed information and estimates from neighboring worker nodes exchanged over a (possibly sparse) time-varying communication graph. The underlying inter-worker communication protocol is adaptive, making communications increasingly (probabilistically) sparse as time progresses. Under minimal conditions on the inter-worker information exchange network and the sensing models, almost sure convergence of the estimate sequences at the worker nodes to the true parameter is established. Further, the paper characterizes the performance of $\mathcal{CREDO}$ in terms of asymptotic covariance of the estimate sequences and specifically establishes the achievability of optimal asymptotic covariance. The analysis reveals an interesting interplay between the algorithm's communication cost~$\mathcal{C}_{t}$ (over $t$ time-steps) and the asymptotic covariance. Most notably, it is shown that $\mathcal{CREDO}$ may be designed to achieve a $\Theta\left(\mathcal{C}_{t}^{-2+\zeta}\right)$ decay of the mean square error~($\zeta>0$, arbitrarily small) at each worker node, which significantly improves over the existing $ \Theta\left(\mathcal{C}_{t}^{-1}\right)$ rates. Simulation examples on both synthetic and real data sets demonstrate $\mathcal{CREDO}$'s communication efficiency.
Statistical properties of sketching algorithms

Summary Sketching is a probabilistic data compression technique that has been largely developed by the computer science community. Numerical operations on big datasets can be intolerably slow; sketching algorithms address this issue by generating a smaller surrogate dataset. Typically, inference proceeds on the compressed dataset. Sketching algorithms generally use random projections to compress the original dataset, and this stochastic generation process makes them amenable to statistical analysis. We argue that the sketched data can be modelled as a random sample, thus placing this family of data compression methods firmly within an inferential framework. In particular, we focus on the Gaussian, Hadamard and Clarkson–Woodruff sketches and their use in single-pass sketching algorithms for linear regression with huge samples. We explore the statistical properties of sketched regression algorithms and derive new distributional results for a large class of sketching estimators. A key result is a conditional central limit theorem for data-oblivious sketches. An important finding is that the best choice of sketching algorithm in terms of mean squared error is related to the signal-to-noise ratio in the source dataset. Finally, we demonstrate the theory and the limits of its applicability on two datasets.
Damped Anderson Acceleration With Restarts and Monotonicity Control for Accelerating EM and EM-like Algorithms

Abstract The expectation-maximization (EM) algorithm is a well-known iterative method for computing maximum likelihood estimates in a variety of statistical problems. Despite its numerous advantages, a main drawback of the EM algorithm is its frequently observed slow convergence which often hinders the application of EM algorithms in high-dimensional problems or in other complex settings. To address the need for more rapidly convergent EM algorithms, we describe a new class of acceleration schemes that build on the Anderson acceleration technique for speeding fixed-point iterations. Our approach is effective at greatly accelerating the convergence of EM algorithms and is automatically scalable to high-dimensional settings. Through the introduction of periodic algorithm restarts and a damping factor, our acceleration scheme provides faster and more robust convergence when compared to un-modified Anderson acceleration, while also improving global convergence. Crucially, our method works as an “off-the-shelf” method in that it may be directly used to accelerate any EM algorithm without relying on the use of any model-specific features or insights. Through a series of simulation studies involving five representative problems, we show that our algorithm is substantially faster than the existing state-of-art acceleration schemes. The acceleration schemes described in this paper are implemented in the R package daarem which is available from the comprehensive R archive network (https://cran.r-project.org). Supplementary materials for this article are available online.
Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low

We study the behavior of a fundamental tool in sparse statistical modeling --the best-subset selection procedure (aka "best-subsets"). Assuming that the underlying linear model is sparse, it is well known, both in theory and in practice, that the best-subsets procedure works extremely well in terms of several statistical metrics (prediction, estimation and variable selection) when the signal to noise ratio (SNR) is high. However, its performance degrades substantially when the SNR is low -- it is outperformed in predictive accuracy by continuous shrinkage methods, such as ridge regression and the Lasso. We explain why this behavior should not come as a surprise, and contend that the original version of the classical best-subsets procedure was, perhaps, not designed to be used in the low SNR regimes. We propose a close cousin of best-subsets, namely, its $\ell_{q}$-regularized version, for $q \in\{1, 2\}$, which (a) mitigates, to a large extent, the poor predictive performance of best-subsets in the low SNR regimes; (b) performs favorably and generally delivers a substantially sparser model when compared to the best predictive models available via ridge regression and the Lasso. Our estimator can be expressed as a solution to a mixed integer second order conic optimization problem and, hence, is amenable to modern computational tools from mathematical optimization. We explore the theoretical properties of the predictive capabilities of the proposed estimator and complement our findings via several numerical experiments.
Network Estimation via Graphon With Node Features

One popular model for network analysis is the exchangeable graph model (ExGM), which is characterized by a two-dimensional function known as a graphon. Estimating an underlying graphon becomes the key of such analysis. Several nonparametric estimation methods have been proposed, and some are provably consistent. However, if certain useful features of the nodes (e.g., age and schools in a social network context) are available, none of these methods were designed to incorporate this source of information to help with the estimation. This paper develops a consistent graphon estimation method that integrates information from both the adjacency matrix itself and node features. We show that properly leveraging the features can improve the estimation. A cross-validation method is proposed to automatically select the tuning parameter of the method.
Modular Bayes screening for high-dimensional predictors

With the routine collection of massive-dimensional predictors in many application areas, screening methods that rapidly identify a small subset of promising predictors have become commonplace. We propose a new MOdular Bayes Screening (MOBS) approach, which involves several novel characteristics that can potentially lead to improved performance. MOBS first applies a Bayesian mixture model to the marginal distribution of the response, obtaining posterior samples of mixture weights, cluster-specific parameters, and cluster allocations for each subject. Hypothesis tests are then introduced, corresponding to whether or not to include a given predictor, with posterior probabilities for each hypothesis available analytically conditionally on unknowns sampled in the first stage and tuning parameters controlling borrowing of information across tests. By marginalizing over the first stage posterior samples, we avoid under-estimation of uncertainty typical of two-stage methods. We greatly simplify the model specification and reduce computational complexity by using {\em modularization}. We provide basic theoretical support for this approach, and illustrate excellent performance relative to competitors in simulation studies and the ability to capture complex shifts beyond simple differences in means. The method is illustrated with applications to genomics by using a very high-dimensional cis-eQTL dataset with roughly 38 million SNPs.