Miracle-dz/UniFAR-FTUs huggingface dataset:
Accelerating Metropolis-Hastings algorithms by Delayed Acceptance

MCMC algorithms such as Metropolis-Hastings algorithms are slowed down by the computation of complex target distributions as exemplified by huge datasets. We offer in this paper a useful generalisation of the Delayed Acceptance approach, devised to reduce the computational costs of such algorithms by a simple and universal divide-and-conquer strategy. The idea behind the generic acceleration is to divide the acceptance step into several parts, aiming at a major reduction in computing time that out-ranks the corresponding reduction in acceptance probability. Each of the components can be sequentially compared with a uniform variate, the first rejection signalling that the proposed value is considered no further. We develop moreover theoretical bounds for the variance of associated estimators with respect to the variance of the standard Metropolis-Hastings and detail some results on optimal scaling and general optimisation of the procedure. We illustrate those accelerating features on a series of examples
Cramér-type moderate deviations for Studentized two-sample $U$-statistics with applications

Two-sample $U$-statistics are widely used in a broad range of applications, including those in the fields of biostatistics and econometrics. In this paper, we establish sharp Cramer-type moderate deviation theorems for Studentized two-sample $U$-statistics in a general framework, including the two-sample $t$-statistic and Studentized Mann-Whitney test statistic as prototypical examples. In particular, a refined moderate deviation theorem with second-order accuracy is established for the two-sample $t$-statistic. These results extend the applicability of the existing statistical methodologies from the one-sample $t$-statistic to more general nonlinear statistics. Applications to two-sample large-scale multiple testing problems with false discovery rate control and the regularized bootstrap method are also discussed.
A Dynamic Structure for High-Dimensional Covariance Matrices and Its Application in Portfolio Allocation

ABSTRACT Estimation of high-dimensional covariance matrices is an interesting and important research topic. In this article, we propose a dynamic structure and develop an estimation procedure for high-dimensional covariance matrices. Asymptotic properties are derived to justify the estimation procedure and simulation studies are conducted to demonstrate its performance when the sample size is finite. By exploring a financial application, an empirical study shows that portfolio allocation based on dynamic high-dimensional covariance matrices can significantly outperform the market from 1995 to 2014. Our proposed method also outperforms portfolio allocation based on the sample covariance matrix, the covariance matrix based on factor models, and the shrinkage estimator of covariance matrix. Supplementary materials for this article are available online.
Robust Estimation of Propensity Score Weights via Subclassification

The propensity score plays a central role in inferring causal effects from observational studies. In particular, weighting and subclassification are two principal approaches to estimate the average causal effect based on estimated propensity scores. Unlike the conventional version of the propensity score subclassification estimator, if the propensity score model is correctly specified, the weighting methods offer consistent and possibly efficient estimation of the average causal effect. However, this theoretical appeal may be diminished in practice by sensitivity to misspecification of the propensity score model. In contrast, subclassification methods are usually more robust to model misspecification. We hence propose to use subclassification for robust estimation of propensity score weights. Our approach is based on the intuition that the inverse probability weighting estimator can be seen as the limit of subclassification estimators as the number of subclasses goes to infinity. By formalizing this intuition, we propose novel propensity score weighting estimators that are both consistent and robust to model misspecification. Empirical studies show that the proposed estimators perform favorably compared to existing methods.
Spatially Varying Empirical Bayes Approach to Area-level models

We consider two-stage hierarchical area-level models based on natural exponentially family for small area estimation. While the areas are treated exchangeably and the model parameters are assumed to be the same over the areas, we might lose the efficiency when there exist spatial heterogeneity. To overcome this problem, we here propose two-stage area-level model with spatially varying model parameters and local marginal likelihood approach to estimating these parameters to compute empirical Bayes estimators of area means. We also discuss some related problems including mean squared error estimation, benchmarked estimation and estimation in non-sampled areas. The performance of the proposed method is evaluated through simulations and applications to two data sets.
Comparing Trajectories on the Size and Shape Space

In this paper we show that trajectory shape analysis should be performed only after obtaining a proper representation before applying ordination methods. In fact, studying the shape of a trajectory means studying how the deformation changes along each path irrespectively of the actual shape to which these deformations apply. The independence of the deformation from the shape to which it is applied is critical: it implies that any shape variation between individuals at the beginning of each trajectories must be completely filtered out. A Parallel Transport, that can be based on various connection types, is necessary to perform such kind of shape data centering. The Levi Civita connection can also be used to transport a deformation. We demonstrate that this procedure does not preserve deformation even in the ane case. We propose a novel procedure called Direct Transport able to perfectly transport deformation in the ane case and to better approximate non ane deformation in comparison to existing tools.
Boosted Nonparametric Hazards with Time-Dependent Covariates

Given functional data from a survival process with time-dependent covariates, we derive a smooth convex representation for its nonparametric log-likelihood functional and obtain its functional gradient. From this we devise a generic gradient boosting procedure for estimating the hazard function nonparametrically. An illustrative implementation of the procedure using regression trees is described to show how to recover the unknown hazard. The generic estimator is consistent if the model is correctly specified; alternatively an oracle inequality can be demonstrated for tree-based models. To avoid overfitting, boosting employs several regularization devices. One of them is step-size restriction, but the rationale for this is somewhat mysterious from the viewpoint of consistency. Our work brings some clarity to this issue by revealing that step-size restriction is a mechanism for preventing the curvature of the risk from derailing convergence.
High-dimensional consistency in score-based and hybrid structure learning

Main approaches for learning Bayesian networks can be classified as constraint-based, score-based or hybrid methods. Although high-dimensional consistency results are available for constraint-based methods like the PC algorithm, such results have not been proved for score-based or hybrid methods, and most of the hybrid methods have not even shown to be consistent in the classical setting where the number of variables remains fixed and the sample size tends to infinity. In this paper, we show that consistency of hybrid methods based on greedy equivalence search (GES) can be achieved in the classical setting with adaptive restrictions on the search space that depend on the current state of the algorithm. Moreover, we prove consistency of GES and adaptively restricted GES (ARGES) in several sparse high-dimensional settings. ARGES scales well to sparse graphs with thousands of variables and our simulation study indicates that both GES and ARGES generally outperform the PC algorithm.
Inference for penalized spline regression: Improving confidence intervals by reducing the penalty

Penalized spline regression is a popular method for scatterplot smoothing, but there has long been a debate on how to construct confidence intervals for penalized spline fits. Due to the penalty, the fitted smooth curve is a biased estimate of the target function. Many methods, including Bayesian intervals and the simple-shift bias-reduction, have been proposed to upgrade the coverage of the confidence intervals, but these methods usually fail to adequately improve the situation at predictor values where the function is sharply curved. In this paper, we develop a novel approach to improving the confidence intervals by using a smaller smoothing strength than that of the spline fits. With a carefully selected amount of reduction in smoothing strength, the confidence intervals achieve nearly nominal coverage without being excessively wide or wiggly. The coverage performance of the proposed method is investigated via simulation experiments in comparison with the bias-correction techniques proposed by Hodges (2013) and Kuusela and Panaretos (2015).
Normal approximation and concentration of spectral projectors of sample covariance

Let $X,X_1,\dots, X_n$ be i.i. d. Gaussian random variables in a separable Hilbert space ${\mathbb H}$ with zero mean and covariance operator $\Sigma={\mathbb E}(X\otimes X),$ and let $\hat \Sigma:=n^{-1}\sum_{j=1}^n (X_j\otimes X_j)$ be the sample (empirical) covariance operator based on $(X_1,\dots, X_n). $ Denote by $P_r$ the spectral projector of $\Sigma$ corresponding to its $r$-th eigenvalue $\mu_r$ and by $\hat P_r$ the empirical counterpart of $P_r. $ The main goal of the paper is to obtain tight bounds on $$ \sup_{x\in {\mathbb R}} \left|{\mathbb P}\left\{\frac{\|\hat P_r-P_r\|_2^2-{\mathbb E}\|\hat P_r-P_r\|_2^2}{{\rm Var}^{1/2}(\|\hat P_r-P_r\|_2^2)}\leq x\right\}-\Phi(x)\right|, $$ where $\|\cdot\|_2$ denotes the Hilbert--Schmidt norm and $\Phi$ is the standard normal distribution function. Such accuracy of normal approximation of the distribution of squared Hilbert--Schmidt error is characterized in terms of so called effective rank of $\Sigma$ defined as ${\bf r}(\Sigma)=\frac{{\rm tr}(\Sigma)}{\|\Sigma\|_{\infty}},$ where ${\rm tr}(\Sigma)$ is the trace of $\Sigma$ and $\|\Sigma\|_{\infty}$ is its operator norm, as well as another parameter characterizing the size of ${\rm Var}(\|\hat P_r-P_r\|_2^2). $ Other results include non-asymptotic bounds and asymptotic representations for the mean squared Hilbert--Schmidt norm error ${\mathbb E}\|\hat P_r-P_r\|_2^2$ and the variance ${\rm Var}(\|\hat P_r-P_r\|_2^2),$ and concentration inequalities for $\|\hat P_r-P_r\|_2^2$ around its expectation.
Bayesian Trend Filtering

We develop a fully Bayesian hierarchical model for trend filtering, itself a new development in nonparametric, univariate regression. The framework more broadly applies to the generalized lasso, but focus is on Bayesian trend filtering. We compare two shrinkage priors, double exponential and generalized double Pareto. A simulation study, comparing Bayesian trend filtering to the original formulation and a number of other popular methods shows our method to improve estimation error while maintaining if not improving coverage probability. Two time series data sets demonstrate Bayesian trend filtering's robustness to possible violations of its assumptions.
Modeling competition between two pharmaceutical drugs using innovation diffusion models

The study of competition among brands in a common category is an interesting strategic issue for involved firms. Sales monitoring and prediction of competitors' performance represent relevant tools for management. In the pharmaceutical market, the diffusion of product knowledge plays a special role, different from the role it plays in other competing fields. This latent feature naturally affects the evolution of drugs' performances in terms of the number of packages sold. In this paper, we propose an innovation diffusion model that takes the spread of knowledge into account. We are motivated by the need of modeling competition of two antidiabetic drugs in the Italian market.
Inference in High-Dimensional Linear Regression Models

We introduce an asymptotically unbiased estimator for the full high-dimensional parameter vector in linear regression models where the number of variables exceeds the number of available observations. The estimator is accompanied by a closed-form expression for the covariance matrix of the estimates that is free of tuning parameters. This enables the construction of confidence intervals that are valid uniformly over the parameter vector. Estimates are obtained by using a scaled Moore-Penrose pseudoinverse as an approximate inverse of the singular empirical covariance matrix of the regressors. The approximation induces a bias, which is then corrected for using the lasso. Regularization of the pseudoinverse is shown to yield narrower confidence intervals under a suitable choice of the regularization parameter. The methods are illustrated in Monte Carlo experiments and in an empirical example where gross domestic product is explained by a large number of macroeconomic and financial indicators.
Estimating the observable population size from biased samples: a new approach to population estimation with capture heterogeneity

Capture-recapture methods aim to estimate the size of a closed population on the basis of multiple incomplete enumerations of individuals. In many applications, the individual probability of being recorded is heterogeneous in the population. Previous studies have suggested that it is not possible to reliably estimate the total population size when capture heterogeneity exists. Here we approach population estimation in the presence of capture heterogeneity as a latent length biased nonparametric density estimation problem on the unit interval. We show that in this setting it is generally impossible to estimate the density on the entire unit interval in finite samples, and that estimators of the population size have high and sometimes unbounded risk when the density has significant mass near zero. As an alternative, we propose estimating the population of individuals with capture probability exceeding some threshold. We provide methods for selecting an appropriate threshold, and show that this approach results in estimators with substantially lower risk than estimators of the total population size, with correspondingly smaller uncertainty, even when the parameter of interest is the total population. The alternative paradigm is demonstrated in extensive simulation studies and an application to snowshoe hare multiple recapture data.
Spatial clustering of time series via mixture of autoregressions models and Markov random fields

Time series data arise in many medical and biological imaging scenarios. In such images, a time series is obtained at each of a large number of spatially dependent data units. It is interesting to organize these data into model‐based clusters. A two‐stage procedure is proposed. In stage 1, a mixture of autoregressions (MoAR) model is used to marginally cluster the data. The MoAR model is fitted using maximum marginal likelihood (MMaL) estimation via a minorization–maximization (MM) algorithm. In stage 2, a Markov random field (MRF) model induces a spatial structure onto the stage 1 clustering. The MRF model is fitted using maximum pseudolikelihood (MPL) estimation via an MM algorithm. Both the MMaL and MPL estimators are proved to be consistent. Numerical properties are established for both MM algorithms. A simulation study demonstrates the performance of the two‐stage procedure. An application to the segmentation of a zebrafish brain calcium image is presented.
Optimal discrimination design for copula models

Optimum experimental design theory has recently been extended for parameter estimation in copula models. However, the choice of the correct dependence structure still requires wider analyses. In this work the issue of copula selection is treated by using discrimination design techniques. The new proposed approach consists in the use of $D_s$-optimality following an extension of corresponding equivalence theory. We also present some examples and highlight the strength of such a criterion as a way to discriminate between various classes of dependences.
Modeling and Estimation for Self-Exciting Spatio-Temporal Models of Terrorist Activity

Spatio-temporal hierarchical modeling is an extremely attractive way to model the spread of crime or terrorism data over a given region, especially when the observations are counts and must be modeled discretely. The spatio-temporal diffusion is placed, as a matter of convenience, in the process model allowing for straightforward estimation of the diffusion parameters through Bayesian techniques. However, this method of modeling does not allow for the existence of self-excitation, or a temporal data model dependency, that has been shown to exist in criminal and terrorism data. In this manuscript we will use existing theories on how violence spreads to create models that allow for both spatio-temporal diffusion in the process model as well as temporal diffusion, or self-excitation, in the data model. We will further demonstrate how Laplace approximations similar to their use in Integrated Nested Laplace Approximation can be used to quickly and accurately conduct inference of self-exciting spatio-temporal models allowing practitioners a new way of fitting and comparing multiple process models. We will illustrate this approach by fitting a self-exciting spatio-temporal model to terrorism data in Iraq and demonstrate how choice of process model leads to differing conclusions on the existence of self-excitation in the data and differing conclusions on how violence is spreading spatio-temporally.
Estimating the number of change-points in a two-dimensional segmentation model without penalization

In computational biology, numerous recent studies have been dedicated to the analysis of the chromatin structure within the cell by two-dimensional segmentation methods. Motivated by this application, we consider the problem of retrieving the diagonal blocks in a matrix of observations. The theoretical properties of the least-squares estimators of both the boundaries and the number of blocks proposed by L\'evy-Leduc et al. [2014] are investigated. More precisely, the contribution of the paper is to establish the consistency of these estimators. A surprising consequence of our results is that, contrary to the onedimensional case, a penalty is not needed for retrieving the true number of diagonal blocks. Finally, the results are illustrated on synthetic data.
Non-strange Weird Resampling for Complex Survival Data

This paper introduces the new data-dependent multiplier bootstrap for non-parametric analysis of survival data, possibly subject to competing risks. The new resampling procedure includes both the general wild bootstrap and the weird bootstrap as special cases. The data may be subject to independent right-censoring and left-truncation. We rigorously prove asymptotic correctness which has in particular been pending for the weird bootstrap. As a consequence, pointwise as well as time-simultaneous inference procedures for, amongst others, the classical survival setting are deduced. We report simulation results and a real data analysis of the cumulative cardiovascular event probability. The simulation results suggest that both the weird bootstrap and use of non-standard multipliers in the wild bootstrap may perform preferably.
A local approach to estimation in discrete loglinear models

We consider two connected aspects of maximum likelihood estimation of the parameter for high-dimensional discrete graphical models: the existence of the maximum likelihood estimate (mle) and its computation. When the data is sparse, there are many zeros in the contingency table and the maximum likelihood estimate of the parameter may not exist. Fienberg and Rinaldo (2012) have shown that the mle does not exists i the data vector belongs to a face of the so-called marginal cone spanned by the rows of the design matrix of the model. Identifying these faces in high-dimension is challenging. In this paper, we take a local approach : we show that one such face, albeit possibly not the smallest one, can be identied by looking at a collection of marginal graphical models generated by induced subgraphs Gi;i = 1;:::;k of G. This is our rst contribution. Our second contribution concerns the composite maximum likelihood estimate. When the dimension of the problem is large, estimating the parameters of a given graphical model through maximum likelihood is challenging, if not impossible. The traditional approach to this problem has been local with the use of composite likelihood based on local conditional likelihoods. A more recent development is to have the components of the composite likelihood be marginal likelihoods centred around each v. We rst show that the estimates obtained by consensus through local conditional and marginal likelihoods are identical. We then study the asymptotic properties of the composite maximum likelihood estimate when both the dimension of the model and the sample size N go to innity. H. Massam gratefully acknowledges support from NSERC Discovery Grant No A8947.
A Bayes Factor for Replications of ANOVA Results

ABSTRACT With an increasing number of replication studies performed in psychological science, the question of how to evaluate the outcome of a replication attempt deserves careful consideration. Bayesian approaches allow to incorporate uncertainty and prior information into the analysis of the replication attempt by their design. The Replication Bayes factor, introduced by Verhagen and Wagenmakers (2014), provides quantitative, relative evidence in favor or against a successful replication. In previous work by Verhagen and Wagenmakers (2014), it was limited to the case of t-tests. In this article, the Replication Bayes factor is extended to F-tests in multigroup, fixed-effect ANOVA designs. Simulations and examples are presented to facilitate the understanding and to demonstrate the usefulness of this approach. Finally, the Replication Bayes factor is compared to other Bayesian and frequentist approaches and discussed in the context of replication attempts. R code to calculate Replication Bayes factors and to reproduce the examples in the article is available at https://osf.io/jv39h/.
A nonparametric bayesian technique for high-dimensional regression

This paper proposes a nonparametric Bayesian framework called VariScan for simultaneous clustering, variable selection, and prediction in high-throughput regression settings. Poisson-Dirichlet processes are utilized to detect lower-dimensional latent clusters of covariates. An adaptive nonlinear prediction model is constructed for the response, achieving a balance between model parsimony and flexibility. Contrary to conventional belief, cluster detection is shown to be aposteriori consistent for a general class of models as the number of covariates and subjects grows. Simulation studies and data analyses demonstrate that VariScan often outperforms several well-known statistical methods.
ASYMPTOTIC DISTRIBUTIONS RELATED TO MILDLY-EXPLOSIVE SECOND ORDER AUTOREGRESSIVE MODELS

In this paper, we consider the normalized least squares estimator of the parameter in a mildly-explosive first-order autoregressive model with dependent errors which are modeled as a mildly-explosive AR(1) process. We prove that the estimator has a Cauchy limit law which provides a bridge be- tween moderate deviation asymptotics and the earlier results on the local to unity and explosive autoregressive models. In particular, the results can be applied to understand the near-integrated second order autoregressive pro- cesses. Simulation studies are also carried out to assess the performance of least squares estimation in finite samples.