Miracle-dz/UniFAR-FTUs huggingface dataset:
Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings

In the practice of point prediction, it is desirable that forecasters receive a directive in the form of a statistical functional. For example, forecasters might be asked to report the mean or a quantile of their predictive distributions. When evaluating and comparing competing forecasts, it is then critical that the scoring function used for these purposes be consistent for the functional at hand, in the sense that the expected score is minimized when following the directive. We show that any scoring function that is consistent for a quantile or an expectile functional can be represented as a mixture of elementary or extremal scoring functions that form a linearly parameterized family. Scoring functions for the mean value and probability forecasts of binary events constitute important examples. The extremal scoring functions admit appealing economic interpretations of quantiles and expectiles in the context of betting and investment problems. The Choquet‐type mixture representations give rise to simple checks of whether a forecast dominates another in the sense that it is preferable under any consistent scoring function. In empirical settings it suffices to compare the average scores for only a finite number of extremal elements. Plots of the average scores with respect to the extremal scoring functions, which we call Murphy diagrams, permit detailed comparisons of the relative merits of competing forecasts.
Another Generalization of Unimodality

A general characterization for α-unimodal distributions was provided by Alamatsaz (1985) who later introduced a multivariate extension of them (Alamatsaz 1993). Here, by solving the related equations, another generalization for unimodality is presented. As a result of this generalization, a simpler proof of a conjecture, as well as a characterization for generalized arcsin distributions and some generalizations of the author’s earlier works, have been obtained. Last, but not the least, it is shown that some elementary methods can be more powerful than some more advanced techniques.
MCMC with Strings and Branes: The Suburban Algorithm

Motivated by the physics of strings and branes, we introduce a general suite of Markov chain Monte Carlo (MCMC) "suburban samplers" (i.e., spread out Metropolis). The suburban algorithm involves an ensemble of statistical agents connected together by a random network. Performance of the collective in reaching a fast and accurate inference depends primarily on the average number of nearest neighbor connections. Increasing the average number of neighbors above zero initially leads to an increase in performance, though there is a critical connectivity with effective dimension d_eff ~ 1, above which "groupthink" takes over, and the performance of the sampler declines.
A unified framework for weighted parametric multiple test procedures

We describe a general framework for weighted parametric multiple test procedures based on the closure principle. We utilize general weighting strategies that can reflect complex study objectives and include many procedures in the literature as special cases. The proposed weighted parametric tests bridge the gap between rejection rules using either adjusted significance levels or adjusted p‐values. This connection is made by allowing intersection hypotheses of the underlying closed test procedure to be tested at level smaller than α. This may be also necessary to take certain study situations into account. For such cases we introduce a subclass of exact α‐level parametric tests that satisfy the consonance property. When the correlation is known only for certain subsets of the test statistics, a new procedure is proposed to fully utilize this knowledge within each subset. We illustrate the proposed weighted parametric tests using a clinical trial example and conduct a simulation study to investigate its operating characteristics.
Regularization and Confounding in Linear Regression for Treatment Effect Estimation

This paper investigates the use of regularization priors in the context of treatment effect estimation using observational data where the number of control variables is large relative to the number of observations. A reparameterized simultaneous regression model is presented which permits prior specifications designed to overcome certain pitfalls of the more conventional direct parametrization. The new approach is illustrated on synthetic and empirical data.
Robustness Metric for Quantifying Causal Model Confidence and Parameter Uncertainty

Many methods of estimating causal models do not provide estimates of confidence in the resulting model. In this work, a metric is proposed for validating the output of a causal model fit; the robustness of the model structure with resampled data. The metric is developed for time series causal models, but is also applicable to non-time series data. The proposed metric may be utilized regardless of the method selected for fitting the causal model. We find that with synthetically generated data, this metric is able to successfully identify the true data generating model in most cases. Additionally, the metric provides both a qualitative measure of model confidence represented by the robustness level as well as accurate estimates of uncertainty in model coefficients which are important in interpreting model results. The use of this metric is demonstrated on both numerically simulated data and a case study from existing causal model literature.
An adaptable generalization of Hotelling’s $T^{2}$ test in high dimension

We propose a two-sample test for detecting the difference between mean vectors in a high-dimensional regime based on a ridge-regularized Hotelling's $T^2$. To choose the regularization parameter, a method is derived that aims at maximizing power within a class of local alternatives. We also propose a composite test that combines the optimal tests corresponding to a specific collection of local alternatives. Weak convergence of the stochastic process corresponding to the ridge-regularized Hotelling's $T^2$ is established and used to derive the cut-off values of the proposed test. Large sample properties are verified for a class of sub-Gaussian distributions. Through an extensive simulation study, the composite test is shown to compare favorably against a host of existing two-sample test procedure in a wide range of settings. The performance of the proposed test procedure is illustrated through an application to a breast cancer data set where the goal is to detect the pathways with different DNA copy number alterations across breast cancer subtypes.
Simulation Study Comparing Two Tests of Second-order Stationarity and Confidence Intervals for Localized Autocovariance

This report compares two tests of second-order stationarity through simulation. It also provides several examples of localised autocovariances and their approximate confidence intervals on different real and simulated data sets. An empirical verification of an asymptotic Gaussianity result is also demonstrated. The commands use to produce figures in a companion paper are also described.
Sequential detection of three-dimensional signals under dependent noise

ABSTRACT We study detection methods for multivariable signals under dependent noise. The main focus is on three-dimensional signals; that is, on signals in the space–time domain. Examples for such signals are multifaceted. They include geographic and climatic data as well as image data that are observed over a fixed time horizon. We assume that the signal is observed as a finite block of noisy samples whereby we are interested in detecting changes from a given reference signal. Our detector statistic is based on a sequential partial sum process, related to classical signal decomposition and reconstruction approaches applied to the sampled signal. We show that this detector process converges weakly under the no change null hypothesis that the signal coincides with the reference signal, provided that the spatial–temporal partial sum process associated with the random field of the noise terms disturbing the sampled signal converges to a Brownian motion. More generally, we also establish the limiting distribution under a wide class of local alternatives that allows for smooth as well as discontinuous changes. Our results also cover extensions to the case in which the reference signal is unknown. We conclude with an extensive simulation study of the detection algorithm.
Light and Widely Applicable MCMC: Approximate Bayesian Inference for Large Datasets

Light and Widely Applicable (LWA-) MCMC is a novel approximation of the Metropolis‐ Hastings kernel targeting a posterior distribution defined on a large number of observations. Inspired by Approximate Bayesian Computation, we design a Markov chain whose transition makes use of an unknown but fixed, fraction of the available data, where the random choice of sub-sample is guided by the fidelity of this sub-sample to the observed data, as measured by summary (or sucient) statistics. LWA‐MCMC is a generic and
Asynchronous Gibbs Sampling

Gibbs sampling is a Markov Chain Monte Carlo (MCMC) method often used in Bayesian learning. MCMC methods can be difficult to deploy on parallel and distributed systems due to their inherently sequential nature. We study asynchronous Gibbs sampling, which achieves parallelism by simply ignoring sequential requirements. This method has been shown to produce good empirical results for some hierarchical models, and is popular in the topic modeling community, but was also shown to diverge for other targets. We introduce a theoretical framework for analyzing asynchronous Gibbs sampling and other extensions of MCMC that do not possess the Markov property. We prove that asynchronous Gibbs can be modified so that it converges under appropriate regularity conditions -- we call this the exact asynchronous Gibbs algorithm. We study asynchronous Gibbs on a set of examples by comparing the exact and approximate algorithms, including two where it works well, and one where it fails dramatically. We conclude with a set of heuristics to describe settings where the algorithm can be effectively used.
A Multiresolution Stochastic Process Model for Predicting Basketball Possession Outcomes

ABSTRACT Basketball games evolve continuously in space and time as players constantly interact with their teammates, the opposing team, and the ball. However, current analyses of basketball outcomes rely on discretized summaries of the game that reduce such interactions to tallies of points, assists, and similar events. In this article, we propose a framework for using optical player tracking data to estimate, in real time, the expected number of points obtained by the end of a possession. This quantity, called expected possession value (EPV), derives from a stochastic process model for the evolution of a basketball possession. We model this process at multiple levels of resolution, differentiating between continuous, infinitesimal movements of players, and discrete events such as shot attempts and turnovers. Transition kernels are estimated using hierarchical spatiotemporal models that share information across players while remaining computationally tractable on very large data sets. In addition to estimating EPV, these models reveal novel insights on players’ decision-making tendencies as a function of their spatial strategy. In the supplementary material, we provide a data sample and R code for further exploration of our model and its results.
Dynamic Question Ordering in Online Surveys

Abstract Online surveys have the potential to support adaptive questions, where later questions depend on earlier responses. Past work has taken a rule-based approach, uniformly across all respondents. We envision a richer interpretation of adaptive questions, which we call Dynamic Question Ordering (DQO), where question order is personalized. Such an approach could increase engagement, and therefore response rate, as well as imputation quality. We present a DQO framework to improve survey completion and imputation. In the general survey-taking setting, we want to maximize survey completion, and so we focus on ordering questions to engage the respondent and collect hopefully all information, or at least the information that most characterizes the respondent, for accurate imputations. In another scenario, our goal is to provide a personalized prediction. Since it is possible to give reasonable predictions with only a subset of questions, we are not concerned with motivating users to answer all questions. Instead, we want to order questions to get information that reduces prediction uncertainty, while not being too burdensome. We illustrate this framework with two case studies, for the prediction and survey-taking settings. We also discuss DQO for national surveys and consider connections between our statistics-based question-ordering approach and cognitive survey methodology.
Inference in generative models using the Wasserstein distance

In purely generative models, one can simulate data given parameters but not necessarily evaluate the likelihood. We use Wasserstein distances between empirical distributions of observed data and empirical distributions of synthetic data drawn from such models to estimate their parameters. Previous interest in the Wasserstein distance for statistical inference has been mainly theoretical, due to computational limitations. Thanks to recent advances in numerical transport, the computation of these distances has become feasible, up to controllable approximation errors. We leverage these advances to propose point estimators and quasi-Bayesian distributions for parameter inference, first for independent data. For dependent data, we extend the approach by using delay reconstruction and residual reconstruction techniques. For large data sets, we propose an alternative distance using the Hilbert space-filling curve, which computation scales as nlogn where n is the size of the data. We provide a theoretical study of the proposed estimators, and adaptive Monte Carlo algorithms to approximate them. The approach is illustrated on four examples: a quantile g-and-k distribution, a toggle switch model from systems biology, a Lotka-Volterra model for plankton population sizes and a L\'evy-driven stochastic volatility model.
Localizing the Ensemble Kalman Particle Filter

Ensemble methods such as the Ensemble Kalman Filter (EnKF) are widely used for data assimilation in large-scale geophysical applications, as for example in numerical weather prediction. There is a growing interest for physical models with higher and higher resolution, which brings new challenges for data assimilation techniques because of the presence of non-linear and non-Gaussian features that are not adequately treated by the EnKF. We propose two new localized algorithms based on the Ensemble Kalman Particle Filter, a hybrid method combining the EnKF and the Particle Filter (PF) in a way that maintains scalability and sample diversity. Localization is a key element of the success of EnKF in practice, but it is much more challenging to apply to PFs. The algorithms that we introduce in the present paper provide a compromise between the EnKF and the PF while avoiding some of the problems of localization for pure PFs. Numerical experiments with a simplified model of cumulus convection based on a modified shallow water equation show that the proposed algorithms perform better than the local EnKF. In particular, the PF nature of the method allows to capture non-Gaussian characteristics of the estimated fields such as the location of wet and dry areas.
The Block Pseudo-Marginal Sampler

The pseudo-marginal (PM) approach is increasingly used for Bayesian inference in statistical models, where the likelihood is intractable but can be estimated unbiasedly. %Examples include random effect models, state-space models and data subsampling in big-data settings. Deligiannidis et al. (2016) show how the PM approach can be made much more efficient by correlating the underlying Monte Carlo (MC) random numbers used to form the estimate of the likelihood at the current and proposed values of the unknown parameters. Their approach greatly speeds up the standard PM algorithm, as it requires a much smaller number of samples or particles to form the optimal likelihood estimate. Our paper presents an alternative implementation of the correlated PM approach, called the block PM, which divides the underlying random numbers into blocks so that the likelihood estimates for the proposed and current values of the parameters only differ by the random numbers in one block. We show that this implementation of the correlated PM can be much more efficient for some specific problems than the implementation in Deligiannidis et al. (2016); for example when the likelihood is estimated by subsampling or the likelihood is a product of terms each of which is given by an integral which can be estimated unbiasedly by randomised quasi-Monte Carlo. Our article provides methodology and guidelines for efficiently implementing the block PM. A second advantage of the the block PM is that it provides a direct way to control the correlation between the logarithms of the estimates of the likelihood at the current and proposed values of the parameters than the implementation in Deligiannidis et al. (2016). We obtain methods and guidelines for selecting the optimal number of samples based on idealized but realistic assumptions.
High-Dimensional Semiparametric Selection Models: Estimation Theory with an Application to the Retail Gasoline Market

This paper proposes a multi-stage projection-based Lasso procedure for the semiparametric sample selection model in high-dimensional settings under a weak nonparametric restriction on the selection correction. In particular, the number of regressors in the main equation, p, and the number of regressors in the selection equation, d, can grow with and exceed the sample size n. The analysis considers the exact sparsity case and the approximate sparsity case. The main theoretical results are finite-sample bounds from which sufficient scaling conditions on the sample size for estimation consistency and variable-selection consistency are established. Statistical efficiency of the proposed estimators is studied via lower bounds on minimax risks and the result shows that, for a family of models with exactly sparse structure on the coefficient vector in the main equation, one of the proposed estimators attains the smallest estimation error up to the (n,d,p)-scaling among a class of procedures in worst-case scenarios. Inference procedures for the coefficients of the main equation, one based on a pivotal Dantzig selector to construct non-asymptotic confidence sets and one based on a post-selection strategy, are discussed. Other theoretical contributions include establishing the non-asymptotic counterpart of the familiar asymptotic oracle results from previous literature: the estimator of the coefficients in the main equation behaves as if the unknown nonparametric component were known, provided the nonparametric component is sufficiently smooth. Small-sample performance of the high-dimensional multi-stage estimation procedure is evaluated by Monte-Carlo simulations and illustrated with an empirical application to the retail gasoline market in the Greater Saint Louis area.
(Non-) asymptotic properties of Stochastic Gradient Langevin Dynamics

Applying standard Markov chain Monte Carlo (MCMC) algorithms to large data sets is computationally infeasible. The recently proposed stochastic gradient Langevin dynamics (SGLD) method circumvents this problem in three ways: it generates proposed moves using only a subset of the data, it skips the Metropolis-Hastings accept-reject step, and it uses sequences of decreasing step sizes. In \cite{TehThierryVollmerSGLD2014}, we provided the mathematical foundations for the decreasing step size SGLD, including consistency and a central limit theorem. However, in practice the SGLD is run for a relatively small number of iterations, and its step size is not decreased to zero. The present article investigates the behaviour of the SGLD with fixed step size. In particular we characterise the asymptotic bias explicitly, along with its dependence on the step size and the variance of the stochastic gradient. On that basis a modified SGLD which removes the asymptotic bias due to the variance of the stochastic gradients up to first order in the step size is derived. Moreover, we are able to obtain bounds on the finite-time bias, variance and mean squared error (MSE). The theory is illustrated with a Gaussian toy model for which the bias and the MSE for the estimation of moments can be obtained explicitly. For this toy model we study the gain of the SGLD over the standard Euler method in the limit of large data sets.
Hierarchical models for independence structures of networks

We introduce a new family of network models, called hierarchical network models, that allow us to represent in an explicit manner the stochastic dependence among the dyads (random ties) of the network. In particular, each member of this family can be associated with a graphical model defining conditional independence clauses among the dyads of the network, called the dependency graph. Every network model with dyadic independence assumption can be generalized to construct members of this new family. Using this new framework, we generalize the Erdös–Rényi and the β models to create hierarchical Erdös–Rényi and β models. We describe various methods for parameter estimation, as well as simulation studies for models with sparse dependency graphs.
Uncertainty quantification under group sparsity

Summary Quantifying the uncertainty in penalized regression under group sparsity is an important open question. We establish, under a high‐dimensional scaling, the asymptotic validity of a modified parametric bootstrap method for the group lasso, assuming a Gaussian error model and mild conditions on the design matrix and the true coefficients. Simulation of bootstrap samples provides simultaneous inferences on large groups of coefficients. Through extensive numerical comparisons, we demonstrate that our bootstrap method performs much better than popular competitors, highlighting its practical utility. The theoretical results generalize to other block norm penalization and sub‐Gaussian errors, which further broadens the potential applications.
Robust causal inference with continuous instruments using the local instrumental variable curve

Instrumental variables are commonly used to estimate effects of a treatment afflicted by unmeasured confounding, and in practice instruments are often continuous (e.g. measures of distance, or treatment preference). However, available methods for continuous instruments have important limitations: they either require restrictive parametric assumptions for identification, or else rely on modelling both the outcome and the treatment process well (and require modelling effect modification by all adjustment covariates). In this work we develop the first semiparametric doubly robust estimators of the local instrumental variable effect curve, i.e. the effect among those who would take treatment for instrument values above some threshold and not below. In addition to being robust to misspecification of either the instrument or treatment or outcome processes, our approach also incorporates information about the instrument mechanism and allows for flexible data‐adaptive estimation of effect modification. We discuss asymptotic properties under weak conditions and use the methods to study infant mortality effects of neonatal intensive care units with high versus low technical capacity, using travel time as an instrument.