Miracle-dz/UniFAR-FTUs huggingface dataset:
Improved bounds for Square-Root Lasso and Square-Root Slope

Extending the results of Bellec, Lecue and Tsybakov [1] to the setting of sparse highdimensional linear regression with unknown variance, we show that two estimators, the Square-Root Lasso and the Square-Root Slope can achieve the optimal minimax prediction rate, which is (s/n) log (p/s), up to some constant, under some mild conditions on the design matrix. Here, n is the sample size, p is the dimension and s is the sparsity parameter. We also prove optimality for the estimation error in the lq-norm, with q in [1, 2] for the Square-Root Lasso, and in the l2 and sorted l1 norms for the Square-Root Slope. Both estimators are adaptive to the unknown variance of the noise. The Square-Root Slope is also adaptive to the sparsity s of the true parameter. Next, we prove that any estimator depending on s which attains the minimax rate admits an adaptive to s version still attaining the same rate. We apply this result to the Square-root Lasso. Moreover, for both estimators, we obtain valid rates for a wide range of confidence levels, and improved concentration properties as in [1] where the case of known variance is treated. Our results are non-asymptotic. ;Classification-JEL: Primary 62G08; secondary 62C20, 62G05.
Satellite conjunction analysis and the false confidence theorem

Satellite conjunction analysis is the assessment of collision risk during a close encounter between a satellite and another object in orbit. A counterintuitive phenomenon has emerged in the conjunction analysis literature, namely, probability dilution, in which lower quality data paradoxically appear to reduce the risk of collision. We show that probability dilution is a symptom of a fundamental deficiency in probabilistic representations of statistical inference, in which there are propositions that will consistently be assigned a high degree of belief, regardless of whether or not they are true. We call this deficiency false confidence. In satellite conjunction analysis, it results in a severe and persistent underestimate of collision risk exposure. We introduce the Martin–Liu validity criterion as a benchmark by which to identify statistical methods that are free from false confidence. Such inferences will necessarily be non-probabilistic. In satellite conjunction analysis, we show that uncertainty ellipsoids satisfy the validity criterion. Performing collision avoidance manoeuvres based on ellipsoid overlap will ensure that collision risk is capped at the user-specified level. Furthermore, this investigation into satellite conjunction analysis provides a template for recognizing and resolving false confidence issues as they occur in other problems of statistical inference.
Kernel‐based tests for joint independence

We investigate the problem of testing whether d possibly multivariate random variables, which may or may not be continuous, are jointly (or mutually) independent. Our method builds on ideas of the two‐variable Hilbert–Schmidt independence criterion but allows for an arbitrary number of variables. We embed the joint distribution and the product of the marginals in a reproducing kernel Hilbert space and define the d‐variable Hilbert–Schmidt independence criterion dHSIC as the squared distance between the embeddings. In the population case, the value of dHSIC is 0 if and only if the d variables are jointly independent, as long as the kernel is characteristic. On the basis of an empirical estimate of dHSIC, we investigate three non‐parametric hypothesis tests: a permutation test, a bootstrap analogue and a procedure based on a gamma approximation. We apply non‐parametric independence testing to a problem in causal discovery and illustrate the new methods on simulated and real data sets.
Pseudo minimum phi-divergence estimator for multinomial logistic regression with complex sample design

This article develops the theoretical framework needed to study the multinomial logistic regression model for complex sample design with pseudo minimum phi-divergence estimators. Through a numerical example and simulation study new estimators are proposed for the parameter of the logistic regression model with overdispersed multinomial distributions for the response variables, the pseudo minimum Cressie-Read divergence estimators, as well as new estimators for the intra-cluster correlation coefficient. The results show that the Binder's method for the intra-cluster correlation coefficient exhibits an excellent performance when the pseudo minimum Cressie-Read divergence estimator, with lambda = 2/3 , is plugged.
Likelihood-Free Inference by Ratio Estimation

We consider the problem of parametric statistical inference when likelihood computations are prohibitively expensive but sampling from the model is possible. Several so-called likelihood-free methods have been developed to perform inference in the absence of a likelihood function. The popular synthetic likelihood approach infers the parameters by modelling summary statistics of the data by a Gaussian probability distribution. In another popular approach called approximate Bayesian computation, the inference is performed by identifying parameter values for which the summary statistics of the simulated data are close to those of the observed data. Synthetic likelihood is easier to use as no measure of `closeness' is required but the Gaussianity assumption is often limiting. Moreover, both approaches require judiciously chosen summary statistics. We here present an alternative inference approach that is as easy to use as synthetic likelihood but not as restricted in its assumptions, and that, in a natural way, enables automatic selection of relevant summary statistic from a large set of candidates. The basic idea is to frame the problem of estimating the posterior as a problem of estimating the ratio between the data generating distribution and the marginal distribution. This problem can be solved by logistic regression, and including regularising penalty terms enables automatic selection of the summary statistics relevant to the inference task. We illustrate the general theory on canonical examples and employ it to perform inference for challenging stochastic nonlinear dynamical systems and high-dimensional summary statistics.
A Bayesian Approach to Modelling Fine-Scale Spatial Dynamics of Non-State Terrorism: World Study, 2002-2013

To this day, terrorism persists as a worldwide threat, as exemplified by the ongoing lethal attacks perpetrated by ISIS in Iraq, Syria, Al Qaeda in Yemen, and Boko Haram in Nigeria. In response, states deploy various counterterrorism policies, the costs of which could be reduced through efficient preventive measures. Statistical models able to account for complex spatio-temporal dependencies have not yet been applied, despite their potential for providing guidance to explain and prevent terrorism. In an effort to address this shortcoming, we employ hierarchical models in a Bayesian context, where the spatial random field is represented by a stochastic partial differential equation. Our results confirm the contagious nature of the lethality of terrorism and the number of lethal terrorist attacks in both space and time. Moreover, the frequency of lethal attacks tends to be higher in richer areas, close to large cities, and within democratic countries. In contrast, attacks are more likely to be lethal far away from large cities, at higher altitudes, in poorer areas, and in locations with higher ethnic diversity. We argue that, on a local scale, the lethality of terrorism and the frequency of lethal attacks are driven by antagonistic mechanisms.
Transfer Learning with Label Noise

Transfer learning aims to improve learning in target domain by borrowing knowledge from a related but different source domain. To reduce the distribution shift between source and target domains, recent methods have focused on exploring invariant representations that have similar distributions across domains. However, when learning this invariant knowledge, existing methods assume that the labels in source domain are uncontaminated, while in reality, we often have access to source data with noisy labels. In this paper, we first show how label noise adversely affect the learning of invariant representations and the correcting of label shift in various transfer learning scenarios. To reduce the adverse effects, we propose a novel Denoising Conditional Invariant Component (DCIC) framework, which provably ensures (1) extracting invariant representations given examples with noisy labels in source domain and unlabeled examples in target domain; (2) estimating the label distribution in target domain with no bias. Experimental results on both synthetic and real-world data verify the effectiveness of the proposed method.
Pseudo‐R2 statistics under complex sampling

Model summaries based on the ratio of fitted and null likelihoods have been proposed for generalised linear models, reducing to the familiar R2 coefficient of determination in the Gaussian model with identity link. In this note I show how to define the Cox–Snell and Nagelkerke summaries under arbitrary probability sampling designs, giving a design‐consistent estimator of the population model summary. It is also shown that for logistic regression models under case–control sampling the usual Cox–Snell and Nagelkerke R2 are not design‐consistent, but are systematically larger than would be obtained with a cross‐sectional or cohort sample from the same population, even in settings where the weighted and unweighted logistic regression estimators are similar or identical. Implementation of the new estimators is straightforward and code is provided in R.
Improving Grid Based Bayesian Methods

In some cases, computational benefit can be gained by exploring the hyper parameter space using a deterministic set of grid points instead of a Markov chain. We view this as a numerical integration problem and make three unique contributions. First, we explore the space using low discrepancy point sets instead of a grid. This allows for accurate estimation of marginals of any shape at a much lower computational cost than a grid based approach and thus makes it possible to extend the computational benefit to a hyper parameter space with higher dimensionality (10 or more). Second, we propose a new, quick and easy method to estimate the marginal using a least squares polynomial and prove the conditions under which this polynomial will converge to the true marginal. Our results are valid for a wide range of point sets including grids, random points and low discrepancy points. Third, we show that further accuracy and efficiency can be gained by taking into consideration the functional decomposition of the integrand and illustrate how this can be done using anchored f-ANOVA on weighted spaces.
Compressed and Penalized Linear Regression

Abstract Modern applications require methods that are computationally feasible on large datasets while retaining good statistical properties. Recent work has focused on developing fast and randomized approximations for solving least squares problems when the data are too large to fit into memory easily or when computations are at a premium. Many of these techniques rely on data-driven subsampling or random compression. In this article, we provide new approximate algorithms for solving penalized least squares problems which have improved statistical performance relative to existing methods. We provide the first efficient methods for tuning parameter selection, compare our methods with current approaches via simulation and application, and provide theoretical intuition which makes explicit the impact of approximation on statistical efficiency and demonstrates the necessity of careful parameter tuning. Supplementary materials for this article are available online.
Network representation using graph root distributions

Exchangeable random graphs serve as an important probabilistic framework for the statistical analysis of network data. At the core of this framework is the parameterization of graph sampling distributions, where existing methods suffer from non-trivial identifiability issues. In this work we develop a new parameterization for general exchangeable random graphs, where the nodes are independent random vectors in a linear space equipped with an indefinite inner product, and the edge probability between two nodes equals the inner product of the corresponding node vectors. Therefore, the distribution of exchangeable random graphs can be represented by a node sampling distribution on this linear space, which we call the "graph root distribution". We study existence and uniqueness of such representations, the topological relationship between the graph root distribution and the exchangeable random graph sampling distribution, and the statistical estimation of graph root distributions.
Spectral clustering in the dynamic stochastic block model

In the present paper, we studied a Dynamic Stochastic Block Model (DSBM) under the assumptions that the connection probabilities, as functions of time, are smooth and that at most $s$ nodes can switch their class memberships between two consecutive time points. We estimate the edge probability tensor by a kernel-type procedure and extract the group memberships of the nodes by spectral clustering. The procedure is computationally viable, adaptive to the unknown smoothness of the functional connection probabilities, to the rate $s$ of membership switching and to the unknown number of clusters. In addition, it is accompanied by non-asymptotic guarantees for the precision of estimation and clustering.
Exchangeable Trait Allocations

Trait allocations are a class of combinatorial structures in which data may belong to multiple groups and may have different levels of belonging in each group. Often the data are also exchangeable, i.e., their joint distribution is invariant to reordering. In clustering---a special case of trait allocation---exchangeability implies the existence of both a de Finetti representation and an exchangeable partition probability function (EPPF), distributional representations useful for computational and theoretical purposes. In this work, we develop the analogous de Finetti representation and exchangeable trait probability function (ETPF) for trait allocations, along with a characterization of all trait allocations with an ETPF. Unlike previous feature allocation characterizations, our proofs fully capture single-occurrence "dust" groups. We further introduce a novel constrained version of the ETPF that we use to establish an intuitive connection between the probability functions for clustering, feature allocations, and trait allocations. As an application of our general theory, we characterize the distribution of all edge-exchangeable graphs, a class of recently-developed models that captures realistic sparse graph sequences.
Comment on Article by Ferreira and Gamerman

To put this paper in context, some background seems worthwhile. Selection bias in one form or another has always been an issue in statistical science, and it has been studied since at least the time when Horvitz and Thompson proposed their simple but ingenious approach to unbiasing estimates of finite population averages when sample items are preferentially selected (Horvitz and Thompson, 1952). Survey statisticians have long since recognized the adverse effect of such bias and the need to adjust for it when computing their estimates. Biostatisticians have also been concerned with this issue in the form of response biased sampling in estimating the relationship between a response Y and a covariate vector Z when instead of sites human subjects are the units (Scott and Wild, 2011). There, inter-subject dependence is ignored due to its complexity and the work of Liang and Zeger (1986) which allows that simplification to be made. The responses Y are assumed to be observed (although that assumption can be relaxed by modeling it) and subject selection is biased by these responses. In contrast, the present paper follows Diggle et al. (2010) and assumes instead that the role of Y is implicit and seen through the point process model that “knows” Y = Y (x), or rather the latent process that generates it, through the intensity function exp {α+ βS(x)}, quite a strong assumption. The just cited work in biostatistics would be of potential relevance in spatial regression, an important topic in environmental epidemiology, but the effect of preferential sampling in that domain, especially on the effect on optimal design as seen in this paper, has not been studied as far as we know.
GAUSSIAN PROCESS MODELS FOR MORTALITY RATES AND IMPROVEMENT FACTORS

Abstract We develop a Gaussian process (GP) framework for modeling mortality rates and mortality improvement factors. GP regression is a nonparametric, data-driven approach for determining the spatial dependence in mortality rates and jointly smoothing raw rates across dimensions, such as calendar year and age. The GP model quantifies uncertainty associated with smoothed historical experience and generates full stochastic trajectories for out-of-sample forecasts. Our framework is well suited for updating projections when newly available data arrives, and for dealing with “edge” issues where credibility is lower. We present a detailed analysis of GP model performance for US mortality experience based on the CDC (Center for Disease Control) datasets. We investigate the interaction between mean and residual modeling, Bayesian and non-Bayesian GP methodologies, accuracy of in-sample and out-of-sample forecasting, and stability of model parameters. We also document the general decline, along with strong age-dependency, in mortality improvement factors over the past few years, contrasting our findings with the Society of Actuaries (SOA) MP-2014 and -2015 models that do not fully reflect these recent trends.
Improved multivariate portmanteau test

A new portmanteau diagnostic test for vector autoregressive moving average (VARMA) models that is based on the determinant of the standardized multivariate residual autocorrelations is derived. The new test statistic may be considered an extension of the univariate portmanteau test statistic suggested by Peňa and Rodríguez (2002) . The asymptotic distribution of the test statistic is derived as well as a chi‐square approximation. However, the Monte–Carlo test is recommended unless the series is very long. Extensive simulation experiments demonstrate the usefulness of this test as well as its improved power performance compared to widely used previous multivariate portmanteau diagnostic check. Two illustrative applications are given.
On Estimation of Conditional Modes Using Multiple Quantile Regressions

We propose an estimation method for the conditional mode when the conditioning variable is high-dimensional. In the proposed method, we first estimate the conditional density by solving quantile regressions multiple times. We then estimate the conditional mode by finding the maximum of the estimated conditional density. The proposed method has two advantages in that it is computationally stable because it has no initial parameter dependencies, and it is statistically efficient with a fast convergence rate. Synthetic and real-world data experiments demonstrate the better performance of the proposed method compared to other existing ones.
Optimizing The Integrator Step Size for Hamiltonian Monte Carlo

Hamiltonian Monte Carlo can provide powerful inference in complex statistical problems, but ultimately its performance is sensitive to various tuning parameters. In this paper we use the underlying geometry of Hamiltonian Monte Carlo to construct a universal optimization criteria for tuning the step size of the symplectic integrator crucial to any implementation of the algorithm as well as diagnostics to monitor for any signs of invalidity. An immediate outcome of this result is that the suggested target average acceptance probability of 0.651 can be relaxed to $0.6 \lesssim a \lesssim 0.9$ with larger values more robust in practice.
Improving the identification of antigenic sites in the H1N1 influenza virus through accounting for the experimental structure in a sparse hierarchical Bayesian model

Understanding how genetic changes allow emerging virus strains to escape the protection afforded by vaccination is vital for the maintenance of effective vaccines. We use structural and phylogenetic differences between pairs of virus strains to identify important antigenic sites on the surface of the influenza A(H1N1) virus through the prediction of haemagglutination inhibition (HI) titre: pairwise measures of the antigenic similarity of virus strains. We propose a sparse hierarchical Bayesian model that can deal with the pairwise structure and inherent experimental variability in the H1N1 data through the introduction of latent variables. The latent variables represent the underlying HI titre measurement of any given pair of virus strains and help to account for the fact that, for any HI titre measurement between the same pair of virus strains, the difference in the viral sequence remains the same. Through accurately representing the structure of the H1N1 data, the model can select virus sites which are antigenic, while its latent structure achieves the computational efficiency that is required to deal with large virus sequence data, as typically available for the influenza virus. In addition to the latent variable model, we also propose a new method, the block‐integrated widely applicable information criterion biWAIC, for selecting between competing models. We show how this enables us to select the random effects effectively when used with the model proposed and we apply both methods to an A(H1N1) data set.
Dynamic degree-corrected blockmodels for social networks: A nonparametric approach

A nonparametric approach to the modelling of social networks using degree-corrected stochastic blockmodels is proposed. The model for static network consists of a stochastic blockmodel using a probit regression formulation, and popularity parameters are incorporated to account for degree heterogeneity. We specify a Dirichlet process prior to detect community structure as well as to induce clustering in the popularity parameters. This approach is flexible yet parsimonious as it allows the appropriate number of communities and popularity clusters to be determined automatically by the data. We further discuss and implement extensions of the static model to dynamic networks. In a Bayesian framework, we perform posterior inference through MCMC algorithms. The models are illustrated using several real-world benchmark social networks.
Methods for Bayesian Variable Selection with Binary Response Data using the EM Algorithm

High-dimensional Bayesian variable selection problems are often solved using computationally expensive Markov Chain Montle Carlo (MCMC) techniques. Recently, a Bayesian variable selection technique was developed for continuous data using the EM algorithm called EMVS. We extend the EMVS method to binary data by proposing both a logistic and probit extension. To preserve the computational speed of EMVS we also implemented the Stochastic Dual Coordinate Descent (SDCA) algorithm. Further, we conduct two extensive simulation studies to show the computational speed of both methods. These simulation studies reveal the power of both methods to quickly identify the correct sparse model. When these EMVS methods are compared to Stochastic Search Variable Selection (SSVS), the EMVS methods surpass SSVS both in terms of computational speed and correctly identifying significant variables. Finally, we illustrate the effectiveness of both methods on two well-known gene expression datasets. Our results mirror the results of previous examinations of these datasets with far less computational cost.