Miracle-dz/UniFAR-FTUs huggingface dataset:
Efficient uniform designs for multi-wave computer experiments

In this paper we tackle the problem of generating uniform designs in very small subregions of computer model input space that have been identified in previous experiments as worthy of further study. The method is capable of producing uniform designs in subregions of computer model input space defined by a membership function that consists of a continuous function passing a threshold test, and does so far more efficiently than current methods when these subregions are small. Our application is designing for regions of input space that are not ruled out by history matching, a statistical methodology applied in numerous diverse scientific applications whereby model runs are used to cut out regions of input space that are incompatible with real world observations. History matching defines a membership function for a region of input space that is not ruled out yet by observations in the form of a distance metric called implausibility. We use this distance metric to drive a new type of Evolutionary Monte Carlo algorithm with a uniform distribution on the not ruled out yet region as its target distribution. The algorithm can locate and generate uniform points within extremely small subspaces of the computer model input space with complex and even disconnected topologies. We illustrate the performance of the technique in comparison to current methods with a number of idealised examples. We then apply our algorithm to generating an optimal design for the not ruled out yet region of a galaxy simulation model called GALFORM following 4 previous waves of history matching where the target region is 0.001% the volume of the input space.
A characterization of strong orthogonal arrays of strength three

In an early paper, He and Tang [Biometrika 100 (2013) 254-260] introduced and studied a new class of designs, strong orthogonal arrays, for computer experiments, and characterized such arrays through generalized orthogonal arrays. The current paper presents a simple characterization for strong orthogonal arrays of strength three. Besides being simple, this new characterization through a notion of semi-embeddability is more direct and penetrating in terms of revealing the structure of strong orthogonal arrays. Some other results on strong orthogonal arrays of strength three are also obtained along the way, and in particular, two $\operatorname {SOA}(54,5,27,3)$'s are constructed.
Some theory for ordinal embedding

Motivated by recent work on ordinal embedding (Kleindessner and von Luxburg, 2014), we derive large sample consistency results and rates of convergence for the problem of embedding points based on triple or quadruple distance comparisons. We also consider a variant of this problem where only local comparisons are provided. Finally, inspired by (Jamieson and Nowak, 2011), we bound the number of such comparisons needed to achieve consistency.
On Single Variable Transformation Approach to Markov Chain Monte Carlo

Random Walk Metropolis Hastings (RWMH) algorithm, is quite inefficient in high dimensions because of its abysmally slow acceptance rate. The slow acceptance rate results from the fact that RWMH separately updates each coordinate of the chain at every step. Dutta and Bhattacharya (2013) proposed a new technique called Transformation based Markov Chain Monte Carlo (TMCMC) aimed at overcoming these problems. This method updates all co-ordinates at a time- ensuring stable acceptance in all dimensions. We have shown here that geometric ergodicity is achieved for sub-exponential targets for two versions of TMCMC- the additive and the additive-multiplicative hybrid TMCMC schemes. Also, we obtain the optimal scaling by maximizing the diffusion speed of the limiting time-scaled diffusion process for TMCMC. We show that the optimal acceptance rate is 0.439 for TMCMC which is almost twice as large as RWMH (0.234). We observe that convergence to stationarity for TMCMC is faster than RWMH but the mixing property in RWMH is relatively better. However TMCMC is more robust with respect to scaling and dimensionality. This is attested by simulation runs on Gaussian and nearest neighbor models.
Fast Computation of Tukey Trimmed Regions and Median in Dimension p > 2

ABSTRACT Given data in , a Tukey κ-trimmed region is the set of all points that have at least Tukey depth κ w.r. t. the data. As they are visual, affine equivariant and robust, Tukey regions are useful tools in nonparametric multivariate analysis. While these regions are easily defined and interpreted, their practical use in applications has been impeded so far by the lack of efficient computational procedures in dimension p > 2. We construct two novel algorithms to compute a Tukey κ-trimmed region, a naïve one and a more sophisticated one that is much faster than known algorithms. Further, a strict bound on the number of facets of a Tukey region is derived. In a large simulation study the novel fast algorithm is compared with the naïve one, which is slower and by construction exact, yielding in every case the same correct results. Finally, the approach is extended to an algorithm that calculates the innermost Tukey region and its barycenter, the Tukey median. Supplementary materials for this article are available online.
Minimax theory of estimation of linear functionals of the deconvolution density with or without sparsity

The present paper considers a problem of estimating a linear functional $\Phi=\int_{-\infty}^\infty \varphi(x) f(x)dx$ of an unknown deconvolution density $f$ on the basis of i.i. d. observations $Y_i = \theta_i + \xi_i$ where $\xi_i$ has a known pdf $g$ and $f$ is the pdf of $\theta_i$. Although various aspects and particular cases of this problem have been treated by a number of authors, there are still many gaps. In particular, there are no minimax lower bounds for an estimator of $\Phi$ for an arbitrary function $\varphi$. The general upper risk bounds cover only the case when the Fourier transform of $\varphi$ exists. Moreover, no theory exists for estimating $\Phi$ when vector of observations is sparse. In addition, until now, the related problem of estimation of functionals $\Phi_n = n^{-1} \sum_{i=1}^n \varphi(\theta_i)$ in indirect observations have been treated as a separate problem with no connection to estimation of $\Phi$. The objective of the present paper is to fill in the gaps and develop the general minimax theory of estimation of $\Phi$ and $\Phi_n$. We offer a general approach to estimation of $\Phi$ (and $\Phi_n$) and provide the upper and the minimax lower risk bounds in the case when function $\varphi$ is square integrable. Furthermore, we extend the theory to the case when Fourier transform of $\varphi$ does not exist and $\Phi$ can be presented as a linear functional of the Fourier transform of $f$ and its derivatives. Finally, we generalize our results to handle the situation when vector $\theta$ is sparse. As a direct application of the proposed theory, we obtain multiple new results and automatically recover existing ones for a variety of problems such as estimation of the $(2M+1)$-th absolute moment or a generalized moment of the deconvolution density, estimation of the mixing cdf or estimation of the mixing pdf with classical and Berkson errors.
Bayesian computation: a perspective on the current state, and sampling backwards and forwards

The past decades have seen enormous im-provements in computational inference based on sta-tistical models, with continual enhancement in a wide range of computational tools, in competition. In Bayesian inference, first and foremost, MCMC techniques con-tinue to evolve, moving from random walk proposals to Langevin drift, to Hamiltonian Monte Carlo, and so on, with both theoretical and algorithmic inputs opening wider access to practitioners. However, this impressive evolution in capacity is confronted by an even steeper increase in the complexity of the models and datasets to be addressed. The difficulties of modelling and then handling ever more complex datasets most likely call for a new type of tool for computational inference that dramatically reduce the dimension and size of the raw data while capturing its essential aspects. Approximate models and algorithms may thus be at the core of the next computational revolution.
Weighted M-estimators for multivariate clustered data: theory and simulation results

We study weighted M-estimators for $\mathbb{R}^d$-valued clustered data and give sufficient conditions for their consistency. Their asymptotic normality is established with estimation of the asymptotic covariance matrix. We address the robustness of these estimators in terms of their breakdown point. Comparison with the unweighted case is performed with some numerical studies. They highlight that optimal weights maximizing the relative efficiency have a bad impact on the breakdown point.
Testing independence in high dimensions with sums of rank correlations

We treat the problem of testing independence between m continuous variables when m can be larger than the available sample size n. We consider three types of test statistics that are constructed as sums or sums of squares of pairwise rank correlations. In the asymptotic regime where both m and n tend to infinity, a martingale central limit theorem is applied to show that the null distributions of these statistics converge to Gaussian limits, which are valid with no specific distributional or moment assumptions on the data. Using the framework of U-statistics, our result covers a variety of rank correlations including Kendall's tau and a dominating term of Spearman's rank correlation coefficient (rho), but also degenerate U-statistics such as Hoeffding's $D$, or the $\tau^*$ of Bergsma and Dassios (2014). As in the classical theory for U-statistics, the test statistics need to be scaled differently when the rank correlations used to construct them are degenerate U-statistics. The power of the considered tests is explored in rate-optimality theory under Gaussian equicorrelation alternatives as well as in numerical experiments for specific cases of more general alternatives.
Refinement revisited with connections to Bayes error, conditional entropy and calibrated classifiers

The concept of refinement from probability elicitation is considered for proper scoring rules. Taking directions from the axioms of probability, refinement is further clarified using a Hilbert space interpretation and reformulated into the underlying data distribution setting where connections to maximal marginal diversity and conditional entropy are considered and used to derive measures that provide arbitrarily tight bounds on the Bayes error. Refinement is also reformulated into the classifier output setting and its connections to calibrated classifiers and proper margin losses are established.
Optimal variable selection in multi-group sparse discriminant analysis ∗

This article considers the problem of multi-group classification in the setting where the number of variables p is larger than the number of observations n. Several methods have been proposed in the literature that address this problem, however their variable selection performance is either unknown or suboptimal to the results known in the two-group case. In this work we provide sharp conditions for the consistent recovery of relevant variables in the multi-group case using the discriminant analysis proposal of Gaynanova et al. (7). We achieve the rates of convergence that attain the optimal scaling of the sample size n, number of variables p and the sparsity level s. These rates are significantly faster than the best known results in the multi-group case. Moreover, they coincide with the minimax optimal rates for the two-group case. We validate our theoretical results with numerical analysis.
Graph-based multivariate conditional autoregressive models

ABSTRACT The conditional autoregressive model is a routinely used statistical model for areal data that arise from, for instances, epidemiological, socio-economic or ecological studies. Various multivariate conditional autoregressive models have also been extensively studied in the literature and it has been shown that extending from the univariate case to the multivariate case is not trivial. The difficulties lie in many aspects, including validity, interpretability, flexibility and computational feasibility of the model. In this paper, we approach the multivariate modelling from an element-based perspective instead of the traditional vector-based perspective. We focus on the joint adjacency structure of elements and discuss graphical structures for both the spatial and non-spatial domains. We assume that the graph for the spatial domain is generally known and fixed while the graph for the non-spatial domain can be unknown and random. We propose a very general specification for the multivariate conditional modelling and then focus on three special cases, which are linked to well-known models in the literature. Bayesian inference for parameter learning and graph learning is provided for the focused cases, and finally, an example with public health data is illustrated.
Bayesian Analysis of ODEs: Solver Optimal Accuracy and Bayes Factors

In most cases in the Bayesian analysis of ODE inverse problems, a numerical solver needs to be used. Therefore, we cannot work with the exact theoretical posterior distribution but only with an approximate posterior derived from the error in the numerical solver. To compare an approximate posterior distribution with the theoretical one, we propose using Bayes factors (BFs), considering both of them as models for the data at hand. From a theoretical point of view, we prove that the theoretical vs. numerical posterior BF tends to 1, in the same order as the numerical solver used. In practice, we illustrate the fact that for higher order solvers (e.g., Runge--Kutta) the BF is already nearly 1 for step sizes that would take far less computational effort. Considerable CPU time may be saved by using coarser solvers that nevertheless produce practically error-free posteriors. Two examples are presented where nearly 90% CPU time is saved, with all inference results being identical to those obtained using a solver.
BET: Bayesian Ensemble Trees for Clustering and Prediction in Heterogeneous Data

We propose a novel "tree-averaging" model that utilizes the ensemble of classification and regression trees (CART). Each constituent tree is estimated with a subset of similar data. We treat this grouping of subsets as Bayesian ensemble trees (BET) and model them as an infinite mixture Dirichlet process. We show that BET adapts to data heterogeneity and accurately estimates each component. Compared with the bootstrap-aggregating approach, BET shows improved prediction performance with fewer trees. We develop an efficient estimating procedure with improved sampling strategies in both CART and mixture models. We demonstrate these advantages of BET with simulations, classification of breast cancer and regression of lung function measurement of cystic fibrosis patients. Keywords: Bayesian CART; Dirichlet Process; Ensemble Approach; Heterogeneity; Mixture of Trees.
The subset argument and consistency of MLE in GLMM: Answer to an open problem and beyond

We give answer to an open problem regarding consistency of the maximum likelihood estimators (MLEs) in generalized linear mixed models (GLMMs) involving crossed random effects. The solution to the open problem introduces an interesting, nonstandard approach to proving consistency of the MLEs in cases of dependent observations. Using the new technique, we extend the results to MLEs under a general GLMM. An example is used to further illustrate the technique.
Parametric density-based optimization of partition in cluster analysis, with applications

We developed an optimal in the natural sense algorithm of partition in cluster analysis based on the densities of observations in the different hypotheses. These densities may be characterized, for instance, as the multivariate so-called "quasi-Gaussian distribution". We describe also the possible applications in technical diagnosis, demography and philology.
Additive inverse regression models with convolution-type operators

aIn a recent paper Birke and Bissantz (2008) considered the problem of nonparametric estimation in inverse regression models with convolution-type operators. For multivariate predictors nonparametric methods suer from the curse of dimensionality and we consider inverse regression models with the additional qualitative assumption of additivity. In these models several additive estimators are studied. In particular, we investigate estimators under the random design assumption which are applicable when observations are not available on a grid. Finally, we compare this estimator with the marginal integration and the non-additive estimator by means of a simulation study. It is demonstrated that the new method yields a substantial improvement of the currently available procedures.
Sparse latent factor models with interactions: Analysis of gene expression data

Sparse latent multi-factor models have been used in many exploratory and predictive problems with high-dimensional multivariate observations. Because of concerns with identifiability, the latent factors are almost always assumed to be linearly related to measured feature variables. Here we explore the analysis of multi-factor models with different structures of interactions between latent factors, including multiplicative effects as well as a more general framework for nonlinear interactions introduced via the Gaussian Process. We utilize sparsity priors to test whether the factors and interaction terms have significant effect. The performance of the models is evaluated through simulated and real data applications in genomics. Variation in the number of copies of regions of the genome is a well-known and important feature of most cancers. We examine interactions between factors directly associated with different chromosomal regions detected with copy number alteration in breast cancer data. In this context, significant interaction effects for specific genes suggest synergies between duplications and deletions in different regions of the chromosome.
Perturbed Gibbs Samplers for Synthetic Data Release

We propose a categorical data synthesizer with a quantifiable disclosure risk. Our algorithm, named Perturbed Gibbs Sampler, can handle high-dimensional categorical data that are often intractable to represent as contingency tables. The algorithm extends a multiple imputation strategy for fully synthetic data by utilizing feature hashing and non-parametric distribution approximations. California Patient Discharge data are used to demonstrate statistical properties of the proposed synthesizing methodology. Marginal and conditional distributions, as well as the coefficients of regression models built on the synthesized data are compared to those obtained from the original data. Intruder scenarios are simulated to evaluate disclosure risks of the synthesized data from multiple angles. Limitations and extensions of the proposed algorithm are also discussed.
AIC, Cp and estimators of loss for elliptically symmetric distributions

In this article, we develop a modern perspective on Akaike's information criterion and Mallows's Cp for model selection, and propose generalisations to spherically and elliptically symmetric distributions. Despite the differences in their respective motivation, Cp and Akaike's information criterion are equivalent in the special case of Gaussian linear regression. In this case, they are also equivalent to a third criterion, an unbiased estimator of the quadratic prediction loss, derived from loss estimation theory. We then show that the form of the unbiased estimator of the quadratic prediction loss under a Gaussian assumption still holds under a more general distributional assumption, the family of spherically symmetric distributions. One of the features of our results is that our criterion does not rely on the specificity of the distribution, but only on its spherical symmetry. The same kind of criterion can be derived for a family of elliptically contoured distribution, which allows correlations, when considering the invariant loss. More specifically, the unbiasedness property is relative to a distribution associated to the original density.
Estimation of the shift parameter in regression models with unknown distribution of the observations

This paper is devoted to the estimation of the shift parameter in a semiparametric regression model when the distribution of the observation times is unknown. Hence, we propose to use a stochastic algorithm which takes into account the estimation of the distribution of the observation times. We establish the almost sure convergence of our estimator and the asymptotic normality. The main result of the paper is that, with little assumptions on the regularity of the regression function, the asymptotic variance obtained is the same as when the distribution is known. In that sense, we improve the recent work of Bercu and Fraysse.
Sparsistency and agnostic inference in sparse PCA

The presence of a sparse "truth" has been a constant assumption in the theoretical analysis of sparse PCA and is often implicit in its methodological development. This naturally raises questions about the properties of sparse PCA methods and how they depend on the assumption of sparsity. Under what conditions can the relevant variables be selected consistently if the truth is assumed to be sparse? What can be said about the results of sparse PCA without assuming a sparse and unique truth? We answer these questions by investigating the properties of the recently proposed Fantope projection and selection (FPS) method in the high-dimensional setting. Our results provide general sufficient conditions for sparsistency of the FPS estimator. These conditions are weak and can hold in situations where other estimators are known to fail. On the other hand, without assuming sparsity or identifiability, we show that FPS provides a sparse, linear dimension-reducing transformation that is close to the best possible in terms of maximizing the predictive covariance.
Finding Regression Outliers With FastRCS

The Residual Congruent Subset (RCS) is a new method for finding outliers in the linear regression setting. Like many other outlier detection procedures, RCS searches for a subset which minimizes a criterion. The difference is that the new criterion was designed to be insensitive to the outliers. RCS is supported by FastRCS, a fast regression and affine equivariant algorithm which we also detail. Both an extensive simulation study and two real data applications show that FastRCS performs better than its competitors.
A Rice method proof of the Null-Space Property over the Grassmannian

The Null-Space Property (NSP) is a necessary and sufficient condition for the recovery of the largest coefficients of solutions to an under-determined system of linear equations. Interestingly, this property governs also the success and the failure of recent developments in high-dimensional statistics, signal processing, error-correcting codes and the theory of polytopes. Although this property is the keystone of $\ell\_{1}$-minimization techniques, it is an open problem to derive a closed form for the phase transition on NSP. In this article, we provide the first proof of NSP using random processes theory and the Rice method. As a matter of fact, our analysis gives non-asymptotic bounds for NSP with respect to unitarily invariant distributions. Furthermore, we derive a simple sufficient condition for NSP.