Miracle-dz/UniFAR-FTUs huggingface dataset:
Nonparametric regression with adaptive truncation via a convex hierarchical penalty

&NA; We consider the problem of nonparametric regression with a potentially large number of covariates. We propose a convex, penalized estimation framework that is particularly well suited to high‐dimensional sparse additive models and combines the appealing features of finite basis representation and smoothing penalties. In the case of additive models, a finite basis representation provides a parsimonious representation for fitted functions but is not adaptive when component functions possess different levels of complexity. In contrast, a smoothing spline‐type penalty on the component functions is adaptive but does not provide a parsimonious representation. Our proposal simultaneously achieves parsimony and adaptivity in a computationally efficient way. We demonstrate these properties through empirical studies and show that our estimator converges at the minimax rate for functions within a hierarchical class. We further establish minimax rates for a large class of sparse additive models. We also develop an efficient algorithm that scales similarly to the lasso with the number of covariates and sample size.
The inflation technique solves completely the classical inference problem

The causal inference problem consists in determining whether a probability distribution over a set of observed variables is compatible with a given causal structure. In [arXiv:1609.00672], one of us introduced a hierarchy of necessary linear programming constraints which all the observed distributions compatible with the considered causal structure must satisfy. In this work, we prove that the inflation hierarchy is complete, i.e., any distribution of the observed variables which does not admit a realization within the considered causal structure will fail one of the inflation tests. More quantitatively, we show that any distribution of measurable events satisfying the $n^{th}$ inflation test is $O\left(\frac{1}{\sqrt{n}}\right)$-close in Euclidean norm to a distribution realizable within the given causal structure. In addition, we show that the corresponding $n^{th}$-order relaxation of the dual problem consisting in maximizing a $k^{th}$ degree polynomial on the observed variables is $O\left(\frac{k^2}{n}\right)$-close to the optimal solution.
Expectation propagation in the large data limit

Expectation propagation (EP) is a widely successful algorithm for variational inference. EP is an iterative algorithm used to approximate complicated distributions, typically to find a Gaussian approximation of posterior distributions. In many applications of this type, EP performs extremely well. Surprisingly, despite its widespread use, there are very few theoretical guarantees on Gaussian EP, and it is quite poorly understood. To analyse EP, we first introduce a variant of EP: averaged EP, which operates on a smaller parameter space. We then consider averaged EP and EP in the limit of infinite data, where the overall contribution of each likelihood term is small and where posteriors are almost Gaussian. In this limit, we prove that the iterations of both averaged EP and EP are simple: they behave like iterations of Newton's algorithm for finding the mode of a function. We use this limit behaviour to prove that EP is asymptotically exact, and to obtain other insights into the dynamic behaviour of EP, e.g. that it may diverge under poor initialization exactly like Newton's method. EP is a simple algorithm to state, but a difficult one to study. Our results should facilitate further research into the theoretical properties of this important method.
Functional linear regression with points of impact

The paper considers functional linear regression, where scalar responses $Y_1,\ldots,Y_n$ are modeled in dependence of i.i. d. random functions $X_1,\ldots,X_n$. We study a generalization of the classical functional linear regression model. It is assumed that there exists an unknown number of "points of impact," that is, discrete observation times where the corresponding functional values possess significant influences on the response variable. In addition to estimating a functional slope parameter, the problem then is to determine the number and locations of points of impact as well as corresponding regression coefficients. Identifiability of the generalized model is considered in detail. It is shown that points of impact are identifiable if the underlying process generating $X_1,\ldots,X_n$ possesses "specific local variation. " Examples are well-known processes like the Brownian motion, fractional Brownian motion or the Ornstein-Uhlenbeck process. The paper then proposes an easily implementable method for estimating the number and locations of points of impact. It is shown that this number can be estimated consistently. Furthermore, rates of convergence for location estimates, regression coefficients and the slope parameter are derived. Finally, some simulation results as well as a real data application are presented.
Recurrence Analysis of Vegetation Time Series and Phase Transitions in Mediterranean Rangelands

Mediterranean rangelands should be conceived as socio-ecological landscapes (SEL) because of the close interaction and coevolution between socio-economic and natural systems. A significant threat to these Mediterranean rangelands is related to uncontrolled fires that can cause potential damages due to the reduction or even the loss of ecosystems. Our results show that time series of forest and grassland for unburned and burned areas are characterized by both periodic and chaotic components. The fire event caused a clear simplification of vegetation structures as well as of SEL dynamics that is more regular and predictable after the burning and less chaotic. However grassland evolution could be more predictable than forest considering the effect of fire disturbance on successional cycles and stages of the two land-cover types. In particular, we applied recurrence analysis with sliding temporal windows three-year length on the original time series. This analysis indicates that grasslands and forests behaved similarly in correspondence with the burning, although their phase states slowly diverge after fire. Recurrence is useful to study the vegetation recovery as it enables mapping landscape transitions derived from remote sensing. The approach helps stakeholders to draw landscape interventions and improve management strategies to sustain the delivery of ecosystem services.
Monte Carlo approximation certificates for k-means clustering

Efficient algorithms for $k$-means clustering frequently converge to suboptimal partitions, and given a partition, it is difficult to detect $k$-means optimality. In this paper, we develop an a posteriori certifier of approximate optimality for $k$-means clustering. The certifier is a sub-linear Monte Carlo algorithm based on Peng and Wei's semidefinite relaxation of $k$-means. In particular, solving the relaxation for small random samples of the dataset produces a high-confidence lower bound on the $k$-means objective, and being sub-linear, our algorithm is faster than $k$-means++ when the number of data points is large. We illustrate the performance of our algorithm with both numerical experiments and a performance guarantee: If the data points are drawn independently from any mixture of two Gaussians over $\mathbb{R}^m$ with identity covariance, then with probability $1-O(1/m)$, our $\operatorname{poly}(m)$-time algorithm produces a 3-approximation certificate with 99% confidence.
Spearman Rank Correlation Screening for Ultrahigh-Dimensional Censored Data

Herein, we propose a Spearman rank correlation-based screening procedure for ultrahigh-dimensional data with censored response cases. The proposed method is model-free without specifying any regression forms of predictors or response variables and is robust under the unknown monotone transformations of these response variable and predictors. The sure-screening and rank-consistency properties are established under some mild regularity conditions. Simulation studies demonstrate that the new screening method performs well in the presence of a heavy-tailed distribution, strongly dependent predictors or outliers, and offers superior performance over the existing nonparametric screening procedures. In particular, the new screening method still works well when a response variable is observed under a high censoring rate. An illustrative example is provided.
Inferring the parameters of a Markov process from snapshots of the steady state

We seek to infer the parameters of an ergodic Markov process from samples taken independently from the steady state. Our focus is on non-equilibrium processes, where the steady state is not described by the Boltzmann measure, but is generally unknown and hard to compute, which prevents the application of established equilibrium inference methods. We propose a quantity we call propagator likelihood, which takes on the role of the likelihood in equilibrium processes. This propagator likelihood is based on fictitious transitions between those configurations of the system which occur in the samples. The propagator likelihood can be derived by minimising the relative entropy between the empirical distribution and a distribution generated by propagating the empirical distribution forward in time. Maximising the propagator likelihood leads to an efficient reconstruction of the parameters of the underlying model in different systems, both with discrete configurations and with continuous configurations. We apply the method to non-equilibrium models from statistical physics and theoretical biology, including the asymmetric simple exclusion process (ASEP), the kinetic Ising model, and replicator dynamics.
MEDL and MEDLA: Methods for Assessment of Scaling by Medians of Log-Squared Nondecimated Wavelet Coefficients

High-frequency measurements and images acquired from various sources in the real world often possess a degree of self-similarity and inherent regular scaling. When data look like a noise, the scaling exponent may be the only informative feature that summarizes such data. Methods for the assessment of self-similarity by estimating Hurst exponent often involve analysis of rate of decay in a spectrum defined in various multiresolution domains. When this spectrum is calculated using discrete non-decimated wavelet transforms, due to increased autocorrelation in wavelet coefficients, the estimators of $H$ show increased bias compared to the estimators that use traditional orthogonal transforms. At the same time, non-decimated transforms have a number of advantages when employed for calculation of wavelet spectra and estimation of Hurst exponents: the variance of the estimator is smaller, input signals and images could be of arbitrary size, and due to the shift-invariance, the local scaling can be assessed as well. We propose two methods based on robust estimation and resampling that alleviate the effect of increased autocorrelation while maintaining all advantages of non-decimated wavelet transforms. The proposed methods extend the approaches in existing literature where the logarithmic transformation and pairing of wavelet coefficients are used for lowering the bias. In a simulation study we use fractional Brownian motions with a range of theoretical Hurst exponents. For such signals for which "true" $H$ is known, we demonstrate bias reduction and overall reduction of the mean-squared error by the two proposed estimators. For fractional Brownian motions, both proposed methods yield estimators of $H$ that are asymptotically normal and unbiased.
Nonreversible Langevin Samplers: Splitting Schemes, Analysis and Implementation

For a given target density, there exist an infinite number of diffusion processes which are ergodic with respect to this density. As observed in a number of papers, samplers based on nonreversible diffusion processes can significantly outperform their reversible counterparts both in terms of asymptotic variance and rate of convergence to equilibrium. In this paper, we take advantage of this in order to construct efficient sampling algorithms based on the Lie-Trotter decomposition of a nonreversible diffusion process into reversible and nonreversible components. We show that samplers based on this scheme can significantly outperform standard MCMC methods, at the cost of introducing some controlled bias. In particular, we prove that numerical integrators constructed according to this decomposition are geometrically ergodic and characterise fully their asymptotic bias and variance, showing that the sampler inherits the good mixing properties of the underlying nonreversible diffusion. This is illustrated further with a number of numerical examples ranging from highly correlated low dimensional distributions, to logistic regression problems in high dimensions as well as inference for spatial models with many latent variables.
Smoothed Hierarchical Dirichlet Process: A Non-Parametric Approach to Constraint Measures

Time-varying mixture densities occur in many scenarios, for example, the distributions of keywords that appear in publications may evolve from year to year, video frame features associated with multiple targets may evolve in a sequence. Any models that realistically cater to this phenomenon must exhibit two important properties: the underlying mixture densities must have an unknown number of mixtures, and there must be some "smoothness" constraints in place for the adjacent mixture densities. The traditional Hierarchical Dirichlet Process (HDP) may be suited to the first property, but certainly not the second. This is due to how each random measure in the lower hierarchies is sampled independent of each other and hence does not facilitate any temporal correlations. To overcome such shortcomings, we proposed a new Smoothed Hierarchical Dirichlet Process (sHDP). The key novelty of this model is that we place a temporal constraint amongst the nearby discrete measures $\{G_j\}$ in the form of symmetric Kullback-Leibler (KL) Divergence with a fixed bound $B$. Although the constraint we place only involves a single scalar value, it nonetheless allows for flexibility in the corresponding successive measures. Remarkably, it also led us to infer the model within the stick-breaking process where the traditional Beta distribution used in stick-breaking is now replaced by a new constraint calculated from $B$. We present the inference algorithm and elaborate on its solutions. Our experiment using NIPS keywords has shown the desirable effect of the model.
The Impact of Confounder Selection in Propensity Scores for Rare Events Data - with Applications to Birth Defects

Our work was motivated by a recent study on birth defects of infants born to pregnant women exposed to a certain medication for treating chronic diseases. Outcomes such as birth defects are rare events in the general population, which often translate to very small numbers of events in the unexposed group. As drug safety studies in pregnancy are typically observational in nature, we control for confounding in this rare events setting using propensity scores (PS). Using our empirical data, we noticed that the estimated odds ratio for birth defects due to exposure varied drastically depending on the specific approach used. The commonly used approaches with PS are matching, stratification, inverse probability weighting (IPW) and regression adjustment. The extremely rare events setting renders the matching or stratification infeasible. In addition, the PS itself may be formed via different approaches to select confounders from a relatively long list of potential confounders. We carried out simulation experiments to compare different combinations of approaches: IPW or regression adjustment, with 1) including all potential confounders without selection, 2) selection based on univariate association between the candidate variable and the outcome, 3) selection based on change in effects (CIE). The simulation showed that IPW without selection leads to extremely large variances in the estimated odds ratio, which help to explain the empirical data analysis results that we had observed. The simulation also showed that IPW with selection based on univariate association with the outcome is preferred over IPW with CIE. Regression adjustment has small variances of the estimated odds ratio regardless of the selection methods used.
Uncovering Longitudinal Healthcare Utilization from Patient-Level Medical Claims Data

The objective of this study is to introduce methodology for studying longitudinal claims data observed at the patient level, with inference on the heterogeneity of healthcare utilization behaviors within large healthcare systems such as Medicaid. The proposed approach is model-based, allowing for visualization of longitudinal utilization behaviors using simple stochastic graphical networks. The approach is general, providing a framework for the study of other chronic conditions wherever longitudinal healthcare utilization data are available. Our methods are inspired by and applied to patient-level Medicaid claims for asthma-diagnosed children diagnosed observed over a period of five years, with a comparison of two neighboring states, Georgia and North Carolina.
Enhancements of Non‐parametric Generalized Likelihood Ratio Test: Bias Correction and Dimension Reduction

Non‐parametric generalized likelihood ratio test is a popular method of model checking for regressions. However, there are two issues that may be the barriers for its powerfulness: existing bias term and curse of dimensionality. The purpose of this paper is thus twofold: a bias reduction is suggested and a dimension reduction‐based adaptive‐to‐model enhancement is recommended to promote the power performance. The proposed test statistic still possesses the Wilks phenomenon and behaves like a test with only one covariate. Thus, it converges to its limit at a much faster rate and is much more sensitive to alternative models than the classical non‐parametric generalized likelihood ratio test. As a by‐product, we also prove that the bias‐corrected test is more efficient than the one without bias reduction in the sense that its asymptotic variance is smaller. Simulation studies and a real data analysis are conducted to evaluate of proposed tests.
V-Splines and Bayes Estimate

Smoothing splines can be thought of as the posterior mean of a Gaussian process regression in a certain limit. By constructing a reproducing kernel Hilbert space with an appropriate inner product, the Bayesian form of the V-spline is derived when the penalty term is a fixed constant instead of a function. An extension to the usual generalized cross-validation formula is utilized to find the optimal V-spline parameters.
On the Ubiquity of Information Inconsistency for Conjugate Priors

Informally, "Information Inconsistency" is the property that has been observed in many Bayesian hypothesis testing and model selection procedures whereby the Bayesian conclusion does not become definitive when the data seems to become definitive. An example is that, when performing a t-test using standard conjugate priors, the Bayes factor of the alternative hypothesis to the null hypothesis remains bounded as the t statistic grows to infinity. This paper shows that information inconsistency is ubiquitous in Bayesian hypothesis testing under conjugate priors. Yet the title does not fully describe the paper, since we also show that theoretically recommended priors, including scale mixtures of conjugate priors and adaptive priors, are information consistent. Hence the paper is simply a forceful warning that use of conjugate priors in testing and model selection is highly problematical, and should be replaced by the information consistent alternatives.
Information Theory and Statistics: an overview

We give an overview of the role of information theory in statistics, and particularly in biostatistics. We recall the basic quantities in information theory; entropy, cross-entropy, conditional entropy, mutual information and Kullback-Leibler risk. Then we examine the role of information theory in estimation theory, where the log-klikelihood can be identified as being an estimator of a cross-entropy. Then the basic quantities are extended to estimators, leading to criteria for estimator selection, such as Akaike criterion and its extensions. Finally we investigate the use of these concepts in Bayesian theory; the cross-entropy of the predictive distribution can be used for model selection; a cross-validation estimator of this cross-entropy is found to be equivalent to the pseudo-Bayes factor.
Is it Correct to Use MLE Method for GRP Parameter Estimation

Analysis of repair systems usually uses an As Good As New or As Bad As Old repair assumptions. In practice, repair actions do not result in such extreme situations, but rather in a complex transitional one, that is imperfect maintenance, i.e. Generalized Renewal Process. Maximum Likelihood Estimation method is often used for reliability parameter estimation, but is it correct to use it for Generalized Renewal Process
Summary Statistics in Approximate Bayesian Computation

This document is due to appear as a chapter of the forthcoming Handbook of Approximate Bayesian Computation (ABC) edited by S. Sisson, Y. Fan, and M. Beaumont. Since the earliest work on ABC, it has been recognised that using summary statistics is essential to produce useful inference results. This is because ABC suffers from a curse of dimensionality effect, whereby using high dimensional inputs causes large approximation errors in the output. It is therefore crucial to find low dimensional summaries which are informative about the parameter inference or model choice task at hand. This chapter reviews the methods which have been proposed to select such summaries, extending the previous review paper of Blum et al. (2013) with recent developments. Related theoretical results on the ABC curse of dimensionality and sufficiency are also discussed.
Multivariate output analysis for Markov chain Monte Carlo

Markov chain Monte Carlo (MCMC) produces a correlated sample for estimating expectations with respect to a target distribution. A fundamental question is when should sampling stop so that we have good estimates of the desired quantities? The key to answering this question lies in assessing the Monte Carlo error through a multivariate Markov chain central limit theorem (CLT). The multivariate nature of this Monte Carlo error largely has been ignored in the MCMC literature. We present a multivariate framework for terminating simulation in MCMC. We define a multivariate effective sample size, estimating which requires strongly consistent estimators of the covariance matrix in the Markov chain CLT; a property we show for the multivariate batch means estimator. We then provide a lower bound on the number of minimum effective samples required for a desired level of precision. This lower bound depends on the problem only in the dimension of the expectation being estimated, and not on the underlying stochastic process. This result is obtained by drawing a connection between terminating simulation via effective sample size and terminating simulation using a relative standard deviation fixed-volume sequential stopping rule; which we demonstrate is an asymptotically valid procedure. The finite sample properties of the proposed method are demonstrated in a variety of examples.