Miracle-dz/UniFAR-FTUs huggingface dataset:
Heterogeneous change point inference

We propose, a heterogeneous simultaneous multiscale change point estimator called ‘H‐SMUCE’ for the detection of multiple change points of the signal in a heterogeneous Gaussian regression model. A piecewise constant function is estimated by minimizing the number of change points over the acceptance region of a multiscale test which locally adapts to changes in the variance. The multiscale test is a combination of local likelihood ratio tests which are properly calibrated by scale‐dependent critical values to keep a global nominal level α, even for finite samples. We show that H‐SMUCE controls the error of overestimation and underestimation of the number of change points. For this, new deviation bounds for F‐type statistics are derived. Moreover, we obtain confidence sets for the whole signal. All results are non‐asymptotic and uniform over a large class of heterogeneous change point models. H‐SMUCE is fast to compute, achieves the optimal detection rate and estimates the number of change points at almost optimal accuracy for vanishing signals, while still being robust. We compare H‐SMUCE with several state of the art methods in simulations and analyse current recordings of a transmembrane protein in the bacterial outer membrane with pronounced heterogeneity for its states. An R‐package is available on line.
Tensor graphical lasso (TeraLasso)

The paper introduces a multiway tensor generalization of the bigraphical lasso which uses a two‐way sparse Kronecker sum multivariate normal model for the precision matrix to model parsimoniously conditional dependence relationships of matrix variate data based on the Cartesian product of graphs. We call this tensor graphical lasso generalization TeraLasso. We demonstrate by using theory and examples that the TeraLasso model can be accurately and scalably estimated from very limited data samples of high dimensional variables with multiway co‐ordinates such as space, time and replicates. Statistical consistency and statistical rates of convergence are established for both the bigraphical lasso and TeraLasso estimators of the precision matrix and estimators of its support (non‐sparsity) set respectively. We propose a scalable composite gradient descent algorithm and analyse the computational convergence rate, showing that the composite gradient descent algorithm is guaranteed to converge at a geometric rate to the global minimizer of the TeraLasso objective function. Finally, we illustrate TeraLasso by using both simulation and experimental data from a meteorological data set, showing that we can accurately estimate precision matrices and recover meaningful conditional dependence graphs from high dimensional complex data sets.
Construction of simultaneous confidence bands for multiple logistic regression models over restricted regions

ABSTRACT This article presents methods for constructing an asymptotic hyperbolic band under the multiple logistic regression model when the predictor variables are restricted to a specific region . Scheffé's method yields unnecessarily wide, and hence conservative, bands if the predictor variables can be restricted to a certain region. Piegorsch and Casella [Confidence bands for logistic regression with restricted predictor variables. Biometrics. 1988;44:739–750] developed a procedure to build an asymptotic confidence band for the multiple logistic regression model over particular regions. Those regions are shown to be special cases of the region , which was first investigated by Seppanen and Uusipaikka [Confidence bands for linear regression over restricted regions. Scand J Stat. 1992;19:73–81] in the multiple linear regression context. This article also provides methods for constructing conservative confidence bands when the restricted region is not of the specified form. Particularly, rectangular restricted regions, which are commonly encountered in practice, are considered. Two examples are given to illustrate the proposed methodology, and one example shows that the proposed procedure outperforms the method given by Piegorsch and Casella (1988).
A pairwise comparison approach to ranking in chess team championships

Chess championships are often organised as a Swiss-system tournament, causing great challenges in ranking the participants due to the different strength of schedules and possible circular triads. The paper suggests that pairwise comparison matrices perform well in similar ranking problems. Some features of the proposed method are illustrated by the results of the 18th European Team Chess Championship. The analysis is able to take into account the influence of different opponents and robust with respect to the scaling technique chosen. The method is simple to compute as a solution of a linear equation system.
On the Equivalence of Factorized Information Criterion Regularization and the Chinese Restaurant Process Prior

Factorized Information Criterion (FIC) is a recently developed information criterion, based on which a novel model selection methodology, namely Factorized Asymptotic Bayesian (FAB) Inference, has been developed and successfully applied to various hierarchical Bayesian models. The Dirichlet Process (DP) prior, and one of its well known representations, the Chinese Restaurant Process (CRP), derive another line of model selection methods. FIC can be viewed as a prior distribution over the latent variable configurations. Under this view, we prove that when the parameter dimensionality $D_{c}=2$, FIC is equivalent to CRP. We argue that when $D_{c}>2$, FIC avoids an inherent problem of DP/CRP, i.e. the data likelihood will dominate the impact of the prior, and thus the model selection capability will weaken as $D_{c}$ increases. However, FIC overestimates the data likelihood. As a result, FIC may be overly biased towards models with less components. We propose a natural generalization of FIC, which finds a middle ground between CRP and FIC, and may yield more accurate model selection results than FIC.
Testing One Hypothesis Multiple Times

Hypothesis testing in presence of a nuisance parameter that is only identifiable under the alternative is challenging in part because standard asymptotic results (e.g., Wilks theorem for the generalized likelihood ratio test) do not apply. Several solutions have been proposed in the statistical literature and their practical implementation often reduces the problem into one of Testing One Hypothesis Multiple times (TOHM). Specifically, a fine discretization of the space of the non-identifiable parameter is specified, and an ensemble of sub-test statistics is obtained to test the null hypothesis against a set of sub-alternative hypothesis where the value of the non-identifiable parameter is fixed at each point of the discretization. The goal is to provide a global p-value as the standard of evidence for comparing the null hypothesis and the alternative hypothesis. In this paper, we first propose a simple and computationally efficient inferential tool to perform TOHM under stringent significance requirements, such as those typically required in the physical sciences. Further, we exploit the underlying discrete nature of the solution to clarify the connection between TOHM and multiple hypothesis testing. This allows us to identify scenarios where the simple Bonferroni correction is equivalent to TOHM, and thus it is not overly conservative.
Bayesian Constrained-Model Selection for Factor Analytic Modeling

My dissertation revolves around Bayesian approaches towards constrained statistical inference in the factor analysis (FA) model. Two interconnected types of restricted-model selection are considered. These types have a natural connection to selection problems in the exploratory FA (EFA) and confirmatory FA (CFA) model and are termed Type I and Type II model selection. Type I constrained-model selection is taken to mean the determination of the appropriate dimensionality of a model. This type of constrained-model selection connects with EFA in the sense of selecting the optimal dimensionality of the latent vector. Type II model selection is taken to mean the determination of appropriate inequality, order or shape restrictions on the parameter space. The dissertation connects Type II constrained-model selection to CFA by focusing on the determination of linear inequality constraints as expressions of the direction and (relative) strength of factor loadings. The figures accompanying this article are taken from the slides of my Division 5 Awards Symposium Invited address at the APA 2015 Annual Convention in Toronto. These slides can be retrieved from \url{this https URL}.
Optimal modification of the LRT for the equality of two high-dimensional covariance matrices

This paper considers the optimal modification of the likelihood ratio test (LRT) for the equality of two high-dimensional covariance matrices. The classical LRT is not well defined when the dimensions are larger than or equal to one of the sample sizes. In this paper, an optimally modified test that works well in cases where the dimensions may be larger than the sample sizes is proposed. In addition, the test is established under the weakest conditions on the moments and the dimensions of the samples. We also present weakly consistent estimators of the fourth moments, which are necessary for the proposed test, when they are not equal to 3. From the simulation results and real data analysis, we find that the performances of the proposed statistics are robust against affine transformations.
Fast estimation of multidimensional adaptive P-spline models

A fast and stable algorithm for estimating multidimensional adaptive P-spline models is presented. We call it as Separation of Overlapping Penalties (SOP) as it is an extension of the \textit{Separation of Anisotropic Penalties} (SAP) algorithm. SAP was originally derived for the estimation of the smoothing parameters of a multidimensional tensor product P-spline model with anisotropic penalties.
Smooth Alternating Direction Methods for Nonsmooth Constrained Convex Optimization

We propose two new alternating direction methods to solve "fully" nonsmooth constrained convex problems. Our algorithms have the best known worst-case iteration-complexity guarantee under mild assumptions for both the objective residual and feasibility gap. Through theoretical analysis, we show how to update all the algorithmic parameters automatically with clear impact on the convergence performance. We also provide a representative numerical example showing the advantages of our methods over the classical alternating direction methods using a well-known feasibility problem.
Bayesian Nonparametric Modeling of Heterogeneous Groups of Censored Data

Datasets containing large samples of time-to-event data arising from several small heterogeneous groups are commonly encountered in statistics. This presents problems as they cannot be pooled directly due to their heterogeneity or analyzed individually because of their small sample size. Bayesian nonparametric modelling approaches can be used to model such datasets given their ability to flexibly share information across groups. In this paper, we will compare three popular Bayesian nonparametric methods for modelling the survival functions of heterogeneous groups. Specifically, we will first compare the modelling accuracy of the Dirichlet process, the hierarchical Dirichlet process, and the nested Dirichlet process on simulated datasets of different sizes, where group survival curves differ in shape or in expectation. We, then, will compare the models on a real-world injury dataset.
Incentivized Advertising: Treatment Effect and Adverse Selection

Incentivized advertising is a new ad format that is gaining popularity in digital mobile advertising. In incentivized advertising, the publisher rewards users for watching an ad. An endemic issue here is adverse selection, where reward-seeking users select into incentivized ad placements to obtain rewards. Adverse selection reduces the publisher's ad profit as well as poses a difficulty to causal inference of the effectiveness of incentivized advertising. To this end, we develop a treatment effect model that allows and controls for unobserved adverse selection, and estimate the model using data from a mobile gaming app that offers both incentivized and non-incentivized ads. We find that rewarding users to watch an ad has an overall positive effect on the ad conversion rate. A user is 27% more likely to convert when being rewarded to watch an ad. However there is a negative offsetting effect that reduces the effectiveness of incentivized ads. Some users are averse to delayed rewards, they prefer to collect their rewards immediately after watching the incentivized ads, instead of pursuing the content of the ads further. For the subset of users who are averse to delayed rewards, the treatment effect is only 13%, while it can be as high as 47% for other users.
Topological density estimation

We introduce \emph{topological density estimation} (TDE), in which the multimodal structure of a probability density function is topologically inferred and subsequently used to perform bandwidth selection for kernel density estimation. We show that TDE has performance and runtime advantages over competing methods of kernel density estimation for highly multimodal probability density functions. We also show that TDE yields useful auxiliary information, that it can determine its own suitability for use, and we explain its performance.
An MM Algorithm for Split Feasibility Problems

The classical multi-set split feasibility problem seeks a point in the intersection of finitely many closed convex domain constraints, whose image under a linear mapping also lies in the intersection of finitely many closed convex range constraints. Split feasibility generalizes important inverse problems including convex feasibility, linear complementarity, and regression with constraint sets. When a feasible point does not exist, solution methods that proceed by minimizing a proximity function can be used to obtain optimal approximate solutions to the problem. We present an extension of the proximity function approach that generalizes the linear split feasibility problem to allow for non-linear mappings. Our algorithm is based on the principle of majorization-minimization, is amenable to quasi-Newton acceleration, and comes complete with convergence guarantees under mild assumptions. Furthermore, we show that the Euclidean norm appearing in the proximity function of the non-linear split feasibility problem can be replaced by arbitrary Bregman divergences. We explore several examples illustrating the merits of non-linear formulations over the linear case, with a focus on optimization for intensity-modulated radiation therapy.
A new method of joint nonparametric estimation of probability density and its support

In this paper we propose a new method of joint nonparametric estimation of probability density and its support. As is well known, nonparametric kernel density estimator has "boundary bias problem" when the support of the population density is not the whole real line. To avoid the unknown boundary effects, our estimator detects the boundary, and eliminates the boundary-bias of the estimator simultaneously. Moreover, we refer an extension to a simple multivariate case, and propose an improved estimator free from the unknown boundary bias.
Logistic push: a regression framework for partial AUC optimization

The area under the receiver operating characteristic curve (AUC) is often used to evaluate the performance of clinical prediction models. Recently, a more refined strategy has been proposed to examine a partial area under the curve (pAUC), which can account for differing costs associated with false negative versus false positive results. Such consideration can substantially increase the clinical utility of prediction models depending on the clinical question. Properties of the pAUC estimator create significant challenges for pAUC-optimal marker selection and model building. As such, current approaches towards these aims can be complex and computationally intensive. We present a simpler method based on weighted logistic regressions. We refer to our strategy as logistic push, due to shared heuristics with the ranking algorithm P-norm push. Logistic push is particularly useful in the high-dimensional setting, where fast and broadly available algorithms for fitting penalized regressions can be used for both marker selection and model fitting.
Warp Bridge Sampling: The Next Generation

Abstract Bridge sampling is an effective Monte Carlo (MC) method for estimating the ratio of normalizing constants of two probability densities, a routine computational problem in statistics, physics, chemistry, and other fields. The MC error of the bridge sampling estimator is determined by the amount of overlap between the two densities. In the case of unimodal densities, Warp-I, II, and III transformations are effective for increasing the initial overlap, but they are less so for multimodal densities. This article introduces Warp-U transformations that aim to transform multimodal densities into unimodal ones (hence “U”) without altering their normalizing constants. The construction of a Warp-U transformation starts with a normal (or other convenient) mixture distribution that has reasonable overlap with the target density p, whose normalizing constant is unknown. The stochastic transformation that maps back to its generating distribution is then applied to p yielding its Warp-U version, which we denote . Typically, is unimodal and has substantially increased overlap with . Furthermore, we prove that the overlap between and is guaranteed to be no less than the overlap between p and , in terms of any f-divergence. We propose a computationally efficient method to find an appropriate , and a simple but effective approach to remove the bias which results from estimating the normalizing constant and fitting with the same data. We illustrate our findings using 10 and 50 dimensional highly irregular multimodal densities, and demonstrate how Warp-U sampling can be used to improve the final estimation step of the Generalized Wang–Landau algorithm, a powerful sampling and estimation approach. Supplementary materials for this article are available online.
Discovering phase transitions with unsupervised learning

Unsupervised learning is a discipline of machine learning which aims at discovering patterns in large data sets or classifying the data into several categories without being trained explicitly. We show that unsupervised learning techniques can be readily used to identify phases and phases transitions of many-body systems. Starting with raw spin configurations of a prototypical Ising model, we use principal component analysis to extract relevant low-dimensional representations of the original data and use clustering analysis to identify distinct phases in the feature space. This approach successfully finds physical concepts such as the order parameter and structure factor to be indicators of a phase transition. We discuss the future prospects of discovering more complex phases and phase transitions using unsupervised learning techniques.
Recursive computation for evaluating the exact $p$-values of temporal and spatial scan statistics

Let $V$ be a finite set of indices, and let $B_i$, $i=1,\ldots,m$, be subsets of $V$ such that $V=\bigcup_{i=1}^{m}B_i$. Let $X_i$, $i\in V$, be independent random variables, and let $X_{B_i}=(X_j)_{j\in B_i}$. In this paper, we propose a recursive computation method to calculate the conditional expectation $E\bigl[\prod_{i=1}^m\chi_i(X_{B_i}) \,|\, N\bigr]$ with $N=\sum_{i\in V}X_i$ given, where $\chi_i$ is an arbitrary function. Our method is based on the recursive summation/integration technique using the Markov property in statistics. To extract the Markov property, we define an undirected graph whose cliques are $B_j$, and obtain its chordal extension, from which we present the expressions of the recursive formula. This methodology works for a class of distributions including the Poisson distribution (that is, the conditional distribution is the multinomial). This problem is motivated from the evaluation of the multiplicity-adjusted $p$-value of scan statistics in spatial epidemiology. As an illustration of the approach, we present the real data analyses to detect temporal and spatial clustering.
An Extended Empirical Saddlepoint Approximation for Intractable Likelihoods

The challenges posed by complex stochastic models used in computational ecology, biology and genetics have stimulated the development of approximate approaches to statistical inference. Here we focus on Synthetic Likelihood (SL), a procedure that reduces the observed and simulated data to a set of summary statistics, and quantifies the discrepancy between them through a synthetic likelihood function. SL requires little tuning, but it relies on the approximate normality of the summary statistics. We relax this assumption by proposing a novel, more flexible, density estimator: the Extended Empirical Saddlepoint approximation. In addition to proving the consistency of SL, under either the new or the Gaussian density estimator, we illustrate the method using two examples. One of these is a complex individual-based forest model for which SL offers one of the few practical possibilities for statistical inference. The examples show that the new density estimator is able to capture large departures from normality, while being scalable to high dimensions, and this in turn leads to more accurate parameter estimates, relative to the Gaussian alternative. The new density estimator is implemented by the esaddle R package, which can be found on the Comprehensive R Archive Network (CRAN).
A Tutorial on Libra: R package for the Linearized Bregman Algorithm in High Dimensional Statistics

The R package, Libra, stands for the LInearized BRegman Al- gorithm in high dimensional statistics. The Linearized Bregman Algorithm is a simple iterative procedure to generate sparse regularization paths of model estimation, which are rstly discovered in applied mathematics for image restoration and particularly suitable for parallel implementation in large scale problems. The limit of such an algorithm is a sparsity-restricted gradient descent ow, called the Inverse Scale Space, evolving along a par- simonious path of sparse models from the null model to over tting ones. In sparse linear regression, the dynamics with early stopping regularization can provably meet the unbiased Oracle estimator under nearly the same condition as LASSO, while the latter is biased. Despite their successful applications, statistical consistency theory of such dynamical algorithms remains largely open except for some recent progress on linear regression. In this tutorial, algorithmic implementations in the package are discussed for several widely used sparse models in statistics, including linear regression, logistic regres- sion, and several graphical models (Gaussian, Ising, and Potts). Besides the simulation examples, various application cases are demonstrated, with real world datasets from diabetes, publications of COPSS award winners, as well as social networks of two Chinese classic novels, Journey to the West and Dream of the Red Chamber.
Analysis of Thompson Sampling for Gaussian Process Optimization in the Bandit Setting

We consider the global optimization of a function over a continuous domain. At every evaluation attempt, we can observe the function at a chosen point in the domain and we reap the reward of the value observed. We assume that drawing these observations are expensive and noisy. We frame it as a continuum-armed bandit problem with a Gaussian Process prior on the function. In this regime, most algorithms have been developed to minimize some form of regret. Contrary to this popular norm, in this paper, we study the convergence of the sequential point $\boldsymbol{x}^t$ to the global optimizer $\boldsymbol{x}^*$ for the Thompson Sampling approach. Under some assumptions and regularity conditions, we show an exponential rate of convergence to the true optimal.