Miracle-dz/UniFAR-FTUs huggingface dataset:
Unbiased Hamiltonian Monte Carlo with couplings

We propose a method for parallelization of Hamiltonian Monte Carlo estimators. Our approach involves constructing a pair of Hamiltonian Monte Carlo chains that are coupled in such a way that they meet exactly after some random number of iterations. These chains can then be combined so that the resulting estimators are unbiased. This allows us to produce independent replicates in parallel and average them to obtain estimators that are consistent in the limit of the number of replicates, rather than in the usual limit of the number of Markov chain iterations. We investigate the scalability of our coupling in high dimensions on a toy example. The choice of algorithmic parameters and the efficiency of our proposed approach are then illustrated on a logistic regression with 300 covariates and a log-Gaussian Cox point processes model with low- to fine-grained discretizations.
Common change point estimation in panel data from the least squares and maximum likelihood viewpoints

We establish the convergence rates and asymptotic distributions of the common break change-point estimators, obtained by least squares and maximum likelihood in panel data models and compare their asymptotic variances. Our model assumptions accommodate a variety of commonly encountered probability distributions and, in particular, models of particular interest in econometrics beyond the commonly analyzed Gaussian model, including the zero-inflated Poisson model for count data, and the probit and tobit models. We also provide novel results for time dependent data in the signal-plus-noise model, with emphasis on a wide array of noise processes, including Gaussian process, MA$(\infty)$ and $m$-dependent processes. The obtained results show that maximum likelihood estimation requires a stronger signal-to-noise model identifiability condition compared to its least squares counterpart. Finally, since there are three different asymptotic regimes that depend on the behavior of the norm difference of the model parameters before and after the change point, which cannot be realistically assumed to be known, we develop a novel data driven adaptive procedure that provides valid confidence intervals for the common break, without requiring a priori knowledge of the asymptotic regime the problem falls in.
Fast spatial inference in the homogeneous Ising model

The Ising model is important in statistical modeling and inference in many applications, however its normalizing constant, mean number of active vertices and mean spin interaction are intractable. We provide accurate approximations that make it possible to calculate these quantities numerically. Simulation studies indicate good performance when compared to Markov Chain Monte Carlo methods and at a tiny fraction of the time. The methodology is also used to perform Bayesian inference in a functional Magnetic Resonance Imaging activation detection experiment.
Extended Gauss-Newton and Gauss-Newton-ADMM Algorithms for Low-Rank Matrix Optimization

We develop a generic Gauss-Newton (GN) framework for solving a class of nonconvex optimization problems involving low-rank matrix variables. As opposed to standard Gauss-Newton method, our framework allows one to handle general smooth convex cost function via its surrogate. The main complexity-per-iteration consists of the inverse of two rank-size matrices and at most six small matrix multiplications to compute a closed form Gauss-Newton direction, and a backtracking linesearch. We show, under mild conditions, that the proposed algorithm globally and locally converges to a stationary point of the original nonconvex problem. We also show empirically that the Gauss-Newton algorithm achieves much higher accurate solutions compared to the well studied alternating direction method (ADM). Then, we specify our Gauss-Newton framework to handle the symmetric case and prove its convergence, where ADM is not applicable without lifting variables. Next, we incorporate our Gauss-Newton scheme into the alternating direction method of multipliers (ADMM) to design a GN-ADMM algorithm for solving the low-rank optimization problem. We prove that, under mild conditions and a proper choice of the penalty parameter, our GN-ADMM globally converges to a stationary point of the original problem. Finally, we apply our algorithms to solve several problems in practice such as low-rank approximation, matrix completion, robust low-rank matrix recovery, and matrix recovery in quantum tomography. The numerical experiments provide encouraging results to motivate the use of nonconvex optimization.
LPG: a four-groups probabilistic approach to leveraging pleiotropy in genome-wide association studies

To date, genome-wide association studies (GWAS) have successfully identified tens of thousands of genetic variants among a variety of traits/diseases, shedding a light on the genetic architecture of complex diseases. Polygenicity of complex diseases, which refers to the phenomenon that a vast number of risk variants collectively contribute to the heritability of complex diseases with modest individual effects, have been widely accepted. This imposes a major challenge towards fully characterizing the genetic bases of complex diseases. An immediate implication of polygenicity is that a much larger sample size is required to detect risk variants with weak/moderate effects. Meanwhile, accumulating evidence suggests that different complex diseases can share genetic risk variants, a phenomenon known as pleiotropy. In this study, we propose a statistical framework for Leveraging Pleiotropic effects in large-scale GWAS data (LPG). LPG utilizes a variational Bayesian expectation-maximization (VBEM) algorithm, making it computationally efficient and scalable for genome-wide scale analysis. To demon- strate the advantage of LPG over existing methods that do not leverage pleiotropy, we conducted extensive simulation studies and also applied LPG to analyze three au- toimmune disorders (Crohn's disease, rheumatoid arthritis and Type 1 diabetes). The results indicate that LPG can improve the power of prioritization of risk variants and accuracy of risk prediction by leveraging pleiotropy. The software is available at http- s://github.com/Shufeyangyi2015310117/LPG.
On the convergence rate of the three operator splitting scheme

The three operator splitting scheme was recently proposed by [Davis and Yin, 2015] as a method to optimize composite objective functions with one convex smooth term and two convex (possibly non-smooth) terms for which we have access to their proximity operator. In this short note we provide an alternative proof for the sublinear rate of convergence of this method.
Causal Structure Learning

Graphical models can represent a multivariate distribution in a convenient and accessible form as a graph. Causal models can be viewed as a special class of graphical models that represent not only the distribution of the observed system but also the distributions under external interventions. They hence enable predictions under hypothetical interventions, which is important for decision making. The challenging task of learning causal models from data always relies on some underlying assumptions. We discuss several recently proposed structure learning algorithms and their assumptions, and we compare their empirical performance under various scenarios.
Unbiased multi-index Monte Carlo

ABSTRACT We introduce a new class of Monte Carlo-based approximations of expectations of random variables such that their laws are only available via certain discretizations. Sampling from the discretized versions of these laws can typically introduce a bias. In this paper, we show how to remove that bias, by introducing a new version of multi-index Monte Carlo (MIMC) that has the added advantage of reducing the computational effort, relative to i.i. d. sampling from the most precise discretization, for a given level of error. We cover extensions of results regarding variance and optimality criteria for the new approach. We apply the methodology to the problem of computing an unbiased mollified version of the solution of a partial differential equation with random coefficients. A second application concerns the Bayesian inference (the smoothing problem) of an infinite-dimensional signal modeled by the solution of a stochastic partial differential equation that is observed on a discrete space grid and at discrete times. Both applications are complemented by numerical simulations.
ESTIMATION AND INFERENCE FOR MOMENTS OF RATIOS WITH ROBUSTNESS AGAINST LARGE TRIMMING BIAS

Researchers often trim observations with small values of the denominator A when they estimate moments of the form $\mathbb {E}[B/A]$ . Large trimming is common in practice to reduce variance, but it incurs a large bias. This paper provides a novel method of correcting the large trimming bias. If a researcher is willing to assume that the joint distribution between A and B is smooth, then the trimming bias may be estimated well. Along with the proposed bias correction method, we also develop an inference method. Practical advantages of the proposed method are demonstrated through simulation studies, where the data generating process entails a heavy-tailed distribution of $B/A$ . Applying the proposed method to the Compustat database, we analyze the history of external financial dependence of U.S. manufacturing firms for years 2000–2010.
Semi-parametric estimation of the variogram of a Gaussian process with stationary increments

We consider the semi-parametric estimation of a scale parameter of a one-dimensional Gaussian process with known smoothness. We suggest an estimator based on quadratic variations and on the moment method. We provide asymptotic approximations of the mean and variance of this estimator, together with asymptotic normality results, for a large class of Gaussian processes. We allow for general mean functions and study the aggregation of several estimators based on various variation sequences. In extensive simulation studies, we show that the asymptotic results accurately depict the finite-sample situations already for small to moderate sample sizes. We also compare various variation sequences and highlight the efficiency of the aggregation procedure.
An online updating approach for testing the proportional hazards assumption with streams of survival data

The Cox model—which remains the first choice for analyzing time‐to‐event data, even for large data sets—relies on the proportional hazards (PH) assumption. When survival data arrive sequentially in chunks, a fast and minimally storage intensive approach to test the PH assumption is desirable. We propose an online updating approach that updates the standard test statistic as each new block of data becomes available and greatly lightens the computational burden. Under the null hypothesis of PH, the proposed statistic is shown to have the same asymptotic distribution as the standard version computed on an entire data stream with the data blocks pooled into one data set. In simulation studies, the test and its variant based on most recent data blocks maintain their sizes when the PH assumption holds and have substantial power to detect different violations of the PH assumption. We also show in simulation that our approach can be used successfully with “big data” that exceed a single computer's computational resources. The approach is illustrated with the survival analysis of patients with lymphoma cancer from the Surveillance, Epidemiology, and End Results Program. The proposed test promptly identified deviation from the PH assumption, which was not captured by the test based on the entire data.
NEON+: Accelerated Gradient Methods for Extracting Negative Curvature for Non-Convex Optimization

Accelerated gradient (AG) methods are breakthroughs in convex optimization, improving the convergence rate of the gradient descent method for optimization with smooth functions. However, the analysis of AG methods for non-convex optimization is still limited. It remains an open question whether AG methods from convex optimization can accelerate the convergence of the gradient descent method for finding local minimum of non-convex optimization problems. This paper provides an affirmative answer to this question. In particular, we analyze two renowned variants of AG methods (namely Polyak's Heavy Ball method and Nesterov's Accelerated Gradient method) for extracting the negative curvature from random noise, which is central to escaping from saddle points. By leveraging the proposed AG methods for extracting the negative curvature, we present a new AG algorithm with double loops for non-convex optimization~\footnote{this is in contrast to a single-loop AG algorithm proposed in a recent manuscript~\citep{AGNON}, which directly analyzed the Nesterov's AG method for non-convex optimization and appeared online on November 29, 2017. However, we emphasize that our work is an independent work, which is inspired by our earlier work~\citep{NEON17} and is based on a different novel analysis. }, which converges to second-order stationary point $\x$ such that $\|\nabla f(\x)\|\leq \epsilon$ and $\nabla^2 f(\x)\geq -\sqrt{\epsilon} I$ with $\widetilde O(1/\epsilon^{1.75})$ iteration complexity, improving that of gradient descent method by a factor of $\epsilon^{-0.25}$ and matching the best iteration complexity of second-order Hessian-free methods for non-convex optimization.
Towards Standardization of Data Licenses: The Montreal Data License

This paper provides a taxonomy for the licensing of data in the fields of artificial intelligence and machine learning. The paper's goal is to build towards a common framework for data licensing akin to the licensing of open source software. Increased transparency and resolving conceptual ambiguities in existing licensing language are two noted benefits of the approach proposed in the paper. In parallel, such benefits may help foster fairer and more efficient markets for data through bringing about clearer tools and concepts that better define how data can be used in the fields of AI and ML. The paper's approach is summarized in a new family of data license language - \textit{the Montreal Data License (MDL)}. Alongside this new license, the authors and their collaborators have developed a web-based tool to generate license language espousing the taxonomies articulated in this paper.
Self-dual forms: action, Hamiltonian and compactification

It has been shown that, by adding an extra free field that decouples from the dynamics, one can construct actions for interacting 2n-form fields with self-dual field strengths in 4n+ 2 dimensions. In this paper we analyze canonical formulation of these theories, and show that the resulting Hamiltonian reduces to the sum of two Hamiltonians with independent degrees of freedom. One of them is free and has no physical consequence, while the other contains the physical degrees of freedom with the desired interactions. For the special cases of chiral scalars in two dimensions and chiral two form fields in six dimensions, we discuss compactification of these theories respectively on a circle and a two dimensional torus, and show that we recover the expected properties of these systems, including S-duality invariance in four dimensions.
Identifying the effect of public holidays on daily demand for gas

To reduce operational costs and to ensure security of supply, gas distribution networks require accurate forecasts of the demand for gas. Among domestic and commercial customers, demand relates primarily to the weather and patterns of life and work. Public holidays have a pronounced effect which often spreads into neighbouring days. We call this spread the ‘proximity effect’. Traditionally, the days over which the proximity effect is felt are prespecified in fixed windows around each holiday, allowing no uncertainty in their identification. We are motivated by an application to modelling daily gas demand in two large British regions. We introduce a novel model which does not fix the days on which the proximity effect is felt. Our approach uses a four‐state, non‐homogeneous hidden Markov model, with cyclic dynamics, where the classification of days as public holidays is observed, but the assignment of days as ‘pre‐holiday’, ‘post‐holiday’ or ‘normal’ days is unknown. The number of days to the preceding and succeeding holidays guide transitions between states. We apply Bayesian inference and illustrate the benefit of our modelling approach. A version of the model is now being used by one of the UK's regional distribution networks.
Domain of Inverse Double Arcsine Transformation

To combine the proportions from different studies for meta-analysis, Freeman and Tukey double arcsine tranformation can be useful for normalization and variance stabilization. The inverse function of the double arcsine transformation has been also derived in the literature to recover the original scale of the proportion after aggregation. In this brief note, we present the domain and range of the inverse double arcsine transformation both analytically and graphically. We notice an erratic behavior in the mathematical formula for the inverse double arcsine tranformation at both limits of its domain, and propose approximation methods for both small and large samples. We also propose a simple accuracy measure, the maximum percent error (MPE), of the large sample approximation, which can be used to determine the sample size that would provide a certain accuracy level, and conversely to determine the accuracy level of the approximation given a sample size.
Validity and Robustness of Tests in Survival Analysis under Covariate-Adaptive Randomization

Covariate-adaptive randomization is popular in clinical trials with sequentially arrived patients for balancing treatment assignments across prognostic factors which may have influence on the response. However, there exists no theoretical work about testing hypotheses under covariate-adaptive randomization in survival analysis, although covariate-adaptive randomization has been used in survival analysis for a long time and its main application is in survival analysis. Often times, practitioners would simply adopt a conventional test such as the log-rank test or score test to compare two treatments, which is controversial since tests derived under simple randomization may not be valid under other randomization schemes. In this article, we prove that the log-rank test valid under simple randomization is conservative in terms of type I error under covariate-adaptive randomization, and the robust score test developed under simple randomization is no longer robust under covariate-adaptive randomization. We then propose a calibration type log-rank or score test that is valid and robust under covariate-adaptive randomization. Furthermore, we obtain Pitman's efficacy of log-rank and score tests to compare their asymptotic relative efficiency. Simulation studies about the type I error and power of various tests are presented under several popular randomization schemes.
Prediction of the margin of victory only from team rankings for regular season games in NCAA men’s basketball

The main objective of this article was to investigate the extent to which the margin of victory can be predicted solely by the rankings of the opposing teams in NCAA Division I men’s basketball games. Several past studies have modeled this relationship for the games played during the March Madness tournament. This work aimed at verifying whether the models advocated in previous studies can accurately predict the margin of victory in regular season games. Indeed, most previous articles have shown that a simple quadratic regression model provides fairly accurate predictions of the margin of victory when team rankings only range from 1 to 16. Does that still hold true when team rankings increase to 351? Do the model assumptions hold? Can semi- or nonparametric methods that yield even better results (i.e. predicted margins of victory that more closely resemble actual results) be found? The analyses presented in this article suggest that the answer is “yes” to all three questions!
Heteroskedastic PCA: Algorithm, optimality, and applications

Principal component analysis (PCA) and singular value decomposition (SVD) are widely used in statistics, machine learning, and applied mathematics. It has been well studied in the case of homoskedastic noise, where the noise levels of the contamination are homogeneous. In this paper, we consider PCA and SVD in the presence of heteroskedastic noise, which arises naturally in a range of applications. We introduce a general framework for heteroskedastic PCA and propose an algorithm called HeteroPCA, which involves iteratively imputing the diagonal entries to remove the bias due to heteroskedasticity. This procedure is computationally efficient and provably optimal under the generalized spiked covariance model. A key technical step is a deterministic robust perturbation analysis on the singular subspace, which can be of independent interest. The effectiveness of the proposed algorithm is demonstrated in a suite of applications, including heteroskedastic low-rank matrix denoising, Poisson PCA, and SVD based on heteroskedastic and incomplete data.
GaussianProcesses.jl: A Nonparametric Bayes Package for the Julia Language

Gaussian processes are a class of flexible nonparametric Bayesian tools that are widely used across the sciences, and in industry, to model complex data sources. Key to applying Gaussian process models is the availability of well-developed open source software, which is available in many programming languages. In this paper, we present a tutorial of the GaussianProcesses.jl package that has been developed for the Julia language. GaussianProcesses.jl utilises the inherent computational benefits of the Julia language, including multiple dispatch and just-in-time compilation, to produce a fast, flexible and user-friendly Gaussian processes package. The package provides a range of mean and kernel functions with supporting inference tools to fit the Gaussian process models, as well as a range of alternative likelihood functions to handle non-Gaussian data (e.g. binary classification models). The package makes efficient use of existing Julia packages to provide users with a range of optimization and plotting tools.
A Bayesian Approach to Restricted Latent Class Models for Scientifically-Structured Clustering of Multivariate Binary Outcomes

This paper presents a model-based method for clustering multivariate binary observations that incorporates constraints consistent with the scientific context. The approach is motivated by the precision medicine problem of identifying autoimmune disease patient subsets or classes who may require different treatments. We start with a family of restricted latent class models or RLCMs (e.g., Xu and Shang, 2018). However, in the motivating example and many others like it, the unknown number of classes and the definition of classes using binary states are among the targets of inference. We use a Bayesian approach to RCLMs in order to use informative prior assumptions on the number and definitions of latent classes to be consistent with scientific knowledge so that the posterior distribution tends to concentrate on smaller numbers of clusters and sparser binary patterns. The paper derives a posterior sampling algorithm based on Markov chain Monte Carlo with split-merge updates to efficiently explore the space of clustering allocations. Through simulations under the assumed model and realistic deviations from it, we demonstrate greater interpretability of results and superior finite-sample clustering performance for our method compared to common alternatives. The methods are illustrated with an analysis of protein data to detect clusters representing autoantibody classes among scleroderma patients.