Miracle-dz/UniFAR-FTUs huggingface dataset:
Latent factor models for density estimation

Although discrete mixture modelling has formed the backbone of the literature on Bayesian density estimation, there are some well-known disadvantages. As an alternative to discrete mixtures, we propose a class of priors based on random nonlinear functions of a uniform latent variable with an additive residual. The induced prior for the density is shown to have desirable properties, including ease of centring on an initial guess, large support, posterior consistency and straightforward computation via Gibbs sampling. Some advantages over discrete mixtures, such as Dirichlet process mixtures of Gaussian kernels, are discussed and illustrated via simulations and an application.
Optimal selection of reduced rank estimators of high-dimensional matrices

We introduce a new criterion, the Rank Selection Criterion (RSC), for selecting the optimal reduced rank estimator of the coefficient matrix in multivariate response regression models. The corresponding RSC estimator minimizes the Frobenius norm of the fit plus a regularization term proportional to the number of parameters in the reduced rank model. The rank of the RSC estimator provides a consistent estimator of the rank of the coefficient matrix; in general, the rank of our estimator is a consistent estimate of the effective rank, which we define to be the number of singular values of the target matrix that are appropriately large. The consistency results are valid not only in the classic asymptotic regime, when n, the number of responses, and p, the number of predictors, stay bounded, and m, the number of observations, grows, but also when either, or both, n and p grow, possibly much faster than m. We establish minimax optimal bounds on the mean squared errors of our estimators. Our finite sample performance bounds for the RSC estimator show that it achieves the optimal balance between the approximation error and the penalty term. Furthermore, our procedure has very low computational complexity, linear in the number of candidate models, making it particularly appealing for large scale problems. We contrast our estimator with the nuclear norm penalized least squares (NNP) estimator, which has an inherently higher computational complexity than RSC, for multivariate regression models. We show that NNP has estimation properties similar to those of RSC, albeit under stronger conditions. However, it is not as parsimonious as RSC. We offer a simple correction of the NNP estimator which leads to consistent rank estimation. We verify and illustrate our theoretical findings via an extensive simulation study.
Sure screening for estimating equations in ultra-high dimensions

As the number of possible predictors generated by high-throughput experiments continues to increase, methods are needed to quickly screen out unimportant covariates. Model-based screening methods have been proposed and theoretically justified, but only for a few specific models. Model-free screening methods have also recently been studied, but can have lower power to detect important covariates. In this paper we propose EEScreen, a screening procedure that can be used with any model that can be fit using estimating equations, and provide unified results on its finite-sample screening performance. EEScreen thus generalizes many recently proposed model-based and model-free screening procedures. We also propose iEEScreen, an iterative version of EEScreen, and show that it is closely related to a recently studied boosting method for estimating equations. We show via simulations for two different estimating equations that EEScreen and iEEScreen are useful and flexible screening procedures, and demonstrate our methods on data from a multiple myeloma study.
An l1-Oracle Inequality for the Lasso

The Lasso has attracted the attention of many authors these last years. While many efforts have been made to prove that the Lasso behaves like a variable selection procedure at the price of strong (though unavoidable) assumptions on the geometric structure of these variables, much less attention has been paid to the analysis of the performance of the Lasso as a regularization algorithm. Our first purpose here is to provide a conceptually very simple result in this direction. We shall prove that, provided that the regularization parameter is properly chosen, the Lasso works almost as well as the deterministic Lasso. This result does not require any assumption at all, neither on the structure of the variables nor on the regression function. Our second purpose is to introduce a new estimator particularly adapted to deal with infinite countable dictionaries. This estimator is constructed as an l0-penalized estimator among a sequence of Lasso estimators associated to a dyadic sequence of growing truncated dictionaries. The selection procedure automatically chooses the best level of truncation of the dictionary so as to make the best tradeoff between approximation, l1-regularization and sparsity. From a theoretical point of view, we shall provide an oracle inequality satisfied by this selected Lasso estimator. The oracle inequalities established for the Lasso and the selected Lasso estimators shall enable us to derive rates of convergence on a wide class of functions, showing that these estimators perform at least as well as greedy algorithms. Besides, we shall prove that the rates of convergence achieved by the selected Lasso estimator are optimal in the orthonormal case by bounding from below the minimax risk on some Besov bodies. Finally, some theoretical results about the performance of the Lasso for infinite uncountable dictionaries will be studied in the specific framework of neural networks. All the oracle inequalities presented in this paper are obtained via the application of a single general theorem of model selection among a collection of nonlinear models which is a direct consequence of the Gaussian concentration inequality. The key idea that enables us to apply this general theorem is to see l1-regularization as a model selection procedure among l1-balls.
Sparse recovery under matrix uncertainty

We consider the model y - Y0 +, Z- X + 3? where the random vector y ∈ ℝ n and the random n × p matrix Z are observed, the n x p matrix X is unknown, Θ is an n x p random noise matrix, ξ ∈ ℝ n is a noise independent of Θ, and θ * is a vector of unknown parameters to be estimated. The matrix uncertainty is in the fact that X is observed with additive error. For dimensions p that can be much larger than the sample size n, we consider the estimation of sparse vectors θ * . Under matrix uncertainty, the Lasso and Dantzig selector turn out to be extremely unstable in recovering the sparsity pattern (i.e., of the set of nonzero components of θ * ), even if the noise level is very small. We suggest new estimators called matrix uncertainty selectors (or, shortly, the MU-selectors) which are close to θ * in different norms and in the prediction risk if the restricted eigenvalue assumption on X is satisfied. We also show that under somewhat stronger assumptions, these estimators recover correctly the sparsity pattern.
Statistical classification techniques for photometric supernova typing

Future photometric supernova surveys will produce vastly more candidates than can be followed up spectroscopically, highlighting the need for effective classification methods based on light curves alone. Here we introduce boosting and kernel density estimation techniques which have minimal astrophysical input, and compare their performance on 20 000 simulated Dark Energy Survey light curves. We demonstrate that these methods perform very well provided a representative sample of the full population is used for training. Interestingly, we find that they do not require the redshift of the host galaxy or candidate supernova. However, training on the types of spectroscopic subsamples currently produced by supernova surveys leads to poor performance due to the resulting bias in training, and we recommend that special attention be given to the creation of representative training samples. We show that given a typical non-representative training sample, S, one can expect to pull out a representative subsample of about 10 per cent of the size of S, which is large enough to outperform the methods trained on all of S.
Using machine learning for discovery in synoptic survey imaging data

Modern time-domain surveys continuously monitor large swaths of the sky to look for astronomical variability. Astrophysical discovery in such data sets is complicated by the fact that detections of real transient and variable sources are highly outnumbered by bogus detections caused by imperfect subtractions, atmospheric effects and detector artefacts. In this work we present a machine learning (ML) framework for discovery of variability in time-domain imaging surveys. Our ML methods provide probabilistic statements, in near real time, about the degree to which each newly observed source is astrophysically relevant source of variable brightness. We provide details about each of the analysis steps involved, including compilation of the training and testing sets, construction of descriptive image-based and contextual features, and optimization of the feature subset and model tuning parameters. Using a validation set of nearly 30,000 objects from the Palomar Transient Factory, we demonstrate a missed detection rate of at most 7.7% at our chosen false-positive rate of 1% for an optimized ML classifier of 23 features, selected to avoid feature correlation and over-fitting from an initial library of 42 attributes. Importantly, we show that our classification methodology is insensitive to mis-labelled training data up to a contamination of nearly 10%, making it easier to compile sufficient training sets for accurate performance in future surveys. This ML framework, if so adopted, should enable the maximization of scientific gain from future synoptic survey and enable fast follow-up decisions on the vast amounts of streaming data produced by such experiments.
Robust subspace recovery by geodesically convex optimization

We introduce Tyler's M-estimator to robustly recover the underlying linear model from a data set contaminated by outliers. We prove that the objective function of this estimator is geodesically convex on the manifold of all positive definite matrices and have a unique minimizer. Besides, we prove that when inliers (i.e., points that are not outliers) are sampled from a subspace and the percentage of outliers is bounded by some number, then under some very weak assumptions a commonly used algorithm of this estimator can recover the underlying subspace exactly. We also show that empirically this algorithm compares favorably with other convex algorithms of subspace recovery.
Adaptive parallel tempering algorithm

Parallel tempering is a generic Markov chain Monte Carlo sampling method which allows good mixing with multimodal target distributions, where conventional Metropolis-Hastings algorithms often fail. The mixing properties of the sampler depend strongly on the choice of tuning parameters, such as the temperature schedule and the proposal distribution used for local exploration. We propose an adaptive algorithm which tunes both the temperature schedule and the parameters of the random-walk Metropolis kernel automatically. We prove the convergence of the adaptation and a strong law of large numbers for the algorithm. We illustrate the performance of our method with examples. Our empirical findings indicate that the algorithm can cope well with different kind of scenarios without prior tuning.
Convergence properties of pseudo-marginal Markov chain Monte Carlo algorithms

We study convergence properties of pseudo-marginal Markov chain Monte Carlo algorithms (Andrieu and Roberts [Ann. Statist. 37 (2009) 697-725]). We find that the asymptotic variance of the pseudo-marginal algorithm is always at least as large as that of the marginal algorithm. We show that if the marginal chain admits a (right) spectral gap and the weights (normalised estimates of the target density) are uniformly bounded, then the pseudo-marginal chain has a spectral gap. In many cases, a similar result holds for the absolute spectral gap, which is equivalent to geometric ergodicity. We consider also unbounded weight distributions and recover polynomial convergence rates in more specific cases, when the marginal algorithm is uniformly ergodic or an independent Metropolis-Hastings or a random-walk Metropolis targeting a super-exponential density with regular contours. Our results on geometric and polynomial convergence rates imply central limit theorems. We also prove that under general conditions, the asymptotic variance of the pseudo-marginal algorithm converges to the asymptotic variance of the marginal algorithm if the accuracy of the estimators is increased.
Flexible Shrinkage Estimation in High-Dimensional Varying Coefficient Models

We consider the problem of simultaneous variable selection and constant coefficient identification in high-dimensional varying coefficient models based on B-spline basis expansion. Both objectives can be considered as some type of model selection problems and we show that they can be achieved by a double shrinkage strategy. We apply the adaptive group Lasso penalty in models involving a diverging number of covariates, which can be much larger than the sample size, but we assume the number of relevant variables is smaller than the sample size via model sparsity. Such so-called ultra-high dimensional settings are especially challenging in semiparametric models as we consider here and has not been dealt with before. Under suitable conditions, we show that consistency in terms of both variable selection and constant coefficient identification can be achieved, as well as the oracle property of the constant coefficients. Even in the case that the zero and constant coefficients are known a priori, our results appear to be new in that it reduces to semivarying coefficient models (a.k.a. partially linear varying coefficient models) with a diverging number of covariates. We also theoretically demonstrate the consistency of a semiparametric BIC-type criterion in this high-dimensional context, extending several previous results. The finite sample behavior of the estimator is evaluated by some Monte Carlo studies.
On Wishart distribution

This paper proposes a unified approach that enables the Wishart distribution to be studied simultaneously in the real, complex, quaternion and octonion cases. In particular, the noncentral generalised Wishart distribution, the joint density of the eigenvalues and the distribution of the maximum eigenvalue are obtained for real normed division algebras.
Degrees of Equivalence in a Key Comparison

In an interlaboratory key comparison, a data analysis procedure for this comparison was proposed and recommended by CIPM [1, 2, 3], therein the degrees of equivalence of measurement standards of the laboratories participated in the comparison and the ones between each two laboratories were introduced but a corresponding clear and plausible measurement model was not given. Authors in [4] offered possible measurement models for a given comparison and a suitable model was selected out after rigorous analyzing steps for expectation values of these degrees of equivalence. The systematic laboratory-effects model was then selected as a right one in this report. Those models were all based on the one true value existence assumption. However in the year 2008, a new version of the Vocabulary for International Metrology (VIM) [7] was issued where the true value of a given measurement standard should be now perceived as multi true values which following a given statistics distribution. Applying this perception of true values of a measurement standard with combination of the steps in [4], measurement models have been developed and degrees of equivalence have been analyzed. The results show that although with new definition, the systematic laboratory-effects model is still the reasonable one in a given key comparison.
Spectral approximations in machine learning

In many areas of machine learning, it becomes necessary to find the eigenvector decompositions of large matrices. We discuss two methods for reducing the computational burden of spectral decompositions: the more venerable Nystom extension and a newly introduced algorithm based on random projections. Previous work has centered on the ability to reconstruct the original matrix. We argue that a more interesting and relevant comparison is their relative performance in clustering and classification tasks using the approximate eigenvectors as features. We demonstrate that performance is task specific and depends on the rank of the approximation.
Statistical inference for time-changed Lévy processes via composite characteristic function estimation

In this article, the problem of semi-parametric inference on the parameters of a multidimensional L\'{e}vy process $L_t$ with independent components based on the low-frequency observations of the corresponding time-changed L\'{e}vy process $L_{\mathcal{T}(t)}$, where $\mathcal{T}$ is a nonnegative, nondecreasing real-valued process independent of $L_t$, is studied. We show that this problem is closely related to the problem of composite function estimation that has recently gotten much attention in statistical literature. Under suitable identifiability conditions, we propose a consistent estimate for the L\'{e}vy density of $L_t$ and derive the uniform as well as the pointwise convergence rates of the estimate proposed. Moreover, we prove that the rates obtained are optimal in a minimax sense over suitable classes of time-changed L\'{e}vy models. Finally, we present a simulation study showing the performance of our estimation algorithm in the case of time-changed Normal Inverse Gaussian (NIG) L\'{e}vy processes.
Quantile Based Variable Mining : Detection, FDR based Extraction and Interpretation

This paper outlines a unified framework for high dimensional variable selection for classification problems. Traditional approaches to finding interesting variables mostly utilize only partial information through moments (like mean difference). On the contrary, in this paper we address the question of variable selection in full generality from a distributional point of view. If a variable is not important for classification, then it will have similar distributional aspect under different classes. This simple and straightforward observation motivates us to quantify `How and Why' the distribution of a variable changes over classes through CR-statistic. The second contribution of our paper is to develop and investigate the FDR based thresholding technology from a completely new point of view for adaptive thresholding, which leads to a elegant algorithm called CDfdr. This paper attempts to show how all of these problems of detection, extraction and interpretation for interesting variables can be treated in a unified way under one broad general theme - comparison analysis. It is proposed that a key to accomplishing this unification is to think in terms of the quantile function and the comparison density. We illustrate and demonstrate the power of our methodology using three real data sets.
Analysis of 1:1 Matched Cohort Studies and Twin Studies, with Binary Exposures and Binary Outcomes

To improve confounder adjustments, observational studies are often matched on potential confounders. While matched case-control studies are common and well covered in the literature, our focus here is on matched cohort studies, which are less common and sparsely dis- cussed in the literature. Matched data also arise naturally in twin stud- ies, as a cohort of exposure-discordant twins can be viewed as being matched on a large number of potential confounders. The analysis of twin studies will be given special attention. We give an overview of vari- ous analysis methods for matched cohort studies with binary exposures and binary outcomes. In particular, our aim is to answer the following questions: (1) What are the target parameters in the common analysis methods? (2) What are the underlying assumptions in these methods? (3) How do the methods compare in terms of statistical power?
Combining Predictive Distributions

Predictive distributions need to be aggregated when probabilistic forecasts are merged, or when expert opinions expressed in terms of probability distributions are fused. We take a prediction space approach that applies to discrete, mixed discrete-continuous and continuous predictive distributions alike, and study combination formulas for cumulative distribution functions from the perspectives of coherence, probabilistic and conditional calibration, and dispersion. Both linear and non-linear aggregation methods are investigated, including generalized, spread-adjusted and beta-transformed linear pools. The effects and techniques are demonstrated theoretically, in simulation examples, and in case studies on density forecasts for S&P 500 returns and daily maximum temperature at Seattle-Tacoma Airport.
On the identifiability of copulas in bivariate competing risks models

In competing risks models, the joint distribution of the event times is not identifiable even when the margins are fully known, which has been referred to as the “identifiability crisis in competing risks analysis” (Crowder, 1991). We model the dependence between the event times by an unknown copula and show that identification is actually possible within many frequently used families of copulas. The result is then extended to the case where one margin is unknown. The Canadian Journal of Statistics 41: 291–303; 2013 © 2013 Statistical Society of Canada
Estimation of high-dimensional low-rank matrices

Suppose that we observe entries or, more generally, linear combinations of entries of an unknown m x T-matrix A corrupted by noise. We are particularly interested in the high-dimensional setting where the number mT of unknown entries can be much larger than the sample size N. Motivated by several applications, we consider estimation of matrix A under the assumption that it has small rank. This can be viewed as dimension reduction or sparsity assumption. In order to shrink toward a low-rank representation, we investigate penalized least squares estimators with a Schatten-p quasi-norm penalty term, p ≤ 1. We study these estimators under two possible assumptions—a modified version of the restricted isometry condition and a uniform bound on the ratio "empirical norm induced by the sampling operator/Frobenius norm. " The main results are stated as nonasymptotic upper bounds on the prediction risk and on the Schatten-q risk of the estimators, where q ∈ [p, 2]. The rates that we obtain for the prediction risk are of the form rm/N (for m = T), up to logarithmic factors, where r is the rank of A. The particular examples of multi-task learning and matrix completion are worked out in detail. The proofs are based on tools from the theory of empirical processes. As a by-product, we derive bounds for the kth entropy numbers of the quasi-convex Schatten class embeddings S M p → S M 2 , p < 1, which are of independent interest.
Quasi-Likelihood and/or Robust Estimation in High Dimensions

We consider the theory for the high-dimensional generalized linear model with the Lasso. After a short review on theoretical results in literature, we present an extension of the oracle results to the case of quasi-likelihood loss. We prove bounds for the prediction error and $\ell_1$-error. The results are derived under fourth moment conditions on the error distribution. The case of robust loss is also given. We moreover show that under an irrepresentable condition, the $\ell_1$-penalized quasi-likelihood estimator has no false positives.