Miracle-dz/UniFAR-FTUs huggingface dataset:
Adaptive estimation of the rank of the coefficient matrix in high-dimensional multivariate response regression models

We consider the multivariate response regression problem with a regression coefficient matrix of low, unknown rank. In this setting, we analyze a new criterion for selecting the optimal reduced rank. This criterion differs notably from the one proposed in Bunea, She and Wegkamp [7] in that it does not require estimation of the unknown variance of the noise, nor depends on a delicate choice of a tuning parameter. We develop an iterative, fully data-driven procedure, that adapts to the optimal signal to noise ratio. This procedure finds the true rank in a few steps with overwhelming probability. At each step, our estimate increases, while at the same time it does not exceed the true rank. Our finite sample results hold for any sample size and any dimension, even when the number of responses and of covariates grow much faster than the number of observations. We perform an extensive simulation study that confirms our theoretical findings. The new method performs better and more stable than that in [7] in both low- and high-dimensional settings.
Manifold unwrapping using density ridges

Research on manifold learning within a density ridge estimation framework has shown great potential in recent work for both estimation and de-noising of manifolds, building on the intuitive and well-defined notion of principal curves and surfaces. However, the problem of unwrapping or unfolding manifolds has received relatively little attention within the density ridge approach, despite being an integral part of manifold learning in general. This paper proposes two novel algorithms for unwrapping manifolds based on estimated principal curves and surfaces for one- and multi-dimensional manifolds respectively. The methods of unwrapping are founded in the realization that both principal curves and principal surfaces will have inherent local maxima of the probability density function. Following this observation, coordinate systems that follow the shape of the manifold can be computed by following the integral curves of the gradient flow of a kernel density estimate on the manifold. Furthermore, since integral curves of the gradient flow of a kernel density estimate is inherently local, we propose to stitch together local coordinate systems using parallel transport along the manifold. We provide numerical experiments on both real and synthetic data that illustrates clear and intuitive unwrapping results comparable to state-of-the-art manifold learning algorithms.
GAN and VAE from an Optimal Transport Point of View

This short article revisits some of the ideas introduced in arXiv:1701.07875 and arXiv:1705.07642 in a simple setup. This sheds some lights on the connexions between Variational Autoencoders (VAE), Generative Adversarial Networks (GAN) and Minimum Kantorovitch Estimators (MKE).
Mass Volume Curves and Anomaly Ranking

This paper aims at formulating the issue of ranking multivariate unlabeled observations depending on their degree of abnormality as an unsupervised statistical learning task. In the 1-d situation, this problem is usually tackled by means of tail estimation techniques: univariate observations are viewed as all the more `abnormal' as they are located far in the tail(s) of the underlying probability distribution. It would be desirable as well to dispose of a scalar valued `scoring' function allowing for comparing the degree of abnormality of multivariate observations. Here we formulate the issue of scoring anomalies as a M-estimation problem by means of a novel functional performance criterion, referred to as the Mass Volume curve (MV curve in short), whose optimal elements are strictly increasing transforms of the density almost everywhere on the support of the density. We first study the statistical estimation of the MV curve of a given scoring function and we provide a strategy to build confidence regions using a smoothed bootstrap approach. Optimization of this functional criterion over the set of piecewise constant scoring functions is next tackled. This boils down to estimating a sequence of empirical minimum volume sets whose levels are chosen adaptively from the data, so as to adjust to the variations of the optimal MV curve, while controling the bias of its approximation by a stepwise curve. Generalization bounds are then established for the difference in sup norm between the MV curve of the empirical scoring function thus obtained and the optimal MV curve.
A Critique of Dyadic Design

Dyadic research designs concern data that comprises interactions among actors. Dyadic approaches unambiguously constitute the most frequent designs employed in the empirical study of international politics, but what do such designs cary with them in terms of theoretical claims and statistical problems? These two issues are closely intertwined. When testing hypotheses empirically, the statistical model must be a careful operationalization of the theory being tested. Given that the theoretical and statistical cannot be separated, we discuss dyadic research designs from these two perspectives; highlighting model misspecification, erroneous assumptions about independence of events, artificial levels of analysis, and the incoherent treatment of multilateral/multiparty events on the theoretical side and difficult-to-escape challenges to valid inference on the statistical side.
A Note on a Sum of Lognormals

This note considers the applicability of Gauss-Hermite quadrature and direct numerical quadrature for computation of moment generating function (mgf) and the derivatives. A preprocessing using the asymptotic technique is employed while computing the characteristic function (chf) using Gauss Hermite quadrature while this is optional for mgf. The mgf of the low and high amplitude regions of a single lognormal variable and the derivatives is examined and attention is drawn to the effect of variance. The problem of inversion of the mgf/chf of a sum of lognormals to obtain the CDF/pdf is considered with special reference to methods related to Post Widder technique, Gaussian quadrature and the Fourier series method. The method based on the complex exponential integral which makes use of the derivative of the cumulant is an alternative. Segmentation of the mgf/chf on the basis of the derivative structure which indicates activity rate is shown to be useful.
Using particle swarm optimization to search for locally $D$-optimal designs for mixed factor experiments with binary response

Identifying optimal designs for generalized linear models with a binary response can be a challenging task, especially when there are both continuous and discrete independent factors in the model. Theoretical results rarely exist for such models, and the handful that do exist come with restrictive assumptions. This paper investigates the use of particle swarm optimization (PSO) to search for locally $D$-optimal designs for generalized linear models with discrete and continuous factors and a binary outcome and demonstrates that PSO can be an effective method. We provide two real applications using PSO to identify designs for experiments with mixed factors: one to redesign an odor removal study and the second to find an optimal design for an electrostatic discharge study. In both cases we show that the $D$-efficiencies of the designs found by PSO are much better than the implemented designs. In addition, we show PSO can efficiently find $D$-optimal designs on a prototype or an irregularly shaped design space, provide insights on the existence of minimally supported optimal designs, and evaluate sensitivity of the $D$-optimal design to mis-specifications in the link function.
A Divide and Conquer Strategy for High Dimensional Bayesian Factor Models

We propose a distributed computing framework, based on a divide and conquer strategy and hierarchical modeling, to accelerate posterior inference for high-dimensional Bayesian factor models. Our approach distributes the task of high-dimensional covariance matrix estimation to multiple cores, solves each subproblem separately via a latent factor model, and then combines these estimates to produce a global estimate of the covariance matrix. Existing divide and conquer methods focus exclusively on dividing the total number of observations $n$ into subsamples while keeping the dimension $p$ fixed. Our approach is novel in this regard: it includes all of the $n$ samples in each subproblem and, instead, splits the dimension $p$ into smaller subsets for each subproblem. The subproblems themselves can be challenging to solve when $p$ is large due to the dependencies across dimensions. To circumvent this issue, we specify a novel hierarchical structure on the latent factors that allows for flexible dependencies across dimensions, while still maintaining computational efficiency. Our approach is readily parallelizable and is shown to have computational efficiency of several orders of magnitude in comparison to fitting a full factor model. We report the performance of our method in synthetic examples and a genomics application.
A Multi-Resolution Approximation for Massive Spatial Datasets

ABSTRACT Automated sensing instruments on satellites and aircraft have enabled the collection of massive amounts of high-resolution observations of spatial fields over large spatial regions. If these datasets can be efficiently exploited, they can provide new insights on a wide variety of issues. However, traditional spatial-statistical techniques such as kriging are not computationally feasible for big datasets. We propose a multi-resolution approximation (M-RA) of Gaussian processes observed at irregular locations in space. The M-RA process is specified as a linear combination of basis functions at multiple levels of spatial resolution, which can capture spatial structure from very fine to very large scales. The basis functions are automatically chosen to approximate a given covariance function, which can be nonstationary. All computations involving the M-RA, including parameter inference and prediction, are highly scalable for massive datasets. Crucially, the inference algorithms can also be parallelized to take full advantage of large distributed-memory computing environments. In comparisons using simulated data and a large satellite dataset, the M-RA outperforms a related state-of-the-art method. Supplementary materials for this article are available online.
How Many Components should be Retained from a Multivariate Time Series PCA

We report on the results of two new approaches to considering how many principal components to retain from an analysis of a multivariate time series. The first is by using a "heat map" based approach. A heat map in this context refers to a series of principal component coefficients created by applying a sliding window to a multivariate time series. Furthermore the heat maps can provide detailed insights into the evolution of the structure of each principal component over time. The second is by examining the change of the angle of the principal component over time within the high-dimensional data space. We provide evidence that both are useful in studying structure and evolution of a multivariate time series.
Causal Inference: A Missing Data Perspective

Inferring causal effects of treatments is a central goal in many disciplines. The potential outcomes framework is a main statistical approach to causal inference, in which a causal effect is defined as a comparison of the potential outcomes of the same units under different treatment conditions. Because for each unit at most one of the potential outcomes is observed and the rest are missing, causal inference is inherently a missing data problem. Indeed, there is a close analogy in the terminology and the inferential framework between causal inference and missing data. Despite the intrinsic connection between the two subjects, statistical analyses of causal inference and missing data also have marked differences in aims, settings and methods. This article provides a systematic review of causal inference from the missing data perspective. Focusing on ignorable treatment assignment mechanisms, we discuss a wide range of causal inference methods that have analogues in missing data analysis, such as imputation, inverse probability weighting and doubly-robust methods. Under each of the three modes of inference--Frequentist, Bayesian, and Fisherian randomization--we present the general structure of inference for both finite-sample and super-population estimands, and illustrate via specific examples. We identify open questions to motivate more research to bridge the two fields.
Selection of variables and decision boundaries for functional data via bi-level selection

Sparsity-inducing penalties are useful tools for variable selection and they are also effective for regression settings where the data are functions. We consider the problem of selecting not only variables but also decision boundaries in logistic regression models for functional data, using the sparse regularization. The functional logistic regression model is estimated by the framework of the penalized likelihood method with the sparse group lasso-type penalty, and then tuning parameters are selected using the model selection criterion. The effectiveness of the proposed method is investigated through real data analysis.
Statistical Analysis of Persistence Intensity Functions

Persistence diagrams are two-dimensional plots that summarize the topological features of functions and are an important part of topological data analysis. A problem that has received much attention is how deal with sets of persistence diagrams. How do we summarize them, average them or cluster them? One approach -- the persistence intensity function -- was introduced informally by Edelsbrunner, Ivanov, and Karasev (2012). Here we provide a modification and formalization of this approach. Using the persistence intensity function, we can visualize multiple diagrams, perform clustering and conduct two-sample tests.
Simultaneous Detection of Signal Regions With Applications in Genome-Wide Association Studies

We consider in this paper detection of signal regions associated with disease outcomes in Genome-Wide Association Studies (GWAS). Gene- or region-based methods have become increasingly popular in GWAS as a complementary approach to traditional individual variant analysis. However, these methods test for the association between an outcome and the genetic variants in a pre-specified region, e.g., a gene. In view of massive intergenic regions in GWAS and substantial interests in identifying signal regions for subsequent fine mapping, we propose a computationally efficient quadratic scan (Q-SCAN) statistic based method to detect the existence and the locations of signal regions by scanning the genome continuously. The proposed method accounts for the correlation (linkage disequilibrium) among genetic variants, and allows for signal regions to have both signal and neutral variants, and signal variants whose effects can be in different directions. We study the asymptotic properties of the proposed Q-SCAN statistics. We derive an asymptotic threshold that controls for the family-wise error rate, and show that under regularity conditions the proposed method consistently selects the true signal regions. We perform simulation studies to evaluate the finite sample performance of the proposed method. Our simulation results show that the proposed procedure outperforms the existing methods, especially when signal regions have signal variants whose effects are in different directions, or are contaminated with neutral variants, or have correlated variants. We apply the proposed method to analyze a lung cancer genome-wide association study to identify the genetic regions that are associated with lung cancer risk.
A goodness-of-fit test for stochastic block models

The stochastic block model is a popular tool for studying community structures in network data. We develop a goodness-of-fit test for the stochastic block model. The test statistic is based on the largest singular value of a residual matrix obtained by subtracting the estimated block mean effect from the adjacency matrix. Asymptotic null distribution is obtained using recent advances in random matrix theory. The test is proved to have full power against alternative models with finer structures. These results naturally lead to a consistent sequential testing estimate of the number of communities.
Directed networks with a noisy bi-degree sequence

Although a lot of approaches are developed to release network data with a differentially privacy guarantee, inference using noisy data in many network models is still unknown or not properly this http URL this paper, we release the bi-degree sequences of directed networks under a general additive noisy mechanism with the Laplace mechanism as a special case and use the $p_0$ model for inferring the degree parameters. We show that the estimator of the parameter without the denoised process is asymptotically consistent and normally distributed. This is contrast sharply with known results that valid inference such as the existence and consistency of the estimator needs the denoised process. Along the way, a new phenomenon is revealed in which an additional variance factor appears in the asymptotic variance of the estimator when the noise becomes large. Further, we propose an efficient algorithm for finding the closet point lying in the set of all graphical bi-degree sequences under the global $L_1$ optimization problem. The algorithm simultaneously produces a synthetic directed graph. Numerical studies demonstrate our theoretical findings.
High-Dimension, Low Sample Size Asymptotics of Canonical Correlation Analysis

An asymptotic behavior of canonical correlation analysis is studied when dimension d grows and the sample size n is fxed. In particular, we are interested in the conditions for which CCA works or fails in the HDLSS situation. This technical report investigates those conditions in a rather simplified setting where there exists one pair of directions in two sets of random variables with non-zero correlation between two sets of scores on them. Proofs and an extensive simulation study supports the findings.
Model-based Dashboards for Customer Analytics

Automating the customer analytics process is crucial for companies that manage distinct customer bases. In such data-rich and dynamic environments, visualization plays a key role in understanding events of interest. These ideas have led to the popularity of analytics dashboards, yet academic research has paid scant attention to these managerial needs. We develop a probabilistic, nonparametric framework for understanding and predicting individual-level spending using Gaussian process priors over latent functions that describe customer spending along calendar time, interpurchase time, and customer lifetime dimensions. These curves form a dashboard that provides a visual model-based representation of purchasing dynamics that is easily comprehensible. The model flexibly and automatically captures the form and duration of the impact of events that influence spend propensity, even when such events are unknown a-priori. We illustrate the use of our Gaussian Process Propensity Model (GPPM) on data from two popular mobile games. We show that the GPPM generalizes hazard and buy-till-you-die models by incorporating calendar time dynamics while simultaneously accounting for recency and lifetime effects. It therefore provides insights about spending propensity beyond those available from these models. Finally, we show that the GPPM outperforms these benchmarks both in fitting and forecasting real and simulated spend data.
Efficient Estimation of Partially Linear Models for Spatial Data over Complex Domains

In this paper, we study the estimation of partially linear models for spatial data distributed over complex domains. We use bivariate splines over triangulations to represent the nonparametric component on an irregular two-dimensional domain. The proposed method is formulated as a constrained minimization problem which does not require constructing finite elements or locally supported basis functions. Thus, it allows an easier implementation of piecewise polynomial representations of various degrees and various smoothness over an arbitrary triangulation. Moreover, the constrained minimization problem is converted into an unconstrained minimization via a QR decomposition of the smoothness constraints, which leads to a penalized least squares method to estimate the model. The estimators of the parameters are proved to be asymptotically normal under some regularity conditions. The estimator of the bivariate function is consistent, and its rate of convergence is also established. The proposed method enables us to construct confidence intervals and permits inference for the parameters. The performance of the estimators is evaluated by two simulation examples and by a real data analysis.
Full-Pulse Tomographic Reconstruction with Deep Neural Networks

Abstract Plasma tomography consists of reconstructing a two-dimensional radiation profile of a poloidal cross section of a fusion device based on line-integrated measurements along several lines of sight. The reconstruction process is computationally intensive, and in practice, only a few reconstructions are usually computed per pulse. In this work, we trained a deep neural network based on a large collection of sample tomograms that have been produced at JET over several years. Once trained, the network is able to reproduce those results with high accuracy. More importantly, it can compute all the tomographic reconstructions for a given pulse in just a few seconds. This makes it possible to visualize several phenomena—such as plasma heating, disruptions, and impurity transport—over the course of the entire pulse.
A note on the posterior inference for the Yule–Simon distribution

ABSTRACT The Yule–Simon distribution has been out of the radar of the Bayesian community, so far. In this note, we propose an explicit Gibbs sampling scheme when a Gamma prior is chosen for the shape parameter. The performance of the algorithm is illustrated with simulation studies, including count data regression, and a real data application to text analysis. We compare our proposal to the frequentist counterparts showing better performance of our algorithm when a small sample size is considered.
A Geometric Approach for Bounding Average Stopping Time

We propose a geometric approach for bounding average stopping times defined in terms of sums of i.i. d. random variables. We consider stopping times in the hyperspace of sample number and sample sum. Our techniques relies on exploring geometric properties of continuity or stopping regions. Especially, we make use of the concepts of convex hull, convex sets and supporting hyperplane. Explicit formulae and efficiently computable bounds are obtained for average stopping times. Our techniques can be applied to bound average stopping times involving random vectors, nonlinear stopping boundary, and constraints of sample number. Moreover, we establish a stochastic characteristic of convex sets and generalize Jensen's inequality, Wald's equations and Lorden's inequality, which are useful for investigating average stopping times.
Robust mixture modelling using sub-Gaussian stable distribution

Heavy-tailed distributions are widely used in robust mixture modelling due to possessing thick tails. As a computationally tractable subclass of the stable distributions, sub-Gaussian $\alpha$-stable distribution received much interest in the literature. Here, we introduce a type of expectation maximization algorithm that estimates parameters of a mixture of sub-Gaussian stable distributions. A comparative study, in the presence of some well-known mixture models, is performed to show the robustness and performance of the mixture of sub-Gaussian $\alpha$-stable distributions for modelling, simulated, synthetic, and real data.