Miracle-dz/UniFAR-FTUs huggingface dataset:
Copula Correlation: An Equitable Dependence Measure and Extension of Pearson's Correlation

In Science, Reshef et al. (2011) proposed the concept of equitability for measures of dependence between two random variables. To this end, they proposed a novel measure, the maximal information coefficient (MIC). Recently a PNAS paper (Kinney and Atwal, 2014) gave a mathematical definition for equitability. They proved that MIC in fact is not equitable, while a fundamental information theoretic measure, the mutual information (MI), is self-equitable. In this paper, we show that MI also does not correctly reflect the proportion of deterministic signals hidden in noisy data. We propose a new equitability definition based on this scenario. The copula correlation (Ccor), based on the L1-distance of copula density, is shown to be equitable under both definitions. We also prove theoretically that Ccor is much easier to estimate than MI. Numerical studies illustrate the properties of the measures.
Minimum Scoring Rule Inference

Proper scoring rules are devices for encouraging honest assessment of probability distributions. Just like log‐likelihood, which is a special case, a proper scoring rule can be applied to supply an unbiased estimating equation for any statistical model, and the theory of such equations can be applied to understand the properties of the associated estimator. In this paper, we discuss some novel applications of scoring rules to parametric inference. In particular, we focus on scoring rule test statistics, and we propose suitable adjustments to allow reference to the usual asymptotic chi‐squared distribution. We further explore robustness and interval estimation properties, by both theory and simulations.
MATHEMATICAL ENGINEERING TECHNICAL REPORTS Determinantal Point Process Priors for Bayesian Variable Selection in Linear Regression

We propose discrete determinantal point processes (DPPs) for priors on the model parameter in Bayesian variable selection. By our variable selection method, collinear predictors are less likely to be selected simultaneously because of the repulsion property of discrete DPPs. Three types of DPP priors are proposed. We show the efficiency of the proposed priors through numerical experiments and applications to collinear datasets.
Penalty, Shrinkage, and Preliminary Test Estimators under Full Model Hypothesis

This paper considers a multiple regression model and compares, under full model hypothesis, analytically as well as by simulation, the performance characteristics of some popular penalty estimators such as ridge regression, LASSO, adaptive LASSO, SCAD, and elastic net versus Least Squares Estimator, restricted estimator, preliminary test estimator, and Stein-type estimators when the dimension of the parameter space is smaller than the sample space dimension. We find that RR uniformly dominates LSE, RE, PTE, SE and PRSE while LASSO, aLASSO, SCAD, and EN uniformly dominates LSE only. Further, it is observed that neither penalty estimators nor Stein-type estimator dominate one another.
A new distribution for robust least squares

A new distribution is introduced, which we call the twin-t distribution. This distribution is heavy-tailed like the t distribution, but closer to normality in the central part of the curve. Its properties are described, e.g. the pdf, the distribution function, moments, and random number generation. This distribution could have many applications, but here we focus on its use as an aid to robustness. We give examples of its application in robust regression and in curve fitting. Extensions such as skew and multivariate twin-t distributions, and a twin of
Data Assimilation: A Mathematical Introduction

This book provides a systematic treatment of the mathematical underpinnings of work in data assimilation, covering both theoretical and computational approaches. Specifically the authors develop a unified mathematical framework in which a Bayesian formulation of the problem provides the bedrock for the derivation, development and analysis of algorithms; the many examples used in the text, together with the algorithms which are introduced and discussed, are all illustrated by the MATLAB software detailed in the book and made freely available online. The book is organized into nine chapters: the first contains a brief introduction to the mathematical tools around which the material is organized; the next four are concerned with discrete time dynamical systems and discrete time data; the last four are concerned with continuous time dynamical systems and continuous time data and are organized analogously to the corresponding discrete time chapters. This book is aimed at mathematical researchers interested in a systematic development of this interdisciplinary field, and at researchers from the geosciences, and a variety of other scientific fields, who use tools from data assimilation to combine data with time-dependent models. The numerous examples and illustrations make understanding of the theoretical underpinnings of data assimilation accessible. Furthermore, the examples, exercises and MATLAB software, make the book suitable for students in applied mathematics, either through a lecture course, or through self-study.
Posterior consistency for Bayesian inverse problems through stability and regression results

In the Bayesian approach, the a priori knowledge about the input of a mathematical model is described via a probability measure. The joint distribution of the unknown input and the data is then conditioned, using Bayes’ formula, giving rise to the posterior distribution on the unknown input. In this setting we prove posterior consistency for nonlinear inverse problems: a sequence of data is considered, with diminishing fluctuations around a single truth and it is then of interest to show that the resulting sequence of posterior measures arising from this sequence of data concentrates around the truth used to generate the data. Posterior consistency justifies the use of the Bayesian approach very much in the same way as error bounds and convergence results for regularization techniques do. As a guiding example, we consider the inverse problem of reconstructing the diffusion coefficient from noisy observations of the solution to an elliptic PDE in divergence form. This problem is approached by splitting the forward operator into the underlying continuum model and a simpler observation operator based on the output of the model. In general, these splittings allow us to conclude posterior consistency provided a deterministic stability result for the underlying inverse problem and a posterior consistency result for the Bayesian regression problem with the push-forward prior. Moreover, we prove posterior consistency for the Bayesian regression problem based on the regularity, the tail behaviour and the small ball probabilities of the prior.
Exploring Hoover and Perez's experimental designs using global sensitivity analysis

This paper investigates variable-selection procedures in regression that make use of global sensitivity analysis. The approach is combined with existing algorithms and it is applied to the time series regression designs proposed by Hoover and Perez. A comparison of an algorithm employing global sensitivity analysis and the (optimized) algorithm of Hoover and Perez shows that the former significantly improves the recovery rates of original specifications.
A space-time covariance function for spatio-temporal random processes and spatio-temporal prediction (kriging)

We consider a stationary spatio-temporal random process and assume that we have a sample. By defining a sequence of discrete Fourier transforms at canonical frequencies at each location, and using these complex valued random varables as observed sample, we obtain expressions for the spatio-temporal covariance functions and the spectral density functions of the spatio-temporal random processes. These spectra correspond to non separable class of random processes. The spatio-temporal covariance functions, obtained here are functions of the spatial distance and the temporal frequency and are similar to Matern class. These are in terms of modified Bessel functions of the second kind. and the parameters are in terms of the second order spectral density functions of the random proces and the spatial distances. We consider the estimation of the parameters of the covariance function and also briefly mention their asymptotic properties. The estimation of the entire data at a known location, and also the estimation of a value given the above sample is also considered. The predictors are obtained using the vectors of Discrete Fourier Transforms. We also describe a statistical test for testing the independence of the m spatial time series (testing for spatial independence) using the Finite Fourier Transforms and it is based on the likelihood ratio test of complex valued random variables The methods are illustrated with real data. Keywords: Discrete Fourier Transforms, Covariance functions, Spectral density functions, Space-Time Processses, Prediction(kriging) Laplacian operators, Frequency Variogram, Tests for independence, Whittle likelihood.
Properties and Applications of some Distributions derived from Frullani's integral

Frullani’s integral dates from 1821, but a probabilistic interpretation of it has never been made. In this paper, Frullani’s integral formula is shown to result from mixing a lifetime distribution by allowing the logarithm of the scale factor to be uniformly distributed over a finite range. This gives a class of long-tailed distributions related to slash distributions, where the pdf is simply expressed in terms of the survival function of the ‘parent’ distribution. The resulting survival distributions have all moments finite, and can exhibit the bimodal hazard functions sometimes seen in practice. A distribution of this type analogous to the t-distribution is derived, the corresponding multivariate distributions are given, and two skewed versions of this distribution are derived. The use of the mixed distributions for inference is exemplified by fitting them to several datasets. It is expected that there will be many applications, in health, reliability, telecommunications, finance, etc.
From GWAS to transcriptomics in prospective cancer design - new statistical challenges

Background. With the increasing interest in post-GWAS research which represents a transition from genome-wide association discovery to analysis of functional mechanisms, attention has been lately focused on the potential of including various biological material in epidemiological studies. In particular, exploration of the carcinogenic process through transcriptional analysis at the epidemiological level opens up new horizons in functional analysis and causal inference, and requires a new design together with adequate analysis procedures. Results. In this article, we present the post-genome design implemented in the NOWAC cohort as an example of a prospective nested case-control study built for transcriptomics use, and discuss analytical strategies to explore the changes occurring in transcriptomics during the carcinogenic process in association with questionnaire information. We emphasize the inadequacy of survival analysis models usually considered in GWAS for post-genome design, and propose instead to parameterize the gene trajectories during the carcinogenic process. Conclusions. This novel approach, in which transcriptomics are considered as potential intermediate biomarkers of cancer and exposures, offers a flexible framework which can include various biological assumptions.
Speckle Patterns and 2-Dimensional Spatial Models

The result of 2-dimensional Gaussian lattice fit to a speckle intensity pattern based on a linear model that includes nearest-neighbor interactions is presented. We also include a Monte Carlo simulation of the same spatial speckle pattern that takes the nearest-neighbor interactions into account. These nearest-neighbor interactions lead to a spatial variance structure on the lattice. The resulting spatial pattern fluctuates in value from point to point in a manner characteristic of a stationary stochastic process. The value at a lattice point in the simulation is interpreted as an inten-sity level and the difference in values in neighboring cells produces a fluctuating intensity pattern on the lattice. Changing the size of the mesh changes the relative size of the speckles. Increasing the mesh size tends to average out the intensity in the direction of the mean of the stationary process.
The Generalized Mean Information Coefficient

Reshef & Reshef recently published a paper in which they present a method called the Maximal Information Coefficient (MIC) that can detect all forms of statistical dependence between pairs of variables as sample size goes to infinity. While this method has been praised by some, it has also been criticized for its lack of power in finite samples. We seek to modify MIC so that it has higher power in detecting associations for limited sample sizes. Here we present the Generalized Mean Information Coefficient (GMIC), a generalization of MIC which incorporates a tuning parameter that can be used to modify the complexity of the association favored by the measure. We define GMIC and prove it maintains several key asymptotic properties of MIC. Its increased power over MIC is demonstrated using a simulation of eight different functional relationships at sixty different noise levels. The results are compared to the Pearson correlation, distance correlation, and MIC. Simulation results suggest that while generally GMIC has slightly lower power than the distance correlation measure, it achieves higher power than MIC for many forms of underlying association. For some functional relationships, GMIC surpasses all other statistics calculated. Preliminary results suggest choosing a moderate value of the tuning parameter for GMIC will yield a test that is robust across underlying relationships. GMIC is a promising new method that mitigates the power issues suffered by MIC, at the possible expense of equitability. Nonetheless, distance correlation was in our simulations more powerful for many forms of underlying relationships. At a minimum, this work motivates further consideration of maximal information-based nonparametric exploration (MINE) methods as statistical tests of independence.
Covariance pattern mixture models for the analysis of multivariate heterogeneous longitudinal data

We propose a novel approach for modeling multivariate longitudinal data in the presence of unobserved heterogeneity for the analysis of the Health and Retirement Study (HRS) data. Our proposal can be cast within the framework of linear mixed models with discrete individual random intercepts; however, differently from the standard formulation, the proposed Covariance Pattern Mixture Model (CPMM) does not require the usual local independence assumption. The model is thus able to simultaneously model the heterogeneity, the association among the responses and the temporal dependence structure. We focus on the investigation of temporal patterns related to the cognitive functioning in retired American respondents. In particular, we aim to understand whether it can be affected by some individual socio-economical characteristics and whether it is possible to identify some homogenous groups of respondents that share a similar cognitive profile. An accurate description of the detected groups allows government policy interventions to be opportunely addressed. Results identify three homogenous clusters of individuals with specific cognitive functioning, consistent with the class conditional distribution of the covariates. The flexibility of CPMM allows for a different contribution of each regressor on the responses according to group membership. In so doing, the identified groups receive a global and accurate phenomenological characterization.
Convergence rates of empirical block length selectors for block bootstrap

We investigate the accuracy of two general non-parametric methods for estimating optimal block lengths for block bootstraps with time series - the first proposed in the seminal paper of Hall, Horowitz and Jing (Biometrika 82 (1995) 561-574) and the second from Lahiri et al. (Stat. Methodol. 4 (2007) 292-321). The relative performances of these general methods have been unknown and, to provide a comparison, we focus on rates of convergence for these block length selectors for the moving block bootstrap (MBB) with variance estimation problems under the smooth function model. It is shown that, with suitable choice of tuning parameters, the optimal convergence rate of the first method is $O_p(n^{-1/6})$ where $n$ denotes the sample size. The optimal convergence rate of the second method, with the same number of tuning parameters, is shown to be $O_p(n^{-2/7})$, suggesting that the second method may generally have better large-sample properties for block selection in block bootstrap applications beyond variance estimation. We also compare the two general methods with other plug-in methods specifically designed for block selection in variance estimation, where the best possible convergence rate is shown to be $O_p(n^{-1/3})$ and achieved by a method from Politis and White (Econometric Rev. 23 (2004) 53-70).
Identifiability of directed Gaussian graphical models with one latent source

We study parameter identifiability of directed Gaussian graphical models with one latent variable. In the scenario we consider, the latent variable is a confounder that forms a source node of the graph and is a parent to all other nodes, which correspond to the observed variables. We give a graphical condition that is sufficient for the Jacobian matrix of the parametrization map to be full rank, which entails that the parametrization is generically finite-to-one, a fact that is sometimes also referred to as local identifiability. We also derive a graphical condition that is necessary for such identifiability. Finally, we give a condition under which generic parameter identifiability can be determined from identifiability of a model associated with a subgraph. The power of these criteria is assessed via an exhaustive algebraic computational study on models with 4, 5, and 6 observable variables.
An Algorithm for Nonlinear, Nonparametric Model Choice and Prediction

We introduce an algorithm which, in the context of nonlinear regression on vector-valued explanatory variables, aims to choose those combinations of vector components that provide best prediction. The algorithm is constructed specifically so that it devotes attention to components that might be of relatively little predictive value by themselves, and so might be ignored by more conventional methodology for model choice, but which, in combination with other difficult-to-find components, can be particularly beneficial for prediction. The design of the algorithm is also motivated by a desire to choose vector components that become redundant once appropriate combinations of other, more relevant components are selected. Our theoretical arguments show these goals are met in the sense that, with probability converging to 1 as sample size increases, the algorithm correctly determines a small, fixed number of variables on which the regression mean, g say, depends, even if dimension diverges to infinity much faster than n. Moreover, the estimated regression mean based on those variables approximates g with an error that, to first order, equals the error which would arise if we were told in advance the correct variables. In this sense, the estimator achieves oracle performance. Our numerical work indicates that the algorithm is suitable for very high dimensional problems, where it keeps computational labor in check by using a novel sequential argument, and also for more conventional prediction problems, where dimension is relatively low.
A new permutation test statistic for complete block designs

We introduce a nonparametric test statistic for the permutation test in complete block designs. We find the region in which the statistic exists and consider particularly its properties on the boundary of the region. Further, we prove that saddlepoint approximations for tail probabilities can be obtained inside the interior of this region. Finally, numerical examples are given showing that both accuracy and power of the new statistic improves on these properties of the classical $F$-statistic under some non-Gaussian models and equals them for the Gaussian case.
Regularization of $\ell_1$ minimization for dealing with outliers and noise in Statistics and Signal Recovery

We study the robustness properties of $\ell_1$ norm minimization for the classical linear regression problem with a given design matrix and contamination restricted to the dependent variable. We perform a fine error analysis of the $\ell_1$ estimator for measurements errors consisting of outliers coupled with noise. We introduce a new estimation technique resulting from a regularization of $\ell_1$ minimization by inf-convolution with the $\ell_2$ norm. Concerning robustness to large outliers, the proposed estimator keeps the breakdown point of the $\ell_1$ estimator, and reduces to least squares when there are not outliers. We present a globally convergent forward-backward algorithm for computing our estimator and some numerical experiments confirming its theoretical properties.
Graphical Fermat's Principle and Triangle-Free Graph Estimation

We consider the problem of estimating undirected triangle-free graphs of high dimensional distributions. Triangle-free graphs form a rich graph family which allows arbitrary loopy structures but 3-cliques. For inferential tractability, we propose a graphical Fermat's principle to regularize the distribution family. Such principle enforces the existence of a distribution-dependent pseudo-metric such that any two nodes have a smaller distance than that of two other nodes who have a geodesic path include these two nodes. Guided by this principle, we show that a greedy strategy is able to recover the true graph. The resulting algorithm only requires a pairwise distance matrix as input and is computationally even more efficient than calculating the minimum spanning tree. We consider graph estimation problems under different settings, including discrete and nonparametric distribution families. Thorough numerical results are provided to illustrate the usefulness of the proposed method.