Miracle-dz/UniFAR-FTUs huggingface dataset:
A Multinomial Model for Comorbidity in England of Longstanding CVD, Diabetes, and Obesity

From a public health perspective, previous research on comorbidity tends to have focused on identifying the most prevalent groupings of illnesses that demonstrate comorbidity, particularly among the elderly population, already in receipt of care. In contrast, little attention has been paid to possible socio-economic factors associated with increased rates of comorbidity or to the possibility of wider unrevealed need. Given the known relationship between CVD, diabetes and obesity and the strong socio-economic gradients in risk factors for each of the three diseases as single morbidities, this paper uses the Health Survey for England to examine the demographic and socio-economic determinants of each of the seven disease combinations in the English population. Using a multinomial logistic model, this research finds that gender is a significant predictor for all seven disease combinations. However, gender was not as influential as individual age or socio-economic profile. With regard to ethnicity, the black population presents a high obesity, diabetes and diabetes-related comorbidity risk, whilst the Asian population presents a high risk for diabetes and diabetes-related comorbidity but a low risk for obesity and comorbidity. Across the seven disease combinations, risk was lowest for those individuals with a high income (4 out of 7), in-work (4 out of 7), home owning (3 out of 7) and degree educated (3 out of 7). Finally, smokers have a lower risk rate of obesity (and related) than ex-smokers relative to individuals that never smoked (in all cases controlling for all other factors). The important influence of socioeconomic factors has implications for the spatial demand for services and the policy solutions adopted to tackle the increasing prevalence of comorbidity.
Variance estimation for Brier Score decomposition

The Brier Score is a widely used criterion to assess the quality of probabilistic predictions of binary events. The expectation value of the Brier Score can be decomposed into the sum of three components called reliability, resolution and uncertainty, which characterize different forecast attributes. Given a dataset of forecast probabilities and corresponding binary verifications, these three components can be estimated empirically. Here, propagation of uncertainty is used to derive expressions that approximate the sampling variances of the estimated components. Variance estimates are provided for both the traditional estimators, as well as for refined estimators that include a bias correction. Applications of the derived variance estimates to artificial data illustrate their validity and application to a meteorological prediction problem illustrates a possible usage case. The observed increase of variance of the bias‐corrected estimators is discussed.
Model-based clustering of multivariate binary data with dimension reduction

Clustering methods with dimension reduction have been receiving considerable wide interest in statistics lately and a lot of methods to simultaneously perform clustering and dimension reduction have been proposed. This work presents a novel procedure for simultaneously determining the optimal cluster structure for multivariate binary data and the subspace to represent that cluster structure. The method is based on a finite mixture model of multivariate Bernoulli distributions, and each component is assumed to have a low-dimensional representation of the cluster structure. This method can be considered an extension of the traditional latent class analysis model. Sparsity is introduced to the loading values, which produces the low-dimensional subspace, for enhanced interpretability and more stable extraction of the subspace. An EM-based algorithm is developed to efficiently solve the proposed optimization problem. We demonstrate the effectiveness of the proposed method by applying it to a simulation study and real datasets.
The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.
Wishart Generator Distribution

The Wishart distribution and its generalizations are among the most prominent probability distributions in multivariate statistical analysis, arising naturally in applied research and as a basis for theoretical models. In this paper, we generalize the Wishart distribution utilizing a different approach that leads to the Wishart generator distribution with the Wishart distribution as a special case. It is not restricted, however some special cases are exhibited. Important statistical characteristics of the Wishart generator distribution are derived from the matrix theory viewpoint. Estimation is also touched upon as a guide for further research from the classical approach as well as from the Bayesian paradigm. The paper is concluded by giving applications of two special cases of this distribution in calculating the product of beta functions and astronomy.
Non-subjective power analysis to detect G*E interactions in Genome-Wide Association Studies in presence of confounding factor

It is generally acknowledged that most complex diseases are affected in part by interactions between genes and genes and/or between genes and environmental factors. Taking into account environmental exposures and their interactions with genetic factors in genome-wide association studies (GWAS) can help to identify high-risk subgroups in the population and provide a better understanding of the disease. For this reason, many methods have been developed to detect gene-environment (G*E) interactions. Despite this, few loci that interact with environmental exposures have been identified so far. Indeed, the modest effect of G*E interactions as well as confounding factors entail low statistical power to detect such interactions. In this work, we provide a simulated dataset in order to study methods for detecting G*E interactions in GWAS in presence of confounding factor and population structure. Our work applies a recently introduced non-subjective method for H1 simulations called waffect and exploits the publicly available HapMap project to build a datasets with real genotypes and population structures. We use this dataset to study the impact of confounding factors and compare the relative performance of popular methods such as PLINK, random forests and linear mixed models to detect G*E interactions. Presence of confounding factor is an obstacle to detect G*E interactions in GWAS and the approaches considered in our power study all have insufficient power to detect the strong simulated interaction. Our simulated dataset could help to develop new methods which account for confounding factors through latent exposures in order to improve power.
FFT-Based Fast Computation of Multivariate Kernel Estimators with Unconstrained Bandwidth Matrices

The problem of fast computation of multivariate kernel density estimation (KDE) is still an open research problem. In our view, the existing solutions do not resolve this matter in a satisfactory way. One of the most elegant and efficient approach utilizes the fast Fourier transform. Unfortunately, the existing FFT-based solution suffers from a serious limitation, as it can accurately operate only with the constrained (i.e., diagonal) multivariate bandwidth matrices. In this paper we describe the problem and give a satisfactory solution. The proposed solution may be successfully used also in other research problems, for example for the fast computation of the optimal bandwidth for KDE.
An Information-Theoretic Alternative to the Cronbach's Alpha Coefficient of Item Reliability

We propose an information-theoretic alternative to the popular Cronbach alpha coefficient of reliability. Particularly suitable for contexts in which instruments are scored on a strictly nonnumeric scale, our proposed index is based on functions of the entropy of the distributions of defined on the sample space of responses. Our reliability index tracks the Cronbach alpha coefficient uniformly while offering several other advantages discussed in great details in this paper.
Thresholds in choice behaviour and the size of travel time savings

Travel time savings are usually the most substantial economic benefit of transport infrastructure projects. However, questions surround whether small time savings are as valuable per unit as larger savings. Thresholds in individual choice behaviour are one reason cited for a discounted unit value for small time savings. We demonstrate different approaches for modelling these thresholds using synthetic and stated choice data. We show that the consideration of thresholds is important, even if the discounted unit value for small travel time savings is rejected for transport project appraisal. If an existing threshold is ignored in model estimation, the value of travel time savings will be biased. The presented procedure might also be useful to model thresholds in other contexts of choice behaviour.
Inferring gene-gene interactions and functional modules using sparse canonical correlation analysis

Networks pervade many disciplines of science for analyzing complex systems with interacting components. In particular, this concept is commonly used to model interactions between genes and identify closely associated genes forming functional modules. In this paper, we focus on gene group interactions and infer these interactions using appropriate partial correlations between genes, that is, the conditional dependencies between genes after removing the influences of a set of other functionally related genes. We introduce a new method for estimating group interactions using sparse canonical correlation analysis (SCCA) coupled with repeated random partition and subsampling of the gene expression data set. By considering different subsets of genes and ways of grouping them, our interaction measure can be viewed as an aggregated estimate of partial correlations of different orders. Our approach is unique in evaluating conditional dependencies when the correct dependent sets are unknown or only partially known. As a result, a gene network can be constructed using the interaction measures as edge weights and gene functional groups can be inferred as tightly connected communities from the network. Comparisons with several popular approaches using simulated and real data show our procedure improves both the statistical significance and biological interpretability of the results. In addition to achieving considerably lower false positive rates, our procedure shows better performance in detecting important biological pathways.
DepthProc: An R Package for Robust Exploration of Multidimensional Economic Phenomena

Data depth concept oers a variety of powerful and user friendly tools for robust exploration and inference for multivariate socio-economic phenomena. The oered techniques may be successfully used in cases of lack of our knowledge on parametric models generating data due to their nonparametric nature. This paper presents the R package DepthProc, which is available under GPL-2 licence on CRAN and R-forge servers for Windows, Linux and OS X platform. The package consist of among others successful implementations of several data depth techniques involving multivariate quantile-quantile plots, multivariate scatter estimators, multivariate Wilcoxon tests, robust regressions. In order to show the package capabilities, real dataset concerning United Nations Fourth Millennium Goal is used.
Nonparametric statistical inference for the context tree of a stationary ergodic process

We consider the problem of estimating the context tree of a stationary ergodic process with finite alphabet without imposing additional conditions on the process. As a starting point we introduce a Hamming metric in the space of irreducible context trees and we use the properties of the weak topology in the space of ergodic stationary processes to prove that if the Hamming metric is unbounded, there exist no consistent estimators for the context tree. Even in the bounded case we show that there exist no two-sided confidence bounds. However we prove that one-sided inference is possible in this general setting and we construct a consistent estimator that is a lower bound for the context tree of the process with an explicit formula for the coverage probability. We develop an efficient algorithm to compute the lower bound and we apply the method to test a linguistic hypothesis about the context tree of codified written texts in European Portuguese.
On discrete approximations of stable distributions

In some fields of applications of stable distributions, especially in economics, it appears, that data have distributions similar to stable in a large region, but do not have such heavy tails. Our aim in this note is to propose several methods of approximation of stable distributions by some discrete distributions, which may have different tail behavior. In a sense the introduced distributions form an alternative to tempered stable distributions that combine Gaussian and stable behavior.
Nonparametric inference in a stereological model with oriented cylinders applied to dual phase steel

Oriented circular cylinders in an opaque medium are used to represent certain microstructural objects in steel. The opaque medium is sliced parallel to the cylinder axes of symmetry and the cut-plane contains the observable rectangular profiles of the cylinders. A one-to-one relation between the joint density of the squared radius and height of the 3D cylinders and the joint density of the squared half-width and height of the observable 2D rectangles is established. We propose a nonparametric estimation procedure to estimate the distributions and expectations of various quantities of interest, such as the cylinder radius, height, aspect ratio, surface area and volume from the observed 2D rectangle widths and heights. Also, the covariance between the radius and height of a cylinder is estimated. The asymptotic behavior of these estimators is established to yield point-wise confidence intervals for the expectations and point-wise confidence sets for the distributions of the quantities of interest. Many of these quantities can be linked to the mechanical properties of the material, and are, therefore, useful for industry. We illustrate the mathematical model and estimation procedures using a banded microstructure for which nearly 90 \textmu m of depth have been observed via serial sectioning.
Estimation in a change-point non linear quantile model

ABSTRACT This paper considers a non linear quantile model with change-points. The quantile estimation method, which as a particular case includes median model, is more robust with respect to other traditional methods when model errors contain outliers. Under relatively weak assumptions, the convergence rate and asymptotic distribution of change-point and of regression parameter estimators are obtained. Numerical study by Monte Carlo simulations shows the performance of the proposed method for non linear model with change-points.
denoiseR: A Package for Low Rank Matrix Estimation

We present the R package denoiseR dedicated to low-rank matrix estimation. First, we briefly review existing methods including singular value shrinkage and a parametric bootstrap approach. Then, we discuss how to extend the methods to missing values and suggest a general iterative imputation algorithm. It includes an extension of the Stein Unbiased Risk Estimate to missing values for selecting tuning parameters. Finally, we compare and apply the methods with many experiments.
Uncertainty in Test Score Data and Classically Defined Reliability of Tests and Test Batteries, using a New Method for Test Dichotomisation

As with all measurements, the measurement of examinee ability, in terms of scores that the examinee obtains in a test, is also error-ridden. The quantification of such error or uncertainty in the test score data--or rather the complementary test reliability--is pursued within the paradigm of Classical Test Theory in a variety of ways, with no existing method of finding reliability, isomorphic to the theoretical definition that parametrises reliability as the ratio of the true score variance and observed (i.e. error-ridden) score variance. Thus, multiple reliability coefficients for the same test have been advanced. This paper describes a much needed method of obtaining reliability of a test as per its theoretical definition, via a single administration of the test, by using a new fast method of splitting of a given test into parallel halves, achieving near-coincident empirical distributions of the two halves. The method has the desirable property of achieving splitting on the basis of difficulty of the questions (or items) that constitute the test, thus allowing for fast computation of reliability even for very large test data sets, i.e. test data obtained by a very large examinee sample. An interval estimate for the true score is offered, given an examinee score, subsequent to the determination of the test reliability. This method of finding test reliability as per the classical definition can be extended to find reliability of a set or battery of tests; a method for determination of the weights implemented in the computation of the weighted battery score is discussed. We perform empirical illustration of our method on real and simulated tests, and on a real test battery comprising two constituent tests.
Toward Computerized Efficient Estimation in Infinite-Dimensional Models

ABSTRACT Despite the risk of misspecification they are tied to, parametric models continue to be used in statistical practice because they are simple and convenient to use. In particular, efficient estimation procedures in parametric models are easy to describe and implement. Unfortunately, the same cannot be said of semiparametric and nonparametric models. While the latter often reflect the level of available scientific knowledge more appropriately, performing efficient inference in these models is generally challenging. The efficient influence function is a key analytic object from which the construction of asymptotically efficient estimators can potentially be streamlined. However, the theoretical derivation of the efficient influence function requires specialized knowledge and is often a difficult task, even for experts. In this article, we present a novel representation of the efficient influence function and describe a numerical procedure for approximating its evaluation. The approach generalizes the nonparametric procedures of Frangakis et al. and Luedtke, Carone, and van der Laan to arbitrary models. We present theoretical results to support our proposal and illustrate the method in the context of several semiparametric problems. The proposed approach is an important step toward automating efficient estimation in general statistical models, thereby rendering more accessible the use of realistic models in statistical analyses. Supplementary materials for this article are available online.
Optimal Gaussian approximations to the posterior for log-linear models with Diaconis-Ylvisaker priors

In contingency table analysis, sparse data is frequently encountered for even modest numbers of variables, resulting in non-existence of maximum likelihood estimates. A common solution is to obtain regularized estimates of the parameters of a log-linear model. Bayesian methods provide a coherent approach to regularization, but are often computationally intensive. Conjugate priors ease computational demands, but the conjugate Diaconis-Ylvisaker priors for the parameters of log-linear models do not give rise to closed form credible regions, complicating posterior inference. Here we derive the optimal Gaussian approximation to the posterior for log-linear models with Diaconis-Ylvisaker priors, and provide convergence rate and finite-sample bounds for the Kullback-Leibler divergence between the exact posterior and the optimal Gaussian approximation. We demonstrate empirically in simulations and a real data application that the approximation is highly accurate, even in relatively small samples. The proposed approximation provides a computationally scalable and principled approach to regularized estimation and approximate Bayesian inference for log-linear models.
Statistical Modeling for Spatio-Temporal Degradation Data

This paper investigates the modeling of an important class of degradation data, which are collected from a spatial domain over time; for example, the surface quality degradation. Like many existing time-dependent stochastic degradation models, a special random field is constructed for modeling the spatio-temporal degradation process. In particular, we express the degradation at any spatial location and time as an additive superposition of two stochastic components: a dynamic spatial degradation generation process, and a spatio-temporal degradation propagation process. Some unique challenges are addressed, including the spatial heterogeneity of the degradation process, the spatial propagation of degradation to neighboring areas, the anisotropic and space-time non-separable covariance structure often associated with a complex spatio-temporal degradation process, and the computational issue related to parameter estimation. When the spatial dependence is ignored, we show that the proposed spatio-temporal degradation model incorporates some existing pure time-dependent degradation processes as its special cases. We also show the connection, under special conditions, between the proposed model and general physical degradation processes which are often defined by stochastic partial differential equations. A numerical example is presented to illustrate the modeling approach and model validation.
Regression Analysis for Microbiome Compositional Data

One important problem in microbiome analysis is to identify the bacterial taxa that are associated with a response, where the microbiome data are summarized as the composition of the bacterial taxa at different taxonomic levels. This paper considers regression analysis with such compositional data as covariates. In order to satisfy the subcompositional coherence of the results, linear models with a set of linear constraints on the regression coefficients are introduced. Such models allow regression analysis for subcompositions and include the log-contrast model for compositional covariates as a special case. A penalized estimation procedure for estimating the regression coefficients and for selecting variables under the linear constraints is developed. A method is also proposed to obtain de-biased estimates of the regression coefficients that are asymptotically unbiased and have a joint asymptotic multivariate normal distribution. This provides valid confidence intervals of the regression coefficients and can be used to obtain the $p$-values. Simulation results show the validity of the confidence intervals and smaller variances of the de-biased estimates when the linear constraints are imposed. The proposed methods are applied to a gut microbiome data set and identify four bacterial genera that are associated with the body mass index after adjusting for the total fat and caloric intakes.