Miracle-dz/UniFAR-FTUs huggingface dataset:
Factor Analysis of Interval Data

This paper presents a factor analysis model for symbolic data, focusing on the particular case of interval-valued variables. The proposed method describes the correlation structure among the measured interval-valued variables in terms of a few underlying, but unobservable, uncorrelated interval-valued variables, called \textit{common factors}. Uniform and Triangular distributions are considered within each observed interval. We obtain the corresponding sample mean, variance and covariance assuming a general Triangular distribution. In our proposal, factors are extracted either by Principal Component or by Principal Axis Factoring, performed on the interval-valued variables correlation matrix. To estimate the values of the common factors, usually called \textit{factor scores}, two approaches are considered, which are inspired in methods for real-valued data: the Bartlett and the Anderson-Rubin methods. In both cases, the estimated values are obtained solving an optimization problem that minimizes a function of the weighted squared Mallows distance between quantile functions. Explicit expressions for the quantile function and the squared Mallows distance are derived assuming a general Triangular distribution. The applicability of the method is illustrated using two sets of data: temperature and precipitation in cities of the United States of America between the years 1971 and 2000 and measures of car characteristics of different makes and models. Moreover, the method is evaluated on synthetic data with predefined correlation structures.
Regret bounds for Narendra-Shapiro bandit algorithms

Abstract Narendra-Shapiro (NS) algorithms are bandit-type algorithms developed in the 1960s. NS-algorithms have been deeply studied in infinite horizon but little non-asymptotic results exist for this type of bandit algorithms. In this paper, we focus on a non-asymptotic study of the regret and address the following question: are Narendra-Shapiro bandit algorithms competitive from this point of view? In our main result, we obtain some uniform explicit bounds for the regret of (over)-penalized-NS algorithms. We also extend to the multi-armed case some convergence properties of penalized-NS algorithms towards a stationary Piecewise Deterministic Markov Process (PDMP). Finally, we establish some new sharp mixing bounds for these processes.
Improving Estimations in Quantile Regression Model with Autoregressive Errors

An important issue is that the respiratory mortality may be a result of air pollution which can be measured by the following variables: temperature, relative humidity, carbon monoxide, sulfur dioxide, nitrogen dioxide, hydrocarbons, ozone, and particulates. The usual way is to fit a model using the ordinary least squares regression, which has some assumptions, also known as Gauss-Markov assumptions, on the error term showing white noise process of the regression model. However, in many applications, especially for this example, these assumptions are not satisfied. Therefore, in this study, a quantile regression approach is used to model the respiratory mortality using the mentioned explanatory variables. Moreover, improved estimation techniques such as preliminary testing and shrinkage strategies are also obtained when the errors are autoregressive. A Monte Carlo simulation experiment, including the quantile penalty estimators such as lasso, ridge, and elastic net, is designed to evaluate the performances of the proposed techniques. Finally, the theoretical risks of the listed estimators are given.
Data Analysis Recipes: Using Markov Chain Monte Carlo

Markov Chain Monte Carlo (MCMC) methods for sampling probability density functions (combined with abundant computational resources) have transformed the sciences, especially in performing probabilistic inferences, or fitting models to data. In this primarily pedagogical contribution, we give a brief overview of the most basic MCMC method and some practical advice for the use of MCMC in real inference problems. We give advice on method choice, tuning for performance, methods for initialization, tests of convergence, troubleshooting, and use of the chain output to produce or report parameter estimates with associated uncertainties. We argue that autocorrelation time is the most important test for convergence, as it directly connects to the uncertainty on the sampling estimate of any quantity of interest. We emphasize that sampling is a method for doing integrals; this guides our thinking about how MCMC output is best used.
A Fast and Flexible Algorithm for the Graph-Fused Lasso

We propose a new algorithm for solving the graph-fused lasso (GFL), a method for parameter estimation that operates under the assumption that the signal tends to be locally constant over a predefined graph structure. Our key insight is to decompose the graph into a set of trails which can then each be solved efficiently using techniques for the ordinary (1D) fused lasso. We leverage these trails in a proximal algorithm that alternates between closed form primal updates and fast dual trail updates. The resulting techinque is both faster than previous GFL methods and more flexible in the choice of loss function and graph structure. Furthermore, we present two algorithms for constructing trail sets and show empirically that they offer a tradeoff between preprocessing time and convergence rate.
BDSAR: a new package on Bregman divergence for Bayesian simultaneous autoregressive models

BDSAR is an R package which estimates distances between probability distributions and facilitates a dynamic and powerful analysis of diagnostics for Bayesian models from the class of Simultaneous Autoregressive (SAR) spatial models. The package offers a new and fine plot to compare models as well as it works in an intuitive way to allow any analyst to easily build fine plots. These are helpful to promote insights about influential observations in the data.
Sparse Generalized Principal Component Analysis for Large-scale Applications beyond Gaussianity

Principal Component Analysis (PCA) is a dimension reduction technique. It produces inconsistent estimators when the dimensionality is moderate to high, which is often the problem in modern large-scale applications where algorithm scalability and model interpretability are difficult to achieve, not to mention the prevalence of missing values. While existing sparse PCA methods alleviate inconsistency, they are constrained to the Gaussian assumption of classical PCA and fail to address algorithm scalability issues. We generalize sparse PCA to the broad exponential family distributions under high-dimensional setup, with built-in treatment for missing values. Meanwhile we propose a family of iterative sparse generalized PCA (SG-PCA) algorithms such that despite the non-convexity and non-smoothness of the optimization task, the loss function decreases in every iteration. In terms of ease and intuitive parameter tuning, our sparsity-inducing regularization is far superior to the popular Lasso. Furthermore, to promote overall scalability, accelerated gradient is integrated for fast convergence, while a progressive screening technique gradually squeezes out nuisance dimensions of a large-scale problem for feasible optimization. High-dimensional simulation and real data experiments demonstrate the efficiency and efficacy of SG-PCA.
The Gaussian Process modelling module in UQLab

We introduce the Gaussian process (GP) modelling module developed within the UQLab software framework. The novel design of the GP-module aims at providing seamless integration of GP modelling into any uncertainty quantification workflow, as well as a standalone surrogate modelling tool. We first briefly present the key mathematical tools at the basis of GP modelling (a.k.a. Kriging), as well as the associated theoretical and computational framework. We then provide an extensive overview of the available features of the software and demonstrate its flexibility and user-friendliness. Finally, we showcase the usage and the performance of the software on several applications borrowed from different fields of engineering. These include a basic surrogate of a well-known analytical benchmark function, a hierarchical Kriging example applied to wind turbine aero-servo-elastic simulations and a more complex geotechnical example that requires a non-stationary, user-defined correlation function. The GP-module, like the rest of the scientific code that is shipped with UQLab, is open source (BSD license).
Empirical risk minimization and complexity of dynamical models

A dynamical model consists of a continuous self-map $T: \mathcal{X} \to \mathcal{X}$ of a compact state space $\mathcal{X}$ and a continuous observation function $f: \mathcal{X} \to \mathbb{R}$. This paper considers the fitting of a parametrized family of dynamical models to an observed real-valued stochastic process using empirical risk minimization. The limiting behavior of the minimum risk parameters is studied in a general setting. We establish a general convergence theorem for minimum risk estimators and ergodic observations. We then study conditions under which empirical risk minimization can effectively separate the signal from the noise in an additive observational noise model. The key, necessary condition in the latter results is that the family of dynamical models has limited complexity, which is quantified through a notion of entropy for families of infinite sequences. Close connections between entropy and limiting average mean widths for stationary processes are established.
A Computational Framework for Multivariate Convex Regression and Its Variants

ABSTRACT We study the nonparametric least squares estimator (LSE) of a multivariate convex regression function. The LSE, given as the solution to a quadratic program with O(n2) linear constraints (n being the sample size), is difficult to compute for large problems. Exploiting problem specific structure, we propose a scalable algorithmic framework based on the augmented Lagrangian method to compute the LSE. We develop a novel approach to obtain smooth convex approximations to the fitted (piecewise affine) convex LSE and provide formal bounds on the quality of approximation. When the number of samples is not too large compared to the dimension of the predictor, we propose a regularization scheme—Lipschitz convex regression—where we constrain the norm of the subgradients, and study the rates of convergence of the obtained LSE. Our algorithmic framework is simple and flexible and can be easily adapted to handle variants: estimation of a nondecreasing/nonincreasing convex/concave (with or without a Lipschitz bound) function. We perform numerical studies illustrating the scalability of the proposed algorithm—on some instances our proposal leads to more than a 10,000-fold improvement in runtime when compared to off-the-shelf interior point solvers for problems with n = 500.
Symbolic Calculus in Mathematical Statistics: A Review

In the last ten years, the employment of symbolic methods has substantially extended both the theory and the applications of statistics and probability. This survey reviews the development of a symbolic technique arising from classical umbral calculus, as introduced by Rota and Taylor in $1994. $ The usefulness of this symbolic technique is twofold. The first is to show how new algebraic identities drive in discovering insights among topics apparently very far from each other and related to probability and statistics. One of the main tools is a formal generalization of the convolution of identical probability distributions, which allows us to employ compound Poisson random variables in various topics that are only somewhat interrelated. Having got a different and deeper viewpoint, the second goal is to show how to set up algorithmic processes performing efficiently algebraic calculations. In particular, the challenge of finding these symbolic procedures should lead to a new method, and it poses new problems involving both computational and conceptual issues. Evidence of efficiency in applying this symbolic method will be shown within statistical inference, parameter estimation, L\'evy processes, and, more generally, problems involving multivariate functions. The symbolic representation of Sheffer polynomial sequences allows us to carry out a unifying theory of classical, Boolean and free cumulants. Recent connections within random matrices have extended the applications of the symbolic method.
Shape Parameter Estimation

Performance of machine learning approaches depends strongly on the choice of misfit penalty, and correct choice of penalty parameters, such as the threshold of the Huber function. These parameters are typically chosen using expert knowledge, cross-validation, or black-box optimization, which are time consuming for large-scale applications. We present a principled, data-driven approach to simultaneously learn the model pa- rameters and the misfit penalty parameters. We discuss theoretical properties of these joint inference problems, and develop algorithms for their solution. We show synthetic examples of automatic parameter tuning for piecewise linear-quadratic (PLQ) penalties, and use the approach to develop a self-tuning robust PCA formulation for background separation.
Asymptotic goodness-of-fit tests for point processes based on scaled empirical K-functions

ABSTRACT We study sequences of scaled edge-corrected empirical (generalized) K-functions (modifying Ripley's K-function) each of them constructed from a single observation of a d-dimensional fourth-order stationary point process in a sampling window which grows together with some scaling rate unboundedly as . Under some natural assumptions it is shown that the normalized difference between scaled empirical and scaled theoretical K-function converges weakly to a mean zero Gaussian process with simple covariance function. This result suggests discrepancy measures between empirical and theoretical K-function with known limit distribution which allow to perform goodness-of-fit tests for checking a hypothesized point process based only on its intensity and (generalized) K-function. Similar test statistics are derived for testing the hypothesis that two independent point processes in have the same distribution without explicit knowledge of their intensities and K-functions.
A Non‐Gaussian Spatio‐Temporal Model for Daily Wind Speeds Based on a Multi‐Variate Skew‐t Distribution

Facing increasing domestic energy consumption from population growth and industrialization, Saudi Arabia is aiming to reduce its reliance on fossil fuels and to broaden its energy mix by expanding investment in renewable energy sources, including wind energy. A preliminary task in the development of wind energy infrastructure is the assessment of wind energy potential, a key aspect of which is the characterization of its spatio‐temporal behavior. In this study we examine the impact of internal climate variability on seasonal wind power density fluctuations over Saudi Arabia using 30 simulations from the Large Ensemble Project (LENS) developed at the National Center for Atmospheric Research. Furthermore, a spatio‐temporal model for daily wind speed is proposed with neighbor‐based cross‐temporal dependence, and a multi‐variate skew‐t distribution to capture the spatial patterns of higher‐order moments. The model can be used to generate synthetic time series over the entire spatial domain that adequately reproduce the internal variability of the LENS dataset.
Pivotal Quantities with Arbitrary Small Skewness

In this paper we present randomization methods to enhance the accuracy of the central limit theorem (CLT) based inferences about the population mean $\mu$. We introduce a broad class of randomized versions of the Student $t$-statistic, the classical pivot for $\mu$, that continue to possess the pivotal property for $\mu$ and their skewness can be made arbitrarily small, for each fixed sample size $n$. Consequently, these randomized pivots admit CLTs with smaller errors. The randomization framework in this paper also provides an explicit relation between the precision of the CLTs for the randomized pivots and the volume of their associated confidence regions for the mean for both univariate and multivariate data. This property allows regulating the trade-off between the accuracy and the volume of the randomized confidence regions discussed in this paper.
A Sampling Strategy for Projecting to Permutations in the Graph Matching Problem

In the context of the graph matching problem we propose a novel method for projecting a matrix $Q$, which may be a doubly stochastic matrix, to a permutation matrix $P.$ We observe that there is an intuitve mapping, depending on a given $Q,$ from the set of $n$-dimensional permutation matrices to sets of points in $\mathbb{R}^n$. The mapping has a number of geometrical properties that allow us to succesively sample points in $\mathbb{R}^n$ in a manner similar to simulated annealing, where our objective is to minimise the graph matching norm found using the permutation matrix corresponding to each of the points. Our sampling strategy is applied to the QAPLIB benchmark library and outperforms the PATH algorithm in two-thirds of cases. Instead of using linear assignment, the incorporation of our sampling strategy as a projection step into algorithms such as PATH itself has the potential to achieve even better results.
Local average causal effects and superefficiency

Recent approaches in causal inference have proposed estimating average causal effects that are local to some subpopulation, often for reasons of efficiency. These inferential targets are sometimes data-adaptive, in that they are dependent on the empirical distribution of the data. In this short note, we show that if researchers are willing to adapt the inferential target on the basis of efficiency, then extraordinary gains in precision can be obtained. Specifically, when causal effects are heterogeneous, any asymptotically normal and root-$n$ consistent estimator of the population average causal effect is superefficient for a data-adaptive local average causal effect. Our result illustrates the fundamental gain in statistical certainty afforded by indifference about the inferential target.
Nonparametric estimation of galaxy cluster's emissivity and point source detection in astrophysics with two lasso penalties

Astrophysicists are interested in recovering the 3D gas emissivity of a galaxy cluster from a 2D image taken by a telescope. A blurring phenomenon and presence of point sources make this inverse problem even harder to solve. The current state-of-the-art technique is two step: first identify the location of potential point sources, then mask these locations and deproject the data. We instead model the data as a Poisson generalized linear model (involving blurring, Abel and wavelets operators) regularized by two lasso penalties to induce sparse wavelet representation and sparse point sources. The amount of sparsity is controlled by two quantile universal thresholds. As a result, our method outperforms the existing one.
A Guide to Comparing the Performance of VA Algorithms

The literature comparing the performance of algorithms for assigning cause of death using verbal autopsy data is fractious and does not reach a consensus on which algorithms perform best, or even how to do the comparison. This manuscript explains the challenges and suggests a way forward. A universal challenge is the lack of standard training and testing data. This limits meaningful comparisons between algorithms, and further, limits the ability of any algorithm to classify verbal autopsy deaths by cause in a way that is widely generalizable across regions and through time. Verbal autopsy algorithms utilize a variety of information to describe the relationship between verbal autopsy symptoms and causes of death - called symptom-cause information (SCI). A crowd sourced, public archive of SCI managed by the World Health Organization (WHO) is suggested as a way to address the lack of SCI for developing, testing, and comparing verbal autopsy coding algorithms, and additionally, as a way to ensure that algorithm-assigned causes of death are as accurate and comparable across regions and through time as possible.
Distributions and Statistical Power of Optimal Signal-Detection Methods In Finite Cases

In big data analysis for detecting rare and weak signals among $n$ features, some grouping-test methods such as Higher Criticism test (HC), Berk-Jones test (B-J), and $\phi$-divergence test share the similar asymptotical optimality when $n \rightarrow \infty$. However, in practical data analysis $n$ is frequently small and moderately large at most. In order to properly apply these optimal tests and wisely choose them for practical studies, it is important to know how to get the p-values and statistical power of them. To address this problem in an even broader context, this paper provides analytical solutions for a general family of goodness-of-fit (GOF) tests, which covers these optimal tests. For any given i.i. d. and continuous distributions of the input test statistics of the $n$ features, both p-value and statistical power of such a GOF test can be calculated. By calculation we compared the finite-sample performances of asymptotically optimal tests under the normal mixture alternative. Results show that HC is the best choice when signals are rare, while B-J is more robust over various signal patterns. In the application to a real genome-wide association study, results illustrate that the p-value calculation works well, and the optimal tests have potentials for detecting novel disease genes with weak genetic effects. The calculations have been implemented in an R package SetTest and published on the CRAN.
An optimal $(\epsilon,\delta)$-approximation scheme for the mean of random variables with bounded relative variance

Randomized approximation algorithms for many #P-complete problems (such as the partition function of a Gibbs distribution, the volume of a convex body, the permanent of a $\{0,1\}$-matrix, and many others) reduce to creating random variables $X_1,X_2,\ldots$ with finite mean $\mu$ and standard deviation$\sigma$ such that $\mu$ is the solution for the problem input, and the relative standard deviation $|\sigma/\mu| \leq c$ for known $c$. Under these circumstances, it is known that the number of samples from the $\{X_i\}$ needed to form an $(\epsilon,\delta)$-approximation $\hat \mu$ that satisfies $\mathbb{P}(|\hat \mu - \mu| > \epsilon \mu) \leq \delta$ is at least $(2-o(1))\epsilon^{-2} c^2\ln(1/\delta)$. We present here an easy to implement $(\epsilon,\delta)$-approximation $\hat \mu$ that uses $(2+o(1))c^2\epsilon^{-2}\ln(1/\delta)$ samples. This achieves the same optimal running time as other estimators, but without the need for extra conditions such as bounds on third or fourth moments.
Big Data Analysis Using Shrinkage Strategies

In this paper, we apply shrinkage strategies to estimate regression coefficients efficiently for the high-dimensional multiple regression model, where the number of samples is smaller than the number of predictors. We assume in the sparse linear model some of the predictors have very weak influence on the response of interest. We propose to shrink estimators more than usual. Specifically, we use integrated estimation strategies in sub and full models and shrink the integrated estimators by incorporating a bounded measurable function of some weights. The exhibited double shrunken estimators improve the prediction performance of sub models significantly selected from existing Lasso-type variable selection methods. Monte Carlo simulation studies as well as real examples of eye data and Riboavin data confirm the superior performance of the estimators in the high-dimensional regression model.