Miracle-dz/UniFAR-FTUs huggingface dataset:
A Piecewise Deterministic Markov Process via $(r,\theta)$ swaps in hyperspherical coordinates

Recently, a class of stochastic processes known as piecewise deterministic Markov processes has been used to define continuous-time Markov chain Monte Carlo algorithms with a number of attractive properties, including compatibility with stochastic gradients like those typically found in optimization and variational inference, and high efficiency on certain big data problems. Not many processes in this class that are capable of targeting arbitrary invariant distributions are currently known, and within one subclass all previously known processes utilize linear transition functions. In this work, we derive a process whose transition function is nonlinear through solving its Fokker-Planck equation in hyperspherical coordinates. We explore its behavior on Gaussian targets, as well as a Bayesian logistic regression model with synthetic data. We discuss implications to both the theory of piecewise deterministic Markov processes, and to Bayesian statisticians as well as physicists seeking to use them for simulation-based computation.
Pathway Lasso: Estimate and Select Sparse Mediation Pathways with High Dimensional Mediators

In many scientific studies, it becomes increasingly important to delineate the causal pathways through a large number of mediators, such as genetic and brain mediators. Structural equation modeling (SEM) is a popular technique to estimate the pathway effects, commonly expressed as products of coefficients. However, it becomes unstable to fit such models with high dimensional mediators, especially for a general setting where all the mediators are causally dependent but the exact causal relationships between them are unknown. This paper proposes a sparse mediation model using a regularized SEM approach, where sparsity here means that a small number of mediators have nonzero mediation effects between a treatment and an outcome. To address the model selection challenge, we innovate by introducing a new penalty called Pathway Lasso. This penalty function is a convex relaxation of the non-convex product function, and it enables a computationally tractable optimization criterion to estimate and select many pathway effects simultaneously. We develop a fast ADMM-type algorithm to compute the model parameters, and we show that the iterative updates can be expressed in closed form. On both simulated data and a real fMRI dataset, the proposed approach yields higher pathway selection accuracy and lower estimation bias than other competing methods.
Statistics Educational Challenge in the

What do we teach and what should we teach? An honest answer to this question is painful, very painful--what we teach lags decades behind what we practice. How can we reduce this `gap' to prepare a data science workforce of trained next-generation statisticians? This is a challenging open problem that requires many well-thought-out experiments before finding the secret sauce. My goal in this article is to lay out some basic principles and guidelines (rather than creating a pseudo-curriculum based on cherry-picked topics) to expedite this process for finding an `objective' solution.
Nonparametric Quantile-Based Causal Discovery

Telling cause from effect using observational data is a challenging problem, especially in the bivariate case. Contemporary methods often assume an independence between the cause and the generating mechanism of the effect given the cause. From this postulate, they derive asymmetries to uncover causal relationships. In this work, we propose such an approach, based on the link between Kolmogorov complexity and quantile scoring. We use a nonparametric conditional quantile estimator based on copulas to implement our procedure, thus avoiding restrictive assumptions about the joint distribution between cause and effect. In an extensive study on real and synthetic data, we show that quantile copula causal discovery (QCCD) compares favorably to state-of-the-art methods, while at the same time being computationally efficient and scalable.
Transforming cumulative hazard estimates

Time to event outcomes are often evaluated on the hazard scale, but interpreting hazards may be difficult. Recently, there has been concern in the causal inference literature that hazards actually have a built in selection-effect that prevents simple causal interpretations. This is even a problem in randomized controlled trials, where hazard ratios have become a standard measure of treatment effects. Modeling on the hazard scale is nevertheless convenient, e.g. to adjust for covariates. Using hazards for intermediate calculations may therefore be desirable. Here, we provide a generic method for transforming hazard estimates consistently to other scales at which these built in selection effects are avoided. The method is based on differential equations, and generalize a well known relation between the Nelson-Aalen and Kaplan-Meier estimators. Using the martingale central limit theorem we also find that covariances can be estimated consistently for a large class of estimators, thus allowing for rapid calculations of confidence intervals. Hence, given cumulative hazard estimates based on e.g. Aalen's additive hazard model, we can obtain many other parameters without much more effort. We present several examples and associated estimators. Coverage and convergence speed is explored using simulations, suggesting that reliable estimates can be obtained in real-life scenarios.
Unbiased Simulation for Optimizing Stochastic Function Compositions

In this paper, we introduce an unbiased gradient simulation algorithms for solving convex optimization problem with stochastic function compositions. We show that the unbiased gradient generated from the algorithm has finite variance and finite expected computation cost. We then combined the unbiased gradient simulation with two variance reduced algorithms (namely SVRG and SCSG) and showed that the proposed optimization algorithms based on unbiased gradient simulations exhibit satisfactory convergence properties. Specifically, in the SVRG case, the algorithm with simulated gradient can be shown to converge linearly to optima in expectation and almost surely under strong convexity. Finally, for the numerical experiment,we applied the algorithms to two important cases of stochastic function compositions optimization: maximizing the Cox's partial likelihood model and training conditional random fields.
Integrating Semantic Knowledge to Tackle Zero-shot Text Classification

Insufficient or even unavailable training data of emerging classes is a big challenge of many classification tasks, including text classification. Recognising text documents of classes that have never been seen in the learning stage, so-called zero-shot text classification, is therefore difficult and only limited previous works tackled this problem. In this paper, we propose a two-phase framework together with data augmentation and feature augmentation to solve this problem. Four kinds of semantic knowledge (word embeddings, class descriptions, class hierarchy, and a general knowledge graph) are incorporated into the proposed framework to deal with instances of unseen classes effectively. Experimental results show that each and the combination of the two phases achieve the best overall accuracy compared with baselines and recent approaches in classifying real-world texts under the zero-shot scenario.
Anchor regression: Heterogeneous data meet causality

We consider the problem of predicting a response variable from a set of covariates on a data set that differs in distribution from the training data. Causal parameters are optimal in terms of predictive accuracy if in the new distribution either many variables are affected by interventions or only some variables are affected, but the perturbations are strong. If the training and test distributions differ by a shift, causal parameters might be too conservative to perform well on the above task. This motivates anchor regression, a method that makes use of exogenous variables to solve a relaxation of the ‘causal’ minimax problem by considering a modification of the least‐squares loss. The procedure naturally provides an interpolation between the solutions of ordinary least squares (OLS) and two‐stage least squares. We prove that the estimator satisfies predictive guarantees in terms of distributional robustness against shifts in a linear class; these guarantees are valid even if the instrumental variable assumptions are violated. If anchor regression and least squares provide the same answer (‘anchor stability’), we establish that OLS parameters are invariant under certain distributional changes. Anchor regression is shown empirically to improve replicability and protect against distributional shifts.
Strictly Proper Kernel Scoring Rules and Divergences with an Application to Kernel Two-Sample Hypothesis Testing

We study strictly proper scoring rules in the Reproducing Kernel Hilbert Space. We propose a general Kernel Scoring rule and associated Kernel Divergence. We consider conditions under which the Kernel Score is strictly proper. We then demonstrate that the Kernel Score includes the Maximum Mean Discrepancy as a special case. We also consider the connections between the Kernel Score and the minimum risk of a proper loss function. We show that the Kernel Score incorporates more information pertaining to the projected embedded distributions compared to the Maximum Mean Discrepancy. Finally, we show how to integrate the information provided from different Kernel Divergences, such as the proposed Bhattacharyya Kernel Divergence, using a one-class classifier for improved two-sample hypothesis testing results.
Center-Outward Distribution Functions, Quantiles, Ranks, and Signs in $\mathbb{R}^d$

Univariate concepts as quantile and distribution functions involving ranks and signs, do not canonically extend to $\mathbb{R}^d, d\geq 2$. Palliating that has generated an abundant literature. Chapter 1 shows that, unlike the many definitions that have been proposed so far, the measure transportation-based ones introduced in Chernozhukov et al. (2017) enjoy all the properties that make univariate quantiles and ranks successful tools for semiparametric statistical inference. We therefore propose a new center-outward definition of multivariate distribution and quantile functions, along with their empirical counterparts, for which we obtain a Glivenko-Cantelli result. Our approach is geometric and, contrary to the Monge-Kantorovich one in Chernozhukov et al. (2017), does not require any moment assumptions. The resulting ranks and signs are strictly distribution-free, and maximal invariant under the action of a data-driven class of (order-preserving) transformations generating the family of absolutely continuous distributions; that property is the theoretical foundation of the semiparametric efficiency preservation property of ranks. The corresponding quantiles are equivariant under the same transformations. The empirical proposed distribution functions are defined at observed values only. A continuous extension to the entire $\mathbb{R}^d$, yielding continuous empirical quantile contours while preserving the monotonicity and Glivenko-Cantelli features is desirable. Such extension requires solving a nontrivial problem of smooth interpolation under cyclical monotonicity constraints. A complete solution of that problem is given in Chapter 2; we show that the resulting distribution and quantile functions are Lipschitz, and provide a sharp lower bound for the Lipschitz constants. A numerical study of empirical center-outward quantile contours and their consistency is conducted.
Spatial trend analysis of gridded temperature data at varying spatial scales

Abstract. Classical assessments of trends in gridded temperature data perform independent evaluations across the grid, thus, ignoring spatial correlations in the trend estimates. In particular, this affects assessments of trend significance as evaluation of the collective significance of individual tests is commonly neglected. In this article we build a space–time hierarchical Bayesian model for temperature anomalies where the trend coefficient is modelled by a latent Gaussian random field. This enables us to calculate simultaneous credible regions for joint significance assessments. In a case study, we assess summer season trends in 65 years of gridded temperature data over Europe. We find that while spatial smoothing generally results in larger regions where the null hypothesis of no trend is rejected, this is not the case for all subregions.
Few-Shot Deep Adversarial Learning for Video-Based Person Re-Identification

Video-based person re-identification (re-ID) refers to matching people across camera views from arbitrary unaligned video footages. Existing methods rely on supervision signals to optimise a projected space under which the distances between inter/intra-videos are maximised/minimised. However, this demands exhaustively labelling people across camera views, rendering them unable to be scaled in large networked cameras. Also, it is noticed that learning effective video representations with view invariance is not explicitly addressed for which features exhibit different distributions otherwise. Thus, matching videos for person re-ID demands flexible models to capture the dynamics in time-series observations and learn view-invariant representations with access to limited labeled training samples. In this paper, we propose a novel few-shot deep learning approach to video-based person re-ID, to learn comparable representations that are discriminative and view-invariant. The proposed method is developed on the variational recurrent neural networks (VRNNs) and trained adversarially to produce latent variables with temporal dependencies that are highly discriminative yet view-invariant in matching persons. Through extensive experiments conducted on three benchmark datasets, we empirically show the capability of our method in creating view-invariant temporal features and state-of-the-art performance achieved by our method.
Uncertainty quantification of the parameters and predictions of a phenomenological constitutive model for thermally induced phase transformation in Ni–Ti shape memory alloys

In this work, a probabilistic calibration approach has been adopted to quantify the uncertainties of the parameters and predictions of a phenomenological shape memory alloys (SMAs) constitutive model which has been used to predict the thermally induced phase transformation of Ni–Ti SMAs. Furthermore, the impact of the adopted calibration method, which enables the determination of the uncertainty bounds of the model predictions, on the design of robust engineering applications has been discussed. To this end prior to the probabilistic model calibration, a design of experiments has been performed in order to identify the most influential parameters on the response of the system and thus reduce the dimensionality of the problem. Subsequently, uncertainty quantification (UQ) of the influential parameters has been carried out through Bayesian Markov Chain Monte Carlo (MCMC). The assessed uncertainties in the model parameters has been then propagated to the model predictions using an approximate approach based on the variance–covariance matrix of the MCMC-calibrated model parameters and then an explicit propagation of uncertainty through MCMC-based sampling. The determined 95% Bayesian confidence intervals of the model predictions, by the latter methods, have been demonstrated and compared. Additionally, good agreement between the experimentally measured and model predicted SMA hysteresis loops has been observed where the experimental data are situated within the predicted 95% Bayesian confidence intervals. Finally, the application of the MCMC-based UQ/UP approach in decision making for experimental design has also been shown by comparing the information that can be gained by performing multiple repetitive experiments under identical thermo-mechanical conditions versus experiments under different conditions.
Doubly Robust Inference With Nonprobability Survey Samples

Abstract We establish a general framework for statistical inferences with nonprobability survey samples when relevant auxiliary information is available from a probability survey sample. We develop a rigorous procedure for estimating the propensity scores for units in the nonprobability sample, and construct doubly robust estimators for the finite population mean. Variance estimation is discussed under the proposed framework. Results from simulation studies show the robustness and the efficiency of our proposed estimators as compared to existing methods. The proposed method is used to analyze a nonprobability survey sample collected by the Pew Research Center with auxiliary information from the Behavioral Risk Factor Surveillance System and the Current Population Survey. Our results illustrate a general approach to inference with nonprobability samples and highlight the importance and usefulness of auxiliary information from probability survey samples. Supplementary materials for this article are available online.
Hadamard Matrices, Quaternions, and the Pearson Chi-square Statistic

We present a symbolic decomposition of the Pearson chi-square statistic with unequal cell probabilities, by presenting Hadamard-type matrices whose columns are eigenvectors of the variance-covariance matrix of the cell counts. All of the eigenvectors have non-zero values so each component test uses all cell probabilities in a way that makes it intuitively interpretable. When all cell probabilities are distinct and unrelated we establish that such decomposition is only possible when the number of multinomial cells is a small power of 2. For higher powers of 2, we show, using the theory of orthogonal designs, that the targeted decomposition is possible when appropriate relations are imposed on the cell probabilities, the simplest of which is when the probabilities are equal and the decomposition is reduced to the one obtained by Hadamard matrices. Simulations are given to illustrate the sensitivity of various components to changes in location, scale skewness and tail probability, as well as to illustrate the potential improvement in power when the cell probabilities are changed.
On efficiency of the plug-in principle for estimating smooth integrated functionals of a nonincreasing density

We consider the problem of estimating smooth integrated functionals of a monotone nonincreasing density $f$ on $[0,\infty)$ using the nonparametric maximum likelihood based plug-in estimator. We find the exact asymptotic distribution of this natural (tuning parameter-free) plug-in estimator, properly normalized. In particular, we show that the simple plug-in estimator is always $\sqrt{n}$-consistent, and is additionally asymptotically normal with zero mean and the semiparametric efficient variance for estimating a subclass of integrated functionals. Compared to the previous results on this topic (see e.g., Nickl (2007), Gine and Nickl (2008), Jankowski (2014), and Sohl (2015)) our results hold for a much larger class of functionals (which include linear and non-linear functionals) under less restrictive assumptions on the underlying $f$ --- we do not require $f$ to be (i) smooth, (ii) bounded away from $0$, or (iii) compactly supported. Further, when $f$ is the uniform distribution on a compact interval we explicitly characterize the asymptotic distribution of the plug-in estimator --- which now converges at a non-standard rate --- thereby extending the results in Groeneboom and Pyke (1983) for the case of the quadratic functional.
Can the Wild Bootstrap be Tamed into a General Analysis of Covariance Model

It is well known that the F test is severly affected by heteroskedasticity in unbalanced analysis of covariance (ANCOVA) models. Currently available remedies for such a scenario are either based on heteroskedasticity-consistent covariance matrix estimation (HCCME) or bootstrap techniques. However, the HCCME approach tends to be liberal in small samples. Therefore, we propose a combination of HCCME and a wild bootstrap technique. We prove the theoretical validity of our approach and investigate its performance in an extensive simulation study in comparison to existing procedures. The results indicate that our proposed test remedies all problems of the ANCOVA F test and its heteroskedasticityconsistent alternatives. Our test only requires very general conditions, thus being applicable in a broad range of real-life settings.
Minimizing Sum of Truncated Convex Functions and Its Applications

ABSTRACT In this article, we study a class of problems where the sum of truncated convex functions is minimized. In statistical applications, they are commonly encountered when ℓ0-penalized models are fitted and usually lead to NP-Hard non-convex optimization problems. In this article, we propose a general algorithm for the global minimizer in low-dimensional settings. We also extend the algorithm to high-dimensional settings, where an approximate solution can be found efficiently. We introduce several applications where the sum of truncated convex functions is used, compare our proposed algorithm with other existing algorithms in simulation studies, and show its utility in edge-preserving image restoration on real data.
Optimal Spectral Shrinkage and PCA With Heteroscedastic Noise

This paper studies the related problems of prediction, covariance estimation, and principal component analysis for the spiked covariance model with heteroscedastic noise. We consider an estimator of the principal components based on whitening the noise, and we derive optimal singular value and eigenvalue shrinkers for use with these estimated principal components. Underlying these methods are new asymptotic results for the high-dimensional spiked model with heteroscedastic noise, and consistent estimators for the relevant population parameters. We extend previous analysis on out-of-sample prediction to the setting of predictors with whitening. We demonstrate certain advantages of noise whitening. Specifically, we show that in a certain asymptotic regime, optimal singular value shrinkage with whitening converges to the best linear predictor, whereas without whitening it converges to a suboptimal linear predictor. We prove that for generic signals, whitening improves estimation of the principal components, and increases a natural signal-to-noise ratio of the observations. We also show that for rank one signals, our estimated principal components achieve the asymptotic minimax rate.
Multi-Armed Bandit Problem and Batch UCB Rule

We obtain the upper bound of the loss function for a strategy in the multi-armed bandit problem with Gaussian distributions of incomes. Considered strategy is an asymptotic generalization of the strategy proposed by J. Bather for the multi-armed bandit problem and using UCB rule, i.e. choosing the action corresponding to the maximum of the upper bound of the confidence interval of the current estimate of the expected value of one-step income. Results are obtained with the help of invariant description of the control on the unit horizon in the domain of close distributions because just there the loss function attains its maximal values. UCB rule is widely used in machine learning. It can be also used for the batch data processing optimization if there are two alternative processing methods available with different a priori unknown efficiencies.