Miracle-dz/UniFAR-FTUs huggingface dataset:
Propensity Score Weighting for Causal Inference with Multi-valued Treatments

This article proposes a unified framework, the balancing weights, for estimating causal effects with multi-valued treatments using propensity score weighting. These weights incorporate the generalized propensity score to balance the weighted covariate distribution of each treatment group, all weighted toward a common pre-specified target population. The class of balancing weights include several existing approaches such as inverse probability weights and trimming weights as special cases. Within this framework, we propose a class of target estimands based on linear contrasts and their corresponding nonparametric weighting estimators. We further propose the generalized overlap weights, constructed as the product of the inverse probability weights and the harmonic mean of the generalized propensity scores, to focus on the target population with the most overlap in covariates. These weights are bounded and thus bypass the problem of extreme propensities. We show that the generalized overlap weights minimize the total asymptotic variance of the nonparametric estimators for the pairwise contrasts within the class of balancing weights. We also develop two new balance check criteria and a sandwich variance estimator for estimating the causal effects with generalized overlap weights. We illustrate these methods by simulations and apply them to study the racial disparities in medical expenditure.
Counting With Focus for Free

This paper aims to count arbitrary objects in images. The leading counting approaches start from point annotations per object from which they construct density maps. Then, their training objective transforms input images to density maps through deep convolutional networks. We posit that the point annotations serve more supervision purposes than just constructing density maps. We introduce ways to repurpose the points for free. First, we propose supervised focus from segmentation, where points are converted into binary maps. The binary maps are combined with a network branch and accompanying loss function to focus on areas of interest. Second, we propose supervised focus from global density, where the ratio of point annotations to image pixels is used in another branch to regularize the overall density estimation. To assist both the density estimation and the focus from segmentation, we also introduce an improved kernel size estimator for the point annotations. Experiments on six datasets show that all our contributions reduce the counting error, regardless of the base network, resulting in state-of-the-art accuracy using only a single network. Finally, we are the first to count on WIDER FACE, allowing us to show the benefits of our approach in handling varying object scales and crowding levels. Code is available at https://github.com/shizenglin/Counting-with-Focus-for-Free
Associating Growth in Infancy and Cognitive Performance in Early Childhood: A functional data analysis approach

Physical growth traits can be naturally represented by continuous functions. In a large dataset of infancy growth patterns, we develop a practical approach to infer statistical associations between growth-trajectories and IQ performance in early childhood. The main objective of this study is to show how to assess physical growth curves and detect if particular infancy growth patterns are associated with differences in IQ (Full-scale WASI scores) in later ages using a semi-parametric functional response model. Additionally, we investigate the association between different growth measurements in terms of their cross-correlation with each other, their correlation with later IQ, as well as their time-varying dynamics. This analysis framework can easily incorporate or learn population information in a non-parametric way, rendering the existence of prior population charts partially redundant.
Statistical Evaluation of Spectral Methods for Anomaly Detection in Networks

Monitoring of networks for anomaly detection has attracted a lot of attention in recent years especially with the rise of connected devices and social networks. This is of importance as anomaly detection could span a wide range of application, from detecting terrorist cells in counter-terrorism efforts to phishing attacks in social network circles. For this reason, numerous techniques for anomaly detection have been introduced. However, application of these techniques to more complex network models is hindered by various challenges such as the size of the network being investigated, how much apriori information is needed, the size of the anomalous graph, among others. A recent technique introduced by Miller et al, which relies on a spectral framework for anomaly detection, has the potential to address many of these challenges. In their discussion of the spectral framework, three algorithms were proposed that relied on the eigenvalues and eigenvectors of the residual matrix of a binary network. The authors demonstrated the ability to detect anomalous subgraphs that were less than 1% of the network size. However, to date, there is little work that has been done to evaluate the statistical performance of these algorithms. This study investigates the statistical properties of the spectral methods, specifically the Chi-square and L1 norm algorithm proposed by Miller. We will analyze the performance of the algorithm using simulated networks and also extend the method's application to count networks. Finally we will make some methodological improvements and recommendations to both algorithms.
Energy distance and kernel mean embeddings for two-sample survival testing

In this article a new family of tests is proposed for the comparison problem of the equality of distribution of two-sample under right censoring scheme. The tests are based on energy distance and kernels mean embedding, are calibrated by permutations and are consistent against all alternatives. The good performance of the new tests in real situations with finite samples is established with a simulation study.
Pairwise likelihood inference for the multivariate ordered probit model

This paper provides a closed form expression for the pairwise score vector for the multivariate ordered probit model. This result has several implications in likelihood-based inference. It is indeed used both to speed-up gradient based optimization routines for point estimation, and to provide a building block to compute standard errors and confidence intervals by means of the Godambe matrix.
Second-order Stein: SURE for SURE and other applications in high-dimensional inference

Stein's formula states that a random variable of the form $z^\top f(z) - {\rm{div}} f(z)$ is mean-zero for all functions $f$ with integrable gradient. Here, ${\rm{div}} f$ is the divergence of the function $f$ and $z$ is a standard normal vector. A Second Order Stein formula is proposed to characterize the variance of such random variables. In the Gaussian sequence model, a remarkable consequence of Stein's formula is Stein's Unbiased Risk Estimate (SURE) of the mean square risk of almost any given estimator $\hat\mu$ for the unknown mean vector. A first application of the Second Order Stein formula is an Unbiased Risk Estimate of the risk of SURE itself (SURE for SURE): a simple unbiased estimate provides information about the squared distance between SURE and the squared estimation error of $\hat\mu$. SURE for SURE has a simple form and can be computed explicitly for differentiable $\hat\mu$, for example the Lasso and the Elastic Net. Other applications of the Second Order Stein formula are provided in high-dimensional regression. This includes novel bounds on the variance of the size of the model selected by the Lasso, and a general semi-parametric scheme to de-bias an almost differentiable initial estimator in order to estimate a low-dimensional projection of the unknown regression coefficient vector.
Learning to Transfer Examples for Partial Domain Adaptation

Domain adaptation is critical for learning in new and unseen environments. With domain adversarial training, deep networks can learn disentangled and transferable features that effectively diminish the dataset shift between the source and target domains for knowledge transfer. In the era of Big Data, large-scale labeled datasets are readily available, stimulating the interest in partial domain adaptation (PDA), which transfers a recognizer from a large labeled domain to a small unlabeled domain. It extends standard domain adaptation to the scenario where target labels are only a subset of source labels. Under the condition that target labels are unknown, the key challenges of PDA are how to transfer relevant examples in the shared classes to promote positive transfer and how to ignore irrelevant ones in the source domain to mitigate negative transfer. In this work, we propose a unified approach to PDA, Example Transfer Network (ETN), which jointly learns domain-invariant representations across domains and a progressive weighting scheme to quantify the transferability of source examples. A thorough evaluation on several benchmark datasets shows that ETN consistently achieves state-of-the-art results for various partial domain adaptation tasks.
Survival forests under test: Impact of the proportional hazards assumption on prognostic and predictive forests for amyotrophic lateral sclerosis survival

We investigate the effect of the proportional hazards assumption on prognostic and predictive models of the survival time of patients suffering from amyotrophic lateral sclerosis. We theoretically compare the underlying model formulations of several variants of survival forests and implementations thereof, including random forests for survival, conditional inference forests, Ranger, and survival forests with L1 splitting, with two novel variants, namely distributional and transformation survival forests. Theoretical considerations explain the low power of log-rank-based splitting in detecting patterns in non-proportional hazards situations in survival trees and corresponding forests. This limitation can potentially be overcome by the alternative split procedures suggested herein. We empirically investigated this effect using simulation experiments and a re-analysis of the Pooled Resource Open-Access ALS Clinical Trials database of amyotrophic lateral sclerosis survival, giving special emphasis to both prognostic and predictive models.
Graphical models for extremes

Conditional independence, graphical models and sparsity are key notions for parsimonious statistical models and for understanding the structural relationships in the data. The theory of multivariate and spatial extremes describes the risk of rare events through asymptotically justified limit models such as max‐stable and multivariate Pareto distributions. Statistical modelling in this field has been limited to moderate dimensions so far, partly owing to complicated likelihoods and a lack of understanding of the underlying probabilistic structures. We introduce a general theory of conditional independence for multivariate Pareto distributions that enables the definition of graphical models and sparsity for extremes. A Hammersley–Clifford theorem links this new notion to the factorization of densities of extreme value models on graphs. For the popular class of Hüsler–Reiss distributions we show that, similarly to the Gaussian case, the sparsity pattern of a general extremal graphical model can be read off from suitable inverse covariance matrices. New parametric models can be built in a modular way and statistical inference can be simplified to lower dimensional marginals. We discuss learning of minimum spanning trees and model selection for extremal graph structures, and we illustrate their use with an application to flood risk assessment on the Danube river.
Homogeneity testing under finite location‐scale mixtures

The testing problem for the order of finite mixture models has a long history and remains an active research topic. Since Ghosh & Sen (1985) revealed the hard‐to‐manage asymptotic properties of the likelihood ratio test, many successful alternative approaches have been developed. The most successful attempts include the modified likelihood ratio test and the EM‐test, which lead to neat solutions for finite mixtures of univariate normal distributions, finite mixtures of single‐parameter distributions, and several mixture‐like models. The problem remains challenging, and there is still no generic solution for location‐scale mixtures. In this article, we provide an EM‐test solution for homogeneity for finite mixtures of location‐scale family distributions. This EM‐test has nonstandard limiting distributions, but we are able to find the critical values numerically. We use computer experiments to obtain appropriate values for the tuning parameters. A simulation study shows that the fine‐tuned EM‐test has close to nominal type I errors and very good power properties. Two application examples are included to demonstrate the performance of the EM‐test.
Decay estimates for large velocities in the Boltzmann equation without cutoff

We consider solutions $f=f(t,x,v)$ to the full (spatially inhomogeneous) Boltzmann equation with periodic spatial conditions $x \in \mathbb T^d$, for hard and moderately soft potentials \emph{without the angular cutoff assumption}, and under the \emph{a priori} assumption that the main hydrodynamic fields, namely the local mass $\int\_v f(t,x,v)$ and local energy $\int\_v f(t,x,v)|v|^2$ and local entropy $\int\_v f(t,x,v) \ln f(t,x,v)$, are controlled along time. We establish quantitative estimates of \emph{propagation} in time of "pointwise polynomial moments", i.e. $\sup\_{x,v} f(t,x,v) (1+|v|)^q$, $q \ge 0$. In the case of hard potentials, we also prove \emph{appearance} of these moments for all $q \ge 0$. In the case of moderately soft potentials we prove the \emph{appearance} of low-order pointwise moments.
Solutions to Sparse Multilevel Matrix Problems

We define and solve classes of sparse matrix problems that arise in multilevel modeling and data analysis. The classes are indexed by the number of nested units, with two-level problems corresponding to the common situation in which data on level 1 units are grouped within a two-level structure. We provide full solutions for two-level and three-level problems and their derivations provide blueprints for the challenging, albeit rarer in applications, higher level versions of the problem. Whilst our linear system solutions are a concise recasting of existing results, our matrix inverse sub-block results are novel and facilitate streamlined computation of standard errors in frequentist inference as well as allowing streamlined mean field variational Bayesian inference for models containing higher level random effects.
Power in High‐Dimensional Testing Problems

Fan, Liao, and Yao (2015) recently introduced a remarkable method for increasing the asymptotic power of tests in high‐dimensional testing problems. If applicable to a given test, their power enhancement principle leads to an improved test that has the same asymptotic size, has uniformly non‐inferior asymptotic power, and is consistent against a strictly broader range of alternatives than the initially given test. We study under which conditions this method can be applied and show the following: In asymptotic regimes where the dimensionality of the parameter space is fixed as sample size increases, there often exist tests that cannot be further improved with the power enhancement principle. However, when the dimensionality of the parameter space increases sufficiently slowly with sample size and a marginal local asymptotic normality (LAN) condition is satisfied, every test with asymptotic size smaller than 1 can be improved with the power enhancement principle. While the marginal LAN condition alone does not allow one to extend the latter statement to all rates at which the dimensionality increases with sample size, we give sufficient conditions under which this is the case.
The exponential distribution analog of the Grubbs–Weaver method

Abstract Grubbs and Weaver (1947) suggest a minimum-variance unbiased estimator for the population standard deviation of a normal random variable, where a random sample is drawn and a weighted sum of the ranges of subsamples is calculated. The optimal choice involves using as many subsamples of size eight as possible. They verified their results numerically for samples of size up to 100, and conjectured that their “rule of eights” is valid for all sample sizes. Here we examine the analogous problem where the underlying distribution is exponential and find that a “rule of fours” yields optimality and prove the result rigorously.
Uniform Hanson-Wright type concentration inequalities for unbounded entries via the entropy method

This paper is devoted to uniform versions of the Hanson-Wright inequality for a random vector $X \in \mathbb{R}^n$ with independent subgaussian components. The core technique of the paper is based on the entropy method combined with truncations of both gradients of functions of interest and of the coordinates of $X$ itself. Our results recover, in particular, the classic uniform bound of Talagrand (1996) for Rademacher chaoses and the more recent uniform result of Adamczak (2015), which holds under certain rather strong assumptions on the distribution of $X$. We provide several applications of our techniques: we establish a version of the standard Hanson-Wright inequality, which is tighter in some regimes. Extending our techniques we show a version of the dimension-free matrix Bernstein inequality that holds for random matrices with a subexponential spectral norm. We apply the derived inequality to the problem of covariance estimation with missing observations and prove an almost optimal high probability version of the recent result of Lounici (2014). Finally, we show a uniform Hanson-Wright type inequality in the Ising model under Dobrushin's condition. A closely related question was posed by Marton (2003).
Statistical matching of non-Gaussian data

The statistical matching problem is a data integration problem with structured missing data. The general form involves the analysis of multiple datasets that only have a strict subset of variables jointly observed across all datasets. The simplest version involves two datasets, labelled A and B, with three variables of interest $X, Y$ and $Z$. Variables $X$ and $Y$ are observed in dataset A and variables $X$ and $Z$ are observed in dataset $B$. Statistical inference is complicated by the absence of joint $(Y, Z)$ observations. Parametric modelling can be challenging due to identifiability issues and the difficulty of parameter estimation. We develop computationally feasible procedures for the statistical matching of non-Gaussian data using suitable data augmentation schemes and identifiability constraints. Nearest-neighbour imputation is a common alternative technique due to its ease of use and generality. Nearest-neighbour matching is based on a conditional independence assumption that may be inappropriate for non-Gaussian data. The violation of the conditional independence assumption can lead to improper imputations. We compare model based approaches to nearest-neighbour imputation on a number of flow cytometry datasets and find that the model based approach can address some of the weaknesses of the nonparametric nearest-neighbour technique.
Prediction and estimation consistency of sparse multi-class penalized optimal scoring

Sparse linear discriminant analysis via penalized optimal scoring is a successful tool for classification in high-dimensional settings. While the variable selection consistency of sparse optimal scoring has been established, the corresponding prediction and estimation consistency results have been lacking. We bridge this gap by providing probabilistic bounds on out-of-sample prediction error and estimation error of multi-class penalized optimal scoring allowing for diverging number of classes.
Multithreshold change plane model: Estimation theory and applications in subgroup identification

We propose a multithreshold change plane regression model which naturally partitions the observed subjects into subgroups with different covariate effects. The underlying grouping variable is a linear function of observed covariates and thus multiple thresholds produce change planes in the covariate space. We contribute a novel two‐stage estimation approach to determine the number of subgroups, the location of thresholds, and all other regression parameters. In the first stage we adopt a group selection principle to consistently identify the number of subgroups, while in the second stage change point locations and model parameter estimates are refined by a penalized induced smoothing technique. Our procedure allows sparse solutions for relatively moderate‐ or high‐dimensional covariates. We further establish the asymptotic properties of our proposed estimators under appropriate technical conditions. We evaluate the performance of the proposed methods by simulation studies and provide illustrations using two medical data examples. Our proposal for subgroup identification may lead to an immediate application in personalized medicine.
Frame-constrained total variation regularization for white noise regression

Despite the popularity and practical success of total variation (TV) regularization for function estimation, surprisingly little is known about its theoretical performance in a statistical setting. While TV regularization has been known for quite some time to be minimax optimal for denoising one-dimensional signals, for higher dimensions this remains elusive until today. In this paper we consider frame-constrained TV estimators including many well-known (overcomplete) frames in a white noise regression model, and prove their minimax optimality w.r. t. $L^q$-risk ($1<q\leq \frac{d+2}{d}\wedge 2$) up to a logarithmic factor in any dimension $d\geq 1$. Overcomplete frames are an established tool in mathematical imaging and signal recovery, and their combination with TV regularization has been shown to give excellent results in practice, which our theory now confirms. Our results rely on a new interpolation inequality to relate the frames and certain Besov spaces to the risk functional.
A simulation-based approach to estimate joint model of longitudinal and event-time data with many missing longitudinal observations

Joint models of longitudinal and event-time data have been extensively studied and applied in many different fields. Estimation of joint models is challenging, most present procedures are computational expensive and have a strict requirement on data quality. In this study, a novel simulation-based procedure is proposed to estimate a general family of joint models, which include many widely-applied joint models as special cases. Our procedure can easily handle low-quality data where longitudinal observations are systematically missed for some of the covariate dimensions. In addition, our estimation procedure is compatible with parallel computing framework when combining with stochastic descending algorithm, it is perfectly applicable to massive data and therefore suitable for many financial applications. Consistency and asymptotic normality of our estimator are proved, a simulation study is conducted to illustrate its effectiveness. Finally, as an application, the procedure is applied to estimate pre-payment probability of a massive consumer-loan dataset drawn from one biggest P2P loan platform of China.