Miracle-dz/UniFAR-FTUs huggingface dataset:
Predicting Race and Ethnicity From the Sequence of Characters in a Name

To answer questions about racial inequality, we often need a way to infer race and ethnicity from a name. Until now, a bulk of the focus has been on optimally exploiting the last names list provided by the Census Bureau. But there is more information in the first names, especially for African Americans. To estimate the relationship between full names and race, we exploit the Florida voter registration data and the Wikipedia data. In particular, we model the relationship between the sequence of characters in a name, and race and ethnicity using Long Short Term Memory Networks. Our out of sample (OOS) precision and recall for the full name model estimated on the Florida Voter Registration data is . 83 and . 84 respectively. This compares to OOS precision and recall of . 79 and . 81 for the last name only model. Commensurate numbers for Wikipedia data are . 73 and . 73 for the full name model and . 66 and . 67 for the last name model. To illustrate the use of this method, we apply our method to the campaign finance data to estimate the share of donations made by people of various racial groups.
Change-point detection in high-dimensional covariance structure

In this paper we introduce a novel approach for an important problem of break detection. Specifically, we are interested in detection of an abrupt change in the covariance structure of a high-dimensional random process -- a problem, which has applications in many areas e.g., neuroimaging and finance. The developed approach is essentially a testing procedure involving a choice of a critical level. To that end a non-standard bootstrap scheme is proposed and theoretically justified under mild assumptions. Theoretical study features a result providing guaranties for break detection. All the theoretical results are established in a high-dimensional setting (dimensionality $p \gg n$). Multiscale nature of the approach allows for a trade-off between sensitivity of break detection and localization. The approach can be naturally employed in an on-line setting. Simulation study demonstrates that the approach matches the nominal level of false alarm probability and exhibits high power, outperforming a recent approach.
A perturbation analysis of some Markov chains models with time-varying parameters

We study some regularity properties in locally stationary Markov models which are fundamental for controlling the bias of nonparametric kernel estimators. In particular, we provide an alternative to the standard notion of derivative process developed in the literature and that can be used for studying a wide class of Markov processes. To this end, for some families of V-geometrically ergodic Markov kernels indexed by a real parameter u, we give conditions under which the invariant probability distribution is differentiable with respect to u, in the sense of signed measures. Our results also complete the existing literature for the perturbation analysis of Markov chains, in particular when exponential moments are not finite. Our conditions are checked on several original examples of locally stationary processes such as integer-valued autoregressive processes, categorical time series or threshold autoregressive processes.
Statistical clustering of temporal networks through a dynamic stochastic block model

Statistical node clustering in discrete time dynamic networks is an emerging field that raises many challenges. Here, we explore statistical properties and frequentist inference in a model that combines a stochastic block model for its static part with independent Markov chains for the evolution of the nodes groups through time. We model binary data as well as weighted dynamic random graphs (with discrete or continuous edges values). Our approach, motivated by the importance of controlling for label switching issues across the different time steps, focuses on detecting groups characterized by a stable within‐group connectivity behaviour. We study identifiability of the model parameters and propose an inference procedure based on a variational expectation–maximization algorithm as well as a model selection criterion to select the number of groups. We carefully discuss our initialization strategy which plays an important role in the method and we compare our procedure with existing procedures on synthetic data sets. We also illustrate our approach on dynamic contact networks: one of encounters between high school students and two others on animal interactions. An implementation of the method is available as an R package called dynsbm.
A local ensemble transform Kalman particle filter for convective‐scale data assimilation

Ensemble data assimilation methods such as the ensemble Kalman filter (EnKF) are a key component of probabilistic weather forecasting. They represent the uncertainty in the initial conditions by an ensemble that incorporates information coming from the physical model with the latest observations. High‐resolution numerical weather prediction models run at operational centres are able to resolve nonlinear and non‐Gaussian physical phenomena such as convection. There is therefore a growing need to develop ensemble assimilation algorithms able to deal with non‐Gaussianity while staying computationally feasible. In the present article, we address some of these needs by proposing a new hybrid algorithm based on the ensemble Kalman particle filter. It is fully formulated in ensemble space and uses a deterministic scheme such that it has the ensemble transform Kalman filter (ETKF) instead of the stochastic EnKF as a limiting case. A new criterion for choosing the proportion of particle filter and ETKF updates is also proposed. The new algorithm is implemented in the Consortium for Small‐scale Modeling (COSMO) framework and numerical experiments in a quasi‐operational convective‐scale set‐up are conducted. The results show the feasibility of the new algorithm in practice and indicate the strong potential of such local hybrid methods, in particular for forecasting non‐Gaussian variables such as wind and hourly precipitation.
2 3 O ct 2 01 8 MULTILINEAR COMPRESSIVE SENSING AND AN APPLICATION TO CONVOLUTIONAL LINEAR NETWORKS

We study a deep linear network endowed with a structure. It takes the form of a matrix X obtained by multiplying K matrices (called factors and corresponding to the action of the layers). The action of each layer (i.e. a factor) is obtained by applying a ﬁxed linear operator to a vector of parameters satisfying a constraint. The number of layers is not limited. Assuming that X is given and factors have been estimated, the error between the product of the estimated factors and X (i.e. the reconstruction error) is either the statistical or the empirical risk. In this paper, we provide necessary and suﬃcient conditions on the network topology under which a stability property holds. The stability property requires that the error on the parameters deﬁning the factors (i.e. the stability of the recovered parameters) scales linearly with the reconstruction error (i.e. the risk). Therefore, under these conditions on the network topology, any successful learning task leads to stably deﬁned features and therefore interpretable layers/network. In order to do so, we ﬁrst evaluate how the Segre embedding and its inverse distort distances. Then, we show that any deep structured linear network can be cast as a generic multilinear problem (that uses the Segre embedding). This is the tensorial lifting . Using the tensorial lifting, we provide necessary and suﬃcient conditions for the identiﬁability of the factors (up to a scale rearrangement). We ﬁnally provide the necessary and suﬃcient condition called deep-Null Space Property (because of the analogy with the usual Null Space Property in the compressed sensing framework) which guarantees that the stability property holds. We illustrate the theory with a practical example where the deep structured linear network is a convolutional linear network. As expected, the conditions are rather strong but not empty. A simple test on the network topology can be implemented to test if the condition holds.
Bayesian Hierarchical Models With Conjugate Full-Conditional Distributions for Dependent Data From the Natural Exponential Family

Abstract We introduce a Bayesian approach for analyzing (possibly) high-dimensional dependent data that are distributed according to a member from the natural exponential family of distributions. This problem requires extensive methodological advancements, as jointly modeling high-dimensional dependent data leads to the so-called “big n problem. ” The computational complexity of the “big n problem” is further exacerbated when allowing for non-Gaussian data models, as is the case here. Thus, we develop new computationally efficient distribution theory for this setting. In particular, we introduce the “conjugate multivariate distribution,” which is motivated by the Diaconis and Ylvisaker distribution. Furthermore, we provide substantial theoretical and methodological development including: results regarding conditional distributions, an asymptotic relationship with the multivariate normal distribution, conjugate prior distributions, and full-conditional distributions for a Gibbs sampler. To demonstrate the wide-applicability of the proposed methodology, we provide two simulation studies and three applications based on an epidemiology dataset, a federal statistics dataset, and an environmental dataset, respectively. Supplementary materials for this article are available online.
Adaptive estimation in structured factor models with applications to overlapping clustering

This work introduces a novel estimation method, called LOVE, of the entries and structure of a loading matrix A in a sparse latent factor model X = AZ + E, for an observable random vector X in Rp, with correlated unobservable factors Z \in RK, with K unknown, and independent noise E. Each row of A is scaled and sparse. In order to identify the loading matrix A, we require the existence of pure variables, which are components of X that are associated, via A, with one and only one latent factor. Despite the fact that the number of factors K, the number of the pure variables, and their location are all unknown, we only require a mild condition on the covariance matrix of Z, and a minimum of only two pure variables per latent factor to show that A is uniquely defined, up to signed permutations. Our proofs for model identifiability are constructive, and lead to our novel estimation method of the number of factors and of the set of pure variables, from a sample of size n of observations on X. This is the first step of our LOVE algorithm, which is optimization-free, and has low computational complexity of order p2. The second step of LOVE is an easily implementable linear program that estimates A. We prove that the resulting estimator is minimax rate optimal up to logarithmic factors in p. The model structure is motivated by the problem of overlapping variable clustering, ubiquitous in data science. We define the population level clusters as groups of those components of X that are associated, via the sparse matrix A, with the same unobservable latent factor, and multi-factor association is allowed. Clusters are respectively anchored by the pure variables, and form overlapping sub-groups of the p-dimensional random vector X. The Latent model approach to OVErlapping clustering is reflected in the name of our algorithm, LOVE.
Bayesian nonparametric forecasting of monotonic functional time series

We propose a Bayesian nonparametric approach to modelling and predicting a class of functional time series with application to energy markets, based on fully observed, noise-free functional data. Traders in such contexts conceive profitable strategies if they can anticipate the impact of their bidding actions on the aggregate demand and supply curves, which in turn need to be predicted reliably. Here we propose a simple Bayesian nonparametric method for predicting such curves, which take the form of monotonic bounded step functions. We borrow ideas from population genetics by defining a class of interacting particle systems to model the functional trajectory, and develop an implementation strategy which uses ideas from Markov chain Monte Carlo and approximate Bayesian computation techniques and allows to circumvent the intractability of the likelihood. Our approach shows great adaptation to the degree of smoothness of the curves and the volatility of the functional series, proves to be robust to an increase of the forecast horizon and yields an uncertainty quantification for the functional forecasts. We illustrate the model and discuss its performance with simulated datasets and on real data relative to the Italian natural gas market.
Analysis of the Gibbs Sampler for Gaussian hierarchical models via multigrid decomposition

We study the convergence properties of the Gibbs Sampler in the context of posterior distributions arising from Bayesian analysis of Gaussian hierarchical models. We consider centred and non-centred parameterizations as well as their hybrids including the full family of partially non-centred parameterizations. We develop a novel methodology based on multi-grid decompositions to derive analytic expressions for the convergence rates of the algorithm for an arbitrary number of layers in the hierarchy, while previous work was typically limited to the two-level case. Our work gives a complete understanding for the three-level symmetric case and this gives rise to approximations for the non-symmetric case. We also give analogous, if less explicit, results for models of arbitrary level. This theory gives rise to simple and easy-to-implement guidelines for the practical implementation of Gibbs samplers on conditionally Gaussian hierarchical models.
On Reject and Refine Options in Multicategory Classification

ABSTRACT In many real applications of statistical learning, a decision made from misclassification can be too costly to afford; in this case, a reject option, which defers the decision until further investigation is conducted, is often preferred. In recent years, there has been much development for binary classification with a reject option. Yet, little progress has been made for the multicategory case. In this article, we propose margin-based multicategory classification methods with a reject option. In addition, and more importantly, we introduce a new and unique refine option for the multicategory problem, where the class of an observation is predicted to be from a set of class labels, whose cardinality is not necessarily one. The main advantage of both options lies in their capacity of identifying error-prone observations. Moreover, the refine option can provide more constructive information for classification by effectively ruling out implausible classes. Efficient implementations have been developed for the proposed methods. On the theoretical side, we offer a novel statistical learning theory and show a fast convergence rate of the excess ℓ-risk of our methods with emphasis on diverging dimensionality and number of classes. The results can be further improved under a low noise assumption and be generalized to the excess 0-d-1 risk. Finite-sample upper bounds for the reject and reject/refine rates are also provided. A set of comprehensive simulation and real data studies has shown the usefulness of the new learning tools compared to regular multicategory classifiers. Detailed proofs of theorems and extended numerical results are included in the supplemental materials available online.
On estimation of isotonic piecewise constant signals

Consider a sequence of real data points $X_1,\ldots, X_n$ with underlying means $\theta^*_1,\dots,\theta^*_n$. This paper starts from studying the setting that $\theta^*_i$ is both piecewise constant and monotone as a function of the index $i$. For this, we establish the exact minimax rate of estimating such monotone functions, and thus give a non-trivial answer to an open problem in the shape-constrained analysis literature. The minimax rate involves an interesting iterated logarithmic dependence on the dimension, a phenomenon that is revealed through characterizing the interplay between the isotonic shape constraint and model selection complexity. We then develop a penalized least-squares procedure for estimating the vector $\theta^*=(\theta^*_1,\dots,\theta^*_n)^T$. This estimator is shown to achieve the derived minimax rate adaptively. For the proposed estimator, we further allow the model to be misspecified and derive oracle inequalities with the optimal rates, and show there exists a computationally efficient algorithm to compute the exact solution.
A Constructive Approach to High-dimensional Regression

We develop a constructive approach to estimating sparse, high-dimensional linear regression models. The approach is a computational algorithm motivated from the KKT conditions for the $\ell_0$-penalized least squares solutions. It generates a sequence of solutions iteratively, based on support detection using primal and dual information and root finding. We refer to the algorithm as SDAR for brevity. Under a sparse Rieze condition on the design matrix and certain other conditions, we show that with high probability, the $\ell_2$ estimation error of the solution sequence decays exponentially to the minimax error bound in $O(\sqrt{J}\log(R))$ steps; and under a mutual coherence condition and certain other conditions, the $\ell_{\infty}$ estimation error decays to the optimal error bound in $O(\log(R))$ steps, where $J$ is the number of important predictors, $R$ is the relative magnitude of the nonzero target coefficients. Computational complexity analysis shows that the cost of SDAR is $O(np)$ per iteration. Moreover the oracle least squares estimator can be exactly recovered with high probability at the same cost if we know the sparsity level. We also consider an adaptive version of SDAR to make it more practical in applications. Numerical comparisons with Lasso, MCP and greedy methods demonstrate that SDAR is competitive with or outperforms them in accuracy and efficiency.
Machine learning by unitary tensor network of hierarchical tree structure

The resemblance between the methods used in quantum-many body physics and in machine learning has drawn considerable attention. In particular, tensor networks (TNs) and deep learning architectures bear striking similarities to the extent that TNs can be used for machine learning. Previous results used one-dimensional TNs in image recognition, showing limited scalability and flexibilities. In this work, we train two-dimensional hierarchical TNs to solve image recognition problems, using a training algorithm derived from the multi-scale entanglement renormalization ansatz. This approach introduces mathematical connections among quantum many-body physics, quantum information theory, and machine learning. While keeping the TN unitary in the training phase, TN states are defined, which encode classes of images into quantum many-body states. We study the quantum features of the TN states, including quantum entanglement and fidelity. We find these quantities could be properties that characterize the image classes, as well as the machine learning tasks.
Empirical regression quantile process with possible application to risk analysis

The processes of the averaged regression quantiles and of their modifications provide useful tools in the regression models when the covariates are not fully under our control. As an application we mention the probabilistic risk assessment in the situation when the return depends on some exogenous variables. The processes enable to evaluate the expected $\alpha$-shortfall ($0\leq\alpha\leq 1$) and other measures of the risk, recently generally accepted in the financial literature, but also help to measure the risk in environment analysis and elsewhere.
An Improved Modified Cholesky Decomposition Method for Inverse Covariance Matrix Estimation

The modified Cholesky decomposition is commonly used for inverse covariance matrix estimation given a specified order of random variables. However, the order of variables is often not available or cannot be pre-determined. Hence, we propose a novel estimator to address the variable order issue in the modified Cholesky decomposition to estimate the sparse inverse covariance matrix. The key idea is to effectively combine a set of estimates obtained from multiple permutations of variable orders, and to efficiently encourage the sparse structure for the resultant estimate by the use of thresholding technique on the combined Cholesky factor matrix. The consistent property of the proposed estimate is established under some weak regularity conditions. Simulation studies show the superior performance of the proposed method in comparison with several existing approaches. We also apply the proposed method into the linear discriminant analysis for analyzing real-data examples for classification.
Split Regularized Regression

Abstract We propose an approach for fitting linear regression models that splits the set of covariates into groups. The optimal split of the variables into groups and the regularized estimation of the regression coefficients are performed by minimizing an objective function that encourages sparsity within each group and diversity among them. The estimated coefficients are then pooled together to form the final fit. Our procedure works on top of a given penalized linear regression estimator (e.g., Lasso, elastic net) by fitting it to possibly overlapping groups of features, encouraging diversity among these groups to reduce the correlation of the corresponding predictions. For the case of two groups, elastic net penalty and orthogonal predictors, we give a closed form solution for the regression coefficients in each group. We establish the consistency of our method with the number of predictors possibly increasing with the sample size. An extensive simulation study and real-data applications show that in general the proposed method improves the prediction accuracy of the base estimator used in the procedure. Possible extensions to GLMs and other models are discussed. The supplemental material for this article, available online, contains the proofs of our theoretical results and the full results of our simulation study.
QUANTILE TREATMENT EFFECTS IN REGRESSION KINK DESIGNS

The literature on regression kink designs develops identification results for average effects of continuous treatments (Nielsen et al., 2010, American Economic Journal: Economic Policy 2, 185–215; Card et al., 2015, Econometrica 83, 2453–2483), average effects of binary treatments (Dong, 2018, Jump or Kink? Identifying Education Effects by Regression Discontinuity Design without the Discontinuity), and quantile-wise effects of continuous treatments (Chiang and Sasaki, 2019, Journal of Econometrics 210, 405–433), but there has been no identification result for quantile-wise effects of binary treatments to date. In this article, we fill this void in the literature by providing an identification of quantile treatment effects in regression kink designs with binary treatment variables. For completeness, we also develop large sample theories for statistical inference, present a practical guideline on estimation and inference, conduct simulation studies, and provide an empirical illustration.
General notions of regression depth function

As a measure for the centrality of a point in a set of multivariate data, statistical depth functions play important roles in multivariate analysis, because one may conveniently construct descriptive as well as inferential procedures relying on them. Many depth notions have been proposed in the literature to fit to different applications. However, most of them are mainly developed for the location setting. In this paper, we discuss the possibility of extending some of them into the regression setting. A general concept of regression depth function is also provided.