Miracle-dz/UniFAR-FTUs huggingface dataset:
Minimax Optimal Convergence Rates for Estimating Ground Truth from Crowdsourced Labels

Crowdsourcing has become a primary means for label collection in many real-world machine learning applications. A classical method for inferring the true labels from the noisy labels provided by crowdsourcing workers is Dawid-Skene estimator. In this paper, we prove convergence rates of a projected EM algorithm for the Dawid-Skene estimator. The revealed exponent in the rate of convergence is shown to be optimal via a lower bound argument. Our work resolves the long standing issue of whether Dawid-Skene estimator has sound theoretical guarantees besides its good performance observed in practice. In addition, a comparative study with majority voting illustrates both advantages and pitfalls of the Dawid-Skene estimator.
Gaussian graphical model estimation with false discovery rate control

This paper studies the estimation of high dimensional Gaussian graphical model (GGM). Typically, the existing methods depend on regularization techniques. As a result, it is necessary to choose the regularized parameter. However, the precise relationship between the regularized parameter and the number of false edges in GGM estimation is unclear. Hence, it is impossible to evaluate their performance rigorously. In this paper, we propose an alternative method by a multiple testing procedure. Based on our new test statistics for conditional dependence, we propose a simultaneous testing procedure for conditional dependence in GGM. Our method can control the false discovery rate (FDR) asymptotically. The numerical performance of the proposed method shows that our method works quite well.
Bayesian Probabilistic Matrix Factorization: A User Frequency Analysis

Matrix factorization (MF) has become a common approach to collaborative filtering, due to ease of implementation and scalability to large data sets. Two existing drawbacks of the basic model is that it does not incorporate side information on either users or items, and assumes a common variance for all users. We extend the work of constrained probabilistic matrix factorization by deriving the Gibbs updates for the side feature vectors for items (Salakhutdinov and Minh, 2008). We show that this Bayesian treatment to the constrained PMF model outperforms simple MAP estimation. We also consider extensions to heteroskedastic precision introduced in the literature (Lakshminarayanan, Bouchard, and Archambeau, 2011). We show that this tends result in overfitting for deterministic approximation algorithms (ex: Variational inference) when the observed entries in the user / item matrix are distributed in an non-uniform manner. In light of this, we propose a truncated precision model. Our experimental results suggest that this model tends to delay overfitting.
Accelerating inference for diffusions observed with measurement error and large sample sizes using approximate Bayesian computation

In recent years, dynamical modelling has been provided with a range of breakthrough methods to perform exact Bayesian inference. However, it is often computationally unfeasible to apply exact statistical methodologies in the context of large data sets and complex models. This paper considers a nonlinear stochastic differential equation model observed with correlated measurement errors and an application to protein folding modelling. An approximate Bayesian computation (ABC)-MCMC algorithm is suggested to allow inference for model parameters within reasonable time constraints. The ABC algorithm uses simulations of ‘subsamples’ from the assumed data-generating model as well as a so-called ‘early-rejection’ strategy to speed up computations in the ABC-MCMC sampler. Using a considerate amount of subsamples does not seem to degrade the quality of the inferential results for the considered applications. A simulation study is conducted to compare our strategy with exact Bayesian inference, the latter resulting two orders of magnitude slower than ABC-MCMC for the considered set-up. Finally, the ABC algorithm is applied to a large size protein data. The suggested methodology is fairly general and not limited to the exemplified model and data.
Dictionary LASSO: Guaranteed Sparse Recovery under Linear Transformation

We consider the following signal recovery problem: given a measurement matrix $\Phi\in \mathbb{R}^{n\times p}$ and a noisy observation vector $c\in \mathbb{R}^{n}$ constructed from $c = \Phi\theta^* + \epsilon$ where $\epsilon\in \mathbb{R}^{n}$ is the noise vector whose entries follow i.i. d. centered sub-Gaussian distribution, how to recover the signal $\theta^*$ if $D\theta^*$ is sparse {\rca under a linear transformation} $D\in\mathbb{R}^{m\times p}$? One natural method using convex optimization is to solve the following problem: $$\min_{\theta} {1\over 2}\|\Phi\theta - c\|^2 + \lambda\|D\theta\|_1. $$ This paper provides an upper bound of the estimate error and shows the consistency property of this method by assuming that the design matrix $\Phi$ is a Gaussian random matrix. Specifically, we show 1) in the noiseless case, if the condition number of $D$ is bounded and the measurement number $n\geq \Omega(s\log(p))$ where $s$ is the sparsity number, then the true solution can be recovered with high probability; and 2) in the noisy case, if the condition number of $D$ is bounded and the measurement increases faster than $s\log(p)$, that is, $s\log(p)=o(n)$, the estimate error converges to zero with probability 1 when $p$ and $s$ go to infinity. Our results are consistent with those for the special case $D=\bold{I}_{p\times p}$ (equivalently LASSO) and improve the existing analysis. The condition number of $D$ plays a critical role in our analysis. We consider the condition numbers in two cases including the fused LASSO and the random graph: the condition number in the fused LASSO case is bounded by a constant, while the condition number in the random graph case is bounded with high probability if $m\over p$ (i.e., $#text{edge}\over #text{vertex}$) is larger than a certain constant. Numerical simulations are consistent with our theoretical results.
Bayesian Nonparametric Modeling of Higher Order Markov Chains

ABSTRACT We consider the problem of flexible modeling of higher order Markov chains when an upper bound on the order of the chain is known but the true order and nature of the serial dependence are unknown. We propose Bayesian nonparametric methodology based on conditional tensor factorizations, which can characterize any transition probability with a specified maximal order. The methodology selects the important lags and captures higher order interactions among the lags, while also facilitating calculation of Bayes factors for a variety of hypotheses of interest. We design efficient Markov chain Monte Carlo algorithms for posterior computation, allowing for uncertainty in the set of important lags to be included and in the nature and order of the serial dependence. The methods are illustrated using simulation experiments and real world applications. Supplementary materials for this article are available online.
Connection graph Laplacian methods can be made robust to noise

Recently, several data analytic techniques based on connection graph laplacian (CGL) ideas have appeared in the literature. At this point, the properties of these methods are starting to be understood in the setting where the data is observed without noise. We study the impact of additive noise on these methods, and show that they are remarkably robust. As a by-product of our analysis, we propose modifications of the standard algorithms that increase their robustness to noise. We illustrate our results in numerical simulations.
Asymptotic representation theorems for poverty indices

We set general conditions under which the general poverty index, which summarizes all the available indices, is asymptotically represented with some empirical processes. This representation theorem offers a general key, in most directions, for the asymptotic of the bulk of poverty indices and issues in poverty analysis. Our representation results uniformly hold on a large collection of poverty indices. They enable the continuous measure of poverty with longitudinal data. R esum e. Nous introduisons un ensemble coherent de conditions sous lesquelles, l'Index General de Pauvrete, qui englobe presque tous les indices usuels, admet une representation asymptptique sous forme de somme d'un processus empirique fonctionnel et d'un processus empirique special. Cette representation asymptotique offre un ensemble d'outils permettant de trouver rapidement les lois asymptotiques des mesures de pauvrete. Elle est en general utile pour l'analyse de la pauvrete. Elle est donnee de maniere uniforme par rapport au temps etla fonctionnelle de la mesure. Elle est appropriee pour l'´ etude de la pauvrete longitudinale.
Estimation for ultra-high dimensional factor model: a pivotal variable detection based approach

For factor model, the involved covariance matrix often has no row sparse structure because the common factors may lead some variables to strongly associate with many others. Under the ultra-high dimensional paradigm, this feature causes existing methods for sparse covariance matrix in the literature not directly applicable. In this paper, for general covariance matrix, a novel approach to detect these variables that is called the pivotal variables is suggested. Then, two-stage estimation procedures are proposed to handle ultra-high dimensionality in factor model. In these procedures, pivotal variable detection is performed as a screening step and then existing approaches are applied to refine the working model. The estimation efficiency can be promoted under weaker assumptions on the model structure. Simulations are conducted to examine the performance of the new method and a real dataset is analysed for illustration.
A comparison of multiple imputation methods for bivariate hierarchical outcomes

Missing observations are common in cluster randomised trials. Approaches taken to handling such missing data include: complete case analysis, single-level multiple imputation that ignores the clustering, multiple imputation with a fixed effect for each cluster and multilevel multiple imputation. We conducted a simulation study to assess the performance of these approaches, in terms of confidence interval coverage and empirical bias in the estimated treatment effects. Missing-at-random clustered data scenarios were simulated following a full-factorial design. An Analysis of Variance was carried out to study the influence of the simulation factors on each performance measure. When the randomised treatment arm was associated with missingness, complete case analysis resulted in biased treatment effect estimates. Across all the missing data mechanisms considered, the multiple imputation methods provided estimators with negligible bias. Confidence interval coverage was generally in excess of nominal levels (up to 99.8%) following fixed-effects multiple imputation, and too low following single-level multiple imputation. Multilevel multiple imputation led to coverage levels of approximately 95% throughout. The approach to handling missing data was the most influential factor on the bias and coverage. Within each method, the most important factors were the number and size of clusters, and the intraclass correlation coefficient.
Breakdown Properties of the M-Estimators of Multivariate Scatter

The M-estimators of multivariate scatter are known to have breakdown points no greater than 1/(p+1), where p is the dimension of the data. In high dimension, the breakdown points are usually considered to be disappointingly low. This paper studies the breakdown problem in more detail. The exact breakdown points for the M-estimators of scatter are obtained and it is shown that their low values are primarily due to contamination restricted to some plane. If such "coplanar" contamination is not present, then there exists M-estimators which have breakdown points close to 1/2. The effect of coplanar contamination is further examined and is shown to be related to the singularity of the scatter matrix. Finally, the implications of the results of this paper on whether the low breakdown point is necessarily a bad feature and on multivariate outlier detection are briefly discussed. This paper is a reprint of an unpublished 1986 Rutgers Technical Report.
Oracle Inequalities for High Dimensional Vector Autoregressions

This paper establishes non-asymptotic oracle inequalities for the prediction error and estimation accuracy of the LASSO in stationary vector autoregressive models. These inequalities are used to establish consistency of the LASSO even when the number of parameters is of a much larger order of magnitude than the sample size. Furthermore, it is shown that under suitable conditions the number of variables selected is of the right order of magnitude and that no relevant variables are excluded. Next, non-asymptotic probabilities are given for the Adaptive LASSO to select the correct sign pattern (and hence the correct sparsity pattern). Finally conditions under which the Adaptive LASSO reveals the correct sign pattern with probability tending to one are given. Again, the number of parameters may be much larger than the sample size. Some maximal inequalities for vector autoregressions which might be of independent interest are contained in the appendix.
On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA

This work provides a unified analysis of the properties of the sample covariance matrix $\Sigma_n$ over the class of $p\times p$ population covariance matrices $\Sigma$ of reduced effective rank $r_e(\Sigma)$. This class includes scaled factor models and covariance matrices with decaying spectrum. We consider $r_e(\Sigma)$ as a measure of matrix complexity, and obtain sharp minimax rates on the operator and Frobenius norm of $\Sigma_n-\Sigma$, as a function of $r_e(\Sigma)$ and $\|\Sigma\|_2$, the operator norm of $\Sigma$. With guidelines offered by the optimal rates, we define classes of matrices of reduced effective rank over which $\Sigma_n$ is an accurate estimator. Within the framework of these classes, we perform a detailed finite sample theoretical analysis of the merits and limitations of the empirical scree plot procedure routinely used in PCA. We show that identifying jumps in the empirical spectrum that consistently estimate jumps in the spectrum of $\Sigma$ is not necessarily informative for other goals, for instance for the selection of those sample eigenvalues and eigenvectors that are consistent estimates of their population counterparts. The scree plot method can still be used for selecting consistent eigenvalues, for appropriate threshold levels. We provide a threshold construction and also give a rule for checking the consistency of the corresponding sample eigenvectors. We specialize these results and analysis to population covariance matrices with polynomially decaying spectra, and extend it to covariance operators with polynomially decaying spectra. An application to fPCA illustrates how our results can be used in functional data analysis.
Treatment Effects on Ordinal Outcomes: Causal Estimands and Sharp Bounds

Assessing the causal effects of interventions on ordinal outcomes is an important objective of many educational and behavioral studies. Under the potential outcomes framework, we can define causal effects as comparisons between the potential outcomes under treatment and control. However, unfortunately, the average causal effect, often the parameter of interest, is difficult to interpret for ordinal outcomes. To address this challenge, we propose to use two causal parameters, which are defined as the probabilities that the treatment is beneficial and strictly beneficial for the experimental units. However, although well-defined for any outcomes and of particular interest for ordinal outcomes, the two aforementioned parameters depend on the association between the potential outcomes and are therefore not identifiable from the observed data without additional assumptions. Echoing recent advances in the econometrics and biostatistics literature, we present the sharp bounds of the aforementioned causal parameters for ordinal outcomes, under fixed marginal distributions of the potential outcomes. Because the causal estimands and their corresponding sharp bounds are based on the potential outcomes themselves, the proposed framework can be flexibly incorporated into any chosen models of the potential outcomes and is directly applicable to randomized experiments, unconfounded observational studies, and randomized experiments with noncompliance. We illustrate our methodology via numerical examples and three real-life applications related to educational and behavioral research.
A simple modification for improving inference of non-linear dynamical systems

Particle and ensemble filters are increasingly utilized for inference, optimization, and forecast; however, both filtering methods use discrete distributions to simulate continuous state space, a drawback that can lead to degraded performance for non-linear dynamical systems. Here we propose a simple modification, applicable to both particle and ensemble filters, that compensates for this problem. The method randomly replaces one or more model variables or parameters within a fraction of simulated trajectories at each filtering cycle. This modification, termed space re-probing, expands the state space covered by the filter through the introduction of outlying trajectories. We apply the space re-probing modification to three particle filters and three ensemble filters, and use these modified filters to model and forecast influenza epidemics. For both filter types, the space re-probing improves simulation of influenza epidemic curves and the prediction of influenza outbreak peak timing. Further, as fewer particles are needed for the particle filters, the proposed modification reduces the computational cost of these filters.
BADER: Bayesian analysis of differential expression in RNA sequencing data

Identifying differentially expressed genes from RNA sequencing data remains a challenging task because of the considerable uncertainties in parameter estimation and the small sample sizes in typical applications. Here we introduce Bayesian Analysis of Differential Expression in RNA-sequencing data (BADER). Due to our choice of data and prior distributions, full posterior inference for BADER can be carried out efficiently. The method appropriately takes uncertainty in gene variance into account, leading to higher power than existing methods in detecting differentially expressed genes. Moreover, we show that the posterior samples can be naturally integrated into downstream gene set enrichment analyses, with excellent performance in detecting enriched sets. An open-source R package (BADER) that provides a user-friendly interface to a C++ back-end is available on Bioconductor.
State-space modeling of dynamic genetic networks

The genomic reality is a highly complex and dynamic system. The recent development of high-throughput technologies has enabled researchers to measure the abundance of many genes (in the order of thousands) simultaneously. The challenge is to unravel from such measurements, gene/protein or gene/gene or protein/ protein interactions and key biological features of cellular systems. Our goal is to devise a method for inferring transcriptional or gene regulatory networks from high-throughput data sources such as gene expression microarrays with potentially hidden states, such as unmeasured transcription factors (TFs), which satisfies certain Markov properties. We propose a dynamic state space representation. Our method is based on an EM algorithm with an incorporated Kalman smoothing algorithm in the E-step, a bootstrap for confidence intervals to infer the networks and the AIC for model selection. The state space model is an approach with proven effectiveness to reverse engineer transcriptional networks. The proposed method is applied to time course microarray data obtained from well established T-cell. When we applied the method to the T-cell data, we obtained 4, as the optimum number of hidden states. Our results support interesting biological properties in the family of Jun genes. The following genes were mostly seen as regulatory genes. These genes includes FYB, CCNA2, AKT1, TRAF5, CASP4, and CTNNB1. We found interaction between Jun-B and SMN1, and CDC2 activates Jun-D. We found few significant interactions or one-to-one correspondence among the 4 putative transcription factors. Among the important key genes in terms of outward-directed edges, we found genes such as CCNA2, JUNB, CDC2, CASP4, JUND to have a high degree of connectivity. R Computer source code is made available at our website at this http URL
On the Prediction Performance of the Lasso

Although the Lasso has been extensively studied, the relationship between its prediction performance and the correlations of the covariates is not fully understood. In this paper, we give new insights into this relationship in the context of multiple linear regression. We show, in particular, that the incorporation of a simple correlation measure into the tuning parameter can lead to a nearly optimal prediction performance of the Lasso even for highly correlated covariates. However, we also reveal that for moderately correlated covariates, the prediction performance of the Lasso can be mediocre irrespective of the choice of the tuning parameter. We finally show that our results also lead to near-optimal rates for the least-squares estimator with total variation penalty.
Statistical estimation of jump rates for a specific class of Piecewise Deterministic Markov Processes

We consider the class of Piecewise Deterministic Markov Processes (PDMP), whose state space is $\R\_{+}^{*}$, that possess an increasing deterministic motion and that shrink deterministically when they jump. Well known examples for this class of processes are Transmission Control Protocol (TCP) window size process and the processes modeling the size of a "marked" {\it Escherichia coli} cell. Having observed the PDMP until its $n$th jump, we construct a nonparametric estimator of the jump rate $\lambda$. Our main result is that for $D$ a compact subset of $\R\_{+}^{*}$, if $\lambda$ is in the H{\''{o}}lder space ${\mathcal H}^s({\mathcal D})$, the squared-loss error of the estimator is asymptotically close to the rate of $n^{-s/(2s+1)}$. Simulations illustrate the behavior of our estimator.
Variable selection in high-dimensional additive models based on norms of projections

We consider the problem of variable selection in high-dimensional sparse additive models. We focus on the case that the components belong to nonparametric classes of functions. The proposed method is motivated by geometric considerations in Hilbert spaces and consists of comparing the norms of the projections of the data onto various additive subspaces. Under minimal geometric assumptions, we prove concentration inequalities which lead to new conditions under which consistent variable selection is possible. As an application, we establish conditions under which a single component can be estimated with the rate of convergence corresponding to the situation in which the other components are known.