Miracle-dz/UniFAR-FTUs huggingface dataset:
On $\sqrt{n}-$consistency for Bayesian quantile regression based on the misspecified asymmetric Laplace likelihood

The asymmetric Laplace density (ALD) is used as a working likelihood for Bayesian quantile regression. Sriram et al. (2013) derived posterior consistency for Bayesian linear quantile regression based on the misspecified ALD. While their paper also argued for $\sqrt{n}-$consistency, Sriram and Ramamoorthi (2017) highlighted that the argument was only valid for $n^{\alpha}$ rate for $\alpha<1/2$. However, $\sqrt{n}-$rate is necessary to carry out meaningful Bayesian inference based on the ALD. In this paper, we give sufficient conditions for $\sqrt{n}-$consistency in the more general setting of Bayesian non-linear quantile regression based on ALD. In particular, we derive $\sqrt{n}-$consistency for the Bayesian linear quantile regression. Our approach also enables an interesting extension of the linear case when number of parameters $p$ increases with $n$, where we obtain posterior consistency at the rate $n^{\alpha}$ for $\alpha<1/2$.
A Permutation Test on Complex Sample Data

Permutation tests are a distribution-free way of performing hypothesis tests. These tests rely on the condition that the observed data are exchangeable among the groups being tested under the null hypothesis. This assumption is easily satisfied for data obtained from a simple random sample or a controlled study after simple adjustments to the data, but there is no general method for adjusting survey data collected using a complex sample design to allow for permutation tests. In this article, we propose a general method for performing a pseudo-permutation test that accounts for the complex sample design. The proposed method is not a true permutation test in that the new values do not come from the set of observed values in general but of an expanded set of values satisfying a random-effects model on the clustered residuals of a design-consistent estimating equation. We provide a set of conditions under which this procedure leads to consistent test results. Tests using a simulated population and an application analyzing US Bureau of Labor Statistics consumer expenditure data comparing the performance of the proposed method to permutation tests that ignore the sample design demonstrate that it is necessary to account for the design features in order to obtain reasonable p value estimates.
Semi-Supervised Linear Regression

Abstract We study a regression problem where for some part of the data we observe both the label variable (Y) and the predictors ( ), while for other part of the data only the predictors are given. Such a problem arises, for example, when observations of the label variable are costly and may require a skilled human agent. When the conditional expectation is not exactly linear, one can consider the best linear approximation to the conditional expectation, which can be estimated consistently by the least-square estimates (LSE). The latter depends only on the labeled data. We suggest improved alternative estimates to the LSE that use also the unlabeled data. Our estimation method can be easily implemented and has simply described asymptotic properties. The new estimates asymptotically dominate the usual standard procedures under certain non-linearity condition of ; otherwise, they are asymptotically equivalent. The performance of the new estimator for small sample size is investigated in an extensive simulation study. A real data example of inferring homeless population is used to illustrate the new methodology.
Measurement error in continuous endpoints in randomised trials: Problems and solutions

In randomised trials, continuous endpoints are often measured with some degree of error. This study explores the impact of ignoring measurement error and proposes methods to improve statistical inference in the presence of measurement error. Three main types of measurement error in continuous endpoints are considered: classical, systematic, and differential. For each measurement error type, a corrected effect estimator is proposed. The corrected estimators and several methods for confidence interval estimation are tested in a simulation study. These methods combine information about error‐prone and error‐free measurements of the endpoint in individuals not included in the trial (external calibration sample). We show that, if measurement error in continuous endpoints is ignored, the treatment effect estimator is unbiased when measurement error is classical, while Type‐II error is increased at a given sample size. Conversely, the estimator can be substantially biased when measurement error is systematic or differential. In those cases, bias can largely be prevented and inferences improved upon using information from an external calibration sample, of which the required sample size increases as the strength of the association between the error‐prone and error‐free endpoint decreases. Measurement error correction using already a small (external) calibration sample is shown to improve inferences and should be considered in trials with error‐prone endpoints. Implementation of the proposed correction methods is accommodated by a new software package for R.
Heteroskedasticity-robust inference in linear regression models

This paper considers inference in heteroskedastic linear regression models with many control variables. The slope coefficients on these variables are nuisance parameters. Our setting allows their number to grow with the sample size, possibly at the same rate, in which case they are not consistently estimable. A prime example of this setting are models with many (possibly multi-way) fixed effects. The presence of many nuisance parameters introduces an incidental-parameter problem in the usual heteroskedasticity-robust estimators of the covariance matrix, rendering them biased and inconsistent. Hence, tests based on these estimators are size distorted even in large samples. An alternative covariance-matrix estimator that is conditionally unbiased and remains consistent is presented and supporting simulation results are provided.
Flexible Sensitivity Analysis for Observational Studies Without Observable Implications

Abstract A fundamental challenge in observational causal inference is that assumptions about unconfoundedness are not testable from data. Assessing sensitivity to such assumptions is therefore important in practice. Unfortunately, some existing sensitivity analysis approaches inadvertently impose restrictions that are at odds with modern causal inference methods, which emphasize flexible models for observed data. To address this issue, we propose a framework that allows (1) flexible models for the observed data and (2) clean separation of the identified and unidentified parts of the sensitivity model. Our framework extends an approach from the missing data literature, known as Tukey’s factorization, to the causal inference setting. Under this factorization, we can represent the distributions of unobserved potential outcomes in terms of unidentified selection functions that posit a relationship between treatment assignment and unobserved potential outcomes. The sensitivity parameters in this framework are easily interpreted, and we provide heuristics for calibrating these parameters against observable quantities. We demonstrate the flexibility of this approach in two examples, where we estimate both average treatment effects and quantile treatment effects using Bayesian nonparametric models for the observed data.
GPMatch: A Bayesian Doubly Robust Approach to Causal Inference with Gaussian Process Covariance Function As a Matching Tool

Gaussian process (GP) covariance function is proposed as a matching tool in GPMatch within a full Bayesian framework under relatively weaker causal assumptions. The matching is accomplished by utilizing GP prior covariance function to define matching distance. We show that GPMatch provides a doubly robust estimate of the averaged treatment effect (ATE) much like the G-estimation, the ATE is correctly estimated when either conditions are satisfied: 1) the GP mean function correctly specifies potential outcome \(Y^{(0)}\); or 2) the GP covariance function correctly specifies matching structure. Simulation studies were carried out without assuming any known matching structure nor functional form of the outcomes. The results demonstrate that GPMatch enjoys well calibrated frequentist properties, and outperforms many widely used methods including Bayesian Additive Regression Trees. The case study compares effectiveness of early aggressive use of biological medication in treating children with newly diagnosed Juvenile Idiopathic Arthritis, using data extracted from electronic medical records.
Adaptive Non-Parametric Regression With the $K$-NN Fused Lasso

The fused lasso, also known as total-variation denoising, is a locally-adaptive function estimator over a regular grid of design points. In this paper, we extend the fused lasso to settings in which the points do not occur on a regular grid, leading to an approach for non-parametric regression. This approach, which we call the $K$-nearest neighbors ($K$-NN) fused lasso, involves (i) computing the $K$-NN graph of the design points; and (ii) performing the fused lasso over this $K$-NN graph. We show that this procedure has a number of theoretical advantages over competing approaches: specifically, it inherits local adaptivity from its connection to the fused lasso, and it inherits manifold adaptivity from its connection to the $K$-NN approach. We show that excellent results are obtained in a simulation study and on an application to flu data. For completeness, we also study an estimator that makes use of an $\epsilon$-graph rather than a $K$-NN graph, and contrast this with the $K$-NN fused lasso.
Bounded Regression with Gaussian Process Projection

Examples with bound information on the regression function and density abound in many real applications. We propose a novel approach for estimating such functions by incorporating the prior knowledge on the bounds. Specially, a Gaussian process is first imposed on the regression function whose posterior distribution is then projected onto the bounded space. The resulting projected measure is then used for inference. The projected sample path has closed form which facilitates efficient computations. In particular, our projection approach maintains a comparable computational efficiency with that of the original GP. The proposed method yield predictions that respects bound constraints everywhere, while allows varying bounds across the input domain. An extensive simulation study is carried out which demonstrates that the performance of our approach dominates that of the competitors. An application to real data set is also considered.
Distributed statistical inference for massive data

This paper considers distributed statistical inference for general symmetric statistics %that encompasses the U-statistics and the M-estimators in the context of massive data where the data can be stored at multiple platforms in different locations. In order to facilitate effective computation and to avoid expensive communication among different platforms, we formulate distributed statistics which can be conducted over smaller data blocks. The statistical properties of the distributed statistics are investigated in terms of the mean square error of estimation and asymptotic distributions with respect to the number of data blocks. In addition, we propose two distributed bootstrap algorithms which are computationally effective and are able to capture the underlying distribution of the distributed statistics. Numerical simulation and real data applications of the proposed approaches are provided to demonstrate the empirical performance.
A Feature‐Based Procedure for Detecting Technical Outliers in Water‐Quality Data From In Situ Sensors

Outliers due to technical errors in water‐quality data from in situ sensors can reduce data quality and have a direct impact on inference drawn from subsequent data analysis. However, outlier detection through manual monitoring is infeasible given the volume and velocity of data the sensors produce. Here we introduce an automated procedure, named oddwater, that provides early detection of outliers in water‐quality data from in situ sensors caused by technical issues. Our oddwater procedure is used to first identify the data features that differentiate outlying instances from typical behaviors. Then, statistical transformations are applied to make the outlying instances stand out in a transformed data space. Unsupervised outlier scoring techniques are applied to the transformed data space, and an approach based on extreme value theory is used to calculate a threshold for each potential outlier. Using two data sets obtained from in situ sensors in rivers flowing into the Great Barrier Reef lagoon, Australia, we show that oddwater successfully identifies outliers involving abrupt changes in turbidity, conductivity, and river level, including sudden spikes, sudden isolated drops, and level shifts, while maintaining very low false detection rates. We have implemented this oddwater procedure in the open source R package oddwater.
Scoring Alternative Forecast Distributions: Completing the Kullback Distance Complex

We develop two surprising new results regarding the use of proper scoring rules for evaluating the predictive quality of two alternative sequential forecast distributions. Both of the proponents prefer to be awarded a score derived from the other's distribution rather than a score awarded on the basis of their own. A Pareto optimal exchange of their scoring outcomes provides the basis for a comparison of forecast quality that is preferred by both forecasters, and also evades a feature of arbitrariness inherent in using the forecasters' own achieved scores. The well-known Kullback divergence, used as a measure of information, is evaluated via the entropies in the two forecast distributions and the two cross-entropies between them. We show that Kullback's symmetric measure needs to be appended by three component measures if it is to characterise completely the information content of the two asserted probability forecasts. Two of these do not involve entropies at all. The resulting 'Kullback complex' supported by the 4-dimensional measure is isomorphic to an equivalent vector measure generated by the forecasters' expectations of their scores, each for one's own score and for the other's score. We foreshadow the results of a sophisticated application of the Pareto relative scoring procedure for actual sequentional observations, and we propose a standard format for evaluation.
Network Response Regression for Modeling Population of Networks with Covariates

Multiple-network data are fast emerging in recent years, where a separate network over a common set of nodes is measured for each individual subject, along with rich subject covariates information. Existing network analysis methods have primarily focused on modeling a single network, and are not directly applicable to multiple networks with subject covariates. In this article, we propose a new network response regression model, where the observed networks are treated as matrix-valued responses, and the individual covariates as predictors. The new model characterizes the population-level connectivity pattern through a low-rank intercept matrix, and the parsimonious effects of subject covariates on the network through a sparse slope tensor. We formulate the parameter estimation as a non-convex optimization problem, and develop an efficient alternating gradient descent algorithm. We establish the non-asymptotic error bound for the actual estimator from our optimization algorithm. Built upon this error bound, we derive the strong consistency for network community recovery, as well as the edge selection consistency. We demonstrate the efficacy of our method through intensive simulations and two brain connectivity studies.
Lasso tuning through the flexible-weighted bootstrap

Regularized regression approaches such as the Lasso have been widely adopted for constructing sparse linear models in high-dimensional datasets. A complexity in fitting these models is the tuning of the parameters which control the level of introduced sparsity through penalization. The most common approach to select the penalty parameter is through $k$-fold cross-validation. While cross-validation is used to minimise the empirical prediction error, approaches such as the $m$-out-of-$n$ paired bootstrap which use smaller training datasets provide consistency in selecting the non-zero coefficients in the oracle model, performing well in an asymptotic setting but having limitations when $n$ is small. In fact, for models such as the Lasso there is a monotonic relationship between the size of training sets and the penalty parameter. We propose a generalization of these methods for selecting the regularization parameter based on a flexible-weighted bootstrap procedure that mimics the $m$-out-of-$n$ bootstrap and overcomes its challenges for all sample sizes. Through simulation studies we demonstrate that when selecting a penalty parameter, the choice of weights in the bootstrap procedure can be used to dictate the size of the penalty parameter and hence the sparsity of the fitted model. We empirically illustrate our weighted bootstrap procedure by applying the Lasso to integrate clinical and microRNA data in the modeling of Alzheimer's disease. In both the real and simulated data we find a narrow part of the parameter space to perform well, emulating an $m$-out-of-$n$ bootstrap, and that our procedure can be used to improve interpretation of other optimization heuristics.
Restricted eigenvalue property for corrupted Gaussian designs

Motivated by the construction of tractable robust estimators via convex relaxations, we present conditions on the sample size which guarantee an augmented notion of Restricted Eigenvalue-type condition for Gaussian designs. Such a notion is suitable for high-dimensional robust inference in a Gaussian linear model and a multivariate Gaussian model when samples are corrupted by outliers either in the response variable or in the design matrix. Our proof technique relies on simultaneous lower and upper bounds of two random bilinear forms with very different behaviors. Such simultaneous bounds are used for balancing the interaction between the parameter vector and the estimated corruption vector as well as for controlling the presence of corruption in the design. Our technique has the advantage of not relying on known bounds of the extreme singular values of the associated Gaussian ensemble nor on the use of mutual incoherence arguments. A relevant consequence of our analysis, compared to prior work, is that a significantly sharper restricted eigenvalue constant can be obtained under weaker assumptions. In particular, the sparsity of the unknown parameter and the number of outliers are allowed to be completely independent of each other.
Strong consistency of the AIC, BIC, $C_p$ and KOO methods in high-dimensional multivariate linear regression

Variable selection is essential for improving inference and interpretation in multivariate linear regression. Although a number of alternative regressor selection criteria have been suggested, the most prominent and widely used are the Akaike information criterion (AIC), Bayesian information criterion (BIC), Mallow's $C_p$, and their modifications. However, for high-dimensional data, experience has shown that the performance of these classical criteria is not always satisfactory. In the present article, we begin by presenting the necessary and sufficient conditions (NSC) for the strong consistency of the high-dimensional AIC, BIC, and $C_p$, based on which we can identify some reasons for their poor performance. Specifically, we show that under certain mild high-dimensional conditions, if the BIC is strongly consistent, then the AIC is strongly consistent, but not vice versa. This result contradicts the classical understanding. In addition, we consider some NSC for the strong consistency of the high-dimensional kick-one-out (KOO) methods introduced by Zhao et al. (1986) and Nishii et al. (1988). Furthermore, we propose two general methods based on the KOO methods and prove their strong consistency. The proposed general methods remove the penalties while simultaneously reducing the conditions for the dimensions and sizes of the regressors. A simulation study supports our consistency conclusions and shows that the convergence rates of the two proposed general KOO methods are much faster than those of the original methods.
User-Friendly Covariance Estimation for Heavy-Tailed Distributions

We offer a survey of selected recent results on covariance estimation for heavy-tailed distributions. By unifying scattered ideas appeared in the literature, we propose user-friendly methods that facilitate practical implementation. Specifically, we introduce element-wise and spectrum-wise truncation operators, as well as their $M$-estimator counterparts, to robustify the sample covariance matrix. Different from the classical notion of robustness which is typically characterized by the breakdown property, we focus on the tail robustness that is evidenced by the nonasymptotic deviation property of the estimator for data with only finite fourth moments. The key observation is that the robustification parameter needs to adapt to the sample size, dimensionality and moment to achieve optimal tradeoff between bias and robustness. Furthermore, to facilitate their practical use, we propose tuning-free procedures that automatically calibrate the tuning parameters. Applications to a series of structured models in high dimensions, including the bandable covariance estimation, sparse precision matrix estimation, low-rank covariance estimation, and covariance estimation and multiple testing under factor models are revisited. Numerical examples lend strong support to our proposed methodology.
Separable correlation and maximum likelihood

We consider estimation of the covariance matrix of a multivariate normal distribution when the correlation matrix is separable in the sense that it factors as a Kronecker product of two smaller matrices. A computationally convenient coordinate descent-type algorithm is developed for maximum likelihood estimation. Simulations indicate our method often gives smaller estimation error than some common alternatives when correlation is separable, and that correctly sized tests for correlation separability can be obtained using a parametric bootstrap. Using dissolved oxygen data from the Upper Mississippi River, we illustrate how our model can lead to interesting scientific findings that may be missed when using competing models.
Hypothesis Testing in Functional Linear Concurrent Regression

We propose a novel method for testing the null hypothesis of no effect of a covariate on the response in the context of functional linear concurrent regression. We establish an equivalent random effects formulation of our functional regression model under which our testing problem reduces to testing for zero variance component for random effects. For this purpose, we use a one-sided score test approach, which is an extension of the classical score test. We provide theoretical justification as to why our testing procedure has the right levels (asymptotically) under null using standard assumptions. Using numerical simulations, we show that our testing method has the desired type I error rate as well as a higher power than the bootstrapped F test currently existing in the literature. Our model and testing procedure are shown to give good performances even when the data is sparsely observed, and the covariate is contaminated with noise. We also illustrate our method by applying to two real data applications: the gait data and dietary calcium absorption study data.