cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.6084/m9.figshare.13100293
title: Local Linear Forests

description: Random forests are a powerful method for nonparametric regression, but are limited in their ability to fit smooth signals. Taking the perspective of random forests as an adaptive kernel method, we pair the forest kernel with a local linear regression adjustment to better capture smoothness. The resulting procedure, <i>local linear forests</i>, enables us to improve on asymptotic rates of convergence for random forests with smooth signals, and provides substantial gains in accuracy on both real and simulated data. We prove a central limit theorem valid under regularity conditions on the forest and smoothness constraints, and propose a computationally efficient construction for confidence intervals. Moving to a causal inference application, we discuss the merits of local regression adjustments for heterogeneous treatment effect estimation, and give an example on a dataset exploring the effect word choice has on attitudes to the social safety net. Last, we include simulation results on real and generated data. A software implementation is available in the R package grf. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.13100293.v1
title: Local Linear Forests

description: Random forests are a powerful method for nonparametric regression, but are limited in their ability to fit smooth signals. Taking the perspective of random forests as an adaptive kernel method, we pair the forest kernel with a local linear regression adjustment to better capture smoothness. The resulting procedure, <i>local linear forests</i>, enables us to improve on asymptotic rates of convergence for random forests with smooth signals, and provides substantial gains in accuracy on both real and simulated data. We prove a central limit theorem valid under regularity conditions on the forest and smoothness constraints, and propose a computationally efficient construction for confidence intervals. Moving to a causal inference application, we discuss the merits of local regression adjustments for heterogeneous treatment effect estimation, and give an example on a dataset exploring the effect word choice has on attitudes to the social safety net. Last, we include simulation results on real and generated data. A software implementation is available in the R package grf. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.12903987
title: Improving Bayesian Local Spatial Models in Large Datasets

description: Environmental processes resolved at a sufficiently small scale in space and time inevitably display nonstationary behavior. Such processes are both challenging to model and computationally expensive when the data size is large. Instead of modeling the global non-stationarity explicitly, local models can be applied to disjoint regions of the domain. The choice of the size of these regions is dictated by a bias-variance trade-off; large regions will have smaller variance and larger bias, whereas small regions will have higher variance and smaller bias. From both the modeling and computational point of view, small regions are preferable to better accommodate the non-stationarity. However, in practice, large regions are necessary to control the variance. We propose a novel Bayesian three-step approach that allows for smaller regions without compromising the increase of the variance that would follow. We are able to propagate the uncertainty from one step to the next without issues caused by reusing the data. The improvement in inference also results in improved prediction, as our simulated example shows. We illustrate this new approach on a dataset of simulated high-resolution wind speed data over Saudi Arabia. Supplemental files for this article are available online.
DOI: 10.6084/m9.figshare.12903987.v2
title: Improving Bayesian Local Spatial Models in Large Datasets

description: Environmental processes resolved at a sufficiently small scale in space and time inevitably display nonstationary behavior. Such processes are both challenging to model and computationally expensive when the data size is large. Instead of modeling the global non-stationarity explicitly, local models can be applied to disjoint regions of the domain. The choice of the size of these regions is dictated by a bias-variance trade-off; large regions will have smaller variance and larger bias, whereas small regions will have higher variance and smaller bias. From both the modeling and computational point of view, small regions are preferable to better accommodate the non-stationarity. However, in practice, large regions are necessary to control the variance. We propose a novel Bayesian three-step approach that allows for smaller regions without compromising the increase of the variance that would follow. We are able to propagate the uncertainty from one step to the next without issues caused by reusing the data. The improvement in inference also results in improved prediction, as our simulated example shows. We illustrate this new approach on a dataset of simulated high-resolution wind speed data over Saudi Arabia. Supplemental files for this article are available online.
DOI: 10.6084/m9.figshare.12903987.v1
title: Improving Bayesian Local Spatial Models in Large Data Sets

description: Environmental processes resolved at a sufficiently small scale in space and time inevitably display non-stationary behavior. Such processes are both challenging to model and computationally expensive when the data size is large. Instead of modeling the global non-stationarity explicitly, local models can be applied to disjoint regions of the domain. The choice of the size of these regions is dictated by a bias-variance trade-off; large regions will have smaller variance and larger bias, whereas small regions will have higher variance and smaller bias. From both the modeling and computational point of view, small regions are preferable to better accommodate the non-stationarity. However, in practice, large regions are necessary to control the variance. We propose a novel Bayesian three-step approach that allows for smaller regions without compromising the increase of the variance that would follow. We are able to propagate the uncertainty from one step to the next without issues caused by reusing the data. The improvement in inference also results in improved prediction, as our simulated example shows. We illustrate this new approach on a data set of simulated high-resolution wind speed data over Saudi Arabia.
DOI: 10.6084/m9.figshare.13333849.v1
title: Tensor Canonical Correlation Analysis with Convergence and Statistical Guarantees

description: In many applications, such as classification of images or videos, it is of interest to develop a framework for tensor data instead of an ad-hoc way of transforming data to vectors due to the computational and under-sampling issues. In this paper, we study convergence and statistical properties of two-dimensional canonical correlation analysis (Lee and Choi, 2007) under an assumption that data come from a probabilistic model. We show that carefully initialized the power method converges to the optimum and provide a finite sample bound. Then we extend this framework to tensor-valued data and propose the higher-order power method, which is commonly used in tensor decomposition, to extract the canonical directions. Our method can be used effectively in a large-scale data setting by solving the inner least squares problem with a stochastic gradient descent, and we justify convergence via the theory of Lojasiewicz’s inequalities without any assumption on data generating process and initialization. For practical applications, we further develop (a) an inexact updating scheme which allows us to use the state-of-the-art stochastic gradient descent algorithm, (b) an effective initialization scheme which alleviates the problem of local optimum in non-convex optimization, and (c) a deflation procedure for extracting several canonical components. Empirical analyses on challenging data including gene expression and air pollution indexes in Taiwan, show the effectiveness and efficiency of the proposed methodology. Our results fill a missing, but crucial, part in the literature on tensor data.
DOI: 10.6084/m9.figshare.13275287.v1
title: An explicit mean-covariance parameterization for multivariate response linear regression

description: We develop a new method to fit the multivariate response linear regression model that exploits a parametric link between the regression coefficient matrix and the error covariance matrix. Specifically, we assume that the correlations between entries in the multivariate error random vector are proportional to the cosines of the angles between their corresponding regression coefficient matrix columns, so as the angle between two regression coefficient matrix columns decreases, the correlation between the corresponding errors increases. We highlight two models under which this parameterization arises: a latent variable reduced-rank regression model and the errors-in-variables regression model. We propose a novel non-convex weighted residual sum of squares criterion which exploits this parameterization and admits a new class of penalized estimators. The optimization is solved with an accelerated proximal gradient descent algorithm. Our method is used to study the association between microRNA expression and cancer drug activity measured on the NCI-60 cell lines. An R package implementing our method, MCMVR, is available online.
DOI: 10.6084/m9.figshare.12903888
title: Shrinking the Covariance Matrix Using Convex Penalties on the Matrix-Log Transformation

description: For <i>q</i>-dimensional data, penalized versions of the sample covariance matrix are important when the sample size is small or modest relative to <i>q</i>. Since the negative log-likelihood under multivariate normal sampling is convex in Σ−1, the inverse of the covariance matrix, it is common to consider additive penalties which are also convex in Σ−1. More recently, Deng and Tsui and Yu et al. have proposed penalties which are strictly functions of the roots of Σ and are convex in log Σ, but not in Σ−1. The resulting penalized optimization problems, though, are neither convex in log Σ nor in Σ−1. In this article, however, we show these penalized optimization problems to be geodesically convex in Σ. This allows us to establish the existence and uniqueness of the corresponding penalized covariance matrices. More generally, we show that geodesic convexity in Σ is equivalent to convexity in log Σ for penalties which are functions of the roots of Σ. In addition, when using such penalties, the resulting penalized optimization problem reduces to a <i>q</i>-dimensional convex optimization problem on the logs of the roots of Σ, which can then be readily solved via Newton’s algorithm. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.12903888.v2
title: Shrinking the Covariance Matrix Using Convex Penalties on the Matrix-Log Transformation

description: For <i>q</i>-dimensional data, penalized versions of the sample covariance matrix are important when the sample size is small or modest relative to <i>q</i>. Since the negative log-likelihood under multivariate normal sampling is convex in Σ−1, the inverse of the covariance matrix, it is common to consider additive penalties which are also convex in Σ−1. More recently, Deng and Tsui and Yu et al. have proposed penalties which are strictly functions of the roots of Σ and are convex in log Σ, but not in Σ−1. The resulting penalized optimization problems, though, are neither convex in log Σ nor in Σ−1. In this article, however, we show these penalized optimization problems to be geodesically convex in Σ. This allows us to establish the existence and uniqueness of the corresponding penalized covariance matrices. More generally, we show that geodesic convexity in Σ is equivalent to convexity in log Σ for penalties which are functions of the roots of Σ. In addition, when using such penalties, the resulting penalized optimization problem reduces to a <i>q</i>-dimensional convex optimization problem on the logs of the roots of Σ, which can then be readily solved via Newton’s algorithm. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.12903888.v1
title: Shrinking the Covariance Matrix using Convex Penalties on the Matrix-Log Transformation

description: For <i>q</i>-dimensional data, penalized versions of the sample covariance matrix are important when the sample size is small or modest relative to <i>q</i>. Since the negative log-likelihood under multivariate normal sampling is convex in Σ−1, the inverse of the covariance matrix, it is common to consider additive penalties which are also convex in Σ−1. More recently, Deng and Tsui (2013) and Yu et al. (2017) have proposed penalties which are strictly functions of the roots of Σ and are convex in log Σ, but not in Σ−1. The resulting penalized optimization problems, though, are neither convex in log Σ nor in Σ−1. In this paper, however, we show these penalized optimization problems to be geodesically convex in Σ. This allows us to establish the existence and uniqueness of the corresponding penalized covariance matrices. More generally, we show that geodesic convexity in Σ is equivalent to convexity in log Σ for penalties which are functions of the roots of Σ. In addition, when using such penalties, the resulting penalized optimization problem reduces to a <i>q</i>-dimensional convex optimization problem on the logs of the roots of Σ, which can then be readily solved via Newton’s algorithm.
DOI: 10.6084/m9.figshare.13577940
title: Cluster Optimized Proximity Scaling

description: Proximity scaling methods such as multidimensional scaling represent objects in a low-dimensional configuration so that fitted object distances optimally approximate object proximities. Besides finding the optimal configuration, an additional goal may be to make statements about the cluster arrangement of objects. This fails if the configuration lacks appreciable clusteredness. We present cluster optimized proximity scaling (COPS), which attempts to find a configuration that exhibits clusteredness. In COPS, a flexible parameterized scaling loss function that may emphasize differentiation information in the proximities is augmented with an index (OPTICS Cordillera) that penalizes lack of clusteredness of the configuration. We present two variants of this, one for finding a configuration directly and one for hyperparameter selection for parametric stresses. We apply both to a functional magnetic resonance imaging dataset on neural representations of mental states in a social cognition task and show that COPS improves clusteredness of the configuration, enabling visual identification of clusters of mental states. Online supplementary materials are available including an R package and a document with additional details.
DOI: 10.6084/m9.figshare.13577940.v1
title: Cluster Optimized Proximity Scaling

description: Proximity scaling methods such as multidimensional scaling represent objects in a low-dimensional configuration so that fitted object distances optimally approximate object proximities. Besides finding the optimal configuration, an additional goal may be to make statements about the cluster arrangement of objects. This fails if the configuration lacks appreciable clusteredness. We present cluster optimized proximity scaling (COPS), which attempts to find a configuration that exhibits clusteredness. In COPS, a flexible parameterized scaling loss function that may emphasize differentiation information in the proximities is augmented with an index (OPTICS Cordillera) that penalizes lack of clusteredness of the configuration. We present two variants of this, one for finding a configuration directly and one for hyperparameter selection for parametric stresses. We apply both to a functional magnetic resonance imaging dataset on neural representations of mental states in a social cognition task and show that COPS improves clusteredness of the configuration, enabling visual identification of clusters of mental states. Online supplementary materials are available including an R package and a document with additional details.
DOI: 10.6084/m9.figshare.13017382
title: Asymptotically Exact Data Augmentation: Models, Properties, and Algorithms

description: Data augmentation, by the introduction of auxiliary variables, has become an ubiquitous technique to improve convergence properties, simplify the implementation or reduce the computational time of inference methods such as Markov chain Monte Carlo ones. Nonetheless, introducing appropriate auxiliary variables while preserving the initial target probability distribution and offering a computationally efficient inference cannot be conducted in a systematic way. To deal with such issues, this article studies a unified framework, coined asymptotically exact data augmentation (AXDA), which encompasses both well-established and more recent approximate augmented models. In a broader perspective, this article shows that AXDA models can benefit from interesting statistical properties and yield efficient inference algorithms. In non-asymptotic settings, the quality of the proposed approximation is assessed with several theoretical results. The latter are illustrated on standard statistical problems. Supplementary materials including computer code for this article are available online.
DOI: 10.6084/m9.figshare.13017382.v2
title: Asymptotically Exact Data Augmentation: Models, Properties, and Algorithms

description: Data augmentation, by the introduction of auxiliary variables, has become an ubiquitous technique to improve convergence properties, simplify the implementation or reduce the computational time of inference methods such as Markov chain Monte Carlo ones. Nonetheless, introducing appropriate auxiliary variables while preserving the initial target probability distribution and offering a computationally efficient inference cannot be conducted in a systematic way. To deal with such issues, this article studies a unified framework, coined asymptotically exact data augmentation (AXDA), which encompasses both well-established and more recent approximate augmented models. In a broader perspective, this article shows that AXDA models can benefit from interesting statistical properties and yield efficient inference algorithms. In non-asymptotic settings, the quality of the proposed approximation is assessed with several theoretical results. The latter are illustrated on standard statistical problems. Supplementary materials including computer code for this article are available online.
DOI: 10.6084/m9.figshare.13017382.v1
title: Asymptotically exact data augmentation: models, properties and algorithms

description: Data augmentation, by the introduction of auxiliary variables, has become an ubiquitous technique to improve convergence properties, simplify the implementation or reduce the computational time of inference methods such as Markov chain Monte Carlo ones. Nonetheless, introducing appropriate auxiliary variables while preserving the initial target probability distribution and offering a computationally efficient inference cannot be conducted in a systematic way. To deal with such issues, this paper studies a unified framework, coined asymptotically exact data augmentation (AXDA), which encompasses both well-established and more recent approximate augmented models. In a broader perspective, this paper shows that AXDA models can benefit from interesting statistical properties and yield efficient inference algorithms. In non-asymptotic settings, the quality of the proposed approximation is assessed with several theoretical results. The latter are illustrated on standard statistical problems. Supplementary materials including computer code for this paper are available online.
DOI: 10.6084/m9.figshare.13017426.v1
title: Markov Chain Importance Sampling – a highly efficient estimator for MCMC

description: Markov chain (MC) algorithms are ubiquitous in machine learning and statistics and many other disciplines. Typically, these algorithms can be formulated as acceptance rejection methods. In this work we present a novel estimator applicable to these methods, dubbed Markov chain importance sampling (MCIS), which efficiently makes use of rejected proposals. For the unadjusted Langevin algorithm, it provides a novel way of correcting the discretization error. Our estimator satisfies a central limit theorem and improves on error per CPU cycle, often to a large extent. As a by-product it enables estimating the normalizing constant, an important quantity in Bayesian machine learning and statistics.
DOI: 10.6084/m9.figshare.13637944
title: False Discovery Rates to Detect Signals from Incomplete Spatially Aggregated Data

description: There are a number of ways to test for the absence/presence of a spatial signal in a completely observed fine-resolution image. One of these is a powerful nonparametric procedure called enhanced false discovery rate (EFDR). A drawback of EFDR is that it requires the data to be defined on regular pixels in a rectangular spatial domain. Here, we develop an EFDR procedure for possibly incomplete data defined on irregular small areas. Motivated by statistical learning, we use conditional simulation (CS) to condition on the available data and simulate the full rectangular image at its finest resolution many times (<i>M</i>, say). EFDR is then applied to each of these simulations resulting in <i>M</i> estimates of the signal and <i>M</i> statistically dependent <i>p</i>-values. Averaging over these estimates yields a single, combined estimate of a possible signal, but inference is needed to determine whether there really is a signal present. We test the original null hypothesis of no signal by combining the M p-values into a single <i>p</i>-value using copulas and a composite likelihood. If the null hypothesis of no signal is rejected, we use the combined estimate. We call this new procedure EFDR-CS and, to demonstrate its effectiveness, we show results from a simulation study; an experiment where we introduce aggregation and incompleteness into temperature-change data in the Asia-Pacific; and an application to total-column carbon dioxide from satellite remote sensing data over a region of the Middle East, Afghanistan, and the western part of Pakistan. Supplementary materials for this article are available online.