cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.6084/m9.figshare.14531785.v1
title: Manifold optimization Assisted Gaussian Variational Approximation*

description: Gaussian variational approximation is a popular methodology to approximate posterior distributions in Bayesian inference especially in high dimensional and large data settings. To control the computational cost while being able to capture the correlations among the variables, the low rank plus diagonal structure was introduced in the previous literature for the Gaussian covariance matrix. For a specific Bayesian learning task, the uniqueness of the solution is usually ensured by imposing stringent constraints on the parameterized covariance matrix, which could break down during the optimization process. In this paper, we consider two special covariance structures by applying the Stiefel manifold and Grassmann manifold constraints, to address the optimization difficulty in such factorization architectures. To speed up the updating process with minimum hyperparameter-tuning efforts, we design two new schemes of Riemannian stochastic gradient descent methods and compare them with other existing methods of optimizing on manifolds. In addition to fixing the identification issue, results from both simulation and empirical experiments prove the ability of the proposed methods of obtaining competitive accuracy and comparable converge speed in both high-dimensional and large-scale learning tasks.
DOI: 10.6084/m9.figshare.11603547.v2
title: Fagaceae collection

description: This fileset includes matrices and inferences relating to inferences and discussions about phylogenetic relationships in extant and extinct Fagaceae. See readme.txt in each archive for content and labelling conventions.<br><br><i><b>Version 1</b></i>Data, inferences, and exemplary artwork relevant for testing believes expressed in <br>Wilf P, Nixon KC, Gandolfo MA, Cúneo RA (2019). Eocene Fagaceae from Patagonia and Gondwanan legacy in Asian rainforests. <i>Science </i><b>364, </b>972based on a "phylogenetic analysis" of seven morphological traits.<i>Ockham's Razor applied but not used: can we do DNA-scaffolding with seven characters?</i><i>Why we want to map trait evolution on networks, pt. 1 – Introduction; pt. 2 – Topological ambiguity.</i><i><br></i><i><b>Version 2</b><br></i>Morphology folder updated.<i>Surviving parsimonists: just tree-naive or tree-blindfolded?</i><br><i></i><br><br>
DOI: 10.6084/m9.figshare.11593905.v1
title: Estimating Truncated Functional Linear Models With a Nested Group Bridge Approach

description: We study a scalar-on-function truncated linear regression model which assumes that the functional predictor does not influence the response when the time passes a certain cutoff point. We approach this problem from the perspective of locally sparse modeling, where a function is locally sparse if it is zero on a substantial portion of its defining domain. In the truncated linear model, the slope function is exactly a locally sparse function that is zero beyond the cutoff time. A locally sparse estimate then gives rise to an estimate of the cutoff time. We propose a nested group bridge penalty that is able to specifically shrink the tail of a function. Combined with the B-spline basis expansion and penalized least squares, the nested group bridge approach can identify the cutoff time and produce a smooth estimate of the slope function simultaneously. The proposed nested group bridge estimator is shown to be consistent, while its numerical performance is illustrated by simulation studies. The proposed nested group bridge method is demonstrated with an application of determining the effect of the past engine acceleration on the current particulate matter emission. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.13865000
title: Multi-resolution filters for massive spatio-temporal data

description: Spatio-temporal datasets are rapidly growing in size. For example, environmental variables are measured with increasing resolution by increasing numbers of automated sensors mounted on satellites and aircraft. Using such data, which are typically noisy and incomplete, the goal is to obtain complete maps of the spatio-temporal process, together with uncertainty quantification. We focus here on real-time filtering inference in linear Gaussian state-space models. At each time point, the state is a spatial field evaluated on a very large spatial grid, making exact inference using the Kalman filter computationally infeasible. Instead, we propose a multi-resolution filter (MRF), a highly scalable and fully probabilistic filtering method that resolves spatial features at all scales. We prove that the MRF matrices exhibit a particular block-sparse multi-resolution structure that is preserved under filtering operations through time. We describe connections to existing methods, including hierarchical matrices from numerical mathematics. We also discuss inference on time-varying parameters using an approximate Rao-Blackwellized particle filter, in which the integrated likelihood is computed using the MRF. Using a simulation study and a real satellite-data application, we show that the MRF strongly outperforms competing approaches. Supplementary materials include Python code for reproducing the simulations, some detailed properties of the MRF and auxiliary theoretical results.
DOI: 10.6084/m9.figshare.13865000.v2
title: Multi-resolution filters for massive spatio-temporal data

description: Spatio-temporal datasets are rapidly growing in size. For example, environmental variables are measured with increasing resolution by increasing numbers of automated sensors mounted on satellites and aircraft. Using such data, which are typically noisy and incomplete, the goal is to obtain complete maps of the spatio-temporal process, together with uncertainty quantification. We focus here on real-time filtering inference in linear Gaussian state-space models. At each time point, the state is a spatial field evaluated on a very large spatial grid, making exact inference using the Kalman filter computationally infeasible. Instead, we propose a multi-resolution filter (MRF), a highly scalable and fully probabilistic filtering method that resolves spatial features at all scales. We prove that the MRF matrices exhibit a particular block-sparse multi-resolution structure that is preserved under filtering operations through time. We describe connections to existing methods, including hierarchical matrices from numerical mathematics. We also discuss inference on time-varying parameters using an approximate Rao-Blackwellized particle filter, in which the integrated likelihood is computed using the MRF. Using a simulation study and a real satellite-data application, we show that the MRF strongly outperforms competing approaches. Supplementary materials include Python code for reproducing the simulations, some detailed properties of the MRF and auxiliary theoretical results.
DOI: 10.6084/m9.figshare.13865000.v1
title: Multi-Resolution Filters for Massive Spatio-Temporal Data

description: Spatio-temporal datasets are rapidly growing in size. For example, environmental variables are measured with increasing resolution by increasing numbers of automated sensors mounted on satellites and aircraft. Using such data, which are typically noisy and incomplete, the goal is to obtain complete maps of the spatio-temporal process, together with uncertainty quantification. We focus here on real-time filtering inference in linear Gaussian state-space models. At each time point, the state is a spatial field evaluated on a very large spatial grid, making exact inference using the Kalman filter computationally infeasible. Instead, we propose a multi-resolution filter (MRF), a highly scalable and fully probabilistic filtering method that resolves spatial features at all scales. We prove that the MRF matrices exhibit a particular block-sparse multi-resolution structure that is preserved under filtering operations through time. We describe connections to existing methods, including hierarchical matrices from numerical mathematics. We also discuss inference on time-varying parameters using an approximate Rao-Blackwellized particle filter, in which the integrated likelihood is computed using the MRF. Using a simulation study and a real satellite-data application, we show that the MRF strongly outperforms competing approaches. Supplementary materials include Python code for reproducing the simulations, some detailed properties of the MRF and auxiliary theoretical results.
DOI: 10.6084/m9.figshare.12075876.v1
title: Forward Event-Chain Monte Carlo: Fast Sampling by Randomness Control in Irreversible Markov Chains

description: Irreversible and rejection-free Monte Carlo methods, recently developed in physics under the name event-chain and known in statistics as piecewise deterministic Monte Carlo (PDMC), have proven to produce clear acceleration over standard Monte Carlo methods, thanks to the reduction of their random-walk behavior. However, while applying such schemes to standard statistical models, one generally needs to introduce an additional randomization for sake of correctness. We propose here a new class of event-chain Monte Carlo methods that reduces this extra-randomization to a bare minimum. We compare the efficiency of this new methodology to standard PDMC and Monte Carlo methods. Accelerations up to several magnitudes and reduced dimensional scalings are exhibited. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.14916681
title: Reproducible Hyperparameter Optimization

description: A key issue in current machine learning research is the lack of reproducibility. We illustrate what role hyperparameter search plays in this problem and how regular hyperparameter search methods can lead to a large variance in outcomes due to non-deterministic model training during hyperparameter optimization. The variation in outcomes poses a problem both for reproducibility of the hyperparameter search itself and comparisons of different methods each optimized using hyperparameter search. In addition, the fact that hyperparameter search may result in non-optimal hyperparameter settings may affect other studies since hyperparameter settings are often copied from previously published research. To remedy this issue we define the mean prediction error across model training runs as the objective for the hyperparameter search. We then propose a hypothesis testing procedure that makes inference on the mean performance of each hyperparameter setting and results in an equivalence class of hyperparameter settings that are not distinguishable in performance. We further embed this procedure into a group sequential testing framework to increase efficiency in terms of the average number of model training replicates required. Empirical results on machine learning benchmarks show that at equal computation the proposed method reduces the variation in hyperparameter search outcomes by up to 90 percent while resulting in equal or lower mean prediction errors when compared to standard random search and Bayesian optimization. Moreover, the sequential testing framework successfully reduces computation while preserving performance of the method. Code to reproduce the results is available in the supplementary materials and online<sup>1</sup>.
DOI: 10.6084/m9.figshare.14916681.v1
title: Reproducible Hyperparameter Optimization

description: A key issue in current machine learning research is the lack of reproducibility. We illustrate what role hyperparameter search plays in this problem and how regular hyperparameter search methods can lead to a large variance in outcomes due to non-deterministic model training during hyperparameter optimization. The variation in outcomes poses a problem both for reproducibility of the hyperparameter search itself and comparisons of different methods each optimized using hyperparameter search. In addition, the fact that hyperparameter search may result in non-optimal hyperparameter settings may affect other studies since hyperparameter settings are often copied from previously published research. To remedy this issue we define the mean prediction error across model training runs as the objective for the hyperparameter search. We then propose a hypothesis testing procedure that makes inference on the mean performance of each hyperparameter setting and results in an equivalence class of hyperparameter settings that are not distinguishable in performance. We further embed this procedure into a group sequential testing framework to increase efficiency in terms of the average number of model training replicates required. Empirical results on machine learning benchmarks show that at equal computation the proposed method reduces the variation in hyperparameter search outcomes by up to 90 percent while resulting in equal or lower mean prediction errors when compared to standard random search and Bayesian optimization. Moreover, the sequential testing framework successfully reduces computation while preserving performance of the method. Code to reproduce the results is available in the supplementary materials and online<sup>1</sup>.
DOI: 10.6084/m9.figshare.4545697
title: Penalized Estimation in Large-Scale Generalized Linear Array Models

description: Large-scale generalized linear array models (GLAMs) can be challenging to fit. Computation and storage of its tensor product design matrix can be impossible due to time and memory constraints, and previously considered design matrix free algorithms do not scale well with the dimension of the parameter vector. A new design matrix free algorithm is proposed for computing the penalized maximum likelihood estimate for GLAMs, which, in particular, handles nondifferentiable penalty functions. The proposed algorithm is implemented and available via the R package glamlasso. It combines several ideas—previously considered separately—to obtain sparse estimates while at the same time efficiently exploiting the GLAM structure. In this article, the convergence of the algorithm is treated and the performance of its implementation is investigated and compared to that of glmnet on simulated as well as real data. It is shown that the computation time for glamlasso scales favorably with the size of the problem when compared to glmnet. Supplementary materials, in the form of R code, data and visualizations of results, are available online.
DOI: 10.6084/m9.figshare.4545697.v7
title: Penalized Estimation in Large-Scale Generalized Linear Array Models

description: Large-scale generalized linear array models (GLAMs) can be challenging to fit. Computation and storage of its tensor product design matrix can be impossible due to time and memory constraints, and previously considered design matrix free algorithms do not scale well with the dimension of the parameter vector. A new design matrix free algorithm is proposed for computing the penalized maximum likelihood estimate for GLAMs, which, in particular, handles nondifferentiable penalty functions. The proposed algorithm is implemented and available via the R package glamlasso. It combines several ideas—previously considered separately—to obtain sparse estimates while at the same time efficiently exploiting the GLAM structure. In this article, the convergence of the algorithm is treated and the performance of its implementation is investigated and compared to that of glmnet on simulated as well as real data. It is shown that the computation time for glamlasso scales favorably with the size of the problem when compared to glmnet. Supplementary materials, in the form of R code, data and visualizations of results, are available online.
DOI: 10.6084/m9.figshare.4528016.v3
title: A Cross-Entropy Approach to the Estimation of Generalized Linear Multilevel Models

description: In this article, we use the cross-entropy method for noisy optimization for fitting generalized linear multilevel models through maximum likelihood. We propose specifications of the instrumental distributions for positive and bounded parameters that improve the computational performance. We also introduce a new stopping criterion, which has the advantage of being problem-independent. In a second step we find, by means of extensive Monte Carlo experiments, the most suitable values of the input parameters of the algorithm. Finally, we compare the method to the benchmark estimation technique based on numerical integration. The cross-entropy approach turns out to be preferable from both the statistical and the computational point of view. In the last part of the article, the method is used to model the probability of firm exits in the healthcare industry in Italy. Supplemental materials are available online.
DOI: 10.6084/m9.figshare.5183224
title: Approximate Bayesian Computation and Model Assessment for Repulsive Spatial Point Processes

description: In many applications involving spatial point patterns, we find evidence of inhibition or repulsion. The most commonly used class of models for such settings are the Gibbs point processes. A recent alternative, at least to the statistical community, is the determinantal point process. Here, we examine model fitting and inference for both of these classes of processes in a Bayesian framework. While usual MCMC model fitting can be available, the algorithms are complex and are not always well behaved. We propose using approximate Bayesian computation (ABC) for such fitting. This approach becomes attractive because, though likelihoods are very challenging to work with for these processes, generation of realizations given parameter values is relatively straightforward. As a result, the ABC fitting approach is well-suited for these models. In addition, such simulation makes them well-suited for posterior predictive inference as well as for model assessment. We provide details for all of the above along with some simulation investigation and an illustrative analysis of a point pattern of tree data exhibiting repulsion. R code and datasets are included in the supplementary material.
DOI: 10.6084/m9.figshare.5183224.v2
title: Approximate Bayesian Computation and Model Assessment for Repulsive Spatial Point Processes

description: In many applications involving spatial point patterns, we find evidence of inhibition or repulsion. The most commonly used class of models for such settings are the Gibbs point processes. A recent alternative, at least to the statistical community, is the determinantal point process. Here, we examine model fitting and inference for both of these classes of processes in a Bayesian framework. While usual MCMC model fitting can be available, the algorithms are complex and are not always well behaved. We propose using approximate Bayesian computation (ABC) for such fitting. This approach becomes attractive because, though likelihoods are very challenging to work with for these processes, generation of realizations given parameter values is relatively straightforward. As a result, the ABC fitting approach is well-suited for these models. In addition, such simulation makes them well-suited for posterior predictive inference as well as for model assessment. We provide details for all of the above along with some simulation investigation and an illustrative analysis of a point pattern of tree data exhibiting repulsion. R code and datasets are included in the supplementary material.
DOI: 10.6084/m9.figshare.4545697.v6
title: Penalized Estimation in Large-Scale Generalized Linear Array Models

description: Large-scale generalized linear array models (GLAMs) can be challenging to fit. Computation and storage of its tensor product design matrix can be impossible due to time and memory constraints, and previously considered design matrix free algorithms do not scale well with the dimension of the parameter vector. A new design matrix free algorithm is proposed for computing the penalized maximum likelihood estimate for GLAMs, which, in particular, handles nondifferentiable penalty functions. The proposed algorithm is implemented and available via the R package glamlasso. It combines several ideas—previously considered separately—to obtain sparse estimates while at the same time efficiently exploiting the GLAM structure. In this article, the convergence of the algorithm is treated and the performance of its implementation is investigated and compared to that of glmnet on simulated as well as real data. It is shown that the computation time for glamlasso scales favorably with the size of the problem when compared to glmnet. Supplementary materials, in the form of R code, data and visualizations of results, are available online.
DOI: 10.6084/m9.figshare.4528016.v2
title: A Cross-Entropy Approach to the Estimation of Generalized Linear Multilevel Models

description: In this article, we use the cross-entropy method for noisy optimization for fitting generalized linear multilevel models through maximum likelihood. We propose specifications of the instrumental distributions for positive and bounded parameters that improve the computational performance. We also introduce a new stopping criterion, which has the advantage of being problem-independent. In a second step we find, by means of extensive Monte Carlo experiments, the most suitable values of the input parameters of the algorithm. Finally, we compare the method to the benchmark estimation technique based on numerical integration. The cross-entropy approach turns out to be preferable from both the statistical and the computational point of view. In the last part of the article, the method is used to model the probability of firm exits in the healthcare industry in Italy. Supplemental materials are available online.
DOI: 10.6084/m9.figshare.5183224.v1
title: Approximate Bayesian Computation and Model Assessment for Repulsive Spatial Point Processes

description: In many applications involving spatial point patterns, we find evidence of inhibition or repulsion. The most commonly used class of models for such settings are the Gibbs point processes. A recent alternative, at least to the statistical community, is the determinantal point process. Here, we examine model fitting and inference for both of these classes of processes in a Bayesian framework. While usual MCMC model fitting can be available, the algorithms are complex and are not always well behaved. We propose using approximate Bayesian computation (ABC) for such fitting. This approach becomes attractive because, though likelihoods are very challenging to work with for these processes, generation of realizations given parameter values is relatively straightforward. As a result, the ABC fitting approach is well-suited for these models. In addition, such simulation makes them well-suited for posterior predictive inference as well as for model assessment. We provide details for all of the above along with some simulation investigation and an illustrative analysis of a point pattern of tree data exhibiting repulsion. R code and datasets are included in the supplementary material.
DOI: 10.6084/m9.figshare.4528016
title: A Cross-Entropy Approach to the Estimation of Generalized Linear Multilevel Models

description: In this article, we use the cross-entropy method for noisy optimization for fitting generalized linear multilevel models through maximum likelihood. We propose specifications of the instrumental distributions for positive and bounded parameters that improve the computational performance. We also introduce a new stopping criterion, which has the advantage of being problem-independent. In a second step we find, by means of extensive Monte Carlo experiments, the most suitable values of the input parameters of the algorithm. Finally, we compare the method to the benchmark estimation technique based on numerical integration. The cross-entropy approach turns out to be preferable from both the statistical and the computational point of view. In the last part of the article, the method is used to model the probability of firm exits in the healthcare industry in Italy. Supplemental materials are available online.