cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.6084/m9.figshare.7862876.v3
title: Damped Anderson Acceleration With Restarts and Monotonicity Control for Accelerating EM and EM-like Algorithms

description: The expectation-maximization (EM) algorithm is a well-known iterative method for computing maximum likelihood estimates in a variety of statistical problems. Despite its numerous advantages, a main drawback of the EM algorithm is its frequently observed slow convergence which often hinders the application of EM algorithms in high-dimensional problems or in other complex settings. To address the need for more rapidly convergent EM algorithms, we describe a new class of acceleration schemes that build on the Anderson acceleration technique for speeding fixed-point iterations. Our approach is effective at greatly accelerating the convergence of EM algorithms and is automatically scalable to high-dimensional settings. Through the introduction of periodic algorithm restarts and a damping factor, our acceleration scheme provides faster and more robust convergence when compared to un-modified Anderson acceleration, while also improving global convergence. Crucially, our method works as an “off-the-shelf” method in that it may be directly used to accelerate any EM algorithm without relying on the use of any model-specific features or insights. Through a series of simulation studies involving five representative problems, we show that our algorithm is substantially faster than the existing state-of-art acceleration schemes. The acceleration schemes described in this paper are implemented in the R package daarem which is available from the comprehensive R archive network (https://cran.r-project.org). Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7932509.v3
title: Projection Pursuit Based on Gaussian Mixtures and Evolutionary Algorithms

description: We propose a projection pursuit (PP) algorithm based on Gaussian mixture models (GMMs). The negentropy obtained from a multivariate density estimated by GMMs is adopted as the PP index to be maximized. For a fixed dimension of the projection subspace, the GMM-based density estimation is projected onto that subspace, where an approximation of the negentropy for Gaussian mixtures is computed. Then, genetic algorithms are used to find the optimal, orthogonal projection basis by maximizing the former approximation. We show that this semiparametric approach to PP is flexible and allows highly informative structures to be detected, by projecting multivariate datasets onto a subspace, where the data can be feasibly visualized. The performance of the proposed approach is shown on both artificial and real datasets. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7981790.v1
title: Variable-Domain Functional Principal Component Analysis

description: We introduce a novel method of principal component analysis for data with varying domain lengths for each functional observation. We refer to this technique as variable-domain functional principal component analysis, or vd-FPCA. We fit a trivariate smoother using penalized thin plate splines to estimate the covariance as a function of the domain length. Principal components are then calculated through eigen-decomposition of the estimated covariance matrix, conditional on the domain length. We apply vd-FPCA in two functional data settings, first to daily measures of patient wellness during a stay in the ICU, and second, to accelerometer recordings of repeated in-lab movements. In each example, vd-FPCA uses fewer principal components than typical FPCA methods to explain a greater proportion of the variation in the data. We also find the principal components provide greater flexibility in interpretation with respect to domain length than traditional approaches. These methods are easily implementable through standard statistical software and applicable to a wide variety of datasets involving continuous observations over a variable domain. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7999502
title: Simultaneous Registration and Clustering for Multidimensional Functional Data

description: The clustering for functional data with misaligned problems has drawn much attention in the last decade. Most methods do the clustering after those functional data being registered and there has been little research using both functional and scalar variables. In this article, we propose a simultaneous registration and clustering model via two-level models, allowing the use of both types of variables and also allowing simultaneous registration and clustering. For the data collected from subjects in different groups, a Gaussian process functional regression model with time warping is used as the first level model; an allocation model depending on scalar variables is used as the second level model providing further information over the groups. The former carries out registration and modeling for the multidimensional functional data (two-dimensional curves) at the same time. This methodology is implemented using an EM algorithm, and is examined on both simulated data and real data. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7859750.v3
title: Incremental Mixture Importance Sampling With Shotgun Optimization

description: <b>This article proposes a general optimization strategy, which combines results from different optimization or parameter estimation methods to overcome shortcomings of a single method. Shotgun optimization is developed as a framework which employs different optimization strategies, criteria, or conditional targets to enable wider likelihood exploration. The introduced shotgun optimization approach is embedded into an incremental mixture importance sampling algorithm to produce improved posterior samples for multimodal densities and creates robustness in cases where the likelihood and prior are in disagreement. Despite using different optimization approaches, the samples are combined into samples from a single target posterior. The diversity of the framework is demonstrated on parameter estimation from differential equation models employing diverse strategies including numerical solutions and approximations thereof. Additionally the approach is demonstrated on mixtures of discrete and continuous parameters and is shown to ease estimation from synthetic likelihood models. R code of the implemented examples can be found at</b>https://github.com/BiljanaJSJ/IMIS-ShOpt<b>. Supplementary materials for this article are available online.</b>
DOI: 10.6084/m9.figshare.7862810.v2
title: Improving Spectral Clustering Using the Asymptotic Value of the Normalized Cut

description: Spectral clustering (SC) is a popular and versatile clustering method based on a relaxation of the normalized graph cut objective. Despite its popularity, selecting the number of clusters and tuning the important scaling parameter remain challenging problems in practical applications of SC. Popular heuristics have been proposed, but corresponding theoretical results are scarce. In this article, we investigate the asymptotic value of the normalized cut for an increasing sample assumed to arise from an underlying probability distribution. Based on this, we find strong connections between spectral and density clustering. This enables us to provide recommendations for selecting the number of clusters and setting the scaling parameter in a data driven manner. An algorithm inspired by these recommendations is proposed, which we have found to exhibit strong performance in a range of applied domains. An R implementation of the algorithm is available from https://github.com/DavidHofmeyr/spuds. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7859750.v2
title: Incremental Mixture Importance Sampling With Shotgun Optimization

description: <b>This article proposes a general optimization strategy, which combines results from different optimization or parameter estimation methods to overcome shortcomings of a single method. Shotgun optimization is developed as a framework which employs different optimization strategies, criteria, or conditional targets to enable wider likelihood exploration. The introduced shotgun optimization approach is embedded into an incremental mixture importance sampling algorithm to produce improved posterior samples for multimodal densities and creates robustness in cases where the likelihood and prior are in disagreement. Despite using different optimization approaches, the samples are combined into samples from a single target posterior. The diversity of the framework is demonstrated on parameter estimation from differential equation models employing diverse strategies including numerical solutions and approximations thereof. Additionally the approach is demonstrated on mixtures of discrete and continuous parameters and is shown to ease estimation from synthetic likelihood models. R code of the implemented examples can be found at</b>https://github.com/BiljanaJSJ/IMIS-ShOpt<b>. Supplementary materials for this article are available online.</b>
DOI: 10.6084/m9.figshare.7862876.v2
title: Damped Anderson Acceleration With Restarts and Monotonicity Control for Accelerating EM and EM-like Algorithms

description: The expectation-maximization (EM) algorithm is a well-known iterative method for computing maximum likelihood estimates in a variety of statistical problems. Despite its numerous advantages, a main drawback of the EM algorithm is its frequently observed slow convergence which often hinders the application of EM algorithms in high-dimensional problems or in other complex settings. To address the need for more rapidly convergent EM algorithms, we describe a new class of acceleration schemes that build on the Anderson acceleration technique for speeding fixed-point iterations. Our approach is effective at greatly accelerating the convergence of EM algorithms and is automatically scalable to high-dimensional settings. Through the introduction of periodic algorithm restarts and a damping factor, our acceleration scheme provides faster and more robust convergence when compared to un-modified Anderson acceleration, while also improving global convergence. Crucially, our method works as an “off-the-shelf” method in that it may be directly used to accelerate any EM algorithm without relying on the use of any model-specific features or insights. Through a series of simulation studies involving five representative problems, we show that our algorithm is substantially faster than the existing state-of-art acceleration schemes. The acceleration schemes described in this paper are implemented in the R package daarem which is available from the comprehensive R archive network (https://cran.r-project.org). Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7981724
title: The Generalized Ridge Estimator of the Inverse Covariance Matrix

description: The ridge inverse covariance estimator is generalized to allow for entry-wise penalization. An efficient algorithm for its evaluation is proposed. Its computational accuracy is benchmarked against implementations of specific cases the generalized ridge inverse covariance estimator encompasses. The proposed estimator shrinks toward a user-specified, nonrandom target matrix and is shown to be positive definite and consistent. It is pointed out how the generalized ridge inverse covariance estimator can be used to obtain a generalization of the graphical lasso estimator as well as of its elastic net counterpart. The usage of the presented estimator is illustrated in graphical modeling of omics data. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7859756.v1
title: Flexible and Interpretable Models for Survival Data

description: As data sets continue to increase in size, there is growing interest in methods for prediction that are both flexible and interpretable. A flurry of recent work on this topic has focused on additive modeling in the regression setting, and in particular, on the use of data-adaptive non-linear functions that can be used to flexibly model each covariate’s effect, conditional on the other features in the model. In this paper, we extend this recent line of work to the survival setting. We develop an additive Cox proportional hazards model, in which each additive function is obtained by trend filtering, so that the fitted functions are piece-wise polynomial with adaptively-chosen knots. An efficient proximal gradient descent algorithm is used to fit the model. We demonstrate its performance in simulations and in application to a primary biliary cirrhosis data set, as well as a data set consisting of time to publication for clinical trials in the biomedical literature.
DOI: 10.6084/m9.figshare.7932509.v2
title: Projection Pursuit Based on Gaussian Mixtures and Evolutionary Algorithms

description: We propose a projection pursuit (PP) algorithm based on Gaussian mixture models (GMMs). The negentropy obtained from a multivariate density estimated by GMMs is adopted as the PP index to be maximized. For a fixed dimension of the projection subspace, the GMM-based density estimation is projected onto that subspace, where an approximation of the negentropy for Gaussian mixtures is computed. Then, genetic algorithms are used to find the optimal, orthogonal projection basis by maximizing the former approximation. We show that this semiparametric approach to PP is flexible and allows highly informative structures to be detected, by projecting multivariate datasets onto a subspace, where the data can be feasibly visualized. The performance of the proposed approach is shown on both artificial and real datasets. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7999502.v3
title: Simultaneous Registration and Clustering for Multidimensional Functional Data

description: The clustering for functional data with misaligned problems has drawn much attention in the last decade. Most methods do the clustering after those functional data being registered and there has been little research using both functional and scalar variables. In this article, we propose a simultaneous registration and clustering model via two-level models, allowing the use of both types of variables and also allowing simultaneous registration and clustering. For the data collected from subjects in different groups, a Gaussian process functional regression model with time warping is used as the first level model; an allocation model depending on scalar variables is used as the second level model providing further information over the groups. The former carries out registration and modeling for the multidimensional functional data (two-dimensional curves) at the same time. This methodology is implemented using an EM algorithm, and is examined on both simulated data and real data. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.8063765.v3
title: Beyond Prediction: A Framework for Inference With Variational Approximations in Mixture Models

description: Variational inference is a popular method for estimating model parameters and conditional distributions in hierarchical and mixed models, which arise frequently in many settings in the health, social, and biological sciences. Variational inference in a frequentist context works by approximating intractable conditional distributions with a tractable family and optimizing the resulting lower bound on the log-likelihood. The variational objective function is typically less computationally intensive to optimize than the true likelihood, enabling scientists to fit rich models even with extremely large datasets. Despite widespread use, little is known about the general theoretical properties of estimators arising from variational approximations to the log-likelihood, which hinders their use in inferential statistics. In this article, we connect such estimators to profile <i>M</i>-estimation, which enables us to provide regularity conditions for consistency and asymptotic normality of variational estimators. Our theory also motivates three methodological improvements to variational inference: estimation of the asymptotic model-robust covariance matrix, a one-step correction that improves estimator efficiency, and an empirical assessment of consistency. We evaluate the proposed results using simulation studies and data on marijuana use from the National Longitudinal Study of Youth. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7859756
title: Flexible and Interpretable Models for Survival Data

description: As data sets continue to increase in size, there is growing interest in methods for prediction that are both flexible and interpretable. A flurry of recent work on this topic has focused on additive modeling in the regression setting, and in particular, on the use of data-adaptive non-linear functions that can be used to flexibly model each covariate’s effect, conditional on the other features in the model. In this paper, we extend this recent line of work to the survival setting. We develop an additive Cox proportional hazards model, in which each additive function is obtained by trend filtering, so that the fitted functions are piece-wise polynomial with adaptively-chosen knots. An efficient proximal gradient descent algorithm is used to fit the model. We demonstrate its performance in simulations and in application to a primary biliary cirrhosis data set, as well as a data set consisting of time to publication for clinical trials in the biomedical literature.
DOI: 10.6084/m9.figshare.7981724.v3
title: The Generalized Ridge Estimator of the Inverse Covariance Matrix

description: The ridge inverse covariance estimator is generalized to allow for entry-wise penalization. An efficient algorithm for its evaluation is proposed. Its computational accuracy is benchmarked against implementations of specific cases the generalized ridge inverse covariance estimator encompasses. The proposed estimator shrinks toward a user-specified, nonrandom target matrix and is shown to be positive definite and consistent. It is pointed out how the generalized ridge inverse covariance estimator can be used to obtain a generalization of the graphical lasso estimator as well as of its elastic net counterpart. The usage of the presented estimator is illustrated in graphical modeling of omics data. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.8063765.v2
title: Beyond Prediction: A Framework for Inference With Variational Approximations in Mixture Models

description: Variational inference is a popular method for estimating model parameters and conditional distributions in hierarchical and mixed models, which arise frequently in many settings in the health, social, and biological sciences. Variational inference in a frequentist context works by approximating intractable conditional distributions with a tractable family and optimizing the resulting lower bound on the log-likelihood. The variational objective function is typically less computationally intensive to optimize than the true likelihood, enabling scientists to fit rich models even with extremely large datasets. Despite widespread use, little is known about the general theoretical properties of estimators arising from variational approximations to the log-likelihood, which hinders their use in inferential statistics. In this article, we connect such estimators to profile <i>M</i>-estimation, which enables us to provide regularity conditions for consistency and asymptotic normality of variational estimators. Our theory also motivates three methodological improvements to variational inference: estimation of the asymptotic model-robust covariance matrix, a one-step correction that improves estimator efficiency, and an empirical assessment of consistency. We evaluate the proposed results using simulation studies and data on marijuana use from the National Longitudinal Study of Youth. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7981724.v2
title: The Generalized Ridge Estimator of the Inverse Covariance Matrix

description: The ridge inverse covariance estimator is generalized to allow for entry-wise penalization. An efficient algorithm for its evaluation is proposed. Its computational accuracy is benchmarked against implementations of specific cases the generalized ridge inverse covariance estimator encompasses. The proposed estimator shrinks toward a user-specified, nonrandom target matrix and is shown to be positive definite and consistent. It is pointed out how the generalized ridge inverse covariance estimator can be used to obtain a generalization of the graphical lasso estimator as well as of its elastic net counterpart. The usage of the presented estimator is illustrated in graphical modeling of omics data. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.7999502.v2
title: Simultaneous Registration and Clustering for Multidimensional Functional Data

description: The clustering for functional data with misaligned problems has drawn much attention in the last decade. Most methods do the clustering after those functional data being registered and there has been little research using both functional and scalar variables. In this article, we propose a simultaneous registration and clustering model via two-level models, allowing the use of both types of variables and also allowing simultaneous registration and clustering. For the data collected from subjects in different groups, a Gaussian process functional regression model with time warping is used as the first level model; an allocation model depending on scalar variables is used as the second level model providing further information over the groups. The former carries out registration and modeling for the multidimensional functional data (two-dimensional curves) at the same time. This methodology is implemented using an EM algorithm, and is examined on both simulated data and real data. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9034328
title: Efficient Construction of Test Inversion Confidence Intervals Using Quantile Regression

description: Modern problems in statistics often include estimators of high computational complexity and with complicated distributions. Statistical inference on such estimators usually relies on asymptotic normality assumptions, however, such assumptions are often not applicable for available sample sizes, due to dependencies in the data. A common alternative is the use of resampling procedures, such as bootstrapping, but these may be computationally intensive to an extent that renders them impractical for modern problems. In this article, we develop a method for fast construction of test-inversion bootstrap confidence intervals. Our approach uses quantile regression to model the quantile of an estimator conditional on the true value of the parameter. We apply this to the Watterson estimator of mutation rate in a standard coalescent model. We demonstrate an improved efficiency of up to 40% from using quantile regression compared to state of the art methods based on stochastic approximation, as measured by the number of simulations required to achieve comparable accuracy. Supplementary materials for this article are available online.