cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.6084/m9.figshare.12066147.v1
title: Predicting the Output From a Stochastic Computer Model When a Deterministic Approximation is Available

description: Statistically modeling the output of a stochastic computer model can be difficult to do accurately without a large simulation budget. We alleviate this problem by exploiting readily available deterministic approximations to efficiently learn about the respective stochastic computer models. This is done via the summation of two Gaussian processes; one responsible for modeling the deterministic approximation, the other responsible for using such approximation to better statistically model the stochastic computer model. The developed method provides high predictive performance and increased confidence that complicated features of a stochastic computer model are captured, even when the simulation budget is small. Several synthetic computer models are used to outline the capabilities of this method, and two real-world examples are used to display its practical utility. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.12141069.v3
title: A Tree-Based Semi-Varying Coefficient Model for the COM-Poisson Distribution

description: We propose a tree-based semi-varying coefficient model for the Conway–Maxwell–Poisson (CMP or COM-Poisson) distribution which is a two-parameter generalization of the Poisson distribution and is flexible enough to capture both under-dispersion and over-dispersion in count data. The advantage of tree-based methods is their scalability to high-dimensional data. We develop CMPMOB, an estimation procedure for a semi-varying coefficient model, using model-based recursive partitioning (MOB). The proposed framework is broader than the existing MOB framework as it allows node-invariant effects to be included in the model. To simplify the computational burden of the exhaustive search employed in the original MOB algorithm, a new split point estimation procedure is proposed by borrowing tools from change point estimation methodology. The proposed method uses only the estimated score functions without fitting models for each split point and, therefore, is computationally simpler. Since the tree-based methods only provide a piece-wise constant approximation to the underlying smooth function, we further propose the CMPBoost semi-varying coefficient model which uses the gradient boosting procedure for estimation. The usefulness of the proposed methods are illustrated using simulation studies and a real example from a bike sharing system in Washington, DC. Supplementary files for this article are available online.
DOI: 10.6084/m9.figshare.11973531.v1
title: Co-clustering of ordinal data via latent continuous random variables and not missing at random entries

description: This paper is about the co-clustering of ordinal data. Such data are very common on e-commerce platforms where customers rank the products/services they bought. In more detail, we focus on arrays of ordinal (possibly missing) data involving two disjoint sets of individuals/objects corresponding to the rows/columns of the arrays. Typically, an observed entry (i, j) in the array is an ordinal score assigned by the individual/row i to the object/column j. A new generative model for arrays of ordinal data is introduced along with an inference algorithm for parameters estimation. The model accounts for not missing at random (NMAR) data and relies on latent continuous random variables. The fitting allows to simultaneously co-cluster the rows and columns of an array. The estimation of the model parameters is performed via a classification expectation maximization (C-EM) algorithm. A model selection criterion is formally obtained to select the number of row and column clusters. In order to show that our approach reaches and often outperforms the state of the art, we carry out numerical experiments on synthetic data. Finally, applications on real datasets highlight the model capacity to deal with very sparse arrays.
DOI: 10.6084/m9.figshare.11973558.v2
title: Bayesian Variational Inference for Exponential Random Graph Models

description: Deriving Bayesian inference for exponential random graph models (ERGMs) is a challenging “doubly intractable” problem as the normalizing constants of the likelihood and posterior density are both intractable. Markov chain Monte Carlo (MCMC) methods which yield Bayesian inference for ERGMs, such as the exchange algorithm, are asymptotically exact but computationally intensive, as a network has to be drawn from the likelihood at every step using, for instance, a “tie no tie” sampler. In this article, we develop a variety of variational methods for Gaussian approximation of the posterior density and model selection. These include nonconjugate variational message passing based on an adjusted pseudolikelihood and stochastic variational inference. To overcome the computational hurdle of drawing a network from the likelihood at each iteration, we propose stochastic gradient ascent with biased but consistent gradient estimates computed using adaptive self-normalized importance sampling. These methods provide attractive fast alternatives to MCMC for posterior approximation. We illustrate the variational methods using real networks and compare their accuracy with results obtained via MCMC and Laplace approximation. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.11973546
title: Poisson Kernel-Based Clustering on the Sphere: Convergence Properties, Identifiability, and a Method of Sampling

description: Spherical or directional data arise in many applications of interest. Furthermore, many nondirectional datasets can be usefully re-expressed in the form of directions and analyzed as spherical data. We have proposed a clustering algorithm using mixtures of Poisson-kernel-based densities (PKBD) on the sphere. We prove convergence of the associated generalized EM-algorithm, investigate the identifiability of various forms of PKBD mixture model, and study in detail its performance via simulation and application to real data. Specifically, we discuss a method and simulate data from a variety of PKBD models, then study the performance of the algorithm in terms of adjusted Rand index, macro-precision, and macro-recall. Finally, we compare the PKBD clustering method with other algorithms for clustering data on the sphere. Supplementary materials are available online and provide proofs of the theoretical results and the associated computer code.
DOI: 10.6084/m9.figshare.12075876
title: Forward Event-Chain Monte Carlo: Fast Sampling by Randomness Control in Irreversible Markov Chains

description: Irreversible and rejection-free Monte Carlo methods, recently developed in physics under the name event-chain and known in statistics as piecewise deterministic Monte Carlo (PDMC), have proven to produce clear acceleration over standard Monte Carlo methods, thanks to the reduction of their random-walk behavior. However, while applying such schemes to standard statistical models, one generally needs to introduce an additional randomization for sake of correctness. We propose here a new class of event-chain Monte Carlo methods that reduces this extra-randomization to a bare minimum. We compare the efficiency of this new methodology to standard PDMC and Monte Carlo methods. Accelerations up to several magnitudes and reduced dimensional scalings are exhibited. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.12141069.v2
title: A Tree-Based Semi-Varying Coefficient Model for the COM-Poisson Distribution

description: We propose a tree-based semi-varying coefficient model for the Conway–Maxwell–Poisson (CMP or COM-Poisson) distribution which is a two-parameter generalization of the Poisson distribution and is flexible enough to capture both under-dispersion and over-dispersion in count data. The advantage of tree-based methods is their scalability to high-dimensional data. We develop CMPMOB, an estimation procedure for a semi-varying coefficient model, using model-based recursive partitioning (MOB). The proposed framework is broader than the existing MOB framework as it allows node-invariant effects to be included in the model. To simplify the computational burden of the exhaustive search employed in the original MOB algorithm, a new split point estimation procedure is proposed by borrowing tools from change point estimation methodology. The proposed method uses only the estimated score functions without fitting models for each split point and, therefore, is computationally simpler. Since the tree-based methods only provide a piece-wise constant approximation to the underlying smooth function, we further propose the CMPBoost semi-varying coefficient model which uses the gradient boosting procedure for estimation. The usefulness of the proposed methods are illustrated using simulation studies and a real example from a bike sharing system in Washington, DC. Supplementary files for this article are available online.
DOI: 10.6084/m9.figshare.12141111
title: Improving Estimation in Functional Linear Regression With Points of Impact: Insights Into Google AdWords

description: The functional linear regression model with points of impact (PoI) is a recent augmentation of the classical functional linear model with many practically important applications. In this article, however, we demonstrate that the existing data-driven procedure for estimating the parameters of this regression model can be very instable and inaccurate. The tendency to omit relevant PoI is a particularly problematic aspect resulting in omitted-variable biases. We explain the theoretical reason for this problem and propose a new sequential estimation algorithm that leads to significantly improved estimation results. Our estimation algorithm is compared with the existing estimation procedure using an in-depth simulation study. The applicability is demonstrated using data from Google AdWords, today’s most important platform for online advertisements. The R-package FunRegPoI and additional R-codes are provided in the online supplementary materials.
DOI: 10.6084/m9.figshare.11932974.v3
title: Consensus Monte Carlo for Random Subsets Using Shared Anchors

description: We present a consensus Monte Carlo algorithm that scales existing Bayesian nonparametric models for clustering and feature allocation to big data. The algorithm is valid for any prior on random subsets such as partitions and latent feature allocation, under essentially any sampling model. Motivated by three case studies, we focus on clustering induced by a Dirichlet process mixture sampling model, inference under an Indian buffet process prior with a binomial sampling model, and with a categorical sampling model. We assess the proposed algorithm with simulation studies and show results for inference with three datasets: an MNIST image dataset, a dataset of pancreatic cancer mutations, and a large set of electronic health records. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.11973546.v2
title: Poisson Kernel-Based Clustering on the Sphere: Convergence Properties, Identifiability, and a Method of Sampling

description: Spherical or directional data arise in many applications of interest. Furthermore, many nondirectional datasets can be usefully re-expressed in the form of directions and analyzed as spherical data. We have proposed a clustering algorithm using mixtures of Poisson-kernel-based densities (PKBD) on the sphere. We prove convergence of the associated generalized EM-algorithm, investigate the identifiability of various forms of PKBD mixture model, and study in detail its performance via simulation and application to real data. Specifically, we discuss a method and simulate data from a variety of PKBD models, then study the performance of the algorithm in terms of adjusted Rand index, macro-precision, and macro-recall. Finally, we compare the PKBD clustering method with other algorithms for clustering data on the sphere. Supplementary materials are available online and provide proofs of the theoretical results and the associated computer code.
DOI: 10.6084/m9.figshare.12075876.v3
title: Forward Event-Chain Monte Carlo: Fast Sampling by Randomness Control in Irreversible Markov Chains

description: Irreversible and rejection-free Monte Carlo methods, recently developed in physics under the name event-chain and known in statistics as piecewise deterministic Monte Carlo (PDMC), have proven to produce clear acceleration over standard Monte Carlo methods, thanks to the reduction of their random-walk behavior. However, while applying such schemes to standard statistical models, one generally needs to introduce an additional randomization for sake of correctness. We propose here a new class of event-chain Monte Carlo methods that reduces this extra-randomization to a bare minimum. We compare the efficiency of this new methodology to standard PDMC and Monte Carlo methods. Accelerations up to several magnitudes and reduced dimensional scalings are exhibited. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.11932974.v2
title: Consensus Monte Carlo for Random Subsets Using Shared Anchors

description: <b>We present a consensus Monte Carlo algorithm that scales existing Bayesian nonparametric models for clustering and feature allocation to big data. The algorithm is valid for any prior on random subsets such as partitions and latent feature allocation, under essentially any sampling model. Motivated by three case studies, we focus on clustering induced by a Dirichlet process mixture sampling model, inference under an Indian buffet process prior with a binomial sampling model, and with a categorical sampling model. We assess the proposed algorithm with simulation studies and show results for inference with three datasets: an MNIST image dataset, a dataset of pancreatic cancer mutations, and a large set of electronic health records. Supplementary materials for this article are available online.</b>
DOI: 10.6084/m9.figshare.11973546.v1
title: Poisson Kernel-Based Clustering on the Sphere: Convergence Properties, Identifiability, and a Method of Sampling

description: Spherical or directional data arise in many applications of interest. Furthermore, many nondirectional datasets can be usefully re-expressed in the form of directions and analyzed as spherical data. We have proposed a clustering algorithm using mixtures of Poisson-kernel-based densities (PKBD) on the sphere. We prove convergence of the associated generalized EM-algorithm, investigate the identifiability of various forms of PKBD mixture model, and study in detail its performance via simulation and application to real data. Specifically, we discuss a method and simulate data from a variety of PKBD models, then study the performance of the algorithm in terms of adjusted Rand index, macro-precision, and macro-recall. Finally, we compare the PKBD clustering method with other algorithms for clustering data on the sphere. Supplementary materials are available online and provide proofs of the theoretical results and the associated computer code.
DOI: 10.6084/m9.figshare.12075876.v2
title: Forward Event-Chain Monte Carlo: Fast sampling by randomness control in irreversible Markov chains

description: Irreversible and rejection-free Monte Carlo methods, recently developed in physics under the name Event-Chain and known in Statistics as Piecewise Deterministic Monte Carlo (PDMC), have proven to produce clear acceleration over standard Monte Carlo methods, thanks to the reduction of their random-walk behavior. However, while applying such schemes to standard statistical models, one generally needs to introduce an additional randomization for sake of correctness. We propose here a new class of Event-Chain Monte Carlo methods that reduces this extra-randomization to a bare minimum. We compare the efficiency of this new methodology to standard PDMC and Monte Carlo methods. Accelerations up to several magnitudes and reduced dimensional scalings are exhibited.
DOI: 10.6084/m9.figshare.12141111.v2
title: Improving Estimation in Functional Linear Regression With Points of Impact: Insights Into Google AdWords

description: The functional linear regression model with points of impact (PoI) is a recent augmentation of the classical functional linear model with many practically important applications. In this article, however, we demonstrate that the existing data-driven procedure for estimating the parameters of this regression model can be very instable and inaccurate. The tendency to omit relevant PoI is a particularly problematic aspect resulting in omitted-variable biases. We explain the theoretical reason for this problem and propose a new sequential estimation algorithm that leads to significantly improved estimation results. Our estimation algorithm is compared with the existing estimation procedure using an in-depth simulation study. The applicability is demonstrated using data from Google AdWords, today’s most important platform for online advertisements. The R-package FunRegPoI and additional R-codes are provided in the online supplementary materials.
DOI: 10.6084/m9.figshare.12413117
title: Delayed Acceptance ABC-SMC

description: Approximate Bayesian computation (ABC) is now an established technique for statistical inference used in cases where the likelihood function is computationally expensive or not available. It relies on the use of a model that is specified in the form of a simulator, and approximates the likelihood at a parameter value <i>θ</i> by simulating auxiliary data sets <i>x</i> and evaluating the distance of <i>x</i> from the true data <i>y</i>. However, ABC is not computationally feasible in cases where using the simulator for each <i>θ</i> is very expensive. This article investigates this situation in cases where a cheap, but approximate, simulator is available. The approach is to employ delayed acceptance Markov chain Monte Carlo within an ABC sequential Monte Carlo sampler to, in a first stage of the kernel, use the cheap simulator to rule out parts of the parameter space that are not worth exploring, so that the “true” simulator is only run (in the second stage of the kernel) where there is a reasonable chance of accepting proposed values of <i>θ</i>. We show that this approach can be used quite automatically, with few tuning parameters. Applications to stochastic differential equation models and latent doubly intractable distributions are presented. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.12844499.v2
title: Trace Ratio Optimization for High-Dimensional Multi-Class Discrimination

description: In multi-class discrimination with high-dimensional data, identifying a lower-dimensional subspace with maximum class separation is crucial. We propose a new optimization criterion for finding such a discriminant subspace, which is the ratio of two traces: the trace of between-class scatter matrix and the trace of within-class scatter matrix. Since this problem is not well-defined for high-dimensional data, we propose to regularize the within trace and maximize the between trace. A careful investigation reveals that this optimization has an innate connection to the eigenvalue decomposition of an indefinite matrix. For the sake of better interpretability of the classifier, we also consider a sparse estimation via a group-wise soft-thresholding. Interesting relationships between the proposed method and some classical methods such as Fisher’s linear discriminant analysis and its variants are discussed. Empirical examples with simulated and real datasets suggest that the proposed method works well and is often better than some existing approaches in a wide range of problems, with respect to both variable selectivity and classification accuracy. Supplementary files for this article are available online.
DOI: 10.6084/m9.figshare.12413117.v3
title: Delayed Acceptance ABC-SMC

description: Approximate Bayesian computation (ABC) is now an established technique for statistical inference used in cases where the likelihood function is computationally expensive or not available. It relies on the use of a model that is specified in the form of a simulator, and approximates the likelihood at a parameter value <i>θ</i> by simulating auxiliary data sets <i>x</i> and evaluating the distance of <i>x</i> from the true data <i>y</i>. However, ABC is not computationally feasible in cases where using the simulator for each <i>θ</i> is very expensive. This article investigates this situation in cases where a cheap, but approximate, simulator is available. The approach is to employ delayed acceptance Markov chain Monte Carlo within an ABC sequential Monte Carlo sampler to, in a first stage of the kernel, use the cheap simulator to rule out parts of the parameter space that are not worth exploring, so that the “true” simulator is only run (in the second stage of the kernel) where there is a reasonable chance of accepting proposed values of <i>θ</i>. We show that this approach can be used quite automatically, with few tuning parameters. Applications to stochastic differential equation models and latent doubly intractable distributions are presented. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.12844499
title: Trace Ratio Optimization for High-Dimensional Multi-Class Discrimination

description: In multi-class discrimination with high-dimensional data, identifying a lower-dimensional subspace with maximum class separation is crucial. We propose a new optimization criterion for finding such a discriminant subspace, which is the ratio of two traces: the trace of between-class scatter matrix and the trace of within-class scatter matrix. Since this problem is not well-defined for high-dimensional data, we propose to regularize the within trace and maximize the between trace. A careful investigation reveals that this optimization has an innate connection to the eigenvalue decomposition of an indefinite matrix. For the sake of better interpretability of the classifier, we also consider a sparse estimation via a group-wise soft-thresholding. Interesting relationships between the proposed method and some classical methods such as Fisher’s linear discriminant analysis and its variants are discussed. Empirical examples with simulated and real datasets suggest that the proposed method works well and is often better than some existing approaches in a wide range of problems, with respect to both variable selectivity and classification accuracy. Supplementary files for this article are available online.
DOI: 10.6084/m9.figshare.12844499.v1
title: Trace Ratio Optimization for High-Dimensional Multi-Class Discrimination

description: In multi-class discrimination with high-dimensional data, identifying a lower-dimensional subspace with maximum class separation is crucial. We propose a new optimization criterion for finding such a discriminant subspace, which is the ratio of two traces: the trace of between-class scatter matrix and the trace of within-class scatter matrix. Since this problem is not well-defined for high-dimensional data, we propose to regularize the within trace and maximize the between trace. A careful investigation reveals that this optimization has an innate connection to the eigenvalue decomposition of an indefinite matrix. For the sake of better interpretability of the classifier, we also consider a sparse estimation via a group-wise soft-thresholding. Interesting relationships between the proposed method and some classical methods such as Fisher’s linear discriminant analysis and its variants are discussed. Empirical examples with simulated and real data sets suggest that the proposed method works well and is often better than some existing approaches in a wide range of problems, with respect to both variable selectivity and classification accuracy.