cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.6084/m9.figshare.8156327.v1
title: Anomaly Detection in Streaming Nonstationary Temporal Data

description: This article proposes a framework that provides early detection of anomalous series within a large collection of nonstationary streaming time-series data. We define an anomaly as an observation, that is, very unlikely given the recent distribution of a given system. The proposed framework first calculates a boundary for the system’s typical behavior using extreme value theory. Then a sliding window is used to test for anomalous series within a newly arrived collection of series. The model uses time series features as inputs, and a density-based comparison to detect any significant changes in the distribution of the features. Using various synthetic and real world datasets, we demonstrate the wide applicability and usefulness of our proposed framework. We show that the proposed algorithm can work well in the presence of noisy nonstationarity data within multiple classes of time series. This framework is implemented in the open source R package <i>oddstream</i>. R code and data are available in the online supplementary materials.
DOI: 10.6084/m9.figshare.8156327.v2
title: Anomaly Detection in Streaming Nonstationary Temporal Data

description: This article proposes a framework that provides early detection of anomalous series within a large collection of nonstationary streaming time-series data. We define an anomaly as an observation, that is, very unlikely given the recent distribution of a given system. The proposed framework first calculates a boundary for the system’s typical behavior using extreme value theory. Then a sliding window is used to test for anomalous series within a newly arrived collection of series. The model uses time series features as inputs, and a density-based comparison to detect any significant changes in the distribution of the features. Using various synthetic and real world datasets, we demonstrate the wide applicability and usefulness of our proposed framework. We show that the proposed algorithm can work well in the presence of noisy nonstationarity data within multiple classes of time series. This framework is implemented in the open source R package <i>oddstream</i>. R code and data are available in the online supplementary materials.
DOI: 10.6084/m9.figshare.9034331
title: Parallelization of a Common Changepoint Detection Method

description: In recent years, various means of efficiently detecting changepoints have been proposed, with one popular approach involving minimizing a penalized cost function using dynamic programming. In some situations, these algorithms can have an expected computational cost that is linear in the number of data points; however, the worst case cost remains quadratic. We introduce two means of improving the computational performance of these methods, both based on parallelizing the dynamic programming approach. We establish that parallelization can give substantial computational improvements: in some situations the computational cost decreases roughly quadratically in the number of cores used. These parallel implementations are no longer guaranteed to find the true minimum of the penalized cost; however, we show that they retain the same asymptotic guarantees in terms of their accuracy in estimating the number and location of the changes. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9116864
title: Bivariate Residual Plots With Simulation Polygons

description: When using univariate models, goodness of fit can be assessed through many different methods, including graphical tools such as half-normal plots with a simulation envelope. This is straightforward due to the notion of ordering of a univariate sample, which can readily reveal possible outliers. In the bivariate case, however, it is often difficult to detect extreme points and verify whether a sample of residuals is a reasonable realization from a fitted model. We propose a new framework, implemented as the bivrp R package, available on CRAN. Our framework uses the same principles of the simulation envelope in a half-normal plot, but as a simulation polygon for each point in a bivariate sample. By using algorithms of convex hull construction and polygon area reduction, we describe how our method works and illustrate its functionality with examples using simulated bivariate normal data and real bivariate count data. We show how different model diagnostics can produce different results and pinpoint potential drawbacks of our approach, such as the limitations in terms of computational burden. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9172628
title: Bayesian model averaging over tree-based dependence structures for multivariate extremes

description: Describing the complex dependence structure of extreme phenomena is particularly challenging. To tackle this issue we develop a novel statistical method that describes extremal dependence taking advantage of the inherent tree-based dependence structure of the max-stable nested logistic distribution, and that identifies possible clusters of extreme variables using reversible jump Markov chain Monte Carlo techniques. Parsimonious representations are achieved when clusters of extreme variables are found to be completely independent. Moreover, we significantly decrease the computational complexity of full likelihood inference by deriving a recursive formula for the likelihood function of the nested logistic model. The method’s performance is verified through extensive simulation experiments which also compare different likelihood procedures. The new methodology is used to investigate the dependence relationships between extreme concentrations of multiple pollutants in California and how these concentrations are related to extreme weather conditions. Overall, we show that our approach allows for the representation of complex extremal dependence structures and has valid applications in multivariate data analysis, such as air pollution monitoring, where it can guide policymaking.
DOI: 10.6084/m9.figshare.8295500
title: A Function Emulation Approach for Doubly Intractable Distributions

description: Doubly intractable distributions arise in many settings, for example, in Markov models for point processes and exponential random graph models for networks. Bayesian inference for these models is challenging because they involve intractable normalizing “constants” that are actually functions of the parameters of interest. Although several computational methods have been developed for these models, each can be computationally burdensome or even infeasible for many problems. We propose a novel algorithm that provides computational gains over existing methods by replacing Monte Carlo approximations to the normalizing function with a Gaussian process-based approximation. We provide theoretical justification for this method. We also develop a closely related algorithm that is applicable more broadly to any likelihood function that is expensive to evaluate. We illustrate the application of our methods to challenging simulated and real data examples, including an exponential random graph model, a Markov point process, and a model for infectious disease dynamics. The algorithm shows significant gains in computational efficiency over existing methods, and has the potential for greater gains for more challenging problems. For a random graph model example, we show how this gain in efficiency allows us to carry out accurate Bayesian inference when other algorithms are computationally impractical. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.8967998
title: Testing Sparsity-Inducing Penalties

description: Many penalized maximum likelihood estimators correspond to posterior mode estimators under specific prior distributions. Appropriateness of a particular class of penalty functions can therefore be interpreted as the appropriateness of a prior for the parameters. For example, the appropriateness of a lasso penalty for regression coefficients depends on the extent to which the empirical distribution of the regression coefficients resembles a Laplace distribution. We give a testing procedure of whether or not a Laplace prior is appropriate and accordingly, whether or not using a lasso penalized estimate is appropriate. This testing procedure is designed to have power against exponential power priors which correspond to ℓq penalties. Via simulations, we show that this testing procedure achieves the desired level and has enough power to detect violations of the Laplace assumption when the numbers of observations and unknown regression coefficients are large. We then introduce an adaptive procedure that chooses a more appropriate prior and corresponding penalty from the class of exponential power priors when the null hypothesis is rejected. We show that this can improve estimation of the regression coefficients both when they are drawn from an exponential power distribution and when they are drawn from a spike-and-slab distribution. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.8398058
title: Diagonal Discriminant Analysis With Feature Selection for High-Dimensional Data

description: We introduce a new method of performing high-dimensional discriminant analysis (DA), which we call multiDA. Starting from multiclass diagonal DA classifiers which avoid the problem of high-dimensional covariance estimation we construct a hybrid model that seamlessly integrates feature selection components. Our feature selection component naturally simplifies to weights which are simple functions of likelihood ratio test statistics allowing natural comparisons with traditional hypothesis testing methods. We provide heuristic arguments suggesting desirable asymptotic properties of our algorithm with regard to feature selection. We compare our method with several other approaches, showing marked improvements in regard to prediction accuracy, interpretability of chosen features, and fast run time. We demonstrate such strengths of our model by showing strong classification performance on publicly available high-dimensional datasets, as well as through multiple simulation studies. We make an R package available implementing our approach. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9762731.v2
title: Valid Inference Corrected for Outlier Removal

description: Ordinary least square (OLS) estimation of a linear regression model is well-known to be highly sensitive to outliers. It is common practice to (1) identify and remove outliers by looking at the data and (2) to fit OLS and form confidence intervals and <i>p</i>-values on the remaining data as if this were the original data collected. This standard “detect-and-forget” approach has been shown to be problematic, and in this article we highlight the fact that it can lead to invalid inference and show how recently developed tools in selective inference can be used to properly account for outlier detection and removal. Our inferential procedures apply to a general class of outlier removal procedures that includes several of the most commonly used approaches. We conduct simulations to corroborate the theoretical results, and we apply our method to three real datasets to illustrate how our inferential results can differ from the traditional <i>detect-and-forget</i> strategy. A companion R package, outference, implements these new procedures with an interface that matches the functions commonly used for inference with lm in R. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9805397
title: Fast Nonseparable Gaussian Stochastic Process With Application to Methylation Level Interpolation

description: Gaussian stochastic process (GaSP) has been widely used as a prior over functions due to its flexibility and tractability in modeling. However, the computational cost in evaluating the likelihood is O(n3), where <i>n</i> is the number of observed points in the process, as it requires to invert the covariance matrix. This bottleneck prevents GaSP being widely used in large-scale data. We propose a general class of nonseparable GaSP models for multiple functional observations with a fast and exact algorithm, in which the computation is linear (<i>O</i>(<i>n</i>)) and exact, requiring no approximation to compute the likelihood. We show that the commonly used linear regression and separable models are special cases of the proposed nonseparable GaSP model. Through the study of an epigenetic application, the proposed nonseparable GaSP model can accurately predict the genome-wide DNA methylation levels and compares favorably to alternative methods, such as linear regression, random forest, and localized Kriging method. The supplementary materials of this article are online and the algorithm for fast computation is implemented in the FastGaSP R package on CRAN. Supplemental materials for this article are available online.
DOI: 10.6084/m9.figshare.9805397.v1
title: Fast Nonseparable Gaussian Stochastic Process With Application to Methylation Level Interpolation

description: Gaussian stochastic process (GaSP) has been widely used as a prior over functions due to its flexibility and tractability in modeling. However, the computational cost in evaluating the likelihood is O(n3), where <i>n</i> is the number of observed points in the process, as it requires to invert the covariance matrix. This bottleneck prevents GaSP being widely used in large-scale data. We propose a general class of nonseparable GaSP models for multiple functional observations with a fast and exact algorithm, in which the computation is linear (<i>O</i>(<i>n</i>)) and exact, requiring no approximation to compute the likelihood. We show that the commonly used linear regression and separable models are special cases of the proposed nonseparable GaSP model. Through the study of an epigenetic application, the proposed nonseparable GaSP model can accurately predict the genome-wide DNA methylation levels and compares favorably to alternative methods, such as linear regression, random forest, and localized Kriging method. The supplementary materials of this article are online and the algorithm for fast computation is implemented in the FastGaSP R package on CRAN. Supplemental materials for this article are available online.
DOI: 10.6084/m9.figshare.9764813
title: Compressed and Penalized Linear Regression

description: Modern applications require methods that are computationally feasible on large datasets while retaining good statistical properties. Recent work has focused on developing fast and randomized approximations for solving least squares problems when the data are too large to fit into memory easily or when computations are at a premium. Many of these techniques rely on data-driven subsampling or random compression. In this article, we provide new approximate algorithms for solving penalized least squares problems which have improved statistical performance relative to existing methods. We provide the first efficient methods for tuning parameter selection, compare our methods with current approaches via simulation and application, and provide theoretical intuition which makes explicit the impact of approximation on statistical efficiency and demonstrates the necessity of careful parameter tuning. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9764813.v2
title: Compressed and Penalized Linear Regression

description: Modern applications require methods that are computationally feasible on large datasets while retaining good statistical properties. Recent work has focused on developing fast and randomized approximations for solving least squares problems when the data are too large to fit into memory easily or when computations are at a premium. Many of these techniques rely on data-driven subsampling or random compression. In this article, we provide new approximate algorithms for solving penalized least squares problems which have improved statistical performance relative to existing methods. We provide the first efficient methods for tuning parameter selection, compare our methods with current approaches via simulation and application, and provide theoretical intuition which makes explicit the impact of approximation on statistical efficiency and demonstrates the necessity of careful parameter tuning. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9764813.v1
title: Compressed and Penalized Linear Regression

description: Modern applications require methods that are computationally feasible on large datasets while retaining good statistical properties. Recent work has focused on developing fast and randomized approximations for solving least squares problems when the data are too large to fit into memory easily or when computations are at a premium. Many of these techniques rely on data-driven subsampling or random compression. In this article, we provide new approximate algorithms for solving penalized least squares problems which have improved statistical performance relative to existing methods. We provide the first efficient methods for tuning parameter selection, compare our methods with current approaches via simulation and application, and provide theoretical intuition which makes explicit the impact of approximation on statistical efficiency and demonstrates the necessity of careful parameter tuning. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9954989
title: Heteroscedastic BART via Multiplicative Regression Trees

description: Bayesian additive regression trees (BART) has become increasingly popular as a flexible and scalable nonparametric regression approach for modern applied statistics problems. For the practitioner dealing with large and complex nonlinear response surfaces, its advantages include a matrix-free formulation and the lack of a requirement to prespecify a confining regression basis. Although flexible in fitting the mean, BART has been limited by its reliance on a constant variance error model. Alleviating this limitation, we propose HBART, a nonparametric heteroscedastic elaboration of BART. In BART, the mean function is modeled with a sum of trees, each of which determines an additive contribution to the mean. In HBART, the variance function is further modeled with a product of trees, each of which determines a multiplicative contribution to the variance. Like the mean model, this flexible, multidimensional variance model is entirely nonparametric with no need for the prespecification of a confining basis. Moreover, with this enhancement, HBART can provide insights into the potential relationships of the predictors with both the mean and the variance. Practical implementations of HBART with revealing new diagnostic plots are demonstrated with simulated and real data on used car prices and song year of release. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9959627.v1
title: Testing One Hypothesis Multiple Times: The Multidimensional Case

description: The identification of new rare signals in data, the detection of a sudden change in a trend, and the selection of competing models are among the most challenging problems in statistical practice. These challenges can be tackled using a test of hypothesis where a nuisance parameter is present only under the alternative, and a computationally efficient solution can be obtained by the “testing one hypothesis multiple times” (TOHM) method. In the one-dimensional setting, a fine discretization of the space of the non identifiable parameter is specified, and a global <i>p</i>-value is obtained by approximating the distribution of the supremum of the resulting stochastic process. In this article, we propose a computationally efficient inferential tool to perform TOHM in the multidimensional setting. Here, the approximations of interest typically involve the expected Euler characteristics (EC) of the excursion set of the underlying random field. We introduce a simple algorithm to compute the EC in multiple dimensions and for arbitrarily large significance levels. This leads to an highly generalizable computational tool to perform hypothesis testing under nonstandard regularity conditions. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9693164
title: Scalable Bayesian Inference for Coupled Hidden Markov and Semi-Markov Models

description: Bayesian inference for coupled hidden Markov models frequently relies on data augmentation techniques for imputation of the hidden state processes. Considerable progress has been made on developing such techniques, mainly using Markov chain Monte Carlo (MCMC) methods. However, as the dimensionality and complexity of the hidden processes increase some of these methods become inefficient, either because they produce MCMC chains with high autocorrelation or because they become computationally intractable. Motivated by this fact we developed a novel MCMC algorithm, which is a modification of the forward filtering backward sampling algorithm, that achieves a good balance between computation and mixing properties, and thus can be used to analyze models with large numbers of hidden chains. Even though our approach is developed under the assumption of a Markovian model, we show how this assumption can be relaxed leading to minor modifications in the algorithm. Our approach is particularly well suited to epidemic models, where the hidden Markov chains represent the infection status of an individual through time. The performance of our method is assessed on simulated data on epidemic models for the spread of <i>Escherichia coli</i> O157:H7 in cattle. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9959627
title: Testing One Hypothesis Multiple Times: The Multidimensional Case

description: The identification of new rare signals in data, the detection of a sudden change in a trend, and the selection of competing models are among the most challenging problems in statistical practice. These challenges can be tackled using a test of hypothesis where a nuisance parameter is present only under the alternative, and a computationally efficient solution can be obtained by the “testing one hypothesis multiple times” (TOHM) method. In the one-dimensional setting, a fine discretization of the space of the non identifiable parameter is specified, and a global <i>p</i>-value is obtained by approximating the distribution of the supremum of the resulting stochastic process. In this article, we propose a computationally efficient inferential tool to perform TOHM in the multidimensional setting. Here, the approximations of interest typically involve the expected Euler characteristics (EC) of the excursion set of the underlying random field. We introduce a simple algorithm to compute the EC in multiple dimensions and for arbitrarily large significance levels. This leads to an highly generalizable computational tool to perform hypothesis testing under nonstandard regularity conditions. Supplementary materials for this article are available online.
DOI: 10.6084/m9.figshare.9954926
title: Longitudinal Principal Component Analysis With an Application to Marketing Data

description: We propose a longitudinal principal component analysis method for multivariate longitudinal data using a random-effects eigen-decomposition, where the eigen-decomposition uses longitudinal information through nonparametric splines and the multivariate random effects incorporate significant store-wise heterogeneity. Our method can effectively analyze large marketing data containing sales information for products from hundreds of stores over an 11-year time period. The proposed method leads to more accurate estimation and interpretation compared to existing approaches. We illustrate our method through simulation studies and an application to marketing data from IRI. Supplementary materials for this article are available online.