The ability to quantify complex relationships within multivariate time series is a key component of modelling many physical systems, from the climate to brains and other biophysical phenomena . Unfortunately, even testing the significance of simple dependence measures, such as Pearson correlation, is complicated by altered sampling properties when autocorrelation is present in the individual time series . Moreover, it has been recently established that commonly used multivariate dependence measures---such as Granger causality---can produce substantially inaccurate results when applying classical hypothesis-testing procedures to digitally-filtered time series. Here , we suggest thatthe digital filtering-induced bias in Granger causality is an effect of autocorrelation, and we present a principled statistical framework for the hypothesis testing of a large family of linear-dependence measures between multiple autocorrelated time series. Our approach unifies the theoretical foundations established by Bartlett and others on variance estimators for autocorrelated signals with the more intricate multivariate measures of linear dependence. Specifically, we derive the sampling distributions and subsequent hypothesis tests for any measure that can be decomposed into terms that involve independent partial correlations, which we show includes Granger causality and mutual information under a multivariate linear-Gaussian model. In doing so, we provide the first exact tests for inferring linear dependence between vector autoregressive processes with limited data. Using numerical simulations and brain-imaging datasets, we demonstrate that our newly developed tests maintain the expected false-positive rate (FPR) with minimally-sufficient samples, while the classical log-likelihood ratio tests can yield an unbounded FPR depending on the parameters chosen.\\%DIF > for both measures, with and without prewhitening of the signals. These findings suggest that many dependencies reported in the scientific literature may have been, and may continue to be, spuriously reported or missed if modified hypothesis tests are not used when analysing time series.
Inferring linear dependence between time series is central to our understanding of natural and artificial systems . Unfortunately, the hypothesis tests that are used to determine statistically significant directed or multivariate relationships from time-series data often yield spurious associations (Type I errors) or omit causal relationships (Type II errors). This is due to the autocorrelation present in the analysed time series -- a property that is ubiquitous across diverse applications, from brain dynamics to climate change. Here we show that, for limited data, this issue cannot be mediated by fitting a time-series model alone (e.g., in Granger causality or prewhitening approaches), and instead that the degrees of freedom in statistical tests should be altered to account for the effective sample size induced by cross-correlations in the observations. This insight enabled us to derive modified hypothesis tests for any multivariate correlation-based measures of linear dependence between covariance-stationary time series, including Granger causality and mutual information with Gaussian marginals. We use both numerical simulations (generated by autoregressive models and digital filtering) as well as recorded fMRI-neuroimaging data to show that our tests are unbiased for a variety of stationary time series. Our experiments demonstrate that the commonly used F- and \chi^2-tests can induce significant false-positive rates of up to 100\\%DIF > for both measures, with and without prewhitening of the signals. These findings suggest that many dependencies reported in the scientific literature may have been, and may continue to be, spuriously reported or missed if modified hypothesis tests are not used when analysing time series.
R
The ability to quantify complex relationships within multivariate
Inferring linear dependence between
R
a key component of modelling many physical systems, from the climate to brains and other biophysical phenomena
central to our understanding of natural and artificial systems
R
even testing the significance of simple dependence measures, such as Pearson correlation, is complicated by altered sampling properties when autocorrelation is
the hypothesis tests that are used to determine statistically significant directed or multivariate relationships from time-series data often yield spurious associations (Type I errors) or omit causal relationships (Type II errors). This is due to the autocorrelation
R
individual time series . Moreover, it has been recently established that commonly used multivariate dependence measures---such as Granger causality---can produce substantially inaccurate results when applying classical hypothesis-testing procedures to digitally-filtered time series. Here , we suggest thatthe digital filtering-induced bias
analysed time series -- a property that is ubiquitous across diverse applications, from brain dynamics to climate change. Here we show that, for limited data, this issue cannot be mediated by fitting a time-series model alone (e.g.,
R
is an effect of autocorrelation, and we present a principled statistical framework for the hypothesis testing of a large family of linear-dependence measures between multiple autocorrelated time series. Our approach unifies the theoretical foundations established by Bartlett and others on variance estimators for autocorrelated signals with the more intricate multivariate measures of linear dependence. Specifically, we derive the sampling distributions and subsequent
or prewhitening approaches), and instead that the degrees of freedom in statistical tests should be altered to account for the effective sample size induced by cross-correlations in the observations. This insight enabled us to derive modified
R
measure that can be decomposed into terms that involve independent partial correlations, which we show includes
multivariate correlation-based measures of linear dependence between covariance-stationary time series, including
R
under a multivariate linear-Gaussian model. In doing so, we provide the first exact tests for inferring linear dependence between vector autoregressive processes with limited data. Using numerical simulations and brain-imaging datasets, we demonstrate that our newly developed tests maintain the expected
with Gaussian marginals. We use both numerical simulations (generated by autoregressive models and digital filtering) as well as recorded fMRI-neuroimaging data to show that our tests are unbiased for a variety of stationary time series. Our experiments demonstrate that the commonly used F- and \chi^2-tests can induce significant
R
rate (FPR) with minimally-sufficient samples, while the classical log-likelihood ratio tests can yield an unbounded FPR depending on the parameters chosen.
rates of up to 100