Structural covariance analysis is a promising and increasingly used structural Magnetic Resonance Imaging (MRI ) analysis method which characterises the co-relations of morphology between brain regions over a group of subjects. However, to our knowledge, little has been investigated in terms of the comparability of results between different data sets of healthy human subjects as well as the reliability of results over  different rescan sessions and FreeSurfer versions. Our results show (significant) differences of structural covariance between two data sets of age- and sex-matched healthy human adults. This low comparability is unaltered by site correction, and is most severe when using average cortical thickness as a measure of cortical morphology. The low comparability further extends to significant differencesin graph theoretic measures. Similar results of low reliabilitywere found when comparing rescan sessions of the same subjects, and even between different FreeSurfer versions of the same scans. To understand our observations of low reliability, we estimated a measurement error covariance, and we show that it is largest in cortical thickness. On simulated datawe further demonstrated that stronger errors cause stronger attenuation of correlations and proportionally decreases reliability, which agrees with associations we observed in the real data. To our knowledge this study is the first highlight the problem of reliability and comparability in structural covariance . Practically, we suggest that (1) combining data from different scanners and sites for structural covariance analysis in a naive manner should be avoided,  (2) surface area and volume should be preferred as morphological measures of structural covariance over cortical thickness , and (3) some analysis of robustness of the results should be performed for every structural covariancestudy .
Structural covariance analysis is a widely used structural MRI analysis method which characterises the co-relations of morphology between brain regions over a group of subjects. To our knowledge, little has been investigated in terms of the comparability of results between different data sets or the reliability of results over the same subjects in different rescan sessions , image resolutions, or FreeSurfer versions. In terms of comparability, our results show substantial differences in the structural covariance matrix between data sets of age- and sex-matched healthy human adults. These differences persist after site correction, they are exacerbated by low sample sizes, and they are most pronounced when using average cortical thickness as a morphological measure. Down-stream graph theoretic analyses further show statistically significant differences. In terms of reliability, substantial differences were also found when comparing repeated scan sessions of the same subjects, and image resolutions and FreeSurfer versions of the same image. We could further estimate the relative measurement error and showed that it is largest when using thickness. With simulated data, we argue that cortical thickness is least reliable because of larger relative measurement errors . Practically, we make the following recommendations (1) pooling subjects across sites into one group should be avoided, particularly if sites differ in image resolutions, demographics, or preprocessing; (2) surface area and volume should be preferred as morphological measures over cortical thickness ; (3) a large number of subjects should be used to estimate structural covariance; (4) measurement error should be assessed where repeated measurements are available; (5) if combining sites is critical, univariate site-correction is insufficient, but error covariance should be explicitly measured and modelled .
R
promising and increasingly used structural Magnetic Resonance Imaging (MRI )
widely used structural MRI
R
However, to
To
R
of healthy human subjects as well as
or
A
the same subjects in
R
and
, image resolutions, or
R
Our results show (significant) differences of structural covariance between two
In terms of comparability, our results show substantial differences in the structural covariance matrix between
R
This low comparability is unaltered by
These differences persist after
R
and is most severe
they are exacerbated by low sample sizes, and they are most pronounced
R
measure of cortical morphology. The low comparability further extends to significant differencesin graph theoretic measures. Similar results of low reliabilitywere
morphological measure. Down-stream graph theoretic analyses further show statistically significant differences. In terms of reliability, substantial differences were also
R
rescan
repeated scan
R
even between different
image resolutions and
R
scans. To understand our observations of low reliability, we estimated a measurement error covariance, and we show
image. We could further estimate the relative measurement error and showed
R
in cortical thickness. On simulated datawe further demonstrated that stronger errors cause stronger attenuation of correlations and proportionally decreases reliability, which agrees with associations we observed in the real data. To our knowledge this study is the first highlight the problem of reliability and comparability in structural covariance
when using thickness. With simulated data, we argue that cortical thickness is least reliable because of larger relative measurement errors
R
suggest that
make the following recommendations
R
combining data from different scanners and sites for structural covariance analysis in a naive manner
pooling subjects across sites into one group
A
particularly if sites differ in image resolutions, demographics, or preprocessing;
D
of structural covariance
R
, and
;
R
some analysis of robustness of the results should be performed for every structural covariancestudy
a large number of subjects should be used to estimate structural covariance; (4) measurement error should be assessed where repeated measurements are available; (5) if combining sites is critical, univariate site-correction is insufficient, but error covariance should be explicitly measured and modelled