Response to ‘Inconsistencies and inaccuracies in reporting on choice of endpoints and of statistical results in RCT of maternal diet by Lewis et al.’

Dear Editor, Thank you for allowing us the opportunity to address Lewis et al.’s query regarding our publication ‘Impact of maternal diet on neonatal anthropometry: a randomized controlled trial’, in Pediatric Obesity. Regarding the query on trial registration, we respectfully fail to see how a reader could draw the conclusion that the Donnelly et al. publication was anything other than an exploratory analysis of anthropometric outcomes using data from the original ROLO trial (http://www.isrctn.com/ISRCTN54392969). In contrast to Lewis et al.’s claims, anthropometric measures were mentioned in the trial protocol registration, and did not form part of the set of primary and secondary end points, which were analysed for the original ROLO trial (1), which was published in 2012. The trial registration mentions neonatal anthropometry including birth weight, length and head circumference but did not exclude the measurement of further anthropometric measurements on a subgroup. The Donnelly et al. study was a secondary data analysis, consisting of exploratory comparisons of the original intervention and control groups as clearly detailed in the paper: ‘As part of the study, neonatal anthropometric measurements were taken on a subgroup of 265 neonates. One hundred twenty-six (47.5%) were in the intervention group and 139 (52.5%) were in the control group’. It therefore does not conform to a typical epidemiological observational study, as the randomized intervention is a key predictor, and benefits from many of the rigorous standards of the original study. Lewis et al. also state that the protocol does not list glucose intolerance as a primary or secondary outcome; however, it is clearly stated in the protocol that at 12 weeks, fasting blood glucose (and insulin, leptin and IGF-1) will be taken and that they would be used to assess baseline insulin resistance and glucose status. It is also stated in the trial protocol that participants will undergo a glucose challenge test at 28 weeks to determine the presence of gestational diabetes, i.e. degree of glucose intolerance. While this was not a primary outcome, the results of this analysis have been published (1,2). We thank Lewis et al. for discovering and highlighting the typographical error in the abstract on the mean thigh circumference in the control group, which eluded both our, the peer reviewers, and the editor’s notice. We emphasize that the P-value presented for a group comparison of thigh circumference was correct (P = 0.04) and our conclusions remain unaltered; Lewis et al.’s estimate of P = 0.0116 differs entirely because of the imprecision of the summary statistics from which it was derived. The error in table 3 resulted from a miscalculation of the mean summation of skin-folds, which were, as Lewis et al. pointed out, 28.48 mm SD ± 5.6 in the intervention group and 28.74 mm SD ± 5.2 in the control group. The conclusion arising from a t-test comparing these groups are unchanged (P = 0.723). Multiple testing issues are contentious in medical research. There is a clear need for, on one hand, intervention studies that can be evaluated quantitatively via prespecified tests and non-subjective metrics, while on the other hand, there must be room for unplanned discoveries to arise from exploratory research and for those findings be communicated to the scientific community within the peerreview framework. Lewis et al. should not confuse the demands for transparent and reproducible research, which we support, with an exigency that all research be conducted at the level of evidence expected of a blinded randomized controlled trial. The readers are referred to Vandenbroucke (3) and to Andrew Gelman’s paper ‘The garden of forking paths’ (4) for enlightening discussions of these issues, which all biomedical researchers should be aware of while conducting their own research and reviewing published findings. To specifically address the use of multiple testing corrections, in the context of Donnelly et al.’s secondary data analysis, the epidemiological view is that each test arguably stands on its own as an independent clinical hypothesis even if not statistically independent, and that exploring many outcomes with validation through replication is a realistic approach in the absence of strong a priori hypotheses. The commonly used Bonferroni procedure is known to be extremely conservative in cases of tests on several correlated scores, with an increased type II error rate leading to the exclusion of excess true effects. As Lewis et al. correctly point out, the thigh circumference difference would not be significant at a Bonferroniadjusted significance threshold of 0.0036. However, the balance of type I and type II errors in exploratory medical research should be in favour of avoiding the latter – that is to achieve adequate power to detect unexpected effects. The study of Donnelly et al. was adequately powered for individual anthropometric measures to detect a clinically relevant difference of 0.5 cm, and furthermore, we believe that the conclusions drawn were not hyperbolic in their presentation, accompanied as they were by a discussion of the limitations of thigh circumference as a measure of adiposity and a final paragraph calling for further research. While we regret the confusion Lewis et al. encountered on reading Donnelly et al. in the context of the published L E T T E R T O T H E E D IT O R