Reply to: "The data do not support the existence of an 'Old Boy network' in science. Some critical comments on a study by Massen et al."
Venue: Scientific Reports
Year: 2020

We commend Van de Velde and Heller for their goal of keeping high scientific standards, and underline the importance of replication. We too are vested in such standards and are genuinely interested in the results of alternative statistical analyses. That said, we feel that the reported results do not harm the conclusions from our original paper, namely, that male scientists are very likely to follow up on requests to share their science with other males, and that such helping behavior was less frequently observed between any other sex combination of requester and participant1. Whereas the alternative analyses presented by Van de Velde and Heller may cast some uncertainty on the found effect, they did yield comparable results, and importantly showed a similar trend (P 0.061) for the reported interaction. We hope that our disagreement about the specific analyses will not distract us from the core problem: structural gender biases in academia. Van de Velde and Heller take issue with the interpretation of the three way interaction between sex of the requester, sex of the participant and condition on the likelihood of scientist sharing their science, which we found to be statistically significant in our original study1. They do so based on the contrasts produced by the statistical software they used. However, these contrasts are not very informative when concerning a three way interaction. The reason being that a three way interaction reports variation in a two way interaction across the third level and this is not reflected in contrasts between each combination of these three factors. In our case that was the two way interaction between sex of the requester and sex of the participant across conditions, and this was also verified by the post hoc analyses (with corrections) that we provided in the original manuscript; i.e. both the two way interaction being significant, and a significant difference in effect size of this two way interaction between the two conditions. We acknowledge that it might have been more parsimonious to test for the two way interaction effect while controlling for condition, and are pleased to see that when doing this Van de Velde and Heller find a similar trend with respect to the two way interaction between the sex of the requester and the sex of the participant (P 0.061) Second, Van de Velde and Heller take issue with our modeling procedure, as they argue we have violated the principle of marginality. This principle, however, has been heavily debated2,3, and in fact, when clear hypotheses are provided, like we did in our original study, it is even advised to deviate from this principle4. Importantly, however, when Van de Velde and Heller reproduce model selection according to their standards, the crucial interaction term (i.e. sex of requester with sex of participant) is maintained in the final model, and, albeit not statistically significant at P 0.05, again they find a similar trend (P 0.061; see the supplements to their comment) There are multiple procedures one can follow when analyzing data. The interaction between sex of the requester and sex of the participant can also be analyzed using a dummy variable that reflects the sex combination of both (FF, FM, MF, and MM) In our original analyses (and submission) we did this and also here this dummy variable was maintained in the best fitting model, and we found that sex combination had a significant open