A total of 159 of 312 reviews included free-text comments applicable for the methodological study. Wrong sampling unit was mentioned by one fourth of 80 referees. Only one referee mentioned the incorrect use of a parametric test in the analysis of data whose distribution was nonparametric. An English version seemed to be accepted more easily than a national-language version of the same manuscript. Most referees spontaneously mentioned the shortcomings of the manuscripts only as part of their overall judgment.
Referees differ in many respects, such as time spent on reviewing and experience as a referee.
The project also included a study of the usefulness of referees' statements to editors and their ability to detect and comment spontaneously on common flaws and shortcomings. Both manuscripts presented cross-sectional studies comparing patient characteristics with a control group. The manuscripts also contained suboptimal standards of problem formulation, interpretation, language style, and so on. These qualitative characteristics could, for obvious reasons, only be measured semiquantitatively. For safety reasons, both manuscripts only described pathogenetic aspects of the two conditions to avoid even a hypothetical influence on the poststudy behavior of referees, who were all clinicians.