DOI: 10.5061/dryad.6nh4fc2
title: Data from: The effectiveness of journals as arbiters of scientific quality

description: Academic publishers purport to be arbiters of knowledge, aiming to publish
 studied that advance the frontiers of their research domain. Yet the
 effectiveness of journal editors at identifying novel and important
 research is generally unknown, in part because of the confidential nature
 of the editorial and peer-review process. Using questionnaires, we
 evaluated the degree to which journals are effective arbiters of
 scientific impact in the domain of Ecology, quantified by three key
 criteria. First, journals discriminated against low-impact manuscripts:
 the probability of rejection increased as the number of citations gained
 by the published paper decreased. Second, journals were more likely to
 publish high-impact manuscripts (those that obtained citations in 90th
 percentile for their journal) than run-of-the-mill manuscripts; editors
 were only 23 and 41% as likely to reject an eventual high-impact paper
 (pre- versus post-review rejection) compared to a run-of-the-mill paper.
 Third, editors did occasionally reject papers that went on to be highly
 cited. Error rates were low, however: only 3.8% of rejected papers gained
 more citations than the median article in the journal that rejected them,
 and only 9.2% of rejected manuscripts went on to be high-impact papers in
 the (generally lower impact factor) publishing journal. The effectiveness
 of scientific arbitration increased with journal prominence, although some
 highly prominent journals were no more effective than much less prominent
 ones. We conclude that the academic publishing system, founded on peer
 review, appropriately recognises the significance of research contained in
 manuscripts, as measured by the number of citations that manuscripts
 obtain after publication, even though some errors are made. We therefore
 recommend that authors reduce publication delays by choosing journals
 appropriate to the significance of their research.
description: Questionnaire data for
 DRYAD 2018 07 26Data from questionnaires
 sent to authors of papers in the research domain of Ecology. The column
 header are mostly self-explanatory. They are: ResponseID: a serial number
 linking rows corresponding to multiple submissions of the same manuscript
 (MS). Note that some MSs were submitted repeatedly to the same journal.
 Invited MS: Was the manuscript invited by the journal for publication?
 Round start: A serial number for the round of submission of each
 manuscript, beginning with 1, and incrementing upwards. Round end: A
 serial number for the outcome of each submission round, beginning with 2
 and incrementing upwards. Note that the final round of submission is given
 identifier 99. Journal start: To which journal was the MS submitted in
 this round? Journal end: To which journal was the MS next submitted? Note
 that this may be the same as Journal Start, if the manuscript was not
 rejected. If it was rejected, then it will (typically) differ. JIF start:
 What was the submission-year-specific impact factor of the starting
 journal? JIF end: What was the submission-year-specific impact factor of
 the ending journal? Year start: In what year was the submission made? Year
 end: In what year was the next submission made? Outcome: What was the
 outcome of the submission to the starting journal? cumulative n journals:
 To how many journals had this MS previously been submitted? n citations:
 How many citations were obtained by this article, once published? Note
 that N citations was current as of 16 August 2017.