Research depends on rapid dissemination of results, and computer science researchers have been naturally attracted to the idea of placing their papers on the Web. Indeed, in 1998 the Computing Research Repository (CoRR) was established as a partnership between the ACM, arXiv.org, and others as an online repository for research papers in computer science [Halpern, 2000] . However, CoRR seems to have had fairly limited popularity for the first decade of its existence (Section 3), with researchers preferring to post e-prints on their individual web sites [McCallum et al., 2000a , Giles et al., 1998 ]. At the same time, we are seeing a budding revolution in the way that publication and peer review is conducted in computer science. Some researchers have argued for all submissions and reviews to be public [LeCun, 2009] , and for four years the International Conference on Learning Representations (ICLR) has followed this model successfully, increasing from 67 submissions in 2013 up to 647 submissions in 2017. 1 To support experimentation with new and open reviewing models, the OpenReview system [ope] was designed specifically to allowing conferences to configure a wide range of reviewing and dissemination policies and levels of openness; a good overview of the debate about reviewing models is provided by the accompanying paper [Soergel et al., 2013] . An important aspect of how research is disseminated in computer science is whether, where, and when papers are made electronically available. For many years, researchers have made papers available via individual home pages, and some researchers use or are mandated to use institutional repositories. However, public e-print services like CoRR/arXiv have the advantage of allowing researchers to receive automatic notifications when new relevant papers are uploaded (and incidentally, also of facilitating the study of the literature in aggregate, as in the current paper). There is also the question of when papers should be made public, i.e. immediately after the authors judge them ready for public consumption, which we will call prepublication, or only after they have been accepted by a peer reviewed conference or journal. Advocates argue that prepublication can have a dramatic effect on the rate of change in a field. For example, in deep learning, it is now impossible to stay up to date without following submissions in arXiv closely. In some well-known cases, by the time an influential paper has been presented in a conference, follow-on work has already appeared, and indeed, is sometimes presented in the same conference session. However, an important disadvantage of pervasive prepublication is that it significantly complicates double-blind reviewing, as a centralized e-print repository can send alerts to many potential reviewers. Many research communities within computer science have had multi-year discussions about the tradeoffs among e-print servers, prepublication, and reviewing models, with concomitant town hall meetings at conferences, blog posts, and so on. 2 As the comments on some of these blogs indicate, the debate among researchers can be spirited, even if it is carried out with only the most limited of evidence. Although all indications are that computer scientists are prepublishing their articles more and more, there is little information about the size of this trend or how it varies across areas of computer science. We aim to quantify the extent to which computer scientists deposit papers in centralized e-print repositories, and whether these e-prints are preprints that are made public before or during the review process, or whether e-prints are typically posted only upon the paper's acceptance into a peer-reviewed venue. Specifically, we measure the percentage of computer science publications that have been posted as e-prints to arXiv.org. We consider all papers in the past decade that have been published in 63 of the most selective conferences across computer science, and matched their metadata to the arXiv preprint server. Our main findings are: 1 The text and reviews of all accepted and rejected ICLR 2017 submissions are available at
With the evergrowing rates of articles being submitted to toptier conferences in Computer Science and pre-print repositories REF , there is a need to expedite the peer review process.
Popularity of arXiv.org within Computer Science
ArXiv
ArXiv
Computer Science