IS PEER REVIEW IN DECLINE

I. INTRODUCTION For the past half-century or more, peer-reviewed journals have played a central role in the evaluation and dissemination of scientific research. The Internet has enhanced scientific communication in many ways, and there is considerable excitement around new institutions for disseminating research. (1) A more sobering thought, however, is that new technologies can also be disruptive. This paper begins with some documentation of recent trends in economics publishing. Roughly, the trends involve economists from top-ranked departments publishing fewer papers in many top journals. This could be a welcome change but is also potentially troublesome: one way in which the Internet could be disruptive is that it could allow high-profile researchers to disseminate their work without subjecting it to peer review, which in turn could lead to a broader unraveling of the peer-review system. The majority of the paper is then devoted to analyses of several additional data sources with the goal of understanding better what may be causing the observed trends. The facts part of this paper documents two main facts: 1. Economists in top-ranked departments now publish very few papers in top field journals. There is a marked decline in such publications between the early 1990s and early 2000s. 2. Comparing the early 2000s with the early 1990s, there is a decline in both the absolute number of papers and the share of papers in the top general interest journals written by Harvard economics department faculty. Although the second fact just concerns one department, I see it as potentially important to understanding what is happening because it comes at a time when Harvard is widely regarded (I believe correctly) as having ascended to the top position in the profession. The "decline-of-peer-review" theory I allude to in the title is that the necessity of going through the peer-review process has lessened for high-status authors: in the old days peer-reviewed journals were by far the most effective means of reaching readers, whereas with the growth of the Internet high-status authors can now post papers online and exploit their reputation to attract readers. Many alternate explanations are possible. I focus on four theories: the decline-in-peer-review theory and three alternatives. 1. The trends could be a consequence of top-school authors' being crowded out of the top journals by other researchers. Several such stories have an optimistic message, for example, there is more talent entering the profession, old pro-elite biases are being broken down, more schools are encouraging faculty to do cutting-edge research, and the Internet is enabling more cutting-edge research by breaking down informational barriers that had hampered researchers outside the top schools. (2) 2. The trends could be a consequence of the growth of revisions at economics journals discussed in Ellison (2002a, 2002b). In this more pessimistic theory, highly productive researchers must abandon some projects and/or seek out faster outlets to conserve the time now required to publish their most important works. 3. The trends could simply reflect that field journals have declined in quality in some relative sense and become a less attractive place to publish. This theory is meant to encompass also the rise of new journals, which is not obviously desirable or undesirable. The majority of this paper is devoted to examining various data sources that provide additional details about how economics publishing has changed over the past decade. These are intended both to sharpen understanding of the facts to be explained and to provide tests of auxiliary predictions of the theories. Two main sources of information are used: data on publications and data on citations. The publication data include department-level counts of publications in various additional journals, an individual-level dataset containing records of publications in a subset of journals for thousands of economists, and a very small dataset containing complete data on a few authors' publication records. …