${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

Review Boards for Scientiﬁc Publishingnathan hagen
While the spread of open access publishing for technical and scientific papers has
improved some long-standing problems in the scientific publishing pipeline, it
has worsened others. Meanwhile, peer review and the scientific publication system
in general have come under increasingly intense criticism, provoking many reform
ideas. Despite the strident language of reformists and the widespread opinion
that the situation is worsening, reform ideas have generally received a lukewarm
response among researchers. I argue that this complacency is a reaction to reforms
that ignore the priorities of readers and how readers’ needs have shaped the
publishing world today. I outline a path for improving the peer-review systemthrough the use of permanent review boards — to accommodate the needs ofreaders, reviewers, and authors alike — and show how to get from where we are
now to where we should be.
Keywords: peer review, scientific communication, scholarly publishing

introduction
While the open access movement has made sweeping changes to thescientiﬁc publishing landscape, there is a widespread impression thatthe changes are incomplete, that a second set of reforms are needed
to make scientiﬁc and technical work more accessible and useful to thescientiﬁc community.1 Editors are ﬁnding it increasingly difﬁcult tolocate qualiﬁed reviewers for manuscripts, reviewers complain of themounting burden on their time and of authors taking advantage of thepeer-review system, and authors complain of reviewer misbehaviour andof unfairness in the system. Calls for reform have included opening up
peer review to public scrutiny or replacing peer review with different
evaluation mechanisms, removing restrictions on publication, and making
the whole pre- and post-publication phase more interactive with readers
and reviewers.
While these are all attractive ideas for improving the current state ofaffairs, the reform literature has so far been unsuccessful in bringing
Journal of Scholarly Publishing July 2018 doi: 10.3138/jsp.49.4.03

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

420

Journal of Scholarly Publishingabout widespread changes to the scientiﬁc publishing world. A prominent
reason for researchers’ lukewarm response to reform proposals is thatreform discussions generally focus on the perspectives of authors andpublishers but ignore the fundamental needs of readers. It is the needsof readers (the audience of fellow researchers and other informed consumers of original scientiﬁc research) as much as the needs of publishers
themselves that have shaped the scientiﬁc publishing world today, andthis perspective is essential for a realistic assessment of what reforms
are promising and possible, and what are not.
The overriding need of readers that shapes the scientiﬁc publishinglandscape is the need to locate research results that are both interesting
and relevant to a topic of interest, in the face of an ocean of material thatis either uninteresting or irrelevant. To this end, journals ask peer
reviewers to evaluate whether submitted manuscripts are both a good ﬁt
for the journal’s topical area and likely to be of interest to the journal’s
readership. These two ﬁltering criteria are thus designed to help readerssift through and cull the literature. This is essential because we as readersof scientiﬁc literature spend a signiﬁcant part of our time looking forinteresting new ideas, as a means of studying a subject or of developing
new research directions. Naturally, we try to make this time spentsearching and selecting as efﬁcient as possible. Peer review and the focus
of subject-speciﬁc journals are the primary ways we ﬁlter information
and reduce the volume of the vast scientiﬁc literature. In this sense, we
can say that the majority of the scientiﬁc readership wants the difﬁculties
and frustrations that peer review causes to authors. Even a ﬂawed ﬁlter is
far more useful to readers than none at all. This tension between theneeds of authors to reach as wide an audience as possible and the needsof readers to limit the literature to as narrow a stream as possible is thefundamental problem that drives scientiﬁc communication and yet is
not widely appreciated.
While readers’ needs are underappreciated, writers too have legitimate
complaints. Yet while many writers complain of the ugly featuresof peer review — reviewers’ haste and their lack of objectivity andprofessionalism — the scientiﬁc community has yet to ﬁnd a ﬁltering
mechanism that can replace peer review with something better. The
need for human evaluation of manuscripts, by those who should havesufﬁcient technical knowledge on the subject to assess their value, is something we have been unable to escape. As a result, reviewers are called on

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

Review Boards for Scientiﬁc Publishingto assume the gatekeeping role of literature ﬁlters for journal readers.
While the peer-review system has ﬂaws just as any human institution
does, it has suffered accumulating criticism of late, and many see it as
the weak point in the current scientiﬁc publishing pipeline. I propose
to strengthen it and make it more ﬂexible at the same time by creating
permanent bodies of reviewers — review boards — with established andpermanent names by which others can recognize them. This would be agroup of journal reviewers that specialize in a speciﬁc area, who would
agree to come together as a single body under the journal’s guidance
and form a review board for reviewing manuscripts. In the area of
optical engineering, for example, a lens design review board can cometogether and agree to review manuscripts on that subject. While this may
seem no different than the current system, it has the critical difference thatthe review board would be a ﬁxed group with long-standing membershipso that it has an enduring interest in maintaining its reputation. This
feedback and self-policing mechanism is missing from today’s ad hoc
review teams assembled individually for each manuscript.
In the discussion below, I ﬁrst survey some of the landscape of
proposed publishing reform and consider how the various ideas can beﬁtted together into a coherent reform program that meets the needs ofreaders while also reducing some of the strains in scientiﬁc publishing. I
show how permanent review boards can be formed with little disruption
to the existing system and argue that this simple organization can form
the foundation needed for implementing many of the reforms.

survey of peer-review reform ideas
A common criticism of peer review questions whether peer review‘works’ at all and suggests that the answer depends on what we thinkpeer review is for.2 While most researchers suggest that peer review is amethod of selecting the best papers to publish, or a means of improving
the quality of publications, a better answer is that peer review is primarily
for readers — to help select what to read. It is only secondarily to improve
quality, though the two are of course related. And in this light, thecriticism that peer review ‘doesn’t work’3 sounds hollow since no one
has been able to suggest an alternative that does work. For example, there
are often complaints that peer review fails to catch technical errors.
However, it is optimistic to expect reviewers to detect many technicalerrors in a couple hours of their time. Instead, what a reviewer can say

421

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

422

Journal of Scholarly Publishingfor certain is whether he or she understands the work and whether it
seems important and interesting. So, rather than consider ideas ofeliminating peer review, I instead consider how we might improve on
the existing system and shore up its weaknesses.
A common complaint among authors is that the publishing process is
far more difﬁcult than it should be. Reviewers are often unreasonable
and nasty;4 publishers charge exorbitant prices to archive manuscripts
and place them behind pay walls; journals restrict the kind of resultsthat can be published,5 and so on. Reviewers, too, have their grievances.
The review process is more inefﬁcient than it should be; authors areoften lazy and submit work that is poorly explained and difﬁcult toread; and many manuscripts pass through multiple review cycles withdifferent journals, wasting reviewers’ time. Finally, even reviewers feelthat binary pass/fail decisions (the ultimate decision to accept or rejecta manuscript that may follow rounds of revision) are an overly crude
mechanism for evaluating manuscripts. These complaints about thecurrent system have prompted a number of reform ideas, a representative
list of which includes the following:
Minimize publishing costs — Allow authors to archive manuscripts
for only a nominal fee.6
Pre- versus post-publication review — Use post-publication reviewsof manuscripts instead of pre-publication reviews.7
Graded versus binary review — Rather than giving a binary-valued
accept/reject review, allow reviewers to apply a numerical grade to apaper.8
Open versus blind review — Use open reviews, in which thereviewers are known and their comments can be viewed by everyone.9
Permanent review boards — Gather subject-expert reviewers into
permanent bodies that can manage themselves, instead of the current
system of forming ad hoc teams for each manuscript.
I consider each of these reform ideas in turn.
Minimize Publishing Costs
One reform proposal is to eliminate the barriers to publishing andallow authors to publish whatever and whenever they want. This promises
to reduce costs to authors, improve the speed of publication, and avoid

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

Review Boards for Scientiﬁc Publishingcontent restrictions such as the inability to publish negative results.10
The successful open preprint archiving system arXiv is a model for thisidea,11 motivating some writers to claim that there is no longer a needfor journals to host manuscripts at all.12 Rather, manuscripts can behosted in an archive, allowing a journal to focus instead on acting as
a curation interface to the archive. The Discrete Analysis journal, forexample, was formed along these lines and has demonstrated that it ispossible to run an ‘overlay journal’ in this way and keep costs under$10 per manuscript.13
While this proposal excites some authors with its potential, it hasreceived a lukewarm response in the scientiﬁc community generally.
Authors who use arXiv for their manuscripts typically go on to submit
their preprints to peer-reviewed journals14 — behaviour that may seem
surprising, given that the gatekeeping function of peer review is probably
the greatest cause of frustration with the current system. However, whenviewed from the perspective of readers, we can see why the responseto this idea has been weak: eliminating the gatekeeping function ofjournals — the ﬁnancial barriers to publishing and the need to meet therequirements of reviewers — would likely produce a ﬂood of material.
The worry is that quality will suffer and that readers will have difﬁcultyﬁltering the literature to locate manuscripts of interest. Thus, the overlay
journal Discrete Analysis, while removing ﬁnancial barriers to publication, implicitly acknowledges the priority of reader needs by preserving
the gatekeeping function of peer review.
An additional worry about automatic archiving is that many papers
will suffer from unclear writing and grammatical errors. Journals currently
minimize these errors by requiring papers to be copy-edited prior to
publication, and this greatly improves the readability of the literature.
With automatic archiving, we can expect increased frustration withpoor writing quality, so it will likely prove necessary to indicate in some
way that an archived paper has been not just peer reviewed but also
editorially endorsed, allowing readers to ﬁlter out those manuscriptsthat are likely of poor writing quality.
We can conclude that as long as we maintain readers’ ability to ﬁlter
the stream of publications according to their interests, and at a level ofquality that they want, then minimizing publishing costs through lowcost archiving will be accepted by the community. Making this work
will require that publishers provide sufﬁcient metadata for ﬁltering423

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

424

Journal of Scholarly Publishingalgorithms to customize to each reader’s needs and will also require thedevelopment of a user-friendly platform for developing individualized
ﬁltering algorithms.
Pre- versus Post-review
In the current publication pipeline, a manuscript is submitted by anauthor to a journal, the journal editor assembles a team of reviewers toevaluate the paper, and reviewers are asked to recommend whether or
not the manuscript is a good ﬁt for the journal. In what can be considered a requirement for the previous idea of immediate archiving ofmanuscripts, one reform proposal is to eliminate the gatekeeping function of peer review — that is, we should allow submitted manuscripts to
be archived without review and instead perform evaluations after publication. A touted advantage for post-publication evaluation of manuscripts isthat it is open-ended, allowing for something like the comment forums
often a part of Internet blogs.15
Because this proposal eliminates the pre-publication ﬁlter that is so
useful for readers, such a reform would need to provide an alternativeﬁltering mechanism, such as attaching a review ‘badge’ or grade to apaper to signal to readers that the paper has been evaluated by peers
and was considered a noteworthy research report.16
Graded versus Binary Review
While peer reviewers under the current system are asked to evaluate
manuscripts with a binary decision (approved or not approved for
publication, often after rounds of revision), a more useful measure forﬁltering purposes is a numerical grade evaluation.17 This ﬁts in nicely
with the ideas of post-publication review and automatic archiving, sinceunevaluated papers can be labelled as such, and an evaluation grade
allows readers to construct ﬁlters based on grade. A reader, for example,
could choose to look at only those publications that have been reviewed
and have received grades better than, say, three out of ﬁve. Adjusting thelevel of the ﬁlter would allow a reader to tighten or loosen the stream ofmaterial that the ﬁlter permits through.
Open versus Blind Review
The impressive success of the open-source software movement, andits use of community reviewing of code to improve quality, has inﬂuenced

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

Review Boards for Scientiﬁc Publishing

many to try something similar for reviews of scholarly manuscripts.
Whereas reviewer evaluations are typically closed and cannot be viewed
by readers, it has been suggested that evaluations should instead be open
and that open review would promote better behaviour by reviewers since
their work would be publicly scrutinized.18 There is, however, disagreement about whether the quality of reviews would be improved by public
exposure,19 and attempts at measuring the positive or negative effects ofpublic exposure have been frustrated by methodological problems ofevaluation and quantiﬁcation.20 Making the problem worse, journals
making use of open peer review have also found that a signiﬁcant numberof reviewers refuse requests to review manuscripts out of concern for the
public scrutiny.21
While open peer review can come in many forms, the success of any
one depends on whether it serves readers’ needs. Looking at journals
that have implemented different comment forums for manuscripts,
however, we see that these forums have so far attracted little interest,22
indicating that readers do not ﬁnd them useful. This should probably
come as no surprise when we consider that readers already feel swampedby the amount of material to read, so the low information density provided
by comment forums is often a poor use of readers’ time.
Although advocates of peer-review reform are generally sanguine
about the potential of open review, the journals that use open review
show clearly that, while reviewers are better behaved in their writing,they are also more reluctant to reject papers as uninteresting. This may
be great for authors, but it is a major problem from the perspective of
readers, who need a strong ﬁlter mechanism. It may be that permanent
review boards can play a useful role here to make open review a practicalreality by protecting individuals from censure through their membershipin a group that is collectively responsible for reviewing.
Permanent Review Boards
In the most common form of manuscript review for modern journals,
an assistant editor assigned to the manuscript seeks reviewers among thevarious researchers that he or she considers subject-area experts. Manyrequests are declined, so the editor typically must send several requests
sequentially to assemble a volunteer review team of two or three. The
inevitable result is that the process of locating reviewers is a burden for
editors. In addition, researchers currently have little incentive to devote425

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

426

Journal of Scholarly Publishingserious effort to performing reviews because the work is performed
anonymously and without reward.
One way of addressing these issues is to form groups of subject-area
reviewers into independent review boards. If the boards are establishedas permanent entities and keep membership records, reviewer members
can receive recognition for their work, much as journal editors arerecognized for theirs. Moreover, a permanent body can also cultivate areputation, giving them visibility and an incentive to maintain theirreputation by giving high-quality reviews. An example of implementing
such a board would be to bring together as members any potential
reviewers recommended to the board and who also agree to review aminimum of x papers per year. Each manuscript under review by theboard is assigned to three members, for example, with two submitting
their reviews to the third for compiling the reports into a consensus
review. This could be as simple as appending the three and averaging
the three evaluation grades, or it could be more complex depending on
the accepted practices of a particular board. If the number of board
members is insufﬁcient, the board may agree to take on auxiliary reviewers,
while maintaining the requirement that the consensus is always compiled
by a board member. If such a group is sufﬁciently protective of its reputation, it may also require the reviewers to evaluate one another’s reviews.
As a result, a journal’s assistant editor can assign a paper to be reviewed
not by an ad hoc anonymous team but rather by a journal-sanctioned
board, with a list of members having subject-matter expertise, that has
performed many reviews and has consistently demonstrated its expertise.
By having board members agree in advance to perform a given numberof reviews each month, the burden of repeatedly assembling review
teams disappears — replaced by the lighter and more useful burden of
administering the board. Finally, the board’s reputation is that of thegroup rather than of the individuals, striking a balance between the safetyof anonymity within the group and the exposure of open review — agroup small enough to build a recognizable reputation yet large enoughto anonymize its members for any individual review. In addition, forthose who feel that paying reviewers for their effort is called for,23 thelong-term stability of the board and the obligation for members to
perform a given number of reviews over the duration of membershippaves the way for this to happen.

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

Review Boards for Scientiﬁc Publishing

how archiving and review boards affect the system
Each of the reform ideas discussed above shows promise for improving
the existing state of affairs. With a carefully designed system, the proposed reforms can be made to reinforce one another for a new scientiﬁcpublishing ecosystem.
An example from an author’s perspective on how a manuscript would
make its way through this new ecosystem is as follows. An author submits
a manuscript to a publisher’s online archive, paying a nominal fee to doso. The archivist performs a cursory, minimal review of the submission.
If the archivist determines that the submission is ‘not obviously of low
value,’24 then it is quickly posted to the archive. Once the manuscript isposted, the author, worried that any problems of grammar or clarity
may affect the peer-review grade, requests that it be editorially reviewed
for writing quality by paying a fee to the journal. After a few days,
the author receives the editor’s response, and after implementing thesuggested changes, the submission receives editorial approval and ismarked as such. The author next selects a review board for the subject
area matching the content of the research and submits a request forpeer review, paying another fee to the journal. (All these fees can, ofcourse, be bundled together instead of paid out piecemeal.) After acouple weeks, the board grades the paper for its level of interest, andthis grade is automatically attached to the manuscript. For an open
review system, the board’s comments will be attached to the manuscript
as well. The author responds to the reviewer comments and revises themanuscript and then requests that the board revise their review. The
board reevaluates the manuscript and updates their grade. At each stepalong this path, the publisher’s website maintains a daily feed of information, announcing all manuscripts whose status has changed during
the previous day. An algorithm can monitor this stream and ﬁlter it
based on any reader’s settings, restricting the material to only thosepapers that match the reader’s requirements of review grade, editorial
approval, keywords, and so forth.
From a reviewer’s perspective, the process appears as follows. On theﬁrst of each month, a board member receives an email containing a link
to a manuscript to review, email contacts for two other members of thereview team, and a pre-review summary generated by an evaluation
algorithm. The evaluation algorithm looks for things such as plagiarismand for related publications by the same authors — things that are often427

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

428

Journal of Scholarly Publishingeasier for an algorithm to do than for a human.25 Algorithms applied tocomplex data can be inaccurate, however, so the pre-review data areused as information for the reviewer to take under advisement. After
writing comments and grading the manuscript, the reviewer waits forthe remaining two reviewers to complete the same steps, after which
the full set of reviews becomes visible. Since he or she is the primary
reviewer for this manuscript, the reviewer compiles the consensus review
according to the board’s rules, possibly with some emails to ask theother reviewers for more feedback, and submits the ﬁnal result. The
consensus grade and comments are attached to the posted manuscript.
Finally, according to the board’s rules, the reviewers must also submit
grades for their fellow members’ work, and so the reviewer also submits
those to the review board’s Web interface.
From a reader’s perspective, the process is not much different fromwhat it is today, with the exception that the reader needs to supply thejournal with more speciﬁc information about the type of publication
he or she wants to receive notiﬁcation of. Thus, a reader can specify toreceive news of peer-reviewed papers with grades of three or higher inthe ﬁeld of optical engineering and grades of two or better in the ﬁeldof lens design. With this ﬁlter deﬁnition, the reader will receive a weekly
update email announcing every manuscript whose status has changed
during the week with those criteria.
A major advantage that this new ecosystem has over the existing one
is that by separating the content review and editorial review steps fromthe archiving step of manuscript preparation, it allows journals to nudge
authors into better performance. Authors are often lazy and are willingto submit sloppy writing, knowing that they will receive professional
editing without cost once the manuscript has received peer-review
approval. However, if journals separate the costs for archiving, editorial
correction, and peer review, then those authors requiring multiple
reviews, or heavy editorial effort, will be forced to pay higher fees.26
This rewards authors who are willing to take on some of this work
themselves and penalizes those who are not, providing healthier incentives for the system. While this clearly also penalizes non-ﬂuent writers,
it does so by removing an implicit subsidy in the current publishingpipeline. This can be explicitly inserted back into the system if the journal
desires, by redistributing the editorial costs to the archiving and peerreview steps, as is done now. The advantage of healthier incentives,

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

Review Boards for Scientiﬁc Publishing

however, is sacriﬁced — incentives whereby those who can perform areencouraged to do so, thereby reducing costs for everyone else.
A second advantage of the new ecosystem is that it also penalizes
the inefﬁciency caused by ‘forum shopping’: the common practice ofsubmitting work to the highest-regarded journal and, if rejected, thensubmitting to the next journal down and so forth until the work is
ﬁnally accepted.27 This often involves multiple submission and review
cycles, with a lot of redundant work. More recently, some authors havebegun to climb the ladder in the opposite direction by submitting tolower journals and, anticipating rejection, collecting reviewer comments
in order to learn how to massage the paper into one acceptable for ahigher journal. In the proposed ecosystem, because manuscripts are
typically not rejected outright but instead are given a permanent review
grade, there is no longer an incentive for multiple submissions in different
forums. After the ﬁrst submission, any second submission would be a
duplicate publication.
Below I summarize the various problems and how the proposals
address them, from the perspectives of the different roles: author, reviewer, editor, and reader. I follow this with a consideration of drawbacks to this alternative publishing system.
Author Grievances
Expense and wait — Automatic archiving makes the initial publication process almost instantaneous. The expenses of publishing, for
editorial work and for administering peer review, are difﬁcult to
reduce, but the authors who can afford to forgo them will have thefreedom to do so.
Reviewer misbehaviour — Review boards will police member
behaviour to maintain their reputation, and authors can choose
which board to send their manuscript to, among those approved by
the publisher archive.
Reviewer Grievances
Forum shopping — With automatic archiving, forum shopping is nolonger useful for authors.
Overburdened reviewers — Elimination of forum shopping decreases
the burden on reviewer time, and algorithmic pre-review can help429

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

430

Journal of Scholarly Publishinglighten some of the tasks that reviewers are called on to do, such asplagiarism checking. The use of graded reviews means that authors
will have a stronger incentive to write clearly, helping to reduce
review time.
Lack of reviewer recognition — The idea of improving recognition
for the work of reviewers is an issue that often comes up in
discussions of scientiﬁc publishing reform, and membership on
publisher review boards is a means of achieving this.
Editor Grievances
Authors creating false reviewers — Many journal submission systems
have recently started to ask authors for reviewer suggestions to help
lighten the load on editors. However, this has also caused someauthors to take advantage of the system and create fake reviewers.
With review boards, however, this type of unethical behaviour will bemuch more difﬁcult for authors to pull off.
Much time spent assembling review teams — Most of this burden istaken up by review boards.
Reader Grievances
Excessive focus on journal prestige — Journal prestige is widely used
as a proxy for research quality, but this association weakens when weallow for the differentiation of review board reputations. The prestige
of a journal is largely a by-product of the selectivity of its board, andthis becomes clearer when the two are differentiated from one another.
That is, readers will learn to focus more on the reputation of the board
and less on the name of the journal.
Drawbacks
 There is a clear bias against non-native writers of English in the

proposed pipeline. This can be mended by requiring all authors
to pay editorial expenses, as now, or with an explicit subsidy for
non-native speakers.
 In the current system, if a manuscript is rejected due to reviewer
misunderstanding, there are second chances. The author canrewrite and resubmit elsewhere. In the new system, some form of${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

Review Boards for Scientiﬁc Publishing

re-review appeal process will be needed to address such errors thatcreate bad evaluations. But if review boards are protective of theirreputations, they will maintain a mechanism for correcting any
errors that may occur.
 With the creation of review boards comes a set of new administrative tasks. It is possible that the overall burden on publishers andon reviewers may be left unchanged.

how to get there from here
Critics of the current publishing system suggest that radical changes arenecessary, but the discussion above makes clear that the changes needed
are actually moderate. While moving to a near-automatic archiving
system and the use of review boards may appear to be sweeping changes
to the publishing cycle, it is important to realize that they can be implemented with minimal disturbance to the current system. A publisher, forexample, can create a new journal name connected to this archive andallow authors to submit to the archive system or to the traditional
system in parallel, or can allow any manuscript evaluated with a highgrade to be published in the traditional journal instead of the archive.
Review boards, too, can be implemented gradually and without disturbing the current system. A journal can start by forming a coupleof small review boards with reviewers who agree to try the new system.
Editors can send a few manuscripts to the boards instead of working
to assemble review teams, and the consensus review can then be usedexactly as other reviews are currently.
Moreover, there is no need to eliminate the current subscription
system for journal readers. If publishers forgo charging authors for editing
and formatting, then the authors can in return agree to give the publishers
rights to restrict viewing of the papers to the set of paid subscribers —basically the same system that we have now. It seems likely that moving
to an archival system will eventually relax this long-standing system, butthere is little reason to think that it will disappear overnight.
conclusions
It has been little acknowledged that the central problem in scientiﬁc communication is the tension that exists between authors and readers — thebattle to reach a wide audience versus the battle to defend one’s time.
For the last century, readers have waged their battle using peer reviewers

431

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

432

Journal of Scholarly Publishing

to defend the gates to publication, a ﬁght that has come under increasingstrain with the multiplying number of gates to defend and the swellingarmies of authors assaulting them. It is time for both sides to switch
tactics and reduce the intensity of conﬂict.
I propose to establish an arXiv-like archiving system to improve thesituation for authors while also implementing review boards to give
reviewers the permanent administrative structure they need to have theirwork recognized and to let publishers make more efﬁcient use of theirskills. If implemented in tandem with the development of ﬁltering software tools that allow readers to take advantage of the improved ﬂexibility,
many of the choke points in the existing publishing pipeline will relax.
As a teacher, Professor nathan hagen currently works in the Optical Engineering
Department of Utsunomiya University in Japan. As an author, he publishes researchin optical science. As a reviewer, he was recently awarded an Outstanding Reviewer
Award by OSA, the Optical Society. His email address is nh@hagenlab.org and his
ORCID is 0000-0003-1122-9081.

notes
1. Brian A. Nosek and Yoav Bar-Anan, ‘Scientiﬁc Utopia: I. Opening Scientiﬁc
Communication,’ Psychological Inquiry 23 (2012): 217–44; Jonathan P. Tennant,
Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner,
Daniel Mietchen, Yehia Elkhatib, et al., ‘A Multi-disciplinary Perspective on
Emergent and Future Innovations in Peer Review,’ F1000 Research 6 (2017):
1151–214.
2. Richard Smith, ‘Peer Review: A Flawed Process at the Heart of Science andJournals,’ Journal of the Royal Society of Medicine 99 (2006): 178–82.
3. Larry Wasserman, ‘A World without Referees,’ February 20, 2012, http://
www.stat.cmu.edu/~larry/Peer-Review.pdf; John Bohannon, ‘Who’s Afraid ofPeer Review?,’ Science 342 (2013): 60–64; Richard Smith, ‘The Peer Review
Drugs Don’t Work,’ Times Higher Education, May 28, 2015, https://www.
timeshighereducation.com/content/the-peer-review-drugs-dont-work; Vitek Tracz,‘The Five Deadly Sins of Science Publishing,’ F1000 Research 4 (2015): 6962–68.
4. Aliaksandr Birukou, Joseph R. Wakeling, Claudio Bartolini, Fabio Casati, Maurizio
Marchese, Katsiaryna Mirylenka, Nardine Osman, et al., ‘Alternatives to Peer
Review: Novel Approaches for Research Evaluation,’ Frontiers in Computational
Neuroscience 5 (2011), doi:10.3389/fncom.2011.00056.
5. Jeffrey D. Scargle, ‘Publication Bias: The ‘File-Drawer’ Problem in ScientiﬁcInference,’ Journal of Scientific Exploration 14, no. 1 (2000): 91–106.

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

Review Boards for Scientiﬁc Publishing
6. Sharon J. Rogers and Charlene S. Hurt, ‘How Scholarly Communication Should
Work in the 21st Century,’ Chronicle of Higher Education, October 18, 1989,
https://www.chronicle.com/article/How-Scholarly-Communication/67083; Paul
Ginsparg, ‘Can Peer Review Be Better Focused?,’ Science and Technology Libraries
22 (2002): 5–19.
7. Nicolas Chopin, Andrew Gelman, Kerrie L. Mengersen, and Christian P. Robert,‘In Praise of the Referee,’ arXiv.org, May 19, 2012, http://arxiv.org/abs/1205.4304v1;
Richard Walker and Pascal Rocha da Silva, ‘Emerging Trends in Peer Review — a
Survey,’ Frontiers in Neuroscience 9 (2015), doi:10.3389/fnins.2015.00169.
8. John W. T. Smith, ‘The Deconstructed Journal — a New Model for Academic
Publishing,’ Learned Publishing 12 (1999): 79–91; Stefano Mizzaro, ‘Quality
Control in Scholarly Publishing: A New Proposal,’ Journal of the Association for
Information Science and Technology 54 (2003): 989–1005.
9. Smith, ‘Peer Review: A Flawed Process,’ 178; Axel Boldt, ‘Extending ArXiv.org
to Achieve Open Peer Review and Publishing,’ Journal of Scholarly Publishing 42(2011): 238–42; Christopher Lee, ‘Open Peer Review by a Selected-Papers Network,’
Frontiers in Computational Neuroscience 6 (2012), doi:10.3389/fncom.2012.00001;
Emily Ford, ‘Deﬁning and Characterizing Open Peer Review: A Review of the
Literature,’ Journal of Scholarly Publishing 44 (2013): 311–27.
10. Kay Dickersin and Yuan-I Min, ‘Publication Bias: The Problem That Won’t Go
Away,’ Annals of the New York Academy of Sciences 703 (1993): 135–48.
11. See ArXiv e-print service, https://arxiv.org; Paul Ginsparg, ‘First Steps towards
Electronic Research Communication,’ Computers in Physics 8 (1994): 390–97.
12. Andrew Gelman, ‘My Proposal for JASA,’ Statistical Modeling, Causal Inference,and Social Science (blog), April 19, 2017, http://andrewgelman.com/2017/04/19/
my-proposal-for-jasa/.
13. Rogers and Hurt, ‘How Scholarly Communication Should Work,’ A56; see
Discrete Analysis Journal, http://discreteanalysisjournal.com/.
14. Edwin A. Henneken, Michael J. Kurtz, Guenther Eichhorn, Alberto Accomazzi,
Carolyn S. Grant, Donna Thompson, Elizabeth Bohlen, et al., ‘E-prints andJournal Articles in Astronomy: A Productive Co-existence,’ Learned Publishing
20 (2007): 16–22; Vincent Larivire, Cassidy R. Sugimoto, Benoit Macaluso, Staša
Milojević, Blaise Cronin, and Mike Thelwall, ‘arXiv E-prints and the Journal of
Record: An Analysis of Roles and Relationships,’ Journal of the Association for
Information Science and Technology 65 (2014): 1157–69.
15. Nikolaus Kriegeskorte and Diana Deca, eds., Beyond Open Access: Visions forOpen Evaluation of Scientific Papers by Post-publication Peer Review (Lausanne,
Switzerland: Frontiers Media, 2012).

433

${protocol}://www.utpjournals.press/doi/pdf/10.3138/jsp.49.4.03 - Nathan Hagen <nh@hagenlab.org> - Friday, July 27, 2018 6:46:51 PM - IP Address:160.12.132.8

434

Journal of Scholarly Publishing
16. Jason Priem and Bradley M. Hemminger, ‘Decoupling the Scholarly Journal,’
Frontiers in Computational Neuroscience 6 (2012), doi:10.3389/fncom.2012.00019.
17. Smith, ‘Deconstructed Journal,’ 79.
18. Jelte M. Wicherts, Rogier A. Kievit, Marjan Bakker, and Denny Borsboom,‘Letting the Daylight In: Reviewing the Reviewers and Other Ways to Maximize
Transparency in Science,’ Frontiers in Computational Neuroscience 6 (2012),
doi:10.3389/fncom.2012.00020.
19. Ford, ‘Deﬁning and Characterizing Open Peer Review,’ 311; Elizabeth C. Moylan,
Simon Harold, Ciaran O’Neill, and Maria K. Kowalczuk, ‘Open, Single-Blind,
Double-Blind: Which Peer Review Process Do You Prefer?,’ BMC Pharmacology
and Toxicology 15 (2014): 55–59.
20. Tom Jefferson, Philip Alderson, Elizabeth Wager, and Frank Davidoff, ‘Effectsof Editorial Peer Review: A Systematic Review,’ Journal of the American Medical
Association 287 (2002): 2784–90.
21. Moylan et al., ‘Open, Single-Blind, Double-Blind,’ 55.
22. Christen Brownlee, ‘Peer Review under the Microscope,’ Science News 170(2006): 392–93.
23. Eleftherios P. Diamandis, ‘The Current Peer Review System Is Unsustainable —Awaken the Paid Reviewer Force!,’ Clinical Biochemistry 50 (2017): 461–63.
24. Paul Ginsparg, ‘Scholarly Information Architecture, 1989–2015,’ Data Science
Journal 3 (2004): 29–37.
25. Daria Sorokina, Johannes Gehrke, Simeon Warner, and Paul Ginsparg, ‘Plagiarism
Detection in arXiv,’ arXiv.org, February 1, 2007, https://arxiv.org/abs/cs/0702012;
Daniel T. Citron and Paul Ginsparg, ‘Patterns of Text Reuse in a Scientiﬁc
Corpus,’ Proceedings of the National Academy of Sciences of the United States ofAmerica 112 (2015): 25–35.
26. Stevan Harnad, ‘No-Fault Peer Review Charges: The Price of Selectivity Need
Not Be Access Denied or Delayed,’ D-Lib Magazine 16 (2010), doi:10.1045/
july2010-harnad.
27. V. Calcagno, E. Demoinet, K. Gollner, L. Guidi, D. Ruths, and C. de Mazancourt,‘Flows of Research Manuscripts among Scientiﬁc Journals Reveal Hidden Submission Patterns,’ Science 338 (2012): 1065–69.