Peer review without editors
By: Matthew Mckeever
Tags: Cryptography, Peer Review, Academia, Decentralization

Many disciplines in academia rely on mutually anonymous peer review to verify the quality of work and thereby to accredit the academics who produce that work. This is most notably evident in the journal system, which works as so. Authors write papers about their research which they send to journals, who consider it for publication. Journals are run by editors who have at least two main functions: to perform initial quality control, rejecting immediately clearly unsuitable papers, and to find qualified reviewers to assess the papers they don’t immediately reject.

There’s a third role that to some extent overlaps or enables the latter role: editors are a sort of middle person. By that I mean that they stand in between the author and the reviewer, enabling mutual anonymity and, hopefully, using their knowledge to match papers and reviewers in the best way possible.

They ensure mutual anonymity by sending to reviewers and potential reviewers manuscripts without author-identifying information, and by not disclosing the identity of reviewers to authors. Anonymity, arguably, is desirable, because it minimises reviewers making decisions on the basis of irrelevant information (status, friendship, enemyship, etc.), and prevents authors using their knowledge of the author’s identity to grant favour or disfavour.

As is familiar, though, the journal system of peer review is creaking. The basic problem is that the number of submissions has greatly increased but the number of journals, and editors employed at those journals, hasn’t increased that much. Moreover, even were this not the case, it’s arguable that the system is misdesigned. If an editor’s function is primarily matching papers and reviewers, it’s arguable they aren’t best placed to perform that function, simply because they know too little. They don’t know, in particular, the particularities of the reviewers they invite — their schedule, busyness, whether they’ve reviewed a given paper before, and so on (including most notably their very existence (any editor will almost never know all the potential reviewers of a paper, just because often there are many, and an editor’s perspective on the world is limited)) — and this leads to mistakes: to allocating papers to less than ideal reviewers, to sending invitations to people too busy to respond (perhaps not even in the profession any more).

Realising this, we might ask for a better way. And a first question to ask is — assuming editors lack knowledge as to who might aptly review a paper — who does have that knowledge?

And the answer is arguably the potential reviewers themselves. They can tell, given a paper, whether they are apt to review it, given their skills, current busyness, and so on. Is there a way to harness the reviewers’ particular knowledge, to cut out the editorial middle man and match authors and reviewers while retaining anonymity?

I will argue that ideas in cryptography suggest an answer in the affirmative. In order to get all the moving pieces on the table takes a bit of work, so let me just lay out the nature of the system I have in mind.

We should imagine a communications channel on which people can post messages. In particular, authors can post messages containing or linking to papers they have written (call them paper messages), and reviewers can post messages with their judgements of those papers (judgement messages). The communications channel has the following properties: anyone can verify that any paper message has been submitted by someone qualified and worthy of attention, but no one can determine which person submitted it. And anyone can verify that a judgement message was written by someone qualified to judge it without anyone being able to determine who wrote the review. Mutual anonymity is thus preserved.

We can imagine, moreover, that reviewers can just browse the site, when they have availability, looking for papers they could publish. This system would give us a means, then, of letting reviewers harness their superior knowledge of their own circumstances to provide truly peer-to-peer, disintermediated, peer review.

We can do this thanks to the cryptographic concept of ring signatures. The basic idea is that ring signatures let us send messages with the following two properties:

(i) Anyone can verify that a message was produced by someone from a given set of people

(ii) No one can determine which person from the set sent the message.

Using these properties, an author submits a paper along with a list of possible authors, including themselves. Imagine I am an expert in the semantics of names, as are Zora, Xavier, and Vera, and my academic community knows this. If I want to submit a paper for review, I send a message like this:

M1. Here is the link to my paper. Signed: Zora, Xaver, Vera, or Matt.

Anyone can verify that the message came from one of us, and so any potential reviewer who knows that we are experts in the field (easily, publicly available information) will know it’s worth their time to review the paper. Ring signatures on the author’s side, then, to some extent lets us do without the triage that editors perform and which in the current system ensures reviewers that their time is not being wasted.

But the real benefit comes with reviewing. Using exactly the same idea, a qualified reviewer can send a message like this:

M2. I read the paper linked in m1. I recommend acceptance. Signed: David, Elaine, Federio, or Georgi

Assuming that David, Elaine, Federico and Georgi are also experts in the field — again, publicly available information — both author and any onlooker can see that the paper has been positively assessed as worthy of the scholarly community’s intention, without compromising the identity of the reviewer. Moreover, because ring signatures enable a signer to subsequently reveal that they signed a message, the author can subsequently take credit for their paper (and, in ways not fully worked out here, in a development of this idea we could ensure that reviewers also get credit for reviewing, without revealing their identity).

Before jumping into the somewhat technical details — which anyway I’m only going to sketch in a very preliminary way, and indeed using a less than state of the art framework — I want to point out the very large benefits such a system could have. In one way of understanding it, it allows accreditation without the journal system, thus without relying on a bottle-neck-producing, ill-equiped editor standing between author and reviewer. It is much more transparent — the participants of the reviewing process are there for anyone to see, even if their exact identity always remains a secret. If a paper is recommended, we can verify that someone qualified made the determination. In the current system, the (outside, in my opinion) possibility of editorial malfeasance prevents such a guarantee (an editor could just accept a paper their friend submitted without sending it out for review). It prevents gatekeeping: if you work on a topic that the trendy journals aren’t interested in, that doesn’t prevent your work being accredited by others in your area. Moreover, it gives us a fine-grained assessement of the credibility of a recommendation. If each of David, Elaine, Federio, and Georgi are known to be world-leaders, then we can, arguably, place a lot of confidence in their judgement. If they are less renowned, we can place correspondingly less trust in the paper. A journal is only as good as its referees, a fact completely obfuscated by the current system that will be somewhat evident in my proposed replacement. Finally, if we allow — again in refinements going beyond the sketch presented here — a way to search messages, we can present a way for reviewers to directly get in contact — through a veil of anonymity — with authors to give verdicts, without relying on the central planning of the editor. This system thus promises to work at scale by relying on genuinely peer-to-peer accreditation in a way that the current system straightforwardly fails to do. All of this means that I think this is an idea that should be taken very seriously as we think about how academia should look in the 21st century.

In a sense, you can stop reading now, if you want. The following is my attempt to explain the cryptography behind ring signatures. There are several reasons not to read this: I am not much of a mathematician, and so my exposition might be poor or simply wrong, and most likely any final implementation of these ideas would look very different, using different systems and a different design. The above informal description, and the below slightly more formal exposition, are meant to be proofs of concept, and not final words.

Added 23 June 2020: From summer 2020 I’m going to move my occasional writing from medium to tinyletter. If you want to read more from me in your inbox, please consider signing up: https://tinyletter.com/mittmattmutt. I’ll post relatively infrequently, and hopefully interestingly, on the same sort of themes as the blog, so: popular philosophy/explainers, culture, literature, politics/economics, etc. I might also do things like brief reviews of books I read and so on.

RSA Digital Signatures

I recommend checking out the wikipedia, from which I’ve taken some of the below and which will also tell you background details about the history and discoverers that I don’t feel like repeating.

We’re going to explain the notion of a digital signature, a way of harnessing some number theory to provide very solid evidence that a given message was sent by the person who claims to have sent it.

The basic idea behind a digital signature is as follows. Participants in a given system produce public keys — functions, roughly, from integers to integers — in accordance with a given algorithm a. In carrying out a, one also thereby creates a private key (also a function) known only to you, with the following property: the private key is the functional inverse of the public identity.

Using private_Bernie to refer to the private key produced by Bernie and public_Bernie to refer to his public key, this means that for all x, public_Bernie(private_Bernie(x))=x. And of course Bernie is arbitrary here: this holds for any private/public key pair.

That’s the first important thing. The second is that the private key is easily knowable on the basis of the public key if you’ve carried out the algorithm a but close to impossible to determine on the basis of the private identity if you haven’t.

To reiterate, the public and private keys are notable for having the following three properties:

(*) They are or determine functions that are inverses to each other, i.e. for all inputs x, public(private(x))=x.

(*) It is very difficult to determine the private identity on the basis of the public identity

(*) It is easy to determine the private identity if you have created the public identity.

Functions satisfying these constraints let one do some very interesting things. Imagine Emmy wants to send a message. She creates a message, say ‘hello world!’; call its representation in some number format m. Assume the existence of a publicly known function (let’s call it h, for reasons I won’t bother to reveal but will help if you look at other literature) that maps a message m one-one to some number. Assume there’s some communications channel on which one can send messages, on which Emmy sends: ‘m, signed: Emmy, private(h(m))’. If someone wants to discern that indeed Emmy sent the message (which will of course look to them something like ‘m, signed: Emmy, 12121211212’, they can do this. Find out Emmy’s publicly available public key (imagine it’s listed on her website or social media). Compute public_Emmy(12121211212), yielding a number. Apply the publicly known function h to the result. If you get back m again, then you can very confident that Emmy sent the message, and so private_Emmy(x) can be said to function like a digital signature of the message, in the same way that an actual signature can be used to guarantee a check or letter came from its signer.

Why is this? Well, you know that the message was sent by someone with the inverse of Emmy’s public key, i.e. her private key. You know that it’s very hard to work out the private key unless you created the public key, and so you know that it’s likely that whoever created the private key also created the public key. If there is a reliable way for people to indicate which public keys are theirs (such as by posting them on their website), then you can use that reliable source to link a person with their public key, can use the inversion to assume the sender possesses the public key’s associated private key, and use the infeasibility of determining the private key other than by creating it to exclude everyone but the creator of the public key, namely the person mentioned in the reliable source, namely Emmy.

Part of the magic here is that private_Emmy and public_Emmy are inverses, and yet knowing public_Emmy doesn’t let one know private_Emmy (often, the inverse of a function is a piece of basic mathematical knowledge — if you know a given function multiplies by 5, you know an inverse of it is division by 5). If it did, then since public keys are public, the whole thing would be useless. It’s here where the mathematical trickery comes in, and although I won’t do a full dress proof of how it works, I want to at least sketch the algorithm and some of the underlying maths that makes things work.

The algorithm a

Here’s the algorithm, taken straight from wikipedia, and assuming you know what modular arithmetic is. I recommend going through the worked example on the wiki to get the beginnings of a sense of what’s happening.

(1) Pick two large prime numbers, p and q

(2) Compute n=p x q

(3) Work out Euler’s totient function ϕ(n), which is the number of numbers <n that are coprime with it, i.e. the cardinality of {x|gcd(n,x)=1}, where gcd stands for greatest common divisor.

(4) Pick a number coprime with, and less than, ϕ(n), and set it as e. The pair (e, n) is your public key.

(5) Solve ed ≡1 mod (ϕ(n)), where ≡ denotes the relation of modular congruence. The private key is (d,n).

We can now say in a bit more detail how signing works. Emmy takes h(m) and applies the function associated with the her private key to it, which is to say we raise h(m) to d mod n, and so we send the message ‘m, signed: Emmy, h(m)ᵈ’. Then recipient then uses the public key to invert the signature, computing (h(m)ᵈ)ᵉ mod n, resulting in h(m), and thence to m, thereby enabling the recipient to be pretty confident that Emmy signed the message.

The key identity that’s doing the work here is (h(m)ᵈ)ᵉ ≡h(m) mod n. And just skimming the above, you might be puzzled as to how this could be. The equation that involves both e and d, in (4), however, is not mod n, but is instead mod ϕ(n). One might wonder how we can manipulate an identity mod ϕ(n) into an identity mod n.

Properly to explain this would be to do something badly that others have done well (see here, which I’m copying from, and especially the linked pdf which explains everything from the ground up and is the best explanation I found after days of searching), so let me just glance at a couple of the relevant properties, and then prove a special case of the identity. So, note (where ϕ is phi because the typographical limitations of medium):

(*) Euler’s totient function theorem says that for all x coprime with n, aᵖʰᶦ⁽ⁿ⁾ ≡1 mod n.

(*) It follows from what modular arithmetic is that if ed ≡1 mod (phi(n)) then ed=1+k phi(n) for some k.

From here, we can do some manipulation, to prove the special case where the message m and the modulus n are relatively prime (the above link proves the more general case). We can rearrange (h(m)ᵈ)ᵉ in our sought identity as:

h(m)ᵈᵉ ≡h(m) mod n

Thence:

h(m)¹⁺ᵏᵖʰᶦ⁽ⁿ⁾≡h(m) mod n for some k, from the second fact above

Rearranging:

h(m).(h(m)ᵖʰᶦ⁽ⁿ⁾)ᵏ≡h(m) mod n for some k

Then, from the Euler theorem:

h(m).1ᵏ≡h(m) mod n

And from a fact about modular arithmetic I don’t get we can get that:

h(m)≡h(m) mod n

Which is to say we’ve shown (h(m)ᵈ)ᵉ gets us to h(m), which is what we were after. The basic idea is that we use the modular arithmetic fact to rephrase ed in terms of phi(n) mod n and then use Euler’s identity to get rid of the phi(n) term altogether.

While that might (might!) explain the inverse fact, it doesn’t explain the other crucial fact I adverted to, namely that without having created it, it’s very hard to determine the private key from the public key. And the basic reason for that is that the best thing one can do to that end is try to determine the numbers p,q such that pq=n, and that problem — determining the factorization of an integer — is thought to be extremely difficult as the number n gets bigger (and n, in RSA implementations, will typically be very long indeed).

That is the first thing we wanted to do done — now we know how to make digital signatures. In the next section, we’ll see how to expand this idea to create a scheme that allows one person from a group to sign a message without anyone being able to know who.

Ring Signatures

The basic idea is easy: a ring signature is a signature that one can attach to a message that reveals that someone signed the message without revealing who signed it. I suggested above we could use it to preserve the anonymity of both author and reviewer, by having some authors group sign messages with a list including themselves as well as other plausible options for people who wrote the paper, and by having reviewers group sign messages with a list including themselves as well as other plausible options for people who could have reviewed it.

In more detail: if you want to sign group-sign a message, take all the people you’re including in the ring and get their public keys public₁,public₂,…,publicₙ₋₁. Then straight up make up random values for each of these people, x₁,x₂,…,xₙ₋₁, and calculate public₁(x₁), public₁(x₁),publicₙ₋₁(xₙ₋₁). Next, made up a random initiating value v.

Create a function f(v, y₁,y₁,…,yₙ)=v such that given any n-1 values, the final yₙ is uniquely determined (I will be derelict in my duties and not tell you what the function is, at least in this draft; see the original paper ‘How To Leak A Secret’ by Ron Rivest, Adi Shamir, and Yael Tauman). Then plug in public₁(x₁), public₁(x₁),publicₙ₋₁(xₙ₋₁) as y₁,y₂,…,yₙ₋₁, thus uniquely determining yₙ. Next append all the publicₙs, including your own, and all the made up x’s to begin to make the ring signature:

S(f, v, public₁(x₁), public₁(x₁),publicₙ₋₁, publicₘₑ, x₁, x₂,…,xₙ₋₁,?)

We need to fill in ? such that publicₘₑ(?)=yₙ solves the f equation, given all the other values. But we can solve that by applying the inverse privateₘₑ to yₙ, which will yield the x such that publicₘₑ(x)=yₙ. We then add that final, computed and not made up, x value to our signature:

S(f, v, public₁(x₁), public₁(x₁),publicₙ₋₁, publicₘₑ, x₁,x₂,…,xₙ₋₁,x)

Any user can verify that a signature by working applying the public keys to the x’s, and making sure the resulting y’s satisfy the f equation, and by the design any user knows that a private key associated with one of the listed public keys must have been used to generate one of the x’s, but there is no way to know which. I haven’t shown that here — I can only, at this stage, ask the reader to take it on trust and direct them to the original paper, reiterating that this is merely meant to be a proof of concept to be elaborated on and improved.

A ring signature from the original paper. In this sketch I have omitted many important details.

Some Comments

Before finishing, I want to take a step back and consider some possible objections and refinements. How, you might think, should an author go about populating the ring they sign a paper they submit with? That shouldn’t be difficult: simply get the public keys of other scholars. There would need to be a convention whereby authors listed in rings who aren’t the actual authors don’t reveal that they aren’t the authors; but if this idea could be valuable, it’s hard to see reasons for people not acceding to that convention. The same thing applies for referees — they shouldn’t reveal, of a review in whose signature they appear, that they didn’t sign it. There is perhaps a risk, in both cases, that scholars will be fearful of the presumably rather mild suspicion that others will level of them, either for submitting a paper judged poor, or a review judged poor or harsh or lax.

At the very beginning, I talked a bit about quality control — about ensuring only qualified authors receive referee attention, and ensuring only qualified reviewers assess them. You might worry about this from several directions — you might think quality is a subjective or exclusionary notion, or — and more importantly — you might point out that a crucial merit of the existing system is that it allows people without qualifications to gain them. Any system that didn’t permit new members, because it required of them qualifications they could only get by being members of the system would, obviously, be hopeless. But this isn’t fatal: we just add some system that allows as yet unqualified people (say, graduate students) to submit without qualifications. There are probably many other things to say, and comments are very welcome.