How to referee a math paper
Eric Rowland ∗May 26, 2024
I have refereed over 90 papers and, after submitting my report for most of them, I felt I did a bad job.
After all, if I’m refereeing, the implication is that I’menforcing the rules. I’m supposed to certify a paper
as correct or find an error. But usually I can’t dothat. The authors worked collectively for months toprove their theorems, and I can only spare a few daysor, more realistically, hours. How can I understanda complex argument better than its own authors inthat time? Unless all the ideas in the paper are ideas
I’ve personally worked with already, being an effective enforcement officer is impossible.
So I come up with filler. I point out that the paper’s title is misleading. I say there should be more
examples accompanying definitions and theorems. I
identify notation that is confusing, overloaded, or unnecessary. I suggest that the authors give more intuition where needed. After making this list of superficial improvements, I’ve used up all the time I canafford to spend, and I send in a report. Did I fail tocomplete my certification duties? Yes.
It took me a while to realize that certification isn’t
what authors need though. They don’t need me tocarefully check their proofs; they were trained extensively for that. Reviewers of particle physics papers
don’t recreate the experiments! That would be ludicrous.
What authors need is help communicating. When
writing their paper, most of their focus is on correctness, so they understandably think of the paper as
consisting of theorem justifications. But this is misaligned with the needs of readers, who hardly evercare about the details of proofs. Readers don’t evencare about theorems necessarily. They want intuition. William Thurston recorded his realization ofthis [5, page 174]:
When I started working on foliations, I had
the conception that what people wanted
was to know the answers. I thought thatwhat they sought was a collection of powerful proven theorems that might be applied
to answer further mathematical questions.
But that’s only one part of the story. Morethan the knowledge, people want personalunderstanding.

There is a huge difference between making correct
arguments and effectively communicating ideas to areader. So the feedback I was giving about examples
and notation isn’t second-rate filler like I thought.
It’s extremely valuable, because that’s where the majority of communication happens.
The good news is that often it’s easy to help authors communicate, and you don’t need to know morethan they do. In fact, it’s better if you know less. You
have a perspective that the authors don’t, because
they never saw their paper for the first time. As anoutsider, you can tell whether the new ideas are explained well and whether enough background information is provided. You can definitely tell where theintuition is missing, and you can tell where the writing is not easy to follow. You can tell how successfully
a coherent picture emerges from all the pieces. Simply pay attention to your own level of understanding.
When something is unclear, don’t accept the blame;∗ Eric Rowland is an associate professor of mathematics at Hofstra University.
His email address is it’s not your lack of intelligence but rather an opporeric.rowland@hofstra.edu.
tunity for the authors to improve the exposition. If1

you’re insightful enough, you can eventually articulate what you wish the authors had told you to help
you see the path more clearly. Sometimes you willthen find that the authors did in fact say what you
needed to hear, but they didn’t say it with enoughclarity or emphasis or redundancy for it to sink in.
Authors have blind spots in all these areas. Unlessthey are exceptionally good writers, they will need
your help.
So is that how to referee a paper? Well, that’s how
to review a paper. We use the two terms interchangeably, but we shouldn’t, because they have completely
opposite connotations! Reviewing is more useful tothe authors and to future readers.

ble remembering all the notation, then so will every
other reader. It’s too easy for authors to introduce
notation, and it’s hard for a reader to keep track.
Authors need checks and balances here. Almost every paper I review has notation that should be eliminated and other notation that should be improved.
Ideally, notation should be self-documenting so it
doesn’t require the reader to remember anything. For
the base-b representation of n, the notation rb (n) is
more suggestive than (n)b , but repb (n) is even better
than rb (n). Mathematicians have a lot to learn fromprogrammers, who figured out a long time ago thatsingle-letter function names are a horrible idea.
If it becomes clear that the authors have not put
as much time into polishing the paper as they should
have, stop reading and kindly recommend rejection,
A short checklist
no matter how good the results are. Yes, you’re allowed to do this, even as a new researcher in yourIf you’re not the right person to review a paper, then field. Your time is valuable. You are providing highly
decline the request and suggest someone better if you skilled work that you are not getting paid for. If thecan. If you do accept, here are specific items to look authors are wasting your time, don’t let them wastefor.
even more of it.
Are the title and abstract accurate? These will receive more traffic than any other part of the paper,so they’ve got to be right! Authors often choose ti- Read freely
tles that are overly grandiose or too vague. Titles
like ‘Binomial coefficients modulo prime powers’ and Rather than reading a paper from start to finish,‘A note on β-expansions of real numbers’ can apply you should absolutely feel free to jump around, beequally well to hundreds of different papers, so they cause other readers will jump around too, in searcharen’t specific enough.
of what’s interesting to them. As you do this, you
Are the theorem statements correct? I have seen may find yourself frustratedly hunting for definitionsquite a number of false theorem statements in submit- or notation you “missed” by not reading linearly. Inted (and published) papers. Inevitably the problem that case, the definitions and notation should be easisn’t with the proof but rather that the theorem state- ier to find, either with more prominent placement orment doesn’t actually say what the authors think it with explicit references to them (or by making thesays. The fix is quite simple, but the authors didn’t notation self-documenting!).
notice there was a problem. Experimenting and comA paper should also be readable on multiple levels.
ing up with your own examples are better ways to find Initially, skip the proofs. The proof details make up
mistakes in a theorem than carefully reading a proof. the bottom level, and hardly anyone will read at thatOn that note: The three most important factors level. A convincing and coherent argument should bethat determine the readability of a paper are (to bor- clear at higher levels, and anyway you need to underrow a real estate adage) examples, examples, exam- stand the outline and major steps before you can unples. Are there enough examples? It’s much easier derstand why any details are relevant. After writing
to understand a theorem through an example than their proofs, authors sometimes forget to go throughthrough its statement.
the paper and provide all the connective tissue, inIs the notation optimal? If you’re having trou- which case you’ll find that information is too frag2

mented to tell how all the parts relate to the wholewithout first putting in a ton of work to reconstruct
the missing connections. It isn’t your job to do thiswork; request that the authors provide it.
Your report will typically begin with a few sentences summarizing the paper and explaining its contribution in the context of other results. If this isdifficult to write, it means the authors neglected toinclude this in their paper! It should also be easy towrite a few paragraphs summarizing the paper (forexample, if you are reviewing for Mathematical Reviews). On smaller scales, it should be easy to summarize each section and determine its role relative tothe others. If not, the authors forgot to summarize.

ers who do want to follow the proof must resort tochecking that one line follows from the previous, andthey will likely make the same subtle mistakes as theauthors.
Here is some advice to authors: Actually write
down your hard-earned intuition. You know which
theorems were difficult to prove and why; readers donot. Convey the significance of your results, and say
where the novel ideas are. Do a little marketing! Advocate for your theorems, so that when a journal’s
guidelines direct the reviewer to only recommend acceptance of important, not just correct, results, they
have some material to work with.

Benefits of reviewing

The ideas behind proofs

By reviewing rather than refereeing, you can approach a paper as an ally of the authors rather thanas a gatekeeper. This is more satisfying for you andmore healthy for the profession. Your job is to makethe paper more digestible for the people who will read
it after publication, whether it appears in the journal
where it’s currently submitted or elsewhere. If you
do that, you have made a positive contribution.
When you’re not holding yourself to the task ofcertifying correctness, there is no need to procrastinate for months as an avoidance coping strategy. I
don’t know whether this is the primary cause of theexcessive turnaround times in mathematical publishing, but it certainly doesn’t help.
Reviewing is still a job, and it still takes time, butmaybe it can become something you look forward to.
Personally I’m still working on that. But without
a doubt, improving the writing of others makes you
a better writer, so in addition to making one paper
better you’re also improving all the future papers you
have yet to write.

Given that you’re already reviewing a paper, why notalso check the details of the proofs? If you have thetime and inclination for this, by all means. But as
your career progresses and your responsibilities accumulate, you will have less time while simultaneously
finding yourself qualified to review more papers. At
some point, certifying correctness will become infeasible. Sure, authors will be grateful if you notice a
mistake in a proof, just like they will be grateful if
you point out a typo, but this isn’t your job.
Much more important than checking proof details
is checking that the ideas behind proofs are statedand stated clearly. A long proof stripped of its ideas
isn’t a vehicle for communication — it’s a magic trick.
But this is an easy mistake to make! Often the few
missing sentences are so fundamental to the proof
that the authors took them for granted by the timethey started writing. Their absence is conspicuous toa fresh pair of eyes.
Mistakes in a proof are much easier to detect whenauthors explain what they’re doing in addition to doing it. To quote Thurston again [5, page 169]:

Other hot takes

People are usually not very good in checking formal correctness of proofs, but they
are quite good at detecting potential weaknesses or flaws in proofs.

As a community, we should discuss refereeing/reviewing more than we currently do. However,additional thoughts can be found in several previous
pieces in the Notices [1, 2, 3, 4].

Without a high-level explanation of a proof, read3

References[1] Arend Bayer, Writing, and reading, referee reports, Noticesof the American Mathematical Society 66 (2019) 363–364.
https://dx.doi.org/10.1090/noti1817[2] Álvaro Lozano-Robledo, How to referee a (math) paper,
Notices of the American Mathematical Society 70 (2023)
71–74.
https://dx.doi.org/10.1090/noti2604[3] Mohammad Sal Moslehian, Attributes of an ideal referee,
Notices of the American Mathematical Society 57 (2010)
1245.
[4] Ken Ono and Robert Schneider, Journal refereeing: merge
with the collective mind, Notices of the American Mathematical Society 67 (2020) 188–189.
https://dx.doi.org/10.1090/noti2021[5] William Thurston, On proof and progress in mathematics,
Bulletin of the American Mathematical Society 30 (1994)
161–177.
https://www.ams.org/journals/bull/1994-30-02/
S0273-0979-1994-00502-6/S0273-0979-1994-00502-6.
pdf MR1249357

4