I got halfway through, which I think gets you to the answer to the money question of what's special about four. Here's my attempt to summarize:

There are bunch of possible reasons conversation size might be limited or optimal at certain numbers. They choose to focus on mentalizing -- your ability to maintain a mental model of another person. (I don't know if they have a strong claim for why they chose this.) One interesting observation that HN will love is that mentalizing is recursive. When I have a mental model of your mind, that model includes why I think your mental model of me is, and so on. If I say something to Fred while George and Harry listen in, I can reason about what Fred will think of what I say, what George will think of what Fred will think of what I say, and what Harry will think of what George will think of what I will think of what Harry will think of... ad infinitum.

By focusing on mentalizing, what matters more is the pairs of people in a conversation more than the number of people. It's about the relation between one person and their reaction to another. Pairs grow faster than linear as the number of participants increases. There are "n(n-1)/2" pairs in a conversation with "n" people.

Then they make a distinction between "inclusive" and "exclusive" pairs. An inclusive pair is one that includes you. So in a three-person conversation, there are three pairs: you-A, you-B, A-B. So two of those pairs are inclusive.

The number of pairs increases quadratically. The number of inclusive pairs increases linearly. At larger group sizes, most pairs of people don't include you.

They claim four is the magic number because that's the largest conversation size before the exclusive pairs outnumber the inclusive ones. With four people, there are three inclusive and four exclusive pairs. With five people, there are four inclusive and six exclusive.

It's a neat observation, but they note themselves that they are basically doing a post-hoc analysis. They started with four and then tried to find some math that makes it special, and eventually found two lines that cross at that point.

I do think there might be something to it. If you assume that the most valuable interactions are ones that involve you (versus deriving value from seeing what two people say to each other), then it stands to reason that you want to avoid conversations where most utterances aren't from you or to you.

But that also presumes (1) people don't derive much value from watching others talk to each other and (2) all participants are communicating to each other equally. Neither of those is true in practice.

I think a smarter way to look at it is that people strive to maximize the total value they get from all pair-wise communications. One way to do that is in a small even-handed conversation. But you can also get that by:

1. Giving a speech where you get to do almost all of the utterances. So even though there are many many pairs, most of those channels are silent, and most of the communication does involve you.

2. Watching a debate where even though your aren't participating, you get a lot of value from what the other two are saying to each other.

3. Less formal approximations of the above. All of us have probably experienced a conversation that grew to larger than four people because a minority of them had more dominant personalities so you end up with a couple of "performers" and some "audience" though people occasionally change sides.

Anyway, fun paper.