Why Referees and Weather Forecasters Are Much Better than You Think
By: Brett Pelham
Tags: Judgement, Data Science, Sports, Decision Making, Personal Growth

Why Referees and Weather Forecasters Are Much Better than You Think

Image by Markus Spiske, courtesy of Unsplash

Have you ever seen a referee blow a call that cost your favorite sports team a big win? I certainly have. So, you and I are probably in the same boat. But that doesn’t mean we’re on the same page. If you’re like most other people on planet earth, you think most referees suck at their jobs. But they don’t. They’re actually very good at them. Allow me to explain. But be patient. I need to explain two separate facts about the human mind — to clarify why referees are much better at their jobs than most people think.

Before I argue that you’re wrong about referees, allow me to compliment you on your knowledge of trivia. I hope you’ll agree that: (a) many people feel nauseous when they spin around too much, (b) Abe Lincoln was the 15th U.S. President, and (c) pheromones are a type of hormone. Speaking of honest Abe, please be honest. How many of these three things did you know already? If you think you already knew at least one of these three things, I have bad news. You didn’t know any of them. I know that because none of them are true. People can feel nauseated, but it is only things (like vomit or poison gas) that are nauseous. If you are nauseous, then you make others want to puke. Further, Abe Lincoln was the 16th President, and pheromones are not hormones. Yes, I’m being nitpicky. But that’s the point. I’m being nitpicky with you in much the same way you’re often nitpicky with referees.

Research on the hindsight bias shows that the minute you tell people something, they often feel like they knew it all along. One way to show the hindsight bias is to show a group of people a list of facts. If you ask people how many of these facts they already knew, most people overreport their knowledge — sort of the way you just did. Having just been shown or told something makes us feel like we already knew it. It is easy to show this by splitting a large group of people into two identical groups. If you make one group answer the questions without first telling them the correct answers, you’ll see that these folks do a lot worse than people say they would have done after having been shown the answers.

Notice that this is very much like what we do when we watch the slow-motion instant replay of a questionable call in a professional sporting event. We see slow-motion video images from multiple cameras stationed at strategic locations on the field or court. And with the benefit of technology-assisted hindsight, we see that what a referee called a foul was actually a charge. Likewise, we see that the ball did break the plane of the end zone, when a referee said it did not. And when our preferred team is on the wrong end of this bad call, we just can’t forget about it. We become convinced that referees suck.

The hindsight bias sets the stage for being tough on referees. But this is only part of the story. An even bigger reason why we fail to appreciate how good referees are at their jobs is that we don’t understand the human visual system. I could merely tell you this, but it’d be better to show you. The guy you see depicted in the image that follows is doing what I’d like you to do. He has closed his left eye tightly and put his left hand right in front of his face. He has held his left thumb up as if hitchhiking. He is staring intently at his left thumbnail with his right eye. That’s right. He’s looking directly at his thumbnail and nothing else. That means he’s not moving his right eye, at all. He has also put a coin in his right hand between his thumb and his pointer finger. And he has answered a question — which I’d also like you to answer. As you copy this model and move your right hand slowly toward your left thumb, how close (in inches) will the coin have to be to your left thumbnail before you can say for sure which side of the coin (heads or tails) is facing you?

Assuming the coin begins about 45 inches from your left thumbnail, this means your answer cannot be larger than 45 inches. Before you make this careful observation, commit to a guess. I’ll tell you that when I first tried a demonstration like this one myself (at the written behest of neuroscientist Antonio Demasio), my guess was 24 inches (2 feet). But I knew I had very good peripheral vision. What’s your guess?

Don’t do the actual activity until you’ve committed to a guess.

OK, now do it.

Unless you are a visual freak of nature, you just found out that the coin in your right hand had to be very, very close to the point where you were staring (your left thumbnail) before you could see anything more than a vague blur. Most people can only see the coin clearly when it is an inch or less away from the point at which they are staring.

This happens, by the way, because there is one and only one spot in the retina (the inside back of your eyeball) that has crystal clear, high-resolution vision. That tiny, pinhead-sized spot is called the fovea, by the way, but I’m sure you already knew that. The idea that we see everything around us all the time — with great clarity — is a powerful illusion. When you think you are seeing all the details of a complex moving scene clearly (say you see someone fouling someone else as she drives to the hoop), what is actually happening is that your brain is remembering and patching together what you’ve seen clearly before, whether you saw it yesterday or just a split second ago.

Human beings have wimpy visual systems. We’re very good at getting the visual lay of the land, but we can only see things in crisp detail when we are directly staring at them. And what we are staring at varies from microsecond to microsecond. In fact, part of a referee’s careful training involves knowing where to look and when. But sports are complicated events, and many things are competing for a referee’s visual attention at any given moment of a game. The result? Mistakes are made. But they are made much less often by trained referees than they would be made by non-referees like you and me.

People think referees suck because they forget that referees are human beings who are located in one and only one physical spot during a game — and who cannot possibly see every fine detail of a play that we’d ideally like them to see. That’s why instant replay is becoming an increasingly important part of many professional sports. But don’t blame the referees for not always seeing in real-time what you saw after-the-fact in slow motion, high resolution, from the angle of the one of twelve cameras that provided the best possible vantage point on the event in question. That’s the hindsight bias. If you’re starting to think you knew that all along, I’m not sure if I should be happy or distressed.

The hindsight bias doesn’t just apply to sports, of course. The main reason most people think weather forecasters suck at their jobs is also the hindsight bias. For decades, studies have shown that weather forecasters are incredibly good at their jobs. When a weather forecaster says that there is a 10% chance of rain, guess how often it rains? On average, the answer is 10%. But when you get caught in a downpour that 10% of the time, it’s natural — because of the hindsight bias — to feel like the weather forecasters should have known that.

To repeat, it is a well-documented fact that weather forecasters predict the immediate future weather very accurately on average. To appreciate this, ask yourself whether you’ve ever tried predicting tomorrow’s weather yourself, by relying on your hunches, or knowing that it is mid-June. Try predicting tomorrow’s high and low temperatures — or tomorrow’s likelihood of rain or snow — using anything other than the advanced statistical models that meteorologists routinely use. I promise you, you’ll be left high and dry, or maybe you’ll drown in your own poor predictions.

Meteorologists predicted this. You and I never could have. Photo by John Middelkoop, courtesy of Unsplash.

The hindsight bias is so powerful that we may even criticize those we love or respect for not knowing the future. If you have ever blamed a friend for doing something that, in retrospect, turned out very badly, the hindsight bias was probably involved. Likewise, if you’ve ever kicked yourself for a decision that seemed wise when you made it but proved to be a big mistake, you may be falling prey to the hindsight bias.

So, the next time one of your reasonable decisions proves to be a mistake, cut yourself a little slack while also learning from the specific mistake. Was this a one-off, or is this same situation likely to come up again? And try to apply the same level of forgiveness that you apply to yourself to referees and meteorologists. In fact, if you ever meet someone who is brave enough to have chosen either of these beleaguered professions, consider letting them know that you appreciate how hard their job is. And thank them for their service.