That’s a very kind reaction, thanks for responding. Here are some notes that might help contextualize my reaction and feelings about the so called Dunning-Kruger Effect, and I hope helps on your exploration of whether you believe this paper’s claims.

I don’t think it’s related to what I was talking about because DK does not show people with less knowledge overestimating themselves, and it just doesn’t have anything to do with the ratio of how much you know vs how much you don’t know… it’s not hypothesizing at all about unknowns or about the boundary of knowledge, so the results can’t be seen as providing any evidence for that topic.

Subsequent papers to DK’s have tried to reproduce the effect, but using skills that require more expertise than what DK tested. Some of them found no such effect, and some found a complete reversal of the effect -- they found the experts underestimating themselves! (Impostor syndrome might be slightly closer to what I was talking about at the top, and impostor syndrome is opposite of DK). I believe the studies showing the opposite of DK are mentioned in the blog post I linked to, which I can’t recommend enough if you’re curious about DK.

The DK paper tested a total of 4 “skills”: the ability to get a joke (I’m not kidding), some basic grammar, some basic logical reasoning questions, and last but not least the ability to know how well you did on the previous questions relative to other people. The questions they actually asked aren’t in the paper, we don’t know what they were, but they did not test things that most people normally accrue expertise in like law, engineering, dance, architecture or medicine.

Only the 4th task - estimating others - is the one where they claim that people overestimate themselves. But remember these are undergrads who were asked to rank others, not to gauge their own ability (to, e.g., get a joke) in any absolute sense. Underestimating other people is not the same thing as overestimating yourself! And yet that’s exactly what they’re claiming.

The whole study had a little over 60 Cornell undergrads volunteering for extra credit in a psych class. The part where people ranked themselves only had half the participants, about 30, and it was done several weeks after the first three questions. Ignoring the potential problems with non-native English speakers, just think about how crazy it is to declare you’ve discovered a meta-cognitive bias for all of humanity after testing only 30 kids of all the same age and roughly the same background in a single Ivy League school in the US. Kids who go to Cornell are often rich, and often told they are smart for their whole lives, validated by getting accepted to an elite school. Why wouldn’t they overestimate themselves? Kids volunteering for extra credit doesn’t include the kids who were doing well enough that they decided not to volunteer, nor of the kids who were doing poor enough that trying to get extra credit was a waste of time. How do they know the kids volunteering for this study even took it seriously? They got the extra credit by showing up, not by being accurate. There are so many problems with the population sample, it seems nuts to draw any conclusions.

Here a link to the paper so you can read it and compare to Tal Yarkoni’s blog post. https://www.researchgate.net/publication/12688660_Unskilled_...

The main question to ask yourself as you read it is “is this the only possible explanation for the effect they’re discussing?” Take special note of how they prime the reader with suggested outcomes before they describe the methodology. They have a hypothesis from the beginning and appear to test the hypothesis, but did absolutely nothing to try to disprove it. I don’t think this paper would be publishable by today’s standards, and compared to hundreds of other papers I’ve read, this one stands out for how much it jumps to conclusions and waxes philosophical without enough evidence.