Scientists need to stop writing papers like this one until they get their own house in order. From a quick read through, I can see quite a few aspects that come across as misleading to me:

1. It says:

"Although the knowledge gained and shared by the scientific community about [COVID] gradually increased, public health professionals prescribed traditional, time-tested, and general epidemiological measures to try to mitigate its spread ... consensus on how to mitigate viral contagion was well established even at the beginning of the pandemic."

But this is nonsense. Nothing about global lockdowns was traditional, time tested or general. Nor was there any consensus about doing these things, with the WHO having previously strongly stated in 2019 that during a pandemic border closures, contact tracing and quarantines were "not recommended in any circumstances".[1] The pandemic started with public health people saying that closing borders or avoiding people who had arrived from China would be racist. They also opened by stating that there was no evidence masks were useful (in fact, because that's what the available research said at the time), then flipped almost overnight to masks being so useful they must be mandated. No new research came out that triggered this. There were quite a few other inversions on apparently basic topics but there's no need to list them all.

Given that sequence of events it is quite crazy to describe what happened with COVID as traditional and time tested, or to claim that there was a well established consensus right from the start about what to do. No such consensus existed and never did: there were people both inside and outside epidemiology opposing pandemic measures the whole time.

2. They say:

"This is why self-reported understanding decreases after people try to generate mechanistic explanations, and why novices are poorer judges of their talents than experts (33, 34)."

This second claim is key to the whole paper's argument, but there are only two citations for it. Citation 34 is of Dunning-Kruger. The DK study has bizarre and severe flaws that make me wonder how it ever became as famous as it did, for example, one of the supposedly general tasks in which they pitched their tiny handful of psych undergrads against the "experts" was literally "joke expertise", a totally subjective task. I wrote up some of the logic and design issues with it three months ago in a HN thread [2].

3. Although it isn't communicated in their abstract, several topics and most notably climate change are exceptions to their claim:

"individuals most opposed were the least knowledgeable about science and genetics but rated their understanding of the technology the highest in the sample. A similar pattern emerged for gene therapy, although not for climate change denial ..."

Probably not a huge surprise here. The sort of people who write articles disagreeing with climatology are often in my experience scientists, former scientists, engineers, even meteorologists. The sort of people who get upset about GM foods aren't.

4. They rely on Mechanical Turk. The exponential growth of Mechanical Turk usage in the social sciences is troubling. MT isn't designed for doing research and it's easy for people to pretend to be in demographics they aren't, moreover, they are financially incentivized to do so. Academics like using MT because it's more convenient and cheaper than going out and doing large scale legwork, but the results have little validity. In particular attention check failure rates are horrendous on this platform but it doesn't seem to bother anyone. They use two platforms and the second explicitly advertises itself by trashing the reliability of MT based studies [3]. Amusingly, the second platform is literally called "Prolific Academic"!

[1] https://apps.who.int/iris/bitstream/handle/10665/329438/9789...

[2] https://news.ycombinator.com/item?id=31119836

[3] https://www.prolific.co/blog/bots-and-data-quality-on-crowds...