Let's take a look then. Firstly, is it replicable? Many academic papers aren't because their methodology either isn't specified, or the methodology isn't derived logically from the hypothesis.

Here they are relying on Wikipedia and "Media Bias/Fact Check" to decide if a source is factual, and then decide anything published by any such source is wrong, and then that anyone who refers to any such article is also wrong. This is replicable at some surface level - you could use the same sources and same tweets to compute the same numbers, but lacks any form of deeper replicability because the choice of these two meta-sources doesn't follow obviously from the hypothesis and they don't specify how they chose them (beyond citing other papers that made the same choice).

So an obvious criticism is how do you know these meta-sources aren't junk? They thought of this, and say:

> To ensure the robustness of our methods, the factuality measure was manually validated to ensure that the articles shared in fact contain misinformation (see Supplemental Information File). We selected a stratified random sample of 50 articles from each of the five predefined factuality levels, totaling 250 articles. These were manually analyzed ... The results are reported in Supplemental Tables S6 and S7,

But if we check the Supplementary Tables then we find no such validation. Ironically, this study itself contains misinformation. The articles they checked aren't listed, nor is how they decided what was or was not true. The tables are a masterclass in circular logic, comparing how accurate articles are against their "factuality group" and discovering that, shock, the academics who chose the sources agree that they are good sources. Nothing is presented that can actually be refined or refuted, because this entire "manual validation" boils down to circular reasoning of the form "the sources we chose are accurate because we checked and say they are accurate". It's just a restatement of the rest of the paper, dressed up with some tables to look sciencey.

In the penultimate paragraph they do inch towards an actual evaluation, but it just shows how pseudo-scientific this paper is. They admit that in many cases sources they put in a low factuality group do report accurate news, but justify the decision by saying (about RT):

> Although each individual article may be factually correct in its reporting on individual battles, the overall story is thus a false narrative of a highly successful ‘special military operation.’ This underscores the limitations of assessing factuality solely based on the accuracy of individual articles, offering further support for the validity of relying on media outlet-level analyses.

This is a Kafka trap. If you report accurate stories you are nonetheless inaccurate because you don't report the things we think you should, and therefore anyone who cites you for anything at all, ever, is "spreading misinformation" regardless of topic or correctness of the statement made. The only type of source anyone is allowed to use is one that promotes the academics preferred slants, regardless of the accuracy of the information published.

There are tens of thousands of studies like this coming out of academia every year that are little more than Guardian-bait. Zero of them are scientific, yet all are presented as if they are. When people call them out on this behavior, they're accused by the left of being "anti science". Where are the intellectual standards?