Oh boy. They do. They do it all the time. Too many of them see themselves as part of a "team", defending each other against outsiders. The Slack chats between virologists talking about the COVID lab leak hypothesis are a good example of this, they were very explicit about their aims there.

Huge funding for replication studies is commonly suggested, but it won't work. I'm basing that view on experience of having read and reported invalid papers across several different fields, spent years following this story and its various ins and outs, and have written extensively about the problems with pseudo-science coming out of official institutions.

Replication is the polite society way to talk about a whole range of problems. People say there's a replication crisis because it sounds a lot better than saying there's a large scale fraud and incompetence crisis. But it's not like there are papers out there that abstractly don't replicate, nobody could have known, and when someone tries and fails then there's suddenly a whole process that's followed to root cause why it failed and fix it. There isn't anything even close to that. What actually happens is that very obvious problems are ignored or kicked into the long grass as long as possible, often indefinitely, and sometimes scientists even argue they're being victimized by those reporting problems.

This article is a reasonable place to start, if you're new to the problem space:

https://fantasticanachronism.com/2020/09/11/whats-wrong-with...

It raises points that are often overlooked due to the "replication crisis" framing, points like:

• Many papers that can be replicated are only replicable because they're of little value (e.g. poor children have worse exam results). We don't want to be flooded with highly replicable science that tells us only what everyone already knew.

• Many papers are replicable but their claims are still invalid because their logic or methodology is wrong. This category covers significant amounts of COVID science, for example.

• It's often unclear exactly what the definition of replicable is.

But zooming out from replicability for a moment, I wrote an article with about some of the more egregious problems here:

https://blog.plan99.net/fake-science-part-i-7e9764571422

We're talking about a scientific culture that can't even stop obviously machine-generated text from being published on a massive scale. No other part of society has problems like this. Even talking about replication seems like a distraction whilst you have papers being published every day that contain obviously Photoshopped images, mathematically impossible numbers and AI generated text, yet nobody cares and getting even one case "resolved" (paper retracted) requires months or years of external pressure.

Think about how big the trust problems are with journalism. How to fix trust in journalism is talked about a lot on the conference circuit, but it's a hard problem. And yet newspapers don't publish garbled gibberish and faked images on a daily basis! That's the scale of the challenge faced with science reform.

And even if you somehow manage to drain that swamp, the first step before full replication is attempted is to get the original data, so the analysis steps can be replicated using the original numbers. Whole swathes of science fail here because academics refuse to reveal their data. This happens even when they've been made to sign a statement agreeing that they'll provide it on request. Again, their employers do nothing, so where's the pressure point? Only massive fiscal punishment could cause culture change here but the NSF has a single institutional goal of dishing out as much money as possible. To quote Alvaro de Menard,

Why is the Replication Markets project funded by the Department of Defense? If you look at the NSF's 2019 Performance Highlights, you'll find items such as "Foster a culture of inclusion through change management efforts" (Status: "Achieved") and "Inform applicants whether their proposals have been declined or recommended for funding in a timely manner" (Status: "Not Achieved"). Pusillanimous reports repeat tired clichés about "training", "transparency", and a "culture of openness" while downplaying the scale of the problem and ignoring the incentives. No serious actions have followed from their recommendations.

It's not that they're trying and failing—they appear to be completely oblivious. We're talking about an organization with an 8 billion dollar budget that is responsible for a huge part of social science funding, and they can't manage to inform people that their grant was declined! These are the people we must depend on to fix everything.