I think our intuitions about science differ somewhat due to different experiences.

I used to have near absolute faith in academic scientists, with much the same reasoning as you: the government pays them merely to do research and doesn't demand any specific outcomes, so they must be less biased than anyone working in industry where there is at least pressure to produce something pleasing for their employer.

Over the years I've read a lot of research papers. Mostly in computer science but sometimes others too. In the past few years I noticed that I tended to be more enthusiastic about reading papers that came out of corporate research labs than pure academic productions. On reflection about why I felt that way, I realised it's because when I thought about the research I'd found most exciting over the years it was usually the research that solved some major (in my eyes) problem in a practical and useful way, and that such papers typically had some corporate email addresses at the top, or were at least collaborations with universities. On the other hand papers that came out of only academia often felt rather useless, trivial or like they were going down blind alleys.

Worse, a very small number of university papers unfortunately were in my eyes outright misleading or appeared to be deliberately deceptive, e.g. making eye-catching claims in the abstract that were undermined or voided by statements right at the end of the paper, or which didn't mention critical details that broke their conclusions. I'm thinking of papers dealing with security technologies I happen to be specialised in. Despite what many would have me believe, my own experience has been that corporate funded research never tried to mislead me in this sort of way, but it's happened multiple times with government funded research.

During the time these feelings were developing the whole replication crisis started. At first it was just a few psychology papers that seemed in trouble. Amy Cuddy was outed as building a speaking career on top of her flawed psychological research. I noticed that despite one of her co-researchers admitting the paper was nonsense in writing and effectively retracting it, Cuddy continued to build up her career on the back of it, aided by journalists who wrote breathless articles about this "inspiring woman", apparently not bothering to spend two minutes Googling her research to check if it replicated. Then more and more papers started getting in trouble. Whole fields of research started to disappear. Nutrition turned out to be riven with fake findings. I learned that venture capitalists avoid investing in biotech firms because of an open secret: that around half of all published microbiology papers coming out of academia didn't replicate. Earlier this year I read https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-rev... which showed how psychiatric researchers investigated the gene 5-HTTLP for decades, published over 400 papers on their "discoveries", and then they all collapsed at once because a single paper showed that the gene couldn't possibly have the claimed effects. Literally that entire area of research was the result of misuse of statistics.

John P Ioannadis published a meta-study that appeared to show "why most published research findings are false":

https://journals.plos.org/plosmedicine/article?id=10.1371/jo...

This paper is not the result of some kind of weird cranko blogger. As far as I know, the paper had huge impact and kicked off the current replication movement in a few subfields (by no means all of them).

I did some of my own research into an economics paper that was being used to make suspicious claims on a topic I was an expert in. It had got picked up by major global newspapers, but on reading the paper I discovered grotesque amounts of scientific fraud. Then I discovered that the Chief Economist of the World Bank had been sounding the alarm about research malpractice and bogus papers in economics for years: his reward for blowing the whistle was to be fired.

Finally, there was this paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4816570/

It basically destroys every area of science based on doing small-scale experiments (where "small scales" means fewer than thousands of participants, i.e. nearly all of them). At a stroke that one paper wipes out huge areas of what we thought was "knowledge".

Now we see climatologists editing their data sets in such a way that the results always fit their theories. I was taught at school that adjusting theories to fit the data is science 101, that you handle uncertainty with error bars and statistics. But that entire principle has been thrown out with barely a sound. How did all this happen, and what else isn't obvious about how science happens in the real world?

The problem is that government funded academics are not free of incentives or bias, as I had once assumed. Yes, the government doesn't tell them what to find. But academics are just as strongly incentivised as any corporate-funded researcher, namely, they're incentivised to find things that are interesting. That is so vague it doesn't sound problematic, but it is. Researchers who fail to find things that are interesting don't get published and thus "perish", to use the well worn aphorism about academic success.

What defines interesting, exactly? Anything eyecatching or novel counts. Replications mostly don't. This is how psychology and biotech got into such a mess.

What defines eyecatching or novel? Anything your peers like. Who are your peers? More academics, with the same incentives. Please them and you'll be rewarded with grants, jobs and funding. Displease them and unless you win the argument so comprehensively there's no doubt possible at all, there will be no reward. You can't do what businessmen do and prove you're right via market success.

The least pleasing thing you can publish is evidence that nobody understands the field of study you're in. If the results of your research is "an economy is too complex to model at this time", or "the climate does not behave like our theories say but there are no other theories" then this is a dead end for an academic career.

The entire system is circular. I find corporate funded research into computer science more exciting than academic research because so much academic research has gone off into the weeds and never came back. Corporate researchers do, at some point, have to justify to their managers why their research has some sort of practical utility even if it might be quite long term. Their findings are integrated into products that just won't work if the research is bogus. Academics are never tested in this way. With no ground truth to anchor them it's easy for entire fields of science to spin off into a groupthink dominated la-la land, in which pleasing your colleagues preconceptions is much more important to your success than actually establishing the truth. Outsiders are the only people who can bring them back.

That's why I want to see more oil companies funding research. I want to see GM food companies funding research into GM foods. I don't believe anymore that they're more biased than academics, they just have different biases, but in the end they're still writing papers and making claims that anyone should be able to check. Academics should not be destroying raw data and forcing outsiders to use millions of FOIA requests in order to check their work. They should be welcoming such scrutiny from whoever gives it and especially from outsiders who have different things to lose - just as they would expect corporate funded science to be subject to their own scrutiny in turn.