Not remotely what is happening, is it?

Actually, I disagree. There's a lot of that happening. You just aren't exposed to it.

You say you'd be genuinely fascinated by a debate about how scientists could be mistaken. OK, let's find out if you mean it. The last debate we had here on this got derailed into a discussion of Conservative party policy. Let's keep it on science this time.

To start, let me say that I'm not a climate skeptic ... yet. I'm not actually sure what I am at the moment. Let's say I'm actually a science skeptic. I think groupthink, bias, corruption and outright incompetence in science is far more widespread than people intuit, and it can distort entire subfields of science. People are starting to realise this in fields like psychology due to the replication crisis but there are problems in many other fields. Now, I wasn't surprised to discover in this comments section that the article was fake propaganda (i.e. "all spending on conservative think tanks for any reason" is "funding denial"). In the past year or so I've dug into climate change skepticism to figure out what they think and why.

Here's what modern climate change skepticism looks like: lots and lots of blogs, with very long entries, consisting of large quantities of data analysis and graphs. If it's professionally funded it sure doesn't look like that. It looks like smart people who know maths and science writing about a field they understand but are outsiders in. They reveal a lot of things from the raw data that are deeply troubling and which I've never read about in the media. I feel strongly I should have read about them there because the concerns are reasonable. I've been able to check some of these claims for myself and found them to be true.

Most troubling to me has been the massive extent to which the global temperature record is now synthetic. Timeseries that once showed the world getting cooler have been fed into ever-more complex algorithms that radically change the entire datasets and any conclusions drawn from them. This article on a skeptic website is one that I double checked for myself:

https://realclimatescience.com/61-fake-data/

It makes a checkable claim - that one of the very few global temperature datasets, the one published by NOAA, comes in "raw" and "adjusted" forms, and that in the adjusted form nearly 60% of all claimed measurements from weather stations are in fact not real measurements at all but the output of a computer model. Additionally the quantity of simulated measurements has shot up massively over time: older temperature data is much less synthetic than the rest of algorithms.

I think here on HN we all know how slippery software can be, so learning this was quite a shock. I couldn't quite believe it in fact but the datasets are public files anyone can download, so I downloaded them and wrote some quick Python scripts to check. The scientists aren't hiding anything: the measurements that are "estimated" using their algorithms are marked with an E in the dataset, just as the skeptic blog claimed, and sure enough, calculating the ratios showed that 1970 for instance only 9% of values were the result of a computer model, but in 2018 it was over 40% and this year is even higher.

Look at this graph to get a sense of the growth of the issue:

https://realclimatescience.com/wp-content/uploads/2019/02/Pe...

And worse, if you take away the output of the models, global warming disappears entirely.

https://realclimatescience.com/wp-content/uploads/2019/02/US...

Now, NOAA have an explanation for this sort of thing: this sort of heavy adjustment is required because weather stations move around, come and go, readings are taken at different times of day, etc. So the data needs processing and cleaning before it's used. That's fine. I can buy that, at least to some extent. But this did make me curious about the code that's doing this adjustment, not least because I kept encountering climate blogs written by apparently rigorous authors who said certain artifacts made them suspect a software bug was introduced at some point.

So I downloaded NOAA's code to assess the quality of the software - software that appears to be required for the world to show any warming trend at all. I'm a software developer and always have been so I'm confident I can take part in this area of climatology, at least in a trivial way. I'd assumed that given the huge sums spent on research and incredible costs being thrown around for tackling climate change, the software in question would be under heavy development and written to the highest possible standards.

Unfortunately it's not. Global temperature data is the output of programs that look like every programmers worst nightmare: large piles of FORTRAN that trace directly back to 1985, that's been patched every few years in unauditable ways. All variables are global. As you would expect, they all have cryptic names. Functions start with explanations of which global variables are used and which aren't. Version control doesn't exist. There are occasional comments at the top of a function summarising that a change was made, so that's at least something, but are they comprehensive? There are no unit tests. None of it was written by professional software engineers. The algorithms and code are both extremely complicated. Much of it is undocumented. Not surprisingly, there is evidence in the comments of bugs being fixed only after going undetected for decades.

The program can be found here if you're curious:

ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v3/software/52i/

This is NOAAs code. I mentioned there are only a few global datasets. One of the others comes from the notorious CRU at the University of East Anglia. When they got hacked their code leaked, and it's even worse. The CRU datasets are totally unreproducible: they appear to have asked a scientist to spend a few months on it at some point, and he wrote extensive notes about how frustrated he got:

http://di2.nu/foia/HARRY_READ_ME-20.html

Their code was much the same as above but apparently, much worse documented, and it seems to have suffered from undetected integer overflow bugs.

That's about where I've got up to. So I'd say the skeptics have got me listening in at least one basic way: all global warming theories are rooted in data that comes from a tiny number of people, processed using code that can't possibly have been reviewed by professionals, and those scientists have adjusted the raw data so much that temperature datasets that used to show average cooling (in the USA) now show rapid warming. That needs addressing just on the grounds of basic scientific integrity.