Bullshit filters are invaluable. The advice here is at best mixed, and several points highly flawed.

I see this as a reputation question, and one fundamentally of whether the source has any concern with being seen credible on the basis of their pronouncements.

By reputation, an assessment of typical reliability, which need not be accurate in all cases, but should be useful in most. There's also the principle that false, misleading, or distracting information is worse than no information, and should be rejected as rapidly and cheaply (with minimum effort, burden, or deliberation) as possible.

Given that prediction is hard, especially about the future, errors will occur. A critical question is how does the source address this?

In the case of one cited example, Paul Krugman, he frequently admits errors, even significant ones. I'd noted some years ago his mea culpa regarding his 1970s outlook on energy.[1] This is only one of numerous instances, some large, some small.[2] Krugman also offers his advice on how to regard errors in predictions:

My view, however, is that you don’t just want to look at whether people have been wrong; you want to ask how they respond when events don’t go the way they predicted.[3]

And he states his own principles on the matter along with numerous examples of his own errors, contrasting with others who are "completely unwilling to admit mistakes":

I try hard not to behave that way. If I make a mistake — like my extreme pessimism about the short-term survival of the euro, or my warnings back in 2003 about a US debt crisis — I do try to admit it, and figure out where I was wrong (I underestimated both Europe’s political cohesion and the extent to which ECB intervention could short-circuit the financial panic; back when, I made a false analogy with countries that borrow in someone else’s currency.) No doubt there have been times when I rewrote history to make myself look better, but I try to avoid that — it’s a major intellectual and moral sin.[4]

That's not to say Krugman practices this perfectly, and by his own admission he allows exceptions, though he seems to give the notion an earnest effort and be self-aware. This contrasts with numerous others, including several of recent conversations I've seen on HN in which refutation after refutation draws nary a note of acknowledgement. Makes for tedious discussion.

If prediction is a speculative venture,[5] then errors will be made. Failure to be wrong is failure to try: "predictions" are simply obvious statements, tautologies, unfalsifiable, or cold readings which will be interpreted as accurate regardless of future outcomes. Occasionally they're made by an entity which has the sole power to determine outcome, whether by controlling events (discretion of action) or judgement ("It's not who votes that counts, but who counts the votes," as Stalin is claimed to have said). See also rigged courts and selection committees.

Which affords another principle: be wary those who determine both decisions and the assessment of outcomes. A recent exchange on HN saw the claim that the US Federal Reserve had changed its definition of inflation. A response correctly noted that whilst the Fed's Dual Mandate is to manage both inflation and unemployment, both are measured by a different entity: the U.S. Department of Labour.

Rather than penalise error, we should instead look to its causes and responses. Is the source systematically biased? Was there poor data? Was analysis flawed? Did the source simply have a bad model or poor conceptual grasp, arguably the case for Krugman's Internet blunder. Or were they simply careless, and the prediction itself a trivial non-serious cherry-picked element of their work? That's Krugman's own admission along with, again, his admission of error:

I must have tossed it off quickly (at the time I was mainly focused on the Asian financial crisis!), then later conflated it in my memory with the NYT piece. Anyway, I was clearly trying to be provocative, and got it wrong, which happens to all of us sometimes.[6]

Worse than those who ignore or fail to correct errors are those who become defensive, engage in projection, invoke tu quoque and whatabboutism, who become hostile and abusive, who deny, and/or hide their past, often through secrecy classifications, assertions of privilage, NDAs, nondisparagement contracts, gag orders, outright threats, or unhealthful tea.

There's one worse level yet: Those who are openly and flagrantly indifferent to the truth.

Such actors are very nearly always dangerous of themselves, whether through presumed or actual immunity or impunity. The dynamic is well-described in Adam Curtis's documentary HyperNormalisation.[7]

Any circumstance in which factual statements are rewarded for anything other than truth valence is ultimately perversely selecting. This may be punishing bad news, favouring good, or rewarding (usually short-term) profit (or audience) over truth value. This last makes any advertising-supported medium inherently suspect.

There are four fallacies associated with smart people being stupid:

- Egocentrism: overly self-centered, stop caring about outcomes for others.

- Omniscience: surround themselves with sycophants.

- Omnipotence: believe they can do anything.

- Invulnerability: believe they can get away with anything.

There's a large literature on truth detection. Science and the scientific method (literally: the process of acquiring knowledge), and the philosophical field of epistemology. The distinction between dialectic and rhetoric, the philosophers and the sophists, the one seeking wisdom, the other promulgating dogma, predates Plato.

I've collected a few earlier references, aimed largely at specific claims or works, though somewhat applicable to sources as well, drawing on Reason Stick, Rory Coker, Derek Muller (Veritasium), Tim Minchin, Harry Frankfurt, Craig Ferguson, Carl Sagan, Alberto Brandolini, Doglas Adams, Nate Silver, Dunning-Kruger, and others:

https://old.reddit.com/r/dredmorbius/comments/28ge14/on_nons...

Dietrich Bonhoeffer would be a good addition:

https://religiousgrounds.wordpress.com/2016/05/11/bonhoeffer...

________________________________

Notes:

1. https://krugman.blogs.nytimes.com/2008/04/22/energy-futures-...

2. A DDG search returns numerous instances, and a few false hits. But the case seems made; Krugman will admit error. https://duckduckgo.com/?q=krugman+"i+was+wrong"+site%3Anytim...

3. https://krugman.blogs.nytimes.com/2013/11/17/what-to-do-when...

4. https://krugman.blogs.nytimes.com/2014/10/02/knaves-fools-an...

5. And for most of you it is. Fellow Timelords, you know who you are, and that the situation is even worse for us.

6. https://www.snopes.com/fact-check/paul-krugman-internets-eff...

7. Celine's Second Law is a partial expression of this. https://old.reddit.com/r/dredmorbius/comments/1wpaud/linus_t...

7. https://en.wikipedia.org/wiki/HyperNormalisation