I think the issue here is that many of these things are extremely complicated but _look_ simple. If you aren't in the weeds of that technology you're primed to attach yourself to the simple answer. This is often exacerbated as our technical vernacular overlaps with English, as well as other technical vernaculars. This can make someone believe that they have more understanding of a topic than they really have. This is very obvious in AI/ML research (hint, usually when people say "AI" they aren't a researcher) because there is a lot of hype around the research and products. But I have a great example of this miscommunication on HN from a comment I made yesterday[0]. I said

> [Confidence] indicate[s] how confident the model is of the result, not how likely the prediction is to be accurate.

The problem here is likely how "confidence" and "likelihood" are used. The words are overloaded. Maybe I should have said "not how probable the prediction is" but this could even be less clear. Most people think likelihood and probability are the same thing.

So there's a lot to why this is happening. Misreadings, ego, fooling ourselves, and more. I think there's only a few solutions though. First, we need to recognize that there's nothing wrong with being wrong. After all, we are all wrong. There is no absolute truth. Our perceptions are just a model of the world, not the world[1]. Second, we have to encourage a culture that encourages updating our opinions as we learn more. Third, maybe we don't need to comment on everything? We need to be careful because we might think we know more than we do, especially since we might know more than the average person and want to help this other person understand (but this doesn't mean we're an expert or even right!). Fourth, we need to recognize that language is actually really complicated and that miscommunication is quite frequent. The purpose of language is to communicate an idea from one brain and pass it to another brain. But this is done through lossy compression. Good faith speaking is doing our best to encode in a fashion that is most likely to be well interpreted by our listener's decoder ("speak to your audience" is hard on the internet. Large audiences have a large variance in priors!). Good faith listening is doing our best to make our decoder align with the _intent_ of the speaker's message. Good faith means we need to recognize the stochastic nature of language and that this is more difficult as the diversity of our audience increases (communication is easy with friends but harder with strangers).

I'm sure others have more points to make and I'd love to hear other possible solutions or disagreements with what I've claimed. Let's communicate to update all our priors.

(I know this was originally about ML, which I research, but I think the question was key on a broader concept. If we want to discuss stochastic parrots or other ML stuff we can definitely do so. Sorry if this was in fact a non sequitur)

Edit: I believe we're seeing this in real time in this thread[2]

[0] https://news.ycombinator.com/item?id=34608009

[1] https://hermiene.net/essays-trans/relativity_of_wrong.html

[2] https://news.ycombinator.com/item?id=34619277