> The web was a paradigm shift for, say, newspaper publishers. It totally upturned their world. But from a technology perspective, it was pretty straightforward. [...] The day before, we were writing daemons to output text over a network socket; we did the same thing the day after.

Again, I think we have a case of "no true paradigm shift" on our hands. My inner you says the same thing about the newspaper business actually: "the day before we were writing news stories and selling ads; we did the same thing the day after." Of course, in the technology case the kinds of software we were writing drastically shifted, which algorithms were important shifted, etc .. but that's also analogously true for news .. the beats and topics changed in value.

I'll go so far as to call this the central problem of sociology; the terms are vague enough that theories built on them can be bent to explain anything. Medicine has this problem too, in that tons of diseases are essentially catch-alls for unexplained pain, discomfort, or inflammation. Chemistry, Physics, or Math not so much. Psychology sometimes. I do think sociology has some value, but not more value than say science fiction or poetry (which also allows readers to think personally interesting thoughts that work with personal/non-objective categories).

^ for the record that's not an original idea. The https://en.wikipedia.org/wiki/Logical_positivism folk beat me to it.

> Humans have always been social primates with modest empirical tendencies. The (social) mechanisms of science turn the knobs a bit away from "social primate" and toward "empirical", but it's a difference of degree, not of kind. It is still a social enterprise. We're still status-oriented primates.

To be concrete, consider the "medicine" offered by https://en.wikipedia.org/wiki/Traditional_Chinese_medicine and then compare with actual medicine. If you'd like to call that a difference in degree that's your right I suppose... but from an external results perspective they appear to me a difference in kind. Maybe that's splitting hairs? Is a permanent marker different from a whiteboard marker by degree or kind? I dunno, I guess maybe I see the glass as 90% full rather than 10% empty here.

I do know that a dataset on the history of TCM would perform extremely poorly at predicting the progress that Chemistry has enjoyed... however a dataset on the history of Physics would do rather well. That seems like a difference of kind to me. Absolutely a difference of category from a ML clustering perspective.

> I disagree on your interpretation of Feynman's story. They published wrong numbers because they didn't want to be wrong in public.

As I'm sure you know from the link, Feynman disagrees with you here:

> When they got a number that was too high above Millikan's, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number close to Millikan's value they didn't look so hard

Fortunately for us, your claim can actually be tested. We just need to see if scientists care about being wrong when the error won't be public. They do. As evidence I #include Neuton's massive catalog of unpublished works, and Feynman's book title "The Pleasure of Finding Things Out". Much of science is an intensely single-player puzzle game. You play because it's fun, not to show off your high score.

I do agree that no one enjoys being wrong in public -- we spellcheck our work for others, not for ourselves, etc. I just think the effect size is waaaaay smaller here than most areas of human endeavor (somewhat higher, but on the order of the status effects that can be seen in crossword puzzle solving behavior). I think Feynman also believes this, as evidenced by the above quote, and his autobiography more generally.

> We totally agree on the scientific ideal. But I think it's vital to acknowledge the divergence between the ideal and actual practice, and to study the causes of that divergence.

I do have a category of "science pretenders" that I use to explain essentially non-scientists that someone has accidentally given tenure to. If you just ignore these people you can still do just fine as far as predicting the future of science goes in my experience and opinion. They don't publish anything interesting.

This is what I was meaning when I earlier freely admitted to being guilty of playing "no true scientist" -- if some university, say, adds a fashion sciences department, that won't make the people employed there scientists, or fashion sciences a science. It will be individuals applying scientific thinking (ie puzzle solving type thought). Because it all depends on small groups of puzzle solvers, that's where the modeling data is. Contrast with non-sciences where the personal tastes of big players need to be modeled, or worse yet other people's guesses as to what good personal taste of the big player is.

As you move to softer sciences I suppose I can see an argument needing to study primate behavior more, I guess I just don't see it for the hard sciences, at least if you're selected a good group to work with. Hmm... maybe not needing large grants to do you research is also important there too. So essentially I think it's important "for non-scientific purposes related to science".