# Book review: How Social Science Got Better

Book review: How Social Science Got Better: Overcoming Bias with More
Evidence, Diversity, and Self-Reflection, by Matt Grossmann.

It\'s easy for me to become disenchanted with social science when so
much of what I read about it is selected from the most pessimistic and
controversial reports.

With this book, Grossmann helped me to correct my biased view of the
field. While plenty of valid criticisms have been made about social
science, many of the complaints lobbed against it are little more than
straw men.

Grossmann offers a sweeping overview of the progress that the field has
made over the past few decades. His tone is optimistic and hearkens back
to Steven Pinker\'s [Better Angels of our
Nature](https://bayesianinvestor.com/blog/index.php/2011/11/23/the-better-angels-of-our-nature/),
while maintaining a rigorous (but dry) style akin to the less
controversial sections of Robin Hanson\'s [Age of
Em](http://ageofem.com). Throughout the book, Grossmann aims to outdo
even Wikipedia in his use of a neutral point of view. Politics and
biases have marred efforts to understand some important topics. Science
helps mitigate these biases. By science, I don\'t mean the features that
typically come to mind first when thinking of science in politicized
contexts. Grossmann highlights the importance of empiricism and reason
in scientific inquiry, rather than authority.

Grossmann reaffirmed some of my hopes that most people who call
themselves scientists use enough empiricism, and enough scientific
heuristics, to improve on human knowledge, in spite of biases.

Most of the book focuses on the quality of the scientific literature,
rather than whether the average person has become more knowledgeable
thanks to science over the past few decades.

### Good Trends

\"Bayesian and frequentist assumptions, while hardly reconciled, are
more regularly compared\".

P-hacking is not getting worse. Researchers are focusing less on
statistical significance, and more on evaluating the sensitivity to
assumptions.

Researchers have been moving away from rational choice models, grand
theories, and Marxism. They\'re replacing those with increased
empiricism.

> Rather than consider one gold standard of experiments, Kubinek says we
> should consider three silver standards, judging each by how much they
> reduce uncertainty around our causal knowledge; experiments,
> qualitative process training, and observational analysis could all
> play a role depending on the current state of research.

High level disagreements between, e.g. [Charles
Murray](https://en.wikipedia.org/wiki/Charles_Murray_(political_scientist))
and his critics have been largely replaced by more specific and more
civil disagreements over narrower issues, with more precise claims on
each side reducing the misunderstandings and more closely approaching
the kinds of disputes that get resolved empirically.

### The Replication Non-Crisis

Replication failures have uncovered systemic problems with wide areas of
science.

Far from dooming those fields, they represent both a symptom of
increasing rigor, and a cause of additional effort to improve the
quality of research.

One really nice replication failure (which didn\'t get enough publicity
for me to notice at the time) was [The Grievance Studies
Affair](https://en.wikipedia.org/wiki/Grievance_studies_affair).
Researchers tried to replicate the [Sokal
hoax](https://en.wikipedia.org/wiki/Sokal_affair). Their falsified (and
mostly silly) gender studies papers got a [very mixed
reaction](https://research.abo.fi/ws/portalfiles/portal/25213489/Lagerspetz+2020+Grievance.pdf).
Four were published in low impact factor journals. They were exposed as
frauds, more than a year after the first one was published, but before
the project planned to reveal the hoax.

That\'s evidence that even in areas where I expect ideology to often
trump science, there are many somewhat competent people who are limiting
the harm done by ideology.

### Fact Checks

I didn\'t check Grossmann\'s claims very carefully. My impression is
that they\'re mostly quite good. One exception stood out: he mentions
\"overpoliced communities\". He cites a
[paper](https://www.annualreviews.org/doi/abs/10.1146/annurev-polisci-060415-093825)
that rants about problems with police and with research about police,
but which doesn\'t seem to document overpolicing. More likely Grossmann
cited the paper for its claims about research, and carelessly assumed
that communities were overpoliced.

The research that I\'ve found which bears directly on
[overpolicing](https://marginalrevolution.com/marginalrevolution/2022/08/still-under-policed-and-over-imprisoned.html)
says the US seems
[underpoliced](https://marginalrevolution.com/marginalrevolution/2020/06/underpoliced-and-overprisoned-revisited.html).

To repeat: this seems like an exception. I expect it would be hard to
find more similarly bad claims, even though the book contains an
unusually high number of factual claims.

Another reference that I checked was about why peer review became seen
as crucial for scientific legitimacy. Grossmann cites [Scientific
Autonomy, Public Accountability, and the Rise of "Peer Review" in the
Cold War United
States](https://gwern.net/doc/statistics/peer-review/2018-baldwin.pdf)
as saying peer review was \"adopted largely to justify public scientific
support\".

The paper documents that the codification of peer review was quite
political. Casual refereeing was moderately common before 1970. There
was a sudden shift to formal, mandatory peer review in the 1970s in the
US. That coincided closely with a decline in the Cold War related
consensus behind government funding of science. The paper provides
evidence that some rather arbitrary political considerations were key
forces behind the elevation of a minor part of scientific publishing to
a sacred ritual. Grossmann\'s summary of this is mostly true, but the
details sound more petty and short-sighted than Grossmann led me to
expect.

### Concluding Thoughts

The book is not exciting, eloquent, or controversial enough to become as
influential as Pinker.

It\'s likely somewhat biased toward only showing the positive trends in
social science. But that\'s what I needed, to remind me that, for many
areas where I\'m not actively researching topics, my other information
sources have been strongly selected for a focus on what has gone wrong
recently.

I highly recommend reading the first few chapters. I\'m unsure whether
to recommend reading the whole book.

\[Thanks to ChatGPT for lightly editing this review.\]