This reply will probably be ignored, since it has been two days and ycombinator has a pretty short attention span, but I have skimmed over the full study.

There are about 10 findings, and the only “high” quality finding is that there appears to be a significant increase in abstinence, especially long term abstinence from people who get treatments which get them in the rooms of AA (“TSF” treatments).

We can dive down in to the details about this finding (percent abstinent) on pages 88 and 89 of the report. They actually use six different studies, but only two of them provide information at the one-year follow-up, so only those two get listed in the summary on page 4 of the report.

Considering that they compared a number of things, and only percentage of subjects abstinent showed, with high confidence, a difference between AA/TSF and other treatments, I’m a little worried we may be seeing a multiple comparison fallacy. Then again, this is unlikely: They are seeing P values of 0.02 and 0.03 (1-in-30 to 1-in-50) for the abstinent numbers, but only look at about 10 factors.

Humphreys 2014, another meta-analysis of multiple studies, showed AA effectiveness which can not be attributed to self-selection, but it saw something different: A fairly small but significant increase in the percentage of days abstinent. While they did see in the 2020 Cochrane review, they did not see it with a high level of confidence (see pages 89-90).

Percentage days heavy drinking — I don’t have a copy of Brandsma 1980 handy, but I think that’s what the “Brandsma 1980 shows that AA results in increased binge drinking!!!111!!!!!” claim parroted by the usual anti-AA crowd talks about — might be increased in the short term after starting AA treatment, based on the one study which has 6-month follow-up figures for this; Brandsma 1980 only saw it at the 3-month follow-up and not later. Then again, the Cochrane study of those numbers show a pretty high P value (0.79, just glancing at those figures), which means we still don’t really know if Brandsma 1980 actually saw something. Scott Alexander dismisses it as an example of a multiple comparison fallacy, and the P values for it in Cochrane 2020 means we probably didn’t actually see something, but this may merit additional investigation.

Cochrane 2020, of course, didn’t look at the longitudinal studies showing really high abstinence rates for people who self-select to go to a lot of AA meetings, but even using only randomized studies, we are seeing a significant increase in number of subjects who get and stay sober when they engage in treatment which encourages AA attendance.

Another thing: There are no recent high-quality studies comparing AA to no treatment (“spontaneous remission”, an expression I see used mainly by people with a pretty strong anti-AA bias) because it’s been established for a long time that AA (or any other treatment you can think of) is better than no treatment. Even Brandsma 1980 saw that going to their AA-like meetings was significantly better than getting no treatment at all.

In summary, with some caveats, we are seeing solid scientific evidence that AA really helps alcoholics get sober and stay sober. Ten years ago, someone could still say “AA does not have a higher success rate than no treatment” with a straight face; today, to claim something like that requires ignoring a lot of scientific evidence.