by dxbydt | 2026-01-25 19:05:19+00:00
such an interesting anomaly! Especially for HN since the underlying stat procedure is very programmer friendly. The job is to create a bipartite graph and count edges. So you want 2 disjoint sets X and Y. X has all the "good" esg companies such as X = {google, meta, microsoft, ...} Then in Y you put the "evil" companies who don't care about esg, say Y = {oracle, revlon, walmart,...}

Now you draw an edge from X to Y. But the trick is, you can only connect one node in X to exactly one node in Y. After you connect the nodes, that particular edge is part of your sample size. But two very big issues.

First clause, the nodes that you connect must be very similar (like same business sector, similar earnings). So you can't connect a google to a pfizer.
The second caveat is, once you connect the two nodes with an edge, the two nodes are used up! No reuse (the stat algo is PSM without replacement which is a mouthful but basically means you can't reuse those two nodes).

The paper authors EIS say they start with 270 nodes. Then they drew 90 edges subject to above rules. King says you cannot draw 90. You can only draw 15. And that's the whole debate!

The reason you cannot draw 90 is subtle. Because the problem basically boils down to how to find your evil twin. So for a microsoft in X, the evil twin might be oracle in Y. But now you cannot reuse that oracle! So if you want to match a google in X, the only comparable match is again with oracle, but hey no reuse, so you are out of luck. If you say I will match google with meta, that's not possible because both google and meta practice esg, so meta isn't your evil twin! in fact meta is also in X, and you cannot draw edge from X to X because its a bipartite graph.

The problem is happening because of herding behavior. All faangs are basically similar ie. they herd together. So everybody ends up in X, because google, microsoft, meta, netflix etc every faang practices esg. So you don't have enough evil twins to match everybody. you match the microsoft with the oracle, but what do you do with the google,meta,netflix ?

So King says the way EIS found the 90 matches is by violating the first clause. (In stat we call this the tight caliper setting). So if you match a google to a walmart, you have violated the first clause, because those two businesses are not similar at all. But by making that fake match, you will get your 90 matches.

So the whole thing hinges upon 90 matches versus 15 matches. So you can ask why do we care. Like, why don't we do perform propensity scoring with just 15 instead of 90 ? Answer to that is even more interesting! After you find the matches, you have to average the stock price returns for the X bucket and the Y bucket. Say the difference of returns between the two buckets is 3%. You can't report "companies doing esg are generating 3% more market return than companies not doing esg. boom! done!". Because you haven't said what the standard error is.

Suppose you only have 4 matches. Say that 3% excess return has a standard deviation of 10%, then standard error becomes 10/sqrt(4) = 5%. So then your conclusion is 3% +- 25%. ie. your confidence interval is [-7%, 13%]. Since 0 is part of your confidence interval, it means you have no result! You've learnt precisely nothing from this whole experiment :(

But suppose you had 100 matches. Then 10/sqrt(100) = 1%, so your interval is 3% +- 21% ie. [1%,5%]. Now you have a real result! Now your paper can say if you do esg, you get 3% excess return, statistically significant ci [1,5], p values golden and then everybody will cite your paper and you get 6000 cites, world famous paper...

So in order to get those 90 matches, the authors must have relaxed the first clause. There is no other way. Essentially, they matched a google with a texas instruments, an apple with a samsung etc. Finding 15 matches is not hard. You can match a Tesla with a GE, ikea with walmart, roche with pfizer, ups with fedex, so on. But finding 90 without violating both clauses in that dataset of 270 companies is not possible.