drphilmarshall/pappy — pull request #5 by eggplantbren (2012-05-14 20:23:45+00:00)
Title: Added entropy-based ESS measurement

Hi Phil, this is a very trivial pull request, and one reason I'm doing it is to learn how to fork. :)

I added one more option for a way of measuring effective sample size. I believe, but can not prove, that this has better properties than Neal's (whose reasoning is based on iid Gaussians) or the simple method.

The Shannon entropy is
H = -\sum p_i log p_i
For a uniform distribution with p_i = 1/N, H = log(N). Therefore exp(H) can be regarded as generalising N to non-uniform distributions.