CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers — pull request #27 by y-p (2013-03-23 19:44:11+00:00)
Title: CH3: various tweaks

Hi cam,

Just read through ch3, here are my notes.
as always, please accept what you like and disregard the rest.

Some notes as a reader:

- re the introduction discussing "space" and "surface", IIUC, an important
role of the prior is to assign **zero** probability to some areas and thereby
forbid the MCMC from entering that region. That also explains the concept
of a "surface" in the space, in that the limited exten of the "surface" corresponds
to the finite area of p(x)>0 in the example.
I suggest that the "0 probablity there" role of the prior be touched upon in the intro.
- quote "This is not as trivial as it may appear." - actually does not appear trivial at all.
- there's a comment in a code cell "# notice the size", which directs my attention
but does not explain what exactly is the point of that param.
- in the "Algorithms to perform MCMC" section
1. step 3 is really the algorithmic counterpoint to the observation data "stretching and shaping" analogy you gave in the intro, it would be good to make that connection at that point.
2. the pseudo code doesn't describe the "else" clause, what happens when we
reject a step.
3. quote:
```
In the pseudo-code algorithm of MCMC above, the only position that matters
is the current position. Thus it is fine to halt an MCMC algorithm and inspect
its progress, with the intention of starting it up again later
```
It's unclear if this a property shared by all MCMC algorithms, an implementation
detail of pyMC's chosen algorithm. is the random walk process always memoryless?
- will definitly try my hand atn interactive webGL visualization of the "mountain
as function of sample size" visualization. what a wonderful example.

- couldn't reproduce the 2x2 cluster plots, I just get blank plots
maybe the ylim is off?
- can't reproduce the first mixture model grap, I get a blank screen
- I reworded the paragraph about choosing a best point estimate of the gaussian
parameters, since it stated it couldn't be done and then proceeded to do so. :)
- fixed a code error in the "Important: Values in a sample " section, misnamed
variable
- I'm note sure I understood this section. is the point you were making that since
various rvs are related, you can't mix and match samples from different indices?
that's what was intended by the "indexing correctly" remark? you state that exactly
after the example, but I found this a bit hard to read.
- in the "Diagnosing Convergence" section, quote
`Earlier I mentioned that the autocorrelation`, I believe that's actually
the first time autocorrelation is mentioned in the document, perhaps
previously edited out?
- there seems to be a spurious markdown cell right below the "thinning" section
- `pymc.Matplot.plot()` section, the second cell suggests there ought to be a
figure there, demonstrating the use of `pymc.Matplot.plot()`, it seems to be missing.

That's it, a bit long I know.

Gorgeous, gorgeous graphs!