> Roberts is collected one long detailed case study. Sample size of 1, no controls, no blinding and tiny experiments.

You might want to actually read what Roberts collects, rather than what you think he collects, because it is apparent that you extrapolate from this post. It's a good way to stroke your ego, but not a good way to have a discussion.

> Isn't this just more evidence that the unblinded nature of Roberts anomaly hunting is being skewed by bias?

No, it isn't, and if you read what he publishes rather than have a knee-jerk reaction to one short summary, you might be able to appreciate that.

> have determined that his methods will produce more false correlations than true ones.

That's actually not true, as far as I know, and I would appreciate if you specified exactly what you are referring to. (saying "everyone knows" is not a good enough answer)

What I do know is that this meticulous record keeping produces bogus results at an alarming rate because of the things it does not consider, like "publish or perish" pressure and the "negative reuslts go in the drawer" effect. This xkcd describes the mechanism http://xkcd.com/882/ , and these extremely well written, peer-reviewed papers, show that most published results (which are golden according to the school of thought you seem to belong to) is actually wrong: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ http://www.reuters.com/article/2012/03/28/us-science-cancer-... (points to a paywalled nature article). In case you are unable to get a copy of the nature paper: 88% of celebrated cancer results published in the last 10 years CAN NOT BE REPLICATED (that is, are most probably WRONG), despite following the "golden rules of science".

> The failure to produce previous controlled experiments is alarming.

What exactly are you referring to?

> I'm just baffled as to why you think that removing controls and oversight and experimenter blinding and lowering the sample size, TO ONE, as well as taking a single data point for some "experiments" would in any way make it better?

Because I actually live in the real world and not in some fantasy world. I don't think removing controls and oversight is a good idea. However, Roberts has no funding for these experiments because there's no way to make money off them. So he does what he finds interesting, in the best way he can, and lets others try to replicate -- and then collects their results as well. He has stumbled on a diet that seems to be more effective than any other out there (in real world terms, that of keeping weight off years later). It only works for 80% of the people, but for those it works wonders. If you can describe a plausible placebo mechanism for this result, please do - I would love to hear one (and why it seems to skip every other popular diet out there)

I do believe that a cheap, safe, easy to do experiment that is not blinded is infinitely more useful than an expensive one that will never be carried out.

The first is not perfect, but yields results that are easy to later work with, and easy to verify whether they are right or wrong by doing the same test again or other test that would pinpoint the effect. It is a weak result, but it is a result of some kind.

The second yields zero information, since it will never be done.

> lowering the sample size, TO ONE, as well as taking a single data point for some "experiments" would in any way make it better?

You don't have to agree with what Roberts is doing. But you will have to excuse me for pointing out that making comments based on what you infer he is doing or claiming from a ten line summary makes you look like an idiot, even if it makes you feel superior.