trentmkelly/LessWrong-43k huggingface dataset:
Metauncertainty
by jimmy — 2009-04-10T23:41:52.946Z

Response to: When (Not) To Use Probabilities
“It appears to be a quite general principle that, whenever there is a randomized way of doing something, then there is a nonrandomized way that delivers better performance but requires more thought.” —E. T. Jaynes
The uncertainty due to vague (non math) language is no different than uncertainty by way of "randomizing" something (after all, probability is in the mind). The principle still holds; you should be able to come up with a better way of doing things if you can put in the extra thought.
In some cases, you can't afford to waste time or it's not worth the thought, but when dealing with things such as the deciding whether to run the LHC or signing up for cryonics, there's time, and it's sorta a big deal, so it pays to do it right.
If you're asked "how likely is X?", you can answer "very unlikely" or "0.127%". The latter may give the impression that the probability is known more precisely than it is, but the first is too vague; both strategies do poorly on the log score.
If you are unsure what probability to state, state this with... another probability distribution.

"My probability distribution over probabilities is an exponential with a mean of 0.127%" isn't vague, it isn't overconfident (at the meta^1 level), and gives you numbers to actually bet on.
The expectation value of the metaprobability distribution (integral from 0 to 1 of Pmeta*p*dp) is equal to the probability you give when trying to maximize your expected log score .
To see this, we write out the expected log score (Integral from 0 to 1 of Pmeta*(p*log(q)+(1-p)log(1-q))dp). If you split this into two integrals and pull out the terms that are independent of p, the integrals just turn into the expectation value of p, and the formula is now that of the log score with p replaced with mean(p). We already know that the log score is maximized when q = p, so in this case we set q = mean(p)
This is a very useful result when dealing with extremes where we are not well calibrated. Instead of punting and saying "err... prolly aint gonna happen", put a probability distribution on your probability distribution and take the mean. For example, if you think X is true, but you don't know if you're 99% sure or 99.999% sure, you've got to bet at ~99.5%.
It is still no guarantee that you'll be right 99.5% of times (by assumption we're not calibrated!), but...
Maybe Theism Is OK 
by byrnema — 2009-04-10T21:09:21.105Z

I would like to argue that there could be a more tolerant view of religion/theism here on Less Wrong. The extent to which theism is vilified here seems disproportionate to me.

It depends on the specific scenario how terrible religion is. It is easy to look at the very worst examples of religion and conclude that religion can be irrational in a terribly wrong way. However, religion can also be nearly rational. Considering that any way we view the world is an illusion to some extent. Indeed the whole point of this site is to learn ways to shed more of our illusions, not that we have no illusions.

There are the religious beliefs that contradict empirical observation and those that are independent of it...

A) Could it be rational for a person to hold beliefs that are independent of empirical observation if (a) the person concedes that they are irrational not empirically based and (b) is willing to drop them if they prove to not be useful?

B) Could it be rational for a person to hold unusual beliefs as a result of contradicting empirical observations?

As a least convenient world exercise, what is the most rational belief in God that you can think of?
Akrasia and Shangri-La
by Eliezer Yudkowsky — 2009-04-10T20:53:14.746Z

Continuation of: The Unfinished Mystery of the Shangri-La Diet

My post about the Shangri-La Diet is there to make a point about akrasia. It's not just an excuse: people really are different and what works for one person sometimes doesn't work for another.

You can never be sure in the realm of the mind... but out in material foodland, I know that I was, in fact, drinking extra-light olive oil in the fashion prescribed. There is no reason within Roberts's theory why it shouldn't have worked.

Which just means Roberts's theory is incomplete. In the complicated mess that is the human metabolism there is something else that needs to be considered. (My guess would be "something to do with insulin".)

But if the actions needed to implement the Shangri-La Diet weren't so simple and verifiable... if some of them took place within the mind... if it took, not a metabolic trick, but willpower to get to that amazing state where dieting comes effortlessly and you can lose 30 pounds...

Then when the Shangri-La Diet didn't work, we unfortunate exceptions would get yelled at for doing it wrong and not having enough willpower. Roberts already seems to think that his diet ought to work for everyone; when someone says it's not working, Roberts tells them to drink more extra-light olive oil or try a slightly different variant of the diet, rather than saying, "This doesn't work for some people and I don't know why."

If the failure had occurred somewhere inside the dark recesses of my mind where it could be blamed on me, rather than within my metabolism...

If Roberts's hypothesis is correct, then I'm sure that plenty of people have made some dietary change, started losing weight due to the disrupted flavor-calorie association, and congratulated themselves on their wonderful willpower for eating less. When I moved out of my parents' home and started eating less and exercising and losing more than a pound a week, you can bet I was congratulating myself on my amazing willpower.

Hah. No, I just stumbled onto a metabolic pot of gold that let me lose a lot of weight using a sustainable expenditure of willpower. When that pot of gold was exhausted, willpower ceased to avail.

(The metabolically privileged don't believe in metabolic privilege, since they are able to lose weight by trying! harder! to diet and exercise, and the diet and exercise act...
The Unfinished Mystery of the Shangri-La Diet
by Eliezer Yudkowsky — 2009-04-10T20:30:15.899Z

Followup to: Beware of Other-Optimizing

Once upon a time, Seth Roberts (a professor of psychology at Berkeley, on the editorial board of Nutrition) noticed that he'd started losing weight while on vacation in Europe. For no apparent reason, he'd stopped wanting to eat.

Some time later, The Shangri-La Diet swept... the econoblogosphere, anyway. People including some respectable economists tried it, found that it actually seemed to work, and told their friends.

The Shangri-La Diet is unfortunately named - I would have called it "the set-point diet". And even worse, the actual procedure sounds like the wackiest fad diet imaginable:

Just drink two tablespoons of extra-light olive oil early in the morning... don't eat anything else for at least an hour afterward... and in a few days it will no longer take willpower to eat less; you'll feel so full all the time, you'll have to remind yourself to eat.

Why? I'm tempted to say "No one knows" just to see what kind of comments would show up, but that would be cheating. Roberts does have a theory motivating the diet, an elegant combination of pieces individually backed by previous experiments:

Your metabolism has a set point, like the setting on a thermostat: when your weight is below the set point, you feel hungry; when your weight is above the set point, you feel full.

But the set point is not a constant; it is raised and lowered by what you eat.

This mechanism in turn seems to be regulated by a flavor-calorie association. (Possibly as a famine-storage mechanism that tries to store more resources when dense food sources are available.) If you eat something with flavor X, which is followed by your metabolism detecting a large source of calories, flavor X will (a) seem more appealing and taste better, and (b) will raise your set point whenever you eat items with flavor X.

Your set point is always naturally dropping, but is raised by eating; usually these forces are in dynamic balance and your weight stays constant.

I'm not going to go into all the existing evidence that backs up each step of this theory, but the theory is very beautiful and elegant. The actual Shangri-La Diet is painfully simple by comparison: consume nearly tasteless extra-light olive oil, being careful not to associate it with any flavors before or after, to raise your body weight a little without rai...
Spay or Neuter Your Irrationalities
by Alicorn — 2009-04-10T20:08:45.734Z

No human person has, so far as I am aware, managed to eradicate all irrationalities from their thinking. They are unavoidable, and this is particularly distressing when the irrationalities are lurking in your brain like rats in the walls and you don't know what they are. Of course you don't know what they are - they are irrationalities, and you are a rationalist, so if you had identified them, they would be dying (quickly or slowly, but dying). It's only natural for someone committed to rationality to want to indiscriminately exterminate the threats to the unattainable goal.
But are they all worth getting rid of?
It is my opinion that they are not: some irrationalities are small and cute and neutered, and can be confined and kept where you can see them, like pet gerbils instead of rats in the walls.
I'll give you an example: I use iTunes for my music organization and listening. iTunes automatically records the number of times I have listened to each song and displays it. Within a given playlist, I irrationally believe that all of these numbers have to match: if I have listened to the theme from The Phantom of the Opera exactly fifty-two times, I have to also have listened to "The Music of the Night" exactly fifty-two times, no matter how much I want to listen to the theme on repeat all afternoon.
Does this make any sense? No, of course not, but it isn't worth my time to get rid of it. It is small - it affects only a tiny corner of my life, and if it starts to get in the way of my musical preferences, I can cheat it by resetting play counts or fast-forwarding through songs (like I could get around the chore of feeding a gerbil with an automatic food dispenser). It is "cute" - I can use it as a conversation starter and people generally find it a mildly entertaining quirk, not evidence that I need psychiatric help. I have it metaphorically neutered - since I make no effort to suppress it, I'm able to recognize the various emotional reactions that satsifying or frustrating this irrational preference creates, and I would notice them if they cropped up anywhere else, where I would deal with them appropriately. I also don't encourage it to memetically spread to others. I keep it where I can see it - I make note of when I take actions to satisfy my irrational preference, and acknowledge in so doing that i...
That Crisis thing seems pretty useful
by Eliezer Yudkowsky — 2009-04-10T17:10:32.945Z

Since there's been much questioning of late over "What good is advanced rationality in the real world?", I'd like to remind everyone that it isn't all about post-doctoral-level reductionism.

In particular, as a technique that seems like it ought to be useful in the real world, I exhibit the highly advanced, difficult, multi-component Crisis of Faith aka Reacting To The Damn Evidence aka Actually Changing Your Mind.

Scanning through this post and the list of sub-posts at the bottom (EDIT: copied to below the fold) should certainly qualify it as "extreme rationality" or "advanced rationality" or "x-rationality" or "Bayescraft" or whatever you want to distinguish from "traditional rationality as passed down from Richard Feynman".

An actual sit-down-for-an-hour Crisis of Faith might be something you'd only use once or twice in every year or two, but on important occasions. And the components are often things that you could practice day in and day out, also to positive effect.

I think this is the strongest foot that I could put forward for "real-world" uses of my essays. (Anyone care to nominate an alternative?)

Below the fold, I copy and paste the list of components from the original post, so that we have them at hand:

Avoiding Your Belief's Real Weak Points - One of the first temptations in a crisis of faith is to doubt the strongest points of your belief, so that you can rehearse your good answers. You need to seek out the most painful spots, not the arguments that are most reassuring to consider.

The Meditation on Curiosity - Roger Zelazny once distinguished between "wanting to be an author" versus "wanting to write", and there is likewise a distinction between wanting to have investigated and wanting to investigate. It is not enough to say "It is my duty to criticize my own beliefs"; you must be curious, and only uncertainty can create curiosity. Keeping in mind Conservation of Expected Evidence may help you Update Yourself Incrementally: For every single point that you consider, and each element of new argument and new evidence, you should not expect your beliefs to shift more (on average) in one direction than another - thus you can be truly curious each time about how it will go.

Cached Thoughts and Pirsig's Original Seeing, to prevent standard thoughts from rushing in and completing the pattern.

The Litany of Gendlin and t

...
How theism works
by Paul Crowley — 2009-04-10T16:16:20.261Z

There's a reason we can all agree on theism as a good source of examples of irrationality.

Let's divide the factors that lead to memetic success into two classes: those based on corresponding to evidence, and those detached from evidence. If we imagine a two-dimensional scattergram of memes rated against these two criteria, we can define a frontier of maximum success, along which any idea can only gain in one criterion by losing on the other. This doesn't imply that evidential and non-evidential success are opposed in general; just that whatever shape memespace has, it will have a convex hull that can be drawn across this border.

Religion is what you get when you push totally for non-evidential memetic success. All ties to reality are essentially cut. As a result, all the other dials can be pushed up to 11. God is not just wise, nice, and powerful - he is all knowing, omnibenificent, and omnipotent. Heaven and Hell are not just pleasant and unpleasant places you can spend a long time in - they are the very best possible and the very worst possible experiences, and for all eternity. Religion doesn't just make people better; it is the sole source of morality. And so on; because all of these things happen "offstage", there's no contradictory evidence when you turn the dials up, so of course they'll end up on the highest settings.

This freedom is theism's defining characteristic. Even the most stupid pseudoscience is to some extent about "evidence": people wouldn't believe in it if they didn't think they had evidence for it, though we now understand the cognitive biases and other effects that lead them to think so. That's why there are no homeopathic cures for amputation.

I agree with other commentators that the drug war is the other real world idea that I would attack here without fear of contradiction, but I would still say that drug prohibition is a model of sanity compared to theism. Theism really is the maddest thing you can believe without being considered mad.

Footnote: This was originally a comment on The uniquely awful example of theism, but I was encouraged to make a top-level post from it. I should point out that there are issues with my dividing line between "evidence-based" and "not evidence-based", since you could argue that mathematics is not evidence-based and nor is the belief that evidence is a good way to learn about the world; however, it should b...
Beware of Other-Optimizing
by Eliezer Yudkowsky — 2009-04-10T01:58:49.232Z

I've noticed a serious problem in which aspiring rationalists vastly overestimate their ability to optimize other people's lives. And I think I have some idea of how the problem arises.

You read nineteen different webpages advising you about personal improvement—productivity, dieting, saving money. And the writers all sound bright and enthusiastic about Their Method, they tell tales of how it worked for them and promise amazing results...

But most of the advice rings so false as to not even seem worth considering. So you sigh, mournfully pondering the wild, childish enthusiasm that people can seem to work up for just about anything, no matter how silly. Pieces of advice #4 and #15 sound interesting, and you try them, but... they don't... quite... well, it fails miserably. The advice was wrong, or you couldn't do it, and either way you're not any better off.

And then you read the twentieth piece of advice—or even more, you discover a twentieth method that wasn't in any of the pages—and STARS ABOVE IT ACTUALLY WORKS THIS TIME.

At long, long last you have discovered the real way, the right way, the way that actually works. And when someone else gets into the sort of trouble you used to have—well, this time you know how to help them. You can save them all the trouble of reading through nineteen useless pieces of advice and skip directly to the correct answer. As an aspiring rationalist you've already learned that most people don't listen, and you usually don't bother—but this person is a friend, someone you know, someone you trust and respect to listen.

And so you put a comradely hand on their shoulder, look them straight in the eyes, and tell them how to do it.

I, personally, get quite a lot of this. Because you see... when you've discovered the way that really works... well, you know better by now than to run out and tell your friends and family. But you've got to try telling Eliezer Yudkowsky. He needs it, and there's a pretty good chance that he'll understand.

It actually did take me a while to understand. One of the critical events was when someone on the Board of the Institute Which May Not Be Named, told me that I didn't need a salary increase to keep up with inflation—because I could be spending substantially less money on food if I used an online coupon ...
The uniquely awful example of theism
by gjm — 2009-04-10T00:30:08.149Z

When an LW contributor is in need of an example of something that (1) is plainly, uncontroversially (here on LW, at least) very wrong but (2) an otherwise reasonable person might get lured into believing by dint of inadequate epistemic hygiene, there seems to be only one example that everyone reaches for: belief in God. (Of course there are different sorts of god-belief, but I don't think that makes it count as more than one example.) Eliezer is particularly fond of this trope, but he's not alone.

How odd that there should be exactly one example. How convenient that there is one at all! How strange that there isn't more than one!

In the population at large (even the smarter parts of it) god-belief is sufficiently widespread that using it as a canonical example of irrationality would run the risk of annoying enough of your audience to be counterproductive. Not here, apparently. Perhaps we-here-on-LW are just better reasoners than everyone else ... but then, again, isn't it strange that there aren't a bunch of other popular beliefs that we've all seen through? In the realm of politics or economics, for instance, surely there ought to be some.

Also: it doesn't seem to me that I'm that a much better thinker than I was a few years ago when (alas) I was a theist; nor does it seem to me that everyone on LW is substantially better at thinking than I am; which makes it hard for me to believe that there's a certain level of rationality that almost everyone here has attained, and that makes theism vanishingly rare.

I offer the following uncomfortable conjecture: We all want to find (and advertise) things that our superior rationality has freed us from, or kept us free from. (Because the idea that Rationality Just Isn't That Great is disagreeable when one has invested time and/or effort and/or identity in rationality, and because we want to look impressive.) We observe our own atheism, and that everyone else here seems to be an atheist too, and not unnaturally we conclude that we've found such a thing. But in fact (I conjecture) LW is so full of atheists not only because atheism is more rational than theism (note for the avoidance of doubt: yes, I agree that atheism is more rational than theism, at least for people in our epistemic situation) but also because

the readership of LW (and, earlier, of OB) is drawn disproportionately from communities that have long been atheist

...
Extreme Rationality: It Could Be Great
by badger — 2009-04-09T22:00:13.538Z

Reply to: Extreme Rationality: It's Not That Great

I considered making this into a comment on Yvain's last post, but I'd like to redirect the discussion slightly. Yvain's warning is important, but we're left with the question of how to turn the current state of the art in rationality into something great. I think we are all on the same page that more is possible. Now we just need to know how to get there.

Even though Yvain disapproved of Eliezer's recent post on day jobs, I thought the two shared a common thread: rationalists should be careful about staying in Far-mode too long. I took Eliezer's point to be more about well-developed rationalist communities, and Yvain's to be about our rag-tag band of aspirants, but I think they are both speaking to the same issue. All of this has to be for a purpose, and we can't become ungrounded.

Near- and Far-mode have to be balanced. This shouldn't be surprising, because in this context, Near and Far roughly equate to applied and theoretical work. The two intermingle and build off one another. The history of math and physics is filled with paired problems: calculus and dynamics, Fourier series and heat distribution, least-squares and astronomy, etc. Real world problems need theory to be solved, but theory needs problems to motivate and test it.

My guess is that any large subject develops through the following iterative alteration between Near and Far:

F1. Develop general theory.
F2. Refine and check for consistency and correctness.
F3. Consolidate theory.
N1. Apply existing theory to problems.
N2. Evaluate successes and failures.
GOTO F1.

This looks like a close relative of our trusty friend, the scientific method, and is similarly idealized. In terms of this process, I think the Less Wrong community is between F2 and F3. We have lots of phrases, techniques, and standard examples laying around, and work has been done on testing them for conceptual soundness. The wiki represents an attempt to begin consolidating this information so we can move onto more applied domains.

Assuming this process is productive, how long will it take to produce something useful? If Newton invented undergraduate material in math and physics, as is often quipped, I think existing x-rationality theory and techniques are on a JR High lev...