trentmkelly/LessWrong-43k huggingface dataset:
The Parable of the Dagger
by Eliezer Yudkowsky — 2008-02-01T20:53:05.000Z

Once upon a time, there was a court jester who dabbled in logic.

The jester presented the king with two boxes. Upon the first box was inscribed:

"Either this box contains an angry frog, or the box with a false inscription contains an angry frog, but not both."

On the second box was inscribed:

"Either this box contains gold and the box with a false inscription contains an angry frog, or this box contains an angry frog and the box with a true inscription contains gold."

And the jester said to the king: "One box contains an angry frog, the other box gold; and one, and only one, of the inscriptions is true."

The king opened the wrong box, and was savaged by an angry frog.

"You see," the jester said, "let us hypothesize that the first inscription is the true one. Then suppose the first box contains gold. Then the other box would have an angry frog, while the box with a true inscription would contain gold, which would make the second statement true as well. Now hypothesize that the first inscription is false, and that the first box contains gold. Then the second inscription would be—"

The king ordered the jester thrown in the dungeons.

A day later, the jester was brought before the king in chains, and shown two boxes.

"One box contains a key," said the king, "to unlock your chains; and if you find the key you are free. But the other box contains a dagger for your heart, if you fail."

And the first box was inscribed:

"Either both inscriptions are true, or both inscriptions are false."

And the second box was inscribed:

"This box contains the key."

The jester reasoned thusly: "Suppose the first inscription is true. Then the second inscription must also be true. Now suppose the first inscription is false. Then again the second inscription must be true. So the second box must contain the key, if the first inscription is true, and also if the first inscription is false. Therefore, the second box must logically contain the key."

The jester opened the second box, and found a dagger.

"How?!" cried the jester in horror, as he was dragged away. "It's logically impossible!"

"It is entirely possible," replied the king. "I merely wrote those inscriptions on two boxes, and then I put the dagger in the second one."

(Adapted from Raymond Smullyan.)
OB Meetup: Millbrae, Thu 21 Feb, 7pm
by Eliezer Yudkowsky — 2008-01-31T23:18:20.000Z

The Overcoming Bias meetup has been scheduled for Thursday, February 21st, at 7pm. We're going to look at locating this in Millbrae within walking distance of the BART / Caltrain station. The particular restaurant I had in mind turns out to be booked for Thursdays, so if you know a good Millbrae restaurant (with a private room?) in walking distance of the train station, please post in the comments. I'll be looking at restaurants shortly.

Why not schedule to a day other than Thursday, you ask?

Because:

Robin Hanson will be in the Bay Area and attending! Woohoo!

If you would be able to make Thursday the 21st, 7pm, in Millbrae, somewhere near the BART/Caltrain, please vote below. No, seriously, please vote, now - the kind of restaurant I have to find depends on how many people will be attending.

Opinion Polls & Market Research
Newcomb's Problem and Regret of Rationality
by Eliezer Yudkowsky — 2008-01-31T19:36:56.000Z

The following may well be the most controversial dilemma in the history of decision theory:

A superintelligence from another galaxy, whom we shall call Omega, comes to Earth and sets about playing a strange little game. In this game, Omega selects a human being, sets down two boxes in front of them, and flies away.

Box A is transparent and contains a thousand dollars.
Box B is opaque, and contains either a million dollars, or nothing.

You can take both boxes, or take only box B.

And the twist is that Omega has put a million dollars in box B iff Omega has predicted that you will take only box B.

Omega has been correct on each of 100 observed occasions so far - everyone who took both boxes has found box B empty and received only a thousand dollars; everyone who took only box B has found B containing a million dollars. (We assume that box A vanishes in a puff of smoke if you take only box B; no one else can take box A afterward.)

Before you make your choice, Omega has flown off and moved on to its next game. Box B is already empty or already full.

Omega drops two boxes on the ground in front of you and flies off.

Do you take both boxes, or only box B?

And the standard philosophical conversation runs thusly:

One-boxer: "I take only box B, of course. I'd rather have a million than a thousand."

Two-boxer: "Omega has already left. Either box B is already full or already empty. If box B is already empty, then taking both boxes nets me $1000, taking only box B nets me $0. If box B is already full, then taking both boxes nets $1,001,000, taking only box B nets $1,000,000. In either case I do better by taking both boxes, and worse by leaving a thousand dollars on the table - so I will be rational, and take both boxes."

One-boxer: "If you're so rational, why ain'cha rich?"

Two-boxer: "It's not my fault Omega chooses to reward only people with irrational dispositions, but it's already too late for me to do anything about that."

There is a large literature on the topic of Newcomblike problems - especially if you consider the Prisoner's Dilemma as a special case, which it is generally held to be. "Paradoxes of Rationality and Cooperation" is an edited volume that includes Newcomb's original essay. For those who read only online material, this PhD thesis summarizes the major standard positions.

I'm ...
Something to Protect
by Eliezer Yudkowsky — 2008-01-30T17:52:49.000Z

In the gestalt of (ahem) Japanese fiction, one finds this oft-repeated motif: Power comes from having something to protect.

I'm not just talking about superheroes that power up when a friend is threatened, the way it works in Western fiction. In the Japanese version it runs deeper than that.

In the X saga it's explicitly stated that each of the good guys draw their power from having someone—one person—who they want to protect. Who? That question is part of X's plot—the "most precious person" isn't always who we think. But if that person is killed, or hurt in the wrong way, the protector loses their power—not so much from magical backlash, as from simple despair. This isn't something that happens once per week per good guy, the way it would work in a Western comic. It's equivalent to being Killed Off For Real—taken off the game board.

The way it works in Western superhero comics is that the good guy gets bitten by a radioactive spider; and then he needs something to do with his powers, to keep him busy, so he decides to fight crime. And then Western superheroes are always whining about how much time their superhero duties take up, and how they'd rather be ordinary mortals so they could go fishing or something.

Similarly, in Western real life, unhappy people are told that they need a "purpose in life", so they should pick out an altruistic cause that goes well with their personality, like picking out nice living-room drapes, and this will brighten up their days by adding some color, like nice living-room drapes. You should be careful not to pick something too expensive, though.

In Western comics, the magic comes first, then the purpose: Acquire amazing powers, decide to protect the innocent. In Japanese fiction, often, it works the other way around.

Of course I'm not saying all this to generalize from fictional evidence. But I want to convey a concept whose deceptively close Western analogue is not what I mean.

I have touched before on the idea that a rationalist must have something they value more than "rationality": The Art must have a purpose other than itself, or it collapses into infinite recursion. But do not mistake me, and think I am advocating that rationalists should pick out a nice altruistic cause, by way of having something to do, because rationali...
Trust in Bayes
by Eliezer Yudkowsky — 2008-01-29T23:12:36.000Z

Followup to: Beautiful Probability, Trust in Math

In Trust in Math, I presented an algebraic proof that 1 = 2, which turned out to be - surprise surprise - flawed. Trusting that algebra, correctly used, will not carry you to an absurd result, is not a matter of blind faith. When we see apparent evidence against algebra's trustworthiness, we should also take into account the massive evidence favoring algebra which we have previously encountered. We should take into account our past experience of seeming contradictions which turned out to be themselves flawed. Based on our inductive faith that we may likely have a similar experience in the future, we look for a flaw in the contrary evidence.

This seems like a dangerous way to think, and it is dangerous, as I noted in "Trust in Math". But, faced with a proof that 2 = 1, I can't convince myself that it's genuinely reasonable to think any other way.

The novice goes astray and says, "The Art failed me."
The master goes astray and says, "I failed my Art."

To get yourself to stop saying "The Art failed me", it's helpful to know the history of people crying wolf on Bayesian math - to be familiar with seeming paradoxes that have been discovered and refuted. Here an invaluable resource is "Paradoxes of Probability Theory", Chapter 15 of E. T. Jaynes's Probability Theory: The Logic of Science (available online).

I'll illustrate with one of Jaynes's examples:

If you have a
conditional probability distribution P(X|C), the unconditional
probability P(X) should be a weighted average of the various P(X|C),
and therefore intermediate between the various P(X|C) in value -
somewhere between the minimum and the maximum P(X|C). That is: If you
flip a coin before rolling a die, and the die is a four-sided die if
the coin comes up heads, or ten-sided if the coin comes up tails, then
(even without doing an exact calculation) you know that the compound
probability of rolling a "1" occupies the range [0.1, 0.25].

Now suppose a two-dimensional array, M cells wide and N cells tall, with positions written (i, j) with i as the horizontal coordinate and j as the vertical coordinate. And suppose a uniform probability distribution over the array: p(i, j) = 1/MN for all i, j. Finally, let X be the event that i < j. ...
The "Intuitions" Behind "Utilitarianism"
by Eliezer Yudkowsky — 2008-01-28T16:29:06.000Z

(Still no Internet access. Hopefully they manage to repair the DSL today.)

I haven't said much about metaethics - the nature of morality - because that has a forward dependency on a discussion of the Mind Projection Fallacy that I haven't gotten to yet. I used to be very confused about metaethics. After my confusion finally cleared up, I did a postmortem on my previous thoughts. I found that my object-level moral reasoning had been valuable and my meta-level moral reasoning had been worse than useless. And this appears to be a general syndrome - people do much better when discussing whether torture is good or bad than when they discuss the meaning of "good" and "bad". Thus, I deem it prudent to keep moral discussions on the object level wherever I possibly can.

Occasionally people object to any discussion of morality on the grounds that morality doesn't exist, and in lieu of jumping over the forward dependency to explain that "exist" is not the right term to use here, I generally say, "But what do you do anyway?" and take the discussion back down to the object level.

Paul Gowder, though, has pointed out that both the idea of choosing a googolplex dust specks in a googolplex eyes over 50 years of torture for one person, and the idea of "utilitarianism", depend on "intuition". He says I've argued that the two are not compatible, but charges me with failing to argue for the utilitarian intuitions that I appeal to.

Now "intuition" is not how I would describe the computations that
underlie human morality and distinguish us, as moralists, from an ideal philosopher of perfect emptiness and/or a rock.
But I am okay with using the word "intuition" as a term of art, bearing
in mind that "intuition" in this sense is not to be contrasted to
reason, but is, rather, the cognitive building block out of which both
long verbal arguments and fast perceptual arguments are constructed.

I see the project of morality as a project of renormalizing
intuition. We have intuitions about things that seem desirable or
undesirable, intuitions about actions that are right or wrong,
intuitions about how to resolve conflicting intuitions, intuitions
about how to systematize specific intuitions into general principles.

Delete all the intuitions, and you ar...
Rationality Quotes 9
by Eliezer Yudkowsky — 2008-01-27T18:00:00.000Z

"A world ought to have a few genuine good guys, and not just a spectrum of people running from bad to worse."
-- Glen Cook, A Shadow of All Night Falling

"You couldn't get a clue during the clue mating season in a field full of horny clues if you smeared your body with clue musk and did the clue mating dance."
-- Edward Flaherty

"We all enter this world in the same way: naked; screaming; soaked in blood. But if you live your life right, that kind of thing doesn't have to stop there."
-- Dana Gould

"Love is a snowmobile racing across the tundra and then suddenly it flips over, pinning you underneath. At night, the ice weasels come."
-- Matt Groening

"Things do get better, all the time, maybe just not as fast as I'd like. I do what I can. Don't ask me to hate, too."
-- Michael Wiik

"Political or military commentators, like astrologers, can survive almost any mistake, because their more devoted followers do not look to them for an appraisal of the facts but for the stimulation of nationalistic loyalties."
-- George Orwell, Notes on Nationalism

"People are always amazed by how much "free time" I have.
They're also amazed that I don't know who Ally McBeal is.
Frankly, I'm amazed that they can't make the connection."
-- Robert Wenzlaff

"Throughout the technology revolution, mankind has consistently sought to improve life by reducing the number of tasks that require physical activity, then sought to counteract the inevitable weight gain by manufacturing food that only looks like food and barely tastes like it."
-- Samuel Stoddard

"Any person who claims to have deep feeling for other human beings should think a long, long time before he votes to have other men kept behind bars - caged. I am not saying there shouldn't be prisons, but there shouldn't be bars. Behind bars, a man never reforms. He will never forget. He never will get completely over the memory of the bars."
-- Malcolm X

"What funding committee will agree to fund a book describing an entire new field that has never before been dreamt of? Committees base their conclusions on a shared understanding of a common body of knowled...
Rationality Quotes 8
by Eliezer Yudkowsky — 2008-01-26T18:00:00.000Z

"Like a lot of people in the computer industry, Keith Malinowski had spent his whole life being the smartest person in the room, and like most of his fellows the experience left him with a rather high opinion of his opinions."
-- Rick Cook, The Wizardry Quested

"The fact that we can become accustomed to anything, however disgusting at first, makes it necessary to examine carefully everything we have become accustomed to."
-- George Bernard Shaw

"Beware `we should...', extend a hand to `how do I...'"
-- Alan Cox

"I assign higher credibility to an institution if liberals accuse it of being conservative and conservatives accuse it of being liberal."
-- Alex F. Bokov

"An open mind is a great place for other people to dump their garbage."
-- Rev. Rock, Church of the Subgenius

"A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait)."
-- Yu Suzuki

"A committee cannot be wrong - only divided. Once it resolves its division, then every part reinforces every other part and its rightness becomes unassailable."
-- John M. Ford, The Princes of the Air

"Gee, thanks for the elementary lesson in metaphysics. And there I thought Being was an undifferentiated unity."
-- Mark Walker

"The old political syllogism "something must be done: this is something: therefore this will be done" appears to be at work here, in spades."
-- Charlie Stross

"The fact that I beat a drum has nothing to do with the fact that I do theoretical physics. Theoretical physics is a human endeavour, one of the higher developments of human beings — and this perpetual desire to prove that people who do it are human by showing that they do other things that a few other humans do (like playing bongo drums) is insulting to me. I am human enough to tell you to go to hell."
-- Richard Feynman

"Java sucks. Java on TV set top boxes will suck so hard it might well inhale people from off their sofa until their heads get wedged in the card slots."
-- Jon Rabone

"Susan was bright enough to know...
Rationality Quotes 7
by Eliezer Yudkowsky — 2008-01-25T18:00:00.000Z

"People don't buy three-eighths-inch drill bits. People buy three-eighths-inch holes."
-- Michael Porter

"I feel more like I do now than I did a while ago."
-- Arifel

"Know thyself, because in the end there's no one else."
-- Living Colour, Solace of You

"Student motivation? I'm gonna start hooking you all up to electrodes."
-- Kevin Giffhorn

"A rock pile ceases to be a rock pile the moment a single man contemplates it, bearing within him the image of a cathedral."
-- Antoine de Saint-Exupery

"I found one day in school a boy of medium size ill-treating a smaller boy. I expostulated, but he replied: 'The bigs hit me, so I hit the babies; that's fair.' In these words he epitomized the history of the human race."
-- Bertrand Russell

"Open Source Software: There are days when I can't figure out whether I'm living in a Socialist utopia or a Libertarian one."
-- Alex Future Bokov

"Supposing you got a crate of oranges that you opened, and you found all the top layer of oranges bad, you would not argue, `The underneath ones must be good, so as to redress the balance.' You would say, `Probably the whole lot is a bad consignment'; and that is really what a scientific person would say about the universe."
-- Bertrand Russell

"This thing not only has all the earmarks of a hoax, it has "HOAX" branded into its flank and it's regulated by the U.S. Department of Hoaxes, Pranks and Shenanigans."
-- Brunching Shuttlecocks

"Natural selection gave a big advantage to those who were good at spotting the pattern of a saber toothed tiger hiding in the bushes, but no advantage to those who were good at solving partial differential equations. It is not mere rhetoric to say that in an absolute sense a janitor has a more intellectually challenging job than a professor of mathematics."
-- John K. Clark

"The simple fact is that non-violent means do not work against Evil. Gandhi's non-violent resistance against the British occupiers had some effect because Britain was wrong, but not Evil. The same is true of the success of non-violent civil rights resistance against de jure ...
Rationality Quotes 6
by Eliezer Yudkowsky — 2008-01-24T18:00:00.000Z

"Gnostic: Knowledge that is so pure that it cannot be explained or proven wrong."
-- Glossary of Zen

"I haven't been wrong since 1961, when I thought I made a mistake."
-- Bob Hudson

"Sometimes I envy that groundless confidence."
-- Vandread

"I remember when I first learned the word 'omniscient'. I thought that was a reasonable goal to aim for."
-- Samantha Atkins

"Our language for describing emotions is very crude... that's what music is for, I guess."
-- Ben Goertzel

"How much money would it take to get YOU to change your sexual orientation? Now add a dollar to that. That's how much Disney could offer you."
-- Leader Kibo

"I don't need The Media to tell me that I should be outraged about a brutal murder. All I need is to be informed that it has happened, and I'll form my own opinion about it."
-- The_Morlock

"Now Charlie, don't forget what happened to the man who suddenly got everything he wished for."
"What?"
"He lived happily ever after."
-- Roald Dahl, Willy Wonka and the Chocolate Factory

"He was never at ease with politics, where good and bad were just, apparently, two ways of looking at the same thing or, at least, were described like that by the people who were on the side Vimes thought of as 'bad'."
-- Terry Pratchett, "The Fifth Elephant"

"Philosophy is the art of asking the wrong questions."
-- J.R. Molloy

"The future is coming, Bill Joy. Like a juggernaut. There'll be no slowing it down. A billion people in India (substantial numbers of programmers riding the wave of the silicon revolution) and more than a billion people in China want a decent standard of living and they're counting on future technologies to deliver it for them. To slow down the future, you would have to nuke them. Are you going to nuke them, Bill? I didn't think so."
-- Jeff Davis

"Yikes, we may need to revise the Wechsler-Bellevue to accomodate some hitherto unrecognized nadir of the intelligence quotient scale."
-- Tom Morse

"Adultery always begins with the adulterer(s) claiming to themsel...
Rationality Quotes 5
by Eliezer Yudkowsky — 2008-01-23T18:00:00.000Z

If you're seeing this, it means that I've moved, but that my Internet access isn't set up yet. I've set up quotes to be posted automatically for the next few days. Don't be surprised if I don't respond to comments!

"Perfection is our goal. Excellence will be tolerated."
-- J. Yahl

"Morality is objective within a given frame of reference."
-- Gordon Worley

"If there were a verb meaning "to believe falsely," it would not have any significant first person, present indicative."
-- Ludwig Wittgenstein

"It takes 50 years for your parents to mature enough that they see you as an independent person."
-- Ralph Lewis

"Impatience is a flaw. There's always just enough time when you do something right, no more, no less. Your sword has no blade. It has only your intention. When that goes astray you have no weapon."
-- C.J. Cherryh, The Paladin

"They're space cannibals. They only eat other space cannibals. Q.E.D."
-- Nikolai Kingsley

"My father said whoever tells the longest story is always the liar. The truth isn't that complicated."
-- Bill Joy, cofounder and Chief Scientist of Sun Microsystems

"My experience tells me that in this complicated world the simplest explanation is usually dead wrong. But I've noticed that the simplest explanation usually sounds right and is far more convincing than any complicated explanation could hope to be."
-- Scott Adams, cartoonist

"There are really good and thoroughly bad people on each side in all wars."
"Nothing is more truly horrifying than the limits of human behavior."
"Determined efforts are better than a miracle."
"Things only get weirder the longer they go on."
"People will love you for who you are ... as long as you're secretly a super-hero."
"Lack of communication leads to 90% of all problems. The other causes being 5% magic and 5% giant robots."
-- Tonbo, Things I've Learned From Anime

"And I heard a voice saying "Give up! Give up!" And that really scared me 'cause it sounded like Ben Kenobi."
-- Rebel Pilot's Lament

"If you're a literary critic, keep in mind that I hate you, too, and I said it first."
&...