trentmkelly/LessWrong-43k huggingface dataset:
Joy in Discovery
by Eliezer Yudkowsky — 2008-03-21T01:19:18.000Z

"Newton was the greatest genius who ever lived, and the most fortunate; for we cannot find more than once a system of the world to establish."
—Lagrange

I have more fun discovering things for myself than reading about them in textbooks. This is right and proper, and only to be expected.

But discovering something that no one else knows—being the first to unravel the secret—

There is a story that one of the first men to realize that stars were burning by fusion—plausible attributions I've seen are to Fritz Houtermans and Hans Bethe—was walking out with his girlfriend of a night, and she made a comment on how beautiful the stars were, and he replied: "Yes, and right now, I'm the only man in the world who knows why they shine."

It is attested by numerous sources that this experience, being the first person to solve a major mystery, is a tremendous high. It's probably the closest experience you can get to taking drugs, without taking drugs—though I wouldn't know.

That can't be healthy.

Not that I'm objecting to the euphoria. It's the exclusivity clause that bothers me. Why should a discovery be worth less, just because someone else already knows the answer?

The most charitable interpretation I can put on the psychology, is that you don't struggle with a single problem for months or years if it's something you can just look up in the library. And that the tremendous high comes from having hit the problem from every angle you can manage, and having bounced; and then having analyzed the problem again, using every idea you can think of, and all the data you can get your hands on—making progress a little at a time—so that when, finally, you crack through the problem, all the dangling pieces and unresolved questions fall into place at once, like solving a dozen locked-room murder mysteries with a single clue.

And more, the understanding you get is real understanding—understanding that embraces all the clues you studied to solve the problem, when you didn't yet know the answer. Understanding that comes from asking questions day after day and worrying at them; understanding that no one else can get (no matter how much you tell them the answer) unless they spend months studying the problem in its historical context, even after it's been solved—a...
Joy in the Merely Real
by Eliezer Yudkowsky — 2008-03-20T06:18:58.000Z

...Do not all charms fly
At the mere touch of cold philosophy?
There was an awful rainbow once in heaven:
We know her woof, her texture; she is given
In the dull catalogue of common things.
—John Keats, Lamia

"Nothing is 'mere'."
—Richard Feynman

You've got to admire that phrase, "dull catalogue of common things". What is it, exactly, that goes in this catalogue? Besides rainbows, that is?

Why, things that are mundane, of course. Things that are normal; things that are unmagical; things that are known, or knowable; things that play by the rules (or that play by any rules, which makes them boring); things that are part of the ordinary universe; things that are, in a word, real.

Now that's what I call setting yourself up for a fall.

At that rate, sooner or later you're going to be disappointed in everything—either it will turn out not to exist, or even worse, it will turn out to be real.

If we cannot take joy in things that are merely real, our lives will always be empty.

For what sin are rainbows demoted to the dull catalogue of common things? For the sin of having a scientific explanation. "We know her woof, her texture", says Keats—an interesting use of the word "we", because I suspect that Keats didn't know the explanation himself. I suspect that just being told that someone else knew was too much for him to take. I suspect that just the notion of rainbows being scientifically explicable in principle would have been too much to take. And if Keats didn't think like that, well, I know plenty of people who do.

I have already remarked that nothing is inherently mysterious—nothing that actually exists, that is. If I am ignorant about a phenomenon, that is a fact about my state of mind, not a fact about the phenomenon; to worship a phenomenon because it seems so wonderfully mysterious, is to worship your own ignorance; a blank map does not correspond to a blank territory, it is just somewhere we haven't visited yet, etc. etc...

Which is to say that everything—everything that actually exists—is liable to end up in "the dull catalogue of common things", sooner or later.

Your choice is either:

Decide that things are allowed

...
Savanna Poets
by Eliezer Yudkowsky — 2008-03-18T18:42:31.000Z

"Poets say science takes away from the beauty of the stars—mere globs of gas atoms. Nothing is "mere". I too can see the stars on a desert night, and feel them. But do I see less or more?
"The vastness of the heavens stretches my imagination—stuck on this carousel my little eye can catch one-million-year-old light. A vast pattern—of which I am a part—perhaps my stuff was belched from some forgotten star, as one is belching there. Or see them with the greater eye of Palomar, rushing all apart from some common starting point when they were perhaps all together. What is the pattern, or the meaning, or the why? It does not do harm to the mystery to know a little about it.
"For far more marvelous is the truth than any artists of the past imagined! Why do the poets of the present not speak of it?
"What men are poets who can speak of Jupiter if he were like a man, but if he is an immense spinning sphere of methane and ammonia must be silent?"
—Richard Feynman, The Feynman Lectures on Physics, Vol I, p. 3-6 (line breaks added)

That's a real question, there on the last line—what kind of poet can write about Jupiter the god, but not Jupiter the immense sphere? Whether or not Feynman meant the question rhetorically, it has a real answer:

If Jupiter is like us, he can fall in love, and lose love, and regain love.
If Jupiter is like us, he can strive, and rise, and be cast down.
If Jupiter is like us, he can laugh or weep or dance.

If Jupiter is an immense spinning sphere of methane and ammonia, it is more difficult for the poet to make us feel.

There are poets and storytellers who say that the Great Stories are timeless, and they never change, they only ever retold. They say, with pride, that Shakespeare and Sophocles are bound by ties of craft stronger than mere centuries; that the two playwrights could have swapped times without a jolt.

Donald Brown once compiled a list of over two hundred "human universals", found in all (or a vast supermajority of) studied human cultures, from San Francisco to the !Kung of the Kalahari Desert. Marriage is on the list, and incest avoidance, and motherly love, and sibling rivalry, and music and envy and dance and storytelling and aesthe...
Fake Reductionism
by Eliezer Yudkowsky — 2008-03-17T22:49:13.000Z

There was an awful rainbow once in heaven:
We know her woof, her texture; she is given
In the dull catalogue of common things.
—John Keats, Lamia

I am guessing—though it is only a guess—that Keats himself did not know the woof and texture of the rainbow. Not the way that Newton understood rainbows. Perhaps not even at all. Maybe Keats just read, somewhere, that Newton had explained the rainbow as "light reflected from raindrops"—

—which was actually known in the 13th century. Newton only added a refinement by showing that the light was decomposed into colored parts, rather than transformed in color. But that put rainbows back in the news headlines. And so Keats, with Charles Lamb and William Wordsworth and Benjamin Haydon, drank "Confusion to the memory of Newton" because "he destroyed the poetry of the rainbow by reducing it to a prism." That's one reason to suspect Keats didn't understand the subject too deeply.

I am guessing, though it is only a guess, that Keats could not have sketched out on paper why rainbows only appear when the Sun is behind your head, or why the rainbow is an arc of a circle.

If so, Keats had a Fake Explanation. In this case, a fake reduction. He'd been told that the rainbow had been reduced, but it had not actually been reduced in his model of the world.

This is another of those distinctions that anti-reductionists fail to get—the difference between professing the flat fact that something is reducible, and seeing it.

In this, the anti-reductionists are not too greatly to be blamed, for it is part of a general problem.

I've written before on seeming knowledge that is not knowledge, and beliefs that are not about their supposed objects but only recordings to recite back in the classroom, and words that operate as stop signs for curiosity rather than answers, and technobabble which only conveys membership in the literary genre of "science"...

There is a very great distinction between being able to see where the rainbow comes from, and playing around with prisms to confirm it, and maybe making a rainbow yourself by spraying water droplets—

—versus some dour-faced philosopher just telling you, "No, there's nothing special about the rainbow. Didn't you hear? Scientists have explained it away. Just ...
Explaining vs. Explaining Away
by Eliezer Yudkowsky — 2008-03-17T01:59:27.000Z

John Keats's Lamia (1819) surely deserves some kind of award for Most Famously Annoying Poetry:

...Do not all charms fly
At the mere touch of cold philosophy?
There was an awful rainbow once in heaven:
We know her woof, her texture; she is given
In the dull catalogue of common things.
Philosophy will clip an Angel's wings,
Conquer all mysteries by rule and line,
Empty the haunted air, and gnomed mine—
Unweave a rainbow...

My usual reply ends with the phrase: "If we cannot learn to take joy in the merely real, our lives will be empty indeed." I shall expand on that tomorrow.

Today I have a different point in mind. Let's just take the lines:

Empty the haunted air, and gnomed mine—
Unweave a rainbow...

Apparently "the mere touch of cold philosophy", i.e., the truth, has destroyed:

Haunts in the air

Gnomes in the mine

Rainbows

Which calls to mind a rather different bit of verse:

One of these things
Is not like the others
One of these things
Doesn't belong

The air has been emptied of its haunts, and the mine de-gnomed—but the rainbow is still there!

In "Righting a Wrong Question", I wrote:

Tracing back the chain of causality, step by step, I discover that my belief that I'm wearing socks is fully explained by the fact that I'm wearing socks... On the other hand, if I see a mirage of a lake in the desert, the correct causal explanation of my vision does not involve the fact of any actual lake in the desert. In this case, my belief in the lake is not just explained, but explained away.

The rainbow was explained. The haunts in the air, and gnomes in the mine, were explained away.

I think this is the key distinction that anti-reductionists don't get about reductionism.

You can see this failure to get the distinction in the classic objection to reductionism:

If reductionism is correct, then even your belief in reductionism is just the mere result of the motion of molecules—why should I listen to anything you say?

The key word, in the above, is mere; a word which implies that accepting reductionism would explain away all the reasoning processes leading up to my acceptance of reductionism, the way that an optical illusion is explained away.

But you can explain how a cognitive process works without it being "mere"! My b...
Reductionism
by Eliezer Yudkowsky — 2008-03-16T06:26:38.000Z

Almost one year ago, in April 2007, Matthew C submitted the following suggestion for an Overcoming Bias topic:

"How and why the current reigning philosophical hegemon (reductionistic materialism) is obviously correct [...], while the reigning philosophical viewpoints of all past societies and civilizations are obviously suspect—"

I remember this, because I looked at the request and deemed it legitimate, but I knew I couldn't do that topic until I'd started on the Mind Projection Fallacy sequence, which wouldn't be for a while...

But now it's time to begin addressing this question. And while I haven't yet come to the "materialism" issue, we can now start on "reductionism".

First, let it be said that I do indeed hold that "reductionism", according to the meaning I will give for that word, is obviously correct; and to perdition with any past civilizations that disagreed.

This seems like a strong statement, at least the first part of it. General Relativity seems well-supported, yet who knows but that some future physicist may overturn it?

On the other hand, we are never going back to Newtonian mechanics. The ratchet of science turns, but it does not turn in reverse. There are cases in scientific history where a theory suffered a wound or two, and then bounced back; but when a theory takes as many arrows through the chest as Newtonian mechanics, it stays dead.

"To hell with what past civilizations thought" seems safe enough, when past civilizations believed in something that has been falsified to the trash heap of history.

And reductionism is not so much a positive hypothesis, as the absence of belief—in particular, disbelief in a form of the Mind Projection Fallacy.

I once met a fellow who claimed that he had experience as a Navy gunner, and he said, "When you fire artillery shells, you've got to compute the trajectories using Newtonian mechanics. If you compute the trajectories using relativity, you'll get the wrong answer."

And I, and another person who was present, said flatly, "No." I added, "You might not be able to compute the trajectories fast enough to get the answers in time—maybe that's what you mean? But the relativistic answer will always be more accurate than the Newtonian one."

"No," he said, "I mean that relativity will give you the wrong answer, because things moving at the speed of artillery s...
Qualitatively Confused
by Eliezer Yudkowsky — 2008-03-14T17:01:08.000Z

I suggest that a primary cause of confusion about the distinction between "belief", "truth", and "reality" is qualitative thinking about beliefs.

Consider the archetypal postmodernist attempt to be clever:

"The Sun goes around the Earth" is true for Hunga Huntergatherer, but "The Earth goes around the Sun" is true for Amara Astronomer! Different societies have different truths!

No, different societies have different beliefs. Belief is of a different type than truth; it's like comparing apples and probabilities.

Ah, but there's no difference between the way you use the word 'belief' and the way you use the word 'truth'! Whether you say, "I believe 'snow is white'", or you say, "'Snow is white' is true", you're expressing exactly the same opinion.

No, these sentences mean quite different things, which is how I can conceive of the possibility that my beliefs are false.

Oh, you claim to conceive it, but you never believe it. As Wittgenstein said, "If there were a verb meaning 'to believe falsely', it would not have any significant first person, present indicative."

And that's what I mean by putting my finger on qualitative reasoning as the source of the problem. The dichotomy between belief and disbelief, being binary, is confusingly similar to the dichotomy between truth and untruth.

So let's use quantitative reasoning instead. Suppose that I assign a 70% probability to the proposition that snow is white. It follows that I think there's around a 70% chance that the sentence "snow is white" will turn out to be true. If the sentence "snow is white" is true, is my 70% probability assignment to the proposition, also "true"? Well, it's more true than it would have been if I'd assigned 60% probability, but not so true as if I'd assigned 80% probability.

When talking about the correspondence between a probability assignment and reality, a better word than "truth" would be "accuracy". "Accuracy" sounds more quantitative, like an archer shooting an arrow: how close did your probability assignment strike to the center of the target?

To make a long story short, it turns out that there's a very natural way of scoring the accuracy of a probability assignment, as compared to reality: just take the logarithm of the probability assigned to the real state of affairs.

So if snow is white, my belief "70%: 'snow is white...
Penguicon & Blook
by Eliezer Yudkowsky — 2008-03-13T17:32:43.000Z

One million cumulative daily visits! Woot n' stuff. Also we're in the top 5,000 of all blogs on Technorati, and one of the top 10 econblogs by Technorati rank.

Seems like a good time to mention that I'll be appearing at Penguicon, a combination open-source/science-fiction convention in Troy, MI, Apr 18-20, as a Nifty. I'll be doing an intro to Bayesian reasoning that you probably don't need if you're reading this, possibly a panel on the Virtues of a Rationalist, some stuff on human intelligence upgrades, and definitely "The Ethics of Ending the World" with Aaron Diaz (Dresden Codak).

After the jump, you can see some proposed cover art for the blook.

For the benefit of the humor impaired: Yes, this is a joke. Erin, my girlfriend, Photoshopped this when she heard I was planning to do a book.

This is all taking longer than I expected - as expected - but I do think I'm getting there.

My current serious strategy for the blook is as follows:

Finish all the important serial material on rationality - the posts that have to be done in month-long sequences. That's probably at least another two months.

Maybe spend another month or two doing large transhumanist sequences, either on the Singularity Institute blog (currently fairly defunct) or here on Overcoming Bias if the readers really want that. My self-imposed deadline here is August 2008.

Switch to writing shorter posts on topics that can be considered independently given the already-written background material - maybe cut back to a Sat-Sun-Tue-Thu schedule. Don't worry, this won't happen anytime soon.

On days when I don't post: spend my time compiling collections of related Overcoming Bias posts into medium-sized ebooks of 50 pages / 20,000 words or thereabouts, broken up into short blog-post-sized sections for easy reading. (This is good because it can be done incrementally, and I tend to bog down when I try to do anything book-sized all at once.) Leave a comment if you have any suggestions on ebook-writing software or ways to get the book design done cheaply.

Publish the ebooks incrementally on Wowio, as a compromise between "information wants to be free" and "authors want to eat". (Email me if you happen to work at/with Wowio, because I'm interested in testing the waters on this sometime soon.) Publish the

...
The Quotation is not the Referent
by Eliezer Yudkowsky — 2008-03-13T00:53:44.000Z

In classical logic, the operational definition of identity is that whenever 'A=B' is a theorem, you can substitute 'A' for 'B' in any theorem where B appears. For example, if (2 + 2) = 4 is a theorem, and ((2 + 2) + 3) = 7 is a theorem, then (4 + 3) = 7 is a theorem.

This leads to a problem which is usually phrased in the following terms: The morning star and the evening star happen to be the same object, the planet Venus. Suppose John knows that the morning star and evening star are the same object. Mary, however, believes that the morning star is the god Lucifer, but the evening star is the god Venus. John believes Mary believes that the morning star is Lucifer. Must John therefore (by substitution) believe that Mary believes that the evening star is Lucifer?

Or here's an even simpler version of the problem. 2 + 2 = 4 is true; it is a theorem that (((2 + 2) = 4) = TRUE). Fermat's Last Theorem is also true. So: I believe 2 + 2 = 4 => I believe TRUE => I believe Fermat's Last Theorem.

Yes, I know this seems obviously wrong. But imagine someone writing a logical reasoning program using the principle "equal terms can always be substituted", and this happening to them. Now imagine them writing a paper about how to prevent it from happening. Now imagine someone else disagreeing with their solution. The argument is still going on.

P'rsnally, I would say that John is committing a type error, like trying to subtract 5 grams from 20 meters. "The morning star" is not the same type as the morning star, let alone the same thing. Beliefs are not planets.

morning star = evening star
"morning star" ≠ "evening star"

The problem, in my view, stems from the failure to enforce the type distinction between beliefs and things. The original error was writing an AI that stores its beliefs about Mary's beliefs about "the morning star" using the same representation as in its beliefs about the morning star.

If Mary believes the "morning star" is Lucifer, that doesn't mean Mary believes the "evening star" is Lucifer, because "morning star" ≠ "evening star". The whole paradox stems from the failure to use quote marks in appropriate places.

You may recall that this is not the first time I've talked about enforcing type discipline—the last time was when I spoke about the er...
Probability is in the Mind
by Eliezer Yudkowsky — 2008-03-12T04:08:30.000Z

Yesterday I spoke of the Mind Projection Fallacy, giving the example of the alien monster who carries off a girl in a torn dress for intended ravishing—a mistake which I imputed to the artist's tendency to think that a woman's sexiness is a property of the woman herself, woman.sexiness, rather than something that exists in the mind of an observer, and probably wouldn't exist in an alien mind.

The term "Mind Projection Fallacy" was coined by the late great Bayesian Master, E. T. Jaynes, as part of his long and hard-fought battle against the accursèd frequentists. Jaynes was of the opinion that probabilities were in the mind, not in the environment—that probabilities express ignorance, states of partial information; and if I am ignorant of a phenomenon, that is a fact about my state of mind, not a fact about the phenomenon.

I cannot do justice to this ancient war in a few words—but the classic example of the argument runs thus:

You have a coin.
The coin is biased.
You don't know which way it's biased or how much it's biased. Someone just told you, "The coin is biased" and that's all they said.
This is all the information you have, and the only information you have.

You draw the coin forth, flip it, and slap it down.

Now—before you remove your hand and look at the result—are you willing to say that you assign a 0.5 probability to the coin having come up heads?

The frequentist says, "No. Saying 'probability 0.5' means that the coin has an inherent propensity to come up heads as often as tails, so that if we flipped the coin infinitely many times, the ratio of heads to tails would approach 1:1. But we know that the coin is biased, so it can have any probability of coming up heads except 0.5."

The Bayesian says, "Uncertainty exists in the map, not in the territory. In the real world, the coin has either come up heads, or come up tails. Any talk of 'probability' must refer to the information that I have about the coin—my state of partial ignorance and partial knowledge—not just the coin itself. Furthermore, I have all sorts of theorems showing that if I don't treat my partial knowledge a certain way, I'll make stupid bets. If I've got to plan, I'll plan for a 50/50 state of uncertainty, where I don't weigh outcomes conditional on heads any more heavily in my mind than outcomes co...