trentmkelly/LessWrong-43k huggingface dataset:
Mind Projection Fallacy
by Eliezer Yudkowsky — 2008-03-11T00:29:02.000Z

In the dawn days of science fiction, alien invaders would occasionally kidnap a girl in a torn dress and carry her off for intended ravishing, as lovingly depicted on many ancient magazine covers. Oddly enough, the aliens never go after men in torn shirts.

Would a non-humanoid alien, with a different evolutionary history and evolutionary psychology, sexually desire a human female? It seems rather unlikely. To put it mildly.

People don't make mistakes like that by deliberately reasoning: "All possible minds are likely to be wired pretty much the same way, therefore a bug-eyed monster will find human females attractive." Probably the artist did not even think to ask whether an alien perceives human females as attractive. Instead, a human female in a torn dress is sexy—inherently so, as an intrinsic property.

They who went astray did not think about the alien's evolutionary history; they focused on the woman's torn dress. If the dress were not torn, the woman would be less sexy; the alien monster doesn't enter into it.

Apparently we instinctively represent Sexiness as a direct attribute of the Woman object, Woman.sexiness, like Woman.height or Woman.weight.

If your brain uses that data structure, or something metaphorically similar to it, then from the inside it feels like sexiness is an inherent property of the woman, not a property of the alien looking at the woman. Since the woman is attractive, the alien monster will be attracted to her—isn't that logical?

E. T. Jaynes used the term Mind Projection Fallacy to denote the error of projecting your own mind's properties into the external world. Jaynes, as a late grand master of the Bayesian Conspiracy, was most concerned with the mistreatment of probabilities as inherent properties of objects, rather than states of partial knowledge in some particular mind. More about this shortly.

But the Mind Projection Fallacy generalizes as an error. It is in the argument over the real meaning of the word sound, and in the magazine cover of the monster carrying off a woman in the torn dress, and Kant's declaration that space by its very nature is flat, and Hume's definition of a priori ideas as those "discoverable by the mere operation of thought, without dependence on what is anywhere existent in the universe"...

(Incidentally, I once read an SF story ab...
Righting a Wrong Question
by Eliezer Yudkowsky — 2008-03-09T13:00:00.000Z

When you are faced with an unanswerable question—a question to which it seems impossible to even imagine an answer—there is a simple trick which can turn the question solvable.

Compare:

"Why do I have free will?"

"Why do I think I have free will?"

The nice thing about the second question is that it is guaranteed to have a real answer, whether or not there is any such thing as free will. Asking "Why do I have free will?" or "Do I have free will?" sends you off thinking about tiny details of the laws of physics, so distant from the macroscopic level that you couldn't begin to see them with the naked eye. And you're asking "Why is X the case?" where X may not be coherent, let alone the case.

"Why do I think I have free will?", in contrast, is guaranteed answerable. You do, in fact, believe you have free will. This belief seems far more solid and graspable than the ephemerality of free will. And there is, in fact, some nice solid chain of cognitive cause and effect leading up to this belief.

If you've already outgrown free will, choose one of these substitutes:

"Why does time move forward instead of backward?" versus "Why do I think time moves forward instead of backward?"

"Why was I born as myself rather than someone else?" versus "Why do I think I was born as myself rather than someone else?"

"Why am I conscious?" versus "Why do I think I'm conscious?"

"Why does reality exist?" versus "Why do I think reality exists?"

The beauty of this method is that it works whether or not the question is confused. As I type this, I am wearing socks. I could ask "Why am I wearing socks?" or "Why do I believe I'm wearing socks?" Let's say I ask the second question. Tracing back the chain of causality, I find:

I believe I'm wearing socks, because I can see socks on my feet.

I see socks on my feet, because my retina is sending sock signals to my visual cortex.

My retina is sending sock signals, because sock-shaped light is impinging on my retina.

Sock-shaped light impinges on my retina, because it reflects from the socks I'm wearing.

It reflects from the socks I'm wearing, because I'm wearing socks.

I'm wearing socks because I put them on.

I put socks on because I believed that otherwise my feet would get cold.

&c.

Tracing back the chain of causality, step by step, I discover that my belief that I'm wearing...
Wrong Questions
by Eliezer Yudkowsky — 2008-03-08T17:11:37.000Z

Where the mind cuts against reality's grain, it generates wrong questions—questions that cannot possibly be answered on their own terms, but only dissolved by understanding the cognitive algorithm that generates the perception of a question.

One good cue that you're dealing with a "wrong question" is when you cannot even imagine any concrete, specific state of how-the-world-is that would answer the question. When it doesn't even seem possible to answer the question.

Take the Standard Definitional Dispute, for example, about the tree falling in a deserted forest. Is there any way-the-world-could-be—any state of affairs—that corresponds to the word "sound" really meaning only acoustic vibrations, or really meaning only auditory experiences?

("Why, yes," says the one, "it is the state of affairs where 'sound' means acoustic vibrations." So Taboo the word 'means', and 'represents', and all similar synonyms, and describe again: How can the world be, what state of affairs, would make one side right, and the other side wrong?)

Or if that seems too easy, take free will: What concrete state of affairs, whether in deterministic physics, or in physics with a dice-rolling random component, could ever correspond to having free will?

And if that seems too easy, then ask "Why does anything exist at all?", and then tell me what a satisfactory answer to that question would even look like.

And no, I don't know the answer to that last one. But I can guess one thing, based on my previous experience with unanswerable questions. The answer will not consist of some grand triumphant First Cause. The question will go away as a result of some insight into how my mental algorithms run skew to reality, after which I will understand how the question itself was wrong from the beginning—how the question itself assumed the fallacy, contained the skew.

Mystery exists in the mind, not in reality. If I am ignorant about a phenomenon, that is a fact about my state of mind, not a fact about the phenomenon itself. All the more so, if it seems like no possible answer can exist: Confusion exists in the map, not in the territory. Unanswerable questions do not mark places where magic enters the universe. They mark places where your mind runs skew to reality.

Such questions must be dissolved. Bad thing...
Dissolving the Question
by Eliezer Yudkowsky — 2008-03-08T03:17:07.000Z

"If a tree falls in the forest, but no one hears it, does it make a sound?"

I didn't answer that question. I didn't pick a position, "Yes!" or "No!", and defend it. Instead I went off and deconstructed the human algorithm for processing words, even going so far as to sketch an illustration of a neural network. At the end, I hope, there was no question left—not even the feeling of a question.

Many philosophers—particularly amateur philosophers, and ancient philosophers—share a dangerous instinct: If you give them a question, they try to answer it.

Like, say, "Do we have free will?"

The dangerous instinct of philosophy is to marshal the arguments in favor, and marshal the arguments against, and weigh them up, and publish them in a prestigious journal of philosophy, and so finally conclude: "Yes, we must have free will," or "No, we cannot possibly have free will."

Some philosophers are wise enough to recall the warning that most philosophical disputes are really disputes over the meaning of a word, or confusions generated by using different meanings for the same word in different places. So they try to define very precisely what they mean by "free will", and then ask again, "Do we have free will? Yes or no?"

A philosopher wiser yet, may suspect that the confusion about "free will" shows the notion itself is flawed. So they pursue the Traditional Rationalist course: They argue that "free will" is inherently self-contradictory, or meaningless because it has no testable consequences. And then they publish these devastating observations in a prestigious philosophy journal.

But proving that you are confused may not make you feel any less confused. Proving that a question is meaningless may not help you any more than answering it.

The philosopher's instinct is to find the most defensible position, publish it, and move on. But the "naive" view, the instinctive view, is a fact about human psychology. You can prove that free will is impossible until the Sun goes cold, but this leaves an unexplained fact of cognitive science: If free will doesn't exist, what goes on inside the head of a human being who thinks it does? This is not a rhetorical question!

It is a fact about human psychology that people think they have free will. Finding a more defensible philosophical po...
Gary Gygax Annihilated at 69
by Eliezer Yudkowsky — 2008-03-06T20:54:56.000Z

Yesterday I heard that Gary Gygax, inventor of Dungeons and Dragons, had died at 69. And I don't understand, I truly don't, why that of all deaths should affect me the way it does.

Every day, people die; 150,000 of them, in fact. Every now and then I read the obituary of a scientist whose work I admired, and I don't feel like this. I should, of course, but I don't. I remember hearing about the death of Isaac Asimov, and more distantly, the death of Robert Heinlein (though I was 8 at the time) and that didn't affect me like this.

I never knew one single thing about Gary Gygax. I don't know if he had a wife or children. I couldn't guess his political opinions, or what he thought about the future of humanity. He was just a name on the cover of books I read until they disintegrated.

I searched on the Net and just found comments from other people feeling the same way. Stopped in their tracks by this one death, and not understanding why, and trying to come up with an explanation for their own feelings. Why him?

I never even really played D&D all that much. I played a little with David Levitt, my best friend in elementary school - I think it was how we initially met, in fact, though the memory fades into oblivion. I remember my father teaching me to play very simple D&D games, around the same time I was entering kindergarten; I remember being upset that I couldn't cast a Shield spell more than once. But mostly, I just read the rulebooks.

There are people who played D&D with their friends, every week or every day, until late at night, in modules that Gary Gygax designed. I understand why they feel sad. But all I did, mostly, was read the rulebooks to myself. Why do I feel the same way?

Did D&D help teach me that when the world is in danger, you are
expected to save it? Did Gary Gygax teach
me to form new worlds in my imagination, or to fantasize about more
interesting powers than money? Is there something about mentally
simulating D&D's rules that taught me how to think? Is it just the sheer amount of total time my imagination spent in those worlds?

I truly don't
know. I truly don't know why I feel this way about Gary Gygax's
death. I don't know why I feel this compulsion to write about it, to tell someone. I don't think I would ...
37 Ways That Words Can Be Wrong
by Eliezer Yudkowsky — 2008-03-06T05:09:49.000Z

Some reader is bound to declare that a better title for this post would be "37 Ways That You Can Use Words Unwisely", or "37 Ways That Suboptimal Use Of Categories Can Have Negative Side Effects On Your Cognition".

But one of the primary lessons of this gigantic list is that saying "There's no way my choice of X can be 'wrong'" is nearly always an error in practice, whatever the theory. You can always be wrong. Even when it's theoretically impossible to be wrong, you can still be wrong. There is never a Get-Out-Of-Jail-Free card for anything you do. That's life.

Besides, I can define the word "wrong" to mean anything I like - it's not like a word can be wrong.

Personally, I think it quite justified to use the word "wrong" when:

A word fails to connect to reality in the first place. Is Socrates a framster? Yes or no? (The Parable of the Dagger.)

Your argument, if it worked, could coerce reality to go a different way by choosing a different word definition. Socrates is a human, and humans, by definition, are mortal. So if you defined humans to not be mortal, would Socrates live forever? (The Parable of Hemlock.)

You try to establish any sort of empirical proposition as being true "by definition". Socrates is a human, and humans, by definition, are mortal. So is it a logical truth if we empirically predict that Socrates should keel over if he drinks hemlock? It seems like there are logically possible, non-self-contradictory worlds where Socrates doesn't keel over - where he's immune to hemlock by a quirk of biochemistry, say. Logical truths are true in all possible worlds, and so never tell you which possible world you live in - and anything you can establish "by definition" is a logical truth. (The Parable of Hemlock.)

You unconsciously slap the conventional label on something, without actually using the verbal definition you just gave. You know perfectly well that Bob is "human", even though, on your definition, you can never call Bob "human" without first observing him to be mortal. (The Parable of Hemlock.)

The act of labeling something with a word, disguises a challengable inductive inference you are making. If the last 11 egg-shaped objects drawn have been blue, and the last 8 cubes drawn have been red, it is a matter of induction to say this rule will

...
Variable Question Fallacies
by Eliezer Yudkowsky — 2008-03-05T06:22:25.000Z

Albert: "Every time I've listened to a tree fall, it made a sound, so I'll guess that other trees falling also make sounds. I don't believe the world changes around when I'm not looking."
Barry: "Wait a minute. If no one hears it, how can it be a sound?"

While writing the dialogue of Albert and Barry in their dispute over whether a falling tree in a deserted forest makes a sound, I sometimes found myself losing empathy with my characters. I would start to lose the gut feel of why anyone would ever argue like that, even though I'd seen it happen many times.

On these occasions, I would repeat to myself, "Either the falling tree makes a sound, or it does not!" to restore my borrowed sense of indignation.

(P or ~P) is not always a reliable heuristic, if you substitute arbitrary English sentences for P. "This sentence is false" cannot be consistently viewed as true or false. And then there's the old classic, "Have you stopped beating your wife?"

Now if you are a mathematician, and one who believes in classical (rather than intuitionistic) logic, there are ways to continue insisting that (P or ~P) is a theorem: for example, saying that "This sentence is false" is not a sentence.

But such resolutions are subtle, which suffices to demonstrate a need for subtlety. You cannot just bull ahead on every occasion with "Either it does or it doesn't!"

So does the falling tree make a sound, or not, or...?

Surely, 2 + 2 = X or it does not? Well, maybe, if it's really the same X, the same 2, and the same + and =. If X evaluates to 5 on some occasions and 4 on another, your indignation may be misplaced.

To even begin claiming that (P or ~P) ought to be a necessary truth, the symbol P must stand for exactly the same thing in both halves of the dilemma. "Either the fall makes a sound, or not!"—but if Albert::sound is not the same as Barry::sound, there is nothing paradoxical about the tree making an Albert::sound but not a Barry::sound.

(The :: idiom is something I picked up in my C++ days for avoiding namespace collisions. If you've got two different packages that define a class Sound, you can write Package1::Sound to specify which Sound you mean. The idiom is not widely known, I think; which is a pity, because I often wish I could use it in writing.)

The variability may be subtle: Albert and Barry...
Rationality Quotes 11
by Eliezer Yudkowsky — 2008-03-04T05:48:25.000Z

"If we let ethical considerations get in the way of scientific hubris, then the feminists have won!"
-- Helarxe

"The trajectory to hell is paved with locally-good intentions."
-- Matt Gingell

"To a mouse, cheese is cheese; that's why mousetraps work."
-- Wendell Johnson, quoted in Language in Thought and Action

"'Ethical consideration' has come to mean reasoning from an ivory tower about abstract non-issues while people die."
-- Zeb Haradon

"I intend to live forever. So far, so good."
-- Rick Potvin

"The accessory optic system: The AOS, extensively studied in the rabbit, arises from a special class of ganglion cells, the cells of Dogiel, that are directionally selective and respond best to slow rates of movement. They project to the terminal nuclei which in turn project to the dorsal cap of Kooy of the inferior olive. The climbing fibers from the olive project to the flocculo-nodular lobe of the cerebellum from where the brain stem occulomotor centers are reached through the vestibular nuclei."
-- MIT Encyclopedia of the Cognitive Sciences, "Visual Anatomy and Physiology"

"Fight for those you have lost, and for those you don't want to lose."
-- Claymore

"Which facts are likely to reappear? The simple facts. How to recognize them? Choose those that seem simple. Either this simplicity is real or the complex elements are indistinguishable. In the first case we're likely to meet this simple fact again either alone or as an element in a complex fact. The second case too has a good chance of recurring since nature doesn't randomly construct such cases."
-- Robert M. Pirsig, Zen and the Art of Motorcycle Maintenance

"Revolutions begin not when the first barricades are erected or even when people lose faith in the old ways of doing things, but rather when they realize that fundamental change is possible."
-- Steven Metz

"First Law of Anime Acoustics: In space, loud sounds, like explosions, are even louder because there is no air to get in the way."
"Law of Inherent Combustibility: Everything explodes. Everything."
"Law of Conservation of Firepower: Any powerful weapon capable of destro...
Rationality Quotes 10
by Eliezer Yudkowsky — 2008-03-03T05:48:02.000Z

"Yes, I am the last man to have walked on the moon, and that's a very dubious and disappointing honor. It's been far too long."
-- Gene Cernan

"Man, you're no smarter than me. You're just a fancier kind of stupid."
-- Spider Robinson, Distraction

"Each problem that I solved became a rule which served afterwards to solve other problems."
-- Rene Descartes, Discours de la Methode

"Faith is Hope given too much credit."
-- Matt Tuozzo

"Your Highness, I have no need of this hypothesis."
-- Pierre-Simon Laplace, to Napoleon, explaining why his works on celestial mechanics made no mention of God.

"'For, finally, one can only judge oneself by one's actions,' thought Elric. 'I have looked at what I have done, not at what I meant to do or thought I would like to do, and what I have done has, in the main, been foolish, destructive, and with little point. Yyrkoon was right to despise me and that was why I hated him so.'"
-- Michael Moorcock, Elric of Melniboné

"You will quickly find that if you are completely and self-deprecatingly truthful about how much you owe other people, the world at large will treat you like you did every bit of the invention yourself and are just being becomingly modest about your innate genius."
-- Eric S. Raymond

"The longer I live the more I see that I am never wrong about anything, and that all the pains that I have so humbly taken to verify my notions have only wasted my time."
-- George Bernard Shaw

"The trouble is that consciousness theories are very easy to dream up... Theories that explain intelligence, on the other hand, are fiendishly difficult to come by and so are profoundly useful. I don't know for sure that intelligence always produces consciousness, but I do know that if you assume it does you'll never be disappointed."
-- John K. Clark

"Intelligence is silence, truth being invisible. But what a racket I make in declaring this."
-- Ned Rorem, "Random Notes from a Diary"

Presently the mage said, speaking softly, "Do you see, Arren, how an act is not, as young men think, like a rock that one picks...
Words as Mental Paintbrush Handles
by Eliezer Yudkowsky — 2008-03-01T23:58:21.000Z

(We should be done with the mathy posts, I think, at least for now. But forgive me if, ironically, I end up resorting to Rationality Quotes for a day or two. I'm currently at the AGI-08 conference, which, as of the first session, is not nearly so bad as I feared.)

Suppose I tell you: "It's the strangest thing: The lamps in this hotel have triangular lightbulbs."

You may or may not have visualized it—if you haven't done it yet, do so now—what, in your mind's eye, does a "triangular lightbulb" look like?

In your mind's eye, did the glass have sharp edges, or smooth?

When the phrase "triangular lightbulb" first crossed my mind—no, the hotel doesn't have them—then as best as my introspection could determine, I first saw a pyramidal lightbulb with sharp edges, then (almost immediately) the edges were smoothed, and then my mind generated a loop of flourescent bulb in the shape of a smooth triangle as an alternative.

As far as I can tell, no deliberative/verbal thoughts were involved—just wordless reflex flinch away from the imaginary mental vision of sharp glass, which design problem was solved before I could even think in words.

Believe it or not, for some decades, there was a serious debate about whether people really had mental images in their mind—an actual picture of a chair somewhere—or if people just naively thought they had mental images (having been misled by "introspection", a very bad forbidden activity), while actually just having a little "chair" label, like a LISP token, active in their brain.

I am trying hard not to say anything like "How spectacularly silly," because there is always the hindsight effect to consider, but: how spectacularly silly.

This academic paradigm, I think, was mostly a deranged legacy of behaviorism, which denied the existence of thoughts in humans, and sought to explain all human phenomena as "reflex", including speech. Behaviorism probably deserves its own post at some point, as it was a perversion of rationalism; but this is not that post.

"You call it 'silly'," you inquire, "but how do you know that your brain represents visual images? Is it merely that you can close your eyes and see them?"

This question used to be harder to answer, back in the day of the controversy. If you wanted to prove the existence of mental imagery "scientifically", rather th...